backport: rely on stdout redirection for supervisord logging (#2711)

[rkuo-danswer]

Description

[Provide a brief description of the changes in this PR]

How Has This Been Tested?

[Describe the tests you ran to verify your changes]

Accepted Risk

[Any know risks or failure modes to point out to reviewers]

Related Issue(s)

[If applicable, link to the issue(s) this PR addresses]

Checklist:

• All of the automated tests pass
• All PR comments are addressed and marked resolved
• If there are migrations, they have been rebased to latest main
• If there are new dependencies, they are added to the requirements
• If there are new environment variables, they are added to all of the deployment methods
• If there are new APIs that don't require auth, they are added to PUBLIC_ENDPOINT_SPECS
• Docker images build and basic functionalities work
• Author has done a final read through of the PR right before merge

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
internal-search	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 7, 2024 10:31pm

[greptile-apps]

PR Summary

This PR modifies the supervisord.conf file to improve log management and worker configuration flexibility for Celery workers and other components in the Danswer backend.

• Updated /backend/supervisord.conf to use stdout_logfile instead of --logfile for Celery workers and beat
• Added stdout_logfile_maxbytes=16MB for log rotation in Celery worker configurations
• Modified celery_worker_light to use environment variables for concurrency and prefetch multiplier
• Removed environment=LOG_FILE_NAME settings for workers, simplifying log file management
• Updated log-redirect-handler to use new log file paths, ensuring proper log aggregation

_{1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings}