[INFRA] No Load Balancer or Worker Pool for Concurrent Requests

[shrixtacy]

Description

No request queue, load balancer, or worker pool. Each FastAPI request runs the full pipeline in-process. Under concurrent load, model API calls exhaust connections, rate limits stack up, and memory grows with in-memory caches. mq_agent.py worker exists but is not integrated into the web backend.

Component

System Architecture, ai_council/worker/mq_agent.py

Acceptance Criteria

Integrate load balancing and the existing message queue worker for handling concurrent async queries. Distribute model API calls across worker pools.

[github-actions]

🙏 Thank you for raising this issue!

We'll review it as soon as possible. We truly appreciate your contributions! ✨

📚 Before you start working:

• ✅ Make sure you've read the README.md
• ✅ Review CONTRIBUTING.md for guidelines
• ✅ Familiarize yourself with CODE_OF_CONDUCT.md

⚠️ Important: Please do NOT create a PR until this issue has been assigned to you.

💡 Want to work on this? Comment "assign me" or "I want to work on this" to get assigned!

[Adithyakp86]

assign this issue to me under INTERNSHIP OSCG

[github-actions]

⚠️ Already Working on Another Issue

Hey @Adithyakp86! You're already assigned to: #167

Contributors can only work on one owner-created issue at a time to ensure quality and fair distribution of work.

💡 Tip: Once you submit a PR and close your current issue, you'll be able to take on new ones! 🚀

[sm-28601]

Hi @shrixtacy , assign me this issue .

[github-actions]

🎉 Issue Assigned!

Congratulations @sm-28601! This issue has been assigned to you.

📋 Guidelines:

• Read the issue description carefully
• Review CONTRIBUTING.md and CODE_OF_CONDUCT.md
• Ask questions if anything is unclear
• Link your PR to this issue when ready
• Follow the project's coding standards

⚠️ Note: You can only work on one owner-created issue at a time. Please complete this before requesting another assignment.

Good luck! 💪

[github-actions]

👋 Gentle Reminder

Hey @sm-28601! This issue was assigned to you 24 hours ago.

⏰ Timeline:

• ✅ 24 hours: Reminder (now)

📋 What to do:

• Create a PR and link it to this issue (mention [INFRA] No Load Balancer or Worker Pool for Concurrent Requests #175 in PR description)
• If you need more time, comment here to let us know
• If you can't work on this, let us know so we can reassign

💡 Tip: Even a draft PR shows progress!

Thanks for contributing! 🚀