Genesis rate limit

[ErikHougaard]

First of all, thank you for developing such an amazing framework. I am just learning how to build agents and your tools have saved me a ton of time.

I have been trying to uses Genesis for creating an agency, and it seems to work like a charm. However, after some time it fails due to the OpenAI rate limit of 30000 TPM.

Would it make sense to implement a throttling feature to prevent this, or is there another trick?

[achimstruve]

I faced the same issue and am running into that token limit problem quite often with swarm agency.

[ArtemShatokhin]

Hey @ErikHougaard,
You can create your own version of genesis agency and set custom token limits for it like so:

from agency_swarm.agency.genesis import GenesisAgency

from dotenv import load_dotenv

load_dotenv()

agency = GenesisAgency(with_browsing=False,
    max_prompt_tokens=10000,
    max_completion_tokens=25000,
)


demo = agency.demo_gradio(height=900)
# or for console version
# agency.run_demo()

If you want to learn more on what these paramters do, here's a link to oai's documentation:
https://platform.openai.com/docs/assistants/deep-dive#max-completion-and-max-prompt-tokens

@achimstruve, you can use the same approach to limit tokens agency-wide, but you can also set the max tokens for each agent idividually, if you want more flexebility for your agency.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Please upgrade to the latest version and test it again.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale. If the issue still persists, please reopen.