How does one use AutoGen with an existing codebase?

[maximedupre]

It seems like, by default, AutoGen

• has no knowledge of your existing codebase
• cannot read or write files on your FS (apart from the executor work dir, which seems to act more as a temporary folder for code to be executed)
• cannot be tasked to make a fix or add a feature to an existing codebase

How would one use AutoGen to make bug fixes/add features in an existing codebase under human supervision?

[ekzhu]

You would need to create tools to expose your code base (or any code base) to your agents.

@GregorD1A1 is working on a PR to add a notebook example: #2121

See tool use tutorial chapter: https://microsoft.github.io/autogen/docs/tutorial/tool-use

[maximedupre]

This PR is very useful 👌

To extend on this, for example, how would one give knowledge of the codebase to an "Engineer" agent? Listing the files in the codebase is a good start, but how would you make the "Engineer" "ingest" all the code, or train it in some way on the code, such that the code suggestions would be context relevant to the whole codebase, and not just one specific file?

Of course, it would have to be "pre-trained", or it would need to vectorize/embed the whole codebase to avoid feeding the whole codebase on every request, as that would be very expensive. (not sure I'm using the right terms lol, still a beginner)

Is RAG the right thing that I should learn next to achieve this goal?

[GregorD1A1]

@maximedupre you raised interesting and important issue I haven't solved yet.

I think the best idea is to start simple. You can add to Engineer prompt few consice sentences that include most important infos about the project, for example: "Common styles placed in path/to/dir/..., don't create style locally when it available in common. File main.py intended to store endpoints only, any additional functions place in different files. Aplication log placed in something/log.txt, check it frequently and never overwrite.". Just keep it short to not distract LLMs attention, and it will be a good starting point for your agent.

If that approach will work for you, you can add next function for actualize that project summany when agent will learn something new about your project.

What about RAGs... well, it's a complicated topic. In my opinion in such case it has no sense to index whole your code base - better idea will be to write description of every file, what it contains and does, and index the descriptions. Then to provide path to the file in the metadata of the record, so you will be able to search appropriate files by description. But here will appear another problem - your agent will need to know what he actually wants to find. So in my opinion it's better to avoid RAGs, at least for beginning, and start simple.

[Itrel]

Take a look at https://github.com/princeton-nlp/SWE-agentv

Princeton University created an open source Agent Computer Interface (ACI) enabling an agent to get information about the code base. An associated research paper should be coming out soon.

[Itrel]

Even better, https://github.com/unit-mesh/auto-dev
From Microsoft.