Groq Client

[marklysze]

This is a Groq Client that provides a way to use the, rather fast, Groq cloud inference service, which provides options to use a few LLMs, such as Meta's Llama 3 and Mistral.AI's Mixtral.

This client supports text generation and function/tool calling (when using certain models as noted by Groq).

Streaming is support, including with function/tool calling.

Token costs are also implemented.

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[marklysze]

LGTM! Thanks Mark!

Great :)

[marklysze]

Sorry, found a bug when testing logging. Minor but will fix if someone wants to log this (or Together.AI)

[marklysze]

Fixed, thank you, good to go :)

[Hk669]

lgtm. thanks @marklysze

[Hk669]

@marklysze if this is merged before the blogpost on non-openai models, please also add the Groq Client in the blogpost.

[marklysze]

@marklysze if this is merged before the blogpost on non-openai models, please also add the Groq Client in the blogpost.

Good call, will do

[marklysze]

Thanks @qingyun-wu! @Hk669, I've updated the blog #2965.