diff --git a/docs/images/external/openai/openai-lockup.png b/docs/images/external/openai/openai-lockup.png new file mode 100644 index 000000000..17f72fc48 Binary files /dev/null and b/docs/images/external/openai/openai-lockup.png differ diff --git a/docs/images/external/openai/openai-logomark.png b/docs/images/external/openai/openai-logomark.png new file mode 100644 index 000000000..7d4202873 Binary files /dev/null and b/docs/images/external/openai/openai-logomark.png differ diff --git a/docs/images/external/openai/openai-white-lockup.png b/docs/images/external/openai/openai-white-lockup.png new file mode 100644 index 000000000..93b6491de Binary files /dev/null and b/docs/images/external/openai/openai-white-lockup.png differ diff --git a/docs/images/external/openai/openai-white-logomark.png b/docs/images/external/openai/openai-white-logomark.png new file mode 100644 index 000000000..97e73e006 Binary files /dev/null and b/docs/images/external/openai/openai-white-logomark.png differ diff --git a/docs/mint.json b/docs/mint.json index 45e61b450..c534fc6a5 100644 --- a/docs/mint.json +++ b/docs/mint.json @@ -87,14 +87,15 @@ { "group": "Integrations", "pages": [ - "v1/integrations/crewai", + "v1/integrations/anthropic", "v1/integrations/autogen", - "v1/integrations/langchain", "v1/integrations/cohere", - "v1/integrations/anthropic", - "v1/integrations/ollama", + "v1/integrations/crewai", + "v1/integrations/langchain", "v1/integrations/litellm", "v1/integrations/multion", + "v1/integrations/ollama", + "v1/integrations/openai", "v1/integrations/rest" ] }, diff --git a/docs/v1/integrations/anthropic.mdx b/docs/v1/integrations/anthropic.mdx index 779d80d6b..6b1351590 100644 --- a/docs/v1/integrations/anthropic.mdx +++ b/docs/v1/integrations/anthropic.mdx @@ -18,10 +18,23 @@ This is a living integration. Should you need any added functionality, message u ```bash pip - pip install agentops anthropic + pip install agentops ``` ```bash poetry - poetry add agentops anthropic + poetry add agentops + ``` + + + + + `anthropic>=0.32.0` is currently supported with additional support for the Computer Use tool. + + + ```bash pip + pip install anthropic + ``` + ```bash poetry + poetry add anthropic ``` diff --git a/docs/v1/integrations/cohere.mdx b/docs/v1/integrations/cohere.mdx index d8fd30edf..587adab6b 100644 --- a/docs/v1/integrations/cohere.mdx +++ b/docs/v1/integrations/cohere.mdx @@ -25,6 +25,19 @@ This is a living integration. Should you need any added functionality, message u poetry add agentops ``` + + + + `cohere>=5.4.0` is currently supported. + + + ```bash pip + pip install cohere + ``` + ```bash poetry + poetry add cohere + ``` + diff --git a/docs/v1/integrations/ollama.mdx b/docs/v1/integrations/ollama.mdx index 31e6512d9..04d849328 100644 --- a/docs/v1/integrations/ollama.mdx +++ b/docs/v1/integrations/ollama.mdx @@ -25,6 +25,16 @@ This is a living integration. Should you need any added functionality, message u ``` + + + ```bash pip + pip install ollama + ``` + ```bash poetry + poetry add ollama + ``` + + diff --git a/docs/v1/integrations/openai.mdx b/docs/v1/integrations/openai.mdx new file mode 100644 index 000000000..0ac716e63 --- /dev/null +++ b/docs/v1/integrations/openai.mdx @@ -0,0 +1,195 @@ +--- +title: OpenAI +description: "AgentOps provides first class support for OpenAI's GPT family of models" +--- + +import CodeTooltip from '/snippets/add-code-tooltip.mdx' +import EnvTooltip from '/snippets/add-env-tooltip.mdx' + + +This is a living integration. Should you need any added functionality, message us on [Discord](https://discord.gg/UgJyyxx7uc)! + + + + First class support for GPT family of models + + + + + + ```bash pip + pip install agentops + ``` + ```bash poetry + poetry add agentops + ``` + + + + + `openai<1.0.0` has limited support while `openai>=1.0.0` is continuously supported. + + + ```bash pip + pip install openai + ``` + ```bash poetry + poetry add openai + ``` + + + To install `openai<1.0.0`, use the following: + + ```bash pip + pip install "openai<1.0.0" + ``` + ```bash poetry + poetry add "openai<1.0.0" + ``` + + + + + + + ```python python + import agentops + from openai import OpenAI + + agentops.init() + client = OpenAI() + ... + # End of program (e.g. main.py) + agentops.end_session("Success") # Success|Fail|Indeterminate + ``` + + + + + + ```python .env + AGENTOPS_API_KEY= + OPENAI_API_KEY= + ``` + + Read more about environment variables in [Advanced Configuration](/v1/usage/advanced-configuration) + + + + Execute your program and visit [app.agentops.ai/drilldown](https://app.agentops.ai/drilldown) to observe your Agent! 🕵️ + + After your run, AgentOps prints a clickable url to console linking directly to your session in the Dashboard + +
+ + + + + + +## Full Examples + + + ```python sync + from openai import OpenAI + import agentops + + agentops.init() + client = OpenAI() + + response = client.chat.completions.create(( + model="gpt-4o-mini", + messages=[{ + "role": "user", + "content": "Write a haiku about AI and humans working together" + }] + ) + + print(response.choices[0].message.content) + agentops.end_session('Success') + ``` + + ```python async + from openai import AsyncOpenAI + import agentops + import asyncio + + async def main(): + agentops.init() + client = AsyncOpenAI() + + response = await client.chat.completions.create( + model="gpt-4o-mini", + messages=[{ + "role": "user", + "content": "Write a haiku about AI and humans working together" + }] + ) + + print(response.choices[0].message.content) + agentops.end_session('Success') + + asyncio.run(main()) + ``` + + + +### Streaming examples + + + ```python sync + from openai import OpenAI + import agentops + + agentops.init() + client = OpenAI() + + stream = client.chat.completions.create( + model="gpt-4o-mini", + stream=True, + messages=[{ + "role": "user", + "content": "Write a haiku about AI and humans working together" + }], + ) + + for chunk in stream: + print(chunk.choices[0].delta.content or "", end="") + + agentops.end_session('Success') + ``` + + ```python async + from openai import AsyncOpenAI + import agentops + import asyncio + + async def main(): + agentops.init() + client = AsyncOpenAI() + + stream = await client.chat.completions.create( + model="gpt-4o-mini", + stream=True, + messages=[{ + "role": "user", + "content": "Write a haiku about AI and humans working together" + }], + ) + + async for chunk in stream: + print(chunk.choices[0].delta.content or "", end="") + + agentops.end_session('Success') + + asyncio.run(main()) + ``` + + + + + + + + + diff --git a/examples/README.md b/examples/README.md index 34270b964..03c7f4528 100644 --- a/examples/README.md +++ b/examples/README.md @@ -21,8 +21,15 @@ At a high level, AgentOps gives you the ability to monitor LLM calls, costs, lat - [Multi-Agent](./multi_agent_example.ipynb) ## Integrations -- [Using Langchain](./langchain_examples.ipynb) -- [Crew.ai](https://github.com/joaomdmoura/crewAI-examples/tree/main/markdown_validator) - - Crew is a framework for developing agents, a number of their example projects use AgentOps -- [Cohere](./cohere_example.ipynb) -- [Anthropic](./anthropic_example.ipynb) \ No newline at end of file +- [AI21](./ai21_examples/ai21_examples.ipynb) +- [Anthropic](./anthropic_examples/)- +- [Autogen](./autogen_examples/) +- [Cohere](./cohere_examples/cohere_example.ipynb) +- [Crew.ai](./crew_examples/) +- [Groq](./multi_agent_groq_example.ipynb) +- [Langchain](./langchain_examples/langchain_examples.ipynb) +- [LiteLLM](./litlelm_examples/litlelm_example.ipynb) +- [Mistral](./mistral_examples/mistral_example.ipynb) +- [MultiOn](./multion_examples/) +- [Ollama](./ollama_examples/ollama_examples.ipynb) +- [OpenAI](./openai_examples/) diff --git a/examples/openai-gpt.ipynb b/examples/openai-gpt.ipynb index f5ecaf070..43bc65af2 100644 --- a/examples/openai-gpt.ipynb +++ b/examples/openai-gpt.ipynb @@ -254,7 +254,7 @@ " agentops.record(\n", " ActionEvent(\n", " action_type=\"Agent says hello\",\n", - " params=str(message),\n", + " logs=str(message),\n", " returns=str(response.choices[0].message.content),\n", " )\n", " )" diff --git a/examples/openai_examples/README.md b/examples/openai_examples/README.md new file mode 100644 index 000000000..6416b46b6 --- /dev/null +++ b/examples/openai_examples/README.md @@ -0,0 +1,21 @@ +# OpenAI integration with AgentOps + +AgentOps supports observability for OpenAI's API for both version 0.0.x and version 1.x. + +To learn more about OpenAI visit [here!](https://www.openai.com) and their documentation [here](https://platform.openai.com/docs/introduction). + +## Getting Started + +### Prerequisites +* An AgentOps account with an API key +* An OpenAI API key + +Refer to the [AgentOps documentation](https://docs.agentops.ai/getting-started/api-key) for more information on how to get an API key. + +### Documentation +The documentation for the OpenAI integration can be found [here](https://docs.agentops.ai/integrations/openai). + +The example notebooks are present in the [openai_examples](./openai_examples/) directory. + +### License +This project is released under the MIT License. \ No newline at end of file diff --git a/examples/openai_examples/openai_example_async.ipynb b/examples/openai_examples/openai_example_async.ipynb new file mode 100644 index 000000000..2576ea700 --- /dev/null +++ b/examples/openai_examples/openai_example_async.ipynb @@ -0,0 +1,235 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# OpenAI Async Example\n", + "\n", + "We are going to create a simple chatbot that creates stories based on a user provided image. The chatbot will use the gpt-4o-mini LLM to generate the story using a user prompt and its vision model to understand the image.\n", + "\n", + "We will track the chatbot with AgentOps and see how it performs!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First let's install the required packages" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -U openai\n", + "%pip install -U agentops" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then import them" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from openai import AsyncOpenAI\n", + "import agentops\n", + "import os\n", + "from dotenv import load_dotenv" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we'll grab our API keys. You can use dotenv like below or however else you like to load environment variables" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "load_dotenv()\n", + "OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\") or \"\"\n", + "AGENTOPS_API_KEY = os.getenv(\"AGENTOPS_API_KEY\") or \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we initialize the AgentOps client." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "agentops.init(AGENTOPS_API_KEY, default_tags=[\"openai-async-example\"])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And we are all set! Note the seesion url above. We will use it to track the chatbot.\n", + "\n", + "Let's create a simple chatbot that generates stories given an image and a user prompt." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "client = AsyncOpenAI(api_key=OPENAI_API_KEY)\n", + "\n", + "system_prompt = \"\"\"\n", + "You are a master storyteller, with the ability to create vivid and engaging stories.\n", + "You have experience in writing for children and adults alike.\n", + "You are given a prompt and you need to generate a story based on the prompt.\n", + "\"\"\"\n", + "\n", + "user_prompt = [\n", + " {\n", + " \"type\": \"text\",\n", + " \"text\": \"Write a mystery thriller story based on your understanding of the provided image.\"},\n", + " {\n", + " \"type\": \"image_url\",\n", + " \"image_url\": {\"url\": f\"https://www.cosy.sbg.ac.at/~pmeerw/Watermarking/lena_color.gif\"},\n", + " },\n", + "]\n", + "\n", + "messages = [\n", + " {\"role\": \"system\", \"content\": system_prompt},\n", + " {\"role\": \"user\", \"content\": user_prompt},\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "async def main():\n", + " response = await client.chat.completions.create(\n", + " model=\"gpt-4o-mini\",\n", + " messages=messages,\n", + " )\n", + "\n", + " print(response.choices[0].message.content)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "await main()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The response is a string that contains the story. We can track this with AgentOps by navigating to the session url and viewing the run." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Streaming Version\n", + "We will demonstrate the streaming version of the API." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "async def main_stream():\n", + " stream = await client.chat.completions.create(\n", + " model=\"gpt-4o-mini\",\n", + " messages=messages,\n", + " stream=True,\n", + " )\n", + "\n", + " async for chunk in stream:\n", + " print(chunk.choices[0].delta.content or \"\", end=\"\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "await main_stream()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that the response is a generator that yields chunks of the story. We can track this with AgentOps by navigating to the session url and viewing the run." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "agentops.end_session(end_state=\"Success\", end_state_reason=\"The story was generated successfully.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We end the session with a success state and a success reason. This is useful if you want to track the success or failure of the chatbot. In that case you can set the end state to failure and provide a reason. By default the session will have an indeterminate end state.\n", + "\n", + "All done!" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "ops", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.15" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/examples/openai_examples/openai_example_sync.ipynb b/examples/openai_examples/openai_example_sync.ipynb new file mode 100644 index 000000000..4cb1a3b15 --- /dev/null +++ b/examples/openai_examples/openai_example_sync.ipynb @@ -0,0 +1,207 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# OpenAI Sync Example\n", + "\n", + "We are going to create a simple chatbot that creates stories based on a prompt. The chatbot will use the gpt-4o-mini LLM to generate the story using a user prompt.\n", + "\n", + "We will track the chatbot with AgentOps and see how it performs!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "First let's install the required packages" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -U openai\n", + "%pip install -U agentops" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Then import them" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from openai import OpenAI\n", + "import agentops\n", + "import os\n", + "from dotenv import load_dotenv" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we'll grab our API keys. You can use dotenv like below or however else you like to load environment variables" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "load_dotenv()\n", + "OPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\") or \"\"\n", + "AGENTOPS_API_KEY = os.getenv(\"AGENTOPS_API_KEY\") or \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we initialize the AgentOps client." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "agentops.init(AGENTOPS_API_KEY, default_tags=[\"openai-sync-example\"])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And we are all set! Note the seesion url above. We will use it to track the chatbot.\n", + "\n", + "Let's create a simple chatbot that generates stories." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "client = OpenAI(api_key=OPENAI_API_KEY)\n", + "\n", + "system_prompt = \"\"\"\n", + "You are a master storyteller, with the ability to create vivid and engaging stories.\n", + "You have experience in writing for children and adults alike.\n", + "You are given a prompt and you need to generate a story based on the prompt.\n", + "\"\"\"\n", + "\n", + "user_prompt = \"Write a story about a cyber-warrior trapped in the imperial time period.\"\n", + "\n", + "messages = [\n", + " {\"role\": \"system\", \"content\": system_prompt},\n", + " {\"role\": \"user\", \"content\": user_prompt},\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "response = client.chat.completions.create(\n", + " model=\"gpt-4o-mini\",\n", + " messages=messages,\n", + ")\n", + "\n", + "print(response.choices[0].message.content)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The response is a string that contains the story. We can track this with AgentOps by navigating to the session url and viewing the run." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Streaming Version\n", + "We will demonstrate the streaming version of the API." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "stream = client.chat.completions.create(\n", + " model=\"gpt-4o-mini\",\n", + " messages=messages,\n", + " stream=True,\n", + ")\n", + "\n", + "for chunk in stream:\n", + " print(chunk.choices[0].delta.content or \"\", end=\"\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that the response is a generator that yields chunks of the story. We can track this with AgentOps by navigating to the session url and viewing the run." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "agentops.end_session(end_state=\"Success\", end_state_reason=\"The story was generated successfully.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We end the session with a success state and a success reason. This is useful if you want to track the success or failure of the chatbot. In that case you can set the end state to failure and provide a reason. By default the session will have an indeterminate end state.\n", + "\n", + "All done!" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "ops", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.15" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}