Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/assets/datadog-observability.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
85 changes: 85 additions & 0 deletions docs/observability/datadog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Agent Observability with Datadog

[Datadog LLM
Observability](https://www.datadoghq.com/product/llm-observability/) helps AI
engineers, data scientists, and application developers quickly develop,
evaluate, and monitor LLM applications. Confidently improve output quality,
performance, costs, and overall risk with structured experiments, end-to-end
tracing across AI agents, and evaluations.

## Overview

Datadog LLM Observability can [automatically instrument and trace your agents
built on Google
ADK](https://docs.datadoghq.com/llm_observability/instrumentation/auto_instrumentation?tab=python#google-adk),
allowing you to:

- **Observe agent executions and interactions** - Automatically capture every
agent run, tool call, and code execution within your agents
- **Capture LLM calls and responses** made with the underlying Google GenAI SDK
- **Debug issues** by providing error rates, token usage and cost, and
out-of-the-box evaluations on your LLM calls and tool usage

## Prerequisites

Sign up for a [Datadog account](https://www.datadoghq.com/) if you do not have
one and [get your API
key](https://docs.datadoghq.com/account_management/api-app-keys/#api-keys).

## Installation

Install the required packages:

```bash
pip install ddtrace
```

## Setup

### Configure Environment Variables

You will also need to specify an ML Application name in the following
environment variables. An ML Application is a grouping of LLM Observability
traces associated with a specific LLM-based application. See [ML Application
Naming
Guidelines](https://docs.datadoghq.com/llm_observability/instrumentation/sdk?tab=python#application-naming-guidelines)
for more information on limitations with ML Application names.

```shell
export DD_API_KEY=<YOUR_DD_API_KEY>
export DD_SITE=<YOUR_DD_SITE>
export DD_LLMOBS_ENABLED=true
export DD_LLMOBS_ML_APP=<YOUR_ML_APP_NAME>
export DD_LLMOBS_AGENTLESS_ENABLED=true
export DD_APM_TRACING_ENABLED=false # Only set this if you are not using Datadog APM
```

Additionally, configure any LLM provider API keys:

```shell
export GEMINI_API_KEY=<YOUR_GEMINI_API_KEY>
# or
export GOOGLE_API_KEY=<YOUR_GOOGLE_API_KEY>
...
```

### Run Your Application

Once you have configured your environment variables, you can run your
application and start observing your LLM-based applications.

```shell
ddtrace-run python your_application.py
```

## Observe

Navigate to the [Datadog LLM Observability Traces
View](https://app.datadoghq.com/llm/traces) to see the traces generated by your
application.

![datadog-observability.png](../assets/datadog-observability.png)

## Support and Resources
- [Datadog LLM Observability](https://www.datadoghq.com/product/llm-observability/)
- [Datadog Support](https://docs.datadoghq.com/help/)
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,7 @@ nav:
- Cloud Trace: observability/cloud-trace.md
- AgentOps: observability/agentops.md
- Arize AX: observability/arize-ax.md
- Datadog: observability/datadog.md
- Freeplay: observability/freeplay.md
- Monocle: observability/monocle.md
- Phoenix: observability/phoenix.md
Expand Down