Skip to content

Commit d8f07b0

Browse files
isaacmbrownCopilot
andauthored
Add concepts page for OpenTelemetry (#62726)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
1 parent 90ce488 commit d8f07b0

3 files changed

Lines changed: 55 additions & 0 deletions

File tree

content/copilot/concepts/agents/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ children:
2222
- /anthropic-claude
2323
- /about-agent-skills
2424
- /enterprise-management
25+
- /opentelemetry
2526
contentType: concepts
2627
---
2728

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
title: OpenTelemetry for agent monitoring
3+
shortTitle: OpenTelemetry
4+
intro: Understand how {% data variables.product.prodname_copilot_short %} agents perform and interact with models and tools.
5+
versions:
6+
feature: copilot
7+
contentType: concepts
8+
category:
9+
- Manage Copilot for a team
10+
---
11+
12+
OpenTelemetry (OTel) is an open source observability framework. It provides a standard way to collect telemetry events and metrics and export them to compatible observability tools. For more information, see [What is OpenTelemetry?](https://opentelemetry.io/docs/what-is-opentelemetry/) on the OTel website.
13+
14+
When you enable OTel monitoring, you can send data from users' {% data variables.product.prodname_copilot_short %} clients to an OTel-compatible backend. This lets you analyze agent sessions and understand agent usage across your enterprise.
15+
16+
## What data does {% data variables.product.prodname_copilot_short %} send?
17+
18+
{% data variables.product.prodname_copilot_short %} sends three types of data:
19+
20+
* **Traces** show the flow of an agent session and connect each step, including model calls and tool use. For example, a trace can show an agent calling a model, using the `readFile` tool, and calling the model again to produce a response.
21+
* **Metrics** are numeric measurements that help you identify patterns over time. For example, token usage metrics track the number of input and output tokens used in model calls.
22+
* **Events** record individual actions at a specific point in time. For example, an edit feedback event records whether a user accepted or rejected an agent edit.
23+
24+
By default, the data does not include prompts, responses, or tool arguments. You can choose to capture this content, but it may contain sensitive information, such as code, file contents, and user prompts.
25+
26+
## Enabling OpenTelemetry
27+
28+
To collect OTel data from users' {% data variables.product.prodname_copilot_short %} clients:
29+
30+
1. **Set up an observability backend.** Choose a secure backend that supports the OpenTelemetry Protocol (OTLP). Some backends can receive OTLP data directly. For other backends, deploy an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) to receive, process, and forward the data. For examples, see [Use with observability backends](https://code.visualstudio.com/docs/agents/guides/monitoring-agents#_use-with-observability-backends) in the {% data variables.product.prodname_vscode_shortname %} documentation.
31+
1. **Configure users' clients.** Set configuration values to enable OTel in each client and configure it to send data to your OTLP endpoint by configuring the endpoint, headers, and authentication token.
32+
1. **Interpret data.** Use the collected data to analyze sessions and identify trends. For an example implementation, see [Monitor AI coding agents with Grafana](https://learn.microsoft.com/en-gb/azure/managed-grafana/grafana-opentelemetry-app-insights) in the Microsoft documentation.
33+
34+
### Configuring users' clients
35+
36+
Enterprises can enforce OTel configuration across supported clients with managed settings. These settings are enforced across users' clients and cannot be overridden. The `telemetry` property includes keys for enabling and configuring OpenTelemetry.
37+
38+
For more information, see [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-agents/configure-enterprise-managed-settings) and [AUTOTITLE](/copilot/reference/enterprise-administrators/enterprise-managed-settings).
39+
40+
## Client documentation
41+
42+
For more details of how OpenTelemetry is used across clients, see:
43+
44+
* [Monitor agent usage with OpenTelemetry](https://code.visualstudio.com/docs/agents/guides/monitoring-agents) in the {% data variables.product.prodname_vscode_shortname %} documentation
45+
* [AUTOTITLE](/copilot/how-tos/copilot-sdk/observability/opentelemetry)
46+
* [AUTOTITLE](/copilot/reference/copilot-cli-reference/cli-command-reference#opentelemetry-monitoring)

content/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-agents/monitor-agentic-activity.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,11 @@ Track agentic activity on {% data variables.product.github %} or through streami
3434
To enable streaming for {% data variables.product.prodname_copilot_short %} agent session events and configure a streaming destination from your enterprise audit log settings, see [AUTOTITLE](/enterprise-cloud@latest/admin/monitoring-activity-in-your-enterprise/reviewing-audit-logs-for-your-enterprise/streaming-the-audit-log-for-your-enterprise#enabling-audit-log-streaming-of-copilot-agent-session-events).
3535

3636
In addition to streaming, you can also retrieve Copilot usage data through the REST API. See [AUTOTITLE](/rest/copilot/copilot-usage-metrics#get-copilot-usage-records-for-an-enterprise).
37+
38+
## Enabling OpenTelemetry
39+
40+
OpenTelemetry (OTel) is an open source observability framework. It provides a standard way to collect telemetry events and metrics and export them to compatible observability tools.
41+
42+
When you enable OTel monitoring, you can send data from users' {% data variables.product.prodname_copilot_short %} clients to an OTel-compatible backend. This lets you analyze agent sessions and understand agent usage across your enterprise.
43+
44+
For more information, see [AUTOTITLE](/copilot/concepts/agents/opentelemetry).

0 commit comments

Comments
 (0)