|
| 1 | +--- |
| 2 | +title: OpenTelemetry |
| 3 | +description: Send traces to ZeroEval using the OpenTelemetry collector |
| 4 | +--- |
| 5 | + |
| 6 | +ZeroEval provides native support for the OpenTelemetry Protocol (OTLP), allowing you to send traces from any OpenTelemetry-instrumented application directly to ZeroEval's API. This guide shows you how to configure the OpenTelemetry collector to export traces to ZeroEval. |
| 7 | + |
| 8 | + |
| 9 | +## Prerequisites |
| 10 | + |
| 11 | +- A ZeroEval API key (get one from your [workspace settings](https://app.zeroeval.com/settings/api-keys)) |
| 12 | +- OpenTelemetry collector installed ([installation guide](https://opentelemetry.io/docs/collector/getting-started/)) |
| 13 | + |
| 14 | +## Configuration |
| 15 | + |
| 16 | +Create a collector configuration file (`otel-collector-config.yaml`): |
| 17 | + |
| 18 | +```yaml |
| 19 | +receivers: |
| 20 | + otlp: |
| 21 | + protocols: |
| 22 | + grpc: |
| 23 | + endpoint: 0.0.0.0:4317 |
| 24 | + http: |
| 25 | + endpoint: 0.0.0.0:4318 |
| 26 | + |
| 27 | +processors: |
| 28 | + batch: |
| 29 | + timeout: 1s |
| 30 | + send_batch_size: 1024 |
| 31 | + |
| 32 | + # ZeroEval-specific attributes |
| 33 | + attributes: |
| 34 | + actions: |
| 35 | + - key: deployment.environment |
| 36 | + value: "production" # or staging, development, etc. |
| 37 | + action: upsert |
| 38 | + |
| 39 | +exporters: |
| 40 | + otlphttp: |
| 41 | + endpoint: https://api.zeroeval.com |
| 42 | + headers: |
| 43 | + Authorization: "Bearer YOUR_ZEROEVAL_API_KEY" |
| 44 | + traces_endpoint: https://api.zeroeval.com/v1/traces |
| 45 | + |
| 46 | +service: |
| 47 | + pipelines: |
| 48 | + traces: |
| 49 | + receivers: [otlp] |
| 50 | + processors: [batch, attributes] |
| 51 | + exporters: [otlphttp] |
| 52 | +``` |
| 53 | +
|
| 54 | +## Docker Deployment |
| 55 | +
|
| 56 | +For containerized deployments, use this Docker Compose configuration: |
| 57 | +
|
| 58 | +```yaml |
| 59 | +version: '3.8' |
| 60 | + |
| 61 | +services: |
| 62 | + otel-collector: |
| 63 | + image: otel/opentelemetry-collector-contrib:latest |
| 64 | + container_name: otel-collector |
| 65 | + command: ["--config=/etc/otel-collector-config.yaml"] |
| 66 | + volumes: |
| 67 | + - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml |
| 68 | + ports: |
| 69 | + - "4317:4317" # OTLP gRPC receiver |
| 70 | + - "4318:4318" # OTLP HTTP receiver |
| 71 | + - "8888:8888" # Prometheus metrics |
| 72 | + environment: |
| 73 | + - ZEROEVAL_API_KEY=${ZEROEVAL_API_KEY} |
| 74 | + restart: unless-stopped |
| 75 | +``` |
| 76 | +
|
| 77 | +## Environment-based Configuration |
| 78 | +
|
| 79 | +To avoid hardcoding sensitive information, use environment variables: |
| 80 | +
|
| 81 | +```yaml |
| 82 | +exporters: |
| 83 | + otlphttp: |
| 84 | + endpoint: https://api.zeroeval.com |
| 85 | + headers: |
| 86 | + Authorization: "Bearer ${env:ZEROEVAL_API_KEY}" |
| 87 | + traces_endpoint: https://api.zeroeval.com/v1/traces |
| 88 | +``` |
| 89 | +
|
| 90 | +Then set the environment variable: |
| 91 | +
|
| 92 | +```bash |
| 93 | +export ZEROEVAL_API_KEY="your-api-key-here" |
| 94 | +``` |
| 95 | + |
| 96 | +## Advanced Configuration |
| 97 | + |
| 98 | +### Custom Session Management |
| 99 | + |
| 100 | +ZeroEval automatically creates sessions based on service attributes. You can customize this behavior by adding specific attributes: |
| 101 | + |
| 102 | +```yaml |
| 103 | +processors: |
| 104 | + attributes: |
| 105 | + actions: |
| 106 | + # Custom session ID |
| 107 | + - key: zeroeval.session.id |
| 108 | + value: "custom-session-uuid" |
| 109 | + action: upsert |
| 110 | + # Custom session name |
| 111 | + - key: zeroeval.session.name |
| 112 | + value: "User Journey - Checkout Flow" |
| 113 | + action: upsert |
| 114 | +``` |
| 115 | +
|
| 116 | +### Filtering and Sampling |
| 117 | +
|
| 118 | +To reduce data volume, configure sampling: |
| 119 | +
|
| 120 | +```yaml |
| 121 | +processors: |
| 122 | + probabilistic_sampler: |
| 123 | + sampling_percentage: 10 # Sample 10% of traces |
| 124 | + |
| 125 | +service: |
| 126 | + pipelines: |
| 127 | + traces: |
| 128 | + receivers: [otlp] |
| 129 | + processors: [batch, attributes, probabilistic_sampler] |
| 130 | + exporters: [otlphttp] |
| 131 | +``` |
| 132 | +
|
| 133 | +### Resource Detection |
| 134 | +
|
| 135 | +Automatically detect and add resource attributes: |
| 136 | +
|
| 137 | +```yaml |
| 138 | +processors: |
| 139 | + resourcedetection: |
| 140 | + detectors: [env, system, docker, k8s_node] |
| 141 | + system: |
| 142 | + hostname_sources: ["os"] |
| 143 | + |
| 144 | +service: |
| 145 | + pipelines: |
| 146 | + traces: |
| 147 | + receivers: [otlp] |
| 148 | + processors: [resourcedetection, batch, attributes] |
| 149 | + exporters: [otlphttp] |
| 150 | +``` |
| 151 | +
|
| 152 | +## Kubernetes Deployment |
| 153 | +
|
| 154 | +For Kubernetes environments, use this ConfigMap and Deployment: |
| 155 | +
|
| 156 | +```yaml |
| 157 | +apiVersion: v1 |
| 158 | +kind: ConfigMap |
| 159 | +metadata: |
| 160 | + name: otel-collector-config |
| 161 | +data: |
| 162 | + otel-collector-config.yaml: | |
| 163 | + receivers: |
| 164 | + otlp: |
| 165 | + protocols: |
| 166 | + grpc: |
| 167 | + endpoint: 0.0.0.0:4317 |
| 168 | + http: |
| 169 | + endpoint: 0.0.0.0:4318 |
| 170 | + |
| 171 | + processors: |
| 172 | + batch: |
| 173 | + timeout: 1s |
| 174 | + |
| 175 | + k8sattributes: |
| 176 | + extract: |
| 177 | + metadata: |
| 178 | + - k8s.namespace.name |
| 179 | + - k8s.deployment.name |
| 180 | + - k8s.pod.name |
| 181 | + |
| 182 | + exporters: |
| 183 | + otlphttp: |
| 184 | + endpoint: https://api.zeroeval.com |
| 185 | + headers: |
| 186 | + Authorization: "Bearer ${env:ZEROEVAL_API_KEY}" |
| 187 | + traces_endpoint: https://api.zeroeval.com/v1/traces |
| 188 | + |
| 189 | + service: |
| 190 | + pipelines: |
| 191 | + traces: |
| 192 | + receivers: [otlp] |
| 193 | + processors: [batch, k8sattributes] |
| 194 | + exporters: [otlphttp] |
| 195 | +
|
| 196 | +--- |
| 197 | +apiVersion: apps/v1 |
| 198 | +kind: Deployment |
| 199 | +metadata: |
| 200 | + name: otel-collector |
| 201 | +spec: |
| 202 | + replicas: 2 |
| 203 | + selector: |
| 204 | + matchLabels: |
| 205 | + app: otel-collector |
| 206 | + template: |
| 207 | + metadata: |
| 208 | + labels: |
| 209 | + app: otel-collector |
| 210 | + spec: |
| 211 | + containers: |
| 212 | + - name: otel-collector |
| 213 | + image: otel/opentelemetry-collector-contrib:latest |
| 214 | + args: ["--config=/etc/otel-collector-config.yaml"] |
| 215 | + env: |
| 216 | + - name: ZEROEVAL_API_KEY |
| 217 | + valueFrom: |
| 218 | + secretKeyRef: |
| 219 | + name: zeroeval-secret |
| 220 | + key: api-key |
| 221 | + ports: |
| 222 | + - containerPort: 4317 |
| 223 | + name: otlp-grpc |
| 224 | + - containerPort: 4318 |
| 225 | + name: otlp-http |
| 226 | + volumeMounts: |
| 227 | + - name: config |
| 228 | + mountPath: /etc/otel-collector-config.yaml |
| 229 | + subPath: otel-collector-config.yaml |
| 230 | + volumes: |
| 231 | + - name: config |
| 232 | + configMap: |
| 233 | + name: otel-collector-config |
| 234 | + |
| 235 | +--- |
| 236 | +apiVersion: v1 |
| 237 | +kind: Service |
| 238 | +metadata: |
| 239 | + name: otel-collector |
| 240 | +spec: |
| 241 | + selector: |
| 242 | + app: otel-collector |
| 243 | + ports: |
| 244 | + - name: otlp-grpc |
| 245 | + port: 4317 |
| 246 | + targetPort: 4317 |
| 247 | + - name: otlp-http |
| 248 | + port: 4318 |
| 249 | + targetPort: 4318 |
| 250 | +``` |
| 251 | +
|
| 252 | +## Verifying Your Setup |
| 253 | +
|
| 254 | +Once configured, verify that traces are being sent to ZeroEval: |
| 255 | +
|
| 256 | +1. **Check Collector Logs**: |
| 257 | + ```bash |
| 258 | + docker logs otel-collector |
| 259 | + # or for Kubernetes |
| 260 | + kubectl logs -l app=otel-collector |
| 261 | + ``` |
| 262 | + |
| 263 | +2. **Monitor Collector Metrics**: |
| 264 | + Access the Prometheus metrics endpoint at `http://localhost:8888/metrics` to see export statistics. |
| 265 | + |
| 266 | +3. **View Traces in ZeroEval**: |
| 267 | + Navigate to your [ZeroEval dashboard](https://app.zeroeval.com) to see incoming traces. |
| 268 | + |
| 269 | +## Troubleshooting |
| 270 | + |
| 271 | +### Authentication Errors |
| 272 | + |
| 273 | +If you see 401 errors, verify: |
| 274 | +- Your API key is valid and active |
| 275 | +- The `Authorization` header format is exactly: `Bearer YOUR_API_KEY` |
| 276 | +- The API key has the necessary permissions |
| 277 | + |
| 278 | +### Connection Issues |
| 279 | + |
| 280 | +For connection problems: |
| 281 | +- Ensure the endpoint is `https://api.zeroeval.com/v1/traces` |
| 282 | +- Check network connectivity and firewall rules |
| 283 | +- Verify TLS/SSL certificates if behind a proxy |
0 commit comments