[bot] Anthropic Message Batches API is not instrumented (mis-tagged as a generic LLM span)

[braintrust-bot]

Summary

The Anthropic instrumentation module (anthropic_2_2_0) only has shape-aware tagging for the Messages API (messages().create/streaming). It has no support at all for the Message Batches API (client.messages().batches()), a stable, GA, actively-developed part of the Anthropic Java SDK for submitting many message-generation requests as a single asynchronous job.

Because the generic transport-swap (TracingHttpClient) still intercepts the batch HTTP calls, a batches().create(...) call does produce a span — but it is actively mis-tagged rather than simply absent: it gets span_attributes.type = "llm" (as if it were a real model call), a low-information span name ("batches", from the path-segment fallback), no model in metadata, and no input_json at all — because the request/response shapes for batches are structurally different from a single Messages call and none of the existing field checks match them.

What is missing

In braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java:

• getSpanName() (lines 511–522) switches on provider + ":" + lastPathSegment. For POST /v1/messages/batches, the last segment is "batches", which doesn't match the PROVIDER_NAME_ANTHROPIC + ":messages" case, so it falls through to default -> lastSegment, yielding the span name "batches" instead of something like "anthropic.messages.batches.create".
• tagAnthropicRequest() (lines 208–255) unconditionally sets span_attributes = {"type":"llm"} (line 216) even though a batch-create call isn't itself a model invocation. It only populates metadata.model and braintrust.input_json when requestJson.has("model") / requestJson.has("messages") (lines 231, 235) — but a BatchCreateParams request body has neither at the top level; it has a requests[] array where each entry is {"custom_id": "...", "params": {"model": ..., "messages": ..., "max_tokens": ..., ...}}. Every one of the (potentially many) inline generation requests in the batch is silently dropped from the span.
• tagAnthropicResponse() (lines 258–301) dumps the whole response body as output_json (harmless for batch-create, since the response is just batch job metadata: id, processing_status, request_counts, results_url) and only extracts metrics from a top-level usage object (line 270), which a batch-create/retrieve response never has — so no metrics are produced (correctly, since none exist yet at creation time, but there's also no instrumentation of the batch results retrieval, which is where the actual per-request usage and message outputs become available).
• There is no wrapping at all of batches().results(...)/resultsStreaming(...), so even after a batch completes, iterating its per-request results (each containing a custom_id and either a succeeded message with full usage, or an error) produces zero spans — unlike a normal messages().create() call, none of the individual generation results in a batch get any Braintrust span.
• No test or example anywhere in the repo exercises client.messages().batches() in any form (confirmed via repo-wide grep for batch/Batch in anthropic_2_2_0 — zero matches).

Braintrust docs status: supported (in a sibling SDK) / not_found (for Java)

• The Java section of https://www.braintrust.dev/docs/integrations/ai-providers/anthropic states only: "Braintrust emits spans for the Anthropic Messages API. Each span captures the input messages and response content," with a spans table listing exactly one row — "Anthropic Messages API spans" covering messages().create() calls including streaming. Batches are not mentioned for Java at all: not_found.
• The Python section of the same page states: "Braintrust emits spans for the Anthropic SDK's messages, batches, and managed agents APIs," with a spans-table row for anthropic.messages.batches.* explicitly covering the Batches API: supported (in the Python SDK). This establishes Message Batches tracing as an existing, documented Braintrust capability that the Java SDK has not brought to parity.

Upstream sources

• Anthropic Java SDK Batches sub-client: client.messages().batches().create(BatchCreateParams), .retrieve(...), .list(...), .cancel(...), .resultsStreaming(BatchResultsParams) — official Java code samples in the batch-processing guide: https://platform.claude.com/docs/en/build-with-claude/batch-processing (mirrored at https://docs.anthropic.com/en/docs/build-with-claude/batch-processing)
• Batch create request shape — each requests[] entry has custom_id + params containing "the standard Messages API parameters" (model, max_tokens, messages, system, tools, thinking, etc.), i.e. inline generation requests, not file/S3 references: same guide, "Prepare and create your batch" section
• Batch results shape — streamed JSONL keyed by custom_id, each with result.type of succeeded (containing the full message), errored, canceled, or expired: https://platform.claude.com/docs/en/api/messages/batches/results
• Currency/stability — Anthropic's official API release notes confirm this is an actively maintained, GA surface (e.g. March 30, 2026 entry raising max_tokens to 300k specifically "on the Message Batches API"; the Claude-on-AWS platform announcement listing "the full Messages API, Files API, Message Batches API, Claude Managed Agents" as core stable surfaces): https://platform.claude.com/docs/en/release-notes/api

Local files inspected

• braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 208–255 (tagAnthropicRequest: no top-level model/messages in a batch-create body, so metadata/input_json stay empty), lines 258–301 (tagAnthropicResponse: whole-body dump plus usage-only metrics, batch-create/retrieve responses have neither usage nor useful whole-body content), lines 511–522 (getSpanName: "batches" path segment falls through to the generic default case)
• braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java and ContextCapturingProxy.java — generic transport-swap/service-graph-following wrapper; produces a span for any Anthropic HTTP call including batches, but has no batch-specific logic
• braintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/dev/braintrust/instrumentation/anthropic/v2_2_0/BraintrustAnthropicTest.java and BraintrustAnthropicPromptCachingTest.java — no test exercises batches() in any form
• examples/anthropic-instrumentation/ — no batch example exists
• Repo-wide grep for batch/Batch under braintrust-sdk/instrumentation/anthropic_2_2_0/ — zero matches