Skip to content

fix(middleware)!: name HTTP spans by route template, not raw path#30

Merged
gauchito91 merged 3 commits into
developfrom
fix/span-name-cardinality-pii
Jul 21, 2026
Merged

fix(middleware)!: name HTTP spans by route template, not raw path#30
gauchito91 merged 3 commits into
developfrom
fix/span-name-cardinality-pii

Conversation

@gauchito91

Copy link
Copy Markdown
Contributor

Problema

Spans HTTP SERVER eram nomeados a partir do caminho resolvido com os IDs embutidos (ex.: GET /v1/dict/statistics/keys/<chave-pix>), causando:

  1. Explosão de cardinalidade — um serviço gerou ~50k valores distintos de span.name, ~72% das séries ativas no Mimir de prod (o spanmetrics connector usa span.name como dimensão).
  2. Vazamento de PII (LGPD) — chave Pix / CPF / CNPJ trafegavam crus no nome do span até o backend de traces.

Solução

Nomear spans pela convenção OTel {method} {http.route} usando o template de rota (baixa cardinalidade), que exclui PII estruturalmente:

  • Span criado com nome só-método (PII-free na criação — o sampler e spans descartados/não-casados nunca carregam identificador).
  • Após o roteamento, rename para {method} {http.route} (guardado por IsRecording()); em 404/não-casado mantém só-método e omite http.route (conforme spec).

Correção do hazard de ciclo de vida (ADR-001)

O fix expôs que o EndTracingSpans, por LIFO do Fiber, desenrola antes do WithTelemetry aplicar o rename/atributos — encerrando o span cedo e descartando http.route/status/rename silenciosamente (ocorre inclusive na ordem correta do standard, não só em consumidores mal-ordenados). Adicionado flag owned: WithTelemetry é o único encerrador do seu span raiz; EndTracingSpans pula spans owned mas preserva o encerramento de spans estrangeiros/handler-criados. sync.Once mantido para idempotência (par gRPC + fallback).

Processo (Ring pre-dev)

Planejado via fluxo Ring completo (research → PRD → TRD → tasks, em docs/pre-dev/fix-span-name-cardinality-pii/). TRD revisado por engenheiro Go. Implementação revisada por 3 reviewers (qualidade + correção + segurança) — 3/3 PASS, sem achados bloqueantes.

Testes

  • go build + go test -tags=unit ./... verdes; golangci-lint 0 issues.
  • Novos: 3 permutações de ordem de middleware; nome só-método na criação; naming por template (chaves numérica+UUID colapsam, nenhuma vaza); units de routeAttribute.
  • Preservado: TestEndTracingSpans_EndsFinalContextSpan (fallback de span estrangeiro).

⚠️ BREAKING CHANGE

O formato do nome dos spans HTTP muda do caminho cru para {method} {http.route}. Dashboards/queries que agrupam traces pelo nome antigo do span precisam ser atualizados. Samplers baseados em nome agora veem só-método na criação.

Follow-ups (não neste PR)

  • Atributo url.path ainda carrega o path resolvido (sanitizeURL limpa só query params, não templatiza segmentos) → 2º vetor de PII para segmentos como /keys/<cpf>. Fix separado.
  • Aplicar semântica owned ao par gRPC se um rename pós-handler for adicionado no futuro (hoje desnecessário).
  • Migração do plugin-br-pix-indirect-btg para consumir esta versão (feature separada, corrige a ordem do middleware).

🤖 Generated with Claude Code

HTTP SERVER spans were named from the resolved request path with IDs
embedded (e.g. "GET /v1/dict/statistics/keys/<pix-key>"), which
(1) exploded spanmetrics cardinality — one service produced ~50k
distinct span.name values, ~72% of active series in prod Mimir — and
(2) leaked PII (Pix keys, CPF/CNPJ) raw into span names shipped to the
trace backend.

Name spans per the OTel convention "{method} {http.route}" using the
low-cardinality route template, which structurally excludes PII:
- Create the span with a method-only name (PII-free at creation, so the
  sampler and any dropped/unmatched spans never carry an identifier).
- After routing resolves, rename to "{method} {http.route}" (guarded by
  IsRecording()); on unmatched/404 keep the method-only name and omit
  http.route, per spec.

Fix the span-lifecycle ordering hazard this exposed: a separately
registered EndTracingSpans unwinds first under Fiber LIFO and ended the
span before WithTelemetry applied the rename/attributes, silently
dropping them (present in the standard ordering too, not just misordered
consumers). Add an `owned` flag so WithTelemetry is the sole ender of its
root span; EndTracingSpans skips owned spans but still ends
foreign/handler-created spans (fallback preserved). sync.Once is retained
for double-end idempotency (gRPC pair + foreign-span path).

Remove the now-dead replaceUUIDWithPlaceholder helper (the span name no
longer derives from the path). gRPC interceptors are intentionally left
unchanged (named from info.FullMethod; no PII/cardinality).

Tests: 3 middleware-ordering permutations, method-only creation name,
template naming with numeric+UUID keys collapsing, routeAttribute units.

BREAKING CHANGE: HTTP server span names change from the raw request path
to "{method} {http.route}". Dashboards/queries grouping traces by the old
span name must be updated. Name-based samplers now see a method-only name
at span creation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
X-Lerian-Ref: 0x1
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 27eac70e-2412-4dcd-8c84-9775edfd4a9d

📥 Commits

Reviewing files that changed from the base of the PR and between 325d218 and 4773b1e.

📒 Files selected for processing (1)
  • middleware/telemetry.go

📝 Walkthrough

Walkthrough

HTTP telemetry now creates spans with method-only names, renames them to route templates after routing, and assigns root-span finalization to WithTelemetry. Documentation defines the design, while tests cover routing, PII/cardinality constraints, middleware ordering, and single-span completion.

Changes

HTTP Span Naming and Lifecycle

Layer / File(s) Summary
Naming and lifecycle design
docs/pre-dev/fix-span-name-cardinality-pii/*
PRD, research, tasks, and TRD documents define route-template naming, lifecycle ownership, fallbacks, validation, and breaking-change requirements.
HTTP and gRPC span creation and finalization
middleware/helpers.go, middleware/telemetry.go
HTTP spans start with method-only names, are renamed using route templates after routing, and owned HTTP and gRPC spans are finalized without duplicate ending. UUID placeholder helpers are removed.
Route and span behavior validation
middleware/helpers_test.go, middleware/telemetry_span_lifecycle_test.go, middleware/telemetry_test.go
Tests cover route extraction, unmatched routes, middleware ordering, single finalization, identifier-free names, creation-time naming, and updated expectations.

Sequence Diagram(s)

sequenceDiagram
  participant FiberRouter
  participant WithTelemetry
  participant OpenTelemetrySpan
  participant EndTracingSpans
  WithTelemetry->>OpenTelemetrySpan: Start span with method-only name
  WithTelemetry->>FiberRouter: Execute request routing
  FiberRouter-->>WithTelemetry: Return route template
  WithTelemetry->>OpenTelemetrySpan: Set route-based name and attributes
  EndTracingSpans->>WithTelemetry: Check span ownership
  WithTelemetry->>OpenTelemetrySpan: End owned root span once
Loading

Possibly related PRs

✨ Finishing Touches
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch fix/span-name-cardinality-pii

Comment @coderabbitai help to get the list of available commands.

@lerian-studio lerian-studio added size/L PR changes 500–999 lines docs Documentation, llms.txt and markdown content middleware HTTP/gRPC observability middleware tests Unit, integration and end-to-end tests labels Jul 20, 2026
@lerian-studio

lerian-studio commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

🔍 PR Validation Summary

✅ PR Mergeable — no blocking failures

Check Status Blocking
Source Branch ✅ success yes
PR Title ✅ success yes
PR Description ✅ success yes
PR Size ✅ success no
Auto Labels ✅ success no
PR Metadata ✅ success no

🔍 View workflow run

@lerian-studio

lerian-studio commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

🔒 Security Scan Results — lib-observability

🚫 PR Blocked — 1 blocking finding

Stage Status Blocking?
Filesystem Scan ⚠️ 1 finding 🔴 Yes
Docker Image Scan ➖ Skipped
Docker Hub Health Score ➖ Skipped
Pre-release Version Check ✅ Clean

Trivy

Filesystem Scan

Severity Library Vulnerability Installed Fixed Title
⚪ UNKNOWN golang.org/x/text CVE-2026-56852 v0.38.0 0.39.0 Infinite loop on invalid input in golang.org/x/text

Pre-release Version Check

✅ No unstable version pins found.


🔍 View full scan logs

@lerian-studio

lerian-studio commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

📊 Unit Test Coverage Report: lib-observability

Metric Value
Overall Coverage 85.7% ✅ PASS
Threshold 80%

Coverage by Package

Package Coverage
github.com/LerianStudio/lib-observability/v2/assert 97.9%
github.com/LerianStudio/lib-observability/v2/constants 83.3%
github.com/LerianStudio/lib-observability/v2/log 94.9%
github.com/LerianStudio/lib-observability/v2/metrics 91.4%
github.com/LerianStudio/lib-observability/v2/middleware 76.8%
github.com/LerianStudio/lib-observability/v2/redaction 95.8%
github.com/LerianStudio/lib-observability/v2/runtime 80.4%
github.com/LerianStudio/lib-observability/v2/tracing 84.8%
github.com/LerianStudio/lib-observability/v2/zap 96.0%
github.com/LerianStudio/lib-observability/v2 91.5%

Generated by Go PR Analysis workflow

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@middleware/helpers_test.go`:
- Around line 90-92: Replace assert.NoError with require.NoError for the
app.Test calls in the affected test cases before accessing resp, including the
instances around the user request, subsequent request at the additional
locations, and any resp.Body dereferences. Keep the existing response cleanup
and assertions unchanged.

In `@middleware/telemetry_span_lifecycle_test.go`:
- Around line 137-183: Add a test case in the existing telemetry middleware
ordering tests for registering EndTracingSpans before WithTelemetry, using
app.Use(mid.EndTracingSpans) followed by app.Use(mid.WithTelemetry(tel)). Send
the request and assert a successful response, then verify the finalized route
name and http.route through assertRootSpanEndedOnce.

In `@middleware/telemetry.go`:
- Around line 525-530: Update the gRPC interceptor span setup near the existing
ownership comment to mark the span as owned, preserving finalization until
WithTelemetryInterceptor completes its post-handler attribute and status
recording. Remove or revise the outdated explanation that intentionally leaves
the span unowned; do not rely on a regression test as a substitute for restoring
the ordering guarantee.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 38c41a0d-fb22-4b58-8de0-37182f3ae7eb

📥 Commits

Reviewing files that changed from the base of the PR and between eb521b5 and d3b33f8.

📒 Files selected for processing (9)
  • docs/pre-dev/fix-span-name-cardinality-pii/prd.md
  • docs/pre-dev/fix-span-name-cardinality-pii/research.md
  • docs/pre-dev/fix-span-name-cardinality-pii/tasks.md
  • docs/pre-dev/fix-span-name-cardinality-pii/trd.md
  • middleware/helpers.go
  • middleware/helpers_test.go
  • middleware/telemetry.go
  • middleware/telemetry_span_lifecycle_test.go
  • middleware/telemetry_test.go
💤 Files with no reviewable changes (1)
  • middleware/helpers.go

Comment thread middleware/helpers_test.go
Comment thread middleware/telemetry_span_lifecycle_test.go
Comment thread middleware/telemetry.go Outdated
- Mark the gRPC WithTelemetryInterceptor span as owned and skip it in
  EndTracingSpansInterceptor, mirroring the HTTP pair. The gRPC span records
  rpc.method / rpc.grpc.status_code / handler error status AFTER the handler
  returns, so an end-interceptor that unwinds first could drop them — the same
  ordering hazard the HTTP owned flag fixes. (CodeRabbit)
- helpers_test.go: use require.NoError before dereferencing resp so a failed
  app.Test aborts instead of panicking on resp.Body and hiding the real cause.
  (CodeRabbit)
- telemetry_span_lifecycle_test.go: add the 4th ordering permutation —
  EndTracingSpans registered BEFORE WithTelemetry (outermost, unwinds last) —
  asserting the owned span is not double-ended and finalization is preserved.
  (CodeRabbit)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
X-Lerian-Ref: 0x1

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
middleware/telemetry.go (1)

449-460: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Only skip when the active span matches the owned root span
state.owned is inherited by child contexts, so a handler/interceptor that installs a distinct span can still trip this early return and leave that active span open. Compare trace.SpanFromContext(endCtx) with state.span before skipping, and apply the same check in both:

  • middleware/telemetry.go#L449-L460
  • middleware/telemetry.go#L569-L576
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@middleware/telemetry.go` around lines 449 - 460, In middleware/telemetry.go
lines 449-460 and 569-576, update the state.owned early-return condition to skip
only when trace.SpanFromContext(endCtx) matches state.span; otherwise call
state.End(). Apply the same matching check at both sites so child contexts with
distinct active spans are finalized correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@middleware/telemetry.go`:
- Around line 449-460: In middleware/telemetry.go lines 449-460 and 569-576,
update the state.owned early-return condition to skip only when
trace.SpanFromContext(endCtx) matches state.span; otherwise call state.End().
Apply the same matching check at both sites so child contexts with distinct
active spans are finalized correctly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: f19e18d2-4935-44bb-9230-926dd5933564

📥 Commits

Reviewing files that changed from the base of the PR and between d3b33f8 and 325d218.

📒 Files selected for processing (3)
  • middleware/helpers_test.go
  • middleware/telemetry.go
  • middleware/telemetry_span_lifecycle_test.go

@gauchito91

Copy link
Copy Markdown
Contributor Author

⚠️ Nota sobre o check security / security_scan (vermelho = falso positivo)

O check de segurança está falhando, mas não há vulnerabilidade neste PR. Investiguei o job e o scan de segurança em si passa — o que falha é o step de postar o comentário de resultado no PR, um bug no workflow compartilhado.

Evidência (job 88472871834):

Step Resultado
Trivy Filesystem Scan ✅ success — Report Summary: go.mod / gomod / Secrets: - (0 findings)
Pre-release Version Check ✅ success
Post Security Scan Results to PR failure

O step de report falha com:

::error::pr-security-reporter: github-script produced no output.
The script step likely failed (API error, rate limit, or runtime exception).

Sem output do github-script, o gate falha por precaução (Failing to prevent bypass) e emite a mensagem enganosa "Security vulnerabilities found" — mas nenhuma vulnerabilidade foi encontrada (Trivy limpo, gosec pass, Go Analysis/Security pass).

Causa: bug no reporter do workflow compartilhado LerianStudio/github-actions-shared-workflows (step "Post Security Scan Results to PR"), não neste código. É consistente (falhou em 2 runs seguidos), então re-run não resolve. O diff deste PR não toca go.mod/go.sum.

Ação: reportar o bug ao time dono do workflow. O merge não deve ser bloqueado por este check falso-positivo.

…return

Add the required blank line before the final `return resp, err` in
EndTracingSpansInterceptor's owned-span branch (wsl_v5 "too many lines
above return"). No behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
X-Lerian-Ref: 0x1
@gauchito91
gauchito91 merged commit f1ec77b into develop Jul 21, 2026
20 of 21 checks passed
@github-actions
github-actions Bot deleted the fix/span-name-cardinality-pii branch July 21, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation, llms.txt and markdown content middleware HTTP/gRPC observability middleware size/L PR changes 500–999 lines tests Unit, integration and end-to-end tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants