Skip to content

fix(web): filter AI usage-limit 402 errors out of Sentry - #5058

Open
eshurakov wants to merge 1 commit into
mainfrom
fix/sentry-filter-ai-usage-limit-402
Open

fix(web): filter AI usage-limit 402 errors out of Sentry#5058
eshurakov wants to merge 1 commit into
mainfrom
fix/sentry-filter-ai-usage-limit-402

Conversation

@eshurakov

Copy link
Copy Markdown
Contributor

Summary

Filters AI_APICallError with statusCode: 402 out of Sentry in beforeSend, alongside the existing tRPC 4xx filter.

Why

The AI gateway returns 402 (usage_limit_exceeded — "Add credits to continue, or switch to a free model", llm-proxy-helpers.ts) when the caller's own credit balance is exhausted. Server-side callers acting on behalf of a user — e.g. Kilo Bot processing /api/internal/bot-session-callback — surface this as an unhandled AI_APICallError (statusCode 402), which captureException then reports. Each event just means "a user ran out of credits mid-workflow": expected per-user billing state, not an application bug, and it scales with free/low-credit usage.

This is safe to filter globally because upstream provider 402s (our provider account) never reach AI SDK callers: the gateway converts them to a 503 and reports them itself via captureProxyError({ trackInSentry: true }) in app/api/openrouter/[...path]/route.ts, so that failure mode stays visible.

Changes

  • apps/web/sentry.server.config.ts: add exported isAIUsageLimitError predicate (duck-typed like the existing isTRPC4xxError, so the ai package stays out of the Sentry init bundle) and apply it in beforeSend.
  • apps/web/src/sentry.server.config.test.ts: tests for the predicate, including one built from the real APICallError class to pin the duck-type contract.

Verification

  • jest src/sentry.server.config.test.ts: 24/24 pass (run with a scratch config identical to jest.config.ts minus the DB-backed global setup — this sandbox has no Docker/Postgres; the standard config's setup requires it. The suite itself has no DB dependency, so CI should behave the same).
  • oxlint on the changed test file: clean.
  • oxfmt applied to both files.
  • Scoped tsgo typecheck of both changed files: clean. Full apps/web typecheck not run (its @kilocode/trpc build prerequisite exceeded the sandbox time limit).

The AI gateway returns 402 (usage_limit_exceeded) when the caller's own
credit balance is exhausted, and the AI SDK surfaces that as
AI_APICallError with statusCode 402 - e.g. Kilo Bot calling the gateway
on behalf of a user who is out of credits in bot-session-callback
processing. That is expected per-user billing state, not an application
bug, so reporting it to Sentry is noise.

Drop these events in beforeSend alongside the existing tRPC 4xx filter.
Upstream provider 402s (our provider account) are unaffected: the
gateway converts them to a 503 and reports them itself via
captureProxyError.

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
@kilo-code-bot

kilo-code-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the Sentry beforeSend filter for AI usage-limit 402 errors and its tests; the duck-typed predicate is type-safe, the claim that upstream provider 402s are converted to 503 and reported separately was verified against the gateway route, and no platform-side 402 path is hidden by the filter.

Files Reviewed (2 files)
  • apps/web/sentry.server.config.ts
  • apps/web/src/sentry.server.config.test.ts

Reviewed by kimi-k3 · Input: 62.1K · Output: 13.5K · Cached: 633.9K

Review guidance: REVIEW.md from base branch main

@chrarnoldus

Copy link
Copy Markdown
Contributor

Would it be better to fix the source?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants