Skip to content

fix(ai-gateway): avoid understating OpenRouter data collection - #5057

Open
chrarnoldus wants to merge 3 commits into
mainfrom
lying-scourge
Open

fix(ai-gateway): avoid understating OpenRouter data collection#5057
chrarnoldus wants to merge 3 commits into
mainfrom
lying-scourge

Conversation

@chrarnoldus

@chrarnoldus chrarnoldus commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Problem

OpenRouter can offer the same model through multiple routes from one provider. For example, a provider may have both a standard route that retains prompts and a zero-data-retention (ZDR) route that does not.

The model-list API returns only one of those routes for each model. When it returns the ZDR route, our provider sync stores retainsPrompts: false even though requests can also use the provider's data-retaining route. Which policy appears in our UI can therefore change based on the route selected by OpenRouter.

Fix

When building the provider snapshot, combine the selected model route's policy with the provider-wide policy:

  • report training when either policy allows training
  • report prompt retention when either policy allows retention
  • preserve stricter policies added for free or Kilo-exclusive models

This does not group providers by name or slug prefix. Each model is combined only with the provider whose OpenRouter-filtered model response contained it. The transformation returns a new model instead of modifying OpenRouter's parsed response.

Live example

OpenRouter currently exposes x-ai/grok-4.5 through both xai and xai/zdr routes. The model-list response selected SpaceXAI (ZDR) with retainsPrompts: false, while the xAI provider-wide policy reports retainsPrompts: true. After this change, Kilo reports that the model may retain prompts.

Verification

  • focused model data-policy tests: 4 passed
  • pnpm --filter web typecheck
  • pnpm --filter web lint
  • targeted formatting check
  • git diff --check

@chrarnoldus chrarnoldus self-assigned this Aug 5, 2026
@kilo-code-bot

kilo-code-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incrementally reviewed commit 6ad0b7f5 ("clarify provider policy behavior"): a documentation-only change on top of the previously approved fe9a7dea — the new JSDoc on withWorstProviderDataPolicy accurately describes the OR-merge implementation, the updated comment in sync-providers.ts correctly states that a model may surface a ZDR route while the provider also offers data-retaining routes, and both renamed tests still precisely match their asserted behavior (ZDR route merged with retention-allowing provider policy, and model-reported collection preserved). No logic, control-flow, or API changes; no memory-leak surface; no markdown/landing-page content in scope.

Files Reviewed (3 files)
  • apps/web/src/lib/ai-gateway/providers/openrouter/model-data-policy.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/model-data-policy.test.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/sync-providers.ts
Previous Review Summaries (2 snapshots, latest commit fe9a7de)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit fe9a7de)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incrementally reviewed commit fe9a7dea ("make policy merge functional"): the refactor from mutating applyWorstProviderDataPolicy to pure withWorstProviderDataPolicy is behavior-preserving — the merge still runs after applyFreeEndpointDataPolicy (free-endpoint true/true policies preserved), every post-merge consumer reads only the deduplicated normalized copies (no code relied on the old in-place mutation), the dedup winner is unchanged, missing-provider extra models were never merged in either version, and data_policy full replacement drops nothing (schema has only training/retainsPrompts). Tests correctly assert both worst-case directions plus the new non-mutation guarantee. No memory-leak surface (no retained references, timers, or subscriptions) and no markdown changes in scope.

Files Reviewed (3 files)
  • apps/web/src/lib/ai-gateway/providers/openrouter/model-data-policy.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/model-data-policy.test.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/sync-providers.ts

Previous review (commit 600d33f)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the worst-of provider data-policy merge for OpenRouter route variants: the OR-merge in applyWorstProviderDataPolicy is semantically correct (operator precedence verified), ordering after applyFreeEndpointDataPolicy preserves free-endpoint true/true policies, injected and kilo-exclusive models are covered, the only downstream readers (modelTrains/modelRetainsPrompts and the org providers UI) behave identically via the explicit values they previously got from fallback, and the new tests cover both worst-case directions. No memory-leak surface (no retained references, timers, or subscriptions) and no markdown/image changes in scope.

Files Reviewed (3 files)
  • apps/web/src/lib/ai-gateway/providers/openrouter/model-data-policy.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/model-data-policy.test.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/sync-providers.ts

Reviewed by kimi-k3 · Input: 33.8K · Output: 4.7K · Cached: 176.1K

Review guidance: REVIEW.md from base branch main

@chrarnoldus chrarnoldus changed the title fix(ai-gateway): use worst provider data policy fix(ai-gateway): show worst provider data policy in case of conflict Aug 5, 2026
@chrarnoldus chrarnoldus changed the title fix(ai-gateway): show worst provider data policy in case of conflict fix(ai-gateway): avoid understating OpenRouter data collection Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants