Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ See [Choosing A CPA Panel](https://seakee.github.io/CPA-Manager-Plus/docs/en/gui

- Break down calls, tokens, cost, latency, and failures by model, provider, account, credential, API key, project, channel, and time range.
- Track input, output, reasoning, cache, service tier, and long-context pricing semantics.
- Sync model prices from LiteLLM and OpenRouter, with local overrides for aliases or internal models.
- Sync model prices from models.dev first, with LiteLLM and OpenRouter fallbacks plus local overrides for aliases or internal models.
- Open the [Usage Analytics Demo](https://seakee.github.io/CPA-Manager-Plus/#/demo/usage-analytics).

### Account Health, Quota, And Automation
Expand Down
34 changes: 32 additions & 2 deletions apps/docs/en/manual/model-prices.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,24 @@ Open the [Model Prices Demo](https://seakee.github.io/CPA-Manager-Plus/#/demo/mo

## Price Sources

- Public metadata actively synchronized from LiteLLM or OpenRouter.
- Public metadata synchronized from models.dev first, with LiteLLM and OpenRouter used as fallbacks when the preferred source is unavailable or lacks a model.
- Local prices added or overridden by the user.
- Entries for aliases, internal names, or provider-specific variants.

Synchronization only occurs when the user triggers it and may use the current Manager Server proxy configuration.

The same models.dev model ID may be offered by multiple providers, and a real model ID may itself contain `/`. CPAMP compares source identities separately from original model IDs and only matches automatically when the complete pricing metadata, including tiers and experimental-mode prices, is consistent. Identity or price conflicts remain in the candidate-confirmation flow; a same-named LiteLLM or OpenRouter entry cannot bypass that conflict.

The current sync maps models.dev `cost.input`, `cost.output`, `cost.cache_read`, and `cost.cache_write`, converts valid `cost.tiers` context tiers into CPAMP billing rules, and maps `experimental.modes.fast.cost` to short-context Fast/Priority prices. The complete model object remains available in raw metadata; reasoning prices, unknown experimental modes, unknown tier types, and rules that cannot be validated safely do not activate automatic billing.

### Sync failures and last-known-good prices

- When models.dev is temporarily unavailable, CPAMP continues with LiteLLM and OpenRouter.
- A transient models.dev failure cannot automatically replace a stored models.dev price with a lower-priority source; fallback sources may still fill models that have no local price.
- A fallback source may replace a model normally when models.dev responds successfully but does not contain that model.
- If every source fails, synchronization stops before any database write and existing prices remain unchanged.
- A synchronized price remains the last-known-good value until a later successful sync or a manual edit; `syncedAtMs` indicates its freshness.

## Supported Billing Semantics

A price rule may include:
Expand All @@ -26,11 +38,29 @@ A price rule may include:
- Cache read, cache write, and cache creation.
- Fixed per-request cost.
- `service_tier` differences.
- models.dev context-price tiers.
- Long-context thresholds and multipliers.
- Model alias and billing-model mapping.

Models such as GPT-5.6 may vary by context length, service tier, and cache type. CPAMP can only apply a rule when both the request event and price entry contain the required fields.

### Context-tier semantics

- A tier matches only when normalized input tokens are **strictly greater than** `tier.size`; an exact-threshold request stays in the lower band.
- When multiple tiers match, CPAMP selects the highest matching threshold.
- The selected tier's rates apply to the entire request, not only to tokens above the threshold.
- Input, output, or cache rates omitted by a tier inherit the base price; an explicit zero from models.dev remains zero.
- CPAMP currently activates only safely validated tiers with `tier.type = context` and a positive threshold. Other rules remain in raw metadata for inspection.

### Fast/Priority semantics

- `experimental.modes.fast.cost` matches both `fast` usage telemetry and API `priority` telemetry.
- Short-context requests prefer explicit Fast/Priority prices. Missing fields inherit base rates, while explicit zeros remain zero.
- A matched context tier or the legacy GPT long-context rule uses its standard context price without stacking Fast/Priority pricing.
- Non-models.dev entries, older data, and models without an explicit mode price retain the existing multiplier as a compatibility fallback.

Model Prices displays synchronized context tiers and service-tier prices as read-only rules. The current manual editor manages base prices only; saving a manual price explicitly clears existing synchronized advanced rules, with a warning shown before saving.

## Matching Model Names

The client model, CPA alias, provider model, and price-table name may differ. When cost is missing:
Expand All @@ -48,5 +78,5 @@ The page uses a compact model-usage summary to show which prices are active. It

- Provider billing remains authoritative.
- Missing token, service tier, long-context, or cache fields reduce estimate accuracy.
- Subscriptions, grants, tiered prices, and multiple currencies may not fit a single price entry.
- Subscriptions, grants, non-context tiers, unsupported dynamic-mode prices, and multiple currencies may not fit a single price entry.
- Historical cost may be displayed using current prices after an update; the price table is not an immutable billing snapshot.
34 changes: 32 additions & 2 deletions apps/docs/manual/model-prices.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,24 @@ description: 配置 CPA Manager Plus 模型价格、service tier、长上下文

## 价格来源

- LiteLLM OpenRouter 主动同步的公开元数据
- 首选从 models.dev 主动同步的公开元数据;当该来源不可用或缺少模型时,再使用 LiteLLM OpenRouter 回退
- 用户手动添加或覆盖的本地价格。
- 为模型别名、内部名称或 Provider 特定变体维护的条目。

同步只在用户主动触发时发生,可能使用当前 Manager Server 代理设置。

models.dev 中同一个模型 ID 可能由多个 Provider 提供,而且真实模型 ID 本身也可能包含 `/`。CPAMP 会分别比较来源身份和原始模型 ID;只有完整价格元数据(包括阶梯和实验模式价格)明确一致时才自动匹配。任何身份或价格冲突都会进入候选确认流程,LiteLLM 或 OpenRouter 的同名条目不会绕过该冲突。

当前同步会映射 models.dev 的 `cost.input`、`cost.output`、`cost.cache_read` 和 `cost.cache_write`,将有效的 `cost.tiers` 上下文阶梯转换为 CPAMP 计费规则,并将 `experimental.modes.fast.cost` 映射为 Fast/Priority 短上下文价格。完整模型对象仍保存在原始元数据中;reasoning、未知实验模式、未知阶梯类型或无法安全验证的规则不会激活自动计费。

### 同步失败与最后有效价格

- models.dev 暂时不可用时,CPAMP 会继续尝试 LiteLLM 和 OpenRouter。
- 已保存的 models.dev 价格不会因为本次网络失败而被低优先级来源自动覆盖;回退来源仍可补充本地没有价格的模型。
- 只有 models.dev 成功响应但明确不包含某个模型时,才允许回退来源正常替换该模型。
- 如果所有来源都失败,同步在写入数据库前终止,现有价格保持不变。
- 同步价格会一直作为最后有效数据使用,直到后续成功同步或用户手动修改;`syncedAtMs` 可用于判断数据新鲜度。

## 当前支持的计费语义

价格结构可能包括:
Expand All @@ -26,11 +38,29 @@ description: 配置 CPA Manager Plus 模型价格、service tier、长上下文
- Cache read、cache write 和 cache creation。
- 请求级固定费用。
- `service_tier` 差异。
- models.dev 上下文价格阶梯。
- 长上下文阈值和倍率。
- 模型别名与 billing model 映射。

例如 GPT-5.6 及类似模型可能根据上下文长度、service tier 和缓存类型采用不同价格。只有请求事件带有对应字段且价格规则存在时,CPAMP 才能正确计算。

### 上下文阶梯语义

- 阶梯条件是标准化输入 Token **严格大于** `tier.size`;恰好等于阈值时仍使用较低一档。
- 如果多个阶梯都满足条件,选择阈值最高的一档。
- 选中阶梯的费率应用于整个请求,而不是只计算超过阈值的 Token。
- 阶梯中缺失的输入、输出或缓存费率继承基础价格;models.dev 明确提供的零价格会保留为零。
- 目前只自动启用 `tier.type = context`、正数阈值且价格结构可安全验证的阶梯。其他规则仍保留在原始元数据中供排查。

### Fast/Priority 语义

- `experimental.modes.fast.cost` 同时匹配使用数据中的 `fast` 和 API `priority`。
- 短上下文优先使用显式 Fast/Priority 价格;缺失字段继承基础价格,显式零值保持为零。
- 命中上下文阶梯或旧版 GPT 长上下文规则时,使用对应的标准上下文价格,不再叠加 Fast/Priority。
- 非 models.dev、旧数据或没有显式模式价格的模型继续使用现有倍率作为兼容回退。

模型价格页只读展示已同步的上下文阶梯和服务层级价格。当前手动编辑器只维护基础价格;保存手动价格会明确清除该模型已有的同步高级规则,界面会在保存前提示。

## 模型名称匹配

客户端请求名、CPA 路由别名、Provider 实际模型名和价格表名称可能不同。排查成本为空时:
Expand All @@ -48,5 +78,5 @@ description: 配置 CPA Manager Plus 模型价格、service tier、长上下文

- Provider 账单是最终依据。
- 缺失 Token、service tier、长上下文或缓存字段会降低估算精度。
- 包月、赠送额度、阶梯价和多币种不一定能由单一价格条目完整表达
- 包月、赠送额度、非上下文阶梯、未支持的动态模式价格和多币种不一定能由单一价格条目完整表达
- 更新价格后,历史成本可能按当前价格重新展示;价格表不是不可变账单快照。
6 changes: 6 additions & 0 deletions apps/manager-server/cmd/cpa-manager-plus/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,13 +103,17 @@ func runServer() {
accountActionWorker := worker.NewAccountActionCandidateWorker(db, runtimeSettings.AccountActionsAutoDisable)
accountHistoryRollupWorker := worker.NewAccountHistoryRollupWorker(db)
accountHistoryRollupWorker.Start(ctx)
usagePricingRollupWorker := worker.NewUsagePricingRollupWorker(db)
usagePricingRollupWorker.Start(ctx)
serverApp.AppContext().ModelPriceService.SetPricesChangedNotifier(usagePricingRollupWorker.Wake)
var usageHourlyAggregateWorker *worker.UsageHourlyAggregateWorker
if cfg.DashboardHourlyRollupEnabled {
usageHourlyAggregateWorker = worker.NewUsageHourlyAggregateWorker(db)
usageHourlyAggregateWorker.Start(ctx)
}
serverApp.AppContext().UsageService.SetEventsInsertedNotifier(func() {
accountHistoryRollupWorker.Wake()
usagePricingRollupWorker.Wake()
if usageHourlyAggregateWorker != nil {
usageHourlyAggregateWorker.Wake()
}
Expand All @@ -125,6 +129,7 @@ func runServer() {
manager.SetUsageEventHandler(worker.NewUsageEventFanout(
automationRuntime.UsageEventHandler(),
accountHistoryRollupWorker,
usagePricingRollupWorker,
usageHourlyAggregateWorker,
))

Expand Down Expand Up @@ -167,6 +172,7 @@ func runServer() {
usageCacheAccountingMigrationWorker := worker.NewUsageCacheAccountingMigrationWorker(db, func() {
go runUsageResponseMetadataBackfill(ctx, db)
accountHistoryRollupWorker.Wake()
usagePricingRollupWorker.Wake()
if usageHourlyAggregateWorker != nil {
usageHourlyAggregateWorker.Wake()
}
Expand Down
4 changes: 3 additions & 1 deletion apps/manager-server/internal/app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (

type Options struct {
EmbeddedPanel fs.FS
ModelsDevModelPriceSyncURL *string
ModelPriceSyncURL *string
OpenRouterModelPriceSyncURL *string
ServiceID string
Expand Down Expand Up @@ -51,12 +52,13 @@ func New(ctx context.Context, cfg config.Config, options Options) (*Context, err
if startedAt <= 0 {
startedAt = time.Now().UnixMilli()
}
appCtx := FromExisting(
appCtx := FromExistingWithModelsDev(
cfg,
st,
manager,
startedAt,
options.EmbeddedPanel,
options.ModelsDevModelPriceSyncURL,
options.ModelPriceSyncURL,
options.OpenRouterModelPriceSyncURL,
serviceID,
Expand Down
54 changes: 53 additions & 1 deletion apps/manager-server/internal/app/context.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,58 @@ func FromExisting(
openRouterModelPriceSyncURL *string,
serviceID string,
automationRuntimeService ...AutomationRuntimeService,
) *Context {
return fromExisting(
cfg,
st,
collectorManager,
startedAt,
embeddedPanel,
nil,
modelPriceSyncURL,
openRouterModelPriceSyncURL,
serviceID,
automationRuntimeService...,
)
}

func FromExistingWithModelsDev(
cfg config.Config,
st *store.Store,
collectorManager *collector.Manager,
startedAt int64,
embeddedPanel fs.FS,
modelsDevModelPriceSyncURL *string,
modelPriceSyncURL *string,
openRouterModelPriceSyncURL *string,
serviceID string,
automationRuntimeService ...AutomationRuntimeService,
) *Context {
return fromExisting(
cfg,
st,
collectorManager,
startedAt,
embeddedPanel,
modelsDevModelPriceSyncURL,
modelPriceSyncURL,
openRouterModelPriceSyncURL,
serviceID,
automationRuntimeService...,
)
}

func fromExisting(
cfg config.Config,
st *store.Store,
collectorManager *collector.Manager,
startedAt int64,
embeddedPanel fs.FS,
modelsDevModelPriceSyncURL *string,
modelPriceSyncURL *string,
openRouterModelPriceSyncURL *string,
serviceID string,
automationRuntimeService ...AutomationRuntimeService,
) *Context {
var runtimeService AutomationRuntimeService
if len(automationRuntimeService) > 0 {
Expand Down Expand Up @@ -99,7 +151,7 @@ func FromExisting(
DashboardService: dashboardsvc.New(st, cfg.DashboardHourlyRollupEnabled),
CodexInspectionService: codexinspectionsvc.New(st, managerConfigService),
MonitoringService: monitoringsvc.New(st, cfg.DashboardHourlyRollupEnabled),
ModelPriceService: modelpricesvc.NewMultiSource(st, modelPriceSyncURL, openRouterModelPriceSyncURL, managerConfigService),
ModelPriceService: modelpricesvc.NewMultiSourceWithModelsDev(st, modelsDevModelPriceSyncURL, modelPriceSyncURL, openRouterModelPriceSyncURL, managerConfigService),
APIKeyAliasService: apikeyaliassvc.New(st),
AccountActionService: accountactionsvc.New(st, managerConfigService),
AccountProcessingPolicyService: accountProcessingPolicyService,
Expand Down
4 changes: 3 additions & 1 deletion apps/manager-server/internal/httpapi/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ var embeddedPanel embed.FS

const serviceID = "cpa-manager-plus"

var modelsDevModelPriceSyncURL = "https://models.dev/api.json"
var modelPriceSyncURL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
var openRouterModelPriceSyncURL = "https://openrouter.ai/api/v1/models"

Expand All @@ -27,12 +28,13 @@ type Server struct {

func New(cfg config.Config, store *store.Store, collector *collector.Manager, automationRuntimeService ...app.AutomationRuntimeService) *Server {
startedAt := time.Now().UnixMilli()
appCtx := app.FromExisting(
appCtx := app.FromExistingWithModelsDev(
cfg,
store,
collector,
startedAt,
embeddedPanel,
&modelsDevModelPriceSyncURL,
&modelPriceSyncURL,
&openRouterModelPriceSyncURL,
serviceID,
Expand Down
Loading
Loading