llms-full.txt

# cliproxyapi++ LLM Context (Full)
Expanded, line-addressable repository context.

# cliproxyapi++ LLM Context (Concise)
Generated from repository files for agent/dev/user consumption.

## README Highlights
# cliproxyapi++ 🚀
[![Go Report Card](https://goreportcard.com/badge/github.com/KooshaPari/cliproxyapi-plusplus)](https://goreportcard.com/report/github.com/KooshaPari/cliproxyapi-plusplus)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker Pulls](https://img.shields.io/docker/pulls/kooshapari/cliproxyapi-plusplus.svg)](https://hub.docker.com/r/kooshapari/cliproxyapi-plusplus)
[![GitHub Release](https://img.shields.io/github/v/release/KooshaPari/cliproxyapi-plusplus)](https://github.com/KooshaPari/cliproxyapi-plusplus/releases)
English | [中文](README_CN.md)
**cliproxyapi++** is the definitive high-performance, security-hardened fork of [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI). Designed with a "Defense in Depth" philosophy and a "Library-First" architecture, it provides an OpenAI-compatible interface for proprietary LLMs with enterprise-grade stability.
---
## 🏆 Deep Dive: The `++` Advantage
Why choose **cliproxyapi++** over the mainline? While the mainline focus is on open-source stability, the `++` variant is built for high-scale, production environments where security, automated lifecycle management, and broad provider support are critical.
Full feature-by-feature change reference:
- **[Feature Changes in ++](./docs/FEATURE_CHANGES_PLUSPLUS.md)**
### 📊 Feature Comparison Matrix
| Feature | Mainline | CLIProxyAPI+ | **cliproxyapi++** |
| :--- | :---: | :---: | :---: |
| **Core Proxy Logic** | ✅ | ✅ | ✅ |
| **Basic Provider Support** | ✅ | ✅ | ✅ |
| **Standard UI** | ❌ | ✅ | ✅ |
| **Advanced Auth (Kiro/Copilot)** | ❌ | ⚠️ | ✅ **(Full Support)** |
| **Background Token Refresh** | ❌ | ❌ | ✅ **(Auto-Refresh)** |
| **Security Hardening** | Basic | Basic | ✅ **(Enterprise-Grade)** |
| **Rate Limiting & Cooldown** | ❌ | ❌ | ✅ **(Intelligent)** |
| **Core Reusability** | `internal/` | `internal/` | ✅ **(`pkg/llmproxy`)** |
| **CI/CD Pipeline** | Basic | Basic | ✅ **(Signed/Multi-arch)** |
---
## 🔍 Technical Differences & Hardening
### 1. Architectural Evolution: `pkg/llmproxy`
Unlike the mainline which keeps its core logic in `internal/` (preventing external Go projects from importing it), **cliproxyapi++** has refactored its entire translation and proxying engine into a clean, public `pkg/llmproxy` library.
*   **Reusability**: Import the proxy logic directly into your own Go applications.
*   **Decoupling**: Configuration management is strictly separated from execution logic.
### 2. Enterprise Authentication & Lifecycle
*   **Full GitHub Copilot Integration**: Not just an API wrapper. `++` includes a full OAuth device flow, per-credential quota tracking, and intelligent session management.
*   **Kiro (AWS CodeWhisperer) 2.0**: A custom-built web UI (`/v0/oauth/kiro`) for browser-based AWS Builder ID and Identity Center logins.
*   **Background Token Refresh**: A dedicated worker service monitors tokens and automatically refreshes them 10 minutes before expiration, ensuring zero downtime for your agents.
### 3. Security Hardening ("Defense in Depth")
*   **Path Guard**: A custom GitHub Action workflow (`pr-path-guard`) that prevents any unauthorized changes to critical `internal/translator/` logic during PRs.
*   **Device Fingerprinting**: Generates unique, immutable device identifiers to satisfy strict provider security checks and prevent account flagging.
*   **Hardened Docker Base**: Built on a specific, audited Alpine 3.22.0 layer with minimal packages, reducing the potential attack surface.
### 4. High-Scale Operations
*   **Intelligent Cooldown**: Automated "cooling" mechanism that detects provider-side rate limits and intelligently pauses requests to specific providers while routing others.
*   **Unified Model Converter**: A sophisticated mapping layer that allows you to request `claude-3-5-sonnet` and have the proxy automatically handle the specific protocol requirements of the target provider (Vertex, AWS, Anthropic, etc.).
---
## 🚀 Getting Started
### Prerequisites
- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/)
- OR [Go 1.26+](https://golang.org/dl/)
### One-Command Deployment (Docker)
```bash
# Setup deployment
mkdir -p ~/cliproxy && cd ~/cliproxy
curl -o config.yaml https://raw.githubusercontent.com/KooshaPari/cliproxyapi-plusplus/main/config.example.yaml
# Create compose file
cat > docker-compose.yml << 'EOF'
services:
cliproxy:
image: KooshaPari/cliproxyapi-plusplus:latest
container_name: cliproxyapi++
ports: ["8317:8317"]
volumes:
- ./config.yaml:/CLIProxyAPI/config.yaml
- ./auths:/root/.cli-proxy-api
- ./logs:/CLIProxyAPI/logs
restart: unless-stopped
EOF
docker compose up -d
```
---
## 🛠️ Advanced Usage
### Extended Provider Support
`cliproxyapi++` supports a massive registry of providers out-of-the-box:
*   **Direct**: Claude, Gemini, OpenAI, Mistral, Groq, DeepSeek.
*   **Aggregators**: OpenRouter, Together AI, Fireworks AI, Novita AI, SiliconFlow.
*   **Proprietary**: Kiro (AWS), GitHub Copilot, Roo Code, Kilo AI, MiniMax.
### API Specification
The proxy provides two main API surfaces:
1.  **OpenAI Interface**: `/v1/chat/completions` and `/v1/models` (Full parity).
2.  **Management Interface**:
*   `GET /v0/config`: Inspect current (hot-reloaded) config.
*   `GET /v0/oauth/kiro`: Interactive Kiro auth UI.
*   `GET /v0/logs`: Real-time log inspection.
---
## 🤝 Contributing
We maintain strict quality gates to preserve the "hardened" status of the project:
1.  **Linting**: Must pass `golangci-lint` with zero warnings.
2.  **Coverage**: All new translator logic MUST include unit tests.
3.  **Governance**: Changes to core `pkg/` logic require a corresponding Issue discussion.
See **[CONTRIBUTING.md](CONTRIBUTING.md)** for more details.
---
## 📚 Documentation
- **[Docsets](./docs/docsets/)** — Role-oriented documentation sets.
- [Developer (Internal)](./docs/docsets/developer/internal/)
- [Developer (External)](./docs/docsets/developer/external/)
- [Technical User](./docs/docsets/user/)
- [Agent Operator](./docs/docsets/agent/)
- **[Feature Changes in ++](./docs/FEATURE_CHANGES_PLUSPLUS.md)** — Comprehensive list of `++` differences and impacts.
- **[Docs README](./docs/README.md)** — Core docs map.
---
## 🚢 Docs Deploy
Local VitePress docs:
```bash
cd docs
npm install
npm run docs:dev
npm run docs:build
```
GitHub Pages:
- Workflow: `.github/workflows/vitepress-pages.yml`
- URL convention: `https://<owner>.github.io/cliproxyapi-plusplus/`
---
## 📜 License
Distributed under the MIT License. See [LICENSE](LICENSE) for more information.
---
<p align="center">
<b>Hardened AI Infrastructure for the Modern Agentic Stack.</b><br>
Built with ❤️ by the community.
</p>

## Taskfile Tasks
- GO_FILES
- default
- build
- run
- test
- lint
- tidy
- docker:build
- docker:run
- docker:stop
- doctor
- ax:spec

## Documentation Index
- docs/FEATURE_CHANGES_PLUSPLUS.md
- docs/README.md
- docs/docsets/agent/index.md
- docs/docsets/agent/operating-model.md
- docs/docsets/developer/external/index.md
- docs/docsets/developer/external/integration-quickstart.md
- docs/docsets/developer/internal/architecture.md
- docs/docsets/developer/internal/index.md
- docs/docsets/index.md
- docs/docsets/user/index.md
- docs/docsets/user/quickstart.md
- docs/features/architecture/DEV.md
- docs/features/architecture/SPEC.md
- docs/features/architecture/USER.md
- docs/features/auth/SPEC.md
- docs/features/auth/USER.md
- docs/features/operations/SPEC.md
- docs/features/operations/USER.md
- docs/features/providers/SPEC.md
- docs/features/providers/USER.md
- docs/features/security/SPEC.md
- docs/features/security/USER.md
- docs/index.md
- docs/sdk-access.md
- docs/sdk-access_CN.md
- docs/sdk-advanced.md
- docs/sdk-advanced_CN.md
- docs/sdk-usage.md
- docs/sdk-usage_CN.md
- docs/sdk-watcher.md
- docs/sdk-watcher_CN.md

## Markdown Headings
### docs/FEATURE_CHANGES_PLUSPLUS.md
- # cliproxyapi++ Feature Change Reference (`++` vs baseline)
- ## 1. Architecture Changes
- ## 2. Authentication and Identity Changes
- ## 3. Provider and Model Routing Changes
- ## 4. Security and Governance Changes
- ## 5. Operations and Delivery Changes
- ## 6. API and Compatibility Surface
- ## 7. Migration Impact Summary
### docs/README.md
- # cliproxyapi++ Documentation Index
- ## 📚 Documentation Structure
- ## 🚀 Quick Start
- ## 📖 Feature Documentation
- ### 1. Library-First Architecture
- ### 2. Enterprise Authentication
- ### 3. Security Hardening
- ### 4. High-Scale Operations
- ### 5. Provider Registry
- ## 🔧 API Documentation
- ### OpenAI-Compatible API
- ### Management API
- ### Operations API
- ## 🛠️ SDK Documentation
- ### Go SDK
- ## 🚀 Getting Started
- ### 1. Installation
- ### 2. Configuration
- ### 3. Add Credentials
- ### 4. Start Service
- ### 5. Make Request
- ## 🔍 Troubleshooting
- ### Common Issues
- ### Debug Mode
- ### Get Help
- ## 📊 Comparison: cliproxyapi++ vs Mainline
- ## 📝 Contributing
- ## 🔐 Security
- ## 📜 License
- ## 🗺️ Documentation Map
- ## 🤝 Community
### docs/docsets/agent/index.md
- # Agent Operator Docset
- ## Operator Focus
### docs/docsets/agent/operating-model.md
- # Agent Operating Model
- ## Execution Loop
### docs/docsets/developer/external/index.md
- # External Developer Docset
- ## Start Here
### docs/docsets/developer/external/integration-quickstart.md
- # Integration Quickstart
### docs/docsets/developer/internal/architecture.md
- # Internal Architecture
- ## Core Boundaries
- ## Maintainer Rules
### docs/docsets/developer/internal/index.md
- # Internal Developer Docset
- ## Read First
### docs/docsets/index.md
- # Docsets
- ## Developer
- ## User
- ## Agent
### docs/docsets/user/index.md
- # Technical User Docset
- ## Core Paths
### docs/docsets/user/quickstart.md
- # Technical User Quickstart
### docs/features/architecture/DEV.md
- # Developer Guide: Extending Library-First Architecture
- ## Contributing to pkg/llmproxy
- ## Project Structure
- ## Adding a New Provider
- ### Step 1: Define Provider Configuration
- ### Step 2: Implement Translator Interface
- ### Step 3: Implement Provider Executor
- ### Step 4: Register Provider
- ### Step 5: Add Tests
- ## Custom Authentication Flows
- ### Implementing OAuth
- ### Implementing Device Flow
- ## Performance Optimization
- ### Connection Pooling
- ### Rate Limiting Optimization
- ### Caching Strategy
- ## Testing Guidelines
- ### Unit Tests
- ### Integration Tests
- ### Contract Tests
- ## Submitting Changes
- ## API Stability
### docs/features/architecture/SPEC.md
- # Technical Specification: Library-First Architecture (pkg/llmproxy)
- ## Overview
- ## Architecture Migration
- ### Before: Mainline Structure
- ### After: cliproxyapi++ Structure
- ## Core Components
- ### 1. Translation Engine (`pkg/llmproxy/translator`)
- ### 2. Provider Execution (`pkg/llmproxy/provider`)
- ### 3. Configuration Management (`pkg/llmproxy/config`)
- ### 4. Watcher & Synthesis (`pkg/llmproxy/watcher`)
- ## Data Flow
- ### Request Processing Flow
- ### Configuration Reload Flow
- ### Token Refresh Flow
- ## Reusability Patterns
- ### Embedding as Library
- ### Custom Provider Integration
- ### Extending Configuration
- ## Performance Characteristics
- ### Memory Footprint
- ### Concurrency Model
- ### Throughput
- ## Security Considerations
- ### Public API Stability
- ### Input Validation
- ### Error Propagation
- ## Migration Guide
- ### From Mainline internal/
- ### Function Compatibility
- ## Testing Strategy
- ### Unit Tests
- ### Integration Tests
- ### Contract Tests
### docs/features/architecture/USER.md
- # User Guide: Library-First Architecture
- ## What is "Library-First"?
- ## Why Use the Library?
- ### Benefits Over Standalone CLI

## Detailed File Snapshots

### FILE: .goreleaser.yml
0001: builds:
0002:   - id: "cliproxyapi-plusplus"
0003:     env:
0004:       - CGO_ENABLED=0
0005:     goos:
0006:       - linux
0007:       - windows
0008:       - darwin
0009:     goarch:
0010:       - amd64
0011:       - arm64
0012:     main: ./cmd/server/
0013:     binary: cliproxyapi++
0014:     ldflags:
0015:       - -s -w -X 'main.Version={{.Version}}-++' -X 'main.Commit={{.ShortCommit}}' -X 'main.BuildDate={{.Date}}'
0016: archives:
0017:   - id: "cliproxyapi-plusplus"
0018:     format: tar.gz
0019:     format_overrides:
0020:       - goos: windows
0021:         format: zip
0022:     files:
0023:       - LICENSE
0024:       - README.md
0025:       - README_CN.md
0026:       - config.example.yaml
0027:
0028: checksum:
0029:   name_template: 'checksums.txt'
0030:
0031: snapshot:
0032:   name_template: "{{ incpatch .Version }}-next"
0033:
0034: changelog:
0035:   sort: asc
0036:   filters:
0037:     exclude:
0038:       - '^docs:'
0039:       - '^test:'

### FILE: CONTRIBUTING.md
0001: # Contributing to cliproxyapi++
0002:
0003: First off, thank you for considering contributing to **cliproxyapi++**! It's people like you who make this tool better for everyone.
0004:
0005: ## Code of Conduct
0006:
0007: By participating in this project, you agree to abide by our [Code of Conduct](CODE_OF_CONDUCT.md) (coming soon).
0008:
0009: ## How Can I Contribute?
0010:
0011: ### Reporting Bugs
0012: - Use the [Bug Report](https://github.com/KooshaPari/cliproxyapi-plusplus/issues/new?template=bug_report.md) template.
0013: - Provide a clear and descriptive title.
0014: - Describe the exact steps to reproduce the problem.
0015:
0016: ### Suggesting Enhancements
0017: - Check the [Issues](https://github.com/KooshaPari/cliproxyapi-plusplus/issues) to see if the enhancement has already been suggested.
0018: - Use the [Feature Request](https://github.com/KooshaPari/cliproxyapi-plusplus/issues/new?template=feature_request.md) template.
0019:
0020: ### Pull Requests
0021: 1. Fork the repo and create your branch from `main`.
0022: 2. If you've added code that should be tested, add tests.
0023: 3. If you've changed APIs, update the documentation.
0024: 4. Ensure the test suite passes (`go test ./...`).
0025: 5. Make sure your code lints (`golangci-lint run`).
0026:
0027: #### Which repository to use?
0028: - **Third-party provider support**: Submit your PR directly to [KooshaPari/cliproxyapi-plusplus](https://github.com/KooshaPari/cliproxyapi-plusplus).
0029: - **Core logic improvements**: If the change is not specific to a third-party provider, please propose it to the [mainline project](https://github.com/router-for-me/CLIProxyAPI) first.
0030:
0031: ## Governance
0032:
0033: This project follows a community-driven governance model. Major architectural decisions are discussed in Issues before implementation.
0034:
0035: ### Path Guard
0036: We use a `pr-path-guard` to protect critical translator logic. Changes to these paths require explicit review from project maintainers to ensure security and stability.
0037:
0038: ---
0039: Thank you for your contributions!

### FILE: README.md
0001: # cliproxyapi++ 🚀
0002:
0003: [![Go Report Card](https://goreportcard.com/badge/github.com/KooshaPari/cliproxyapi-plusplus)](https://goreportcard.com/report/github.com/KooshaPari/cliproxyapi-plusplus)
0004: [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
0005: [![Docker Pulls](https://img.shields.io/docker/pulls/kooshapari/cliproxyapi-plusplus.svg)](https://hub.docker.com/r/kooshapari/cliproxyapi-plusplus)
0006: [![GitHub Release](https://img.shields.io/github/v/release/KooshaPari/cliproxyapi-plusplus)](https://github.com/KooshaPari/cliproxyapi-plusplus/releases)
0007:
0008: English | [中文](README_CN.md)
0009:
0010: **cliproxyapi++** is the definitive high-performance, security-hardened fork of [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI). Designed with a "Defense in Depth" philosophy and a "Library-First" architecture, it provides an OpenAI-compatible interface for proprietary LLMs with enterprise-grade stability.
0011:
0012: ---
0013:
0014: ## 🏆 Deep Dive: The `++` Advantage
0015:
0016: Why choose **cliproxyapi++** over the mainline? While the mainline focus is on open-source stability, the `++` variant is built for high-scale, production environments where security, automated lifecycle management, and broad provider support are critical.
0017:
0018: Full feature-by-feature change reference:
0019:
0020: - **[Feature Changes in ++](./docs/FEATURE_CHANGES_PLUSPLUS.md)**
0021:
0022: ### 📊 Feature Comparison Matrix
0023:
0024: | Feature | Mainline | CLIProxyAPI+ | **cliproxyapi++** |
0025: | :--- | :---: | :---: | :---: |
0026: | **Core Proxy Logic** | ✅ | ✅ | ✅ |
0027: | **Basic Provider Support** | ✅ | ✅ | ✅ |
0028: | **Standard UI** | ❌ | ✅ | ✅ |
0029: | **Advanced Auth (Kiro/Copilot)** | ❌ | ⚠️ | ✅ **(Full Support)** |
0030: | **Background Token Refresh** | ❌ | ❌ | ✅ **(Auto-Refresh)** |
0031: | **Security Hardening** | Basic | Basic | ✅ **(Enterprise-Grade)** |
0032: | **Rate Limiting & Cooldown** | ❌ | ❌ | ✅ **(Intelligent)** |
0033: | **Core Reusability** | `internal/` | `internal/` | ✅ **(`pkg/llmproxy`)** |
0034: | **CI/CD Pipeline** | Basic | Basic | ✅ **(Signed/Multi-arch)** |
0035:
0036: ---
0037:
0038: ## 🔍 Technical Differences & Hardening
0039:
0040: ### 1. Architectural Evolution: `pkg/llmproxy`
0041: Unlike the mainline which keeps its core logic in `internal/` (preventing external Go projects from importing it), **cliproxyapi++** has refactored its entire translation and proxying engine into a clean, public `pkg/llmproxy` library.
0042: *   **Reusability**: Import the proxy logic directly into your own Go applications.
0043: *   **Decoupling**: Configuration management is strictly separated from execution logic.
0044:
0045: ### 2. Enterprise Authentication & Lifecycle
0046: *   **Full GitHub Copilot Integration**: Not just an API wrapper. `++` includes a full OAuth device flow, per-credential quota tracking, and intelligent session management.
0047: *   **Kiro (AWS CodeWhisperer) 2.0**: A custom-built web UI (`/v0/oauth/kiro`) for browser-based AWS Builder ID and Identity Center logins.
0048: *   **Background Token Refresh**: A dedicated worker service monitors tokens and automatically refreshes them 10 minutes before expiration, ensuring zero downtime for your agents.
0049:
0050: ### 3. Security Hardening ("Defense in Depth")
0051: *   **Path Guard**: A custom GitHub Action workflow (`pr-path-guard`) that prevents any unauthorized changes to critical `internal/translator/` logic during PRs.
0052: *   **Device Fingerprinting**: Generates unique, immutable device identifiers to satisfy strict provider security checks and prevent account flagging.
0053: *   **Hardened Docker Base**: Built on a specific, audited Alpine 3.22.0 layer with minimal packages, reducing the potential attack surface.
0054:
0055: ### 4. High-Scale Operations
0056: *   **Intelligent Cooldown**: Automated "cooling" mechanism that detects provider-side rate limits and intelligently pauses requests to specific providers while routing others.
0057: *   **Unified Model Converter**: A sophisticated mapping layer that allows you to request `claude-3-5-sonnet` and have the proxy automatically handle the specific protocol requirements of the target provider (Vertex, AWS, Anthropic, etc.).
0058:
0059: ---
0060:
0061: ## 🚀 Getting Started
0062:
0063: ### Prerequisites
0064: - [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/)
0065: - OR [Go 1.26+](https://golang.org/dl/)
0066:
0067: ### One-Command Deployment (Docker)
0068:
0069: ```bash
0070: # Setup deployment
0071: mkdir -p ~/cliproxy && cd ~/cliproxy
0072: curl -o config.yaml https://raw.githubusercontent.com/KooshaPari/cliproxyapi-plusplus/main/config.example.yaml
0073:
0074: # Create compose file
0075: cat > docker-compose.yml << 'EOF'
0076: services:
0077:   cliproxy:
0078:     image: KooshaPari/cliproxyapi-plusplus:latest
0079:     container_name: cliproxyapi++
0080:     ports: ["8317:8317"]
0081:     volumes:
0082:       - ./config.yaml:/CLIProxyAPI/config.yaml
0083:       - ./auths:/root/.cli-proxy-api
0084:       - ./logs:/CLIProxyAPI/logs
0085:     restart: unless-stopped
0086: EOF
0087:
0088: docker compose up -d
0089: ```
0090:
0091: ---
0092:
0093: ## 🛠️ Advanced Usage
0094:
0095: ### Extended Provider Support
0096: `cliproxyapi++` supports a massive registry of providers out-of-the-box:
0097: *   **Direct**: Claude, Gemini, OpenAI, Mistral, Groq, DeepSeek.
0098: *   **Aggregators**: OpenRouter, Together AI, Fireworks AI, Novita AI, SiliconFlow.
0099: *   **Proprietary**: Kiro (AWS), GitHub Copilot, Roo Code, Kilo AI, MiniMax.
0100:
0101: ### API Specification
0102: The proxy provides two main API surfaces:
0103: 1.  **OpenAI Interface**: `/v1/chat/completions` and `/v1/models` (Full parity).
0104: 2.  **Management Interface**:
0105:     *   `GET /v0/config`: Inspect current (hot-reloaded) config.
0106:     *   `GET /v0/oauth/kiro`: Interactive Kiro auth UI.
0107:     *   `GET /v0/logs`: Real-time log inspection.
0108:
0109: ---
0110:
0111: ## 🤝 Contributing
0112:
0113: We maintain strict quality gates to preserve the "hardened" status of the project:
0114: 1.  **Linting**: Must pass `golangci-lint` with zero warnings.
0115: 2.  **Coverage**: All new translator logic MUST include unit tests.
0116: 3.  **Governance**: Changes to core `pkg/` logic require a corresponding Issue discussion.
0117:
0118: See **[CONTRIBUTING.md](CONTRIBUTING.md)** for more details.
0119:
0120: ---
0121:
0122: ## 📚 Documentation
0123:
0124: - **[Docsets](./docs/docsets/)** — Role-oriented documentation sets.
0125:   - [Developer (Internal)](./docs/docsets/developer/internal/)
0126:   - [Developer (External)](./docs/docsets/developer/external/)
0127:   - [Technical User](./docs/docsets/user/)
0128:   - [Agent Operator](./docs/docsets/agent/)
0129: - **[Feature Changes in ++](./docs/FEATURE_CHANGES_PLUSPLUS.md)** — Comprehensive list of `++` differences and impacts.
0130: - **[Docs README](./docs/README.md)** — Core docs map.
0131:
0132: ---
0133:
0134: ## 🚢 Docs Deploy
0135:
0136: Local VitePress docs:
0137:
0138: ```bash
0139: cd docs
0140: npm install
0141: npm run docs:dev
0142: npm run docs:build
0143: ```
0144:
0145: GitHub Pages:
0146:
0147: - Workflow: `.github/workflows/vitepress-pages.yml`
0148: - URL convention: `https://<owner>.github.io/cliproxyapi-plusplus/`
0149:
0150: ---
0151:
0152: ## 📜 License
0153:
0154: Distributed under the MIT License. See [LICENSE](LICENSE) for more information.
0155:
0156: ---
0157:
0158: <p align="center">
0159:   <b>Hardened AI Infrastructure for the Modern Agentic Stack.</b><br>
0160:   Built with ❤️ by the community.
0161: </p>

### FILE: README_CN.md
0001: # cliproxyapi++ 🚀
0002:
0003: [![Go Report Card](https://goreportcard.com/badge/github.com/KooshaPari/cliproxyapi-plusplus)](https://goreportcard.com/report/github.com/KooshaPari/cliproxyapi-plusplus)
0004: [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
0005: [![Docker Pulls](https://img.shields.io/docker/pulls/kooshapari/cliproxyapi-plusplus.svg)](https://hub.docker.com/r/kooshapari/cliproxyapi-plusplus)
0006: [![GitHub Release](https://img.shields.io/github/v/release/KooshaPari/cliproxyapi-plusplus)](https://github.com/KooshaPari/cliproxyapi-plusplus/releases)
0007:
0008: [English](README.md) | 中文
0009:
0010: **cliproxyapi++** 是 [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) 的高性能、经过安全加固的终极分支版本。它秉持“纵深防御”的开发理念和“库优先”的架构设计，为多种主流及私有大模型提供 OpenAI 兼容接口，并具备企业级稳定性。
0011:
0012: ---
0013:
0014: ## 🏆 深度对比：`++` 版本的优势
0015:
0016: 为什么选择 **cliproxyapi++** 而不是主线版本？虽然主线版本专注于开源社区的稳定性，但 `++` 版本则是为高并发、生产级环境而设计的，在安全性、自动化生命周期管理和广泛的提供商支持方面具有显著优势。
0017:
0018: ### 📊 功能对比矩阵
0019:
0020: | 功能特性 | 主线版本 | CLIProxyAPI+ | **cliproxyapi++** |
0021: | :--- | :---: | :---: | :---: |
0022: | **核心代理逻辑** | ✅ | ✅ | ✅ |
0023: | **基础模型支持** | ✅ | ✅ | ✅ |
0024: | **标准 Web UI** | ❌ | ✅ | ✅ |
0025: | **高级认证 (Kiro/Copilot)** | ❌ | ⚠️ | ✅ **(完整支持)** |
0026: | **后台令牌自动刷新** | ❌ | ❌ | ✅ **(自动刷新)** |
0027: | **安全加固** | 基础 | 基础 | ✅ **(企业级)** |
0028: | **频率限制与冷却** | ❌ | ❌ | ✅ **(智能路由)** |
0029: | **核心逻辑复用** | `internal/` | `internal/` | ✅ **(`pkg/llmproxy`)** |
0030: | **CI/CD 流水线** | 基础 | 基础 | ✅ **(签名/多架构)** |
0031:
0032: ---
0033:
0034: ## 🔍 技术差异与安全加固
0035:
0036: ### 1. 架构演进：`pkg/llmproxy`
0037: 主线版本将核心逻辑保留在 `internal/` 目录下（这会导致外部 Go 项目无法直接导入），而 **cliproxyapi++** 已将整个翻译和代理引擎重构为清晰、公开的 `pkg/llmproxy` 库。
0038: *   **可复用性**: 您可以直接在自己的 Go 应用程序中导入代理逻辑。
0039: *   **解耦**: 实现了配置管理与执行逻辑的严格分离。
0040:
0041: ### 2. 企业级身份认证与生命周期管理
0042: *   **完整 GitHub Copilot 集成**: 不仅仅是 API 包装。`++` 包含完整的 OAuth 设备流登录、每个凭据的额度追踪以及智能会话管理。
0043: *   **Kiro (AWS CodeWhisperer) 2.0**: 提供定制化的 Web 界面 (`/v0/oauth/kiro`)，支持通过浏览器进行 AWS Builder ID 和 Identity Center 登录。
0044: *   **后台令牌刷新**: 专门的后台服务实时监控令牌状态，并在过期前 10 分钟自动刷新，确保智能体任务零停机。
0045:
0046: ### 3. 安全加固（“纵深防御”）
0047: *   **路径保护 (Path Guard)**: 定制的 GitHub Action 工作流 (`pr-path-guard`)，防止在 PR 过程中对关键的 `internal/translator/` 逻辑进行任何未经授权的修改。
0048: *   **设备指纹**: 生成唯一且不可变的设备标识符，以满足严格的提供商安全检查，防止账号被标记。
0049: *   **加固的 Docker 基础镜像**: 基于经过审计的 Alpine 3.22.0 层构建，仅包含最少软件包，显著降低了潜在的攻击面。
0050:
0051: ### 4. 高规模运营支持
0052: *   **智能冷却机制**: 自动化的“冷却”系统可检测提供商端的频率限制，并智能地暂停对特定供应商的请求，同时将流量路由至其他可用节点。
0053: *   **统一模型转换器**: 复杂的映射层，允许您请求 `claude-3-5-sonnet`，而由代理自动处理目标供应商（如 Vertex、AWS、Anthropic 等）的具体协议要求。
0054:
0055: ---
0056:
0057: ## 🚀 快速开始
0058:
0059: ### 先决条件
0060: - 已安装 [Docker](https://docs.docker.com/get-docker/) 和 [Docker Compose](https://docs.docker.com/compose/install/)
0061: - 或安装 [Go 1.26+](https://golang.org/dl/)
0062:
0063: ### 一键部署 (Docker)
0064:
0065: ```bash
0066: # 设置部署目录
0067: mkdir -p ~/cliproxy && cd ~/cliproxy
0068: curl -o config.yaml https://raw.githubusercontent.com/KooshaPari/cliproxyapi-plusplus/main/config.example.yaml
0069:
0070: # 创建 compose 文件
0071: cat > docker-compose.yml << 'EOF'
0072: services:
0073:   cliproxy:
0074:     image: KooshaPari/cliproxyapi-plusplus:latest
0075:     container_name: cliproxyapi++
0076:     ports: ["8317:8317"]
0077:     volumes:
0078:       - ./config.yaml:/CLIProxyAPI/config.yaml
0079:       - ./auths:/root/.cli-proxy-api
0080:       - ./logs:/CLIProxyAPI/logs
0081:     restart: unless-stopped
0082: EOF
0083:
0084: docker compose up -d
0085: ```
0086:
0087: ---
0088:
0089: ## 🛠️ 高级用法
0090:
0091: ### 扩展的供应商支持
0092: `cliproxyapi++` 开箱即用地支持海量模型注册：
0093: *   **直接接入**: Claude, Gemini, OpenAI, Mistral, Groq, DeepSeek.
0094: *   **聚合器**: OpenRouter, Together AI, Fireworks AI, Novita AI, SiliconFlow.
0095: *   **私有协议**: Kiro (AWS), GitHub Copilot, Roo Code, Kilo AI, MiniMax.
0096:
0097: ### API 规范
0098: 代理提供两个主要的 API 表面：
0099: 1.  **OpenAI 兼容接口**: `/v1/chat/completions` 和 `/v1/models`。
0100: 2.  **管理接口**:
0101:     *   `GET /v0/config`: 查看当前（支持热重载）的配置。
0102:     *   `GET /v0/oauth/kiro`: 交互式 Kiro 认证界面。
0103:     *   `GET /v0/logs`: 实时日志查看。
0104:
0105: ---
0106:
0107: ## 🤝 贡献指南
0108:
0109: 我们维持严格的质量门禁，以保持项目的“加固”状态：
0110: 1.  **代码风格**: 必须通过 `golangci-lint` 检查，且无任何警告。
0111: 2.  **测试覆盖**: 所有的翻译器逻辑必须包含单元测试。
0112: 3.  **治理**: 对 `pkg/` 核心逻辑的修改需要先在 Issue 中进行讨论。
0113:
0114: 请参阅 **[CONTRIBUTING.md](CONTRIBUTING.md)** 了解更多详情。
0115:
0116: ---
0117:
0118: ## 📜 开源协议
0119:
0120: 本项目根据 MIT 许可证发行。详情请参阅 [LICENSE](LICENSE) 文件。
0121:
0122: ---
0123:
0124: <p align="center">
0125:   <b>为现代智能体技术栈打造的加固级 AI 基础设施。</b><br>
0126:   由社区倾力打造 ❤️
0127: </p>

### FILE: SECURITY.md
0001: # Security Policy
0002:
0003: ## Supported Versions
0004:
0005: | Version | Supported          |
0006: | ------- | ------------------ |
0007: | 6.0.x   | :white_check_mark: |
0008: | < 6.0   | :x:                |
0009:
0010: ## Reporting a Vulnerability
0011:
0012: We take the security of **cliproxyapi++** seriously. If you discover a security vulnerability, please do NOT open a public issue. Instead, report it privately.
0013:
0014: Please report any security concerns directly to the maintainers at [kooshapari@gmail.com](mailto:kooshapari@gmail.com) (assuming this as the email for KooshaPari).
0015:
0016: ### What to include
0017: - A detailed description of the vulnerability.
0018: - Steps to reproduce (proof of concept).
0019: - Potential impact.
0020: - Any suggested fixes or mitigations.
0021:
0022: We will acknowledge your report within 48 hours and provide a timeline for resolution.
0023:
0024: ## Hardening Measures
0025:
0026: **cliproxyapi++** incorporates several security-hardening features:
0027:
0028: - **Minimal Docker Images**: Based on Alpine Linux to reduce attack surface.
0029: - **Path Guard**: GitHub Actions that monitor and protect critical translation and core logic files.
0030: - **Rate Limiting**: Built-in mechanisms to prevent DoS attacks.
0031: - **Device Fingerprinting**: Enhanced authentication security using device-specific metadata.
0032: - **Dependency Scanning**: Automatic scanning for vulnerable Go modules.
0033:
0034: ---
0035: Thank you for helping keep the community secure!

### FILE: Taskfile.yml
0001: # Taskfile for cliproxyapi++
0002: # Unified DX for building, testing, and managing the proxy.
0003:
0004: version: '3'
0005:
0006: vars:
0007:   BINARY_NAME: cliproxyapi++
0008:   DOCKER_IMAGE: kooshapari/cliproxyapi-plusplus
0009:   GO_FILES:
0010:     sh: find . -name "*.go" | grep -v "vendor"
0011:
0012: tasks:
0013:   default:
0014:     cmds:
0015:       - task --list
0016:     silent: true
0017:
0018:   # -- Build & Run --
0019:   build:
0020:     desc: "Build the cliproxyapi++ binary"
0021:     cmds:
0022:       - go build -o {{.BINARY_NAME}} ./cmd/server
0023:     sources:
0024:       - "**/*.go"
0025:       - "go.mod"
0026:       - "go.sum"
0027:     generates:
0028:       - "{{.BINARY_NAME}}"
0029:
0030:   run:
0031:     desc: "Run the proxy locally with default config"
0032:     deps: [build]
0033:     cmds:
0034:       - ./{{.BINARY_NAME}} --config config.example.yaml
0035:
0036:   # -- Testing & Quality --
0037:   test:
0038:     desc: "Run all Go tests"
0039:     cmds:
0040:       - go test -v ./...
0041:
0042:   lint:
0043:     desc: "Run golangci-lint"
0044:     cmds:
0045:       - golangci-lint run ./...
0046:
0047:   tidy:
0048:     desc: "Tidy Go modules"
0049:     cmds:
0050:       - go mod tidy
0051:
0052:   # -- Docker Operations --
0053:   docker:build:
0054:     desc: "Build Docker image locally"
0055:     cmds:
0056:       - docker build -t {{.DOCKER_IMAGE}}:local .
0057:
0058:   docker:run:
0059:     desc: "Run proxy via Docker"
0060:     cmds:
0061:       - docker compose up -d
0062:
0063:   docker:stop:
0064:     desc: "Stop Docker proxy"
0065:     cmds:
0066:       - docker compose down
0067:
0068:   # -- Health & Diagnostics (UX/DX) --
0069:   doctor:
0070:     desc: "Check environment health for cliproxyapi++"
0071:     cmds:
0072:       - |
0073:         echo "Checking Go version..."
0074:         go version
0075:         echo "Checking dependencies..."
0076:         if [ ! -f go.mod ]; then echo "❌ go.mod missing"; exit 1; fi
0077:         echo "Checking config template..."
0078:         if [ ! -f config.example.yaml ]; then echo "❌ config.example.yaml missing"; exit 1; fi
0079:         echo "Checking Docker..."
0080:         docker --version || echo "⚠️ Docker not installed"
0081:         echo "✅ cliproxyapi++ environment looks healthy!"
0082:
0083:   # -- Agent Experience (AX) --
0084:   ax:spec:
0085:     desc: "Generate or verify agent-readable specs"
0086:     cmds:
0087:       - echo "Checking for llms.txt..."
0088:       - if [ ! -f llms.txt ]; then echo "⚠️ llms.txt missing"; else echo "✅ llms.txt present"; fi

### FILE: cmd/codegen/main.go
0001: package main
0002:
0003: import (
0004: 	"bytes"
0005: 	"encoding/json"
0006: 	"fmt"
0007: 	"go/format"
0008: 	"log"
0009: 	"os"
0010: 	"path/filepath"
0011: 	"strings"
0012: 	"text/template"
0013: )
0014:
0015: type ProviderSpec struct {
0016: 	Name          string                     `json:"name"`
0017: 	YAMLKey       string                     `json:"yaml_key"`
0018: 	GoName        string                     `json:"go_name"`
0019: 	BaseURL       string                     `json:"base_url"`
0020: 	EnvVars       []string                   `json:"env_vars"`
0021: 	DefaultModels []OpenAICompatibilityModel `json:"default_models"`
0022: }
0023:
0024: type OpenAICompatibilityModel struct {
0025: 	Name  string `json:"name"`
0026: 	Alias string `json:"alias"`
0027: }
0028:
0029: const configTemplate = `// Code generated by github.com/router-for-me/CLIProxyAPI/v6/cmd/codegen; DO NOT EDIT.
0030: package config
0031:
0032: import "strings"
0033:
0034: // GeneratedConfig contains generated config fields for dedicated providers.
0035: type GeneratedConfig struct {
0036: {{- range .Providers }}
0037: 	{{- if .YAMLKey }}
0038: 	// {{ .Name | goTitle }}Key defines {{ .Name | goTitle }} configurations.
0039: 	{{ .Name | goTitle }}Key []{{ .Name | goTitle }}Key {{ printf "` + "`" + `yaml:\"%s\" json:\"%s\"` + "`" + `" .YAMLKey .YAMLKey }}
0040: 	{{- end }}
0041: {{- end }}
0042: }
0043:
0044: {{ range .Providers }}
0045: {{- if .YAMLKey }}
0046: // {{ .Name | goTitle }}Key is a type alias for OAICompatProviderConfig for the {{ .Name }} provider.
0047: type {{ .Name | goTitle }}Key = OAICompatProviderConfig
0048: {{- end }}
0049: {{- end }}
0050:
0051: // SanitizeGeneratedProviders trims whitespace from generated provider credential fields.
0052: func (cfg *Config) SanitizeGeneratedProviders() {
0053: 	if cfg == nil {
0054: 		return
0055: 	}
0056: {{- range .Providers }}
0057: 	{{- if .YAMLKey }}
0058: 	for i := range cfg.{{ .Name | goTitle }}Key {
0059: 		entry := &cfg.{{ .Name | goTitle }}Key[i]
0060: 		entry.TokenFile = strings.TrimSpace(entry.TokenFile)
0061: 		entry.APIKey = strings.TrimSpace(entry.APIKey)
0062: 		entry.BaseURL = strings.TrimSpace(entry.BaseURL)
0063: 		entry.ProxyURL = strings.TrimSpace(entry.ProxyURL)
0064: 	}
0065: 	{{- end }}
0066: {{- end }}
0067: }
0068: `
0069:
0070: const synthTemplate = `// Code generated by github.com/router-for-me/CLIProxyAPI/v6/cmd/codegen; DO NOT EDIT.
0071: package synthesizer
0072:
0073: import (
0074: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/config"
0075: )
0076:
0077: // getDedicatedProviderEntries returns the config entries for a dedicated provider.
0078: func (s *ConfigSynthesizer) getDedicatedProviderEntries(p config.ProviderSpec, cfg *config.Config) []config.OAICompatProviderConfig {
0079: 	switch p.YAMLKey {
0080: {{- range .Providers }}
0081: 	{{- if .YAMLKey }}
0082: 	case "{{ .YAMLKey }}":
0083: 		return cfg.{{ .Name | goTitle }}Key
0084: 	{{- end }}
0085: {{- end }}
0086: 	}
0087: 	return nil
0088: }
0089: `
0090:
0091: const registryTemplate = `// Code generated by github.com/router-for-me/CLIProxyAPI/v6/cmd/codegen; DO NOT EDIT.
0092: package config
0093:
0094: // AllProviders defines the registry of all supported LLM providers.
0095: // This is the source of truth for generated config fields and synthesizers.
0096: var AllProviders = []ProviderSpec{
0097: {{- range .Providers }}
0098: 	{
0099: 		Name:    "{{ .Name }}",
0100: 		YAMLKey: "{{ .YAMLKey }}",
0101: 		GoName:  "{{ .GoName }}",
0102: 		BaseURL: "{{ .BaseURL }}",
0103: 		{{- if .EnvVars }}
0104: 		EnvVars: []string{
0105: 			{{- range .EnvVars }}"{{ . }}",{{ end -}}
0106: 		},
0107: 		{{- end }}
0108: 		{{- if .DefaultModels }}
0109: 		DefaultModels: []OpenAICompatibilityModel{
0110: 			{{- range .DefaultModels }}
0111: 			{Name: "{{ .Name }}", Alias: "{{ .Alias }}"},
0112: 			{{- end }}
0113: 		},
0114: 		{{- end }}
0115: 	},
0116: {{- end }}
0117: }
0118: `
0119:
0120: const diffTemplate = `// Code generated by github.com/router-for-me/CLIProxyAPI/v6/cmd/codegen; DO NOT EDIT.
0121: package diff
0122:
0123: import (
0124: 	"fmt"
0125: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/config"
0126: )
0127:
0128: // BuildConfigChangeDetailsGeneratedProviders computes changes for generated dedicated providers.
0129: func BuildConfigChangeDetailsGeneratedProviders(oldCfg, newCfg *config.Config, changes *[]string) {
0130: {{- range .Providers }}
0131: 	{{- if .YAMLKey }}
0132: 	if len(oldCfg.{{ .Name | goTitle }}Key) != len(newCfg.{{ .Name | goTitle }}Key) {
0133: 		*changes = append(*changes, fmt.Sprintf("{{ .Name }}: count %d -> %d", len(oldCfg.{{ .Name | goTitle }}Key), len(newCfg.{{ .Name | goTitle }}Key)))
0134: 	}
0135: 	{{- end }}
0136: {{- end }}
0137: }
0138: `
0139:
0140: func main() {
0141: 	jsonPath := "pkg/llmproxy/config/providers.json"
0142: 	configDir := "pkg/llmproxy/config"
0143: 	authDir := "pkg/llmproxy/auth"
0144:
0145: 	if _, err := os.Stat(jsonPath); os.IsNotExist(err) {
0146: 		// Try fallback for when run from within the config directory
0147: 		jsonPath = "providers.json"
0148: 		configDir = "."
0149: 		authDir = "../auth"
0150: 	}
0151:
0152: 	data, err := os.ReadFile(jsonPath)
0153: 	if err != nil {
0154: 		log.Fatalf("failed to read providers.json from %s: %v", jsonPath, err)
0155: 	}
0156:
0157: 	var providers []ProviderSpec
0158: 	if err := json.Unmarshal(data, &providers); err != nil {
0159: 		log.Fatalf("failed to unmarshal providers: %v", err)
0160: 	}

### FILE: cmd/server/main.go
0001: // Package main provides the entry point for the CLI Proxy API server.
0002: // This server acts as a proxy that provides OpenAI/Gemini/Claude compatible API interfaces
0003: // for CLI models, allowing CLI models to be used with tools and libraries designed for standard AI APIs.
0004: package main
0005:
0006: import (
0007: 	"context"
0008: 	"errors"
0009: 	"flag"
0010: 	"fmt"
0011: 	"io"
0012: 	"io/fs"
0013: 	"net/url"
0014: 	"os"
0015: 	"path/filepath"
0016: 	"strings"
0017: 	"time"
0018:
0019: 	"github.com/joho/godotenv"
0020: 	configaccess "github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/access/config_access"
0021: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/kiro"
0022: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/buildinfo"
0023: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/cmd"
0024: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/config"
0025: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/logging"
0026: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/managementasset"
0027: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/misc"
0028: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/store"
0029: 	_ "github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/translator"
0030: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/tui"
0031: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/usage"
0032: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/util"
0033: 	sdkAuth "github.com/router-for-me/CLIProxyAPI/v6/sdk/auth"
0034: 	coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
0035: 	log "github.com/sirupsen/logrus"
0036: )
0037:
0038: var (
0039: 	Version           = "dev"
0040: 	Commit            = "none"
0041: 	BuildDate         = "unknown"
0042: 	DefaultConfigPath = ""
0043: )
0044:
0045: // init initializes the shared logger setup.
0046: func init() {
0047: 	logging.SetupBaseLogger()
0048: 	buildinfo.Version = Version
0049: 	buildinfo.Commit = Commit
0050: 	buildinfo.BuildDate = BuildDate
0051: }
0052:
0053: // setKiroIncognitoMode sets the incognito browser mode for Kiro authentication.
0054: // Kiro defaults to incognito mode for multi-account support.
0055: // Users can explicitly override with --incognito or --no-incognito flags.
0056: func setKiroIncognitoMode(cfg *config.Config, useIncognito, noIncognito bool) {
0057: 	if useIncognito {
0058: 		cfg.IncognitoBrowser = true
0059: 	} else if noIncognito {
0060: 		cfg.IncognitoBrowser = false
0061: 	} else {
0062: 		cfg.IncognitoBrowser = true // Kiro default
0063: 	}
0064: }
0065:
0066: // main is the entry point of the application.
0067: // It parses command-line flags, loads configuration, and starts the appropriate
0068: // service based on the provided flags (login, codex-login, or server mode).
0069: func main() {
0070: 	fmt.Printf("CLIProxyAPI Version: %s, Commit: %s, BuiltAt: %s\n", buildinfo.Version, buildinfo.Commit, buildinfo.BuildDate)
0071:
0072: 	// Command-line flags to control the application's behavior.
0073: 	var login bool
0074: 	var codexLogin bool
0075: 	var claudeLogin bool
0076: 	var qwenLogin bool
0077: 	var kiloLogin bool
0078: 	var iflowLogin bool
0079: 	var iflowCookie bool
0080: 	var noBrowser bool
0081: 	var oauthCallbackPort int
0082: 	var antigravityLogin bool
0083: 	var kimiLogin bool
0084: 	var kiroLogin bool
0085: 	var kiroGoogleLogin bool
0086: 	var kiroAWSLogin bool
0087: 	var kiroAWSAuthCode bool
0088: 	var kiroImport bool
0089: 	var githubCopilotLogin bool
0090: 	var rooLogin bool
0091: 	var minimaxLogin bool
0092: 	var deepseekLogin bool
0093: 	var groqLogin bool
0094: 	var mistralLogin bool
0095: 	var siliconflowLogin bool
0096: 	var openrouterLogin bool
0097: 	var togetherLogin bool
0098: 	var fireworksLogin bool
0099: 	var novitaLogin bool
0100: 	var projectID string
0101: 	var vertexImport string
0102: 	var configPath string
0103: 	var password string
0104: 	var tuiMode bool
0105: 	var standalone bool
0106: 	var noIncognito bool
0107: 	var useIncognito bool
0108:
0109: 	// Define command-line flags for different operation modes.
0110: 	flag.BoolVar(&login, "login", false, "Login Google Account")
0111: 	flag.BoolVar(&codexLogin, "codex-login", false, "Login to Codex using OAuth")
0112: 	flag.BoolVar(&claudeLogin, "claude-login", false, "Login to Claude using OAuth")
0113: 	flag.BoolVar(&qwenLogin, "qwen-login", false, "Login to Qwen using OAuth")
0114: 	flag.BoolVar(&kiloLogin, "kilo-login", false, "Login to Kilo AI using device flow")
0115: 	flag.BoolVar(&iflowLogin, "iflow-login", false, "Login to iFlow using OAuth")
0116: 	flag.BoolVar(&iflowCookie, "iflow-cookie", false, "Login to iFlow using Cookie")
0117: 	flag.BoolVar(&noBrowser, "no-browser", false, "Don't open browser automatically for OAuth")
0118: 	flag.IntVar(&oauthCallbackPort, "oauth-callback-port", 0, "Override OAuth callback port (defaults to provider-specific port)")
0119: 	flag.BoolVar(&useIncognito, "incognito", false, "Open browser in incognito/private mode for OAuth (useful for multiple accounts)")
0120: 	flag.BoolVar(&noIncognito, "no-incognito", false, "Force disable incognito mode (uses existing browser session)")
0121: 	flag.BoolVar(&antigravityLogin, "antigravity-login", false, "Login to Antigravity using OAuth")
0122: 	flag.BoolVar(&kimiLogin, "kimi-login", false, "Login to Kimi using OAuth")
0123: 	flag.BoolVar(&kiroLogin, "kiro-login", false, "Login to Kiro using Google OAuth")
0124: 	flag.BoolVar(&kiroGoogleLogin, "kiro-google-login", false, "Login to Kiro using Google OAuth (same as --kiro-login)")
0125: 	flag.BoolVar(&kiroAWSLogin, "kiro-aws-login", false, "Login to Kiro using AWS Builder ID (device code flow)")
0126: 	flag.BoolVar(&kiroAWSAuthCode, "kiro-aws-authcode", false, "Login to Kiro using AWS Builder ID (authorization code flow, better UX)")
0127: 	flag.BoolVar(&kiroImport, "kiro-import", false, "Import Kiro token from Kiro IDE (~/.aws/sso/cache/kiro-auth-token.json)")
0128: 	flag.BoolVar(&githubCopilotLogin, "github-copilot-login", false, "Login to GitHub Copilot using device flow")
0129: 	flag.BoolVar(&rooLogin, "roo-login", false, "Login to Roo Code (runs roo auth login)")
0130: 	flag.BoolVar(&minimaxLogin, "minimax-login", false, "MiniMax config instructions (add minimax: block with api-key)")
0131: 	flag.BoolVar(&deepseekLogin, "deepseek-login", false, "Login to DeepSeek using API key (stored in auth-dir)")
0132: 	flag.BoolVar(&groqLogin, "groq-login", false, "Login to Groq using API key (stored in auth-dir)")
0133: 	flag.BoolVar(&mistralLogin, "mistral-login", false, "Login to Mistral using API key (stored in auth-dir)")
0134: 	flag.BoolVar(&siliconflowLogin, "siliconflow-login", false, "Login to SiliconFlow using API key (stored in auth-dir)")
0135: 	flag.BoolVar(&openrouterLogin, "openrouter-login", false, "Login to OpenRouter using API key (stored in auth-dir)")
0136: 	flag.BoolVar(&togetherLogin, "together-login", false, "Login to Together AI using API key (stored in auth-dir)")
0137: 	flag.BoolVar(&fireworksLogin, "fireworks-login", false, "Login to Fireworks AI using API key (stored in auth-dir)")
0138: 	flag.BoolVar(&novitaLogin, "novita-login", false, "Login to Novita AI using API key (stored in auth-dir)")
0139: 	flag.StringVar(&projectID, "project_id", "", "Project ID (Gemini only, not required)")
0140: 	flag.StringVar(&configPath, "config", DefaultConfigPath, "Configure File Path")
0141: 	flag.StringVar(&vertexImport, "vertex-import", "", "Import Vertex service account key JSON file")
0142: 	flag.StringVar(&password, "password", "", "")
0143: 	flag.BoolVar(&tuiMode, "tui", false, "Start with terminal management UI")
0144: 	flag.BoolVar(&standalone, "standalone", false, "In TUI mode, start an embedded local server")
0145:
0146: 	flag.CommandLine.Usage = func() {
0147: 		out := flag.CommandLine.Output()
0148: 		_, _ = fmt.Fprintf(out, "Usage of %s\n", os.Args[0])
0149: 		flag.CommandLine.VisitAll(func(f *flag.Flag) {
0150: 			if f.Name == "password" {
0151: 				return
0152: 			}
0153: 			s := fmt.Sprintf("  -%s", f.Name)
0154: 			name, unquoteUsage := flag.UnquoteUsage(f)
0155: 			if name != "" {
0156: 				s += " " + name
0157: 			}
0158: 			if len(s) <= 4 {
0159: 				s += "	"
0160: 			} else {

### FILE: config.example.yaml
0001: # Server host/interface to bind to. Default is empty ("") to bind all interfaces (IPv4 + IPv6).
0002: # Use "127.0.0.1" or "localhost" to restrict access to local machine only.
0003: host: ""
0004:
0005: # Server port
0006: port: 8317
0007:
0008: # TLS settings for HTTPS. When enabled, the server listens with the provided certificate and key.
0009: tls:
0010:   enable: false
0011:   cert: ""
0012:   key: ""
0013:
0014: # Management API settings
0015: remote-management:
0016:   # Whether to allow remote (non-localhost) management access.
0017:   # When false, only localhost can access management endpoints (a key is still required).
0018:   allow-remote: false
0019:
0020:   # Management key. If a plaintext value is provided here, it will be hashed on startup.
0021:   # All management requests (even from localhost) require this key.
0022:   # Leave empty to disable the Management API entirely (404 for all /v0/management routes).
0023:   secret-key: ""
0024:
0025:   # Disable the bundled management control panel asset download and HTTP route when true.
0026:   disable-control-panel: false
0027:
0028:   # GitHub repository for the management control panel. Accepts a repository URL or releases API URL.
0029:   panel-github-repository: "https://github.com/router-for-me/Cli-Proxy-API-Management-Center"
0030:
0031: # Authentication directory (supports ~ for home directory)
0032: auth-dir: "~/.cli-proxy-api"
0033:
0034: # API keys for authentication
0035: api-keys:
0036:   - "your-api-key-1"
0037:   - "your-api-key-2"
0038:   - "your-api-key-3"
0039:
0040: # Enable debug logging
0041: debug: false
0042:
0043: # Enable pprof HTTP debug server (host:port). Keep it bound to localhost for safety.
0044: pprof:
0045:   enable: false
0046:   addr: "127.0.0.1:8316"
0047:
0048: # When true, disable high-overhead HTTP middleware features to reduce per-request memory usage under high concurrency.
0049: commercial-mode: false
0050:
0051: # Open OAuth URLs in incognito/private browser mode.
0052: # Useful when you want to login with a different account without logging out from your current session.
0053: # Default: false (but Kiro auth defaults to true for multi-account support)
0054: incognito-browser: true
0055:
0056: # When true, write application logs to rotating files instead of stdout
0057: logging-to-file: false
0058:
0059: # Maximum total size (MB) of log files under the logs directory. When exceeded, the oldest log
0060: # files are deleted until within the limit. Set to 0 to disable.
0061: logs-max-total-size-mb: 0
0062:
0063: # Maximum number of error log files retained when request logging is disabled.
0064: # When exceeded, the oldest error log files are deleted. Default is 10. Set to 0 to disable cleanup.
0065: error-logs-max-files: 10
0066:
0067: # When false, disable in-memory usage statistics aggregation
0068: usage-statistics-enabled: false
0069:
0070: # Proxy URL. Supports socks5/http/https protocols. Example: socks5://user:pass@192.168.1.1:1080/
0071: proxy-url: ""
0072:
0073: # When true, unprefixed model requests only use credentials without a prefix (except when prefix == model name).
0074: force-model-prefix: false
0075:
0076: # Number of times to retry a request. Retries will occur if the HTTP response code is 403, 408, 500, 502, 503, or 504.
0077: request-retry: 3
0078:
0079: # Maximum wait time in seconds for a cooled-down credential before triggering a retry.
0080: max-retry-interval: 30
0081:
0082: # Quota exceeded behavior
0083: quota-exceeded:
0084:   switch-project: true # Whether to automatically switch to another project when a quota is exceeded
0085:   switch-preview-model: true # Whether to automatically switch to a preview model when a quota is exceeded
0086:
0087: # Routing strategy for selecting credentials when multiple match.
0088: routing:
0089:   strategy: "round-robin" # round-robin (default), fill-first
0090:
0091: # When true, enable authentication for the WebSocket API (/v1/ws).
0092: ws-auth: false
0093:
0094: # When > 0, emit blank lines every N seconds for non-streaming responses to prevent idle timeouts.
0095: nonstream-keepalive-interval: 0
0096:
0097: # Streaming behavior (SSE keep-alives + safe bootstrap retries).
0098: # streaming:
0099: #   keepalive-seconds: 15   # Default: 0 (disabled). <= 0 disables keep-alives.
0100: #   bootstrap-retries: 1    # Default: 0 (disabled). Retries before first byte is sent.
0101:
0102: # Gemini API keys
0103: # gemini-api-key:
0104: #   - api-key: "AIzaSy...01"
0105: #     prefix: "test" # optional: require calls like "test/gemini-3-pro-preview" to target this credential
0106: #     base-url: "https://generativelanguage.googleapis.com"
0107: #     headers:
0108: #       X-Custom-Header: "custom-value"
0109: #     proxy-url: "socks5://proxy.example.com:1080"
0110: #     models:
0111: #       - name: "gemini-2.5-flash" # upstream model name
0112: #         alias: "gemini-flash"    # client alias mapped to the upstream model
0113: #     excluded-models:
0114: #       - "gemini-2.5-pro"     # exclude specific models from this provider (exact match)
0115: #       - "gemini-2.5-*"       # wildcard matching prefix (e.g. gemini-2.5-flash, gemini-2.5-pro)
0116: #       - "*-preview"          # wildcard matching suffix (e.g. gemini-3-pro-preview)
0117: #       - "*flash*"            # wildcard matching substring (e.g. gemini-2.5-flash-lite)
0118: #   - api-key: "AIzaSy...02"
0119:
0120: # Codex API keys

### FILE: docker-build.ps1
0001: # build.ps1 - Windows PowerShell Build Script
0002: #
0003: # This script automates the process of building and running the Docker container
0004: # with version information dynamically injected at build time.
0005:
0006: # Stop script execution on any error
0007: $ErrorActionPreference = "Stop"
0008:
0009: # --- Step 1: Choose Environment ---
0010: Write-Host "Please select an option:"
0011: Write-Host "1) Run using Pre-built Image (Recommended)"
0012: Write-Host "2) Build from Source and Run (For Developers)"
0013: $choice = Read-Host -Prompt "Enter choice [1-2]"
0014:
0015: # --- Step 2: Execute based on choice ---
0016: switch ($choice) {
0017:     "1" {
0018:         Write-Host "--- Running with Pre-built Image ---"
0019:         docker compose up -d --remove-orphans --no-build
0020:         Write-Host "Services are starting from remote image."
0021:         Write-Host "Run 'docker compose logs -f' to see the logs."
0022:     }
0023:     "2" {
0024:         Write-Host "--- Building from Source and Running ---"
0025:
0026:         # Get Version Information
0027:         $VERSION = (git describe --tags --always --dirty)
0028:         $COMMIT  = (git rev-parse --short HEAD)
0029:         $BUILD_DATE = (Get-Date).ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ssZ")
0030:
0031:         Write-Host "Building with the following info:"
0032:         Write-Host "  Version: $VERSION"
0033:         Write-Host "  Commit: $COMMIT"
0034:         Write-Host "  Build Date: $BUILD_DATE"
0035:         Write-Host "----------------------------------------"
0036:
0037:         # Build and start the services with a local-only image tag
0038:         $env:CLI_PROXY_IMAGE = "cli-proxy-api:local"
0039:
0040:         Write-Host "Building the Docker image..."
0041:         docker compose build --build-arg VERSION=$VERSION --build-arg COMMIT=$COMMIT --build-arg BUILD_DATE=$BUILD_DATE
0042:
0043:         Write-Host "Starting the services..."
0044:         docker compose up -d --remove-orphans --pull never
0045:
0046:         Write-Host "Build complete. Services are starting."
0047:         Write-Host "Run 'docker compose logs -f' to see the logs."
0048:     }
0049:     default {
0050:         Write-Host "Invalid choice. Please enter 1 or 2."
0051:         exit 1
0052:     }
0053: }

### FILE: docker-build.sh
0001: #!/usr/bin/env bash
0002: #
0003: # build.sh - Linux/macOS Build Script
0004: #
0005: # This script automates the process of building and running the Docker container
0006: # with version information dynamically injected at build time.
0007:
0008: # Hidden feature: Preserve usage statistics across rebuilds
0009: # Usage: ./docker-build.sh --with-usage
0010: # First run prompts for management API key, saved to temp/stats/.api_secret
0011:
0012: set -euo pipefail
0013:
0014: STATS_DIR="temp/stats"
0015: STATS_FILE="${STATS_DIR}/.usage_backup.json"
0016: SECRET_FILE="${STATS_DIR}/.api_secret"
0017: WITH_USAGE=false
0018:
0019: get_port() {
0020:   if [[ -f "config.yaml" ]]; then
0021:     grep -E "^port:" config.yaml | sed -E 's/^port: *["'"'"']?([0-9]+)["'"'"']?.*$/\1/'
0022:   else
0023:     echo "8317"
0024:   fi
0025: }
0026:
0027: export_stats_api_secret() {
0028:   if [[ -f "${SECRET_FILE}" ]]; then
0029:     API_SECRET=$(cat "${SECRET_FILE}")
0030:   else
0031:     if [[ ! -d "${STATS_DIR}" ]]; then
0032:       mkdir -p "${STATS_DIR}"
0033:     fi
0034:     echo "First time using --with-usage. Management API key required."
0035:     read -r -p "Enter management key: " -s API_SECRET
0036:     echo
0037:     echo "${API_SECRET}" > "${SECRET_FILE}"
0038:     chmod 600 "${SECRET_FILE}"
0039:   fi
0040: }
0041:
0042: check_container_running() {
0043:   local port
0044:   port=$(get_port)
0045:
0046:   if ! curl -s -o /dev/null -w "%{http_code}" "http://localhost:${port}/" | grep -q "200"; then
0047:     echo "Error: cli-proxy-api service is not responding at localhost:${port}"
0048:     echo "Please start the container first or use without --with-usage flag."
0049:     exit 1
0050:   fi
0051: }
0052:
0053: export_stats() {
0054:   local port
0055:   port=$(get_port)
0056:
0057:   if [[ ! -d "${STATS_DIR}" ]]; then
0058:     mkdir -p "${STATS_DIR}"
0059:   fi
0060:   check_container_running
0061:   echo "Exporting usage statistics..."
0062:   EXPORT_RESPONSE=$(curl -s -w "\n%{http_code}" -H "X-Management-Key: ${API_SECRET}" \
0063:     "http://localhost:${port}/v0/management/usage/export")
0064:   HTTP_CODE=$(echo "${EXPORT_RESPONSE}" | tail -n1)
0065:   RESPONSE_BODY=$(echo "${EXPORT_RESPONSE}" | sed '$d')
0066:
0067:   if [[ "${HTTP_CODE}" != "200" ]]; then
0068:     echo "Export failed (HTTP ${HTTP_CODE}): ${RESPONSE_BODY}"
0069:     exit 1
0070:   fi
0071:
0072:   echo "${RESPONSE_BODY}" > "${STATS_FILE}"
0073:   echo "Statistics exported to ${STATS_FILE}"
0074: }
0075:
0076: import_stats() {
0077:   local port
0078:   port=$(get_port)
0079:
0080:   echo "Importing usage statistics..."
0081:   IMPORT_RESPONSE=$(curl -s -w "\n%{http_code}" -X POST \
0082:     -H "X-Management-Key: ${API_SECRET}" \
0083:     -H "Content-Type: application/json" \
0084:     -d @"${STATS_FILE}" \
0085:     "http://localhost:${port}/v0/management/usage/import")
0086:   IMPORT_CODE=$(echo "${IMPORT_RESPONSE}" | tail -n1)
0087:   IMPORT_BODY=$(echo "${IMPORT_RESPONSE}" | sed '$d')
0088:
0089:   if [[ "${IMPORT_CODE}" == "200" ]]; then
0090:     echo "Statistics imported successfully"
0091:   else
0092:     echo "Import failed (HTTP ${IMPORT_CODE}): ${IMPORT_BODY}"
0093:   fi
0094:
0095:   rm -f "${STATS_FILE}"
0096: }
0097:
0098: wait_for_service() {
0099:   local port
0100:   port=$(get_port)
0101:
0102:   echo "Waiting for service to be ready..."
0103:   for i in {1..30}; do
0104:     if curl -s -o /dev/null -w "%{http_code}" "http://localhost:${port}/" | grep -q "200"; then
0105:       break
0106:     fi
0107:     sleep 1
0108:   done
0109:   sleep 2
0110: }
0111:
0112: if [[ "${1:-}" == "--with-usage" ]]; then
0113:   WITH_USAGE=true
0114:   export_stats_api_secret
0115: fi
0116:
0117: # --- Step 1: Choose Environment ---
0118: echo "Please select an option:"
0119: echo "1) Run using Pre-built Image (Recommended)"
0120: echo "2) Build from Source and Run (For Developers)"

### FILE: docker-compose.yml
0001: services:
0002:   cli-proxy-api:
0003:     image: ${CLI_PROXY_IMAGE:-KooshaPari/cliproxyapi-plusplus:latest}
0004:     pull_policy: always
0005:     build:
0006:       context: .
0007:       dockerfile: Dockerfile
0008:       args:
0009:         VERSION: ${VERSION:-dev}
0010:         COMMIT: ${COMMIT:-none}
0011:         BUILD_DATE: ${BUILD_DATE:-unknown}
0012:     container_name: cliproxyapi++
0013:     # env_file:
0014:     #   - .env
0015:     environment:
0016:       DEPLOY: ${DEPLOY:-}
0017:     ports:
0018:       - "8317:8317"
0019:       - "8085:8085"
0020:       - "1455:1455"
0021:       - "54545:54545"
0022:       - "51121:51121"
0023:       - "11451:11451"
0024:     volumes:
0025:       - ${CLI_PROXY_CONFIG_PATH:-./config.yaml}:/CLIProxyAPI/config.yaml
0026:       - ${CLI_PROXY_AUTH_PATH:-./auths}:/root/.cli-proxy-api
0027:       - ${CLI_PROXY_LOG_PATH:-./logs}:/CLIProxyAPI/logs
0028:     restart: unless-stopped

### FILE: docs/.vitepress/config.ts
0001: import { defineConfig } from "vitepress";
0002:
0003: const repo = process.env.GITHUB_REPOSITORY?.split("/")[1] ?? "cliproxyapi-plusplus";
0004: const isCI = process.env.GITHUB_ACTIONS === "true";
0005:
0006: export default defineConfig({
0007:   title: "cliproxy++",
0008:   description: "cliproxyapi-plusplus documentation",
0009:   base: isCI ? `/${repo}/` : "/",
0010:   cleanUrls: true,
0011:   ignoreDeadLinks: true,
0012:   themeConfig: {
0013:     nav: [
0014:       { text: "Home", link: "/" },
0015:       { text: "API", link: "/api/" },
0016:       { text: "Features", link: "/features/" }
0017:     ],
0018:     socialLinks: [
0019:       { icon: "github", link: "https://github.com/kooshapari/cliproxyapi-plusplus" }
0020:     ]
0021:   }
0022: });

### FILE: docs/FEATURE_CHANGES_PLUSPLUS.md
0001: # cliproxyapi++ Feature Change Reference (`++` vs baseline)
0002:
0003: This document explains what changed in `cliproxyapi++`, why it changed, and how it affects users, integrators, and maintainers.
0004:
0005: ## 1. Architecture Changes
0006:
0007: | Change | What changed in `++` | Why it matters |
0008: |---|---|---|
0009: | Reusable proxy core | Translation and proxy runtime are structured for reusability (`pkg/llmproxy`) | Enables embedding proxy logic into other Go systems and keeps runtime boundaries cleaner |
0010: | Stronger module boundaries | Operational and integration concerns are separated from API surface orchestration | Easier upgrades, clearer ownership, lower accidental coupling |
0011:
0012: ## 2. Authentication and Identity Changes
0013:
0014: | Change | What changed in `++` | Why it matters |
0015: |---|---|---|
0016: | Copilot-grade auth support | Extended auth handling for enterprise Copilot-style workflows | More stable integration for organizations depending on tokenized auth stacks |
0017: | Kiro/AWS login path support | Additional OAuth/login handling pathways and operational UX around auth | Better compatibility for multi-provider enterprise environments |
0018: | Token lifecycle automation | Background refresh and expiration handling | Reduces downtime from token expiry and manual auth recovery |
0019:
0020: ## 3. Provider and Model Routing Changes
0021:
0022: | Change | What changed in `++` | Why it matters |
0023: |---|---|---|
0024: | Broader provider matrix | Expanded provider adapter and model mapping surfaces | More routing options without changing client-side OpenAI API integrations |
0025: | Unified model translation | Stronger mapping between OpenAI-style model requests and provider-native model names | Lower integration friction and fewer provider mismatch errors |
0026: | Cooldown and throttling controls | Runtime controls for rate-limit pressure and provider-specific cooldown windows | Better stability under burst traffic and quota pressure |
0027:
0028: ## 4. Security and Governance Changes
0029:
0030: | Change | What changed in `++` | Why it matters |
0031: |---|---|---|
0032: | Defense-in-depth hardening | Added stricter operational defaults and hardened deployment assumptions | Safer default posture in production environments |
0033: | Protected core path governance | Workflow-level controls around critical core logic paths | Reduces accidental regressions in proxy translation internals |
0034: | Device and session consistency controls | Deterministic identity/session behavior for strict provider checks | Fewer auth anomalies in long-running deployments |
0035:
0036: ## 5. Operations and Delivery Changes
0037:
0038: | Change | What changed in `++` | Why it matters |
0039: |---|---|---|
0040: | Stronger CI/CD posture | Expanded release, build, and guard workflows | Faster detection of regressions and safer release cadence |
0041: | Multi-arch/container focus | Production deployment paths optimized for container-first ops | Better portability across heterogeneous infra |
0042: | Runtime observability surfaces | Improved log and management endpoints | Easier production debugging and incident response |
0043:
0044: ## 6. API and Compatibility Surface
0045:
0046: | Change | What changed in `++` | Why it matters |
0047: |---|---|---|
0048: | OpenAI-compatible core retained | `/v1/chat/completions` and `/v1/models` compatibility maintained | Existing OpenAI-style clients can migrate with minimal API churn |
0049: | Expanded management endpoints | Added operational surfaces for config/auth/runtime introspection | Better operations UX without changing core client API |
0050:
0051: ## 7. Migration Impact Summary
0052:
0053: - **Technical users**: gain higher operational stability, better auth longevity, and stronger multi-provider behavior.
0054: - **External integrators**: keep OpenAI-compatible interfaces while gaining wider provider compatibility.
0055: - **Internal maintainers**: get cleaner subsystem boundaries and stronger guardrails for production evolution.

### FILE: docs/README.md
0001: # cliproxyapi++ Documentation Index
0002:
0003: Welcome to the comprehensive documentation for **cliproxyapi++**, the definitive high-performance, security-hardened fork of CLIProxyAPI.
0004:
0005: ## 📚 Documentation Structure
0006:
0007: This documentation is organized into docsets for each major feature area, with three types of documentation for each:
0008:
0009: - **SPEC.md** - Technical specifications for developers and contributors
0010: - **USER.md** - User guides for operators and developers using the system
0011: - **DEV.md** - Developer guides for extending and customizing the system
0012:
0013: ## 🚀 Quick Start
0014:
0015: **New to cliproxyapi++?** Start here:
0016: - [Main README](../README.md) - Project overview and quick start
0017: - [Getting Started](#getting-started) - Basic setup and first request
0018:
0019: **Using as a library?** See:
0020: - [Library-First Architecture](features/architecture/USER.md) - Embedding in your Go app
0021:
0022: **Deploying to production?** See:
0023: - [Security Hardening](features/security/USER.md) - Security best practices
0024: - [High-Scale Operations](features/operations/USER.md) - Production deployment guide
0025:
0026: ## 📖 Feature Documentation
0027:
0028: ### 1. Library-First Architecture
0029:
0030: **Overview**: The core proxy logic is packaged as a reusable Go library (`pkg/llmproxy`), enabling external Go applications to embed translation, authentication, and provider communication directly.
0031:
0032: - **[Technical Spec](features/architecture/SPEC.md)** - Architecture design, component breakdown, data flows
0033: - **[User Guide](features/architecture/USER.md)** - Quick start, embedding, custom translators
0034: - **[Developer Guide](features/architecture/DEV.md)** - Adding providers, implementing auth flows, performance optimization
0035:
0036: **Key Features**:
0037: - Reusable `pkg/llmproxy` library
0038: - Hot-reload configuration management
0039: - Background token refresh worker
0040: - Custom auth flow support
0041: - Extension points for customization
0042:
0043: ### 2. Enterprise Authentication
0044:
0045: **Overview**: Enterprise-grade authentication management with full lifecycle automation, supporting multiple authentication flows (API keys, OAuth, device authorization).
0046:
0047: - **[Technical Spec](features/auth/SPEC.md)** - Auth architecture, flow implementations, token refresh
0048: - **[User Guide](features/auth/USER.md)** - Adding credentials, multi-credential management, quota tracking
0049:
0050: **Key Features**:
0051: - API key, OAuth 2.0, and device authorization flows
0052: - Automatic token refresh (10 minutes before expiration)
0053: - Multi-credential support with load balancing
0054: - Per-credential quota tracking and rotation
0055: - Encrypted credential storage (optional)
0056:
0057: ### 3. Security Hardening
0058:
0059: **Overview**: "Defense in Depth" security philosophy with multiple layers of protection.
0060:
0061: - **[Technical Spec](features/security/SPEC.md)** - Security architecture, CI enforcement, container hardening
0062: - **[User Guide](features/security/USER.md)** - TLS configuration, encryption, IP filtering, monitoring
0063:
0064: **Key Features**:
0065: - Path Guard CI enforcement for critical code
0066: - Signed releases and multi-arch builds
0067: - Hardened Docker containers (Alpine 3.22.0, non-root, read-only)
0068: - Credential encryption at rest
0069: - Device fingerprinting
0070: - IP allowlisting/denylisting
0071: - Comprehensive audit logging
0072:
0073: ### 4. High-Scale Operations
0074:
0075: **Overview**: Intelligent operations features for production environments.
0076:
0077: - **[Technical Spec](features/operations/SPEC.md)** - Operations architecture, load balancing strategies, health monitoring
0078: - **[User Guide](features/operations/USER.md)** - Production deployment, cooldown management, observability
0079:
0080: **Key Features**:
0081: - Intelligent cooldown (automatic rate limit detection)
0082: - Multiple load balancing strategies (round-robin, quota-aware, latency, cost)
0083: - Provider health checks and self-healing
0084: - Comprehensive metrics (Prometheus)
0085: - Structured logging and distributed tracing
0086: - Alerting and notifications
0087:
0088: ### 5. Provider Registry
0089:
0090: **Overview**: Extensive registry of LLM providers.
0091:
0092: - **[Technical Spec](features/providers/SPEC.md)** - Provider architecture, registry implementation, model mapping
0093: - **[User Guide](features/providers/USER.md)** - Provider configuration, usage examples, troubleshooting
0094:
0095: **Supported Providers**:
0096: - **Direct**: Claude, Gemini, OpenAI, Mistral, Groq, DeepSeek
0097: - **Aggregators**: OpenRouter, Together AI, Fireworks AI, Novita AI, SiliconFlow
0098: - **Proprietary**: Kiro (AWS CodeWhisperer), GitHub Copilot, Roo Code, Kilo AI, MiniMax
0099:
0100: ## 🔧 API Documentation
0101:
0102: ### OpenAI-Compatible API
0103:
0104: **Endpoints**:
0105: - `POST /v1/chat/completions` - Chat completions (streaming and non-streaming)
0106: - `GET /v1/models` - List available models
0107: - `POST /v1/embeddings` - Generate embeddings
0108:
0109: See [API Reference](api/README.md) for complete API documentation.
0110:
0111: ### Management API
0112:
0113: **Endpoints**:
0114: - `GET /v0/management/config` - Inspect current configuration
0115: - `GET /v0/management/auths` - List all credentials
0116: - `POST /v0/management/auths` - Add credential
0117: - `DELETE /v0/management/auths/{provider}` - Remove credential
0118: - `POST /v0/management/auths/{provider}/refresh` - Refresh credential
0119: - `GET /v0/management/logs` - Real-time log inspection
0120:
0121: See [Management API](api/management.md) for complete documentation.
0122:
0123: ### Operations API
0124:
0125: **Endpoints**:
0126: - `GET /health` - Health check
0127: - `GET /metrics` - Prometheus metrics
0128: - `GET /v0/operations/providers/status` - Provider status
0129: - `GET /v0/operations/cooldown/status` - Cooldown status
0130: - `POST /v0/operations/providers/{provider}/recover` - Force recovery
0131:
0132: See [Operations API](api/operations.md) for complete documentation.
0133:
0134: ## 🛠️ SDK Documentation
0135:
0136: ### Go SDK
0137:
0138: **Embedding in Go applications**:
0139: - [SDK Usage](../docs/sdk-usage.md) - Basic embedding
0140: - [SDK Advanced](../docs/sdk-access.md) - Advanced configuration
0141: - [SDK Watcher](../docs/sdk-watcher.md) - Hot-reload and synthesis
0142:
0143: **Code Examples**:
0144: ```go
0145: import "github.com/KooshaPari/cliproxyapi-plusplus/sdk/cliproxy"
0146:
0147: svc, err := cliproxy.NewBuilder().
0148:     WithConfig(cfg).
0149:     WithConfigPath("config.yaml").
0150:     Build()
0151:
0152: ctx := context.Background()
0153: svc.Run(ctx)
0154: ```
0155:
0156: ## 🚀 Getting Started
0157:
0158: ### 1. Installation
0159:
0160: **Docker (Recommended)**:
0161: ```bash
0162: docker pull KooshaPari/cliproxyapi-plusplus:latest
0163: ```
0164:
0165: **Binary**:
0166: ```bash
0167: curl -L https://github.com/KooshaPari/cliproxyapi-plusplus/releases/latest/download/cliproxyapi++-darwin-amd64 -o cliproxyapi++
0168: chmod +x cliproxyapi++
0169: ```
0170:
0171: **Go Module**:
0172: ```bash
0173: go get github.com/KooshaPari/cliproxyapi-plusplus/sdk/cliproxy
0174: ```
0175:
0176: ### 2. Configuration
0177:
0178: Create `config.yaml`:
0179: ```yaml
0180: server:
0181:   port: 8317
0182:
0183: providers:
0184:   claude:
0185:     type: "claude"
0186:     enabled: true
0187:
0188: auth:
0189:   dir: "./auths"
0190:   providers:
0191:     - "claude"
0192: ```
0193:
0194: ### 3. Add Credentials
0195:
0196: ```bash
0197: echo '{"type":"api_key","token":"sk-ant-xxxxx"}' > auths/claude.json
0198: ```
0199:
0200: ### 4. Start Service
0201:
0202: **Docker**:
0203: ```bash
0204: docker run -d \
0205:   -p 8317:8317 \
0206:   -v $(pwd)/config.yaml:/config/config.yaml \
0207:   -v $(pwd)/auths:/auths \
0208:   KooshaPari/cliproxyapi-plusplus:latest
0209: ```
0210:
0211: **Binary**:
0212: ```bash
0213: ./cliproxyapi++ --config config.yaml
0214: ```
0215:
0216: ### 5. Make Request
0217:
0218: ```bash
0219: curl -X POST http://localhost:8317/v1/chat/completions \
0220:   -H "Content-Type: application/json" \

### FILE: docs/docsets/agent/index.md
0001: # Agent Operator Docset
0002:
0003: For teams routing agent workloads through cliproxyapi++.
0004:
0005: ## Operator Focus
0006:
0007: 1. [Operating Model](./operating-model.md)
0008: 2. Multi-provider routing and quota management
0009: 3. Auth lifecycle and refresh controls

### FILE: docs/docsets/agent/operating-model.md
0001: # Agent Operating Model
0002:
0003: ## Execution Loop
0004:
0005: 1. Route request into OpenAI-compatible API surface.
0006: 2. Resolve provider/model translation and auth context.
0007: 3. Execute request with quotas, cooldown, and resilience controls.
0008: 4. Emit structured logs and monitoring signals.

### FILE: docs/docsets/developer/external/index.md
0001: # External Developer Docset
0002:
0003: For engineers embedding cliproxyapi++ into their own systems.
0004:
0005: ## Start Here
0006:
0007: 1. [Integration Quickstart](./integration-quickstart.md)
0008: 2. [Feature Change Reference](../../FEATURE_CHANGES_PLUSPLUS.md)
0009: 3. Core docs in `docs/README.md`, `docs/api/`, and `docs/features/`

### FILE: docs/docsets/developer/external/integration-quickstart.md
0001: # Integration Quickstart
0002:
0003: 1. Start cliproxyapi++ with config and auth storage.
0004: 2. Point OpenAI-compatible clients to proxy `/v1` endpoints.
0005: 3. Validate provider model mapping and fallback behavior.
0006: 4. Add health and quota observability to your platform stack.

### FILE: docs/docsets/developer/internal/architecture.md
0001: # Internal Architecture
0002:
0003: ## Core Boundaries
0004:
0005: 1. API entrypoint and command bootstrap (`cmd/`)
0006: 2. Proxy core and reusable translation runtime (`pkg/llmproxy`)
0007: 3. Authentication and provider adapters
0008: 4. Operational surfaces (config, auth state, logs)
0009:
0010: ## Maintainer Rules
0011:
0012: - Keep translation logic deterministic.
0013: - Preserve OpenAI-compatible API behavior.
0014: - Enforce path and security governance gates.

### FILE: docs/docsets/developer/internal/index.md
0001: # Internal Developer Docset
0002:
0003: For maintainers of cliproxyapi++ internals.
0004:
0005: ## Read First
0006:
0007: 1. [Internal Architecture](./architecture.md)
0008: 2. [Feature Changes in ++](../../FEATURE_CHANGES_PLUSPLUS.md)
0009: 3. `pkg/` and `cmd/` source directories
0010: 4. CI/CD workflows under `.github/workflows/`

### FILE: docs/docsets/index.md
0001: # Docsets
0002:
0003: Audience-specific docs for cliproxyapi++.
0004:
0005: ## Developer
0006:
0007: - [Internal Developer Docset](./developer/internal/)
0008: - [External Developer Docset](./developer/external/)
0009:
0010: ## User
0011:
0012: - [Technical User Docset](./user/)
0013:
0014: ## Agent
0015:
0016: - [Agent Operator Docset](./agent/)

### FILE: docs/docsets/user/index.md
0001: # Technical User Docset
0002:
0003: For operators and technical users running cliproxyapi++.
0004:
0005: ## Core Paths
0006:
0007: 1. [Quickstart](./quickstart.md)
0008: 2. Auth and provider setup docs
0009: 3. Runtime and troubleshooting docs

### FILE: docs/docsets/user/quickstart.md
0001: # Technical User Quickstart
0002:
0003: 1. Configure `config.yaml` from the example.
0004: 2. Start service with Docker or native binary.
0005: 3. Validate `GET /v1/models` and sample chat completions.
0006: 4. Monitor rate limits and provider-specific auth state.

### FILE: docs/features/architecture/DEV.md
0001: # Developer Guide: Extending Library-First Architecture
0002:
0003: ## Contributing to pkg/llmproxy
0004:
0005: This guide is for developers who want to extend the core library functionality: adding new providers, customizing translators, implementing new authentication flows, or optimizing performance.
0006:
0007: ## Project Structure
0008:
0009: ```
0010: pkg/llmproxy/
0011: ├── translator/       # Protocol translation layer
0012: │   ├── base.go       # Common interfaces and utilities
0013: │   ├── claude.go     # Anthropic Claude
0014: │   ├── gemini.go     # Google Gemini
0015: │   ├── openai.go     # OpenAI GPT
0016: │   ├── kiro.go       # AWS CodeWhisperer
0017: │   ├── copilot.go    # GitHub Copilot
0018: │   └── aggregators.go # Multi-provider aggregators
0019: ├── provider/         # Provider execution layer
0020: │   ├── base.go       # Provider interface and executor
0021: │   ├── http.go       # HTTP client with retry logic
0022: │   ├── rate_limit.go # Token bucket implementation
0023: │   └── health.go     # Health check logic
0024: ├── auth/             # Authentication lifecycle
0025: │   ├── manager.go    # Core auth manager
0026: │   ├── oauth.go      # OAuth flows
0027: │   ├── device_flow.go # Device authorization flow
0028: │   └── refresh.go    # Token refresh worker
0029: ├── config/           # Configuration management
0030: │   ├── loader.go     # Config file parsing
0031: │   ├── schema.go     # Validation schema
0032: │   └── synthesis.go  # Config merge logic
0033: ├── watcher/          # Dynamic reload orchestration
0034: │   ├── file.go       # File system watcher
0035: │   ├── debounce.go   # Debouncing logic
0036: │   └── notify.go     # Change notifications
0037: └── metrics/          # Observability
0038:     ├── collector.go  # Metrics collection
0039:     └── exporter.go   # Metrics export
0040: ```
0041:
0042: ## Adding a New Provider
0043:
0044: ### Step 1: Define Provider Configuration
0045:
0046: Add provider config to `config/schema.go`:
0047:
0048: ```go
0049: type ProviderConfig struct {
0050:     Type        string   `yaml:"type" validate:"required,oneof=claude gemini openai kiro copilot myprovider"`
0051:     Enabled     bool     `yaml:"enabled"`
0052:     Models      []ModelConfig `yaml:"models"`
0053:     AuthType    string   `yaml:"auth_type" validate:"required,oneof=api_key oauth device_flow"`
0054:     Priority    int      `yaml:"priority"`
0055:     Cooldown    time.Duration `yaml:"cooldown"`
0056:     Endpoint    string   `yaml:"endpoint"`
0057:     // Provider-specific fields
0058:     CustomField string   `yaml:"custom_field"`
0059: }
0060: ```
0061:
0062: ### Step 2: Implement Translator Interface
0063:
0064: Create `pkg/llmproxy/translator/myprovider.go`:
0065:
0066: ```go
0067: package translator
0068:
0069: import (
0070:     "context"
0071:     "encoding/json"
0072:
0073:     openai "github.com/sashabaranov/go-openai"
0074:     "github.com/KooshaPari/cliproxyapi-plusplus/pkg/llmproxy"
0075: )
0076:
0077: type MyProviderTranslator struct {
0078:     config *config.ProviderConfig
0079: }
0080:
0081: func NewMyProviderTranslator(cfg *config.ProviderConfig) *MyProviderTranslator {
0082:     return &MyProviderTranslator{config: cfg}
0083: }
0084:
0085: func (t *MyProviderTranslator) TranslateRequest(
0086:     ctx context.Context,
0087:     req *openai.ChatCompletionRequest,
0088: ) (*llmproxy.ProviderRequest, error) {
0089:     // Map OpenAI models to provider models
0090:     modelMapping := map[string]string{
0091:         "gpt-4": "myprovider-v1-large",
0092:         "gpt-3.5-turbo": "myprovider-v1-medium",
0093:     }
0094:     providerModel := modelMapping[req.Model]
0095:     if providerModel == "" {
0096:         providerModel = req.Model
0097:     }
0098:
0099:     // Convert messages
0100:     messages := make([]map[string]interface{}, len(req.Messages))
0101:     for i, msg := range req.Messages {
0102:         messages[i] = map[string]interface{}{
0103:             "role":    msg.Role,
0104:             "content": msg.Content,
0105:         }
0106:     }
0107:
0108:     // Build request
0109:     providerReq := &llmproxy.ProviderRequest{
0110:         Method: "POST",
0111:         Endpoint: t.config.Endpoint + "/v1/chat/completions",
0112:         Headers: map[string]string{
0113:             "Content-Type": "application/json",
0114:             "Accept": "application/json",
0115:         },
0116:         Body: map[string]interface{}{
0117:             "model":    providerModel,
0118:             "messages": messages,
0119:             "stream":   req.Stream,
0120:         },
0121:     }
0122:
0123:     // Add optional parameters
0124:     if req.Temperature != 0 {
0125:         providerReq.Body["temperature"] = req.Temperature
0126:     }
0127:     if req.MaxTokens != 0 {
0128:         providerReq.Body["max_tokens"] = req.MaxTokens
0129:     }
0130:
0131:     return providerReq, nil
0132: }
0133:
0134: func (t *MyProviderTranslator) TranslateResponse(
0135:     ctx context.Context,
0136:     resp *llmproxy.ProviderResponse,
0137: ) (*openai.ChatCompletionResponse, error) {
0138:     // Parse provider response
0139:     var providerBody struct {
0140:         ID      string `json:"id"`
0141:         Model   string `json:"model"`
0142:         Choices []struct {
0143:             Message struct {
0144:                 Role    string `json:"role"`
0145:                 Content string `json:"content"`
0146:             } `json:"message"`
0147:             FinishReason string `json:"finish_reason"`
0148:         } `json:"choices"`
0149:         Usage struct {
0150:             PromptTokens     int `json:"prompt_tokens"`
0151:             CompletionTokens int `json:"completion_tokens"`
0152:             TotalTokens      int `json:"total_tokens"`
0153:         } `json:"usage"`
0154:     }
0155:
0156:     if err := json.Unmarshal(resp.Body, &providerBody); err != nil {
0157:         return nil, fmt.Errorf("failed to parse provider response: %w", err)
0158:     }
0159:
0160:     // Convert to OpenAI format
0161:     choices := make([]openai.ChatCompletionChoice, len(providerBody.Choices))
0162:     for i, choice := range providerBody.Choices {
0163:         choices[i] = openai.ChatCompletionChoice{
0164:             Message: openai.ChatCompletionMessage{
0165:                 Role:    openai.ChatMessageRole(choice.Message.Role),
0166:                 Content: choice.Message.Content,
0167:             },
0168:             FinishReason: openai.FinishReason(choice.FinishReason),
0169:         }
0170:     }
0171:
0172:     return &openai.ChatCompletionResponse{
0173:         ID:      providerBody.ID,
0174:         Model:   resp.RequestModel,
0175:         Choices: choices,
0176:         Usage: openai.Usage{
0177:             PromptTokens:     providerBody.Usage.PromptTokens,
0178:             CompletionTokens: providerBody.Usage.CompletionTokens,
0179:             TotalTokens:      providerBody.Usage.TotalTokens,
0180:         },
0181:     }, nil
0182: }
0183:
0184: func (t *MyProviderTranslator) TranslateStream(
0185:     ctx context.Context,
0186:     stream io.Reader,
0187: ) (<-chan *openai.ChatCompletionStreamResponse, error) {
0188:     // Implement streaming translation
0189:     ch := make(chan *openai.ChatCompletionStreamResponse)
0190:
0191:     go func() {
0192:         defer close(ch)
0193:
0194:         scanner := bufio.NewScanner(stream)
0195:         for scanner.Scan() {
0196:             line := scanner.Text()
0197:             if !strings.HasPrefix(line, "data: ") {
0198:                 continue
0199:             }
0200:
0201:             data := strings.TrimPrefix(line, "data: ")
0202:             if data == "[DONE]" {
0203:                 return
0204:             }
0205:
0206:             var chunk struct {
0207:                 ID      string `json:"id"`
0208:                 Choices []struct {
0209:                     Delta struct {
0210:                         Content string `json:"content"`
0211:                     } `json:"delta"`
0212:                     FinishReason *string `json:"finish_reason"`
0213:                 } `json:"choices"`
0214:             }
0215:
0216:             if err := json.Unmarshal([]byte(data), &chunk); err != nil {
0217:                 continue
0218:             }
0219:
0220:             ch <- &openai.ChatCompletionStreamResponse{

### FILE: docs/features/architecture/SPEC.md
0001: # Technical Specification: Library-First Architecture (pkg/llmproxy)
0002:
0003: ## Overview
0004:
0005: **cliproxyapi++** implements a "Library-First" architectural pattern by extracting all core proxy logic from the traditional `internal/` package into a public, reusable `pkg/llmproxy` module. This transformation enables external Go applications to import and embed the entire translation, authentication, and communication engine without depending on the CLI binary.
0006:
0007: ## Architecture Migration
0008:
0009: ### Before: Mainline Structure
0010: ```
0011: CLIProxyAPI/
0012: ├── internal/
0013: │   ├── translator/      # Core translation logic (NOT IMPORTABLE)
0014: │   ├── provider/        # Provider executors (NOT IMPORTABLE)
0015: │   └── auth/            # Auth management (NOT IMPORTABLE)
0016: └── cmd/server/
0017: ```
0018:
0019: ### After: cliproxyapi++ Structure
0020: ```
0021: cliproxyapi++/
0022: ├── pkg/llmproxy/         # PUBLIC LIBRARY (IMPORTABLE)
0023: │   ├── translator/       # Translation engine
0024: │   ├── provider/         # Provider implementations
0025: │   ├── config/           # Configuration synthesis
0026: │   ├── watcher/          # Dynamic reload orchestration
0027: │   └── auth/             # Auth lifecycle management
0028: ├── cmd/server/          # CLI entry point (uses pkg/llmproxy)
0029: └── sdk/cliproxy/        # High-level embedding SDK
0030: ```
0031:
0032: ## Core Components
0033:
0034: ### 1. Translation Engine (`pkg/llmproxy/translator`)
0035:
0036: **Purpose**: Handles bidirectional protocol conversion between OpenAI-compatible requests and proprietary LLM APIs.
0037:
0038: **Key Interfaces**:
0039: ```go
0040: type Translator interface {
0041:     // Convert OpenAI format to provider format
0042:     TranslateRequest(ctx context.Context, req *openai.ChatRequest) (*ProviderRequest, error)
0043:
0044:     // Convert provider response back to OpenAI format
0045:     TranslateResponse(ctx context.Context, resp *ProviderResponse) (*openai.ChatResponse, error)
0046:
0047:     // Stream translation for SSE
0048:     TranslateStream(ctx context.Context, stream io.Reader) (<-chan *openai.ChatChunk, error)
0049:
0050:     // Provider-specific capabilities
0051:     SupportsStreaming() bool
0052:     SupportsFunctions() bool
0053:     MaxTokens() int
0054: }
0055: ```
0056:
0057: **Implemented Translators**:
0058: - `claude.go` - Anthropic Claude API
0059: - `gemini.go` - Google Gemini API
0060: - `openai.go` - OpenAI GPT API
0061: - `kiro.go` - AWS CodeWhisperer (custom protocol)
0062: - `copilot.go` - GitHub Copilot (custom protocol)
0063: - `aggregators.go` - OpenRouter, Together, Fireworks
0064:
0065: **Translation Strategy**:
0066: 1. **Request Normalization**: Parse OpenAI-format request, extract:
0067:    - Messages (system, user, assistant)
0068:    - Tools/functions
0069:    - Generation parameters (temp, top_p, max_tokens)
0070:    - Streaming flag
0071:
0072: 2. **Provider Mapping**: Map OpenAI models to provider endpoints:
0073:    ```
0074:    claude-3-5-sonnet -> claude-3-5-sonnet-20241022 (Anthropic)
0075:    gpt-4 -> gpt-4-turbo-preview (OpenAI)
0076:    gemini-1.5-pro -> gemini-1.5-pro-preview-0514 (Gemini)
0077:    ```
0078:
0079: 3. **Response Normalization**: Convert provider responses to OpenAI format:
0080:    - Standardize usage statistics (prompt_tokens, completion_tokens)
0081:    - Normalize finish reasons (stop, length, content_filter)
0082:    - Map provider-specific error codes to OpenAI error types
0083:
0084: ### 2. Provider Execution (`pkg/llmproxy/provider`)
0085:
0086: **Purpose**: Orchestrates HTTP communication with LLM providers, handling authentication, retry logic, and error recovery.
0087:
0088: **Key Interfaces**:
0089: ```go
0090: type ProviderExecutor interface {
0091:     // Execute a single request (non-streaming)
0092:     Execute(ctx context.Context, auth coreauth.Auth, req *ProviderRequest) (*ProviderResponse, error)
0093:
0094:     // Execute streaming request
0095:     ExecuteStream(ctx context.Context, auth coreauth.Auth, req *ProviderRequest) (<-chan *ProviderChunk, error)
0096:
0097:     // Health check provider
0098:     HealthCheck(ctx context.Context, auth coreauth.Auth) error
0099:
0100:     // Provider metadata
0101:     Name() string
0102:     SupportsModel(model string) bool
0103: }
0104: ```
0105:
0106: **Executor Lifecycle**:
0107: ```
0108: Request -> RateLimitCheck -> AuthValidate -> ProviderExecute ->
0109:     -> Success -> Response
0110:     -> RetryableError -> Backoff -> Retry
0111:     -> NonRetryableError -> Error
0112: ```
0113:
0114: **Rate Limiting**:
0115: - Per-provider token bucket
0116: - Per-credential quota tracking
0117: - Intelligent cooldown on 429 responses
0118:
0119: ### 3. Configuration Management (`pkg/llmproxy/config`)
0120:
0121: **Purpose**: Loads, validates, and synthesizes configuration from multiple sources.
0122:
0123: **Configuration Hierarchy**:
0124: ```
0125: 1. Base config (config.yaml)
0126: 2. Environment overrides (CLI_PROXY_*)
0127: 3. Runtime synthesis (watcher merges changes)
0128: 4. Per-request overrides (query params)
0129: ```
0130:
0131: **Key Structures**:
0132: ```go
0133: type Config struct {
0134:     Server      ServerConfig
0135:     Providers   map[string]ProviderConfig
0136:     Auth        AuthConfig
0137:     Management  ManagementConfig
0138:     Logging     LoggingConfig
0139: }
0140:
0141: type ProviderConfig struct {
0142:     Type        string  // "claude", "gemini", "openai", etc.
0143:     Enabled     bool
0144:     Models      []ModelConfig
0145:     AuthType    string  // "api_key", "oauth", "device_flow"
0146:     Priority    int     // Routing priority
0147:     Cooldown    time.Duration
0148: }
0149: ```
0150:
0151: **Hot-Reload Mechanism**:
0152: - File watcher on `config.yaml` and `auths/` directory
0153: - Debounced reload (500ms delay)
0154: - Atomic config swapping (no request interruption)
0155: - Validation before activation (reject invalid configs)
0156:
0157: ### 4. Watcher & Synthesis (`pkg/llmproxy/watcher`)
0158:
0159: **Purpose**: Orchestrates dynamic configuration updates and background lifecycle management.
0160:
0161: **Watcher Architecture**:
0162: ```go
0163: type Watcher struct {
0164:     configPath     string
0165:     authDir        string
0166:     reloadChan     chan struct{}
0167:     currentConfig  atomic.Value // *Config
0168:     currentAuths   atomic.Value // []coreauth.Auth
0169: }
0170:
0171: // Run starts the watcher goroutine
0172: func (w *Watcher) Run(ctx context.Context) error {
0173:     // 1. Initial load
0174:     w.loadAll()
0175:
0176:     // 2. Watch files
0177:     go w.watchConfig(ctx)
0178:     go w.watchAuths(ctx)
0179:
0180:     // 3. Handle reloads
0181:     for {
0182:         select {
0183:         case <-w.reloadChan:
0184:             w.loadAll()
0185:         case <-ctx.Done():
0186:             return ctx.Err()
0187:         }
0188:     }
0189: }
0190: ```
0191:
0192: **Synthesis Pipeline**:
0193: ```
0194: Config File Changed -> Parse YAML -> Validate Schema ->
0195:     Merge with Existing -> Check Conflicts -> Atomic Swap
0196: ```
0197:
0198: **Background Workers**:
0199: 1. **Token Refresh Worker**: Checks every 5 minutes, refreshes tokens expiring within 10 minutes
0200: 2. **Health Check Worker**: Pings providers every 30 seconds, marks unhealthy providers
0201: 3. **Metrics Collector**: Aggregates request latency, error rates, token usage
0202:
0203: ## Data Flow
0204:
0205: ### Request Processing Flow
0206: ```
0207: HTTP Request (OpenAI format)
0208:     ↓
0209: Middleware (CORS, auth, logging)
0210:     ↓
0211: Handler (Parse request, select provider)
0212:     ↓
0213: Provider Executor (Rate limit check)
0214:     ↓
0215: Translator (Convert to provider format)
0216:     ↓
0217: HTTP Client (Execute provider API)
0218:     ↓
0219: Translator (Convert response)
0220:     ↓

### FILE: docs/features/architecture/USER.md
0001: # User Guide: Library-First Architecture
0002:
0003: ## What is "Library-First"?
0004:
0005: The **Library-First** architecture means that all the core proxy logic (translation, authentication, provider communication) is packaged as a reusable Go library (`pkg/llmproxy`). This allows you to embed the proxy directly into your own applications instead of running it as a separate service.
0006:
0007: ## Why Use the Library?
0008:
0009: ### Benefits Over Standalone CLI
0010:
0011: | Aspect | Standalone CLI | Embedded Library |
0012: |--------|---------------|------------------|
0013: | **Deployment** | Separate process, network calls | In-process, zero network overhead |
0014: | **Configuration** | External config file | Programmatic config |
0015: | **Customization** | Limited to config options | Full code access |
0016: | **Performance** | Network latency + serialization | Direct function calls |
0017: | **Monitoring** | External metrics/logs | Internal hooks/observability |
0018:
0019: ### When to Use Each
0020:
0021: **Use Standalone CLI when**:
0022: - You want a simple, drop-in proxy
0023: - You're integrating with existing OpenAI clients
0024: - You don't need custom logic
0025: - You prefer configuration over code
0026:
0027: **Use Embedded Library when**:
0028: - You're building a Go application
0029: - You need custom request/response processing
0030: - You want to integrate with your auth system
0031: - You need fine-grained control over routing
0032:
0033: ## Quick Start: Embedding in Your App
0034:
0035: ### Step 1: Install the SDK
0036:
0037: ```bash
0038: go get github.com/KooshaPari/cliproxyapi-plusplus/sdk/cliproxy
0039: ```
0040:
0041: ### Step 2: Basic Embedding
0042:
0043: Create `main.go`:
0044:
0045: ```go
0046: package main
0047:
0048: import (
0049:     "context"
0050:     "log"
0051:
0052:     "github.com/KooshaPari/cliproxyapi-plusplus/pkg/llmproxy/config"
0053:     "github.com/KooshaPari/cliproxyapi-plusplus/sdk/cliproxy"
0054: )
0055:
0056: func main() {
0057:     // Load config
0058:     cfg, err := config.LoadConfig("config.yaml")
0059:     if err != nil {
0060:         log.Fatalf("Failed to load config: %v", err)
0061:     }
0062:
0063:     // Build service
0064:     svc, err := cliproxy.NewBuilder().
0065:         WithConfig(cfg).
0066:         WithConfigPath("config.yaml").
0067:         Build()
0068:     if err != nil {
0069:         log.Fatalf("Failed to build service: %v", err)
0070:     }
0071:
0072:     // Run service
0073:     ctx := context.Background()
0074:     if err := svc.Run(ctx); err != nil {
0075:         log.Fatalf("Service error: %v", err)
0076:     }
0077: }
0078: ```
0079:
0080: ### Step 3: Create Config File
0081:
0082: Create `config.yaml`:
0083:
0084: ```yaml
0085: server:
0086:   port: 8317
0087:
0088: providers:
0089:   claude:
0090:     type: "claude"
0091:     enabled: true
0092:     models:
0093:       - name: "claude-3-5-sonnet"
0094:         enabled: true
0095:
0096: auth:
0097:   dir: "./auths"
0098:   providers:
0099:     - "claude"
0100: ```
0101:
0102: ### Step 4: Run Your App
0103:
0104: ```bash
0105: # Add your Claude API key
0106: echo '{"type":"api_key","token":"sk-ant-xxx"}' > auths/claude.json
0107:
0108: # Run your app
0109: go run main.go
0110: ```
0111:
0112: Your embedded proxy is now running on port 8317 with OpenAI-compatible endpoints!
0113:
0114: ## Advanced: Custom Translators
0115:
0116: If you need to support a custom LLM provider, you can implement your own translator:
0117:
0118: ```go
0119: package main
0120:
0121: import (
0122:     "context"
0123:
0124:     "github.com/KooshaPari/cliproxyapi-plusplus/pkg/llmproxy/translator"
0125:     openai "github.com/sashabaranov/go-openai"
0126: )
0127:
0128: // MyCustomTranslator implements the Translator interface
0129: type MyCustomTranslator struct{}
0130:
0131: func (t *MyCustomTranslator) TranslateRequest(
0132:     ctx context.Context,
0133:     req *openai.ChatCompletionRequest,
0134: ) (*translator.ProviderRequest, error) {
0135:     // Convert OpenAI request to your provider's format
0136:     return &translator.ProviderRequest{
0137:         Endpoint: "https://api.myprovider.com/v1/chat",
0138:         Headers: map[string]string{
0139:             "Content-Type": "application/json",
0140:         },
0141:         Body: map[string]interface{}{
0142:             "messages": req.Messages,
0143:             "model":    req.Model,
0144:         },
0145:     }, nil
0146: }
0147:
0148: func (t *MyCustomTranslator) TranslateResponse(
0149:     ctx context.Context,
0150:     resp *translator.ProviderResponse,
0151: ) (*openai.ChatCompletionResponse, error) {
0152:     // Convert provider response back to OpenAI format
0153:     return &openai.ChatCompletionResponse{
0154:         ID:      resp.ID,
0155:         Choices: []openai.ChatCompletionChoice{
0156:             {
0157:                 Message: openai.ChatCompletionMessage{
0158:                     Role:    "assistant",
0159:                     Content: resp.Content,
0160:                 },
0161:             },
0162:         },
0163:     }, nil
0164: }
0165:
0166: // Register your translator
0167: func main() {
0168:     myTranslator := &MyCustomTranslator{}
0169:
0170:     svc, err := cliproxy.NewBuilder().
0171:         WithConfig(cfg).
0172:         WithConfigPath("config.yaml").
0173:         WithCustomTranslator("myprovider", myTranslator).
0174:         Build()
0175:     // ...
0176: }
0177: ```
0178:
0179: ## Advanced: Custom Auth Management
0180:
0181: Integrate with your existing auth system:
0182:
0183: ```go
0184: package main
0185:
0186: import (
0187:     "context"
0188:     "sync"
0189:
0190:     "github.com/KooshaPari/cliproxyapi-plusplus/sdk/cliproxy"
0191: )
0192:
0193: // MyAuthProvider implements TokenClientProvider
0194: type MyAuthProvider struct {
0195:     mu    sync.RWMutex
0196:     tokens map[string]string
0197: }
0198:
0199: func (p *MyAuthProvider) Load(
0200:     ctx context.Context,
0201:     cfg *config.Config,
0202: ) (*cliproxy.TokenClientResult, error) {
0203:     p.mu.RLock()
0204:     defer p.mu.RUnlock()
0205:
0206:     var clients []cliproxy.AuthClient
0207:     for provider, token := range p.tokens {
0208:         clients = append(clients, cliproxy.AuthClient{
0209:             Provider: provider,
0210:             Type:     "api_key",
0211:             Token:    token,
0212:         })
0213:     }
0214:
0215:     return &cliproxy.TokenClientResult{
0216:         Clients: clients,
0217:         Count:   len(clients),
0218:     }, nil
0219: }
0220:

### FILE: docs/features/auth/SPEC.md
0001: # Technical Specification: Enterprise Authentication & Lifecycle
0002:
0003: ## Overview
0004:
0005: **cliproxyapi++** implements enterprise-grade authentication management with full lifecycle automation, supporting multiple authentication flows (API keys, OAuth, device authorization) and automatic token refresh capabilities.
0006:
0007: ## Authentication Architecture
0008:
0009: ### Core Components
0010:
0011: ```
0012: Auth System
0013: ├── Auth Manager (coreauth.Manager)
0014: │   ├── Token Store (File-based)
0015: │   ├── Refresh Worker (Background)
0016: │   ├── Health Checker
0017: │   └── Quota Tracker
0018: ├── Auth Flows
0019: │   ├── API Key Flow
0020: │   ├── OAuth 2.0 Flow
0021: │   ├── Device Authorization Flow
0022: │   └── Custom Provider Flows
0023: └── Credential Management
0024:     ├── Multi-credential support
0025:     ├── Per-credential quota tracking
0026:     └── Automatic rotation
0027: ```
0028:
0029: ## Authentication Flows
0030:
0031: ### 1. API Key Authentication
0032:
0033: **Purpose**: Simple token-based authentication for providers with static API keys.
0034:
0035: **Implementation**:
0036: ```go
0037: type APIKeyAuth struct {
0038:     Token string `json:"token"`
0039: }
0040:
0041: func (a *APIKeyAuth) GetHeaders() map[string]string {
0042:     return map[string]string{
0043:         "Authorization": fmt.Sprintf("Bearer %s", a.Token),
0044:     }
0045: }
0046: ```
0047:
0048: **Supported Providers**: Claude, Gemini, OpenAI, Mistral, Groq, DeepSeek
0049:
0050: **Storage Format** (`auths/{provider}.json`):
0051: ```json
0052: {
0053:   "type": "api_key",
0054:   "token": "sk-ant-xxx",
0055:   "priority": 1,
0056:   "quota": {
0057:     "limit": 1000000,
0058:     "used": 50000
0059:   }
0060: }
0061: ```
0062:
0063: ### 2. OAuth 2.0 Flow
0064:
0065: **Purpose**: Standard OAuth 2.0 authorization code flow for providers requiring user consent.
0066:
0067: **Flow Sequence**:
0068: ```
0069: 1. User initiates auth
0070: 2. Redirect to provider auth URL
0071: 3. User grants consent
0072: 4. Provider redirects with authorization code
0073: 5. Exchange code for access token
0074: 6. Store access + refresh token
0075: ```
0076:
0077: **Implementation**:
0078: ```go
0079: type OAuthFlow struct {
0080:     clientID     string
0081:     clientSecret string
0082:     redirectURL  string
0083:     authURL      string
0084:     tokenURL     string
0085: }
0086:
0087: func (f *OAuthFlow) Start(ctx context.Context) (*AuthResult, error) {
0088:     state := generateSecureState()
0089:     authURL := fmt.Sprintf("%s?response_type=code&client_id=%s&redirect_uri=%s&state=%s",
0090:         f.authURL, f.clientID, f.redirectURL, state)
0091:
0092:     return &AuthResult{
0093:         Method:  "oauth",
0094:         AuthURL: authURL,
0095:         State:   state,
0096:     }, nil
0097: }
0098:
0099: func (f *OAuthFlow) Exchange(ctx context.Context, code string) (*AuthToken, error) {
0100:     // Exchange authorization code for tokens
0101:     resp, err := http.PostForm(f.tokenURL, map[string]string{
0102:         "client_id":     f.clientID,
0103:         "client_secret": f.clientSecret,
0104:         "code":          code,
0105:         "redirect_uri":  f.redirectURL,
0106:         "grant_type":    "authorization_code",
0107:     })
0108:
0109:     // Parse and return tokens
0110: }
0111: ```
0112:
0113: **Supported Providers**: GitHub Copilot (partial)
0114:
0115: ### 3. Device Authorization Flow
0116:
0117: **Purpose**: OAuth 2.0 device authorization grant for headless/batch environments.
0118:
0119: **Flow Sequence**:
0120: ```
0121: 1. Request device code
0122: 2. Display user code and verification URL
0123: 3. User visits URL, enters code
0124: 4. Background polling for token
0125: 5. Receive access token
0126: ```
0127:
0128: **Implementation**:
0129: ```go
0130: type DeviceFlow struct {
0131:     deviceCodeURL string
0132:     tokenURL      string
0133:     clientID      string
0134: }
0135:
0136: func (f *DeviceFlow) Start(ctx context.Context) (*AuthResult, error) {
0137:     resp, err := http.PostForm(f.deviceCodeURL, map[string]string{
0138:         "client_id": f.clientID,
0139:     })
0140:
0141:     var dc struct {
0142:         DeviceCode              string `json:"device_code"`
0143:         UserCode               string `json:"user_code"`
0144:         VerificationURI        string `json:"verification_uri"`
0145:         VerificationURIComplete string `json:"verification_uri_complete"`
0146:         ExpiresIn              int    `json:"expires_in"`
0147:         Interval               int    `json:"interval"`
0148:     }
0149:
0150:     // Parse and return device code info
0151:     return &AuthResult{
0152:         Method:              "device_flow",
0153:         UserCode:            dc.UserCode,
0154:         VerificationURL:     dc.VerificationURI,
0155:         DeviceCode:          dc.DeviceCode,
0156:         Interval:            dc.Interval,
0157:         ExpiresAt:           time.Now().Add(time.Duration(dc.ExpiresIn) * time.Second),
0158:     }, nil
0159: }
0160:
0161: func (f *DeviceFlow) Poll(ctx context.Context, deviceCode string) (*AuthToken, error) {
0162:     ticker := time.NewTicker(time.Duration(f.Interval) * time.Second)
0163:     defer ticker.Stop()
0164:
0165:     for {
0166:         select {
0167:         case <-ctx.Done():
0168:             return nil, ctx.Err()
0169:         case <-ticker.C:
0170:             resp, err := http.PostForm(f.tokenURL, map[string]string{
0171:                 "client_id":   f.clientID,
0172:                 "grant_type":  "urn:ietf:params:oauth:grant-type:device_code",
0173:                 "device_code": deviceCode,
0174:             })
0175:
0176:             var token struct {
0177:                 AccessToken string `json:"access_token"`
0178:                 ExpiresIn   int    `json:"expires_in"`
0179:                 Error       string `json:"error"`
0180:             }
0181:
0182:             if token.Error == "" {
0183:                 return &AuthToken{
0184:                     AccessToken: token.AccessToken,
0185:                     ExpiresAt:   time.Now().Add(time.Duration(token.ExpiresIn) * time.Second),
0186:                 }, nil
0187:             }
0188:
0189:             if token.Error != "authorization_pending" {
0190:                 return nil, fmt.Errorf("device flow error: %s", token.Error)
0191:             }
0192:         }
0193:     }
0194: }
0195: ```
0196:
0197: **Supported Providers**: GitHub Copilot (Full), Kiro (AWS CodeWhisperer)
0198:
0199: ## Provider-Specific Authentication
0200:
0201: ### GitHub Copilot (Full OAuth Device Flow)
0202:
0203: **Authentication Flow**:
0204: 1. Device code request to GitHub
0205: 2. User authorizes via browser
0206: 3. Poll for access token
0207: 4. Refresh token management
0208:
0209: **Token Storage** (`auths/copilot.json`):
0210: ```json
0211: {
0212:   "type": "oauth_device_flow",
0213:   "access_token": "ghu_xxx",
0214:   "refresh_token": "ghr_xxx",
0215:   "expires_at": "2026-02-20T00:00:00Z",
0216:   "quota": {
0217:     "limit": 10000,
0218:     "used": 100
0219:   }
0220: }

### FILE: docs/features/auth/USER.md
0001: # User Guide: Enterprise Authentication
0002:
0003: ## Understanding Authentication in cliproxyapi++
0004:
0005: cliproxyapi++ supports multiple authentication methods for different LLM providers. The authentication system handles credential management, automatic token refresh, and quota tracking seamlessly in the background.
0006:
0007: ## Quick Start: Adding Credentials
0008:
0009: ### Method 1: Manual Configuration
0010:
0011: Create credential files in the `auths/` directory:
0012:
0013: **Claude API Key** (`auths/claude.json`):
0014: ```json
0015: {
0016:   "type": "api_key",
0017:   "token": "sk-ant-xxxxx",
0018:   "priority": 1
0019: }
0020: ```
0021:
0022: **OpenAI API Key** (`auths/openai.json`):
0023: ```json
0024: {
0025:   "type": "api_key",
0026:   "token": "sk-xxxxx",
0027:   "priority": 2
0028: }
0029: ```
0030:
0031: **Gemini API Key** (`auths/gemini.json`):
0032: ```json
0033: {
0034:   "type": "api_key",
0035:   "token": "AIzaSyxxxxx",
0036:   "priority": 3
0037: }
0038: ```
0039:
0040: ### Method 2: Interactive Setup (Web UI)
0041:
0042: For providers with OAuth/device flow, use the web interface:
0043:
0044: **GitHub Copilot**:
0045: 1. Visit `http://localhost:8317/v0/oauth/copilot`
0046: 2. Enter your GitHub credentials
0047: 3. Authorize the application
0048: 4. Token is automatically stored
0049:
0050: **Kiro (AWS CodeWhisperer)**:
0051: 1. Visit `http://localhost:8317/v0/oauth/kiro`
0052: 2. Choose AWS Builder ID or Identity Center
0053: 3. Complete browser-based login
0054: 4. Token is automatically stored
0055:
0056: ### Method 3: CLI Commands
0057:
0058: ```bash
0059: # Add API key
0060: curl -X POST http://localhost:8317/v0/management/auths \
0061:   -H "Content-Type: application/json" \
0062:   -d '{
0063:     "provider": "claude",
0064:     "type": "api_key",
0065:     "token": "sk-ant-xxxxx"
0066:   }'
0067:
0068: # Add with priority
0069: curl -X POST http://localhost:8317/v0/management/auths \
0070:   -H "Content-Type: application/json" \
0071:   -d '{
0072:     "provider": "claude",
0073:     "type": "api_key",
0074:     "token": "sk-ant-xxxxx",
0075:     "priority": 10
0076:   }'
0077: ```
0078:
0079: ## Authentication Methods
0080:
0081: ### API Key Authentication
0082:
0083: **Best for**: Providers with static API keys that don't expire.
0084:
0085: **Supported Providers**:
0086: - Claude (Anthropic)
0087: - OpenAI
0088: - Gemini (Google)
0089: - Mistral
0090: - Groq
0091: - DeepSeek
0092: - And many more
0093:
0094: **Setup**:
0095: ```json
0096: {
0097:   "type": "api_key",
0098:   "token": "your-api-key-here",
0099:   "priority": 1
0100: }
0101: ```
0102:
0103: **Priority**: Lower number = higher priority. Used when multiple credentials exist for the same provider.
0104:
0105: ### OAuth 2.0 Device Flow
0106:
0107: **Best for**: Providers requiring user consent with token refresh capability.
0108:
0109: **Supported Providers**:
0110: - GitHub Copilot
0111: - Kiro (AWS CodeWhisperer)
0112:
0113: **Setup**: Use web UI - automatic handling of device code, user authorization, and token storage.
0114:
0115: **How it Works**:
0116: 1. System requests a device code from provider
0117: 2. You're shown a user code and verification URL
0118: 3. Visit URL, enter code, authorize
0119: 4. System polls for token in background
0120: 5. Token stored and automatically refreshed
0121:
0122: **Example: GitHub Copilot**:
0123: ```bash
0124: # Visit web UI
0125: open http://localhost:8317/v0/oauth/copilot
0126:
0127: # Enter your GitHub credentials
0128: # Authorize the application
0129: # Done! Token is stored and managed automatically
0130: ```
0131:
0132: ### Custom Provider Authentication
0133:
0134: **Best for**: Proprietary providers with custom auth flows.
0135:
0136: **Setup**: Implement custom auth flow in embedded library (see DEV.md).
0137:
0138: ## Quota Management
0139:
0140: ### Understanding Quotas
0141:
0142: Track usage per credential:
0143:
0144: ```json
0145: {
0146:   "type": "api_key",
0147:   "token": "sk-ant-xxxxx",
0148:   "quota": {
0149:     "limit": 1000000,
0150:     "used": 50000,
0151:     "remaining": 950000
0152:   }
0153: }
0154: ```
0155:
0156: **Automatic Quota Tracking**:
0157: - Request tokens are deducted from quota after each request
0158: - Multiple credentials are load-balanced based on remaining quota
0159: - Automatic rotation when quota is exhausted
0160:
0161: ### Setting Quotas
0162:
0163: ```bash
0164: # Update quota via API
0165: curl -X PUT http://localhost:8317/v0/management/auths/claude/quota \
0166:   -H "Content-Type: application/json" \
0167:   -d '{
0168:     "limit": 1000000
0169:   }'
0170: ```
0171:
0172: ### Quota Reset
0173:
0174: Quotas reset automatically based on provider billing cycles (configurable in `config.yaml`):
0175:
0176: ```yaml
0177: auth:
0178:   quota:
0179:     reset_schedule:
0180:       claude: "monthly"
0181:       openai: "monthly"
0182:       gemini: "daily"
0183: ```
0184:
0185: ## Automatic Token Refresh
0186:
0187: ### How It Works
0188:
0189: The refresh worker runs every 5 minutes and:
0190: 1. Checks all credentials for expiration
0191: 2. Refreshes tokens expiring within 10 minutes
0192: 3. Updates stored credentials
0193: 4. Notifies applications of refresh (no downtime)
0194:
0195: ### Configuration
0196:
0197: ```yaml
0198: auth:
0199:   refresh:
0200:     enabled: true
0201:     check_interval: "5m"
0202:     refresh_lead_time: "10m"
0203: ```
0204:
0205: ### Monitoring Refresh
0206:
0207: ```bash
0208: # Check refresh status
0209: curl http://localhost:8317/v0/management/auths/refresh/status
0210: ```
0211:
0212: Response:
0213: ```json
0214: {
0215:   "last_check": "2026-02-19T23:00:00Z",
0216:   "next_check": "2026-02-19T23:05:00Z",
0217:   "credentials_checked": 5,
0218:   "refreshed": 1,
0219:   "failed": 0
0220: }

### FILE: docs/features/operations/SPEC.md
0001: # Technical Specification: High-Scale Operations
0002:
0003: ## Overview
0004:
0005: **cliproxyapi++** is designed for high-scale production environments with intelligent operations features: automated cooldown, load balancing, health checking, and comprehensive observability.
0006:
0007: ## Operations Architecture
0008:
0009: ### Core Components
0010:
0011: ```
0012: Operations Layer
0013: ├── Intelligent Cooldown System
0014: │   ├── Rate Limit Detection
0015: │   ├── Provider-Specific Cooldown
0016: │   ├── Automatic Recovery
0017: │   └── Load Redistribution
0018: ├── Load Balancing
0019: │   ├── Round-Robin Strategy
0020: │   ├── Quota-Aware Strategy
0021: │   ├── Latency-Based Strategy
0022: │   └── Cost-Based Strategy
0023: ├── Health Monitoring
0024: │   ├── Provider Health Checks
0025: │   ├── Dependency Health Checks
0026: │   ├── Service Health Checks
0027: │   └── Self-Healing
0028: └── Observability
0029:     ├── Metrics Collection
0030:     ├── Distributed Tracing
0031:     ├── Structured Logging
0032:     └── Alerting
0033: ```
0034:
0035: ## Intelligent Cooldown System
0036:
0037: ### Rate Limit Detection
0038:
0039: **Purpose**: Automatically detect when providers are rate-limited and temporarily pause requests.
0040:
0041: **Implementation**:
0042: ```go
0043: type RateLimitDetector struct {
0044:     mu                sync.RWMutex
0045:     providerStatus    map[string]ProviderStatus
0046:     detectionWindow   time.Duration
0047:     threshold         int
0048: }
0049:
0050: type ProviderStatus struct {
0051:     InCooldown        bool
0052:     CooldownUntil     time.Time
0053:     RecentErrors      []time.Time
0054:     RateLimitCount    int
0055: }
0056:
0057: func (d *RateLimitDetector) RecordError(provider string, statusCode int) {
0058:     d.mu.Lock()
0059:     defer d.mu.Unlock()
0060:
0061:     status := d.providerStatus[provider]
0062:
0063:     // Check for rate limit (429)
0064:     if statusCode == 429 {
0065:         status.RateLimitCount++
0066:         status.RecentErrors = append(status.RecentErrors, time.Now())
0067:     }
0068:
0069:     // Clean old errors
0070:     cutoff := time.Now().Add(-d.detectionWindow)
0071:     var recent []time.Time
0072:     for _, errTime := range status.RecentErrors {
0073:         if errTime.After(cutoff) {
0074:             recent = append(recent, errTime)
0075:         }
0076:     }
0077:     status.RecentErrors = recent
0078:
0079:     // Trigger cooldown if threshold exceeded
0080:     if status.RateLimitCount >= d.threshold {
0081:         status.InCooldown = true
0082:         status.CooldownUntil = time.Now().Add(5 * time.Minute)
0083:         status.RateLimitCount = 0
0084:     }
0085:
0086:     d.providerStatus[provider] = status
0087: }
0088: ```
0089:
0090: ### Cooldown Duration
0091:
0092: **Provider-specific cooldown periods**:
0093: ```yaml
0094: providers:
0095:   claude:
0096:     cooldown:
0097:       enabled: true
0098:       default_duration: "5m"
0099:       rate_limit_duration: "10m"
0100:       error_duration: "2m"
0101:   openai:
0102:     cooldown:
0103:       enabled: true
0104:       default_duration: "3m"
0105:       rate_limit_duration: "5m"
0106:       error_duration: "1m"
0107: ```
0108:
0109: ### Automatic Recovery
0110:
0111: **Recovery mechanisms**:
0112: ```go
0113: type CooldownRecovery struct {
0114:     detector *RateLimitDetector
0115:     checker  *HealthChecker
0116: }
0117:
0118: func (r *CooldownRecovery) Run(ctx context.Context) {
0119:     ticker := time.NewTicker(30 * time.Second)
0120:     defer ticker.Stop()
0121:
0122:     for {
0123:         select {
0124:         case <-ctx.Done():
0125:             return
0126:         case <-ticker.C:
0127:             r.attemptRecovery()
0128:         }
0129:     }
0130: }
0131:
0132: func (r *CooldownRecovery) attemptRecovery() {
0133:     for provider, status := range r.detector.providerStatus {
0134:         if status.InCooldown && time.Now().After(status.CooldownUntil) {
0135:             // Try health check
0136:             if err := r.checker.Check(provider); err == nil {
0137:                 // Recovery successful
0138:                 r.detector.ExitCooldown(provider)
0139:                 log.Infof("Provider %s recovered from cooldown", provider)
0140:             }
0141:         }
0142:     }
0143: }
0144: ```
0145:
0146: ### Load Redistribution
0147:
0148: **Redistribute requests away from cooldown providers**:
0149: ```go
0150: type LoadRedistributor struct {
0151:     providerRegistry map[string]ProviderExecutor
0152:     cooldownDetector *RateLimitDetector
0153: }
0154:
0155: func (l *LoadRedistributor) SelectProvider(providers []string) (string, error) {
0156:     // Filter out providers in cooldown
0157:     available := []string{}
0158:     for _, provider := range providers {
0159:         if !l.cooldownDetector.IsInCooldown(provider) {
0160:             available = append(available, provider)
0161:         }
0162:     }
0163:
0164:     if len(available) == 0 {
0165:         return "", fmt.Errorf("all providers in cooldown")
0166:     }
0167:
0168:     // Select from available providers
0169:     return l.selectFromAvailable(available)
0170: }
0171: ```
0172:
0173: ## Load Balancing Strategies
0174:
0175: ### Strategy Interface
0176:
0177: ```go
0178: type LoadBalancingStrategy interface {
0179:     Select(providers []string, metrics *ProviderMetrics) (string, error)
0180:     Name() string
0181: }
0182: ```
0183:
0184: ### Round-Robin Strategy
0185:
0186: ```go
0187: type RoundRobinStrategy struct {
0188:     counters map[string]int
0189:     mu       sync.Mutex
0190: }
0191:
0192: func (s *RoundRobinStrategy) Select(providers []string, metrics *ProviderMetrics) (string, error) {
0193:     s.mu.Lock()
0194:     defer s.mu.Unlock()
0195:
0196:     if len(providers) == 0 {
0197:         return "", fmt.Errorf("no providers available")
0198:     }
0199:
0200:     // Get counter for first provider (all share counter)
0201:     counter := s.counters["roundrobin"]
0202:     selected := providers[counter%len(providers)]
0203:
0204:     s.counters["roundrobin"] = counter + 1
0205:
0206:     return selected, nil
0207: }
0208: ```
0209:
0210: ### Quota-Aware Strategy
0211:
0212: ```go
0213: type QuotaAwareStrategy struct{}
0214:
0215: func (s *QuotaAwareStrategy) Select(providers []string, metrics *ProviderMetrics) (string, error) {
0216:     var bestProvider string
0217:     var bestQuota float64
0218:
0219:     for _, provider := range providers {
0220:         quota := metrics.GetQuotaRemaining(provider)

### FILE: docs/features/operations/USER.md
0001: # User Guide: High-Scale Operations
0002:
0003: ## Understanding Operations in cliproxyapi++
0004:
0005: cliproxyapi++ is built for production environments with intelligent operations that automatically handle rate limits, load balance requests, monitor health, and recover from failures. This guide explains how to configure and use these features.
0006:
0007: ## Quick Start: Production Deployment
0008:
0009: ### docker-compose.yml (Production)
0010:
0011: ```yaml
0012: services:
0013:   cliproxy:
0014:     image: KooshaPari/cliproxyapi-plusplus:latest
0015:     container_name: cliproxyapi++
0016:
0017:     # Security
0018:     security_opt:
0019:       - no-new-privileges:true
0020:     read_only: true
0021:     user: "65534:65534"
0022:
0023:     # Resources
0024:     deploy:
0025:       resources:
0026:         limits:
0027:           cpus: '4'
0028:           memory: 2G
0029:         reservations:
0030:           cpus: '1'
0031:           memory: 512M
0032:
0033:     # Health check
0034:     healthcheck:
0035:       test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:8317/health"]
0036:       interval: 30s
0037:       timeout: 10s
0038:       retries: 3
0039:       start_period: 40s
0040:
0041:     # Ports
0042:     ports:
0043:       - "8317:8317"
0044:       - "9090:9090"  # Metrics
0045:
0046:     # Volumes
0047:     volumes:
0048:       - ./config.yaml:/config/config.yaml:ro
0049:       - ./auths:/auths:rw
0050:       - ./logs:/logs:rw
0051:
0052:     # Restart
0053:     restart: unless-stopped
0054: ```
0055:
0056: ## Intelligent Cooldown
0057:
0058: ### What is Cooldown?
0059:
0060: When a provider returns rate limit errors (429), cliproxyapi++ automatically pauses requests to that provider for a configurable cooldown period. This prevents your IP from being flagged and allows the provider to recover.
0061:
0062: ### Configure Cooldown
0063:
0064: **config.yaml**:
0065: ```yaml
0066: server:
0067:   operations:
0068:     cooldown:
0069:       enabled: true
0070:       detection_window: "1m"
0071:       error_threshold: 5  # 5 errors in 1 minute triggers cooldown
0072:
0073: providers:
0074:   claude:
0075:     cooldown:
0076:       enabled: true
0077:       default_duration: "5m"
0078:       rate_limit_duration: "10m"  # Longer cooldown for 429
0079:       error_duration: "2m"        # Shorter for other errors
0080:
0081:   openai:
0082:     cooldown:
0083:       enabled: true
0084:       default_duration: "3m"
0085:       rate_limit_duration: "5m"
0086:       error_duration: "1m"
0087: ```
0088:
0089: ### Monitor Cooldown Status
0090:
0091: ```bash
0092: # Check cooldown status
0093: curl http://localhost:8317/v0/operations/cooldown/status
0094: ```
0095:
0096: Response:
0097: ```json
0098: {
0099:   "providers_in_cooldown": ["claude"],
0100:   "cooldown_periods": {
0101:     "claude": {
0102:       "started_at": "2026-02-19T22:50:00Z",
0103:       "ends_at": "2026-02-19T23:00:00Z",
0104:       "remaining_seconds": 300,
0105:       "reason": "rate_limit"
0106:     }
0107:   }
0108: }
0109: ```
0110:
0111: ### Manual Cooldown Control
0112:
0113: **Force cooldown**:
0114: ```bash
0115: curl -X POST http://localhost:8317/v0/operations/providers/claude/cooldown \
0116:   -H "Content-Type: application/json" \
0117:   -d '{
0118:     "duration": "10m",
0119:     "reason": "manual"
0120:   }'
0121: ```
0122:
0123: **Force recovery**:
0124: ```bash
0125: curl -X POST http://localhost:8317/v0/operations/providers/claude/recover
0126: ```
0127:
0128: ## Load Balancing
0129:
0130: ### Choose a Strategy
0131:
0132: **config.yaml**:
0133: ```yaml
0134: server:
0135:   operations:
0136:     load_balancing:
0137:       strategy: "round_robin"  # Options: round_robin, quota_aware, latency, cost
0138: ```
0139:
0140: **Strategies**:
0141: - `round_robin`: Rotate evenly through providers (default)
0142: - `quota_aware`: Use provider with most remaining quota
0143: - `latency`: Use provider with lowest recent latency
0144: - `cost`: Use provider with lowest average cost
0145:
0146: ### Round-Robin (Default)
0147:
0148: ```yaml
0149: server:
0150:   operations:
0151:     load_balancing:
0152:       strategy: "round_robin"
0153: ```
0154:
0155: **Best for**: Simple deployments with similar providers.
0156:
0157: ### Quota-Aware
0158:
0159: ```yaml
0160: server:
0161:   operations:
0162:     load_balancing:
0163:       strategy: "quota_aware"
0164:
0165: providers:
0166:   claude:
0167:     quota:
0168:       limit: 1000000
0169:       reset: "monthly"
0170:
0171:   openai:
0172:     quota:
0173:       limit: 2000000
0174:       reset: "monthly"
0175: ```
0176:
0177: **Best for**: Managing API quota limits across multiple providers.
0178:
0179: ### Latency-Based
0180:
0181: ```yaml
0182: server:
0183:   operations:
0184:     load_balancing:
0185:       strategy: "latency"
0186:       latency_window: "5m"  # Average over last 5 minutes
0187: ```
0188:
0189: **Best for**: Performance-critical applications.
0190:
0191: ### Cost-Based
0192:
0193: ```yaml
0194: server:
0195:   operations:
0196:     load_balancing:
0197:       strategy: "cost"
0198:
0199: providers:
0200:   claude:
0201:     cost_per_1k_tokens:
0202:       input: 0.003
0203:       output: 0.015
0204:
0205:   openai:
0206:     cost_per_1k_tokens:
0207:       input: 0.005
0208:       output: 0.015
0209: ```
0210:
0211: **Best for**: Cost optimization.
0212:
0213: ### Provider Priority
0214:
0215: ```yaml
0216: providers:
0217:   claude:
0218:     priority: 1  # Higher priority
0219:   gemini:
0220:     priority: 2

### FILE: docs/features/providers/SPEC.md
0001: # Technical Specification: Provider Registry & Support
0002:
0003: ## Overview
0004:
0005: **cliproxyapi++** supports an extensive registry of LLM providers, from direct API integrations to multi-provider aggregators and proprietary protocols. This specification details the provider architecture, supported providers, and extension mechanisms.
0006:
0007: ## Provider Architecture
0008:
0009: ### Provider Types
0010:
0011: ```
0012: Provider Registry
0013: ├── Direct Providers
0014: │   ├── Claude (Anthropic)
0015: │   ├── Gemini (Google)
0016: │   ├── OpenAI
0017: │   ├── Mistral
0018: │   ├── Groq
0019: │   └── DeepSeek
0020: ├── Aggregator Providers
0021: │   ├── OpenRouter
0022: │   ├── Together AI
0023: │   ├── Fireworks AI
0024: │   ├── Novita AI
0025: │   └── SiliconFlow
0026: └── Proprietary Providers
0027:     ├── Kiro (AWS CodeWhisperer)
0028:     ├── GitHub Copilot
0029:     ├── Roo Code
0030:     ├── Kilo AI
0031:     └── MiniMax
0032: ```
0033:
0034: ### Provider Interface
0035:
0036: ```go
0037: type Provider interface {
0038:     // Provider metadata
0039:     Name() string
0040:     Type() ProviderType
0041:
0042:     // Model support
0043:     SupportsModel(model string) bool
0044:     ListModels() []Model
0045:
0046:     // Authentication
0047:     AuthType() AuthType
0048:     RequiresAuth() bool
0049:
0050:     // Execution
0051:     Execute(ctx context.Context, req *Request) (*Response, error)
0052:     ExecuteStream(ctx context.Context, req *Request) (<-chan *Chunk, error)
0053:
0054:     // Capabilities
0055:     SupportsStreaming() bool
0056:     SupportsFunctions() bool
0057:     MaxTokens() int
0058:
0059:     // Health
0060:     HealthCheck(ctx context.Context) error
0061: }
0062: ```
0063:
0064: ### Provider Configuration
0065:
0066: ```go
0067: type ProviderConfig struct {
0068:     Name        string            `yaml:"name"`
0069:     Type        string            `yaml:"type"`
0070:     Enabled     bool              `yaml:"enabled"`
0071:     AuthType    string            `yaml:"auth_type"`
0072:     Endpoint    string            `yaml:"endpoint"`
0073:     Models      []ModelConfig     `yaml:"models"`
0074:     Features    ProviderFeatures  `yaml:"features"`
0075:     Limits      ProviderLimits    `yaml:"limits"`
0076:     Cooldown    CooldownConfig    `yaml:"cooldown"`
0077:     Priority    int               `yaml:"priority"`
0078: }
0079:
0080: type ModelConfig struct {
0081:     Name              string `yaml:"name"`
0082:     Enabled           bool   `yaml:"enabled"`
0083:     MaxTokens         int    `yaml:"max_tokens"`
0084:     SupportsFunctions bool   `yaml:"supports_functions"`
0085:     SupportsStreaming bool   `yaml:"supports_streaming"`
0086: }
0087:
0088: type ProviderFeatures struct {
0089:     Streaming        bool `yaml:"streaming"`
0090:     Functions        bool `yaml:"functions"`
0091:     Vision           bool `yaml:"vision"`
0092:     CodeGeneration   bool `yaml:"code_generation"`
0093:     Multimodal       bool `yaml:"multimodal"`
0094: }
0095:
0096: type ProviderLimits struct {
0097:     RequestsPerMinute int `yaml:"requests_per_minute"`
0098:     TokensPerMinute   int `yaml:"tokens_per_minute"`
0099:     MaxTokensPerReq   int `yaml:"max_tokens_per_request"`
0100: }
0101: ```
0102:
0103: ## Direct Providers
0104:
0105: ### Claude (Anthropic)
0106:
0107: **Provider Type**: `claude`
0108:
0109: **Authentication**: API Key
0110:
0111: **Models**:
0112: - `claude-3-5-sonnet` (max: 200K tokens)
0113: - `claude-3-5-haiku` (max: 200K tokens)
0114: - `claude-3-opus` (max: 200K tokens)
0115:
0116: **Features**:
0117: - Streaming: ✅
0118: - Functions: ✅
0119: - Vision: ✅
0120: - Code generation: ✅
0121:
0122: **Configuration**:
0123: ```yaml
0124: providers:
0125:   claude:
0126:     type: "claude"
0127:     enabled: true
0128:     auth_type: "api_key"
0129:     endpoint: "https://api.anthropic.com"
0130:     models:
0131:       - name: "claude-3-5-sonnet"
0132:         enabled: true
0133:         max_tokens: 200000
0134:         supports_functions: true
0135:         supports_streaming: true
0136:     features:
0137:       streaming: true
0138:       functions: true
0139:       vision: true
0140:       code_generation: true
0141:     limits:
0142:       requests_per_minute: 60
0143:       tokens_per_minute: 40000
0144: ```
0145:
0146: **API Endpoint**: `https://api.anthropic.com/v1/messages`
0147:
0148: **Request Format**:
0149: ```json
0150: {
0151:   "model": "claude-3-5-sonnet-20241022",
0152:   "max_tokens": 1024,
0153:   "messages": [
0154:     {"role": "user", "content": "Hello!"}
0155:   ],
0156:   "stream": true
0157: }
0158: ```
0159:
0160: **Headers**:
0161: ```
0162: x-api-key: sk-ant-xxxx
0163: anthropic-version: 2023-06-01
0164: content-type: application/json
0165: ```
0166:
0167: ### Gemini (Google)
0168:
0169: **Provider Type**: `gemini`
0170:
0171: **Authentication**: API Key
0172:
0173: **Models**:
0174: - `gemini-1.5-pro` (max: 1M tokens)
0175: - `gemini-1.5-flash` (max: 1M tokens)
0176: - `gemini-1.0-pro` (max: 32K tokens)
0177:
0178: **Features**:
0179: - Streaming: ✅
0180: - Functions: ✅
0181: - Vision: ✅
0182: - Multimodal: ✅
0183:
0184: **Configuration**:
0185: ```yaml
0186: providers:
0187:   gemini:
0188:     type: "gemini"
0189:     enabled: true
0190:     auth_type: "api_key"
0191:     endpoint: "https://generativelanguage.googleapis.com"
0192:     models:
0193:       - name: "gemini-1.5-pro"
0194:         enabled: true
0195:         max_tokens: 1000000
0196:     features:
0197:       streaming: true
0198:       functions: true
0199:       vision: true
0200:       multimodal: true
0201: ```
0202:
0203: ### OpenAI
0204:
0205: **Provider Type**: `openai`
0206:
0207: **Authentication**: API Key
0208:
0209: **Models**:
0210: - `gpt-4-turbo` (max: 128K tokens)
0211: - `gpt-4` (max: 8K tokens)
0212: - `gpt-3.5-turbo` (max: 16K tokens)
0213:
0214: **Features**:
0215: - Streaming: ✅
0216: - Functions: ✅
0217: - Vision: ✅ (GPT-4 Vision)
0218:
0219: **Configuration**:
0220: ```yaml

### FILE: docs/features/providers/USER.md
0001: # User Guide: Provider Registry
0002:
0003: ## Understanding Providers in cliproxyapi++
0004:
0005: cliproxyapi++ supports an extensive registry of LLM providers, from direct API integrations (Claude, Gemini, OpenAI) to multi-provider aggregators (OpenRouter, Together AI) and proprietary protocols (Kiro, GitHub Copilot). This guide explains how to configure and use these providers.
0006:
0007: ## Quick Start: Using a Provider
0008:
0009: ### 1. Add Provider Credential
0010:
0011: ```bash
0012: # Claude API key
0013: echo '{"type":"api_key","token":"sk-ant-xxxxx"}' > auths/claude.json
0014:
0015: # OpenAI API key
0016: echo '{"type":"api_key","token":"sk-xxxxx"}' > auths/openai.json
0017:
0018: # Gemini API key
0019: echo '{"type":"api_key","token":"AIzaSyxxxxx"}' > auths/gemini.json
0020: ```
0021:
0022: ### 2. Configure Provider
0023:
0024: **config.yaml**:
0025: ```yaml
0026: providers:
0027:   claude:
0028:     type: "claude"
0029:     enabled: true
0030:     auth_type: "api_key"
0031:
0032:   openai:
0033:     type: "openai"
0034:     enabled: true
0035:     auth_type: "api_key"
0036:
0037:   gemini:
0038:     type: "gemini"
0039:     enabled: true
0040:     auth_type: "api_key"
0041: ```
0042:
0043: ### 3. Make Request
0044:
0045: ```bash
0046: curl -X POST http://localhost:8317/v1/chat/completions \
0047:   -H "Content-Type: application/json" \
0048:   -d '{
0049:     "model": "claude-3-5-sonnet",
0050:     "messages": [{"role": "user", "content": "Hello!"}]
0051:   }'
0052: ```
0053:
0054: ## Direct Providers
0055:
0056: ### Claude (Anthropic)
0057:
0058: **Best for**: Advanced reasoning, long context, vision tasks
0059:
0060: **Models**:
0061: - `claude-3-5-sonnet` - Most capable, 200K context
0062: - `claude-3-5-haiku` - Fast, 200K context
0063: - `claude-3-opus` - High performance, 200K context
0064:
0065: **Configuration**:
0066: ```yaml
0067: providers:
0068:   claude:
0069:     type: "claude"
0070:     enabled: true
0071:     auth_type: "api_key"
0072:     models:
0073:       - name: "claude-3-5-sonnet"
0074:         enabled: true
0075: ```
0076:
0077: **Usage**:
0078: ```bash
0079: curl -X POST http://localhost:8317/v1/chat/completions \
0080:   -H "Content-Type: application/json" \
0081:   -d '{
0082:     "model": "claude-3-5-sonnet",
0083:     "messages": [{"role": "user", "content": "Explain quantum computing"}]
0084:   }'
0085: ```
0086:
0087: ### Gemini (Google)
0088:
0089: **Best for**: Multimodal tasks, long context, cost-effective
0090:
0091: **Models**:
0092: - `gemini-1.5-pro` - 1M context window
0093: - `gemini-1.5-flash` - Fast, 1M context
0094: - `gemini-1.0-pro` - Stable, 32K context
0095:
0096: **Configuration**:
0097: ```yaml
0098: providers:
0099:   gemini:
0100:     type: "gemini"
0101:     enabled: true
0102:     auth_type: "api_key"
0103: ```
0104:
0105: **Usage**:
0106: ```bash
0107: curl -X POST http://localhost:8317/v1/chat/completions \
0108:   -H "Content-Type: application/json" \
0109:   -d '{
0110:     "model": "gemini-1.5-pro",
0111:     "messages": [{"role": "user", "content": "What is machine learning?"}]
0112:   }'
0113: ```
0114:
0115: ### OpenAI
0116:
0117: **Best for**: General purpose, functions, ecosystem
0118:
0119: **Models**:
0120: - `gpt-4-turbo` - 128K context
0121: - `gpt-4` - 8K context
0122: - `gpt-3.5-turbo` - Fast, 16K context
0123:
0124: **Configuration**:
0125: ```yaml
0126: providers:
0127:   openai:
0128:     type: "openai"
0129:     enabled: true
0130:     auth_type: "api_key"
0131: ```
0132:
0133: **Usage**:
0134: ```bash
0135: curl -X POST http://localhost:8317/v1/chat/completions \
0136:   -H "Content-Type: application/json" \
0137:   -d '{
0138:     "model": "gpt-4-turbo",
0139:     "messages": [{"role": "user", "content": "Hello!"}]
0140:   }'
0141: ```
0142:
0143: ## Aggregator Providers
0144:
0145: ### OpenRouter
0146:
0147: **Best for**: Access to 100+ models through one API
0148:
0149: **Features**:
0150: - Unified pricing
0151: - Model comparison
0152: - Easy model switching
0153:
0154: **Configuration**:
0155: ```yaml
0156: providers:
0157:   openrouter:
0158:     type: "openrouter"
0159:     enabled: true
0160:     auth_type: "api_key"
0161: ```
0162:
0163: **Usage**:
0164: ```bash
0165: # Access Claude through OpenRouter
0166: curl -X POST http://localhost:8317/v1/chat/completions \
0167:   -H "Content-Type: application/json" \
0168:   -d '{
0169:     "model": "anthropic/claude-3.5-sonnet",
0170:     "messages": [{"role": "user", "content": "Hello!"}]
0171:   }'
0172: ```
0173:
0174: ### Together AI
0175:
0176: **Best for**: Open-source models at scale
0177:
0178: **Features**:
0179: - Llama, Mistral, and more
0180: - Fast inference
0181: - Cost-effective
0182:
0183: **Configuration**:
0184: ```yaml
0185: providers:
0186:   together:
0187:     type: "together"
0188:     enabled: true
0189:     auth_type: "api_key"
0190: ```
0191:
0192: **Usage**:
0193: ```bash
0194: curl -X POST http://localhost:8317/v1/chat/completions \
0195:   -H "Content-Type: application/json" \
0196:   -d '{
0197:     "model": "meta-llama/Llama-3-70b-chat-hf",
0198:     "messages": [{"role": "user", "content": "Hello!"}]
0199:   }'
0200: ```
0201:
0202: ### Fireworks AI
0203:
0204: **Best for**: Sub-second latency
0205:
0206: **Features**:
0207: - Fast inference
0208: - Open-source models
0209: - API-first
0210:
0211: **Configuration**:
0212: ```yaml
0213: providers:
0214:   fireworks:
0215:     type: "fireworks"
0216:     enabled: true
0217:     auth_type: "api_key"
0218: ```
0219:
0220: **Usage**:

### FILE: docs/features/security/SPEC.md
0001: # Technical Specification: Security Hardening ("Defense in Depth")
0002:
0003: ## Overview
0004:
0005: **cliproxyapi++** implements a comprehensive "Defense in Depth" security philosophy with multiple layers of protection: CI-enforced code integrity, hardened container images, device fingerprinting, and secure credential management.
0006:
0007: ## Security Architecture
0008:
0009: ### Defense Layers
0010:
0011: ```
0012: Layer 1: Code Integrity
0013: ├── Path Guard (CI enforcement)
0014: ├── Signed releases
0015: └── Multi-arch builds
0016:
0017: Layer 2: Container Hardening
0018: ├── Minimal base image (Alpine 3.22.0)
0019: ├── Non-root user
0020: ├── Read-only filesystem
0021: └── Seccomp profiles
0022:
0023: Layer 3: Credential Security
0024: ├── Encrypted storage
0025: ├── Secure file permissions
0026: ├── Token refresh isolation
0027: └── Device fingerprinting
0028:
0029: Layer 4: Network Security
0030: ├── TLS only
0031: ├── Request validation
0032: ├── Rate limiting
0033: └── IP allowlisting
0034:
0035: Layer 5: Operational Security
0036: ├── Audit logging
0037: ├── Secret scanning
0038: ├── Dependency scanning
0039: └── Vulnerability management
0040: ```
0041:
0042: ## Layer 1: Code Integrity
0043:
0044: ### Path Guard CI Enforcement
0045:
0046: **Purpose**: Prevent unauthorized changes to critical translation logic during pull requests.
0047:
0048: **Implementation** (`.github/workflows/pr-path-guard.yml`):
0049: ```yaml
0050: name: Path Guard
0051: on:
0052:   pull_request:
0053:     paths:
0054:       - 'pkg/llmproxy/translator/**'
0055:       - 'pkg/llmproxy/auth/**'
0056:
0057: jobs:
0058:   guard:
0059:     runs-on: ubuntu-latest
0060:     steps:
0061:       - uses: actions/checkout@v4
0062:         with:
0063:           fetch-depth: 0
0064:
0065:       - name: Check path protection
0066:         run: |
0067:           # Only allow changes from trusted maintainers
0068:           if ! git log --format="%an" ${{ github.event.pull_request.base.sha }}..${{ github.sha }} | grep -q "KooshaPari"; then
0069:             echo "::error::Unauthorized changes to protected paths"
0070:             exit 1
0071:           fi
0072:
0073:       - name: Verify no translator logic changes
0074:         run: |
0075:           # Ensure core translation logic hasn't been tampered
0076:           if git diff ${{ github.event.pull_request.base.sha }}..${{ github.sha }} --name-only | grep -q "pkg/llmproxy/translator/.*\.go$"; then
0077:             echo "::warning::Translator logic changed - requires maintainer review"
0078:           fi
0079: ```
0080:
0081: **Protected Paths**:
0082: - `pkg/llmproxy/translator/` - Core translation logic
0083: - `pkg/llmproxy/auth/` - Authentication flows
0084: - `pkg/llmproxy/provider/` - Provider execution
0085:
0086: **Authorization Rules**:
0087: - Only repository maintainers can modify
0088: - All changes require at least 2 maintainer approvals
0089: - Must pass security review
0090:
0091: ### Signed Releases
0092:
0093: **Purpose**: Ensure released artifacts are authentic and tamper-proof.
0094:
0095: **Implementation** (`.goreleaser.yml`):
0096: ```yaml
0097: signs:
0098:   - artifacts: checksum
0099:     args:
0100:       - "--batch"
0101:       - "--local-user"
0102:       - "${GPG_FINGERPRINT}"
0103: ```
0104:
0105: **Verification**:
0106: ```bash
0107: # Download release
0108: wget https://github.com/KooshaPari/cliproxyapi-plusplus/releases/download/v6.0.0/cliproxyapi-plusplus_6.0.0_checksums.txt
0109:
0110: # Download signature
0111: wget https://github.com/KooshaPari/cliproxyapi-plusplus/releases/download/v6.0.0/cliproxyapi-plusplus_6.0.0_checksums.txt.sig
0112:
0113: # Import GPG key
0114: gpg --keyserver keyserver.ubuntu.com --recv-keys XXXXXXXX
0115:
0116: # Verify signature
0117: gpg --verify cliproxyapi-plusplus_6.0.0_checksums.txt.sig cliproxyapi-plusplus_6.0.0_checksums.txt
0118:
0119: # Verify checksum
0120: sha256sum -c cliproxyapi-plusplus_6.0.0_checksums.txt
0121: ```
0122:
0123: ### Multi-Arch Builds
0124:
0125: **Purpose**: Provide consistent security across architectures.
0126:
0127: **Platforms**:
0128: - `linux/amd64`
0129: - `linux/arm64`
0130: - `darwin/amd64`
0131: - `darwin/arm64`
0132:
0133: **CI Build Matrix**:
0134: ```yaml
0135: strategy:
0136:   matrix:
0137:     goos: [linux, darwin]
0138:     goarch: [amd64, arm64]
0139: ```
0140:
0141: ## Layer 2: Container Hardening
0142:
0143: ### Minimal Base Image
0144:
0145: **Base**: Alpine Linux 3.22.0
0146:
0147: **Dockerfile**:
0148: ```dockerfile
0149: FROM alpine:3.22.0 AS builder
0150:
0151: # Install build dependencies
0152: RUN apk add --no-cache \
0153:     ca-certificates \
0154:     gcc \
0155:     musl-dev
0156:
0157: # Build application
0158: COPY . .
0159: RUN go build -o cliproxyapi cmd/server/main.go
0160:
0161: # Final stage - minimal runtime
0162: FROM scratch
0163: COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
0164: COPY --from=builder /cliproxyapi /cliproxyapi
0165:
0166: # Non-root user
0167: USER 65534:65534
0168:
0169: # Read-only filesystem
0170: VOLUME ["/config", "/auths", "/logs"]
0171:
0172: ENTRYPOINT ["/cliproxyapi"]
0173: ```
0174:
0175: **Security Benefits**:
0176: - Minimal attack surface (no shell, no package manager)
0177: - No unnecessary packages
0178: - Static binary linking
0179: - Reproducible builds
0180:
0181: ### Security Context
0182:
0183: **docker-compose.yml**:
0184: ```yaml
0185: services:
0186:   cliproxy:
0187:     image: KooshaPari/cliproxyapi-plusplus:latest
0188:     security_opt:
0189:       - no-new-privileges:true
0190:     read_only: true
0191:     tmpfs:
0192:       - /tmp:noexec,nosuid,size=100m
0193:     cap_drop:
0194:       - ALL
0195:     cap_add:
0196:       - NET_BIND_SERVICE
0197:     user: "65534:65534"
0198: ```
0199:
0200: **Explanation**:
0201: - `no-new-privileges`: Prevent privilege escalation
0202: - `read_only`: Immutable filesystem
0203: - `tmpfs`: Noexec on temporary files
0204: - `cap_drop:ALL`: Drop all capabilities
0205: - `cap_add:NET_BIND_SERVICE`: Only allow binding ports
0206: - `user:65534:65534`: Run as non-root (nobody)
0207:
0208: ### Seccomp Profiles
0209:
0210: **Custom seccomp profile** (`seccomp-profile.json`):
0211: ```json
0212: {
0213:   "defaultAction": "SCMP_ACT_ERRNO",
0214:   "architectures": ["SCMP_ARCH_X86_64", "SCMP_ARCH_AARCH64"],
0215:   "syscalls": [
0216:     {
0217:       "names": ["read", "write", "open", "close", "stat", "fstat", "lstat"],
0218:       "action": "SCMP_ACT_ALLOW"
0219:     },
0220:     {

### FILE: docs/features/security/USER.md
0001: # User Guide: Security Hardening
0002:
0003: ## Understanding Security in cliproxyapi++
0004:
0005: cliproxyapi++ is built with a "Defense in Depth" philosophy, meaning multiple layers of security protect your deployments. This guide explains how to configure and use these security features effectively.
0006:
0007: ## Quick Security Checklist
0008:
0009: **Before deploying to production**:
0010:
0011: ```bash
0012: # 1. Verify Docker image is signed
0013: docker pull KooshaPari/cliproxyapi-plusplus:latest
0014: docker trust verify KooshaPari/cliproxyapi-plusplus:latest
0015:
0016: # 2. Set secure file permissions
0017: chmod 600 auths/*.json
0018: chmod 700 auths/
0019:
0020: # 3. Enable TLS
0021: # Edit config.yaml to enable TLS (see below)
0022:
0023: # 4. Enable encryption
0024: # Generate encryption key and set in config.yaml
0025:
0026: # 5. Configure rate limiting
0027: # Set appropriate limits in config.yaml
0028: ```
0029:
0030: ## Container Security
0031:
0032: ### Hardened Docker Deployment
0033:
0034: **docker-compose.yml**:
0035: ```yaml
0036: services:
0037:   cliproxy:
0038:     image: KooshaPari/cliproxyapi-plusplus:latest
0039:     container_name: cliproxyapi++
0040:
0041:     # Security options
0042:     security_opt:
0043:       - no-new-privileges:true
0044:     read_only: true
0045:     tmpfs:
0046:       - /tmp:noexec,nosuid,size=100m
0047:     cap_drop:
0048:       - ALL
0049:     cap_add:
0050:       - NET_BIND_SERVICE
0051:
0052:     # Non-root user
0053:     user: "65534:65534"
0054:
0055:     # Volumes (writable only for these)
0056:     volumes:
0057:       - ./config.yaml:/config/config.yaml:ro
0058:       - ./auths:/auths:rw
0059:       - ./logs:/logs:rw
0060:       - ./tls:/tls:ro
0061:
0062:     # Network
0063:     ports:
0064:       - "8317:8317"
0065:
0066:     # Resource limits
0067:     deploy:
0068:       resources:
0069:         limits:
0070:           cpus: '2'
0071:           memory: 1G
0072:         reservations:
0073:           cpus: '0.5'
0074:           memory: 256M
0075:
0076:     restart: unless-stopped
0077: ```
0078:
0079: **Explanation**:
0080: - `no-new-privileges`: Prevents processes from gaining more privileges
0081: - `read_only`: Makes container filesystem immutable (attackers can't modify binaries)
0082: - `tmpfs:noexec`: Prevents execution of files in `/tmp`
0083: - `cap_drop:ALL`: Drops all Linux capabilities
0084: - `cap_add:NET_BIND_SERVICE`: Only adds back the ability to bind ports
0085: - `user:65534:65534`: Runs as non-root "nobody" user
0086:
0087: ### Seccomp Profiles (Advanced)
0088:
0089: **Custom seccomp profile**:
0090: ```bash
0091: # Save seccomp profile
0092: cat > seccomp-profile.json << 'EOF'
0093: {
0094:   "defaultAction": "SCMP_ACT_ERRNO",
0095:   "syscalls": [
0096:     {
0097:       "names": ["read", "write", "open", "close", "socket", "bind", "listen"],
0098:       "action": "SCMP_ACT_ALLOW"
0099:     }
0100:   ]
0101: }
0102: EOF
0103:
0104: # Use in docker-compose
0105: security_opt:
0106:   - seccomp:./seccomp-profile.json
0107: ```
0108:
0109: ## TLS Configuration
0110:
0111: ### Enable HTTPS
0112:
0113: **config.yaml**:
0114: ```yaml
0115: server:
0116:   port: 8317
0117:   tls:
0118:     enabled: true
0119:     cert_file: "/tls/tls.crt"
0120:     key_file: "/tls/tls.key"
0121:     min_version: "1.2"
0122:     cipher_suites:
0123:       - "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
0124:       - "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
0125: ```
0126:
0127: ### Generate Self-Signed Certificate (Testing)
0128:
0129: ```bash
0130: # Generate private key
0131: openssl genrsa -out tls.key 2048
0132:
0133: # Generate certificate
0134: openssl req -new -x509 -key tls.key -out tls.crt -days 365 \
0135:   -subj "/C=US/ST=State/L=City/O=Organization/CN=localhost"
0136:
0137: # Set permissions
0138: chmod 600 tls.key
0139: chmod 644 tls.crt
0140: ```
0141:
0142: ### Use Let's Encrypt (Production)
0143:
0144: ```bash
0145: # Install certbot
0146: sudo apt-get install certbot
0147:
0148: # Generate certificate
0149: sudo certbot certonly --standalone -d proxy.example.com
0150:
0151: # Copy to tls directory
0152: sudo cp /etc/letsencrypt/live/proxy.example.com/fullchain.pem tls/tls.crt
0153: sudo cp /etc/letsencrypt/live/proxy.example.com/privkey.pem tls/tls.key
0154:
0155: # Set permissions
0156: sudo chown $USER:$USER tls/tls.key tls/tls.crt
0157: chmod 600 tls/tls.key
0158: chmod 644 tls/tls.crt
0159: ```
0160:
0161: ## Credential Encryption
0162:
0163: ### Enable Encryption
0164:
0165: **config.yaml**:
0166: ```yaml
0167: auth:
0168:   encryption:
0169:     enabled: true
0170:     key: "YOUR_32_BYTE_ENCRYPTION_KEY_HERE"
0171: ```
0172:
0173: ### Generate Encryption Key
0174:
0175: ```bash
0176: # Method 1: Using openssl
0177: openssl rand -base64 32
0178:
0179: # Method 2: Using Python
0180: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
0181:
0182: # Method 3: Using /dev/urandom
0183: head -c 32 /dev/urandom | base64
0184: ```
0185:
0186: ### Environment Variable (Recommended)
0187:
0188: ```yaml
0189: auth:
0190:   encryption:
0191:     enabled: true
0192:     key: "${CLIPROXY_ENCRYPTION_KEY}"
0193: ```
0194:
0195: ```bash
0196: # Set in environment
0197: export CLIPRO_ENCRYPTION_KEY="$(openssl rand -base64 32)"
0198:
0199: # Use in docker-compose
0200: environment:
0201:   - CLIPRO_ENCRYPTION_KEY=${CLIPRO_ENCRYPTION_KEY}
0202: ```
0203:
0204: ### Migrating Existing Credentials
0205:
0206: When enabling encryption, existing credentials remain unencrypted. To encrypt them:
0207:
0208: ```bash
0209: # 1. Enable encryption in config.yaml
0210: # 2. Restart service
0211: # 3. Re-add credentials (they will be encrypted)
0212: curl -X POST http://localhost:8317/v0/management/auths \
0213:   -H "Content-Type: application/json" \
0214:   -d '{
0215:     "provider": "claude",
0216:     "type": "api_key",
0217:     "token": "sk-ant-xxxxx"
0218:   }'
0219: ```
0220:

### FILE: docs/index.md
0001: # cliproxy++
0002:
0003: This is the VitePress entrypoint for cliproxyapi++ documentation.
0004:
0005: ## Audience Docsets
0006:
0007: - [Developer (Internal)](./docsets/developer/internal/)
0008: - [Developer (External)](./docsets/developer/external/)
0009: - [Technical User](./docsets/user/)
0010: - [Agent Operator](./docsets/agent/)
0011:
0012: ## Key References
0013:
0014: - [Feature Changes in ++](./FEATURE_CHANGES_PLUSPLUS.md)
0015: - [Documentation README](./README.md)
0016: - [API Docs](./api/)
0017: - [Feature Docs](./features/)

### FILE: docs/sdk-access.md
0001: # @sdk/access SDK Reference
0002:
0003: The `github.com/router-for-me/CLIProxyAPI/v6/sdk/access` package centralizes inbound request authentication for the proxy. It offers a lightweight manager that chains credential providers, so servers can reuse the same access control logic inside or outside the CLI runtime.
0004:
0005: ## Importing
0006:
0007: ```go
0008: import (
0009:     sdkaccess "github.com/router-for-me/CLIProxyAPI/v6/sdk/access"
0010: )
0011: ```
0012:
0013: Add the module with `go get github.com/router-for-me/CLIProxyAPI/v6/sdk/access`.
0014:
0015: ## Provider Registry
0016:
0017: Providers are registered globally and then attached to a `Manager` as a snapshot:
0018:
0019: - `RegisterProvider(type, provider)` installs a pre-initialized provider instance.
0020: - Registration order is preserved the first time each `type` is seen.
0021: - `RegisteredProviders()` returns the providers in that order.
0022:
0023: ## Manager Lifecycle
0024:
0025: ```go
0026: manager := sdkaccess.NewManager()
0027: manager.SetProviders(sdkaccess.RegisteredProviders())
0028: ```
0029:
0030: * `NewManager` constructs an empty manager.
0031: * `SetProviders` replaces the provider slice using a defensive copy.
0032: * `Providers` retrieves a snapshot that can be iterated safely from other goroutines.
0033:
0034: If the manager itself is `nil` or no providers are configured, the call returns `nil, nil`, allowing callers to treat access control as disabled.
0035:
0036: ## Authenticating Requests
0037:
0038: ```go
0039: result, authErr := manager.Authenticate(ctx, req)
0040: switch {
0041: case authErr == nil:
0042:     // Authentication succeeded; result describes the provider and principal.
0043: case sdkaccess.IsAuthErrorCode(authErr, sdkaccess.AuthErrorCodeNoCredentials):
0044:     // No recognizable credentials were supplied.
0045: case sdkaccess.IsAuthErrorCode(authErr, sdkaccess.AuthErrorCodeInvalidCredential):
0046:     // Supplied credentials were present but rejected.
0047: default:
0048:     // Internal/transport failure was returned by a provider.
0049: }
0050: ```
0051:
0052: `Manager.Authenticate` walks the configured providers in order. It returns on the first success, skips providers that return `AuthErrorCodeNotHandled`, and aggregates `AuthErrorCodeNoCredentials` / `AuthErrorCodeInvalidCredential` for a final result.
0053:
0054: Each `Result` includes the provider identifier, the resolved principal, and optional metadata (for example, which header carried the credential).
0055:
0056: ## Built-in `config-api-key` Provider
0057:
0058: The proxy includes one built-in access provider:
0059:
0060: - `config-api-key`: Validates API keys declared under top-level `api-keys`.
0061:   - Credential sources: `Authorization: Bearer`, `X-Goog-Api-Key`, `X-Api-Key`, `?key=`, `?auth_token=`
0062:   - Metadata: `Result.Metadata["source"]` is set to the matched source label.
0063:
0064: In the CLI server and `sdk/cliproxy`, this provider is registered automatically based on the loaded configuration.
0065:
0066: ```yaml
0067: api-keys:
0068:   - sk-test-123
0069:   - sk-prod-456
0070: ```
0071:
0072: ## Loading Providers from External Go Modules
0073:
0074: To consume a provider shipped in another Go module, import it for its registration side effect:
0075:
0076: ```go
0077: import (
0078:     _ "github.com/acme/xplatform/sdk/access/providers/partner" // registers partner-token
0079:     sdkaccess "github.com/router-for-me/CLIProxyAPI/v6/sdk/access"
0080: )
0081: ```
0082:
0083: The blank identifier import ensures `init` runs so `sdkaccess.RegisterProvider` executes before you call `RegisteredProviders()` (or before `cliproxy.NewBuilder().Build()`).
0084:
0085: ### Metadata and auditing
0086:
0087: `Result.Metadata` carries provider-specific context. The built-in `config-api-key` provider, for example, stores the credential source (`authorization`, `x-goog-api-key`, `x-api-key`, `query-key`, `query-auth-token`). Populate this map in custom providers to enrich logs and downstream auditing.
0088:
0089: ## Writing Custom Providers
0090:
0091: ```go
0092: type customProvider struct{}
0093:
0094: func (p *customProvider) Identifier() string { return "my-provider" }
0095:
0096: func (p *customProvider) Authenticate(ctx context.Context, r *http.Request) (*sdkaccess.Result, *sdkaccess.AuthError) {
0097:     token := r.Header.Get("X-Custom")
0098:     if token == "" {
0099:         return nil, sdkaccess.NewNotHandledError()
0100:     }
0101:     if token != "expected" {
0102:         return nil, sdkaccess.NewInvalidCredentialError()
0103:     }
0104:     return &sdkaccess.Result{
0105:         Provider:  p.Identifier(),
0106:         Principal: "service-user",
0107:         Metadata:  map[string]string{"source": "x-custom"},
0108:     }, nil
0109: }
0110:
0111: func init() {
0112:     sdkaccess.RegisterProvider("custom", &customProvider{})
0113: }
0114: ```
0115:
0116: A provider must implement `Identifier()` and `Authenticate()`. To make it available to the access manager, call `RegisterProvider` inside `init` with an initialized provider instance.
0117:
0118: ## Error Semantics
0119:
0120: - `NewNoCredentialsError()` (`AuthErrorCodeNoCredentials`): no credentials were present or recognized. (HTTP 401)
0121: - `NewInvalidCredentialError()` (`AuthErrorCodeInvalidCredential`): credentials were present but rejected. (HTTP 401)
0122: - `NewNotHandledError()` (`AuthErrorCodeNotHandled`): fall through to the next provider.
0123: - `NewInternalAuthError(message, cause)` (`AuthErrorCodeInternal`): transport/system failure. (HTTP 500)
0124:
0125: Errors propagate immediately to the caller unless they are classified as `not_handled` / `no_credentials` / `invalid_credential` and can be aggregated by the manager.
0126:
0127: ## Integration with cliproxy Service
0128:
0129: `sdk/cliproxy` wires `@sdk/access` automatically when you build a CLI service via `cliproxy.NewBuilder`. Supplying a manager lets you reuse the same instance in your host process:
0130:
0131: ```go
0132: coreCfg, _ := config.LoadConfig("config.yaml")
0133: accessManager := sdkaccess.NewManager()
0134:
0135: svc, _ := cliproxy.NewBuilder().
0136:   WithConfig(coreCfg).
0137:   WithConfigPath("config.yaml").
0138:   WithRequestAccessManager(accessManager).
0139:   Build()
0140: ```
0141:
0142: Register any custom providers (typically via blank imports) before calling `Build()` so they are present in the global registry snapshot.
0143:
0144: ### Hot reloading
0145:
0146: When configuration changes, refresh any config-backed providers and then reset the manager's provider chain:
0147:
0148: ```go
0149: // configaccess is github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/access/config_access
0150: configaccess.Register(&newCfg.SDKConfig)
0151: accessManager.SetProviders(sdkaccess.RegisteredProviders())
0152: ```
0153:
0154: This mirrors the behaviour in `pkg/llmproxy/access.ApplyAccessProviders`, enabling runtime updates without restarting the process.

### FILE: docs/sdk-access_CN.md
0001: # @sdk/access 开发指引
0002:
0003: `github.com/router-for-me/CLIProxyAPI/v6/sdk/access` 包负责代理的入站访问认证。它提供一个轻量的管理器，用于按顺序链接多种凭证校验实现，让服务器在 CLI 运行时内外都能复用相同的访问控制逻辑。
0004:
0005: ## 引用方式
0006:
0007: ```go
0008: import (
0009:     sdkaccess "github.com/router-for-me/CLIProxyAPI/v6/sdk/access"
0010: )
0011: ```
0012:
0013: 通过 `go get github.com/router-for-me/CLIProxyAPI/v6/sdk/access` 添加依赖。
0014:
0015: ## Provider Registry
0016:
0017: 访问提供者是全局注册，然后以快照形式挂到 `Manager` 上：
0018:
0019: - `RegisterProvider(type, provider)` 注册一个已经初始化好的 provider 实例。
0020: - 每个 `type` 第一次出现时会记录其注册顺序。
0021: - `RegisteredProviders()` 会按该顺序返回 provider 列表。
0022:
0023: ## 管理器生命周期
0024:
0025: ```go
0026: manager := sdkaccess.NewManager()
0027: manager.SetProviders(sdkaccess.RegisteredProviders())
0028: ```
0029:
0030: - `NewManager` 创建空管理器。
0031: - `SetProviders` 替换提供者切片并做防御性拷贝。
0032: - `Providers` 返回适合并发读取的快照。
0033:
0034: 如果管理器本身为 `nil` 或未配置任何 provider，调用会返回 `nil, nil`，可视为关闭访问控制。
0035:
0036: ## 认证请求
0037:
0038: ```go
0039: result, authErr := manager.Authenticate(ctx, req)
0040: switch {
0041: case authErr == nil:
0042:     // Authentication succeeded; result carries provider and principal.
0043: case sdkaccess.IsAuthErrorCode(authErr, sdkaccess.AuthErrorCodeNoCredentials):
0044:     // No recognizable credentials were supplied.
0045: case sdkaccess.IsAuthErrorCode(authErr, sdkaccess.AuthErrorCodeInvalidCredential):
0046:     // Credentials were present but rejected.
0047: default:
0048:     // Provider surfaced a transport-level failure.
0049: }
0050: ```
0051:
0052: `Manager.Authenticate` 会按顺序遍历 provider：遇到成功立即返回，`AuthErrorCodeNotHandled` 会继续尝试下一个；`AuthErrorCodeNoCredentials` / `AuthErrorCodeInvalidCredential` 会在遍历结束后汇总给调用方。
0053:
0054: `Result` 提供认证提供者标识、解析出的主体以及可选元数据（例如凭证来源）。
0055:
0056: ## 内建 `config-api-key` Provider
0057:
0058: 代理内置一个访问提供者：
0059:
0060: - `config-api-key`：校验 `config.yaml` 顶层的 `api-keys`。
0061:   - 凭证来源：`Authorization: Bearer`、`X-Goog-Api-Key`、`X-Api-Key`、`?key=`、`?auth_token=`
0062:   - 元数据：`Result.Metadata["source"]` 会写入匹配到的来源标识
0063:
0064: 在 CLI 服务端与 `sdk/cliproxy` 中，该 provider 会根据加载到的配置自动注册。
0065:
0066: ```yaml
0067: api-keys:
0068:   - sk-test-123
0069:   - sk-prod-456
0070: ```
0071:
0072: ## 引入外部 Go 模块提供者
0073:
0074: 若要消费其它 Go 模块输出的访问提供者，直接用空白标识符导入以触发其 `init` 注册即可：
0075:
0076: ```go
0077: import (
0078:     _ "github.com/acme/xplatform/sdk/access/providers/partner" // registers partner-token
0079:     sdkaccess "github.com/router-for-me/CLIProxyAPI/v6/sdk/access"
0080: )
0081: ```
0082:
0083: 空白导入可确保 `init` 先执行，从而在你调用 `RegisteredProviders()`（或 `cliproxy.NewBuilder().Build()`）之前完成 `sdkaccess.RegisterProvider`。
0084:
0085: ### 元数据与审计
0086:
0087: `Result.Metadata` 用于携带提供者特定的上下文信息。内建的 `config-api-key` 会记录凭证来源（`authorization`、`x-goog-api-key`、`x-api-key`、`query-key`、`query-auth-token`）。自定义提供者同样可以填充该 Map，以便丰富日志与审计场景。
0088:
0089: ## 编写自定义提供者
0090:
0091: ```go
0092: type customProvider struct{}
0093:
0094: func (p *customProvider) Identifier() string { return "my-provider" }
0095:
0096: func (p *customProvider) Authenticate(ctx context.Context, r *http.Request) (*sdkaccess.Result, *sdkaccess.AuthError) {
0097:     token := r.Header.Get("X-Custom")
0098:     if token == "" {
0099:         return nil, sdkaccess.NewNotHandledError()
0100:     }
0101:     if token != "expected" {
0102:         return nil, sdkaccess.NewInvalidCredentialError()
0103:     }
0104:     return &sdkaccess.Result{
0105:         Provider:  p.Identifier(),
0106:         Principal: "service-user",
0107:         Metadata:  map[string]string{"source": "x-custom"},
0108:     }, nil
0109: }
0110:
0111: func init() {
0112:     sdkaccess.RegisterProvider("custom", &customProvider{})
0113: }
0114: ```
0115:
0116: 自定义提供者需要实现 `Identifier()` 与 `Authenticate()`。在 `init` 中用已初始化实例调用 `RegisterProvider` 注册到全局 registry。
0117:
0118: ## 错误语义
0119:
0120: - `NewNoCredentialsError()`（`AuthErrorCodeNoCredentials`）：未提供或未识别到凭证。（HTTP 401）
0121: - `NewInvalidCredentialError()`（`AuthErrorCodeInvalidCredential`）：凭证存在但校验失败。（HTTP 401）
0122: - `NewNotHandledError()`（`AuthErrorCodeNotHandled`）：告诉管理器跳到下一个 provider。
0123: - `NewInternalAuthError(message, cause)`（`AuthErrorCodeInternal`）：网络/系统错误。（HTTP 500）
0124:
0125: 除可汇总的 `not_handled` / `no_credentials` / `invalid_credential` 外，其它错误会立即冒泡返回。
0126:
0127: ## 与 cliproxy 集成
0128:
0129: 使用 `sdk/cliproxy` 构建服务时会自动接入 `@sdk/access`。如果希望在宿主进程里复用同一个 `Manager` 实例，可传入自定义管理器：
0130:
0131: ```go
0132: coreCfg, _ := config.LoadConfig("config.yaml")
0133: accessManager := sdkaccess.NewManager()
0134:
0135: svc, _ := cliproxy.NewBuilder().
0136:   WithConfig(coreCfg).
0137:   WithConfigPath("config.yaml").
0138:   WithRequestAccessManager(accessManager).
0139:   Build()
0140: ```
0141:
0142: 请在调用 `Build()` 之前完成自定义 provider 的注册（通常通过空白导入触发 `init`），以确保它们被包含在全局 registry 的快照中。
0143:
0144: ### 动态热更新提供者
0145:
0146: 当配置发生变化时，刷新依赖配置的 provider，然后重置 manager 的 provider 链：
0147:
0148: ```go
0149: // configaccess is github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/access/config_access
0150: configaccess.Register(&newCfg.SDKConfig)
0151: accessManager.SetProviders(sdkaccess.RegisteredProviders())
0152: ```
0153:
0154: 这一流程与 `pkg/llmproxy/access.ApplyAccessProviders` 保持一致，避免为更新访问策略而重启进程。

### FILE: docs/sdk-advanced.md
0001: # SDK Advanced: Executors & Translators
0002:
0003: This guide explains how to extend the embedded proxy with custom providers and schemas using the SDK. You will:
0004: - Implement a provider executor that talks to your upstream API
0005: - Register request/response translators for schema conversion
0006: - Register models so they appear in `/v1/models`
0007:
0008: The examples use Go 1.24+ and the v6 module path.
0009:
0010: ## Concepts
0011:
0012: - Provider executor: a runtime component implementing `auth.ProviderExecutor` that performs outbound calls for a given provider key (e.g., `gemini`, `claude`, `codex`). Executors can also implement `RequestPreparer` to inject credentials on raw HTTP requests.
0013: - Translator registry: schema conversion functions routed by `sdk/translator`. The built‑in handlers translate between OpenAI/Gemini/Claude/Codex formats; you can register new ones.
0014: - Model registry: publishes the list of available models per client/provider to power `/v1/models` and routing hints.
0015:
0016: ## 1) Implement a Provider Executor
0017:
0018: Create a type that satisfies `auth.ProviderExecutor`.
0019:
0020: ```go
0021: package myprov
0022:
0023: import (
0024:   "context"
0025:   "net/http"
0026:
0027:   coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
0028:   clipexec "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/executor"
0029: )
0030:
0031: type Executor struct{}
0032:
0033: func (Executor) Identifier() string { return "myprov" }
0034:
0035: // Optional: mutate outbound HTTP requests with credentials
0036: func (Executor) PrepareRequest(req *http.Request, a *coreauth.Auth) error {
0037:   // Example: req.Header.Set("Authorization", "Bearer "+a.APIKey)
0038:   return nil
0039: }
0040:
0041: func (Executor) Execute(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (clipexec.Response, error) {
0042:   // Build HTTP request based on req.Payload (already translated into provider format)
0043:   // Use per‑auth transport if provided: transport := a.RoundTripper // via RoundTripperProvider
0044:   // Perform call and return provider JSON payload
0045:   return clipexec.Response{Payload: []byte(`{"ok":true}`)}, nil
0046: }
0047:
0048: func (Executor) ExecuteStream(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (<-chan clipexec.StreamChunk, error) {
0049:   ch := make(chan clipexec.StreamChunk, 1)
0050:   go func() { defer close(ch); ch <- clipexec.StreamChunk{Payload: []byte("data: {\"done\":true}\n\n")} }()
0051:   return ch, nil
0052: }
0053:
0054: func (Executor) Refresh(ctx context.Context, a *coreauth.Auth) (*coreauth.Auth, error) {
0055:   // Optionally refresh tokens and return updated auth
0056:   return a, nil
0057: }
0058: ```
0059:
0060: Register the executor with the core manager before starting the service:
0061:
0062: ```go
0063: core := coreauth.NewManager(coreauth.NewFileStore(cfg.AuthDir), nil, nil)
0064: core.RegisterExecutor(myprov.Executor{})
0065: svc, _ := cliproxy.NewBuilder().WithConfig(cfg).WithConfigPath(cfgPath).WithCoreAuthManager(core).Build()
0066: ```
0067:
0068: If your auth entries use provider `"myprov"`, the manager routes requests to your executor.
0069:
0070: ## 2) Register Translators
0071:
0072: The handlers accept OpenAI/Gemini/Claude/Codex inputs. To support a new provider format, register translation functions in `sdk/translator`’s default registry.
0073:
0074: Direction matters:
0075: - Request: register from inbound schema to provider schema
0076: - Response: register from provider schema back to inbound schema
0077:
0078: Example: Convert OpenAI Chat → MyProv Chat and back.
0079:
0080: ```go
0081: package myprov
0082:
0083: import (
0084:   "context"
0085:   sdktr "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
0086: )
0087:
0088: const (
0089:   FOpenAI = sdktr.Format("openai.chat")
0090:   FMyProv = sdktr.Format("myprov.chat")
0091: )
0092:
0093: func init() {
0094:   sdktr.Register(FOpenAI, FMyProv,
0095:     // Request transform (model, rawJSON, stream)
0096:     func(model string, raw []byte, stream bool) []byte { return convertOpenAIToMyProv(model, raw, stream) },
0097:     // Response transform (stream & non‑stream)
0098:     sdktr.ResponseTransform{
0099:       Stream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) []string {
0100:         return convertStreamMyProvToOpenAI(model, originalReq, translatedReq, raw)
0101:       },
0102:       NonStream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) string {
0103:         return convertMyProvToOpenAI(model, originalReq, translatedReq, raw)
0104:       },
0105:     },
0106:   )
0107: }
0108: ```
0109:
0110: When the OpenAI handler receives a request that should route to `myprov`, the pipeline uses the registered transforms automatically.
0111:
0112: ## 3) Register Models
0113:
0114: Expose models under `/v1/models` by registering them in the global model registry using the auth ID (client ID) and provider name.
0115:
0116: ```go
0117: models := []*cliproxy.ModelInfo{
0118:   { ID: "myprov-pro-1", Object: "model", Type: "myprov", DisplayName: "MyProv Pro 1" },
0119: }
0120: cliproxy.GlobalModelRegistry().RegisterClient(authID, "myprov", models)
0121: ```
0122:
0123: The embedded server calls this automatically for built‑in providers; for custom providers, register during startup (e.g., after loading auths) or upon auth registration hooks.
0124:
0125: ## Credentials & Transports
0126:
0127: - Use `Manager.SetRoundTripperProvider` to inject per‑auth `*http.Transport` (e.g., proxy):
0128:   ```go
0129:   core.SetRoundTripperProvider(myProvider) // returns transport per auth
0130:   ```
0131: - For raw HTTP flows, implement `PrepareRequest` and/or call `Manager.InjectCredentials(req, authID)` to set headers.
0132:
0133: ## Testing Tips
0134:
0135: - Enable request logging: Management API GET/PUT `/v0/management/request-log`
0136: - Toggle debug logs: Management API GET/PUT `/v0/management/debug`
0137: - Hot reload changes in `config.yaml` and `auths/` are picked up automatically by the watcher
0138:

### FILE: docs/sdk-advanced_CN.md
0001: # SDK 高级指南：执行器与翻译器
0002:
0003: 本文介绍如何使用 SDK 扩展内嵌代理：
0004: - 实现自定义 Provider 执行器以调用你的上游 API
0005: - 注册请求/响应翻译器进行协议转换
0006: - 注册模型以出现在 `/v1/models`
0007:
0008: 示例基于 Go 1.24+ 与 v6 模块路径。
0009:
0010: ## 概念
0011:
0012: - Provider 执行器：实现 `auth.ProviderExecutor` 的运行时组件，负责某个 provider key（如 `gemini`、`claude`、`codex`）的真正出站调用。若实现 `RequestPreparer` 接口，可在原始 HTTP 请求上注入凭据。
0013: - 翻译器注册表：由 `sdk/translator` 驱动的协议转换函数。内置了 OpenAI/Gemini/Claude/Codex 的互转；你也可以注册新的格式转换。
0014: - 模型注册表：对外发布可用模型列表，供 `/v1/models` 与路由参考。
0015:
0016: ## 1) 实现 Provider 执行器
0017:
0018: 创建类型满足 `auth.ProviderExecutor` 接口。
0019:
0020: ```go
0021: package myprov
0022:
0023: import (
0024:     "context"
0025:     "net/http"
0026:
0027:     coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
0028:     clipexec "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/executor"
0029: )
0030:
0031: type Executor struct{}
0032:
0033: func (Executor) Identifier() string { return "myprov" }
0034:
0035: // 可选：在原始 HTTP 请求上注入凭据
0036: func (Executor) PrepareRequest(req *http.Request, a *coreauth.Auth) error {
0037:     // 例如：req.Header.Set("Authorization", "Bearer "+a.Attributes["api_key"])
0038:     return nil
0039: }
0040:
0041: func (Executor) Execute(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (clipexec.Response, error) {
0042:     // 基于 req.Payload 构造上游请求，返回上游 JSON 负载
0043:     return clipexec.Response{Payload: []byte(`{"ok":true}`)}, nil
0044: }
0045:
0046: func (Executor) ExecuteStream(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (<-chan clipexec.StreamChunk, error) {
0047:     ch := make(chan clipexec.StreamChunk, 1)
0048:     go func() { defer close(ch); ch <- clipexec.StreamChunk{Payload: []byte("data: {\\"done\\":true}\\n\\n")} }()
0049:     return ch, nil
0050: }
0051:
0052: func (Executor) Refresh(ctx context.Context, a *coreauth.Auth) (*coreauth.Auth, error) { return a, nil }
0053: ```
0054:
0055: 在启动服务前将执行器注册到核心管理器：
0056:
0057: ```go
0058: core := coreauth.NewManager(coreauth.NewFileStore(cfg.AuthDir), nil, nil)
0059: core.RegisterExecutor(myprov.Executor{})
0060: svc, _ := cliproxy.NewBuilder().WithConfig(cfg).WithConfigPath(cfgPath).WithCoreAuthManager(core).Build()
0061: ```
0062:
0063: 当凭据的 `Provider` 为 `"myprov"` 时，管理器会将请求路由到你的执行器。
0064:
0065: ## 2) 注册翻译器
0066:
0067: 内置处理器接受 OpenAI/Gemini/Claude/Codex 的入站格式。要支持新的 provider 协议，需要在 `sdk/translator` 的默认注册表中注册转换函数。
0068:
0069: 方向很重要：
0070: - 请求：从“入站格式”转换为“provider 格式”
0071: - 响应：从“provider 格式”转换回“入站格式”
0072:
0073: 示例：OpenAI Chat → MyProv Chat 及其反向。
0074:
0075: ```go
0076: package myprov
0077:
0078: import (
0079:   "context"
0080:   sdktr "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
0081: )
0082:
0083: const (
0084:   FOpenAI = sdktr.Format("openai.chat")
0085:   FMyProv = sdktr.Format("myprov.chat")
0086: )
0087:
0088: func init() {
0089:   sdktr.Register(FOpenAI, FMyProv,
0090:     func(model string, raw []byte, stream bool) []byte { return convertOpenAIToMyProv(model, raw, stream) },
0091:     sdktr.ResponseTransform{
0092:       Stream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) []string {
0093:         return convertStreamMyProvToOpenAI(model, originalReq, translatedReq, raw)
0094:       },
0095:       NonStream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) string {
0096:         return convertMyProvToOpenAI(model, originalReq, translatedReq, raw)
0097:       },
0098:     },
0099:   )
0100: }
0101: ```
0102:
0103: 当 OpenAI 处理器接到需要路由到 `myprov` 的请求时，流水线会自动应用已注册的转换。
0104:
0105: ## 3) 注册模型
0106:
0107: 通过全局模型注册表将模型暴露到 `/v1/models`：
0108:
0109: ```go
0110: models := []*cliproxy.ModelInfo{
0111:   { ID: "myprov-pro-1", Object: "model", Type: "myprov", DisplayName: "MyProv Pro 1" },
0112: }
0113: cliproxy.GlobalModelRegistry().RegisterClient(authID, "myprov", models)
0114: ```
0115:
0116: 内置 Provider 会自动注册；自定义 Provider 建议在启动时（例如加载到 Auth 后）或在 Auth 注册钩子中调用。
0117:
0118: ## 凭据与传输
0119:
0120: - 使用 `Manager.SetRoundTripperProvider` 注入按账户的 `*http.Transport`（例如代理）：
0121:   ```go
0122:   core.SetRoundTripperProvider(myProvider) // 按账户返回 transport
0123:   ```
0124: - 对于原始 HTTP 请求，若实现了 `PrepareRequest`，或通过 `Manager.InjectCredentials(req, authID)` 进行头部注入。
0125:
0126: ## 测试建议
0127:
0128: - 启用请求日志：管理 API GET/PUT `/v0/management/request-log`
0129: - 切换调试日志：管理 API GET/PUT `/v0/management/debug`
0130: - 热更新：`config.yaml` 与 `auths/` 变化会自动被侦测并应用
0131:

### FILE: docs/sdk-usage.md
0001: # CLI Proxy SDK Guide
0002:
0003: The `sdk/cliproxy` module exposes the proxy as a reusable Go library so external programs can embed the routing, authentication, hot‑reload, and translation layers without depending on the CLI binary.
0004:
0005: ## Install & Import
0006:
0007: ```bash
0008: go get github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy
0009: ```
0010:
0011: ```go
0012: import (
0013:     "context"
0014:     "errors"
0015:     "time"
0016:
0017:     "github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/config"
0018:     "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy"
0019: )
0020: ```
0021:
0022: Note the `/v6` module path.
0023:
0024: ## Minimal Embed
0025:
0026: ```go
0027: cfg, err := config.LoadConfig("config.yaml")
0028: if err != nil { panic(err) }
0029:
0030: svc, err := cliproxy.NewBuilder().
0031:     WithConfig(cfg).
0032:     WithConfigPath("config.yaml"). // absolute or working-dir relative
0033:     Build()
0034: if err != nil { panic(err) }
0035:
0036: ctx, cancel := context.WithCancel(context.Background())
0037: defer cancel()
0038:
0039: if err := svc.Run(ctx); err != nil && !errors.Is(err, context.Canceled) {
0040:     panic(err)
0041: }
0042: ```
0043:
0044: The service manages config/auth watching, background token refresh, and graceful shutdown. Cancel the context to stop it.
0045:
0046: ## Server Options (middleware, routes, logs)
0047:
0048: The server accepts options via `WithServerOptions`:
0049:
0050: ```go
0051: svc, _ := cliproxy.NewBuilder().
0052:   WithConfig(cfg).
0053:   WithConfigPath("config.yaml").
0054:   WithServerOptions(
0055:     // Add global middleware
0056:     cliproxy.WithMiddleware(func(c *gin.Context) { c.Header("X-Embed", "1"); c.Next() }),
0057:     // Tweak gin engine early (CORS, trusted proxies, etc.)
0058:     cliproxy.WithEngineConfigurator(func(e *gin.Engine) { e.ForwardedByClientIP = true }),
0059:     // Add your own routes after defaults
0060:     cliproxy.WithRouterConfigurator(func(e *gin.Engine, _ *handlers.BaseAPIHandler, _ *config.Config) {
0061:       e.GET("/healthz", func(c *gin.Context) { c.String(200, "ok") })
0062:     }),
0063:     // Override request log writer/dir
0064:     cliproxy.WithRequestLoggerFactory(func(cfg *config.Config, cfgPath string) logging.RequestLogger {
0065:       return logging.NewFileRequestLogger(true, "logs", filepath.Dir(cfgPath))
0066:     }),
0067:   ).
0068:   Build()
0069: ```
0070:
0071: These options mirror the internals used by the CLI server.
0072:
0073: ## Management API (when embedded)
0074:
0075: - Management endpoints are mounted only when `remote-management.secret-key` is set in `config.yaml`.
0076: - Remote access additionally requires `remote-management.allow-remote: true`.
0077: - See MANAGEMENT_API.md for endpoints. Your embedded server exposes them under `/v0/management` on the configured port.
0078:
0079: ## Provider Metrics
0080:
0081: The proxy exposes a metrics endpoint for routing optimization (cost, latency, throughput):
0082:
0083: - `GET /v1/metrics/providers`: Returns per-provider rolling statistics.
0084:
0085: This endpoint is used by `thegent` to implement routing policies like `cheapest` or `fastest`.
0086:
0087: ## Using the Core Auth Manager
0088:
0089: The service uses a core `auth.Manager` for selection, execution, and auto‑refresh. When embedding, you can provide your own manager to customize transports or hooks:
0090:
0091: ```go
0092: core := coreauth.NewManager(coreauth.NewFileStore(cfg.AuthDir), nil, nil)
0093: core.SetRoundTripperProvider(myRTProvider) // per‑auth *http.Transport
0094:
0095: svc, _ := cliproxy.NewBuilder().
0096:     WithConfig(cfg).
0097:     WithConfigPath("config.yaml").
0098:     WithCoreAuthManager(core).
0099:     Build()
0100: ```
0101:
0102: Implement a custom per‑auth transport:
0103:
0104: ```go
0105: type myRTProvider struct{}
0106: func (myRTProvider) RoundTripperFor(a *coreauth.Auth) http.RoundTripper {
0107:     if a == nil || a.ProxyURL == "" { return nil }
0108:     u, _ := url.Parse(a.ProxyURL)
0109:     return &http.Transport{ Proxy: http.ProxyURL(u) }
0110: }
0111: ```
0112:
0113: Programmatic execution is available on the manager:
0114:
0115: ```go
0116: // Non‑streaming
0117: resp, err := core.Execute(ctx, []string{"gemini"}, req, opts)
0118:
0119: // Streaming
0120: chunks, err := core.ExecuteStream(ctx, []string{"gemini"}, req, opts)
0121: for ch := range chunks { /* ... */ }
0122: ```
0123:
0124: Note: Built‑in provider executors are wired automatically when you run the `Service`. If you want to use `Manager` stand‑alone without the HTTP server, you must register your own executors that implement `auth.ProviderExecutor`.
0125:
0126: ## Custom Client Sources
0127:
0128: Replace the default loaders if your creds live outside the local filesystem:
0129:
0130: ```go
0131: type memoryTokenProvider struct{}
0132: func (p *memoryTokenProvider) Load(ctx context.Context, cfg *config.Config) (*cliproxy.TokenClientResult, error) {
0133:     // Populate from memory/remote store and return counts
0134:     return &cliproxy.TokenClientResult{}, nil
0135: }
0136:
0137: svc, _ := cliproxy.NewBuilder().
0138:   WithConfig(cfg).
0139:   WithConfigPath("config.yaml").
0140:   WithTokenClientProvider(&memoryTokenProvider{}).
0141:   WithAPIKeyClientProvider(cliproxy.NewAPIKeyClientProvider()).
0142:   Build()
0143: ```
0144:
0145: ## Hooks
0146:
0147: Observe lifecycle without patching internals:
0148:
0149: ```go
0150: hooks := cliproxy.Hooks{
0151:   OnBeforeStart: func(cfg *config.Config) { log.Infof("starting on :%d", cfg.Port) },
0152:   OnAfterStart:  func(s *cliproxy.Service) { log.Info("ready") },
0153: }
0154: svc, _ := cliproxy.NewBuilder().WithConfig(cfg).WithConfigPath("config.yaml").WithHooks(hooks).Build()
0155: ```
0156:
0157: ## Shutdown
0158:
0159: `Run` defers `Shutdown`, so cancelling the parent context is enough. To stop manually:
0160:
0161: ```go
0162: ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
0163: defer cancel()
0164: _ = svc.Shutdown(ctx)
0165: ```
0166:
0167: ## Notes
0168:
0169: - Hot reload: changes to `config.yaml` and `auths/` are picked up automatically.
0170: - Request logging can be toggled at runtime via the Management API.
0171: - Gemini Web features (`gemini-web.*`) are honored in the embedded server.

### FILE: docs/sdk-usage_CN.md
0001: # CLI Proxy SDK 使用指南
0002:
0003: `sdk/cliproxy` 模块将代理能力以 Go 库的形式对外暴露，方便在其它服务中内嵌路由、鉴权、热更新与翻译层，而无需依赖可执行的 CLI 程序。
0004:
0005: ## 安装与导入
0006:
0007: ```bash
0008: go get github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy
0009: ```
0010:
0011: ```go
0012: import (
0013:     "context"
0014:     "errors"
0015:     "time"
0016:
0017:     "github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/config"
0018:     "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy"
0019: )
0020: ```
0021:
0022: 注意模块路径包含 `/v6`。
0023:
0024: ## 最小可用示例
0025:
0026: ```go
0027: cfg, err := config.LoadConfig("config.yaml")
0028: if err != nil { panic(err) }
0029:
0030: svc, err := cliproxy.NewBuilder().
0031:     WithConfig(cfg).
0032:     WithConfigPath("config.yaml"). // 绝对路径或工作目录相对路径
0033:     Build()
0034: if err != nil { panic(err) }
0035:
0036: ctx, cancel := context.WithCancel(context.Background())
0037: defer cancel()
0038:
0039: if err := svc.Run(ctx); err != nil && !errors.Is(err, context.Canceled) {
0040:     panic(err)
0041: }
0042: ```
0043:
0044: 服务内部会管理配置与认证文件的监听、后台令牌刷新与优雅关闭。取消上下文即可停止服务。
0045:
0046: ## 服务器可选项（中间件、路由、日志）
0047:
0048: 通过 `WithServerOptions` 自定义：
0049:
0050: ```go
0051: svc, _ := cliproxy.NewBuilder().
0052:   WithConfig(cfg).
0053:   WithConfigPath("config.yaml").
0054:   WithServerOptions(
0055:     // 追加全局中间件
0056:     cliproxy.WithMiddleware(func(c *gin.Context) { c.Header("X-Embed", "1"); c.Next() }),
0057:     // 提前调整 gin 引擎（如 CORS、trusted proxies）
0058:     cliproxy.WithEngineConfigurator(func(e *gin.Engine) { e.ForwardedByClientIP = true }),
0059:     // 在默认路由之后追加自定义路由
0060:     cliproxy.WithRouterConfigurator(func(e *gin.Engine, _ *handlers.BaseAPIHandler, _ *config.Config) {
0061:       e.GET("/healthz", func(c *gin.Context) { c.String(200, "ok") })
0062:     }),
0063:     // 覆盖请求日志的创建（启用/目录）
0064:     cliproxy.WithRequestLoggerFactory(func(cfg *config.Config, cfgPath string) logging.RequestLogger {
0065:       return logging.NewFileRequestLogger(true, "logs", filepath.Dir(cfgPath))
0066:     }),
0067:   ).
0068:   Build()
0069: ```
0070:
0071: 这些选项与 CLI 服务器内部用法保持一致。
0072:
0073: ## 管理 API（内嵌时）
0074:
0075: - 仅当 `config.yaml` 中设置了 `remote-management.secret-key` 时才会挂载管理端点。
0076: - 远程访问还需要 `remote-management.allow-remote: true`。
0077: - 具体端点见 MANAGEMENT_API_CN.md。内嵌服务器会在配置端口下暴露 `/v0/management`。
0078:
0079: ## 使用核心鉴权管理器
0080:
0081: 服务内部使用核心 `auth.Manager` 负责选择、执行、自动刷新。内嵌时可自定义其传输或钩子：
0082:
0083: ```go
0084: core := coreauth.NewManager(coreauth.NewFileStore(cfg.AuthDir), nil, nil)
0085: core.SetRoundTripperProvider(myRTProvider) // 按账户返回 *http.Transport
0086:
0087: svc, _ := cliproxy.NewBuilder().
0088:     WithConfig(cfg).
0089:     WithConfigPath("config.yaml").
0090:     WithCoreAuthManager(core).
0091:     Build()
0092: ```
0093:
0094: 实现每个账户的自定义传输：
0095:
0096: ```go
0097: type myRTProvider struct{}
0098: func (myRTProvider) RoundTripperFor(a *coreauth.Auth) http.RoundTripper {
0099:     if a == nil || a.ProxyURL == "" { return nil }
0100:     u, _ := url.Parse(a.ProxyURL)
0101:     return &http.Transport{ Proxy: http.ProxyURL(u) }
0102: }
0103: ```
0104:
0105: 管理器提供编程式执行接口：
0106:
0107: ```go
0108: // 非流式
0109: resp, err := core.Execute(ctx, []string{"gemini"}, req, opts)
0110:
0111: // 流式
0112: chunks, err := core.ExecuteStream(ctx, []string{"gemini"}, req, opts)
0113: for ch := range chunks { /* ... */ }
0114: ```
0115:
0116: 说明：运行 `Service` 时会自动注册内置的提供商执行器；若仅单独使用 `Manager` 而不启动 HTTP 服务器，则需要自行实现并注册满足 `auth.ProviderExecutor` 的执行器。
0117:
0118: ## 自定义凭据来源
0119:
0120: 当凭据不在本地文件系统时，替换默认加载器：
0121:
0122: ```go
0123: type memoryTokenProvider struct{}
0124: func (p *memoryTokenProvider) Load(ctx context.Context, cfg *config.Config) (*cliproxy.TokenClientResult, error) {
0125:     // 从内存/远端加载并返回数量统计
0126:     return &cliproxy.TokenClientResult{}, nil
0127: }
0128:
0129: svc, _ := cliproxy.NewBuilder().
0130:   WithConfig(cfg).
0131:   WithConfigPath("config.yaml").
0132:   WithTokenClientProvider(&memoryTokenProvider{}).
0133:   WithAPIKeyClientProvider(cliproxy.NewAPIKeyClientProvider()).
0134:   Build()
0135: ```
0136:
0137: ## 启动钩子
0138:
0139: 无需修改内部代码即可观察生命周期：
0140:
0141: ```go
0142: hooks := cliproxy.Hooks{
0143:   OnBeforeStart: func(cfg *config.Config) { log.Infof("starting on :%d", cfg.Port) },
0144:   OnAfterStart:  func(s *cliproxy.Service) { log.Info("ready") },
0145: }
0146: svc, _ := cliproxy.NewBuilder().WithConfig(cfg).WithConfigPath("config.yaml").WithHooks(hooks).Build()
0147: ```
0148:
0149: ## 关闭
0150:
0151: `Run` 内部会延迟调用 `Shutdown`，因此只需取消父上下文即可。若需手动停止：
0152:
0153: ```go
0154: ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
0155: defer cancel()
0156: _ = svc.Shutdown(ctx)
0157: ```
0158:
0159: ## 说明
0160:
0161: - 热更新：`config.yaml` 与 `auths/` 变化会被自动侦测并应用。
0162: - 请求日志可通过管理 API 在运行时开关。
0163: - `gemini-web.*` 相关配置在内嵌服务器中会被遵循。
0164:

### FILE: docs/sdk-watcher.md
0001: # SDK Watcher Integration
0002:
0003: The SDK service exposes a watcher integration that surfaces granular auth updates without forcing a full reload. This document explains the queue contract, how the service consumes updates, and how high-frequency change bursts are handled.
0004:
0005: ## Update Queue Contract
0006:
0007: - `watcher.AuthUpdate` represents a single credential change. `Action` may be `add`, `modify`, or `delete`, and `ID` carries the credential identifier. For `add`/`modify` the `Auth` payload contains a fully populated clone of the credential; `delete` may omit `Auth`.
0008: - `WatcherWrapper.SetAuthUpdateQueue(chan<- watcher.AuthUpdate)` wires the queue produced by the SDK service into the watcher. The queue must be created before the watcher starts.
0009: - The service builds the queue via `ensureAuthUpdateQueue`, using a buffered channel (`capacity=256`) and a dedicated consumer goroutine (`consumeAuthUpdates`). The consumer drains bursts by looping through the backlog before reacquiring the select loop.
0010:
0011: ## Watcher Behaviour
0012:
0013: - `pkg/llmproxy/watcher/watcher.go` keeps a shadow snapshot of auth state (`currentAuths`). Each filesystem or configuration event triggers a recomputation and a diff against the previous snapshot to produce minimal `AuthUpdate` entries that mirror adds, edits, and removals.
0014: - Updates are coalesced per credential identifier. If multiple changes occur before dispatch (e.g., write followed by delete), only the final action is sent downstream.
0015: - The watcher runs an internal dispatch loop that buffers pending updates in memory and forwards them asynchronously to the queue. Producers never block on channel capacity; they just enqueue into the in-memory buffer and signal the dispatcher. Dispatch cancellation happens when the watcher stops, guaranteeing goroutines exit cleanly.
0016:
0017: ## High-Frequency Change Handling
0018:
0019: - The dispatch loop and service consumer run independently, preventing filesystem watchers from blocking even when many updates arrive at once.
0020: - Back-pressure is absorbed in two places:
0021:   - The dispatch buffer (map + order slice) coalesces repeated updates for the same credential until the consumer catches up.
0022:   - The service channel capacity (256) combined with the consumer drain loop ensures several bursts can be processed without oscillation.
0023: - If the queue is saturated for an extended period, updates continue to be merged, so the latest state is eventually applied without replaying redundant intermediate states.
0024:
0025: ## Usage Checklist
0026:
0027: 1. Instantiate the SDK service (builder or manual construction).
0028: 2. Call `ensureAuthUpdateQueue` before starting the watcher to allocate the shared channel.
0029: 3. When the `WatcherWrapper` is created, call `SetAuthUpdateQueue` with the service queue, then start the watcher.
0030: 4. Provide a reload callback that handles configuration updates; auth deltas will arrive via the queue and are applied by the service automatically through `handleAuthUpdate`.
0031:
0032: Following this flow keeps auth changes responsive while avoiding full reloads for every edit.

### FILE: docs/sdk-watcher_CN.md
0001: # SDK Watcher集成说明
0002:
0003: 本文档介绍SDK服务与文件监控器之间的增量更新队列，包括接口契约、高频变更下的处理策略以及接入步骤。
0004:
0005: ## 更新队列契约
0006:
0007: - `watcher.AuthUpdate`描述单条凭据变更，`Action`可能为`add`、`modify`或`delete`，`ID`是凭据标识。对于`add`/`modify`会携带完整的`Auth`克隆，`delete`可以省略`Auth`。
0008: - `WatcherWrapper.SetAuthUpdateQueue(chan<- watcher.AuthUpdate)`用于将服务侧创建的队列注入watcher，必须在watcher启动前完成。
0009: - 服务通过`ensureAuthUpdateQueue`创建容量为256的缓冲通道，并在`consumeAuthUpdates`中使用专职goroutine消费；消费侧会主动“抽干”积压事件，降低切换开销。
0010:
0011: ## Watcher行为
0012:
0013: - `pkg/llmproxy/watcher/watcher.go`维护`currentAuths`快照，文件或配置事件触发后会重建快照并与旧快照对比，生成最小化的`AuthUpdate`列表。
0014: - 以凭据ID为维度对更新进行合并，同一凭据在短时间内的多次变更只会保留最新状态（例如先写后删只会下发`delete`）。
0015: - watcher内部运行异步分发循环：生产者只向内存缓冲追加事件并唤醒分发协程，即使通道暂时写满也不会阻塞文件事件线程。watcher停止时会取消分发循环，确保协程正常退出。
0016:
0017: ## 高频变更处理
0018:
0019: - 分发循环与服务消费协程相互独立，因此即便短时间内出现大量变更也不会阻塞watcher事件处理。
0020: - 背压通过两级缓冲吸收：
0021:   - 分发缓冲（map + 顺序切片）会合并同一凭据的重复事件，直到消费者完成处理。
0022:   - 服务端通道的256容量加上消费侧的“抽干”逻辑，可平稳处理多个突发批次。
0023: - 当通道长时间处于高压状态时，缓冲仍持续合并事件，从而在消费者恢复后一次性应用最新状态，避免重复处理无意义的中间状态。
0024:
0025: ## 接入步骤
0026:
0027: 1. 实例化SDK Service（构建器或手工创建）。
0028: 2. 在启动watcher之前调用`ensureAuthUpdateQueue`创建共享通道。
0029: 3. watcher通过工厂函数创建后立刻调用`SetAuthUpdateQueue`注入通道，然后再启动watcher。
0030: 4. Reload回调专注于配置更新；认证增量会通过队列送达，并由`handleAuthUpdate`自动应用。
0031:
0032: 遵循上述流程即可在避免全量重载的同时保持凭据变更的实时性。

### FILE: examples/custom-provider/main.go
0001: // Package main demonstrates how to create a custom AI provider executor
0002: // and integrate it with the CLI Proxy API server. This example shows how to:
0003: // - Create a custom executor that implements the Executor interface
0004: // - Register custom translators for request/response transformation
0005: // - Integrate the custom provider with the SDK server
0006: // - Register custom models in the model registry
0007: //
0008: // This example uses a simple echo service (httpbin.org) as the upstream API
0009: // for demonstration purposes. In a real implementation, you would replace
0010: // this with your actual AI service provider.
0011: package main
0012:
0013: import (
0014: 	"bytes"
0015: 	"context"
0016: 	"errors"
0017: 	"fmt"
0018: 	"io"
0019: 	"net/http"
0020: 	"net/url"
0021: 	"os"
0022: 	"path/filepath"
0023: 	"strings"
0024: 	"time"
0025:
0026: 	"github.com/gin-gonic/gin"
0027: 	"github.com/router-for-me/CLIProxyAPI/v6/sdk/api"
0028: 	sdkAuth "github.com/router-for-me/CLIProxyAPI/v6/sdk/auth"
0029: 	"github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy"
0030: 	coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
0031: 	clipexec "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/executor"
0032: 	"github.com/router-for-me/CLIProxyAPI/v6/sdk/config"
0033: 	"github.com/router-for-me/CLIProxyAPI/v6/sdk/logging"
0034: 	sdktr "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
0035: )
0036:
0037: const (
0038: 	// providerKey is the identifier for our custom provider.
0039: 	providerKey = "myprov"
0040:
0041: 	// fOpenAI represents the OpenAI chat format.
0042: 	fOpenAI = sdktr.Format("openai.chat")
0043:
0044: 	// fMyProv represents our custom provider's chat format.
0045: 	fMyProv = sdktr.Format("myprov.chat")
0046: )
0047:
0048: // init registers trivial translators for demonstration purposes.
0049: // In a real implementation, you would implement proper request/response
0050: // transformation logic between OpenAI format and your provider's format.
0051: func init() {
0052: 	sdktr.Register(fOpenAI, fMyProv,
0053: 		func(model string, raw []byte, stream bool) []byte { return raw },
0054: 		sdktr.ResponseTransform{
0055: 			Stream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) []string {
0056: 				return []string{string(raw)}
0057: 			},
0058: 			NonStream: func(ctx context.Context, model string, originalReq, translatedReq, raw []byte, param *any) string {
0059: 				return string(raw)
0060: 			},
0061: 		},
0062: 	)
0063: }
0064:
0065: // MyExecutor is a minimal provider implementation for demonstration purposes.
0066: // It implements the Executor interface to handle requests to a custom AI provider.
0067: type MyExecutor struct{}
0068:
0069: // Identifier returns the unique identifier for this executor.
0070: func (MyExecutor) Identifier() string { return providerKey }
0071:
0072: // PrepareRequest optionally injects credentials to raw HTTP requests.
0073: // This method is called before each request to allow the executor to modify
0074: // the HTTP request with authentication headers or other necessary modifications.
0075: //
0076: // Parameters:
0077: //   - req: The HTTP request to prepare
0078: //   - a: The authentication information
0079: //
0080: // Returns:
0081: //   - error: An error if request preparation fails
0082: func (MyExecutor) PrepareRequest(req *http.Request, a *coreauth.Auth) error {
0083: 	if req == nil || a == nil {
0084: 		return nil
0085: 	}
0086: 	if a.Attributes != nil {
0087: 		if ak := strings.TrimSpace(a.Attributes["api_key"]); ak != "" {
0088: 			req.Header.Set("Authorization", "Bearer "+ak)
0089: 		}
0090: 	}
0091: 	return nil
0092: }
0093:
0094: func buildHTTPClient(a *coreauth.Auth) *http.Client {
0095: 	if a == nil || strings.TrimSpace(a.ProxyURL) == "" {
0096: 		return http.DefaultClient
0097: 	}
0098: 	u, err := url.Parse(a.ProxyURL)
0099: 	if err != nil || (u.Scheme != "http" && u.Scheme != "https") {
0100: 		return http.DefaultClient
0101: 	}
0102: 	return &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(u)}}
0103: }
0104:
0105: func upstreamEndpoint(a *coreauth.Auth) string {
0106: 	if a != nil && a.Attributes != nil {
0107: 		if ep := strings.TrimSpace(a.Attributes["endpoint"]); ep != "" {
0108: 			return ep
0109: 		}
0110: 	}
0111: 	// Demo echo endpoint; replace with your upstream.
0112: 	return "https://httpbin.org/post"
0113: }
0114:
0115: func (MyExecutor) Execute(ctx context.Context, a *coreauth.Auth, req clipexec.Request, opts clipexec.Options) (clipexec.Response, error) {
0116: 	client := buildHTTPClient(a)
0117: 	endpoint := upstreamEndpoint(a)
0118:
0119: 	httpReq, errNew := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(req.Payload))
0120: 	if errNew != nil {
0121: 		return clipexec.Response{}, errNew
0122: 	}
0123: 	httpReq.Header.Set("Content-Type", "application/json")
0124:
0125: 	// Inject credentials via PrepareRequest hook.
0126: 	if errPrep := (MyExecutor{}).PrepareRequest(httpReq, a); errPrep != nil {
0127: 		return clipexec.Response{}, errPrep
0128: 	}
0129:
0130: 	resp, errDo := client.Do(httpReq)
0131: 	if errDo != nil {
0132: 		return clipexec.Response{}, errDo
0133: 	}
0134: 	defer func() {
0135: 		if errClose := resp.Body.Close(); errClose != nil {
0136: 			fmt.Fprintf(os.Stderr, "close response body error: %v\n", errClose)
0137: 		}
0138: 	}()
0139: 	body, _ := io.ReadAll(resp.Body)
0140: 	return clipexec.Response{Payload: body}, nil
0141: }
0142:
0143: func (MyExecutor) HttpRequest(ctx context.Context, a *coreauth.Auth, req *http.Request) (*http.Response, error) {
0144: 	if req == nil {
0145: 		return nil, fmt.Errorf("myprov executor: request is nil")
0146: 	}
0147: 	if ctx == nil {
0148: 		ctx = req.Context()
0149: 	}
0150: 	httpReq := req.WithContext(ctx)
0151: 	if errPrep := (MyExecutor{}).PrepareRequest(httpReq, a); errPrep != nil {
0152: 		return nil, errPrep
0153: 	}
0154: 	client := buildHTTPClient(a)
0155: 	return client.Do(httpReq)
0156: }
0157:
0158: func (MyExecutor) CountTokens(context.Context, *coreauth.Auth, clipexec.Request, clipexec.Options) (clipexec.Response, error) {
0159: 	return clipexec.Response{}, errors.New("count tokens not implemented")
0160: }

### FILE: examples/http-request/main.go
0001: // Package main demonstrates how to use coreauth.Manager.HttpRequest/NewHttpRequest
0002: // to execute arbitrary HTTP requests with provider credentials injected.
0003: //
0004: // This example registers a minimal custom executor that injects an Authorization
0005: // header from auth.Attributes["api_key"], then performs two requests against
0006: // httpbin.org to show the injected headers.
0007: package main
0008:
0009: import (
0010: 	"bytes"
0011: 	"context"
0012: 	"errors"
0013: 	"fmt"
0014: 	"io"
0015: 	"net/http"
0016: 	"strings"
0017: 	"time"
0018:
0019: 	coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
0020: 	clipexec "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/executor"
0021: 	log "github.com/sirupsen/logrus"
0022: )
0023:
0024: const providerKey = "echo"
0025:
0026: // EchoExecutor is a minimal provider implementation for demonstration purposes.
0027: type EchoExecutor struct{}
0028:
0029: func (EchoExecutor) Identifier() string { return providerKey }
0030:
0031: func (EchoExecutor) PrepareRequest(req *http.Request, auth *coreauth.Auth) error {
0032: 	if req == nil || auth == nil {
0033: 		return nil
0034: 	}
0035: 	if auth.Attributes != nil {
0036: 		if apiKey := strings.TrimSpace(auth.Attributes["api_key"]); apiKey != "" {
0037: 			req.Header.Set("Authorization", "Bearer "+apiKey)
0038: 		}
0039: 	}
0040: 	return nil
0041: }
0042:
0043: func (EchoExecutor) HttpRequest(ctx context.Context, auth *coreauth.Auth, req *http.Request) (*http.Response, error) {
0044: 	if req == nil {
0045: 		return nil, fmt.Errorf("echo executor: request is nil")
0046: 	}
0047: 	if ctx == nil {
0048: 		ctx = req.Context()
0049: 	}
0050: 	httpReq := req.WithContext(ctx)
0051: 	if errPrep := (EchoExecutor{}).PrepareRequest(httpReq, auth); errPrep != nil {
0052: 		return nil, errPrep
0053: 	}
0054: 	return http.DefaultClient.Do(httpReq)
0055: }
0056:
0057: func (EchoExecutor) Execute(context.Context, *coreauth.Auth, clipexec.Request, clipexec.Options) (clipexec.Response, error) {
0058: 	return clipexec.Response{}, errors.New("echo executor: Execute not implemented")
0059: }
0060:
0061: func (EchoExecutor) ExecuteStream(context.Context, *coreauth.Auth, clipexec.Request, clipexec.Options) (*clipexec.StreamResult, error) {
0062: 	return nil, errors.New("echo executor: ExecuteStream not implemented")
0063: }
0064:
0065: func (EchoExecutor) Refresh(context.Context, *coreauth.Auth) (*coreauth.Auth, error) {
0066: 	return nil, errors.New("echo executor: Refresh not implemented")
0067: }
0068:
0069: func (EchoExecutor) CountTokens(context.Context, *coreauth.Auth, clipexec.Request, clipexec.Options) (clipexec.Response, error) {
0070: 	return clipexec.Response{}, errors.New("echo executor: CountTokens not implemented")
0071: }
0072:
0073: func (EchoExecutor) CloseExecutionSession(sessionID string) {}
0074:
0075: func main() {
0076: 	log.SetLevel(log.InfoLevel)
0077:
0078: 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
0079: 	defer cancel()
0080:
0081: 	core := coreauth.NewManager(nil, nil, nil)
0082: 	core.RegisterExecutor(EchoExecutor{})
0083:
0084: 	auth := &coreauth.Auth{
0085: 		ID:       "demo-echo",
0086: 		Provider: providerKey,
0087: 		Attributes: map[string]string{
0088: 			"api_key": "demo-api-key",
0089: 		},
0090: 	}
0091:
0092: 	// Example 1: Build a prepared request and execute it using your own http.Client.
0093: 	reqPrepared, errReqPrepared := core.NewHttpRequest(
0094: 		ctx,
0095: 		auth,
0096: 		http.MethodGet,
0097: 		"https://httpbin.org/anything",
0098: 		nil,
0099: 		http.Header{"X-Example": []string{"prepared"}},
0100: 	)
0101: 	if errReqPrepared != nil {
0102: 		panic(errReqPrepared)
0103: 	}
0104: 	respPrepared, errDoPrepared := http.DefaultClient.Do(reqPrepared)
0105: 	if errDoPrepared != nil {
0106: 		panic(errDoPrepared)
0107: 	}
0108: 	defer func() {
0109: 		if errClose := respPrepared.Body.Close(); errClose != nil {
0110: 			log.Errorf("close response body error: %v", errClose)
0111: 		}
0112: 	}()
0113: 	bodyPrepared, errReadPrepared := io.ReadAll(respPrepared.Body)
0114: 	if errReadPrepared != nil {
0115: 		panic(errReadPrepared)
0116: 	}
0117: 	fmt.Printf("Prepared request status: %d\n%s\n\n", respPrepared.StatusCode, bodyPrepared)
0118:
0119: 	// Example 2: Execute a raw request via core.HttpRequest (auto inject + do).
0120: 	rawBody := []byte(`{"hello":"world"}`)
0121: 	rawReq, errRawReq := http.NewRequestWithContext(ctx, http.MethodPost, "https://httpbin.org/anything", bytes.NewReader(rawBody))
0122: 	if errRawReq != nil {
0123: 		panic(errRawReq)
0124: 	}
0125: 	rawReq.Header.Set("Content-Type", "application/json")
0126: 	rawReq.Header.Set("X-Example", "executed")
0127:
0128: 	respExec, errDoExec := core.HttpRequest(ctx, auth, rawReq)
0129: 	if errDoExec != nil {
0130: 		panic(errDoExec)
0131: 	}
0132: 	defer func() {
0133: 		if errClose := respExec.Body.Close(); errClose != nil {
0134: 			log.Errorf("close response body error: %v", errClose)
0135: 		}
0136: 	}()
0137: 	bodyExec, errReadExec := io.ReadAll(respExec.Body)
0138: 	if errReadExec != nil {
0139: 		panic(errReadExec)
0140: 	}
0141: 	fmt.Printf("Manager HttpRequest status: %d\n%s\n", respExec.StatusCode, bodyExec)
0142: }

### FILE: examples/translator/main.go
0001: package main
0002:
0003: import (
0004: 	"context"
0005: 	"fmt"
0006:
0007: 	"github.com/router-for-me/CLIProxyAPI/v6/sdk/translator"
0008: 	_ "github.com/router-for-me/CLIProxyAPI/v6/sdk/translator/builtin"
0009: )
0010:
0011: func main() {
0012: 	rawRequest := []byte(`{"messages":[{"content":[{"text":"Hello! Gemini","type":"text"}],"role":"user"}],"model":"gemini-2.5-pro","stream":false}`)
0013: 	fmt.Println("Has gemini->openai response translator:", translator.HasResponseTransformerByFormatName(
0014: 		translator.FormatGemini,
0015: 		translator.FormatOpenAI,
0016: 	))
0017:
0018: 	translatedRequest := translator.TranslateRequestByFormatName(
0019: 		translator.FormatOpenAI,
0020: 		translator.FormatGemini,
0021: 		"gemini-2.5-pro",
0022: 		rawRequest,
0023: 		false,
0024: 	)
0025:
0026: 	fmt.Printf("Translated request to Gemini format:\n%s\n\n", translatedRequest)
0027:
0028: 	claudeResponse := []byte(`{"candidates":[{"content":{"role":"model","parts":[{"thought":true,"text":"Okay, here's what's going through my mind. I need to schedule a meeting"},{"thoughtSignature":"","functionCall":{"name":"schedule_meeting","args":{"topic":"Q3 planning","attendees":["Bob","Alice"],"time":"10:00","date":"2025-03-27"}}}]},"finishReason":"STOP","avgLogprobs":-0.50018133435930523}],"usageMetadata":{"promptTokenCount":117,"candidatesTokenCount":28,"totalTokenCount":474,"trafficType":"PROVISIONED_THROUGHPUT","promptTokensDetails":[{"modality":"TEXT","tokenCount":117}],"candidatesTokensDetails":[{"modality":"TEXT","tokenCount":28}],"thoughtsTokenCount":329},"modelVersion":"gemini-2.5-pro","createTime":"2025-08-15T04:12:55.249090Z","responseId":"x7OeaIKaD6CU48APvNXDyA4"}`)
0029:
0030: 	convertedResponse := translator.TranslateNonStreamByFormatName(
0031: 		context.Background(),
0032: 		translator.FormatGemini,
0033: 		translator.FormatOpenAI,
0034: 		"gemini-2.5-pro",
0035: 		rawRequest,
0036: 		translatedRequest,
0037: 		claudeResponse,
0038: 		nil,
0039: 	)
0040:
0041: 	fmt.Printf("Converted response for OpenAI clients:\n%s\n", convertedResponse)
0042: }

### FILE: pkg/llmproxy/access/config_access/provider.go
0001: package configaccess
0002:
0003: import (
0004: 	"context"
0005: 	"net/http"
0006: 	"strings"
0007:
0008: 	sdkaccess "github.com/router-for-me/CLIProxyAPI/v6/sdk/access"
0009: 	sdkconfig "github.com/router-for-me/CLIProxyAPI/v6/sdk/config"
0010: )
0011:
0012: // Register ensures the config-access provider is available to the access manager.
0013: func Register(cfg *sdkconfig.SDKConfig) {
0014: 	if cfg == nil {
0015: 		sdkaccess.UnregisterProvider(sdkaccess.AccessProviderTypeConfigAPIKey)
0016: 		return
0017: 	}
0018:
0019: 	keys := normalizeKeys(cfg.APIKeys)
0020: 	if len(keys) == 0 {
0021: 		sdkaccess.UnregisterProvider(sdkaccess.AccessProviderTypeConfigAPIKey)
0022: 		return
0023: 	}
0024:
0025: 	sdkaccess.RegisterProvider(
0026: 		sdkaccess.AccessProviderTypeConfigAPIKey,
0027: 		newProvider(sdkaccess.DefaultAccessProviderName, keys),
0028: 	)
0029: }
0030:
0031: type provider struct {
0032: 	name string
0033: 	keys map[string]struct{}
0034: }
0035:
0036: func newProvider(name string, keys []string) *provider {
0037: 	providerName := strings.TrimSpace(name)
0038: 	if providerName == "" {
0039: 		providerName = sdkaccess.DefaultAccessProviderName
0040: 	}
0041: 	keySet := make(map[string]struct{}, len(keys))
0042: 	for _, key := range keys {
0043: 		keySet[key] = struct{}{}
0044: 	}
0045: 	return &provider{name: providerName, keys: keySet}
0046: }
0047:
0048: func (p *provider) Identifier() string {
0049: 	if p == nil || p.name == "" {
0050: 		return sdkaccess.DefaultAccessProviderName
0051: 	}
0052: 	return p.name
0053: }
0054:
0055: func (p *provider) Authenticate(_ context.Context, r *http.Request) (*sdkaccess.Result, *sdkaccess.AuthError) {
0056: 	if p == nil {
0057: 		return nil, sdkaccess.NewNotHandledError()
0058: 	}
0059: 	if len(p.keys) == 0 {
0060: 		return nil, sdkaccess.NewNotHandledError()
0061: 	}
0062: 	authHeader := r.Header.Get("Authorization")
0063: 	authHeaderGoogle := r.Header.Get("X-Goog-Api-Key")
0064: 	authHeaderAnthropic := r.Header.Get("X-Api-Key")
0065: 	queryKey := ""
0066: 	queryAuthToken := ""
0067: 	if r.URL != nil {
0068: 		queryKey = r.URL.Query().Get("key")
0069: 		queryAuthToken = r.URL.Query().Get("auth_token")
0070: 	}
0071: 	if authHeader == "" && authHeaderGoogle == "" && authHeaderAnthropic == "" && queryKey == "" && queryAuthToken == "" {
0072: 		return nil, sdkaccess.NewNoCredentialsError()
0073: 	}
0074:
0075: 	apiKey := extractBearerToken(authHeader)
0076:
0077: 	candidates := []struct {
0078: 		value  string
0079: 		source string
0080: 	}{
0081: 		{apiKey, "authorization"},
0082: 		{authHeaderGoogle, "x-goog-api-key"},
0083: 		{authHeaderAnthropic, "x-api-key"},
0084: 		{queryKey, "query-key"},
0085: 		{queryAuthToken, "query-auth-token"},
0086: 	}
0087:
0088: 	for _, candidate := range candidates {
0089: 		if candidate.value == "" {
0090: 			continue
0091: 		}
0092: 		if _, ok := p.keys[candidate.value]; ok {
0093: 			return &sdkaccess.Result{
0094: 				Provider:  p.Identifier(),
0095: 				Principal: candidate.value,
0096: 				Metadata: map[string]string{
0097: 					"source": candidate.source,
0098: 				},
0099: 			}, nil
0100: 		}
0101: 	}
0102:
0103: 	return nil, sdkaccess.NewInvalidCredentialError()
0104: }
0105:
0106: func extractBearerToken(header string) string {
0107: 	if header == "" {
0108: 		return ""
0109: 	}
0110: 	parts := strings.SplitN(header, " ", 2)
0111: 	if len(parts) != 2 {
0112: 		return header
0113: 	}
0114: 	if strings.ToLower(parts[0]) != "bearer" {
0115: 		return header
0116: 	}
0117: 	return strings.TrimSpace(parts[1])
0118: }
0119:
0120: func normalizeKeys(keys []string) []string {
0121: 	if len(keys) == 0 {
0122: 		return nil
0123: 	}
0124: 	normalized := make([]string, 0, len(keys))
0125: 	seen := make(map[string]struct{}, len(keys))
0126: 	for _, key := range keys {
0127: 		trimmedKey := strings.TrimSpace(key)
0128: 		if trimmedKey == "" {
0129: 			continue
0130: 		}
0131: 		if _, exists := seen[trimmedKey]; exists {
0132: 			continue
0133: 		}
0134: 		seen[trimmedKey] = struct{}{}
0135: 		normalized = append(normalized, trimmedKey)
0136: 	}
0137: 	if len(normalized) == 0 {
0138: 		return nil
0139: 	}
0140: 	return normalized
0141: }

### FILE: pkg/llmproxy/access/config_access/provider_test.go
0001: package configaccess
0002:
0003: import (
0004: 	"context"
0005: 	"net/http/httptest"
0006: 	"testing"
0007:
0008: 	sdkaccess "github.com/router-for-me/CLIProxyAPI/v6/sdk/access"
0009: 	sdkconfig "github.com/router-for-me/CLIProxyAPI/v6/sdk/config"
0010: )
0011:
0012: func findProvider() sdkaccess.Provider {
0013: 	providers := sdkaccess.RegisteredProviders()
0014: 	for _, p := range providers {
0015: 		if p.Identifier() == sdkaccess.DefaultAccessProviderName {
0016: 			return p
0017: 		}
0018: 	}
0019: 	return nil
0020: }
0021:
0022: func TestRegister(t *testing.T) {
0023: 	// Test nil config
0024: 	Register(nil)
0025: 	if findProvider() != nil {
0026: 		t.Errorf("expected provider to be unregistered for nil config")
0027: 	}
0028:
0029: 	// Test empty keys
0030: 	cfg := &sdkconfig.SDKConfig{APIKeys: []string{}}
0031: 	Register(cfg)
0032: 	if findProvider() != nil {
0033: 		t.Errorf("expected provider to be unregistered for empty keys")
0034: 	}
0035:
0036: 	// Test valid keys
0037: 	cfg.APIKeys = []string{"key1"}
0038: 	Register(cfg)
0039: 	p := findProvider()
0040: 	if p == nil {
0041: 		t.Fatalf("expected provider to be registered")
0042: 	}
0043: 	if p.Identifier() != sdkaccess.DefaultAccessProviderName {
0044: 		t.Errorf("expected identifier %q, got %q", sdkaccess.DefaultAccessProviderName, p.Identifier())
0045: 	}
0046: }
0047:
0048: func TestProvider_Authenticate(t *testing.T) {
0049: 	p := newProvider("test-provider", []string{"valid-key"})
0050: 	ctx := context.Background()
0051:
0052: 	tests := []struct {
0053: 		name       string
0054: 		headers    map[string]string
0055: 		query      string
0056: 		wantResult bool
0057: 		wantError  sdkaccess.AuthErrorCode
0058: 	}{
0059: 		{
0060: 			name:       "valid bearer token",
0061: 			headers:    map[string]string{"Authorization": "Bearer valid-key"},
0062: 			wantResult: true,
0063: 		},
0064: 		{
0065: 			name:       "valid plain token",
0066: 			headers:    map[string]string{"Authorization": "valid-key"},
0067: 			wantResult: true,
0068: 		},
0069: 		{
0070: 			name:       "valid google header",
0071: 			headers:    map[string]string{"X-Goog-Api-Key": "valid-key"},
0072: 			wantResult: true,
0073: 		},
0074: 		{
0075: 			name:       "valid anthropic header",
0076: 			headers:    map[string]string{"X-Api-Key": "valid-key"},
0077: 			wantResult: true,
0078: 		},
0079: 		{
0080: 			name:       "valid query key",
0081: 			query:      "?key=valid-key",
0082: 			wantResult: true,
0083: 		},
0084: 		{
0085: 			name:       "valid query auth_token",
0086: 			query:      "?auth_token=valid-key",
0087: 			wantResult: true,
0088: 		},
0089: 		{
0090: 			name:       "invalid token",
0091: 			headers:    map[string]string{"Authorization": "Bearer invalid-key"},
0092: 			wantResult: false,
0093: 			wantError:  sdkaccess.AuthErrorCodeInvalidCredential,
0094: 		},
0095: 		{
0096: 			name:       "no credentials",
0097: 			wantResult: false,
0098: 			wantError:  sdkaccess.AuthErrorCodeNoCredentials,
0099: 		},
0100: 	}
0101:
0102: 	for _, tt := range tests {
0103: 		t.Run(tt.name, func(t *testing.T) {
0104: 			req := httptest.NewRequest("GET", "/"+tt.query, nil)
0105: 			for k, v := range tt.headers {
0106: 				req.Header.Set(k, v)
0107: 			}
0108:
0109: 			res, err := p.Authenticate(ctx, req)
0110: 			if tt.wantResult {
0111: 				if err != nil {
0112: 					t.Errorf("unexpected error: %v", err)
0113: 				}
0114: 				if res == nil {
0115: 					t.Errorf("expected result, got nil")
0116: 				} else if res.Principal != "valid-key" {
0117: 					t.Errorf("expected principal valid-key, got %q", res.Principal)
0118: 				}
0119: 			} else {
0120: 				if err == nil {
0121: 					t.Errorf("expected error, got nil")
0122: 				} else if err.Code != tt.wantError {
0123: 					t.Errorf("expected error code %v, got %v", tt.wantError, err.Code)
0124: 				}
0125: 			}
0126: 		})
0127: 	}
0128: }
0129:
0130: func TestExtractBearerToken(t *testing.T) {
0131: 	cases := []struct {
0132: 		header string
0133: 		want   string
0134: 	}{
0135: 		{"", ""},
0136: 		{"valid-key", "valid-key"},
0137: 		{"Bearer valid-key", "valid-key"},
0138: 		{"bearer valid-key", "valid-key"},
0139: 		{"BEARER valid-key", "valid-key"},
0140: 		{"Bearer  valid-key ", "valid-key"},
0141: 		{"Other token", "Other token"},
0142: 	}
0143: 	for _, tc := range cases {
0144: 		got := extractBearerToken(tc.header)
0145: 		if got != tc.want {
0146: 			t.Errorf("extractBearerToken(%q) = %q, want %q", tc.header, got, tc.want)
0147: 		}
0148: 	}
0149: }
0150:
0151: func TestNormalizeKeys(t *testing.T) {
0152: 	cases := []struct {
0153: 		keys []string
0154: 		want []string
0155: 	}{
0156: 		{nil, nil},
0157: 		{[]string{}, nil},
0158: 		{[]string{" "}, nil},
0159: 		{[]string{" key1 ", "key2", "key1"}, []string{"key1", "key2"}},
0160: 	}

### FILE: pkg/llmproxy/access/reconcile.go
0001: package access
0002:
0003: import (
0004: 	"fmt"
0005: 	"reflect"
0006: 	"sort"
0007: 	"strings"
0008:
0009: 	configaccess "github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/access/config_access"
0010: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/config"
0011: 	sdkaccess "github.com/router-for-me/CLIProxyAPI/v6/sdk/access"
0012: 	log "github.com/sirupsen/logrus"
0013: )
0014:
0015: // ReconcileProviders builds the desired provider list by reusing existing providers when possible
0016: // and creating or removing providers only when their configuration changed. It returns the final
0017: // ordered provider slice along with the identifiers of providers that were added, updated, or
0018: // removed compared to the previous configuration.
0019: func ReconcileProviders(oldCfg, newCfg *config.Config, existing []sdkaccess.Provider) (result []sdkaccess.Provider, added, updated, removed []string, err error) {
0020: 	_ = oldCfg
0021: 	if newCfg == nil {
0022: 		return nil, nil, nil, nil, nil
0023: 	}
0024:
0025: 	result = sdkaccess.RegisteredProviders()
0026:
0027: 	existingMap := make(map[string]sdkaccess.Provider, len(existing))
0028: 	for _, provider := range existing {
0029: 		providerID := identifierFromProvider(provider)
0030: 		if providerID == "" {
0031: 			continue
0032: 		}
0033: 		existingMap[providerID] = provider
0034: 	}
0035:
0036: 	finalIDs := make(map[string]struct{}, len(result))
0037:
0038: 	isInlineProvider := func(id string) bool {
0039: 		return strings.EqualFold(id, sdkaccess.DefaultAccessProviderName)
0040: 	}
0041: 	appendChange := func(list *[]string, id string) {
0042: 		if isInlineProvider(id) {
0043: 			return
0044: 		}
0045: 		*list = append(*list, id)
0046: 	}
0047:
0048: 	for _, provider := range result {
0049: 		providerID := identifierFromProvider(provider)
0050: 		if providerID == "" {
0051: 			continue
0052: 		}
0053: 		finalIDs[providerID] = struct{}{}
0054:
0055: 		existingProvider, exists := existingMap[providerID]
0056: 		if !exists {
0057: 			appendChange(&added, providerID)
0058: 			continue
0059: 		}
0060: 		if !providerInstanceEqual(existingProvider, provider) {
0061: 			appendChange(&updated, providerID)
0062: 		}
0063: 	}
0064:
0065: 	for providerID := range existingMap {
0066: 		if _, exists := finalIDs[providerID]; exists {
0067: 			continue
0068: 		}
0069: 		appendChange(&removed, providerID)
0070: 	}
0071:
0072: 	sort.Strings(added)
0073: 	sort.Strings(updated)
0074: 	sort.Strings(removed)
0075:
0076: 	return result, added, updated, removed, nil
0077: }
0078:
0079: // ApplyAccessProviders reconciles the configured access providers against the
0080: // currently registered providers and updates the manager. It logs a concise
0081: // summary of the detected changes and returns whether any provider changed.
0082: func ApplyAccessProviders(manager *sdkaccess.Manager, oldCfg, newCfg *config.Config) (bool, error) {
0083: 	if manager == nil || newCfg == nil {
0084: 		return false, nil
0085: 	}
0086:
0087: 	existing := manager.Providers()
0088: 	configaccess.Register(&newCfg.SDKConfig)
0089: 	providers, added, updated, removed, err := ReconcileProviders(oldCfg, newCfg, existing)
0090: 	if err != nil {
0091: 		log.Errorf("failed to reconcile request auth providers: %v", err)
0092: 		return false, fmt.Errorf("reconciling access providers: %w", err)
0093: 	}
0094:
0095: 	manager.SetProviders(providers)
0096:
0097: 	if len(added)+len(updated)+len(removed) > 0 {
0098: 		log.Debugf("auth providers reconciled (added=%d updated=%d removed=%d)", len(added), len(updated), len(removed))
0099: 		log.Debugf("auth providers changes details - added=%v updated=%v removed=%v", added, updated, removed)
0100: 		return true, nil
0101: 	}
0102:
0103: 	log.Debug("auth providers unchanged after config update")
0104: 	return false, nil
0105: }
0106:
0107: func identifierFromProvider(provider sdkaccess.Provider) string {
0108: 	if provider == nil {
0109: 		return ""
0110: 	}
0111: 	return strings.TrimSpace(provider.Identifier())
0112: }
0113:
0114: func providerInstanceEqual(a, b sdkaccess.Provider) bool {
0115: 	if a == nil || b == nil {
0116: 		return a == nil && b == nil
0117: 	}
0118: 	if reflect.TypeOf(a) != reflect.TypeOf(b) {
0119: 		return false
0120: 	}
0121: 	valueA := reflect.ValueOf(a)
0122: 	valueB := reflect.ValueOf(b)
0123: 	if valueA.Kind() == reflect.Pointer && valueB.Kind() == reflect.Pointer {
0124: 		return valueA.Pointer() == valueB.Pointer()
0125: 	}
0126: 	return reflect.DeepEqual(a, b)
0127: }

### FILE: pkg/llmproxy/api/handlers/management/api_tools.go
0001: package management
0002:
0003: import (
0004: 	"bytes"
0005: 	"context"
0006: 	"encoding/json"
0007: 	"fmt"
0008: 	"io"
0009: 	"net"
0010: 	"net/http"
0011: 	"net/url"
0012: 	"strings"
0013: 	"time"
0014:
0015: 	"github.com/fxamacker/cbor/v2"
0016: 	"github.com/gin-gonic/gin"
0017: 	log "github.com/sirupsen/logrus"
0018: 	"golang.org/x/net/proxy"
0019: 	"golang.org/x/oauth2"
0020: 	"golang.org/x/oauth2/google"
0021:
0022: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/runtime/geminicli"
0023: 	coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
0024: )
0025:
0026: const defaultAPICallTimeout = 60 * time.Second
0027:
0028: const (
0029: 	geminiOAuthClientID     = "681255809395-oo8ft2oprdrnp9e3aqf6av3hmdib135j.apps.googleusercontent.com"
0030: 	geminiOAuthClientSecret = "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"
0031: )
0032:
0033: var geminiOAuthScopes = []string{
0034: 	"https://www.googleapis.com/auth/cloud-platform",
0035: 	"https://www.googleapis.com/auth/userinfo.email",
0036: 	"https://www.googleapis.com/auth/userinfo.profile",
0037: }
0038:
0039: const (
0040: 	antigravityOAuthClientID     = "1071006060591-tmhssin2h21lcre235vtolojh4g403ep.apps.googleusercontent.com"
0041: 	antigravityOAuthClientSecret = "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"
0042: )
0043:
0044: var antigravityOAuthTokenURL = "https://oauth2.googleapis.com/token"
0045:
0046: type apiCallRequest struct {
0047: 	AuthIndexSnake  *string           `json:"auth_index"`
0048: 	AuthIndexCamel  *string           `json:"authIndex"`
0049: 	AuthIndexPascal *string           `json:"AuthIndex"`
0050: 	Method          string            `json:"method"`
0051: 	URL             string            `json:"url"`
0052: 	Header          map[string]string `json:"header"`
0053: 	Data            string            `json:"data"`
0054: }
0055:
0056: type apiCallResponse struct {
0057: 	StatusCode int                 `json:"status_code"`
0058: 	Header     map[string][]string `json:"header"`
0059: 	Body       string              `json:"body"`
0060: 	Quota      *QuotaSnapshots     `json:"quota,omitempty"`
0061: }
0062:
0063: // APICall makes a generic HTTP request on behalf of the management API caller.
0064: // It is protected by the management middleware.
0065: //
0066: // Endpoint:
0067: //
0068: //	POST /v0/management/api-call
0069: //
0070: // Authentication:
0071: //
0072: //	Same as other management APIs (requires a management key and remote-management rules).
0073: //	You can provide the key via:
0074: //	- Authorization: Bearer <key>
0075: //	- X-Management-Key: <key>
0076: //
0077: // Request JSON (supports both application/json and application/cbor):
0078: //   - auth_index / authIndex / AuthIndex (optional):
0079: //     The credential "auth_index" from GET /v0/management/auth-files (or other endpoints returning it).
0080: //     If omitted or not found, credential-specific proxy/token substitution is skipped.
0081: //   - method (required): HTTP method, e.g. GET, POST, PUT, PATCH, DELETE.
0082: //   - url (required): Absolute URL including scheme and host, e.g. "https://api.example.com/v1/ping".
0083: //   - header (optional): Request headers map.
0084: //     Supports magic variable "$TOKEN$" which is replaced using the selected credential:
0085: //     1) metadata.access_token
0086: //     2) attributes.api_key
0087: //     3) metadata.token / metadata.id_token / metadata.cookie
0088: //     Example: {"Authorization":"Bearer $TOKEN$"}.
0089: //     Note: if you need to override the HTTP Host header, set header["Host"].
0090: //   - data (optional): Raw request body as string (useful for POST/PUT/PATCH).
0091: //
0092: // Proxy selection (highest priority first):
0093: //  1. Selected credential proxy_url
0094: //  2. Global config proxy-url
0095: //  3. Direct connect (environment proxies are not used)
0096: //
0097: // Response (returned with HTTP 200 when the APICall itself succeeds):
0098: //
0099: //	Format matches request Content-Type (application/json or application/cbor)
0100: //	- status_code: Upstream HTTP status code.
0101: //	- header: Upstream response headers.
0102: //	- body: Upstream response body as string.
0103: //	- quota (optional): For GitHub Copilot enterprise accounts, contains quota_snapshots
0104: //	  with details for chat, completions, and premium_interactions.
0105: //
0106: // Example:
0107: //
0108: //	curl -sS -X POST "http://127.0.0.1:8317/v0/management/api-call" \
0109: //	  -H "Authorization: Bearer <MANAGEMENT_KEY>" \
0110: //	  -H "Content-Type: application/json" \
0111: //	  -d '{"auth_index":"<AUTH_INDEX>","method":"GET","url":"https://api.example.com/v1/ping","header":{"Authorization":"Bearer $TOKEN$"}}'
0112: //
0113: //	curl -sS -X POST "http://127.0.0.1:8317/v0/management/api-call" \
0114: //	  -H "Authorization: Bearer 831227" \
0115: //	  -H "Content-Type: application/json" \
0116: //	  -d '{"auth_index":"<AUTH_INDEX>","method":"POST","url":"https://api.example.com/v1/fetchAvailableModels","header":{"Authorization":"Bearer $TOKEN$","Content-Type":"application/json","User-Agent":"cliproxyapi"},"data":"{}"}'
0117: func (h *Handler) APICall(c *gin.Context) {
0118: 	// Detect content type
0119: 	contentType := strings.ToLower(strings.TrimSpace(c.GetHeader("Content-Type")))
0120: 	isCBOR := strings.Contains(contentType, "application/cbor")
0121:
0122: 	var body apiCallRequest
0123:
0124: 	// Parse request body based on content type
0125: 	if isCBOR {
0126: 		rawBody, errRead := io.ReadAll(c.Request.Body)
0127: 		if errRead != nil {
0128: 			c.JSON(http.StatusBadRequest, gin.H{"error": "failed to read request body"})
0129: 			return
0130: 		}
0131: 		if errUnmarshal := cbor.Unmarshal(rawBody, &body); errUnmarshal != nil {
0132: 			c.JSON(http.StatusBadRequest, gin.H{"error": "invalid cbor body"})
0133: 			return
0134: 		}
0135: 	} else {
0136: 		if errBindJSON := c.ShouldBindJSON(&body); errBindJSON != nil {
0137: 			c.JSON(http.StatusBadRequest, gin.H{"error": "invalid body"})
0138: 			return
0139: 		}
0140: 	}
0141:
0142: 	method := strings.ToUpper(strings.TrimSpace(body.Method))
0143: 	if method == "" {
0144: 		c.JSON(http.StatusBadRequest, gin.H{"error": "missing method"})
0145: 		return
0146: 	}
0147:
0148: 	urlStr := strings.TrimSpace(body.URL)
0149: 	if urlStr == "" {
0150: 		c.JSON(http.StatusBadRequest, gin.H{"error": "missing url"})
0151: 		return
0152: 	}
0153: 	parsedURL, errParseURL := url.Parse(urlStr)
0154: 	if errParseURL != nil || parsedURL.Scheme == "" || parsedURL.Host == "" {
0155: 		c.JSON(http.StatusBadRequest, gin.H{"error": "invalid url"})
0156: 		return
0157: 	}
0158:
0159: 	authIndex := firstNonEmptyString(body.AuthIndexSnake, body.AuthIndexCamel, body.AuthIndexPascal)
0160: 	auth := h.authByIndex(authIndex)

### FILE: pkg/llmproxy/api/handlers/management/api_tools_cbor_test.go
0001: package management
0002:
0003: import (
0004: 	"bytes"
0005: 	"encoding/json"
0006: 	"net/http"
0007: 	"net/http/httptest"
0008: 	"testing"
0009:
0010: 	"github.com/fxamacker/cbor/v2"
0011: 	"github.com/gin-gonic/gin"
0012: )
0013:
0014: func TestAPICall_CBOR_Support(t *testing.T) {
0015: 	gin.SetMode(gin.TestMode)
0016:
0017: 	// Create a test handler
0018: 	h := &Handler{}
0019:
0020: 	// Create test request data
0021: 	reqData := apiCallRequest{
0022: 		Method: "GET",
0023: 		URL:    "https://httpbin.org/get",
0024: 		Header: map[string]string{
0025: 			"User-Agent": "test-client",
0026: 		},
0027: 	}
0028:
0029: 	t.Run("JSON request and response", func(t *testing.T) {
0030: 		// Marshal request as JSON
0031: 		jsonData, err := json.Marshal(reqData)
0032: 		if err != nil {
0033: 			t.Fatalf("Failed to marshal JSON: %v", err)
0034: 		}
0035:
0036: 		// Create HTTP request
0037: 		req := httptest.NewRequest(http.MethodPost, "/v0/management/api-call", bytes.NewReader(jsonData))
0038: 		req.Header.Set("Content-Type", "application/json")
0039:
0040: 		// Create response recorder
0041: 		w := httptest.NewRecorder()
0042:
0043: 		// Create Gin context
0044: 		c, _ := gin.CreateTestContext(w)
0045: 		c.Request = req
0046:
0047: 		// Call handler
0048: 		h.APICall(c)
0049:
0050: 		// Verify response
0051: 		if w.Code != http.StatusOK && w.Code != http.StatusBadGateway {
0052: 			t.Logf("Response status: %d", w.Code)
0053: 			t.Logf("Response body: %s", w.Body.String())
0054: 		}
0055:
0056: 		// Check content type
0057: 		contentType := w.Header().Get("Content-Type")
0058: 		if w.Code == http.StatusOK && !contains(contentType, "application/json") {
0059: 			t.Errorf("Expected JSON response, got: %s", contentType)
0060: 		}
0061: 	})
0062:
0063: 	t.Run("CBOR request and response", func(t *testing.T) {
0064: 		// Marshal request as CBOR
0065: 		cborData, err := cbor.Marshal(reqData)
0066: 		if err != nil {
0067: 			t.Fatalf("Failed to marshal CBOR: %v", err)
0068: 		}
0069:
0070: 		// Create HTTP request
0071: 		req := httptest.NewRequest(http.MethodPost, "/v0/management/api-call", bytes.NewReader(cborData))
0072: 		req.Header.Set("Content-Type", "application/cbor")
0073:
0074: 		// Create response recorder
0075: 		w := httptest.NewRecorder()
0076:
0077: 		// Create Gin context
0078: 		c, _ := gin.CreateTestContext(w)
0079: 		c.Request = req
0080:
0081: 		// Call handler
0082: 		h.APICall(c)
0083:
0084: 		// Verify response
0085: 		if w.Code != http.StatusOK && w.Code != http.StatusBadGateway {
0086: 			t.Logf("Response status: %d", w.Code)
0087: 			t.Logf("Response body: %s", w.Body.String())
0088: 		}
0089:
0090: 		// Check content type
0091: 		contentType := w.Header().Get("Content-Type")
0092: 		if w.Code == http.StatusOK && !contains(contentType, "application/cbor") {
0093: 			t.Errorf("Expected CBOR response, got: %s", contentType)
0094: 		}
0095:
0096: 		// Try to decode CBOR response
0097: 		if w.Code == http.StatusOK {
0098: 			var response apiCallResponse
0099: 			if err := cbor.Unmarshal(w.Body.Bytes(), &response); err != nil {
0100: 				t.Errorf("Failed to unmarshal CBOR response: %v", err)
0101: 			} else {
0102: 				t.Logf("CBOR response decoded successfully: status_code=%d", response.StatusCode)
0103: 			}
0104: 		}
0105: 	})
0106:
0107: 	t.Run("CBOR encoding and decoding consistency", func(t *testing.T) {
0108: 		// Test data
0109: 		testReq := apiCallRequest{
0110: 			Method: "POST",
0111: 			URL:    "https://example.com/api",
0112: 			Header: map[string]string{
0113: 				"Authorization": "Bearer $TOKEN$",
0114: 				"Content-Type":  "application/json",
0115: 			},
0116: 			Data: `{"key":"value"}`,
0117: 		}
0118:
0119: 		// Encode to CBOR
0120: 		cborData, err := cbor.Marshal(testReq)
0121: 		if err != nil {
0122: 			t.Fatalf("Failed to marshal to CBOR: %v", err)
0123: 		}
0124:
0125: 		// Decode from CBOR
0126: 		var decoded apiCallRequest
0127: 		if err := cbor.Unmarshal(cborData, &decoded); err != nil {
0128: 			t.Fatalf("Failed to unmarshal from CBOR: %v", err)
0129: 		}
0130:
0131: 		// Verify fields
0132: 		if decoded.Method != testReq.Method {
0133: 			t.Errorf("Method mismatch: got %s, want %s", decoded.Method, testReq.Method)
0134: 		}
0135: 		if decoded.URL != testReq.URL {
0136: 			t.Errorf("URL mismatch: got %s, want %s", decoded.URL, testReq.URL)
0137: 		}
0138: 		if decoded.Data != testReq.Data {
0139: 			t.Errorf("Data mismatch: got %s, want %s", decoded.Data, testReq.Data)
0140: 		}
0141: 		if len(decoded.Header) != len(testReq.Header) {
0142: 			t.Errorf("Header count mismatch: got %d, want %d", len(decoded.Header), len(testReq.Header))
0143: 		}
0144: 	})
0145: }
0146:
0147: func contains(s, substr string) bool {
0148: 	return len(s) > 0 && len(substr) > 0 && (s == substr || len(s) >= len(substr) && s[:len(substr)] == substr || bytes.Contains([]byte(s), []byte(substr)))
0149: }

### FILE: pkg/llmproxy/api/handlers/management/api_tools_test.go
0001: package management
0002:
0003: import (
0004: 	"context"
0005: 	"encoding/json"
0006: 	"io"
0007: 	"net/http"
0008: 	"net/http/httptest"
0009: 	"net/url"
0010: 	"strings"
0011: 	"sync"
0012: 	"testing"
0013: 	"time"
0014:
0015: 	coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
0016: )
0017:
0018: type memoryAuthStore struct {
0019: 	mu    sync.Mutex
0020: 	items map[string]*coreauth.Auth
0021: }
0022:
0023: func (s *memoryAuthStore) List(ctx context.Context) ([]*coreauth.Auth, error) {
0024: 	_ = ctx
0025: 	s.mu.Lock()
0026: 	defer s.mu.Unlock()
0027: 	out := make([]*coreauth.Auth, 0, len(s.items))
0028: 	for _, a := range s.items {
0029: 		out = append(out, a.Clone())
0030: 	}
0031: 	return out, nil
0032: }
0033:
0034: func (s *memoryAuthStore) Save(ctx context.Context, auth *coreauth.Auth) (string, error) {
0035: 	_ = ctx
0036: 	if auth == nil {
0037: 		return "", nil
0038: 	}
0039: 	s.mu.Lock()
0040: 	if s.items == nil {
0041: 		s.items = make(map[string]*coreauth.Auth)
0042: 	}
0043: 	s.items[auth.ID] = auth.Clone()
0044: 	s.mu.Unlock()
0045: 	return auth.ID, nil
0046: }
0047:
0048: func (s *memoryAuthStore) Delete(ctx context.Context, id string) error {
0049: 	_ = ctx
0050: 	s.mu.Lock()
0051: 	delete(s.items, id)
0052: 	s.mu.Unlock()
0053: 	return nil
0054: }
0055:
0056: func TestResolveTokenForAuth_Antigravity_RefreshesExpiredToken(t *testing.T) {
0057: 	var callCount int
0058: 	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
0059: 		callCount++
0060: 		if r.Method != http.MethodPost {
0061: 			t.Fatalf("expected POST, got %s", r.Method)
0062: 		}
0063: 		if ct := r.Header.Get("Content-Type"); !strings.HasPrefix(ct, "application/x-www-form-urlencoded") {
0064: 			t.Fatalf("unexpected content-type: %s", ct)
0065: 		}
0066: 		bodyBytes, _ := io.ReadAll(r.Body)
0067: 		_ = r.Body.Close()
0068: 		values, err := url.ParseQuery(string(bodyBytes))
0069: 		if err != nil {
0070: 			t.Fatalf("parse form: %v", err)
0071: 		}
0072: 		if values.Get("grant_type") != "refresh_token" {
0073: 			t.Fatalf("unexpected grant_type: %s", values.Get("grant_type"))
0074: 		}
0075: 		if values.Get("refresh_token") != "rt" {
0076: 			t.Fatalf("unexpected refresh_token: %s", values.Get("refresh_token"))
0077: 		}
0078: 		if values.Get("client_id") != antigravityOAuthClientID {
0079: 			t.Fatalf("unexpected client_id: %s", values.Get("client_id"))
0080: 		}
0081: 		if values.Get("client_secret") != antigravityOAuthClientSecret {
0082: 			t.Fatalf("unexpected client_secret")
0083: 		}
0084:
0085: 		w.Header().Set("Content-Type", "application/json")
0086: 		_ = json.NewEncoder(w).Encode(map[string]any{
0087: 			"access_token":  "new-token",
0088: 			"refresh_token": "rt2",
0089: 			"expires_in":    int64(3600),
0090: 			"token_type":    "Bearer",
0091: 		})
0092: 	}))
0093: 	t.Cleanup(srv.Close)
0094:
0095: 	originalURL := antigravityOAuthTokenURL
0096: 	antigravityOAuthTokenURL = srv.URL
0097: 	t.Cleanup(func() { antigravityOAuthTokenURL = originalURL })
0098:
0099: 	store := &memoryAuthStore{}
0100: 	manager := coreauth.NewManager(store, nil, nil)
0101:
0102: 	auth := &coreauth.Auth{
0103: 		ID:       "antigravity-test.json",
0104: 		FileName: "antigravity-test.json",
0105: 		Provider: "antigravity",
0106: 		Metadata: map[string]any{
0107: 			"type":          "antigravity",
0108: 			"access_token":  "old-token",
0109: 			"refresh_token": "rt",
0110: 			"expires_in":    int64(3600),
0111: 			"timestamp":     time.Now().Add(-2 * time.Hour).UnixMilli(),
0112: 			"expired":       time.Now().Add(-1 * time.Hour).Format(time.RFC3339),
0113: 		},
0114: 	}
0115: 	if _, err := manager.Register(context.Background(), auth); err != nil {
0116: 		t.Fatalf("register auth: %v", err)
0117: 	}
0118:
0119: 	h := &Handler{authManager: manager}
0120: 	token, err := h.resolveTokenForAuth(context.Background(), auth)
0121: 	if err != nil {
0122: 		t.Fatalf("resolveTokenForAuth: %v", err)
0123: 	}
0124: 	if token != "new-token" {
0125: 		t.Fatalf("expected refreshed token, got %q", token)
0126: 	}
0127: 	if callCount != 1 {
0128: 		t.Fatalf("expected 1 refresh call, got %d", callCount)
0129: 	}
0130:
0131: 	updated, ok := manager.GetByID(auth.ID)
0132: 	if !ok || updated == nil {
0133: 		t.Fatalf("expected auth in manager after update")
0134: 	}
0135: 	if got := tokenValueFromMetadata(updated.Metadata); got != "new-token" {
0136: 		t.Fatalf("expected manager metadata updated, got %q", got)
0137: 	}
0138: }
0139:
0140: func TestResolveTokenForAuth_Antigravity_SkipsRefreshWhenTokenValid(t *testing.T) {
0141: 	var callCount int
0142: 	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
0143: 		callCount++
0144: 		w.WriteHeader(http.StatusInternalServerError)
0145: 	}))
0146: 	t.Cleanup(srv.Close)
0147:
0148: 	originalURL := antigravityOAuthTokenURL
0149: 	antigravityOAuthTokenURL = srv.URL
0150: 	t.Cleanup(func() { antigravityOAuthTokenURL = originalURL })
0151:
0152: 	auth := &coreauth.Auth{
0153: 		ID:       "antigravity-valid.json",
0154: 		FileName: "antigravity-valid.json",
0155: 		Provider: "antigravity",
0156: 		Metadata: map[string]any{
0157: 			"type":         "antigravity",
0158: 			"access_token": "ok-token",
0159: 			"expired":      time.Now().Add(30 * time.Minute).Format(time.RFC3339),
0160: 		},

### FILE: pkg/llmproxy/api/handlers/management/auth_files.go
0001: package management
0002:
0003: import (
0004: 	"bytes"
0005: 	"context"
0006: 	"crypto/rand"
0007: 	"crypto/sha256"
0008: 	"encoding/base64"
0009: 	"encoding/hex"
0010: 	"encoding/json"
0011: 	"errors"
0012: 	"fmt"
0013: 	"io"
0014: 	"net"
0015: 	"net/http"
0016: 	"net/url"
0017: 	"os"
0018: 	"path/filepath"
0019: 	"sort"
0020: 	"strconv"
0021: 	"strings"
0022: 	"sync"
0023: 	"time"
0024:
0025: 	"github.com/gin-gonic/gin"
0026: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/antigravity"
0027: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/claude"
0028: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/codex"
0029: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/copilot"
0030: 	geminiAuth "github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/gemini"
0031: 	iflowauth "github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/iflow"
0032: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/kilo"
0033: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/kimi"
0034: 	kiroauth "github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/kiro"
0035: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/auth/qwen"
0036: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/interfaces"
0037: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/misc"
0038: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/registry"
0039: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/util"
0040: 	sdkAuth "github.com/router-for-me/CLIProxyAPI/v6/sdk/auth"
0041: 	coreauth "github.com/router-for-me/CLIProxyAPI/v6/sdk/cliproxy/auth"
0042: 	log "github.com/sirupsen/logrus"
0043: 	"github.com/tidwall/gjson"
0044: 	"golang.org/x/oauth2"
0045: 	"golang.org/x/oauth2/google"
0046: )
0047:
0048: var lastRefreshKeys = []string{"last_refresh", "lastRefresh", "last_refreshed_at", "lastRefreshedAt"}
0049:
0050: const (
0051: 	anthropicCallbackPort   = 54545
0052: 	geminiCallbackPort      = 8085
0053: 	codexCallbackPort       = 1455
0054: 	geminiCLIEndpoint       = "https://cloudcode-pa.googleapis.com"
0055: 	geminiCLIVersion        = "v1internal"
0056: 	geminiCLIUserAgent      = "google-api-nodejs-client/9.15.1"
0057: 	geminiCLIApiClient      = "gl-node/22.17.0"
0058: 	geminiCLIClientMetadata = "ideType=IDE_UNSPECIFIED,platform=PLATFORM_UNSPECIFIED,pluginType=GEMINI"
0059: )
0060:
0061: type callbackForwarder struct {
0062: 	provider string
0063: 	server   *http.Server
0064: 	done     chan struct{}
0065: }
0066:
0067: var (
0068: 	callbackForwardersMu sync.Mutex
0069: 	callbackForwarders   = make(map[int]*callbackForwarder)
0070: )
0071:
0072: func extractLastRefreshTimestamp(meta map[string]any) (time.Time, bool) {
0073: 	if len(meta) == 0 {
0074: 		return time.Time{}, false
0075: 	}
0076: 	for _, key := range lastRefreshKeys {
0077: 		if val, ok := meta[key]; ok {
0078: 			if ts, ok1 := parseLastRefreshValue(val); ok1 {
0079: 				return ts, true
0080: 			}
0081: 		}
0082: 	}
0083: 	return time.Time{}, false
0084: }
0085:
0086: func parseLastRefreshValue(v any) (time.Time, bool) {
0087: 	switch val := v.(type) {
0088: 	case string:
0089: 		s := strings.TrimSpace(val)
0090: 		if s == "" {
0091: 			return time.Time{}, false
0092: 		}
0093: 		layouts := []string{time.RFC3339, time.RFC3339Nano, "2006-01-02 15:04:05", "2006-01-02T15:04:05Z07:00"}
0094: 		for _, layout := range layouts {
0095: 			if ts, err := time.Parse(layout, s); err == nil {
0096: 				return ts.UTC(), true
0097: 			}
0098: 		}
0099: 		if unix, err := strconv.ParseInt(s, 10, 64); err == nil && unix > 0 {
0100: 			return time.Unix(unix, 0).UTC(), true
0101: 		}
0102: 	case float64:
0103: 		if val <= 0 {
0104: 			return time.Time{}, false
0105: 		}
0106: 		return time.Unix(int64(val), 0).UTC(), true
0107: 	case int64:
0108: 		if val <= 0 {
0109: 			return time.Time{}, false
0110: 		}
0111: 		return time.Unix(val, 0).UTC(), true
0112: 	case int:
0113: 		if val <= 0 {
0114: 			return time.Time{}, false
0115: 		}
0116: 		return time.Unix(int64(val), 0).UTC(), true
0117: 	case json.Number:
0118: 		if i, err := val.Int64(); err == nil && i > 0 {
0119: 			return time.Unix(i, 0).UTC(), true
0120: 		}
0121: 	}
0122: 	return time.Time{}, false
0123: }
0124:
0125: func isWebUIRequest(c *gin.Context) bool {
0126: 	raw := strings.TrimSpace(c.Query("is_webui"))
0127: 	if raw == "" {
0128: 		return false
0129: 	}
0130: 	switch strings.ToLower(raw) {
0131: 	case "1", "true", "yes", "on":
0132: 		return true
0133: 	default:
0134: 		return false
0135: 	}
0136: }
0137:
0138: func startCallbackForwarder(port int, provider, targetBase string) (*callbackForwarder, error) {
0139: 	callbackForwardersMu.Lock()
0140: 	prev := callbackForwarders[port]
0141: 	if prev != nil {
0142: 		delete(callbackForwarders, port)
0143: 	}
0144: 	callbackForwardersMu.Unlock()
0145:
0146: 	if prev != nil {
0147: 		stopForwarderInstance(port, prev)
0148: 	}
0149:
0150: 	addr := fmt.Sprintf("127.0.0.1:%d", port)
0151: 	ln, err := net.Listen("tcp", addr)
0152: 	if err != nil {
0153: 		return nil, fmt.Errorf("failed to listen on %s: %w", addr, err)
0154: 	}
0155:
0156: 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
0157: 		target := targetBase
0158: 		if raw := r.URL.RawQuery; raw != "" {
0159: 			if strings.Contains(target, "?") {
0160: 				target = target + "&" + raw

### FILE: pkg/llmproxy/api/handlers/management/config_basic.go
0001: package management
0002:
0003: import (
0004: 	"encoding/json"
0005: 	"fmt"
0006: 	"io"
0007: 	"net/http"
0008: 	"os"
0009: 	"path/filepath"
0010: 	"strings"
0011: 	"time"
0012:
0013: 	"github.com/gin-gonic/gin"
0014: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/config"
0015: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/util"
0016: 	sdkconfig "github.com/router-for-me/CLIProxyAPI/v6/sdk/config"
0017: 	log "github.com/sirupsen/logrus"
0018: 	"gopkg.in/yaml.v3"
0019: )
0020:
0021: const (
0022: 	latestReleaseURL       = "https://api.github.com/repos/KooshaPari/cliproxyapi-plusplus/releases/latest"
0023: 	latestReleaseUserAgent = "cliproxyapi++"
0024: )
0025:
0026: func (h *Handler) GetConfig(c *gin.Context) {
0027: 	if h == nil || h.cfg == nil {
0028: 		c.JSON(200, gin.H{})
0029: 		return
0030: 	}
0031: 	c.JSON(200, new(*h.cfg))
0032: }
0033:
0034: type releaseInfo struct {
0035: 	TagName string `json:"tag_name"`
0036: 	Name    string `json:"name"`
0037: }
0038:
0039: // GetLatestVersion returns the latest release version from GitHub without downloading assets.
0040: func (h *Handler) GetLatestVersion(c *gin.Context) {
0041: 	client := &http.Client{Timeout: 10 * time.Second}
0042: 	proxyURL := ""
0043: 	if h != nil && h.cfg != nil {
0044: 		proxyURL = strings.TrimSpace(h.cfg.ProxyURL)
0045: 	}
0046: 	if proxyURL != "" {
0047: 		sdkCfg := &sdkconfig.SDKConfig{ProxyURL: proxyURL}
0048: 		util.SetProxy(sdkCfg, client)
0049: 	}
0050:
0051: 	req, err := http.NewRequestWithContext(c.Request.Context(), http.MethodGet, latestReleaseURL, nil)
0052: 	if err != nil {
0053: 		c.JSON(http.StatusInternalServerError, gin.H{"error": "request_create_failed", "message": err.Error()})
0054: 		return
0055: 	}
0056: 	req.Header.Set("Accept", "application/vnd.github+json")
0057: 	req.Header.Set("User-Agent", latestReleaseUserAgent)
0058:
0059: 	resp, err := client.Do(req)
0060: 	if err != nil {
0061: 		c.JSON(http.StatusBadGateway, gin.H{"error": "request_failed", "message": err.Error()})
0062: 		return
0063: 	}
0064: 	defer func() {
0065: 		if errClose := resp.Body.Close(); errClose != nil {
0066: 			log.WithError(errClose).Debug("failed to close latest version response body")
0067: 		}
0068: 	}()
0069:
0070: 	if resp.StatusCode != http.StatusOK {
0071: 		body, _ := io.ReadAll(io.LimitReader(resp.Body, 1024))
0072: 		c.JSON(http.StatusBadGateway, gin.H{"error": "unexpected_status", "message": fmt.Sprintf("status %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))})
0073: 		return
0074: 	}
0075:
0076: 	var info releaseInfo
0077: 	if errDecode := json.NewDecoder(resp.Body).Decode(&info); errDecode != nil {
0078: 		c.JSON(http.StatusBadGateway, gin.H{"error": "decode_failed", "message": errDecode.Error()})
0079: 		return
0080: 	}
0081:
0082: 	version := strings.TrimSpace(info.TagName)
0083: 	if version == "" {
0084: 		version = strings.TrimSpace(info.Name)
0085: 	}
0086: 	if version == "" {
0087: 		c.JSON(http.StatusBadGateway, gin.H{"error": "invalid_response", "message": "missing release version"})
0088: 		return
0089: 	}
0090:
0091: 	c.JSON(http.StatusOK, gin.H{"latest-version": version})
0092: }
0093:
0094: func WriteConfig(path string, data []byte) error {
0095: 	data = config.NormalizeCommentIndentation(data)
0096: 	f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
0097: 	if err != nil {
0098: 		return err
0099: 	}
0100: 	if _, errWrite := f.Write(data); errWrite != nil {
0101: 		_ = f.Close()
0102: 		return errWrite
0103: 	}
0104: 	if errSync := f.Sync(); errSync != nil {
0105: 		_ = f.Close()
0106: 		return errSync
0107: 	}
0108: 	return f.Close()
0109: }
0110:
0111: func (h *Handler) PutConfigYAML(c *gin.Context) {
0112: 	body, err := io.ReadAll(c.Request.Body)
0113: 	if err != nil {
0114: 		c.JSON(http.StatusBadRequest, gin.H{"error": "invalid_yaml", "message": "cannot read request body"})
0115: 		return
0116: 	}
0117: 	var cfg config.Config
0118: 	if err = yaml.Unmarshal(body, &cfg); err != nil {
0119: 		c.JSON(http.StatusBadRequest, gin.H{"error": "invalid_yaml", "message": err.Error()})
0120: 		return
0121: 	}
0122: 	// Validate config using LoadConfigOptional with optional=false to enforce parsing
0123: 	tmpDir := filepath.Dir(h.configFilePath)
0124: 	tmpFile, err := os.CreateTemp(tmpDir, "config-validate-*.yaml")
0125: 	if err != nil {
0126: 		c.JSON(http.StatusInternalServerError, gin.H{"error": "write_failed", "message": err.Error()})
0127: 		return
0128: 	}
0129: 	tempFile := tmpFile.Name()
0130: 	if _, errWrite := tmpFile.Write(body); errWrite != nil {
0131: 		_ = tmpFile.Close()
0132: 		_ = os.Remove(tempFile)
0133: 		c.JSON(http.StatusInternalServerError, gin.H{"error": "write_failed", "message": errWrite.Error()})
0134: 		return
0135: 	}
0136: 	if errClose := tmpFile.Close(); errClose != nil {
0137: 		_ = os.Remove(tempFile)
0138: 		c.JSON(http.StatusInternalServerError, gin.H{"error": "write_failed", "message": errClose.Error()})
0139: 		return
0140: 	}
0141: 	defer func() {
0142: 		_ = os.Remove(tempFile)
0143: 	}()
0144: 	_, err = config.LoadConfigOptional(tempFile, false)
0145: 	if err != nil {
0146: 		c.JSON(http.StatusUnprocessableEntity, gin.H{"error": "invalid_config", "message": err.Error()})
0147: 		return
0148: 	}
0149: 	h.mu.Lock()
0150: 	defer h.mu.Unlock()
0151: 	if WriteConfig(h.configFilePath, body) != nil {
0152: 		c.JSON(http.StatusInternalServerError, gin.H{"error": "write_failed", "message": "failed to write config"})
0153: 		return
0154: 	}
0155: 	// Reload into handler to keep memory in sync
0156: 	newCfg, err := config.LoadConfig(h.configFilePath)
0157: 	if err != nil {
0158: 		c.JSON(http.StatusInternalServerError, gin.H{"error": "reload_failed", "message": err.Error()})
0159: 		return
0160: 	}

### FILE: pkg/llmproxy/api/handlers/management/config_lists.go
0001: package management
0002:
0003: import (
0004: 	"encoding/json"
0005: 	"fmt"
0006: 	"strings"
0007:
0008: 	"github.com/gin-gonic/gin"
0009: 	"github.com/router-for-me/CLIProxyAPI/v6/pkg/llmproxy/config"
0010: )
0011:
0012: // Generic helpers for list[string]
0013: func (h *Handler) putStringList(c *gin.Context, set func([]string), after func()) {
0014: 	data, err := c.GetRawData()
0015: 	if err != nil {
0016: 		c.JSON(400, gin.H{"error": "failed to read body"})
0017: 		return
0018: 	}
0019: 	var arr []string
0020: 	if err = json.Unmarshal(data, &arr); err != nil {
0021: 		var obj struct {
0022: 			Items []string `json:"items"`
0023: 		}
0024: 		if err2 := json.Unmarshal(data, &obj); err2 != nil || len(obj.Items) == 0 {
0025: 			c.JSON(400, gin.H{"error": "invalid body"})
0026: 			return
0027: 		}
0028: 		arr = obj.Items
0029: 	}
0030: 	set(arr)
0031: 	if after != nil {
0032: 		after()
0033: 	}
0034: 	h.persist(c)
0035: }
0036:
0037: func (h *Handler) patchStringList(c *gin.Context, target *[]string, after func()) {
0038: 	var body struct {
0039: 		Old   *string `json:"old"`
0040: 		New   *string `json:"new"`
0041: 		Index *int    `json:"index"`
0042: 		Value *string `json:"value"`
0043: 	}
0044: 	if err := c.ShouldBindJSON(&body); err != nil {
0045: 		c.JSON(400, gin.H{"error": "invalid body"})
0046: 		return
0047: 	}
0048: 	if body.Index != nil && body.Value != nil && *body.Index >= 0 && *body.Index < len(*target) {
0049: 		(*target)[*body.Index] = *body.Value
0050: 		if after != nil {
0051: 			after()
0052: 		}
0053: 		h.persist(c)
0054: 		return
0055: 	}
0056: 	if body.Old != nil && body.New != nil {
0057: 		for i := range *target {
0058: 			if (*target)[i] == *body.Old {
0059: 				(*target)[i] = *body.New
0060: 				if after != nil {
0061: 					after()
0062: 				}
0063: 				h.persist(c)
0064: 				return
0065: 			}
0066: 		}
0067: 		*target = append(*target, *body.New)
0068: 		if after != nil {
0069: 			after()
0070: 		}
0071: 		h.persist(c)
0072: 		return
0073: 	}
0074: 	c.JSON(400, gin.H{"error": "missing fields"})
0075: }
0076:
0077: func (h *Handler) deleteFromStringList(c *gin.Context, target *[]string, after func()) {
0078: 	if idxStr := c.Query("index"); idxStr != "" {
0079: 		var idx int
0080: 		_, err := fmt.Sscanf(idxStr, "%d", &idx)
0081: 		if err == nil && idx >= 0 && idx < len(*target) {
0082: 			*target = append((*target)[:idx], (*target)[idx+1:]...)
0083: 			if after != nil {
0084: 				after()
0085: 			}
0086: 			h.persist(c)
0087: 			return
0088: 		}
0089: 	}
0090: 	if val := strings.TrimSpace(c.Query("value")); val != "" {
0091: 		out := make([]string, 0, len(*target))
0092: 		for _, v := range *target {
0093: 			if strings.TrimSpace(v) != val {
0094: 				out = append(out, v)
0095: 			}