Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 84 additions & 11 deletions ADR.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,89 @@
# Architecture Decision Records
# Architecture Decision Records — CLIProxyAPI++

## ADR-001: CLI Proxy API++
## ADR-001 | Go HTTP Proxy Server | Adopted

**Status**: Accepted
**Status:** Adopted

**Context**: Unified CLI proxy with rich features.
**Context:** Need a high-performance LLM API proxy that can handle concurrent streaming connections with minimal latency overhead.

**Decision**:
- Go-based proxy
- Provider abstraction
- Rich observability
**Decision:** Use Go with net/http for the core proxy server on port 8317, implementing OpenAI-compatible endpoints that translate to multiple backend providers.

**Consequences**:
- Positive: High performance
- Positive: Multi-provider support
**Consequences:**
- Excellent concurrency via goroutines for streaming connections
- Single binary deployment simplifies Docker and bare-metal installs
- Strong stdlib HTTP support reduces external dependencies

---

## ADR-002 | Provider Abstraction Layer | Adopted

**Status:** Adopted

**Context:** The proxy must support multiple LLM providers (GitHub Copilot, Kiro/AWS) with different auth flows and API formats while exposing a unified OpenAI-compatible interface.

**Decision:** Implement a provider abstraction with pluggable auth adapters in `auths/`, model name conversion, and per-provider request/response translation.

**Consequences:**
- Adding new providers requires only a new auth adapter and model mapping
- Model name converter handles provider-specific naming transparently
- Each provider manages its own token lifecycle independently

---

## ADR-003 | OAuth Web UI for Kiro/AWS | Adopted

**Status:** Adopted

**Context:** Kiro authentication requires AWS Builder ID or Identity Center flows that involve browser-based OAuth redirects, unlike Copilot's device code flow.

**Decision:** Embed a web UI at `/v0/oauth/kiro` that handles the full OAuth PKCE flow, token import from IDE, and background refresh.

**Consequences:**
- Users can authenticate via browser without CLI interaction
- Token import supports copying tokens from Kiro IDE directly
- Background goroutine handles token refresh before expiry

---

## ADR-004 | Docker-First Deployment | Adopted

**Status:** Adopted

**Context:** Users need a simple, reproducible deployment that works across platforms with minimal configuration.

**Decision:** Provide `Dockerfile` and `docker-compose.yml` with YAML config via volume mount (`config.yaml`).

**Consequences:**
- Single `docker-compose up` for deployment
- Config changes require only volume-mounted YAML edits
- Multi-provider configuration in a single config file

---

## ADR-005 | Rate Limiting with Smart Cooldown | Adopted

**Status:** Adopted

**Context:** Provider APIs enforce rate limits; aggressive retries waste quota and risk bans.

**Decision:** Implement a rate limiter with smart cooldown manager that tracks per-provider limits and backs off exponentially.

**Consequences:**
- Prevents provider API ban from excessive requests
- Cooldown periods auto-adjust based on response headers
- Metrics module tracks usage for observability

---

## ADR-006 | Plus Fork Strategy | Adopted

**Status:** Adopted

**Context:** CLIProxyAPI++ is an enhanced fork adding multi-provider support, Kiro auth, and observability to the original CLIProxyAPI.

**Decision:** Maintain as independent "plus-plus" fork with additive features, preserving compatibility with upstream API contract.

**Consequences:**
- Upstream changes can be cherry-picked when compatible
- New features (Kiro, metrics, fingerprint) are additive, not breaking
- Original Copilot-only flow remains functional
39 changes: 39 additions & 0 deletions FUNCTIONAL_REQUIREMENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Functional Requirements — CLIProxyAPI++

## FR-PRX: Proxy Core

- **FR-PRX-001:** The system SHALL expose an OpenAI-compatible API on port 8317.
- **FR-PRX-002:** The system SHALL support model name conversion across providers.
- **FR-PRX-003:** The system SHALL handle UTF-8 streaming responses correctly.

## FR-AUTH: Authentication

- **FR-AUTH-001:** The system SHALL support GitHub Copilot OAuth login.
- **FR-AUTH-002:** The system SHALL support Kiro OAuth via web UI at `/v0/oauth/kiro`.
- **FR-AUTH-003:** Kiro auth SHALL support AWS Builder ID and Identity Center login.
- **FR-AUTH-004:** The system SHALL support token import from Kiro IDE.
- **FR-AUTH-005:** Background token refresh SHALL occur before token expiration.

## FR-RL: Rate Limiting

- **FR-RL-001:** The system SHALL enforce configurable request rate limits.
- **FR-RL-002:** The system SHALL implement smart cooldown on provider rate limits.

## FR-MON: Monitoring

- **FR-MON-001:** The system SHALL collect request metrics (count, latency, errors).
- **FR-MON-002:** The system SHALL provide real-time usage monitoring and quota tracking.

## FR-SEC: Security

- **FR-SEC-001:** The system SHALL generate device fingerprints for request attribution.

## FR-DEP: Deployment

- **FR-DEP-001:** The system SHALL support Docker deployment via docker-compose.
- **FR-DEP-002:** Configuration SHALL be via YAML file with volume mount support.

## FR-CFG: Configuration

- **FR-CFG-001:** The system SHALL read configuration from `config.yaml`.
- **FR-CFG-002:** The system SHALL support multiple provider configurations simultaneously.
39 changes: 39 additions & 0 deletions PLAN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Implementation Plan — CLIProxyAPI++

## Phase 1: Core Proxy (Done)

| Task | Description | Depends On | Status |
|------|-------------|------------|--------|
| P1.1 | Go HTTP server on :8317 | — | Done |
| P1.2 | OpenAI-compatible API endpoints | P1.1 | Done |
| P1.3 | Provider abstraction layer | P1.1 | Done |
| P1.4 | Model name converter | P1.3 | Done |
| P1.5 | YAML configuration loading | — | Done |

## Phase 2: Provider Auth (Done)

| Task | Description | Depends On | Status |
|------|-------------|------------|--------|
| P2.1 | GitHub Copilot OAuth | P1.3 | Done |
| P2.2 | Kiro OAuth web UI | P1.3 | Done |
| P2.3 | AWS Builder ID / Identity Center flows | P2.2 | Done |
| P2.4 | Token import from Kiro IDE | P2.2 | Done |
| P2.5 | Background token refresh | P2.1 | Done |

## Phase 3: Enhanced Features (Done)

| Task | Description | Depends On | Status |
|------|-------------|------------|--------|
| P3.1 | Rate limiter | P1.1 | Done |
| P3.2 | Cooldown management | P3.1 | Done |
| P3.3 | Metrics collection | P1.1 | Done |
| P3.4 | Usage checker | P3.3 | Done |
| P3.5 | Device fingerprint | P1.1 | Done |
| P3.6 | UTF-8 stream processing | P1.2 | Done |

## Phase 4: Deployment (Done)

| Task | Description | Depends On | Status |
|------|-------------|------------|--------|
| P4.1 | Docker image build | P1.1 | Done |
| P4.2 | docker-compose configuration | P4.1 | Done |
54 changes: 54 additions & 0 deletions PRD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Product Requirements Document — CLIProxyAPI++

## E1: Multi-Provider LLM Proxy

### E1.1: Provider Abstraction
**As** a developer, **I want** a unified API proxy supporting Claude, Copilot, Kiro, and other providers **so that** I can access any model through a single endpoint.

**Acceptance Criteria:**
- OpenAI-compatible API endpoint at `:8317`
- Provider-specific authentication (OAuth, API key)
- Model name conversion across providers

### E1.2: GitHub Copilot Support
**As** a developer, **I want** GitHub Copilot OAuth login **so that** I can proxy requests through my Copilot subscription.

### E1.3: Kiro (AWS CodeWhisperer) Support
**As** a developer, **I want** Kiro OAuth login via web UI **so that** I can proxy requests through AWS Builder ID or Identity Center.

**Acceptance Criteria:**
- Web-based OAuth at `/v0/oauth/kiro`
- AWS Builder ID and Identity Center login flows
- Token import from Kiro IDE

## E2: Enhanced Features

### E2.1: Rate Limiter
**As** an operator, **I want** built-in rate limiting **so that** API abuse is prevented.

### E2.2: Background Token Refresh
**As** a developer, **I want** automatic token refresh before expiration **so that** requests are never interrupted by expired tokens.

### E2.3: Metrics and Monitoring
**As** an operator, **I want** request metrics collection **so that** I can monitor usage and debug issues.

### E2.4: Device Fingerprint
**As** a security engineer, **I want** device fingerprint generation **so that** requests are tied to specific devices.

### E2.5: Cooldown Management
**As** an operator, **I want** smart cooldown for API rate limits **so that** the proxy backs off gracefully.

### E2.6: Usage Checker
**As** a developer, **I want** real-time usage monitoring **so that** I can track quota consumption.

### E2.7: UTF-8 Stream Processing
**As** a developer, **I want** improved streaming response handling **so that** multi-byte characters are processed correctly.

## E3: Deployment

### E3.1: Docker Deployment
**As** an operator, **I want** one-command Docker deployment **so that** the proxy is running in seconds.

**Acceptance Criteria:**
- `docker-compose.yml` with volume-mounted config
- Pre-built image at `eceasy/cli-proxy-api-plus:latest`
Loading