Skip to content

architecture.md

Latest commit

 

History

History
171 lines (131 loc) · 13 KB

architecture.md

File metadata and controls

171 lines (131 loc) · 13 KB
Raw

Architecture

How Free Provider is put together and why.

High-level flow

OpenAI SDK / browser UI
        │  POST /v1/chat/completions  (Bearer token)
        ▼
   Express server (server/)
        │  validates auth, parses messages
        ▼
   provider router (server/providers/router.mjs)
        │  explicit model → model prefix (gemini-* → Gemini, else ChatGPT)
        │  model "auto"/omitted → next auto slot (alternates chatgpt → gemini,
        │    cycling through paired windows: pair i = chatgpt i + gemini i)
        ▼
   ChatGPTProvider (server/providers/chatgpt.mjs)      GeminiProvider (server/providers/gemini.mjs)
        │  round-robin across N agents, maps models        │  same agent-pool / cooldown machinery
        ▼                                                  ▼
   ChatGPT client (client/index.mjs) × N instances    Gemini guest client (client/gemini.mjs) × N
        │  init → tokens → conduit → pow → turnstile       │  scrape /app → guest session → StreamGenerate
        ▼                                                  ▼
   ChatGPT web backend (chatgpt.com)                  Gemini web backend (gemini.google.com)
        │  SSE stream                                        length-prefixed frames
        ▼                                                  ▼
   Provider re-emits deltas in OpenAI chunk format → response

Repository layout

├── chatgpt.mjs              # Anonymous-only client entry (imports src/)
├── chatgpt-auth.mjs         # Full auth client (login, signup, profile, …)
├── src/                     # Shared client logic (shared by chatgpt.mjs)
│   ├── cloudflare.mjs       # CF retry, UA rotation, cookie jar, proxy
│   ├── pow.mjs              # FNV-1a proof-of-work solver
│   ├── sentinel.mjs         # Sentinel token acquisition
│   ├── sse.mjs              # SSE parser (buffer + streaming)
│   ├── turnstile.mjs        # Turnstile bytecode VM
│   └── upload.mjs           # Azure file upload + multimodal builder
├── client/                  # Self-contained standalone client
│   ├── index.mjs            # ChatGPT class the server actually uses
│   ├── gemini.mjs           # Gemini guest-mode client (scrape + StreamGenerate)
│   └── cloudflare.mjs       # CF retry, UA rotation, cookie jar, proxy pool
└── server/                  # The Free Provider HTTP server
    ├── index.mjs            # Express app, static files, startup banner
    ├── config.mjs           # Port, API key, models, env overrides + .env loader
    ├── middleware/auth.mjs  # Bearer-token guard
    ├── providers/chatgpt.mjs# Adapts ChatGPT client → OpenAI format
    ├── providers/gemini.mjs # Adapts Gemini guest client → OpenAI format
    ├── providers/router.mjs # model-prefix + auto-balancing routing, merged model/pool views
    ├── routes/              # /v1/models, /v1/chat/completions, /v1/pool
    └── public/              # Web chat UI (HTML/CSS/JS)
├── scripts/check-coverage.mjs  # runs the test suite + enforces the 85% coverage gate
└── test/                    # node:test suites (see docs/testing.md)

Server components

server/index.mjs

Sets up Express, serves the web UI from public/, mounts the API routes, and prints the startup banner (including the export OPENAI_API_KEY="…" line). The /favicon.ico route is intentionally placed before auth to avoid 401 noise.

server/config.mjs

Auto-loads .env from the repo root on import (process.loadEnvFile, Node 20.12+ — existing env vars always win), then reads OPENAI_API_KEY, PROXY_URLS, PROXY_FILE, agent/timeout settings and the model list from process.env. A generated API key is saved to .api-key when none is set, and the parsed proxy list (comments stripped, deduplicated) plus all tuning knobs are exported for the rest of the app.

server/middleware/auth.mjs

Reads Authorization: Bearer <key> and compares against config.apiKey. Returns a 401 OpenAI-style error on mismatch.

server/providers/chatgpt.mjs

The bridge between OpenAI and ChatGPT:

  • listModels() — returns OpenAI-format model list
  • chatCompletion() — non-streaming (client.ask)
  • chatCompletionStream() — streaming (client.streamAsk)
  • _resetConversation() — clears conversationId / parentMessageId per request
  • cooldownStatus() / _registerBlock() — per-agent cooldown guard: after MAX_CONSECUTIVE_BLOCKS consecutive Cloudflare blocks on an agent, that agent stops accepting work for COOLDOWN_SECONDS. The server returns 429 only while all agents are cooling down.
  • poolStatus() — live diagnostics for GET /v1/pool: per-agent readiness/cooldown and per-proxy status/bench time, so you can watch the pool rotate and recover.

A single provider singleton is shared by both routes (avoiding duplicate agent pools). A MODEL_MAP translates common OpenAI ids to ChatGPT slugs (e.g. gpt-4o → auto, gpt-5.6-luna → gpt-5-6). Each agent is a ChatGPT client constructed with the configured proxy pool, inter-request delay, streaming retries, and cooldown tuning from config.mjs.

server/providers/gemini.mjs

The ChatGPT provider's mirror for the Gemini web backend. It drives client/gemini.mjs and reuses the same agent-pool pattern (AGENT_COUNT agents, per-agent cooldown, exponential backoff, BLOCK_RE extended with Gemini-specific flags). Serves gemini-3-pro / gemini-3-flash (default) / gemini-3-lite, all owned_by: "gemini-free".

The Gemini client needs no cookies: init() scrapes the build label / session id / language from GET https://gemini.google.com/app, calls the guest maGuAc RPC (sets NID/COMPASS cookies), then POSTs to assistant.lamda.BardFrontendService/StreamGenerate with the guest jspb model headers. Responses are )]}'-prefixed length-delimited frames (not SSE); a StreamingFrameParser reconstructs text deltas and detects completion/error (e.g. usage-limit 1037 or IP-block 1060 → 429).

server/providers/router.mjs

Routes by model prefix: gemini-* → GeminiProvider, everything else → ChatGPTProvider. Exposes getProvider(model) (used per-request by the chat route), listAllModels() (merged, deduplicated model list for /v1/models), and poolStatusAll() ({ chatgpt, gemini } for /v1/pool). Re-exports both provider singletons so routes never build their own pools.

Auto mode adds three helpers:

  • isAutoModel(model) — true when model is missing/empty, "auto", or equals config.defaultModel (default auto).
  • nextAutoSlot() — returns the next { provider, agentIndex } for auto requests. It alternates chatgpt → gemini, and after each full pair advances to the next paired window, wrapping after AGENT_COUNT. Pair i = ChatGPT agent i + Gemini agent i.
  • resetAutoScheduler() — resets the counter (mainly used by tests).

Because the slot is resolved per request, auto mode stays stateless: every request timeshares between backends without any conversation carry-over.

server/routes/chat.mjs

Validates the request, then for streaming accumulates deltas into content and — on completion or error — writes a structured JSON line to server/chat.log:

{"timestamp":"…","type":"completion","stream":true,"model":"gpt-4o","user":["Hi"],"assistant":"Hello!","error":null}

server/public/

A dependency-free (vanilla JS) chat UI with a sidebar, grouped history, markdown rendering, syntax highlighting, and a settings modal.

Key design decisions

Stateless provider

The ChatGPT client is reused across requests, but every request resets conversation state (conversationId/parentMessageId). This:

  • matches OpenAI API semantics (each call is independent),
  • preserves context via the messages array,
  • avoids the anonymous continuation 404 the raw client hits on follow-up messages.

Multi-agent round-robin

The provider holds AGENT_COUNT independent ChatGPT instances (default 3). Each agent has its own CFBypass transport — independent pacing window, cooldown timer, proxy rotation and serialization lock. Requests are distributed round-robin across agents, which gives:

  • Higher throughput — several requests in flight at once, each under its own agent lock (a single agent can only run one message at a time).
  • Resilience — one agent in cooldown doesn't stall the others; 429 only when every agent is cooling down.
  • Isolated sessions — agents rotate their own proxies and accumulate their own CF block history.

OpenAPI → ChatGPT mapping

Because the web backend uses its own model slugs, the provider normalizes model names. Unknown names pass through to the client, which forwards whatever the web backend accepts.

Auto model balancing across paired windows

When the request uses model: "auto" (the default), the router alternates backends instead of pinning one:

  • Sequence: chatgpt → gemini → chatgpt → gemini → ….
  • Paired windows — ChatGPT and Gemini each hold AGENT_COUNT independent agents; pair i is (ChatGPT window i, Gemini window i). Requests cycle pair 0, 1, …, wrapping at AGENT_COUNT.
  • The chosen agentIndex is passed to the provider, which pins that window without disturbing its own round-robin counter — so pairs always stay aligned while explicit-model requests keep plain round-robin semantics.
  • Benefits: both backends share the load (halving hammering on either), and each pair acts as an independent route with its own tokens/session/proxies/cooldown.

Streaming

The client yields parsed SSE chunks { text, done, … }. The provider wraps each as chat.completion.chunk with delta.content, then emits finish_reason: "stop" and finally data: [DONE].

Resilience: retry, cooldown, proxy rotation

The CFBypass transport layer (in client/cloudflare.mjs / src/cloudflare.mjs) owns anti-block strategy:

  • Exponential backoff — retryable failures (429, CF 403/503, 408/502/504) back off baseDelay × 2^attempt (cap 16s), rotate User-Agent, and drop CF challenge cookies.
  • Inter-request delay — a configurable minimum gap between outgoing requests reduces hammering.
  • Adaptive pacing — a sliding-window token bucket (MAX_REQ_PER_WINDOW per PACING_WINDOW_SECONDS) lets normal traffic burst instantly but spaces out repeated requests just under the guard's threshold; a block temporarily halving the allowed rate.
  • Client serialization — ask()/streamAsk() run under a per-client lock so concurrent messages can't corrupt shared token/session state (a source of spurious 403s). Page-init (buildId/deviceId) is cached to cut request count per message.
  • HTTP semantics — the server returns 429 + Retry-After for rate-limit/block/flag conditions so OpenAI-SDK-compatible clients back off correctly, and flushes streaming headers only on the first chunk so pre-stream failures still produce a proper status code.
  • Proxy pool — proxies (http/https/socks4/socks5) come from PROXY_URLS, PROXY_FILE, a single proxyUrl, or client/proxies.txt. Blank lines and # comments in proxy files are stripped. HTTP(S) proxies use undici.ProxyAgent; socks5:// uses undici's built-in Socks5ProxyAgent (which performs the TLS handshake over the tunnel — a custom raw-socket connect would send plaintext and get nginx "400 plain HTTP to HTTPS port"); socks4:// uses the socks package with a custom undici connector that TLS-wraps the tunneled socket for https:. On a block the current proxy is benched for PROXY_COOLDOWN_SECONDS and rotation moves to the next live one. A subset for testing can point at a local Tor (PROXY_URLS=socks5://127.0.0.1:9050).
  • Multi-agent routing — with AGENT_COUNT > 1 the server distributes requests round-robin across independent ChatGPT instances, each with its own proxy rotation, pacing and cooldown state.
  • Instance cooldown — when all proxies are benched (or repeated blocks occur with no proxy), the transport refuses further attempts and reports the wait time as a 429.
  • Streaming retry — fetchStream() retries like ordinary requests (STREAM_RETRIES, default 2) instead of failing on the first non-OK response.

The server provider adds a higher-level guard on top: consecutive blocks surface as 429 server_cooldown with a Retry-After header until the cooldown expires.

Flow through the client

  1. init() — scrape chatgpt.com for buildId/deviceId, build browser config.
  2. getTokens() — call /backend-anon/sentinel/chat-requirements (or the auth variant) to get sentinel token, proof-of-work seed/difficulty, and Turnstile bytecode.
  3. getConduit() — obtain a conduit token from /f/conversation/prepare.
  4. Solve proof-of-work (FNV-1a) and run the Turnstile VM.
  5. POST to /f/conversation with all tokens as headers; parse the SSE reply.
  6. Return streamed deltas (streaming) or the joined text (non-streaming).