Skip to content

PROJECT_OVERVIEW.md

Latest commit

 

History

History
95 lines (66 loc) · 7.05 KB

PROJECT_OVERVIEW.md

File metadata and controls

95 lines (66 loc) · 7.05 KB
Raw

VoidFlag: Project Overview

1. Overview

VoidFlag is a schema-first feature flag and remote configuration management system. Designed to replace stringly-typed and error-prone feature toggles, VoidFlag introduces a custom .vf schema language to rigorously define flags, their data types, and fallback values.

The core problem VoidFlag solves is the lack of type-safety, poor developer experience, and tedious local-testing loops inherent in many feature flag platforms. By pairing a robust schema definition with automatic code generation (SDK clients), local dev-servers, and varying transport strategies, VoidFlag offers a GitOps-friendly, local-first workflow that scales up to a full production system.

2. Architecture

VoidFlag is structured as a Turborepo monorepo with multiple distinct packages and applications:

  • Backend (apps/api): A NestJS application handling core business logic, user auth, project/environment management, and flag distribution.
  • Frontend (apps/frontend): A Next.js application serving as the dashboard for developers to manage flags and environments.
  • Developer CLI (packages/CLI): A toolset for local development (dev server), schema parsing, and client generation.
  • SDK (packages/sdk): A strictly-typed TypeScript client consumed by end-user applications.
  • AST Parser (packages/parser): The lexer and parser for .vf schema files.
  • VS Code Extension (packages/vscode-extension): Syntax highlighting and IDE support for .vf files.

Data Flow

  1. Developers define flags in a schema.vf file.
  2. The CLI generates a strongly-typed TypeScript client representing that specific schema.
  3. In local development, the CLI spins up a Dev Server, capturing API requests from the SDK and streaming flag updates via Server-Sent Events (SSE).
  4. In production, the schema is pushed to the central Backend. The SDK connects to the backend, which determines the optimal transport mechanism (Polling vs. SSE) based on connection stability and the user's billing plan.

3. Components Breakdown

Backend (apps/api)

  • Purpose: Centralized source of truth for projects, user environments, and feature flag states.
  • Key Responsibilities: Rate-limiting (via NestJS Throttler), authentication (JWT/OAuth2), schema syncing (validating pushed .vf schemas), and streaming flag states to connected clients.
  • Notable Implementation: The /connect endpoint inspects the userPlan. Users on the FREE tier are restricted to a polling transport mechanism, while paid tiers receive a persistent sse stream.

Frontend (apps/frontend)

  • Purpose: Developer dashboard.
  • Key Responsibilities: Visualizing flag states, managing environments, and modifying properties like rollout percentages.
  • Notable Implementation: Built on Next.js 16, utilizing modern React hooks, Zustand for local state management, TailwindCSS v4, and shadcn/ui components.

Developer CLI (packages/CLI)

  • Purpose: The bridge between the platform and the developer's local environment.
  • Commands:
    • vf generate: Reads schema.vf, parses the AST, and generates heavily typed voidflag.client.ts code for the developer's app.
    • vf dev: Starts an Express-based local dev server, injecting itself in place of the production backend. It exposes a minimal local UI to toggle flags during local development.
    • vf apply: Pushes local flag states/schemas up to the backend server.

SDK (packages/sdk)

  • Purpose: The runtime library integrated into user applications.
  • Modules/Files: sdk.ts, transport.ts.
  • Implementation: Utilizes djb2 hashing for stable user rollout allocations. Employs lazy-evaluation via an accessor cache Object.defineProperty for optimal read performance.

4. SDK Design

  • Access Patterns: The SDK converts schema definitions into Accessor nodes. This forces O(1) property lookups (e.g., vf.flags.darkMode.value) instead of method calls (vf.get('darkMode')), which is cleaner and highly cacheable.
  • Transport Mechanisms:
    • SSE (SSETransport): Primary real-time mechanism using EventSource. Resolves stream updates incrementally using patches.
    • Polling (PollingTransport): Standard HTTP polling mechanism.
    • Resilience & Backoff: Features a full-jitter exponential backoff algorithm to prevent "thundering herd" scenarios when clients reconnect. If the SSE stream repeatedly fails, the SDK permanently degrades into the Polling transport.
  • Typing Strategy: Generics-heavy (VoidClient<S extends FlagMap>). Extracting types strictly ensures that developers cannot query flags that do not exist or expect an incorrect return type (e.g., expecting a number from a boolean flag).

5. CLI (packages/CLI)

  • Workflows Enabled: Local-first development. The local Dev Server runs completely offline, overriding the SDK's remote endpoints. Developers can test complex feature rollouts on localhost without touching a staging server.
  • Codegen: Central to the UX, the CLI analyzes standard definitions and continuously overwrites a local stub, ensuring zero-latency typescript autocompletion.

6. Custom Schema / File System

  • The .vf Format: A declarative DSL (Domain Specific Language) dedicated solely to feature flags. 
    flag darkMode {
  type bool
  fallback false
}
  • Parser & Lexer: A hand-written lexer/parser (packages/parser). It iterates over strings, tokenizes constraints (flag, type, fallback), and generates a SchemaAST. By owning the grammar, VoidFlag avoids JSON limitations (like lack of comments) and YAML complexities.

7. Tooling & Developer Experience

  • VS Code Extension: voidflag-vscode. Utilizing LSP (Language Server Protocol), it provides real-time .vf syntax highlighting, JSON schema validation for config files (voidflag.config.json), and custom icons.
  • Code Generation: Prevents typos natively. Instead of guessing a string key, developers get IDE intellisense.

8. Key Design Decisions

  • SSE vs Polling (Plan-Based Routing): Connection concurrency can be incredibly expensive for backend architectures. By gracefully reverting free-tier users into long-polling architectures, the platform limits expensive active socket limits on the Node.js APIs.
  • djb2 Hashing for Rollouts: Allows consistent fractional rollouts (e.g., "50% of users"). User X evaluating the flag will always land in the same bucket locally or globally, completely eliminating server-side evaluation calls out to an external system.
  • Schema-First (DSL): Hardcoding a schema text file forces a "source of truth". Flags aren't hidden unreachably in a database; they live beside the code.

9. Strengths

  • Unmatched Type-Safety: End-to-end TS types branching straight from a DSL.
  • Pristine Local Development: The inclusion of an explicit local Dev Server command that mocks the live endpoint eliminates the biggest friction point of feature flags—testing them.
  • Architectural Resilience: The fallback mechanism smoothly degrades connections from SSE to polling without interrupting the user application.