Phase 1: Install and First Inference

.planning/PROJECT.md

@@ -0,0 +1,118 @@
+# Needle
+
+## What This Is
+
+Needle is a lightweight Python package for running a compact language model and
+agent workflows on constrained devices. It provides a native inference facade,
+JAX/Flax reference and training code, LoRA fine-tuning, quantization/export to
+`.cact`, a CLI, and a local playground.
+
+This milestone focuses on making the existing capabilities understandable and
+usable by Python and machine-learning beginners through Chinese-first
+documentation and reproducible end-to-end examples.
+
+## Core Value
+
+A beginner can install Needle and reliably go from a first inference request to
+a fine-tuned, exported model without guessing which assets, commands, or
+runtime constraints apply.
+
+## Requirements
+
+### Validated
+
+- [x] Native `Needle` API supports completion, tool execution, schema extraction,
+  reset, and weight loading — existing in `needle/__init__.py`.
+- [x] CLI exposes fetch/download, run, fine-tune, data generation, build,
+  playground, and related workflows — existing in `needle/cli.py`.
+- [x] JAX/Flax reference model defines the Simple Attention Network, decoding,
+  LoRA training, quantization, and `.cact` export paths — existing in
+  `needle/model/`.
+- [x] Test suite covers model, inference, fine-tuning, export, packaging,
+  environments, and CLI behavior — existing in `tests/`.
+
+### Active
+
+- [ ] DOC-01: Provide a Chinese-first installation and environment guide that
+  explains CPU and GPU options, optional extras, model assets, caches, and
+  offline operation.
+- [ ] DOC-02: Provide a copy-paste quickstart that runs a pre-trained inference
+  example and explains the public `Needle` API and CLI equivalents.
+- [ ] DOC-03: Provide an end-to-end fine-tuning tutorial covering JSONL data
+  format, prompt rendering, LoRA training, checkpoint outputs, and common
+  resource/configuration choices.
+- [ ] DOC-04: Provide an export/deployment tutorial covering merge, quantization,
+  `.cact` generation, engine compatibility, and running the exported artifact.
+- [ ] DOC-05: Document the model structure and data flow, including the native
+  runtime path versus the JAX/Flax training path, attention/MLP components,
+  tokenizer contract, and process-global state constraints.
+- [ ] DOC-06: Add troubleshooting and safety notes for download verification,
+  pickle checkpoints, playground exposure, concurrency, dependency drift, and
+  known training/export limitations; track code hardening as later work.
+- [ ] DOC-07: Keep examples and commands testable on CPU and GPU where supported,
+  with a clear verification checklist for each tutorial.
+
+### Out of Scope
+
+- Rewriting the native inference engine or changing the `.cact` binary contract
+  — documentation should reflect the current implementation first.
+- Adding authentication, hosted multi-user serving, or a production web service
+  — the current playground remains a local development tool.
+- Solving every security or performance concern in this documentation milestone
+  — risks are recorded and prioritized for later implementation phases.
+
+## Context
+
+- The package targets CPython 3.9+ and uses setuptools with optional `train`,
+  `gpu`, `metal`, and `test` extras.
+- Production inference loads a platform-specific native library through ctypes;
+  JAX/Flax/Optax/SentencePiece are used for reference inference, fine-tuning,
+  and export.
+- Model and tokenizer artifacts are commonly fetched from Hugging Face and may
+  be cached locally; `HF_HUB_OFFLINE=1` supports air-gapped use.
+- The native engine keeps process-global active state, so base and tuned agents
+  have ordering and process-isolation constraints.
+- The codebase map in `.planning/codebase/` is the evidence source for current
+  structure, integrations, conventions, testing, and concerns.
+
+## Constraints
+
+- **Audience**: Write for Python/ML beginners, while linking to source paths for
+  readers who need implementation detail.
+- **Language**: Chinese is the primary user-facing documentation language; keep
+  API names, commands, paths, and code identifiers exact.
+- **Platforms**: Cover both CPU-first setup and supported NVIDIA CUDA/Apple
+  Metal acceleration without claiming unsupported combinations.
+- **Compatibility**: Preserve public APIs, CLI behavior, checkpoint formats, and
+  `.cact` tensor ordering while improving documentation.
+- **Verification**: Every tutorial must state prerequisites, expected output,
+  and a practical way to verify success.
+
+## Key Decisions
+
+| Decision | Rationale | Outcome |
+|----------|-----------|---------|
+| Chinese-first documentation | The intended onboarding audience asked for Chinese guidance | - Pending |
+| Cover inference, LoRA, and deployment in one path | Users need a complete journey from install to usable tuned artifact | - Pending |
+| Support CPU and GPU guidance | Hardware availability varies and beginner setup should not assume CUDA | - Pending |
+| Record hardening risks before fixing them | Documentation can prevent misuse without expanding the first milestone into a security rewrite | - Pending |
+
+## Evolution
+
+This document evolves at phase transitions and milestone boundaries.
+
+**After each phase transition** (via `$gsd-transition`):
+1. Requirements invalidated? -> Move to Out of Scope with reason
+2. Requirements validated? -> Move to Validated with phase reference
+3. New requirements emerged? -> Add to Active
+4. Decisions to log? -> Add to Key Decisions
+5. "What This Is" still accurate? -> Update if drifted
+
+**After each milestone** (via `$gsd-complete-milestone`):
+1. Full review of all sections
+2. Core Value check - still the right priority?
+3. Audit Out of Scope - reasons still valid?
+4. Update Context with current state
+
+---
+*Last updated: 2026-08-31 after initialization*

.planning/REQUIREMENTS.md

@@ -0,0 +1,106 @@
+# Requirements: Needle Documentation and Onboarding
+
+**Defined:** 2026-08-31
+**Core Value:** A beginner can install Needle and reliably go from a first inference request to a fine-tuned, exported model without guessing which assets, commands, or runtime constraints apply.
+
+## v1 Requirements
+
+### Installation and Environment
+
+- [x] **INST-01**: A beginner can create an isolated Python 3.9+ environment and install the correct base, training, test, GPU, or Metal extras for their platform.
+- [x] **INST-02**: The guide explains native engine, checkpoint, tokenizer, Hugging Face cache locations, offline mode, and required environment variables.
+- [x] **INST-03**: The guide states a supported CPU/GPU/Metal matrix and gives a CPU fallback when acceleration is unavailable.
+
+### Inference Quickstart
+
+- [x] **INFR-01**: A beginner can run a copy-paste pre-trained inference example and identify the expected successful output.
+- [x] **INFR-02**: The guide demonstrates equivalent CLI and `Needle` Python API flows, including a minimal typed tool call or extraction example.
+- [x] **INFR-03**: Each quickstart includes a verification command and links common errors to troubleshooting guidance.
+
+### Model and Runtime Concepts
+
+- [ ] **MODL-01**: The documentation explains the native ctypes runtime path versus the JAX/Flax reference and training path.
+- [ ] **MODL-02**: The documentation describes the Simple Attention Network components, tokenizer contract, checkpoint formats, and `.cact` artifact roles with source links.
+- [ ] **MODL-03**: The documentation calls out process-global native state, base/tuned agent ordering, and when to use a fresh process.
+
+### LoRA Fine-Tuning
+
+- [ ] **LORA-01**: A beginner can create a valid JSONL fine-tuning dataset with documented fields, chat/tool markers, and target masking behavior.
+- [ ] **LORA-02**: A beginner can run a small local LoRA fine-tuning job, understand its prerequisites and resource knobs, and locate the adapter/checkpoint outputs.
+- [ ] **LORA-03**: The guide explains how to inspect or validate a trained adapter and documents the current limitation around interrupted-run resume.
+
+### Export and Deployment
+
+- [ ] **DEPL-01**: A beginner can merge a LoRA adapter, quantize it, and produce a `.cact` archive using the supported CLI workflow.
+- [ ] **DEPL-02**: The guide explains engine version, tensor order, model geometry, and tokenizer vocabulary compatibility requirements.
+- [ ] **DEPL-03**: A beginner can load the exported archive in a fresh process and compare its output with the reference or base path using a verification checklist.
+
+### Troubleshooting and Safety
+
+- [ ] **SAFE-01**: Troubleshooting maps installation, asset, backend, checkpoint, tokenizer, and export failures to observable symptoms and fixes.
+- [ ] **SAFE-02**: Documentation warns about untrusted pickle checkpoints, native downloads, API keys, and exposing the unauthenticated local playground.
+- [ ] **SAFE-03**: Documentation records known concurrency, dependency drift, performance, and test-coverage limitations without implying they are solved.
+
+### Documentation Quality
+
+- [x] **DOCS-01**: The primary onboarding path is Chinese-first, preserves exact commands/API identifiers, and links every conceptual claim to a source path or official reference.
+- [x] **DOCS-02**: Every tutorial states prerequisites, expected output, cleanup/cache behavior, and a practical verification step for CPU and supported accelerator branches.
+- [ ] **DOCS-03**: Examples are organized by user goal and can be checked in automated tests or a documented manual verification pass.
+
+## v2 Requirements
+
+### Productization
+
+- **PROD-01**: Provide authenticated hosted or multi-user serving guidance.
+- **PROD-02**: Provide a GUI for training job management and artifact browsing.
+- **PROD-03**: Add automatic artifact signing, checksums, or registry provenance enforcement.
+
+### Advanced Learning Material
+
+- **LEARN-01**: Provide notebook-based walkthroughs and interactive visualizations.
+- **LEARN-02**: Publish benchmark tables across model sizes, hardware backends, and quantization levels.
+
+## Out of Scope
+
+| Feature | Reason |
+|---------|--------|
+| Native engine rewrite | This milestone documents the current runtime contract rather than changing it. |
+| New `.cact` format or tensor ordering | Format changes require coordinated engine/export work and are not documentation-only. |
+| Production security hardening | Risks are documented and tracked, but code remediation is a later phase. |
+| Universal OS/GPU support claims | Backend availability changes; docs will state tested/supportable combinations only. |
+
+## Traceability
+
+| Requirement | Phase | Status |
+|-------------|-------|--------|
+| INST-01 | Phase 1 | Complete |
+| INST-02 | Phase 1 | Complete |
+| INST-03 | Phase 1 | Complete |
+| INFR-01 | Phase 1 | Complete |
+| INFR-02 | Phase 1 | Complete |
+| INFR-03 | Phase 1 | Complete |
+| MODL-01 | Phase 2 | Pending |
+| MODL-02 | Phase 2 | Pending |
+| MODL-03 | Phase 2 | Pending |
+| LORA-01 | Phase 3 | Pending |
+| LORA-02 | Phase 3 | Pending |
+| LORA-03 | Phase 3 | Pending |
+| DEPL-01 | Phase 4 | Pending |
+| DEPL-02 | Phase 4 | Pending |
+| DEPL-03 | Phase 4 | Pending |
+| SAFE-01 | Phase 2 | Pending |
+| SAFE-02 | Phase 2 | Pending |
+| SAFE-03 | Phase 2 | Pending |
+| DOCS-01 | Phase 1 | Complete |
+| DOCS-02 | Phase 1 | Complete |
+| DOCS-03 | Phase 4 | Pending |
+
+**Coverage:**
+
+- v1 requirements: 21 total
+- Mapped to phases: 21
+- Unmapped: 0
+
+---
+*Requirements defined: 2026-08-31*
+*Last updated: 2026-08-31 after initial definition*

.planning/ROADMAP.md

@@ -0,0 +1,111 @@
+# Roadmap: Needle Documentation and Onboarding
+
+## Overview
+
+Deliver a Chinese-first beginner journey through the existing Needle lifecycle:
+install and fetch assets, run inference, understand the model/runtime boundary,
+fine-tune with LoRA, and export/deploy a verified `.cact` artifact. Each phase
+ships a usable documentation slice with executable examples and explicit limits.
+
+## Phases
+
+- [x] **Phase 1: Install and First Inference** - Make a clean environment and first response repeatable. (completed 2026-08-31)
+- [ ] **Phase 2: Model and Runtime Concepts** - Explain architecture, artifacts, troubleshooting, and safety boundaries.
+- [ ] **Phase 3: LoRA Fine-Tuning** - Document data preparation and a reproducible local adaptation run.
+- [ ] **Phase 4: Export and Deployment Verification** - Teach merge, quantize, `.cact` loading, and end-to-end checks.
+
+## Phase Details
+
+### Phase 1: Install and First Inference
+
+**Goal**: A beginner can install Needle on CPU or a supported accelerator, obtain required assets, and run a first inference request.
+**Mode**: mvp
+**Depends on**: Nothing (first phase)
+**Requirements**: [INST-01, INST-02, INST-03, INFR-01, INFR-02, INFR-03, DOCS-01, DOCS-02]
+**Success Criteria** (what must be TRUE):
+
+  1. A clean Python 3.9+ environment can be installed using documented commands for CPU and supported accelerator branches.
+  2. A reader can fetch or locate engine/checkpoint/tokenizer assets and knows the cache and offline controls.
+  3. A reader can copy a CLI or `Needle` API example, observe the expected response, and run a verification check.
+  4. Common install and first-run failures link to actionable troubleshooting entries.
+
+**Plans**: 1/2 plans executed
+
+Plans:
+
+- [x] 01-01-PLAN.md
+- [x] 01-02-PLAN.md
+- [x] 01-01: Write installation, asset/cache, and backend matrix guide.
+- [x] 01-02: Write and verify CLI/API inference quickstart with expected output.
+
+### Phase 2: Model and Runtime Concepts
+
+**Goal**: A beginner can explain which runtime path and artifact applies to inference, training, and deployment, and can avoid known unsafe usage.
+**Mode**: mvp
+**Depends on**: Phase 1
+**Requirements**: [MODL-01, MODL-02, MODL-03, SAFE-01, SAFE-02, SAFE-03]
+**Success Criteria** (what must be TRUE):
+
+  1. Architecture documentation traces a request through `Needle`/ctypes and separately through JAX/Flax reference code.
+  2. The roles and compatibility constraints of `.pkl`, tokenizer, LoRA adapter, and `.cact` artifacts are clear with source links.
+  3. A reader can identify process-global state, pickle/download risks, playground exposure risks, and documented performance/dependency limitations.
+  4. Troubleshooting is organized by symptom and includes a safe next action.
+
+**Plans**: 2 plans
+
+Plans:
+
+- [ ] 02-01: Write model architecture, data-flow, and artifact glossary.
+- [ ] 02-02: Write troubleshooting and safety reference from codebase concerns.
+
+### Phase 3: LoRA Fine-Tuning
+
+**Goal**: A beginner can prepare valid supervision data and complete a small, reproducible LoRA fine-tuning run.
+**Mode**: mvp
+**Depends on**: Phase 2
+**Requirements**: [LORA-01, LORA-02, LORA-03]
+**Success Criteria** (what must be TRUE):
+
+  1. A documented JSONL example renders to the expected chat/tool markers and explains which tokens contribute to loss.
+  2. A small local fine-tuning command runs with stated CPU/GPU prerequisites and resource knobs.
+  3. Adapter/checkpoint outputs have documented paths and a validation or inspection step.
+  4. The guide clearly states current interrupted-run resume limitations and safe checkpoint handling.
+
+**Plans**: 2 plans
+
+Plans:
+
+- [ ] 03-01: Document JSONL schema, rendering/masking, and dataset validation.
+- [ ] 03-02: Document and run the LoRA CLI workflow with output verification.
+
+### Phase 4: Export and Deployment Verification
+
+**Goal**: A beginner can turn a trained adapter into a compatible `.cact` artifact and verify it in a fresh runtime process.
+**Mode**: mvp
+**Depends on**: Phase 3
+**Requirements**: [DEPL-01, DEPL-02, DEPL-03, DOCS-03]
+**Success Criteria** (what must be TRUE):
+
+  1. Merge, quantize, and build commands produce a named `.cact` archive from the documented adapter output.
+  2. Engine version, tensor order, geometry, and tokenizer vocabulary checks are explicit and testable.
+  3. A fresh process loads the archive and compares output against a reference/base path using a checklist.
+  4. Examples have a documented manual or automated verification route suitable for CI follow-up.
+
+**Plans**: 2 plans
+
+Plans:
+
+- [ ] 04-01: Write merge/quantize/export/deployment tutorial and compatibility checklist.
+- [ ] 04-02: Add or document example verification commands and review all links/commands.
+
+## Progress
+
+**Execution Order:**
+Phases execute in numeric order: 1 -> 2 -> 3 -> 4
+
+| Phase | Plans Complete | Status | Completed |
+|-------|----------------|--------|-----------|
+| 1. Install and First Inference | 2/2 | Complete    | 2026-08-31 |
+| 2. Model and Runtime Concepts | 0/2 | Not started | - |
+| 3. LoRA Fine-Tuning | 0/2 | Not started | - |
+| 4. Export and Deployment Verification | 0/2 | Not started | - |