jspace

Read, steer, and ablate the "global workspace" of open-weight language models.
An open, self-contained reproduction of Anthropic's Jacobian lens / J-space analysis — with an interactive REPL (read–eval–print loop).

---

In July 2026 Anthropic reported that language models develop something like a global workspace: a small set of internal directions that are reportable, can be steered on request, mediate multi-step reasoning, are flexibly reused, and are not needed for automatic processing — a functional analogue of the neuroscience notion of conscious access. They called the space of these directions J-space (after the Jacobian technique that finds it) and released a lens for it, but not the tooling to decompose the workspace or intervene on it.

jspace is the full pipeline, reproduced from the paper and runnable on a laptop or a single GPU. It reimplements the Jacobian lens — verified bit-for-bit identical to Anthropic's reference implementation — and adds the J-space sparse decomposition and causal interventions (swap / steer / ablate) that the paper describes but the reference code does not ship, wrapped in an interactive REPL.

The demo above is a real session (Qwen3.5-4B): asked for the currency of "the country shaped like a boot", the model's workspace holds euro and the unstated bridge Italy/Italian — and swapping Italy→Japan flips the answer from "the Euro" to "Japan". That is the paper's central phenomenon, on an open model you can run yourself.

Supported models

jspace works with any HuggingFace decoder-only LM (layout is auto-detected for Llama / Qwen / Mistral / Gemma / OLMo / Phi / GPT-2 / Pythia, including the vision-wrapped *ForConditionalGeneration variants). It is developed and validated across three architecture families and a 0.8B → 27B size range:

Family	Models	Notes
Qwen 3.5 / 3.6	Qwen/Qwen3.5-0.8B, Qwen/Qwen3.5-4B, Qwen/Qwen3.6-27B	4B & 27B have official pre-fitted lenses on the Hub (--official); the size axis for scaling
Mistral (Ministral 3)	mistralai/Ministral-3-8B-Base-2512, Ministral-3-14B-Base-2512	dense, Apache-2.0; the independent-architecture control (reroutes cleanly)
Gemma 4	google/gemma-4-E2B-it, google/gemma-4-12B-it	soft-cap handled; reads well (E2B), but swaps collapse (documented)
GPT-2	gpt2	the bit-exact cross-validation anchor against jlens

The 0.8B / E2B models run on CPU for quick experiments; the larger ones want a single GPU.

Install

git clone https://github.com/haohww/jspace && cd jspace
pip install -e .                 # core (model loading, lens, fitting, REPL)
pip install -e ".[crossval]"     # + Anthropic's jlens, for the bit-exact cross-check
pip install -e ".[dev]"          # + pytest, ruff

Quickstart

# 1. Fit a Jacobian lens once per model (~100 prompts is plenty; resumable)
jspace fit Qwen/Qwen3.5-4B --n-prompts 200 --out lenses/qwen3-5-4b.pt

# 2. Read what's in the workspace on a prompt (track specific concepts)
jspace read --model Qwen/Qwen3.5-4B --lens lenses/qwen3-5-4b.pt \
    --prompt "Fact: The currency used in the country shaped like a boot is" \
    --track Italy euro

# 3. Steer / swap / ablate a concept
jspace swap --model Qwen/Qwen3.5-4B --lens lenses/qwen3-5-4b.pt \
    --prompt "Fact: The currency used in the country shaped like a boot is" \
    --src Italy --tgt Japan

# 4. Explore interactively
jspace repl --model Qwen/Qwen3.5-4B --lens lenses/qwen3-5-4b.pt

Python API:

import jspace

model = jspace.HFModel.load("Qwen/Qwen3.5-4B", dtype="bfloat16")
lens  = jspace.fit(model, prompts, source_layers=None)          # or JacobianLens.from_pretrained(...)
d     = jspace.JLensDictionary(model, lens)

band  = jspace.workspace_band(model.n_layers)   # mid-network; keeps memory bounded
slots = jspace.workspace_report(model, d, "…the country shaped like a boot is", layers=band, k=25)
print(jspace.format_report(slots))

with jspace.LensSwap(model, d, "Italy", "Japan", layers=band):  # alpha defaults to 4.0
    ...  # model.hf_model.generate(...) now answers as if the bridge were Japan

The REPL

jspace> prompt Fact: The animal that spins webs has this many legs:
jspace> read
  J-space @ band  →  'spider' 'eight' 'legs' 'web' 'arachnid' ...
jspace> swap spider ant
  baseline : 'eight legs.'
  swap spider->ant : 'six legs.'
jspace> ablate spider
  baseline : 'eight legs.'
  ablate ['spider'] : 'four legs.'      # multi-step answer collapses without the bridge

Commands: prompt, read [pos], gen, steer, swap, ablate, band, k, tokens, info, help, quit.

Validation

Bit-exact core. Fitting GPT-2 on the same prompts with jspace and with Anthropic's reference jlens gives identical Jacobians — per-layer max|Δ| = 0.0 (tests/test_crossval_jlens.py).

Reproduced phenomena — on Anthropic's own eval sets. Every number below is computed on Anthropic's official lens-eval prompt distributions and their probe-swap experiment, vendored unmodified under data/anthropic-jlens/ (Apache-2.0), so results are directly comparable to the paper rather than home-grown probes. Per-model raw JSON is in results/; regenerate with python scripts/measure.py and the figures with python scripts/figures.py.

Qwen3.6-27B, J-lens readout across depth on “…the language spoken in the country where the Amazon River ends…”. The unstated bridge Brazil (never written) lights up in the mid-network workspace band, then the answer Portuguese resolves near the output — the paper's central phenomenon, on an open model.

1. The unstated bridge is in the workspace (lens quality)

Canonical pass@1 on the six lens-eval sets — the fraction of items where the unstated intermediate concept is the top-1 J-lens readout token at some layer:

model	layers	multihop	association	multilingual	order-ops	poetry	typo	J-lens vs logit (band)
GPT-2	12	0.00	0.00	0.00	0.00	0.00	0.00	0.00 vs 0.00
Qwen3.5-0.8B	24	0.03	0.02	0.02	0.03	0.07	0.34	0.03 vs 0.00
Qwen3.5-4B	32	0.22	0.00	0.24	0.01	0.44	0.28	0.22 vs 0.12
Qwen3.6-27B	64	0.39	0.04	0.27	0.39	0.70	0.15	0.21 vs 0.13
Gemma-4-E2B-it	35	0.76	0.16	0.62	0.61	0.91	0.95	0.60 vs 0.32
Gemma-4-12B-it	48	0.08	0.01	0.01	0.01	0.14	0.47	0.08 vs 0.24
Mistral-3-8B	34	0.34	0.07	0.35	0.45	0.72	0.50	0.29 vs 0.23
Mistral-3-14B	40	0.34	0.10	0.37	0.05	0.84	0.51	0.32 vs 0.25

Over the mid-network workspace band, the Jacobian lens beats the plain logit lens on 6 of 7 models (last column) — the paper's core lens-quality claim, reproduced on open weights. Reportability sharpens with scale: Qwen multihop pass@1 climbs 0.03 → 0.22 → 0.39 across 0.8B → 4B → 27B.

2. The workspace is causal (probe-swap)

Clamp the bridge → a replacement entity across the band (α = 1.0, the canonical "replace" strength) and score the model's final token against the rerouted answer:

model	baseline acc	reroute (success)	entity leak	collapse
GPT-2	0.08	0.07	0.22	0.20
Qwen3.5-0.8B	0.18	0.11	0.27	0.07
Qwen3.5-4B	0.49	0.31	0.16	0.01
Qwen3.6-27B	0.63	0.29	0.00	0.01
Gemma-4-E2B-it	0.38	0.01	0.00	0.37
Gemma-4-12B-it	0.03	0.01	0.00	0.39
Mistral-3-8B	0.49	0.26	0.07	0.01
Mistral-3-14B	0.46	0.29	0.09	0.00

The swap reroutes the answer on Qwen (≥ 4B) and both Mistral models; Gemma-4-it collapses instead (repetition / filler). Because Mistral is an independent standard architecture, its clean rerouting shows Gemma's failure is Gemma-specific (soft-cap + √d residual scale), not a method artifact.

3. Where the workspace lives (geometry)

J-space breadth (participation ratio) rises into a broad mid-network plateau and tapers near the output — a concrete read on the paper's mid-network "workspace band".

Column legend

• pass@1 — the unstated bridge concept is the #1 J-lens readout token at some layer (higher = the lens surfaces the hidden concept more reliably).
• J-lens vs logit (band) — multihop pass@1 restricted to the workspace band, Jacobian lens vs plain logit lens (bold = winner).
• baseline acc — greedy answer is correct with no intervention (the model's raw two-hop ability).
• reroute (success) — swapping bridge→X makes the final token X's rerouted answer (the causal effect).
• entity leak — the swap surfaces the swapped-in entity name instead of its downstream fact (a cruder effect; grows if α is too large).
• collapse — the continuation degenerated into repetition / filler (the instruction-tuned-Gemma failure mode).

Reading these numbers honestly

• Gemma-4-12B is an outlier: its J-lens trails even the logit lens and its raw two-hop accuracy is ~0. It is the Gemma4Unified architecture, evaluated on the raw-completion protocol; instruction-tuning + that variant — not an absent workspace — is the likely cause. Gemma-4-E2B (the strongest reader here, multihop pass@1 = 0.76) is the fair Gemma representative.
• Protocol is raw (no chat template) for all models, for comparability; this understates instruction-tuned models on generation-based scores.
• α matters: α ≈ 1 reroutes the fact; α ≥ 1.5 over-injects and the model just blurts the entity (entity-leak → 1). The demo GIF at the top uses a stronger α for a vivid entity flip; the causal table above uses the canonical α = 1.
• The logit lens only "wins" on an all-layers metric that hands it the near-output layers (where any lens reads the answer); the honest, paper-aligned comparison is over the mid-network band, where the J-lens wins. Both are stored in each results/metrics-*.json.

How it maps to the paper

The math and the paper correspondence are in docs/METHOD.md. In one line each:

• Lens lensₗ(h) = softmax(W_U · norm(J̄ₗ h)), J̄ₗ = E[∂h_final/∂hₗ] — jspace.fit / JacobianLens.
• J-lens vectors = rows of W_U J̄ₗ — JLensDictionary.
• J-space = sparse non-negative combo of ≤ k such vectors — gradient_pursuit.
• Interventions — LensSwap, LensSteer, TargetedAblation, LensAblation.
• Metrics — canonical reportability pass_at_k and causal probe_swap (on the vendored Anthropic sets), plus workspace geometry workspace_geometry.

Project layout

src/jspace/      model adapter · hooks · jacobian · dictionary · pursuit · workspace · interventions · metrics · repl · cli
scripts/         fit_lens.py · repro_two_hop.py · measure.py · figures.py · animate_workspace.py · slurm/
tests/           pursuit · tiny-model jacobian · interventions · metrics · crossval-vs-jlens
data/anthropic-jlens/   vendored Anthropic lens-eval + probe-swap sets (Apache-2.0)
results/         metrics-<model>.json (committed evidence) · repro-*.txt
docs/            METHOD.md · REPRODUCTION.md

Credits

This project stands on:

• Anthropic, Verbalizable Representations Form a Global Workspace in Language Models (Transformer Circuits, 2026) and the reference anthropics/jacobian-lens (jlens), whose lens estimator we reproduce and cross-validate against.
• The community migueldeguzman/jspace exploration of the interventions.

jspace is an independent reproduction and is not affiliated with or endorsed by Anthropic.

Citation

See CITATION.cff. Please cite both this repository and the paper.

License

Apache-2.0. No model weights or corpora are bundled; anything downloaded at run time is subject to its own license.