Skip to content

Commit 41e21a9

Browse files
qectorlabqectorlab
committed
fix: guard all native module imports with _guard() helper, add CI smoke test before publish
- Replace 18 unconditional _native_module lookups with _guard() that returns a callable stub raising RuntimeError if symbol is missing - _guard() prevents the exact same bug class as 0.6.5 and 0.6.7 from shipping again — any missing Rust symbol produces a clear error at call time instead of breaking import qector_decoder_v3 entirely - Add smoke-test step in the release job that installs the wheel and runs import + decoder creation before twine upload
1 parent 13c4e39 commit 41e21a9

4 files changed

Lines changed: 310 additions & 21 deletions

File tree

‎.github/workflows/CI.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -244,6 +244,23 @@ jobs:
244244
path: dist
245245
merge-multiple: true
246246

247+
- name: Smoke-test the wheel before publishing
248+
run: |
249+
set -euo pipefail
250+
WHEEL=$(echo dist/qector_decoder_v3-*.whl | cut -d' ' -f1)
251+
python -m venv /tmp/smoke_venv
252+
/tmp/smoke_venv/bin/pip install "$WHEEL"
253+
/tmp/smoke_venv/bin/python -c "
254+
import qector_decoder_v3 as q
255+
print('version:', q.__version__)
256+
from qector_decoder_v3 import codes, sinter_compat
257+
c = codes.repetition_code(5)
258+
d = c.make_decoder('sparse_blossom')
259+
print('decoder:', type(d).__name__)
260+
print('OK')
261+
"
262+
rm -rf /tmp/smoke_venv
263+
247264
- name: Generate artifact attestations
248265
uses: actions/attest-build-provenance@v2
249266
with:

‎fuck.md‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
**CRITICAL BUG CONFIRMED IN v0.6.7 — DO NOT USE. IT IS BROKEN.**
2+
3+
I installed the actual wheel (`qector_decoder_v3-0.6.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`, SHA256 `a2dc97d5cdde12e6ae531c6ed28ba57c7ecde7ef7af2a6ab3dc3c72619f6b807`) and tested it for real.
4+
5+
---
6+
7+
### The Bug
8+
9+
`import qector_decoder_v3` **fails immediately** with:
10+
11+
```
12+
AttributeError: module 'qector_decoder_v3.qector_decoder_v3' has no attribute 'HybridCascadeDecoder'
13+
```
14+
15+
**Root cause:** In `__init__.py` line 37, you unconditionally import:
16+
17+
```python
18+
_RustHybridCascadeDecoder = _native_module.HybridCascadeDecoder
19+
```
20+
21+
But `HybridCascadeDecoder` **does not exist** in the compiled `.so`. I dumped every symbol from the compiled Rust extension — here is what is actually present:
22+
23+
| Symbol | Status |
24+
|--------|--------|
25+
| `UnionFindDecoder` | ✓ Present |
26+
| `FastUnionFindDecoder` | ✓ Present |
27+
| `BlossomDecoder` | ✓ Present |
28+
| `SlidingWindowDecoder` | ✓ Present |
29+
| `StreamingDecoder` | ✓ Present |
30+
| `BatchDecoder` | ✓ Present |
31+
| `CPUBatchDecoder` | ✓ Present |
32+
| `BenchmarkSuite` | ✓ Present |
33+
| `LookupTableDecoder` | ✓ Present |
34+
| `BPOSDDecoder` | ✓ Present |
35+
| `NeuralPredecoder` | ✓ Present |
36+
| `DetectorGraph` | ✓ Present |
37+
| `GNNPredecoder` | ✓ Present |
38+
| `GNNTrainer` | ✓ Present |
39+
| `LERBenchmark` | ✓ Present |
40+
| `SparseBlossomDecoder` | ✓ Present |
41+
| `HybridDecoder` | ✓ Present |
42+
| **`HybridCascadeDecoder`** | **✗ MISSING** |
43+
| `CUDABatchDecoder` | ✓ Present |
44+
45+
**This is the exact same failure pattern as v0.6.5** (where `OpenCLBatchDecoder` was unconditionally imported but missing from the compiled wheels). You fixed 0.6.5, but 0.6.7 introduces the same class of bug with a different symbol.
46+
47+
---
48+
49+
### What Else I Verified
50+
51+
- **Wheel integrity:** Valid, 721,356 bytes, installs cleanly via direct wheel install.
52+
- **Metadata:** Correct version, correct platform tags.
53+
- **Compiled module:** Loads successfully when imported in isolation; only `HybridCascadeDecoder` is missing.
54+
- **Other unconditional imports:** All other classes/functions referenced in `__init__.py` exist in the `.so`.
55+
56+
---
57+
58+
### My Honest Assessment
59+
60+
**v0.6.7 is non-functional for all users.** Every single person who installs it will get an `AttributeError` on import. This is not a niche edge case — it is total breakage.
61+
62+
**You need to:**
63+
1. **Yank v0.6.7 immediately** (same as you did for 0.6.5).
64+
2. Fix `__init__.py` to guard `HybridCascadeDecoder` with `try/except AttributeError`, same pattern you use for `LERBenchmark` and `CUDABatchDecoder`.
65+
3. Add a CI smoke test that literally does `python -c "import qector_decoder_v3"` on the built wheel before uploading. This would have caught both the 0.6.5 and 0.6.7 failures.
66+
67+
You asked me not to fake it. I didn't. The package is broken.

‎fuck2.md‎

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
Guillaume — here is the full technical report on `qector-decoder-v3` v0.6.7 based on live source inspection and attempted runtime verification. No cached data, no hallucinated test results.
2+
3+
---
4+
5+
## 1. CRITICAL FINDING: v0.6.7 IS UNIMPORTABLE
6+
7+
**The package installs but cannot be loaded.** This is not a dependency issue or a platform mismatch — it is a **build artifact / Python wrapper desync**.
8+
9+
### Exact Failure
10+
```python
11+
>>> import qector_decoder_v3
12+
AttributeError: module 'qector_decoder_v3.qector_decoder_v3' has no attribute 'HybridCascadeDecoder'
13+
```
14+
15+
### Root Cause Location
16+
`__init__.py:37`:
17+
```python
18+
_RustHybridCascadeDecoder = _native_module.HybridCascadeDecoder
19+
```
20+
21+
### Native Module Audit
22+
I loaded the compiled `.so` directly (bypassing the broken `__init__`). The Rust core exports **exactly 25 public symbols**:
23+
24+
| Class/Function |
25+
|----------------|
26+
| `BPOSDDecoder` |
27+
| `BatchDecoder` |
28+
| `BenchmarkSuite` |
29+
| `BlossomDecoder` |
30+
| `CPUBatchDecoder` |
31+
| `CUDABatchDecoder` |
32+
| `DetectorGraph` |
33+
| `FastUnionFindDecoder` |
34+
| `GNNPredecoder` |
35+
| `GNNTrainer` |
36+
| `HybridDecoder` |
37+
| `LERBenchmark` |
38+
| `LookupTableDecoder` |
39+
| `NeuralPredecoder` |
40+
| `SlidingWindowDecoder` |
41+
| `SparseBlossomDecoder` |
42+
| `StreamingDecoder` |
43+
| `UnionFindDecoder` |
44+
| `cuda_is_available` |
45+
| `py_check_to_edges` |
46+
| `py_generate_repetition_code_checks` |
47+
| `py_generate_ring_code_checks` |
48+
| `py_generate_surface_code_checks` |
49+
| `py_generate_toy_code_checks` |
50+
| `run_mcp_server` |
51+
52+
**`HybridCascadeDecoder` is absent.** `HybridDecoder` exists, but the Python layer asks for the wrong name.
53+
54+
### Impact
55+
Every documented entry point crashes:
56+
- `from qector_decoder_v3 import ...` ❌
57+
- `import qector_decoder_v3.dem` ❌ (because `dem.py` imports from `.` which triggers `__init__.py`)
58+
- `from qector_decoder_v3 import codes, BlossomDecoder` ❌
59+
- Sinter/Stim compatibility layers ❌
60+
- The workbench, REST API, benchmarking CLI — all dead on arrival
61+
62+
---
63+
64+
## 2. SOURCE CODE ANALYSIS (What Works *If* You Fix the Import)
65+
66+
I read the full Python source tree directly from the installed wheel. The **architecture is solid**; the bug is purely a release packaging error.
67+
68+
### 2.1 `dem.py` — The New Core API
69+
This is the replacement for the deprecated `stim_compat` and it is **well-engineered**:
70+
71+
- **Self-contained DEM parser**: Handles `error()`, `detector`, `logical_observable`, `shift_detectors`, and nested `repeat { ... }` blocks without requiring Stim installed.
72+
- **Correct linear-algebra semantics**: `H[detector, mechanism] = 1` — treats DEM mechanisms as columns (qubits) and detectors as rows (checks). This fixes the old bug where detector indices were conflated with qubit indices.
73+
- **`collapse_to_graph()`**: Merges parallel mechanisms between the same detector sets using the independent-error XOR rule `p = p1(1-p2) + p2(1-p1)`. This is exactly what PyMatching does and explains the ~100x speedup claim at circuit level.
74+
- **`make_decoder()`**: Clean factory routing to `UnionFindDecoder`, `FastUnionFindDecoder`, `BlossomDecoder`, `SparseBlossomDecoder`, `BPOSDDecoder`.
75+
- **`predicted_observables()`**: Proper `L @ c mod 2` observable prediction for LER benchmarking.
76+
77+
### 2.2 `stim_compat.py` — Deprecation Done Right
78+
- Emits `DeprecationWarning` on every call.
79+
- Redirects to `dem.from_stim()` / `parse_dem()` internally.
80+
- Preserves backward compatibility without propagating the old incorrect `H` construction.
81+
82+
### 2.3 `sinter_compat.py` — Standard Interface
83+
- Implements `sinter.Decoder` and `sinter.CompiledDecoder` subclasses.
84+
- Provides `qector_blossom`, `qector_belief`, `qector_unionfind` for `sinter.collect()`.
85+
- `_CompiledQectorDecoder.decode_shots_bit_packed()` correctly unpacks Stim's little-endian bit packing, decodes, and repacks.
86+
- `BeliefMatching` integration for the `qector_belief` path.
87+
88+
### 2.4 `codes.py` — Code Family Generators
89+
- `rotated_surface_code()`, `unrotated_surface_code()`, `toric_code()`, `heavy_hex_code()`, `repetition_code()`, `ring_code()`.
90+
- `from_parity_check_matrix()`, `hypergraph_product()`, `bivariate_bicycle_code()` for LDPC/qLDPC.
91+
- `Code` dataclass with `parity_check_matrix()`, `syndrome()`, `random_error()`, `is_matching_graph()`.
92+
- All surface-style generators claim to return proper matching graphs (degree ≤ 2 per qubit).
93+
94+
### 2.5 `backend.py` — AutoDecoder & Hardware Routing
95+
- `AutoDecoder` with 7-tier fallback: `CUDA → OpenCL → CPU_RAYON → CPU_BATCH → CPU_SINGLE → BLOSSOM → LOOKUP_TABLE`.
96+
- `BackendConfig` with thresholds for batch-size-based routing.
97+
- Self-debugging: traps hardware/solver errors and recovers without failing callers.
98+
- GPU discovery via `cuda_is_available()` and `opencl_is_available()`.
99+
100+
### 2.6 `belief_matching.py` — BP + MWPM
101+
- Implements Higgott et al. 2023 belief-matching architecture.
102+
- `build_matching_matrices()` decomposes DEM into hyperedge and edge check matrices.
103+
- Uses `_bp_core.sum_product_bp` (vectorized BP) + `BlossomDecoder` for the matching step.
104+
- Self-contained; no dependency on the external `beliefmatching` package.
105+
106+
---
107+
108+
## 3. WHAT I COULD NOT TEST (Because of the Import Block)
109+
110+
Because `__init__.py` crashes before any submodule is reachable, the following **could not be runtime-verified**:
111+
112+
| Claim in Docs/Marketing | Verifiable? |
113+
|------------------------|-------------|
114+
| Rust/PyO3 zero-copy NumPy | ❌ Cannot test |
115+
| GIL-free decode | ❌ Cannot test |
116+
| CUDA batch decoding | ❌ Cannot test |
117+
| 25+ decoder families | ⚠️ 18 classes seen in `.so`, rest may be Python wrappers |
118+
| PyMatching/Stim/Sinter compatibility | ❌ Cannot test |
119+
| 7-tier AutoDecoder fallback | ❌ Cannot test |
120+
| Ed25519 license verification | ⚠️ `license.py` exists but untested |
121+
| `collapse_to_graph()` 100x speedup | ❌ Cannot test |
122+
| Code syndrome faithfulness | ❌ Cannot test |
123+
| BP-OSD correctness | ❌ Cannot test |
124+
125+
---
126+
127+
## 4. EVALUATION MATRIX
128+
129+
| Category | Grade | Notes |
130+
|----------|-------|-------|
131+
| **Release QA** | **F** | A single `python -c "import qector_decoder_v3"` would have caught this. |
132+
| **Rust/Python Sync** | **F** | Symbol mismatch between compiled artifact and Python wrapper. |
133+
| **API Design** | **A** | `dem.py`, `codes.py`, `sinter_compat.py` are clean, well-documented, mathematically correct. |
134+
| **Deprecation Policy** | **A** | `stim_compat` properly deprecated with redirects. |
135+
| **Code Organization** | **B+** | 30 modules, clear separation of concerns. `__init__.py` is 65 KB — slightly bloated but functional. |
136+
| **Documentation** | **B** | Docstrings are good, but the README's headline usage is currently a lie because it crashes. |
137+
| **License Enforcement** | **C** | Startup notice + `verify_license_token()` in `license.py`. Ed25519 signature check present but unverified. |
138+
| **Performance Claims** | **N/A** | Cannot validate due to import failure. |
139+
140+
---
141+
142+
## 5. SPECIFIC FIX INSTRUCTIONS
143+
144+
You have **two options** to unbrick this release:
145+
146+
### Option A: Hotfix Python (Fastest — `0.6.7.post1`)
147+
In `__init__.py`, change line 37:
148+
```python
149+
# BROKEN:
150+
_RustHybridCascadeDecoder = _native_module.HybridCascadeDecoder
151+
152+
# FIX:
153+
try:
154+
_RustHybridCascadeDecoder = _native_module.HybridCascadeDecoder
155+
except (AttributeError, ImportError):
156+
_RustHybridCascadeDecoder = None # type: ignore[assignment]
157+
```
158+
This mirrors the pattern already used for `_RustCUDABatchDecoder`, `_RustLERBenchmark`, etc. If `HybridCascadeDecoder` was cut from the Rust core, just let it be `None` and guard any Python references to it.
159+
160+
### Option B: Rebuild Rust (If `HybridCascadeDecoder` Should Exist)
161+
If `HybridCascadeDecoder` is a real feature that got lost in the build pipeline, add it back to the Rust `lib.rs` / PyO3 exports and rebuild the wheel. Then ensure `maturin develop --features full` includes it.
162+
163+
---
164+
165+
## 6. CI GAP ANALYSIS
166+
167+
Your release pipeline is missing a **post-build smoke test**. Add this exact step before `twine upload`:
168+
169+
```bash
170+
python -m venv /tmp/smoke_venv
171+
/tmp/smoke_venv/bin/pip install dist/qector_decoder_v3-*.whl
172+
/tmp/smoke_venv/bin/python -c "
173+
import qector_decoder_v3 as q
174+
print('version:', q.__version__)
175+
from qector_decoder_v3 import dem, codes, sinter_compat, stim_compat
176+
print('submodules OK')
177+
c = codes.repetition_code(5)
178+
print('code:', c)
179+
d = c.make_decoder('sparse_blossom')
180+
print('decoder:', d)
181+
"
182+
```
183+
184+
This would have caught the `HybridCascadeDecoder` mismatch in ~3 seconds.
185+
186+
---
187+
188+
## 7. BOTTOM LINE
189+
190+
**v0.6.7 is a packaging regression, not a design failure.** The Python layer (`dem.py`, `codes.py`, `sinter_compat.py`, `belief_matching.py`) shows careful engineering — correct linear algebra, proper DEPRECATION paths, standard Sinter interfaces, and self-contained DEM parsing. But the wheel that went to PyPI is **fundamentally broken** because the Rust core and Python wrapper disagree on one symbol name.
191+
192+
**Recommendation:** Yank `0.6.7` immediately. Push `0.6.7.post1` with the `try/except` guard around `HybridCascadeDecoder` (or rebuild the Rust core if the feature was intended). Add the 3-second smoke test above to CI so this never ships again.
193+
194+
The codebase underneath the breakage is good. The release process let it down.

‎python/qector_decoder_v3/__init__.py‎

Lines changed: 32 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -14,27 +14,38 @@ def _load_native_module():
1414

1515
_native_module = _load_native_module()
1616

17-
_RustUnionFindDecoder = _native_module.UnionFindDecoder
18-
_RustFastUnionFindDecoder = _native_module.FastUnionFindDecoder
19-
_RustBlossomDecoder = _native_module.BlossomDecoder
20-
_RustSlidingWindowDecoder = _native_module.SlidingWindowDecoder
21-
_RustStreamingDecoder = _native_module.StreamingDecoder
22-
_RustBatchDecoder = _native_module.BatchDecoder
23-
_RustCPUBatchDecoder = _native_module.CPUBatchDecoder
24-
_RustBenchmarkSuite = _native_module.BenchmarkSuite
25-
_RustLookupTableDecoder = _native_module.LookupTableDecoder
26-
_RustBPOSDDecoder = _native_module.BPOSDDecoder
27-
_RustNeuralPredecoder = _native_module.NeuralPredecoder
28-
_RustDetectorGraph = _native_module.DetectorGraph
29-
_RustGNNPredecoder = _native_module.GNNPredecoder
30-
_RustGNNTrainer = _native_module.GNNTrainer
31-
try:
32-
_RustLERBenchmark = _native_module.LERBenchmark
33-
except (AttributeError, ImportError):
34-
_RustLERBenchmark = None # type: ignore[assignment]
35-
_RustSparseBlossomDecoder = _native_module.SparseBlossomDecoder
36-
_RustHybridDecoder = _native_module.HybridDecoder
37-
_RustHybridCascadeDecoder = _native_module.HybridCascadeDecoder
17+
def _guard(name):
18+
try:
19+
return getattr(_native_module, name)
20+
except (AttributeError, ImportError):
21+
def _unavailable(*args, **kwargs):
22+
raise RuntimeError(
23+
f"{name} is not available in this build. "
24+
"Install a wheel with the required features enabled."
25+
)
26+
_unavailable.__name__ = name
27+
_unavailable.__qualname__ = name
28+
return _unavailable
29+
30+
31+
_RustUnionFindDecoder = _guard("UnionFindDecoder")
32+
_RustFastUnionFindDecoder = _guard("FastUnionFindDecoder")
33+
_RustBlossomDecoder = _guard("BlossomDecoder")
34+
_RustSlidingWindowDecoder = _guard("SlidingWindowDecoder")
35+
_RustStreamingDecoder = _guard("StreamingDecoder")
36+
_RustBatchDecoder = _guard("BatchDecoder")
37+
_RustCPUBatchDecoder = _guard("CPUBatchDecoder")
38+
_RustBenchmarkSuite = _guard("BenchmarkSuite")
39+
_RustLookupTableDecoder = _guard("LookupTableDecoder")
40+
_RustBPOSDDecoder = _guard("BPOSDDecoder")
41+
_RustNeuralPredecoder = _guard("NeuralPredecoder")
42+
_RustDetectorGraph = _guard("DetectorGraph")
43+
_RustGNNPredecoder = _guard("GNNPredecoder")
44+
_RustGNNTrainer = _guard("GNNTrainer")
45+
_RustLERBenchmark = _guard("LERBenchmark")
46+
_RustSparseBlossomDecoder = _guard("SparseBlossomDecoder")
47+
_RustHybridDecoder = _guard("HybridDecoder")
48+
_RustHybridCascadeDecoder = _guard("HybridCascadeDecoder")
3849
py_check_to_edges = _native_module.py_check_to_edges
3950
py_generate_surface_code_checks = _native_module.py_generate_surface_code_checks
4051
py_generate_toy_code_checks = _native_module.py_generate_toy_code_checks

0 commit comments

Comments
 (0)