[P0] Delete persisted pending embeddings by exact identity before staged resume regeneration

[100yenadmin]

Parent: #130

Problem

Installed 1.6.10-electric.4 resumed the preserved ClawSweeper staged generation, skipped 3,040 cached snapshot rows owned by the pending checkpoint window, then failed on the first regenerated batch with a duplicate CodeEmbedding.id primary key.

The current resume path intentionally does not restore pending snapshot rows, but pending rows already persisted inside the staged LadybugDB generation can remain. runEmbeddingPipeline force-selects those nodes while its stale-row deletion is gated by the restored hash map, which excludes pending owners. The live VECTOR index can therefore reject CREATE even though the restore side skipped the row.

The failure was contained to .gitnexus/lbug.staged-work; no canonical index or registry row was created.

Required behavior

• Carry exact skipped pending row IDs into the pipeline without treating their hashes as reusable.
• Before each regenerated batch, delete any known persisted pending row by exact primary-key identity.
• Keep delete/reinsert exposure bounded to the current embedding batch.
• Preserve the at-most-5,000-node pending checkpoint contract and existing snapshot preflight guarantees.
• Add a real LadybugDB regression where the staged database already contains the pending row and the snapshot restore deliberately skips it.

Acceptance

The preserved ClawSweeper staged generation resumes on the exact released fix, promotes once, registry/meta/database counts agree, live doctor reports vector-index, and two MCP semantic queries are non-partial. No retry on .4 before the fix.

Evidence: /Volumes/LEXAR/Codex/session-notes/2026-07-20/gitnexus-repository-health-sprint/evals/clawsweeper-electric4-resume.log.

[evaos-code-review-bot]

evaOS issue enrichment

Issue: #162 - [P0] Delete persisted pending embeddings by exact identity before staged resume regeneration
State: open; milestone: GitNexus Repository Health & Large-Repo Reliability Sprint

Related issues/PRs: #130.
Existing labels: P0.
Suggested labels: bug, docs.
Suggested owners: none.

Related context

• EPIC: Restore truthful semantic retrieval and canonical index lifecycle #130 - mentioned in issue metadata; inspect for dependency, duplicate, or prior-decision risk.

Agent-start packet

Problem shape: [P0] Delete persisted pending embeddings by exact identity before staged resume regeneration - Parent: #130 ## Problem Installed 1.6.10-electric.4 resumed the preserved ClawSweeper staged generation, skipped 3,040 cached snapshot rows owned by the pending checkpoint window, then failed on the first regenerated
Product fit: Architecture/integration issue; check repo policy, prior decisions, and operational constraints before implementation.
Implementation wedge: Start with a bounded adapter/contract or dry-run packet before changing live runtime behavior.

Build / borrow / buy scan:

• Build: identify the smallest implementation wedge that proves the issue without broad rollout.
• Borrow: inspect same-repo prior art, repo memory, GitNexus context, and allowlisted cross-repo examples first.
• Buy/use: deferred; no product, architecture, UX, library-choice, integration, roadmap, or market-positioning trigger detected.

Candidate sources:

• VISION.md / repo policy / repo-memory.md / GitNexus packet.
• Same-repo issues and PRs referenced by issue metadata.

Acceptance criteria:

• Preserve and execute the issue's stated acceptance criteria.
• Include source citations for any external OSS/library/current-market claims.
• Keep issue-enrichment comments sticky and idempotent.
• Keep labels, owners, reviewers, and roadmap fields suggestion-only.

Proof plan:

• Focused unit/fixture test for the changed planner behavior.
• Dry-run issue-enrichment evidence packet before live comment posting.
• Redaction check for issue body, citations, and generated markdown.
• Contract or degraded-mode proof before runtime promotion.

Known traps:

• Do not treat related links as proof by themselves; each source needs a reason why it matters.
• Do not scan old issue backlogs or exceed repo-level enrichment throttles.

Non-goals:

• Do not auto-apply labels, owners, reviewers, roadmap fields, or milestones.
• Do not bulk-enrich old backlog issues from this planner output.
• Do not claim external market or OSS research was performed unless cited sources are present.

Context-source taxonomy:

• [enabled] vision_repo_policy_memory_gitnexus: Use repo vision, policy, memory, and GitNexus first before external research.
• [enabled] same_repo_issues_prs: Issue metadata contains same-repo references.
• [deferred] allowlisted_cross_repo_github: Deferred until product, UX, market-positioning, or library-choice signal appears.
• [deferred] external_oss_examples: Deferred for low-research-value issue class.
• [deferred] library_api_docs: Deferred unless a library/API decision is present.
• [deferred] current_market_examples: Deferred for implementation-only issues.
• [enabled] internal_mcp_docs: Use internal docs/MCP context when available; missing context is degraded, not blocking.

Validation suggestions

• Confirm owner, acceptance criteria, and validation evidence before implementation.

Triage gaps

• Owner or reviewer signal not detected in issue body.

No labels, owners, reviewers, or roadmap fields were changed by this bot.

[100yenadmin]

Installed-release acceptance reopened this defect.

1.6.10-electric.5 (release tag target 6c1abb9336c01cdb50d1d8f1ee4a54b460a59a39, containing #163) resumed the preserved ClawSweeper staged generation and reproduced the same duplicate CodeEmbedding primary-key failure before promotion:

• staged generation: 8c990268dd5a6ce8a30a17151031a235
• pending nodes: 4,454
• cached rows excluded from the pending checkpoint window: 3,040
• first regenerated batch: duplicate Function:dashboard/worker.ts:createFastAckComment:0
• canonical index and registry: untouched

The new exact-PK delete path was reached by the supported released artifact, but deleting and immediately recreating the row while the live VECTOR index is loaded is still not sufficient for this preserved LadybugDB generation. This is a real semantic-retrieval release failure, so the issue is reopened for one bounded fix and a real-DB regression proof before another release.

[100yenadmin]

1.6.10-electric.6 also reproduced the duplicate CodeEmbedding.id on the preserved generation before promotion. Dropping and rebuilding HNSW around the pending window was not sufficient for this already-interrupted LadybugDB artifact.

Delivery disposition for this sprint:

• preserve generation 8c990268dd5a6ce8a30a17151031a235 intact with checksums;
• do not open a third speculative source patch;
• recover ClawSweeper through a clean installed-.6 staged rebuild from the same exact source head;
• keep this issue open for an isolated real-artifact root-cause lane.

Canonical storage and the production registry were still untouched by both failed resume attempts. The generic resume claim therefore remains unproven even if the clean staged rebuild succeeds.

[100yenadmin]

Embedding-integrity release hold (2026-07-22): the suspected WorldOS anomaly is independently reproduced on an immutable, checksummed copy of generation 1bc59deb57fb206db030289ad5f48f536cca9d76 (lbug SHA-256 7cc5b28a811a5a88a78ea75ca7cadeef446bd314f3c7a70541a2ff85d91b5e5c).

Raw LadybugDB queries show 19,152 physical CodeEmbedding rows, 17,147 blank primary id values, 15,244 blank nodeId values, only 2,005 rows with complete identity, and 1,618 non-empty embedding references orphaned against the exact union of source-node IDs. All vectors are 2,048-dimensional. Metadata and registry both report the physical 19,152 count, so count agreement masked invalid identity. High-volume source tables in the same generation also contain blank primary IDs, narrowing this to a bulk/staged lifecycle path rather than a column-mapping error. Snapshot hashes were unchanged by the read-only audit and no WAL/lock/recovery sidecars existed.

electric/v1.6.10-electric.7 remains untagged, unpublished, and uninstalled; .6 stays active. PR #173 is held even if CI completes. Release/install and fleet mutation remain blocked until a reduced fixture reproduces the write path, the smallest identity-preservation fix lands, and immutable WorldOS plus ClawSweeper acceptance passes on the exact artifact.