Skip to content

fix: defer bulk-import re-ID until all detections terminal#1552

Open
JasonWildMe wants to merge 6 commits intomainfrom
fix/bulk-import-reid-race
Open

fix: defer bulk-import re-ID until all detections terminal#1552
JasonWildMe wants to merge 6 commits intomainfrom
fix/bulk-import-reid-race

Conversation

@JasonWildMe
Copy link
Copy Markdown
Collaborator

Summary

  • Bug: Bulk imports fire per-encounter identification immediately after each detection callback, before sibling encounters finish detection. Intra-bulk matches are missed on first pass because WBIA's matching index doesn't include annotations from unfinished siblings.
  • Fix: Branch processCallback for bulk imports: defer identification until all ImportTask MediaAssets have terminal detection status (complete|pending|error), then fire one bulk-final ID pass over ALL eligible annotations, claimed atomically via JVM-global registry + persistent Task flag for restart survival.
  • Non-bulk path unchanged — refactored into helpers but identical behavior.

Architecture

Detection callback arrives at processCallback()
   │
   ├── Non-bulk? → existing per-callback ID kickoff (unchanged)
   │
   └── Bulk import (importTaskId in task params)?
         │
         ├── Not all detections terminal? → return (defer)
         │
         ├── Already fired (persistent flag)? → return
         │
         ├── Another thread won claim? → return
         │
         └── Claim won → mark persistent flag
              → enumerate ALL ImportTask annotations
              → filter: matchAgainst && validForIdentification
              → group by encounter, kick off ID per encounter

Files changed

File Change
BulkFinalIdentificationRegistry.java NEW — JVM-global ConcurrentHashMap single-winner claim
Task.java ADD persistent bulkFinalIdentificationFired Boolean
package.jdo ADD JDO mapping for new Task field
ImportTask.java ADD isAllDetectionTerminal() helper
IBEISIA.java REWRITE identification block + 5 extracted helpers

Test plan

  • 24 new unit tests (registry atomicity incl. 50-thread race, bulk task detection, encounter grouping, terminal status)
  • Full mvn test passes (11 pre-existing errors in Settings/ShepherdState tests unrelated to this change)
  • mvn compile clean
  • Independent code review by Codex: "Code is sound, ready for PR"
  • Manual verification on test install with ≥3-encounter bulk import — log line to watch: INFO: fireBulkFinalIdentificationIfReady: ImportTask X -> ID on N annotations across M encounters

🤖 Generated with Claude Code

Bulk imports fire per-encounter identification immediately after each
detection callback, before sibling encounters finish detection. This
causes intra-bulk matches to be missed on the first pass because WBIA's
matching index doesn't yet include annotations from unfinished siblings.

Branch processCallback: when a detection callback belongs to a bulk
import (importTaskId in task parameters), defer identification. On each
callback, check if ALL ImportTask MediaAssets have terminal detection
status. Only the last terminal callback—claimed atomically via a
JVM-global registry keyed on importTaskId plus a persistent flag on
the root IA Task for restart survival—fires a single bulk-final ID
pass over all eligible annotations (matchAgainst && validForIdentification),
grouped by encounter with location filters.

Non-bulk detection-then-identify path is unchanged (refactored into
helpers but identical behavior).

24 new unit tests covering registry atomicity, bulk task detection,
encounter grouping, and terminal status checking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 51.61%. Comparing base (40ea242) to head (5d9f55b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1552   +/-   ##
=======================================
  Coverage   51.61%   51.61%           
=======================================
  Files         308      308           
  Lines       11976    11976           
  Branches     3856     3847    -9     
=======================================
  Hits         6182     6182           
- Misses       5507     5514    +7     
+ Partials      287      280    -7     
Flag Coverage Δ
backend 51.61% <ø> (ø)
frontend 51.61% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JasonWildMe JasonWildMe self-assigned this Apr 20, 2026
@JasonWildMe JasonWildMe requested review from naknomum April 20, 2026 21:41
@goddesswarship
Copy link
Copy Markdown
Contributor

@naknomum For your approval when you return!

@naknomum naknomum added this to the 10.10.5 milestone May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants