Fix OpenSearch index staleness when encounters assigned to individuals#1547
Open
JasonWildMe wants to merge 3 commits into
Open
Fix OpenSearch index staleness when encounters assigned to individuals#1547JasonWildMe wants to merge 3 commits into
JasonWildMe wants to merge 3 commits into
Conversation
Find missed Encounter/individual reindexing opportunities
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1547 +/- ##
=======================================
Coverage 50.74% 50.74%
=======================================
Files 281 281
Lines 10468 10468
Branches 3238 3251 +13
=======================================
Hits 5312 5312
- Misses 4891 4897 +6
+ Partials 265 259 -6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
When a MarkedIndividual's encounter list changes at a mutation site that was not already queueing the individual for a deep reindex, sibling encounters retained stale denormalized fields in OpenSearch, including individualNumberEncounters, individualFirstEncounterDate, etc. Per Codex design review: queue the MarkedIndividual itself — not each encounter — because IndexingManager.addIndexingQueueEntry on an individual triggers opensearchIndexDeep, which cascades to every encounter with built-in per-id dedup. Adds a shared helper IndexingManager.queueIndividualsByIdForDeepReindex(Shepherd, Collection<String>) and calls it post-commit at the missed mutation sites: - AnnotationEdit — annotation-level swap/assign of individuals - EncounterForm — manualID path on new encounter submission - IBEISIA.assignFromIAAPI — IA-driven individual assignment - EncounterVMData — Visual Matcher "matchID" path - ImportIA — per-name individual creation/reuse - merge.jsp — individual merge followed by encounter reassignment - BulkImporter / api.BulkImport — both fg and bg paths via getTouchedIndividualIds() on the importer - StandardImport — end-of-import sweep of individualCache values - ImportSRGD — rows that create/attach individuals - ImportExcelMetadata — first and second flows (latter is dead but kept defensive) - ImportTask.deleteWithRelated — returns surviving individual ids to the BulkImport caller for post-commit queueing - DeleteImportTask — same, inline Deliberately not addressed here (covered by #1548): pre-existing pre-commit queueing in Encounter.processPatch, MarkedIndividual.mergeIndividual, and iaResultsSetID.jsp. Those race against the enclosing commit and survive rollback; moving them requires a cross-cutting refactor that would balloon this PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. IndexingManager.queueIndividualsByIdForDeepReindex was dereferencing the caller's Shepherd, which in servlets is closed by the finally block before the queue call runs (closeDBTransaction). The underlying PM is closed, so getMarkedIndividualQuiet returned null and every affected site silently queued nothing. Open a fresh short-lived read-only Shepherd for the id->object resolution instead. 2. StandardImport.loadIndividual only cached newly-created individuals in individualCache. Pre-existing individuals that had encounters added by the import were never queued for deep reindex — exactly the stale-sibling case #1514 is about. Also cache existing individuals when they are touched via addEncounter. 3. merge.jsp queued inside the finally block unconditionally, so the rollback path still reached the queue. Added a mergeSuccess flag so only the happy path triggers reindex queueing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reopens work from closed PR #1535 on a clean branch off
main, as a foundation for also fixing #1514.Summary
OpenSearch index was not being updated when encounters were assigned to / removed from individuals through various code paths. This PR addresses the direct
individualId/individualDisplayNamestaleness on affected encounters. Issue #1514 (staleindividualNumberEncounterson sibling encounters) is related but will be addressed in a follow-up commit on this branch.Root cause
skipAutoIndexingflag inEncounter.processPatch()prevented auto-indexingIndexingManager.addIndexingQueueEntry()calls with null checks in all affected code pathsFiles changed
Encounter.java: resetskipAutoIndexingand trigger reindex at end ofprocessPatch()IndividualAddEncounter.java: reindex encounter after adding to individualIndividualRemoveEncounter.java: reindex encounter AND old individual after removalIndividualCreateForProject.java: reindex encounter after creating new individualMarkedIndividual.java: reindex all transferred encounters and both individuals inmergeIndividual()iaResultsSetID.jsp: reindex encounter after ID confirmation in all 3 code pathsRelated
individualNumberEncounters) will be added as a follow-up commit on this branch.Test plan
individualID IS NULLfilter