Skip to content

Create functionality to extract LOINC Delta updates to generate additional embeddings#532

Open
BradySkylight wants to merge 9 commits intomainfrom
brady/452-get-loinc-delta
Open

Create functionality to extract LOINC Delta updates to generate additional embeddings#532
BradySkylight wants to merge 9 commits intomainfrom
brady/452-get-loinc-delta

Conversation

@BradySkylight
Copy link
Copy Markdown
Collaborator

@BradySkylight BradySkylight commented May 6, 2026

Description

This PR includes changes to make it possible to perform updates against, at least LOINC for now, various medical terminology value-sets utilized by TTC. Below is a list of the changes you will see:

  • Another refactor that moves the common functionality for All value-sets (general.py) as well as specific functionality for each of the different value-sets/ontologies into their own util python script.
  • The old valuset_sync script has been renamed to 'extraction' and leverages the utils functionality to perform a FULL extraction and saves the various files
  • The readme has been updated to indicate naming changes and location changes for various files/functionality
  • The embedding notebook was also changed to reflect the new order of all LOINC data pulled from LOINC extract so it will work correctly if ever used again.
    • NOTE: There are some formatting changes in this file for some reason and I'm not sure why - ignore these unless you notice an issue
  • Added a maintenance.py to handle maintaining the valueset extracts and embeddings for all valuesets - though only LOINC has been implemented so far for loinc lab names, as this is the only valueset loaded in our model at this point.

Related Issues

Closes #452

Additional Notes

I haven't added in tests, but have tested along the way and have some print statements included for now just to see the end result. The output will be passed to the next step - create the embeddings - which will then be stored as files somewhere and then uploaded into OpenSearch.

Checklist for Reviewers

Please review and complete the following checklist during the review process:

  • The code follows best practices and conventions.
  • The changes implement the desired functionality or fix the reported issue.
  • The tests cover the new changes and pass successfully.
  • Any potential edge cases or error scenarios have been considered.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.80%. Comparing base (3835c51) to head (b0240e0).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #532   +/-   ##
=======================================
  Coverage   95.80%   95.80%           
=======================================
  Files          46       46           
  Lines        2338     2338           
=======================================
  Hits         2240     2240           
  Misses         98       98           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@BradySkylight BradySkylight added this to the SNOINC milestone May 7, 2026
@BradySkylight BradySkylight self-assigned this May 7, 2026
@BradySkylight BradySkylight marked this pull request as ready for review May 7, 2026 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create Process to get delta update for valueset for LOINC

2 participants