feat(api): fix csv parsing logic and ensure feature/ai randomization

[hoangsonww]

PR #1 - fix csv parsing logic and ensure feature/ai randomization

This PR #1 does the following:

• Add a config.csv generation Shell script (under script/assign_cases/generate_config.sh. Running this script will generate the config.csv compatible with our new backend, which includes case feature randomization logic.
• Fix our backend API's CSV parsing and handling logic so that it now accepts the new .csv file structure and make sure that the randomization logic in the .csv file is respected. (To achieve the requested feature where only certain case features, like MEDICAL_HISTORY.Hypertension, are displayed. Each feature now has a ~50% chance of being selected for any particular case).
• Ensure that cases are correctly assigned to lab members and platform users.
• Enhance scalability of the app by refactoring/making app logic more generalizable.
• Add a load_omop.sh script that will automatically load OMOP files (not super important - but it'll help other developers get started with the platform and database real quick!)

The app now looks like:

[Copilot]

Pull Request Overview

This PR fixes the CSV parsing and display configuration logic for the API while introducing a new configuration generation script to support case assignment and randomization. Key changes include updates to type annotations and refactored CSV parsing in src/user/utils/csv_parser.py, the addition of load_omop.sh and generate_config.sh scripts for database initialization and config generation, and refactoring of the case service logic in src/cases/service/case_service.py for more robust tree handling.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/user/utils/csv_parser.py	Updates CSV parser with improved type hints and bucket-based processing.
src/load_omop.sh	Adds a new script for force-reloading OMOP vocabulary and clinical data.
src/cases/service/case_service.py	Refactors building and pruning of case detail tree nodes with enhanced docstrings.
script/assign_cases/generate_config.sh	Introduces a config generation script for case assignments with randomization.
page_config.json	Adds a JSON file for page configuration.
answer_config.json	Adds a JSON configuration for answer forms.

Comments suppressed due to low confidence (1)

page_config.json:2

• The key 'Background' in page_config.json is inconsistent with the expected uppercase 'BACKGROUND' used in the code. Consider renaming it to 'BACKGROUND' to maintain consistency.

"Background":           { "concept_ids": [] }

[Copilot]

[nitpick] The example command line arguments for <cases_per_user> and <ai_per_user> ('20 10') are ambiguous. Please clarify the separation between these two parameters in the documentation.

[Copilot]

Pull Request Overview

This PR updates CSV handling and case assignment logic, adds scripts for configuration and OMOP loading, and refactors case review to support randomized features and styles.

• Enhanced CsvConfigurationParser to bucket rows by (user, case) and build style dictionaries.
• Refactored case_service to prune case details per display config and include AI CRC risk entries.
• Added generate_config.sh for CSV config generation and load_omop.sh for OMOP data loading.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/user/utils/csv_parser.py	Rewrote parser to group rows, validate headers, and apply styles
src/cases/service/case_service.py	Refactored case review workflow to prune tree and merge styles
script/assign_cases/generate_config.sh	New script to generate randomized CSV of user-case-feature rows
src/load_omop.sh	New script to load OMOP vocabulary and clinical data into PostgreSQL
page_config.json	Initial JSON stub for page configuration
answer_config.json	New JSON schema for answer form configuration

Comments suppressed due to low confidence (4)

src/cases/service/case_service.py:350

• The code calls itemgetter but it is not imported. Please add from operator import itemgetter at the top.

important_infos.sort(key=itemgetter("weight"))

src/cases/service/case_service.py:269

• The code uses defaultdict but there is no import for it. Please add from collections import defaultdict at the top.

parent_to_entries: dict[str, list[dict]] = defaultdict(list)

page_config.json:2

• JSON keys like "Background" do not match the uppercase section names (e.g., "BACKGROUND") expected by the code. Update the keys to match exactly.

"Background":           { "concept_ids": [] },

script/assign_cases/generate_config.sh:57

• The script assumes exactly eight positional arguments without checking $#. Add a check for the required argument count and print usage on mismatch.

DB_HOST="$1"