Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: data stratification #361

Merged
merged 1 commit into from
Mar 9, 2024
Merged

Conversation

eatyourgreens
Copy link
Collaborator

  • Replace the tabbed interface for data uploads with a stepper.
  • Add utilities to group subjects by protocol.
  • Add stratification to the stepper.
  • Stratify by dose protocols for the time being.
  • Display each protocol group as a MUI data grid.
  • Display protocol groups as tabs in the trial design view.
  • Add the dosing compartment qname to the protocol model.
  • Add a 'cohort' field to the CSV.

Copy link

codecov bot commented Mar 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.45%. Comparing base (0ff05ef) to head (b12587b).

Additional details and impacted files
@@               Coverage Diff               @@
##           data-upload     #361      +/-   ##
===============================================
+ Coverage        76.41%   76.45%   +0.03%     
===============================================
  Files               96       97       +1     
  Lines             5390     5398       +8     
===============================================
+ Hits              4119     4127       +8     
  Misses            1271     1271              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@eatyourgreens eatyourgreens force-pushed the data-stratification branch 3 times, most recently from 872aef1 to cb6ef21 Compare March 8, 2024 11:57
@eatyourgreens eatyourgreens force-pushed the data-stratification branch 2 times, most recently from b022aea to 9645cdb Compare March 8, 2024 15:22
- Replace the tabbed interface for data uploads with a stepper.
- Add utilities to group subjects by protocol.
- Add stratification to the stepper.
- Stratify by dose protocols for the time being.
- Display each protocol group as a MUI data grid.
- Display protocol groups as tabs in the trial design view.
- Add the dosing compartment qname to the protocol model.
- Add a 'cohort' field to the CSV.
Copy link

sonarqubecloud bot commented Mar 8, 2024

Copy link

sonarqubecloud bot commented Mar 8, 2024

Quality Gate Failed Quality Gate failed for 'pkpdapp-team_pkpdapp_frontend'

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

@eatyourgreens eatyourgreens merged commit d351c69 into data-upload Mar 9, 2024
8 of 9 checks passed
@eatyourgreens eatyourgreens deleted the data-stratification branch March 9, 2024 09:46
eatyourgreens added a commit that referenced this pull request Mar 12, 2024
- Replace the tabbed interface for data uploads with a stepper.
- Add utilities to group subjects by protocol.
- Add stratification to the stepper.
- Stratify by dose protocols for the time being.
- Display each protocol group as a MUI data grid.
- Display protocol groups as tabs in the trial design view.
- Add the dosing compartment qname to the protocol model.
- Add a 'cohort' field to the CSV.
eatyourgreens added a commit that referenced this pull request Mar 19, 2024
- Replace the tabbed interface for data uploads with a stepper.
- Add utilities to group subjects by protocol.
- Add stratification to the stepper.
- Stratify by dose protocols for the time being.
- Display each protocol group as a MUI data grid.
- Display protocol groups as tabs in the trial design view.
- Add the dosing compartment qname to the protocol model.
- Add a 'cohort' field to the CSV.
eatyourgreens added a commit that referenced this pull request Mar 24, 2024
Upload a CSV dataset, set units (time, amount and observation), map CSV columns to model variables (dosing compartments and outputs), then run simulations and compare observation data with simulations in the Simulations tab.

* Map data to model variables

Show a table of dose amount data (and units), with select menus
to map these to model inputs.
Show a table of observation variables (and units), with select menus
to map those to model outputs.

* Add a mapped qname to biomarkers

Store the mapped variable qname on biomarker types. Read it from
OBSERVATION_VARIABLE in a dataset.

* Allow for multiple variable mappings

Map each Administration ID to a dosing compartment.
Map each Observation ID to a model output.
Add new columns to the CSV data, with mappings and optional units.

* Save modified dataset to backend

- load or create a dataset when we start an upload.
- save the dataset when we finish an upload.
- modify the `/datasets/:dataset_id:/csv` endpoint to accept a JSON string.

* Allow for a single unit column

- When there's a single unit column, use that column for both dosing and observations.
- Split the CSV data into dosing rows and observation rows.
- Add administration route to the Map Dosing screen.
- Allow for dimensionless observation units.
- Filter mapped observation variables for compatibility with the observation unit value.

* feat: data stratification (#361)

- Replace the tabbed interface for data uploads with a stepper.
- Add utilities to group subjects by protocol.
- Add stratification to the stepper.
- Stratify by dose protocols for the time being.
- Display each protocol group as a MUI data grid.
- Display protocol groups as tabs in the trial design view.
- Add the dosing compartment qname to the protocol model.
- Add a 'cohort' field to the CSV.

* refactor: optional simulation dosing protocols (#367)

- break `create_myokit_simulator` up into smaller methods.
- break `simulate` up into smaller methods.
- add optional `dosing_protocols` to `create_myokit_simulator`. If set, it
overrides the default dosing variable protocols.

* feat: subject groups

Add a subject group model to the Django app and the API. Subject groups
have a list of subjects and a list of protocols associated with those
subjects. Datasets have a list of subject groups in `dataset.groups`.

Refactor the Data and Trial Design views to use `dataset.groups` for
tabbed views.

* feat: allow for multiple Myokit simulations (#366)

Refactor the Myokit model mixin to allow multiple simulations to be run.
Each dataset subject group has its own simulation, with subject dosing protocols and dosing events.

Update the simulate API to return multiple simulations.

Show multiple plots in the Simulations tab

Test simulations with a project  and dataset

* feat: plot biomarker data in Simulation tab (#371)

- add biomarker data to the dataset serialiser.
- add biomarker data to the `useDataset` hook.
- display the dataset as data grids in the Data tab.
- show biomarkers as scatter plots on simulation plots.

* fix: handle missing observation units in the CSV (#372)

When the uploaded CSV doesn't have an observation unit column, allow
the user to pick a unit from a list of units, then pick a compatible
variable for the selected unit.

* refactor: handle missing units in CSV (#373)

Better handling of missing unit columns in the CSV.
- When time units are missing, prompt to choose a time unit.
- Change the column headings in the submitted CSV to match
headings that are expected by the Python data parser utility.
- Add a preclinical flag to the dosing step during uploads.

* fix: validate observation unit symbols

Makes sure that observation units are valid symbols. Symbols loaded
from a CSV sometimes have encoding issues.

If the symbol isn't valid, show the unit menu instead, so that we can
pick a valid unit and correct the uploading error.

* fix: set Group ID when selected cohort changes

Update the Group ID column when the selected cohort changes.
Use the group column values as the group ID for each row
eg. (1, 2, 3) or (Male, Female.)

* fix: 'map observations' with no selected unit

When there's no selected unit yet, the observation variables menu
should show all possible model variables.

* feat: handle additional doses from a data row (#374)

Map ADDL and II columns, in the CSV, to repeat doses in the model
and simulations.

* feat: filter displayed data by groups (#375)

Add checkboxes to control which subject groups are displayed in the
simulations view.

---------

Co-authored-by: martinjrobins <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant