Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Symbolic Rate law to Pytorch Rate law #201

Merged
merged 23 commits into from
Jul 10, 2023

Conversation

djinnome
Copy link
Contributor

@djinnome djinnome commented Jul 4, 2023

…ook. Will test next.

@djinnome djinnome linked an issue Jul 4, 2023 that may be closed by this pull request
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@djinnome djinnome self-assigned this Jul 4, 2023
@djinnome djinnome added the WIP PR submitter still making changes, not ready for review label Jul 4, 2023
@djinnome djinnome added awaiting review PR submitter awaiting code review from reviewer and removed WIP PR submitter still making changes, not ready for review labels Jul 6, 2023
@djinnome djinnome requested a review from SamWitty July 6, 2023 01:31
@SamWitty
Copy link
Contributor

SamWitty commented Jul 6, 2023

@djinnome , this is tagged as awaiting review but there are some failing tests. Is this actually still a WIP?

@djinnome djinnome added blocked and removed awaiting review PR submitter awaiting code review from reviewer labels Jul 6, 2023
@djinnome
Copy link
Contributor Author

djinnome commented Jul 6, 2023

Hi @SamWitty tests now pass, but I am still blocked on gyorilab/mira#189
If the AMR id is different from the name, then there is no easy way to associate the rate law symbols with the appropriate concept.

@SamWitty
Copy link
Contributor

SamWitty commented Jul 6, 2023

Thanks for the update @djinnome . Could you revise the name of this PR to something more informative?

@SamWitty SamWitty added the WIP PR submitter still making changes, not ready for review label Jul 6, 2023
@djinnome djinnome changed the title I believe I wrote the correct code, based on experiments in the noteb… Rate law Jul 6, 2023
@djinnome djinnome changed the title Rate law Symbolic Rate law to Pytorch Rate law Jul 6, 2023
@djinnome djinnome added awaiting review PR submitter awaiting code review from reviewer and removed WIP PR submitter still making changes, not ready for review blocked labels Jul 6, 2023
…h the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue
…ESystem from main. ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=False
…ESystem from main. ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True
…rch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives
Copy link
Contributor

@SamWitty SamWitty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good overall. Could you rerun the integration demos with these changes? This should produce output files without any mira_param parameters, and with the solution variable names according to the id, not the long name. Please check this and then I'll approve.

src/pyciemss/PetriNetODE/base.py Outdated Show resolved Hide resolved
src/pyciemss/PetriNetODE/base.py Outdated Show resolved Hide resolved
@SamWitty SamWitty added awaiting response PR reviewer awaiting response from submitter and removed awaiting review PR submitter awaiting code review from reviewer labels Jul 7, 2023
…ultiple parameters per transition can be interpreted using compile_rate_law
@djinnome
Copy link
Contributor Author

djinnome commented Jul 7, 2023

I have rerun the integration_demo notebook and it now generates output without aggregate_parameters by default.

@djinnome djinnome added awaiting review PR submitter awaiting code review from reviewer and removed awaiting response PR reviewer awaiting response from submitter labels Jul 7, 2023
Copy link
Contributor

@SamWitty SamWitty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@SamWitty SamWitty merged commit 12a004a into main Jul 10, 2023
3 checks passed
@SamWitty SamWitty deleted the 196-extract-custom-rate-laws-from-the-amr branch July 10, 2023 09:21
anirban-chaudhuri added a commit that referenced this pull request Jul 10, 2023
* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

* Symbolic Rate law to Pytorch Rate law (#201)

* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
anirban-chaudhuri added a commit that referenced this pull request Jul 10, 2023
* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

* Symbolic Rate law to Pytorch Rate law (#201)

* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law

* 12-Month Hackathon Notebooks (#207)

* started hackathon prep scenario notebooks

* more on hackathon scenarios

* more work on hackathon notebooks

* created AMR for scenario1 with constant beta

* Vs hackathon prep (#203)

* beginning of ensemble challenge ipynb

* Update ensemble_challenge.ipynb

* added to scenario2 notebook, ready for scenario3

* added to scenario2 notebook, ready for scenario3

* Update scenario1.ipynb

* Update scenario1

* changes to scenario3, and new AMR

* pre-hackathon prep update

- scenario1 task 1 almost done (pending AMR changes)

- ensemble challenge layout started

* more on scenario notebooks, added AMR to scenario2

* Fixed the nan inside the intervened parameters column output

* updates to scenario3 notebook

* merged from main

---------

Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
anirban-chaudhuri added a commit that referenced this pull request Jul 12, 2023
* Optimize TA4 interface

* Update interfaces.py

* Change setup to fix MIRA name error (#205) (#206)

* changed mira version before bug

* fix error

Co-authored-by: Sam Witty <[email protected]>

* Update demo notebook

* Update demo.ipynb

* Update MIRA loading (#209)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

---------

Co-authored-by: Sam Witty <[email protected]>

* Updated interface with string inputs for QOI

* Interface to optimize after calibrating

* Update demo.ipynb

* Update sample csv file to avoid merge conflicts

* Updating file to avoid merge conflict with main

* Updating from main to fix model loading errors (#211)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

* Symbolic Rate law to Pytorch Rate law (#201)

* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

* Update notebooks to avoid merge conflicts

* Update NB to resolve conflicts

* Update from main to avoid conflicts (#212)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

* Symbolic Rate law to Pytorch Rate law (#201)

* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law

* 12-Month Hackathon Notebooks (#207)

* started hackathon prep scenario notebooks

* more on hackathon scenarios

* more work on hackathon notebooks

* created AMR for scenario1 with constant beta

* Vs hackathon prep (#203)

* beginning of ensemble challenge ipynb

* Update ensemble_challenge.ipynb

* added to scenario2 notebook, ready for scenario3

* added to scenario2 notebook, ready for scenario3

* Update scenario1.ipynb

* Update scenario1

* changes to scenario3, and new AMR

* pre-hackathon prep update

- scenario1 task 1 almost done (pending AMR changes)

- ensemble challenge layout started

* more on scenario notebooks, added AMR to scenario2

* Fixed the nan inside the intervened parameters column output

* updates to scenario3 notebook

* merged from main

---------

Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

* Update demo.ipynb

* Update demo.ipynb

* updated demo with new MIRA model

* Fixed setup.cfg

* Update demo.ipynb

* Updated solution key in demo

* finished demo notebook run

* Added test for samples obtained after OUU

* Update output format and description

* Fixed mira on setup to help pass tests

* Added `self.intervened_samples` to `test_ode_interfaces'

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
anirban-chaudhuri added a commit that referenced this pull request Jul 13, 2023
* Added get_name on the mira parameter keys for safety (#218)

* Added get_name on the mira parameter keys for safety

* Removed test for SIDARTHE which fails due to gyorilab/mira#195

* checked out scenario1 from main so that it doesn't clobber Vignesh's changes

* Added test that confirms gyorilab/mira#196 solves gyorilab/mira#195

* Added pin to latest mira commit

* bump mira (#227)

* bump mira

* bumped AMR file

* Provisional support for time_unit in TA4 interface dataframe (#221)

* Provisional support for time_unit in TA4 interface dataframe

* Missed format identifier

* Default time-unit

* Testing time_unit behaviors

* Updating tests for new TA4 itnerface behavior

* To preserve context in failure messages, changed  to

* Decorator logging function (#232)

* Logging function name where error occurs

* updated text

* Optimize interface for TA4 integration (#204)

* Optimize TA4 interface

* Update interfaces.py

* Change setup to fix MIRA name error (#205) (#206)

* changed mira version before bug

* fix error

Co-authored-by: Sam Witty <[email protected]>

* Update demo notebook

* Update demo.ipynb

* Update MIRA loading (#209)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

---------

Co-authored-by: Sam Witty <[email protected]>

* Updated interface with string inputs for QOI

* Interface to optimize after calibrating

* Update demo.ipynb

* Update sample csv file to avoid merge conflicts

* Updating file to avoid merge conflict with main

* Updating from main to fix model loading errors (#211)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

* Symbolic Rate law to Pytorch Rate law (#201)

* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

* Update notebooks to avoid merge conflicts

* Update NB to resolve conflicts

* Update from main to avoid conflicts (#212)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

* Symbolic Rate law to Pytorch Rate law (#201)

* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law

* 12-Month Hackathon Notebooks (#207)

* started hackathon prep scenario notebooks

* more on hackathon scenarios

* more work on hackathon notebooks

* created AMR for scenario1 with constant beta

* Vs hackathon prep (#203)

* beginning of ensemble challenge ipynb

* Update ensemble_challenge.ipynb

* added to scenario2 notebook, ready for scenario3

* added to scenario2 notebook, ready for scenario3

* Update scenario1.ipynb

* Update scenario1

* changes to scenario3, and new AMR

* pre-hackathon prep update

- scenario1 task 1 almost done (pending AMR changes)

- ensemble challenge layout started

* more on scenario notebooks, added AMR to scenario2

* Fixed the nan inside the intervened parameters column output

* updates to scenario3 notebook

* merged from main

---------

Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

* Update demo.ipynb

* Update demo.ipynb

* updated demo with new MIRA model

* Fixed setup.cfg

* Update demo.ipynb

* Updated solution key in demo

* finished demo notebook run

* Added test for samples obtained after OUU

* Update output format and description

* Fixed mira on setup to help pass tests

* Added `self.intervened_samples` to `test_ode_interfaces'

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

* 234 visualization support for trajectory distributions (#235)

* Trajectory distribution schema

* Sample trajectory data and notebook

* Interactive legend

* Removing extraneous encoding

* replace-by-name utility added

Removes index-based replacements (a source of brittleness in the substitutions being done.)

* Types improvements and linting fixes

* Relabeling

* Example trajectories working

* calibration viz notebook

* clean up

* heatmap vega

* triangle plot in triangle_vega_demo

* Points and tracks on trajectory visualization.

* Ignoring typechecking temporaries

* Adjusting z-order

* EXPERIEMENT: dataframe based input to trajectory visualizations

* REVERTING to prep for a TA4-defined dataframe.

* Ignoring temporaries that start with an underscore.

* Joining line for provided datapoints.

* Color control.

* TA4 dataframe: Distributions and points working

* Distributions, traces and points all work from TA4 interface dataframe.

* Improved Color handling

* triangle notebook

* vega with n-dimensional heatmap

* vega with n-dimensional heatmap

* Basic timepoint markers added

* Permanantly labeled timepoint markers.

* Support to NOT join points

This is useful when the sample points are relatively dense.

* More flexible timepoint for traces.

* LotkaVolterra examples

NOTE: Its possible the `convert_to_output_format` has a bug.  It is returning step-ids, not time-values.  As-such, intervention markers were set to 250 instead of 25.

* Trajectory and barycenter/proababilty-density plots
Extended integration interface to preserve timepoint (not just timepoint_id)

Renamed 'vega' to 'plots'
Updated examples
(Some) Utility tests
Cleanup of various files

* Basic support for calibrate chart

* Calibration supports name-discovery from the dataset.

* Provisional support for time_unit in TA4 interface dataframe

* Missed format identifier

* Updated TA4 dataframe interface.

* Using 'Optional' for types instead of Union with None

* Visualzation of non-ensembles supported.

* Demo of visulaizations through the interface.

* Default time-unit

* Testing time_unit behaviors

* Updating tests for new TA4 itnerface behavior

* To preserve context in failure messages, changed  to

* Tests for visualizations; Visuals avaialble through the TA4 interface.

---------

Co-authored-by: Oostrom, Marjolein T <[email protected]>

* add json files to package (#240)

* Updated setup.cfg to work with pyciemss install (#242)

* add json files to package

* For some reason the last pr worked locally but not with the simulation service install. hoping this solves it

---------

Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Joseph Cottam <[email protected]>
Co-authored-by: marshHawk4 <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: Oostrom, Marjolein T <[email protected]>
SamWitty added a commit that referenced this pull request Jul 13, 2023
* Ensemble quantiles

- Updated `convert_to_output_format` to return quantiles for ensemble models

* Updated utilities to produce quantiles as extra output for ensembles

* Update sample_results.csv

resolving conflicts with main branch

* Updating from main (#230)

* Added get_name on the mira parameter keys for safety (#218)

* Added get_name on the mira parameter keys for safety

* Removed test for SIDARTHE which fails due to gyorilab/mira#195

* checked out scenario1 from main so that it doesn't clobber Vignesh's changes

* Added test that confirms gyorilab/mira#196 solves gyorilab/mira#195

* Added pin to latest mira commit

* bump mira (#227)

* bump mira

* bumped AMR file

---------

Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: Sam Witty <[email protected]>

* Update interface_utils.py

* Data formatting update

* Added options for data formatting

* Finalized notebook

* resolving conflicts with main for notebook

* resolving conflicts in csv files

* resolving conflicts with main

* resolving conflict on rate law test

* resolving conflicts on interfaces

* Update from main branch (#249)

* Added get_name on the mira parameter keys for safety (#218)

* Added get_name on the mira parameter keys for safety

* Removed test for SIDARTHE which fails due to gyorilab/mira#195

* checked out scenario1 from main so that it doesn't clobber Vignesh's changes

* Added test that confirms gyorilab/mira#196 solves gyorilab/mira#195

* Added pin to latest mira commit

* bump mira (#227)

* bump mira

* bumped AMR file

* Provisional support for time_unit in TA4 interface dataframe (#221)

* Provisional support for time_unit in TA4 interface dataframe

* Missed format identifier

* Default time-unit

* Testing time_unit behaviors

* Updating tests for new TA4 itnerface behavior

* To preserve context in failure messages, changed  to

* Decorator logging function (#232)

* Logging function name where error occurs

* updated text

* Optimize interface for TA4 integration (#204)

* Optimize TA4 interface

* Update interfaces.py

* Change setup to fix MIRA name error (#205) (#206)

* changed mira version before bug

* fix error

Co-authored-by: Sam Witty <[email protected]>

* Update demo notebook

* Update demo.ipynb

* Update MIRA loading (#209)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

---------

Co-authored-by: Sam Witty <[email protected]>

* Updated interface with string inputs for QOI

* Interface to optimize after calibrating

* Update demo.ipynb

* Update sample csv file to avoid merge conflicts

* Updating file to avoid merge conflict with main

* Updating from main to fix model loading errors (#211)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

* Symbolic Rate law to Pytorch Rate law (#201)

* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

* Update notebooks to avoid merge conflicts

* Update NB to resolve conflicts

* Update from main to avoid conflicts (#212)

* Change setup to fix MIRA name error (#205)

* changed mira version before bug

* fix error

* Add utilities for loading distributions from AMR (#200)

* added mira distribution loading

* added normal distribution

* fixed Normal2 and Normal3

* nit

* added minimal mira_distribution_to_pyro test

* Symbolic Rate law to Pytorch Rate law (#201)

* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law

* 12-Month Hackathon Notebooks (#207)

* started hackathon prep scenario notebooks

* more on hackathon scenarios

* more work on hackathon notebooks

* created AMR for scenario1 with constant beta

* Vs hackathon prep (#203)

* beginning of ensemble challenge ipynb

* Update ensemble_challenge.ipynb

* added to scenario2 notebook, ready for scenario3

* added to scenario2 notebook, ready for scenario3

* Update scenario1.ipynb

* Update scenario1

* changes to scenario3, and new AMR

* pre-hackathon prep update

- scenario1 task 1 almost done (pending AMR changes)

- ensemble challenge layout started

* more on scenario notebooks, added AMR to scenario2

* Fixed the nan inside the intervened parameters column output

* updates to scenario3 notebook

* merged from main

---------

Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

* Update demo.ipynb

* Update demo.ipynb

* updated demo with new MIRA model

* Fixed setup.cfg

* Update demo.ipynb

* Updated solution key in demo

* finished demo notebook run

* Added test for samples obtained after OUU

* Update output format and description

* Fixed mira on setup to help pass tests

* Added `self.intervened_samples` to `test_ode_interfaces'

---------

Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>

* 234 visualization support for trajectory distributions (#235)

* Trajectory distribution schema

* Sample trajectory data and notebook

* Interactive legend

* Removing extraneous encoding

* replace-by-name utility added

Removes index-based replacements (a source of brittleness in the substitutions being done.)

* Types improvements and linting fixes

* Relabeling

* Example trajectories working

* calibration viz notebook

* clean up

* heatmap vega

* triangle plot in triangle_vega_demo

* Points and tracks on trajectory visualization.

* Ignoring typechecking temporaries

* Adjusting z-order

* EXPERIEMENT: dataframe based input to trajectory visualizations

* REVERTING to prep for a TA4-defined dataframe.

* Ignoring temporaries that start with an underscore.

* Joining line for provided datapoints.

* Color control.

* TA4 dataframe: Distributions and points working

* Distributions, traces and points all work from TA4 interface dataframe.

* Improved Color handling

* triangle notebook

* vega with n-dimensional heatmap

* vega with n-dimensional heatmap

* Basic timepoint markers added

* Permanantly labeled timepoint markers.

* Support to NOT join points

This is useful when the sample points are relatively dense.

* More flexible timepoint for traces.

* LotkaVolterra examples

NOTE: Its possible the `convert_to_output_format` has a bug.  It is returning step-ids, not time-values.  As-such, intervention markers were set to 250 instead of 25.

* Trajectory and barycenter/proababilty-density plots
Extended integration interface to preserve timepoint (not just timepoint_id)

Renamed 'vega' to 'plots'
Updated examples
(Some) Utility tests
Cleanup of various files

* Basic support for calibrate chart

* Calibration supports name-discovery from the dataset.

* Provisional support for time_unit in TA4 interface dataframe

* Missed format identifier

* Updated TA4 dataframe interface.

* Using 'Optional' for types instead of Union with None

* Visualzation of non-ensembles supported.

* Demo of visulaizations through the interface.

* Default time-unit

* Testing time_unit behaviors

* Updating tests for new TA4 itnerface behavior

* To preserve context in failure messages, changed  to

* Tests for visualizations; Visuals avaialble through the TA4 interface.

---------

Co-authored-by: Oostrom, Marjolein T <[email protected]>

* add json files to package (#240)

* Updated setup.cfg to work with pyciemss install (#242)

* add json files to package

* For some reason the last pr worked locally but not with the simulation service install. hoping this solves it

---------

Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Joseph Cottam <[email protected]>
Co-authored-by: marshHawk4 <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: Oostrom, Marjolein T <[email protected]>

* Added back changes for quantile estimations

* Added time units

* Update test_ensemble_interfaces.py

---------

Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: Sam Witty <[email protected]>
Co-authored-by: Joseph Cottam <[email protected]>
Co-authored-by: marshHawk4 <[email protected]>
Co-authored-by: sabinala <[email protected]>
Co-authored-by: vsella <[email protected]>
Co-authored-by: Jeremy Zucker <[email protected]>
Co-authored-by: Oostrom, Marjolein T <[email protected]>
marjoleinpnnl pushed a commit that referenced this pull request Aug 10, 2023
* I believe I wrote the correct code, based on experiments in the notebook. Will test next.

* FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta'

* Added Symbolic_Deriv_Experiments notebook

* Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time

* Clarified the bug in the Symbolic derivatives notebook

* Expected and actual derivative match

* Time varying parameter rate law correctly read

* Thought we added this already

* Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net

* Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default

* Removed unnecessary pygraphviz dependency

* Unit test to fail when concept name does not equal rate law symbols

* All tests pass with default compile_rate_law_p = False

* Merged from main. removed dependency on older version of mira

* point mira to the github repo main branch

* point mira to the github repo main branch

* load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue

* merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main.  ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True

* observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives

* aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting review PR submitter awaiting code review from reviewer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extract custom rate laws from the AMR
2 participants