Skip to content

Add docs for manual stats calculations outside of plots - #215

Merged
jthorton merged 7 commits into
mainfrom
stats_calcs
Jun 3, 2026
Merged

Add docs for manual stats calculations outside of plots#215
jthorton merged 7 commits into
mainfrom
stats_calcs

Conversation

@jthorton

@jthorton jthorton commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #188 with new docs on calculating the metrics reported in scatter plots manually using the helper dataframe functions.

Drafted by me and refined by Claude

Todos

Notable points that this PR has either accomplished or will accomplish.

  • TODO 1

Questions

  • Question1

Checklist

  • Added a news entry for new features, bug fixes, or other user facing changes.

Status

  • Ready to go

Tips

  • Comment "pre-commit.ci autofix" to have pre-commit.ci atomically format your PR.
    Since this will create a commit, it is best to make this comment when you are finished with your work.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@codecov

codecov Bot commented Jun 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.53%. Comparing base (1504318) to head (cea3bbd).
⚠️ Report is 96 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #215      +/-   ##
==========================================
+ Coverage   97.40%   97.53%   +0.13%     
==========================================
  Files          22       22              
  Lines        2308     2470     +162     
==========================================
+ Hits         2248     2409     +161     
- Misses         60       61       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jthorton

jthorton commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

pre-commit.ci autofix

@jthorton
jthorton requested review from IAlibay and hannahbaumann June 2, 2026 12:23
@@ -0,0 +1,826 @@
{

@hannahbaumann hannahbaumann Jun 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cinnabar.stats module exposes are wide range of deviation and correlation metrics, see the API documentation for a complete list.

Below you have a list of the acceptable strings, maybe having them here together with the meaning of the acronym could be helpful?


Reply via ReviewNB

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to link to the API docs too.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Titles sizes seem off - I think "loading RBFE Results" should be the same size, if not bigger than "Available Metrics"?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks both, added a table linking the short-hand name to the function and a description and added a link to the API docs but had to change to myst-nb to get it to work.

@@ -0,0 +1,826 @@
{

@hannahbaumann hannahbaumann Jun 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_relative_dataframe() returns a DataFrame of every directly simulated edge together with the corresponding experimental ΔΔG derived from the absolute experimental values, the dataframe is also grouped and sorted by the source column to make computing metrics for a particular method straightforward.

Maybe break into two sentences.


Reply via ReviewNB

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,826 @@
{

@hannahbaumann hannahbaumann Jun 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line #17.    print("Absolute ΔG metrics:")


Reply via ReviewNB

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks good old copy-paste mistakes!

@hannahbaumann hannahbaumann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jthorton lgtm, just left some small comments. Some of the senteces in the docs are quite long, maybe some of the comma separations could also be separate sentences?

@@ -0,0 +1,826 @@
{

@IAlibay IAlibay Jun 3, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot of context here that I would add to the text for this section. I.e. expand "we will use some RBFE data generated with OpenFE "


Reply via ReviewNB

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, though I want to avoid explaining this in every tutorial. How about linking to the FEMap API tutorial?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd want to at least say "This data contains a mixture of experimental values, and calculated values generated via two different methods". Relying on linking out to another page for that much information can often serve as more of a distraction than anything else.

Comment thread examples/calculating-metrics.ipynb
Comment thread examples/calculating-metrics.ipynb
Comment thread examples/calculating-metrics.ipynb
Comment thread examples/calculating-metrics.ipynb
@jthorton

jthorton commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

pre-commit.ci autofix

@@ -0,0 +1,1659 @@
{

@hannahbaumann hannahbaumann Jun 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the metrix table there's a small spelling mistake "differnece"


Reply via ReviewNB

@hannahbaumann hannahbaumann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, lgtm, just the small spelling mistake!

@IAlibay IAlibay left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall lgtm, but it would be good to do the one thing re: what the OpenFE data contains. Nothing too verbose just "experimental data + 2 sets of calculated data generated via two different methods".

@jthorton
jthorton merged commit 2cf0f6f into main Jun 3, 2026
9 checks passed
@jthorton
jthorton deleted the stats_calcs branch June 3, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add manual stats calculation tutorial

3 participants