-
Notifications
You must be signed in to change notification settings - Fork 16
Core concepts #162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Core concepts #162
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
0e49e61
first pass at core concepts theory pages
jthorton 4f65968
Add plotting core concepts
jthorton 9e11b0f
Update docs/concepts/estimators.rst
jthorton 8773c07
Update docs/concepts/femap.rst
jthorton f152b93
Update docs/concepts/plotting.rst
jthorton a329e75
Update docs/concepts/plotting.rst
jthorton 4670ec0
Update docs/concepts/plotting.rst
jthorton 7d7d8ca
docs feedback
jthorton ec82569
add API links for plotting and feedback
jthorton File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| .. _estimators: | ||
|
|
||
| =============================== | ||
| Absolute Free Energy Estimators | ||
| =============================== | ||
|
|
||
| Relative free energy calculations produce :math:`\Delta\Delta G` (differences between two ligands). To compare these with experiment, | ||
| or to rank ligands by affinity, we need absolute free energies (:math:`\Delta G`) for each ligand. | ||
|
|
||
| This requires an **estimator**: a method that takes the network of relative free energies and produces absolute values. | ||
|
|
||
|
|
||
| Maximum Likelihood Estimation (MLE) | ||
| ----------------------------------- | ||
|
|
||
| The Maximum Likelihood Estimation (MLE) [1]_ method is the **default** estimator used in cinnabar to obtain absolute | ||
| free energies (:math:`\Delta G`) from a network of relative free energies (:math:`\Delta\Delta G`). | ||
|
|
||
|
|
||
| The Core Idea | ||
| ~~~~~~~~~~~~~~ | ||
| To place every ligand on a common absolute scale, we need to find a set of :math:`\Delta G` values that best explain all | ||
| relative differences (:math:`\Delta\Delta G`) simultaneously. The MLE method does this by asking: | ||
|
|
||
| What set of :math:`\Delta G` values makes the observed data most likely, given the reported uncertainties? | ||
|
|
||
| This framing naturally integrates all edges and cycles in the graph simultaneously. | ||
|
|
||
| The Likelihood Function | ||
| ~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Suppose we have a network with two ligands ``i`` and ``j``, with observed relative free energy :math:`\Delta\Delta G_{ij}` and | ||
| uncertainty :math:`\sigma_{ij}`. The model assumes each measurement is normally distributed: | ||
|
|
||
| .. math:: | ||
|
|
||
| \Delta\Delta G_{ij} \approx \mathcal{N}(\Delta G_j - \Delta G_i, \sigma_{ij}^2) | ||
|
|
||
| The **likelihood** is the product of probabilities for all edges in the graph. The MLE procedure finds the set of :math:`\Delta G` | ||
| values that maximises this likelihood (or equivalently, minimises the negative log-likelihood). | ||
|
|
||
| Uncertainty Propagation | ||
| ~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Input uncertainties (:math:`\sigma_{ij}`) are explicitly included in the likelihood function. This means more precise edges | ||
| (smaller uncertainty) have greater weight in determining the solution. However, this does mean that high confidence but low | ||
| accuracy edges can impact the entire network and so robust uncertainty estimates on input data are crucial. | ||
|
|
||
|
|
||
| Centering of Results | ||
| ~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| The absolute :math:`\Delta G` scale is arbitrary: adding a constant to all :math:`\Delta G` values does not change any relative differences :math:`\Delta\Delta G`. | ||
| As a result, the MLE solution is typically centred around zero (or another chosen reference). To compare with experimental | ||
|
IAlibay marked this conversation as resolved.
|
||
| values, an experimental shift must be applied. By default ``cinnabar`` will align the mean of predicted and | ||
| experimental :math:`\Delta G` in the plotting functions. | ||
|
|
||
|
|
||
| Limitations | ||
| ~~~~~~~~~~~ | ||
|
|
||
| - The MLE method **can not** use multiple independent measurements of the same edge to improve precision automatically. | ||
| Each edge must be represented by a single :math:`\Delta\Delta G` and uncertainty. If multiple measurements are available, | ||
| they should be combined externally (e.g. via weighted averaging) before input to the estimator. | ||
|
|
||
|
|
||
|
|
||
| References | ||
| ~~~~~~~~~~~ | ||
|
|
||
| .. [1] Xu, H., 2019. Optimal measurement network of pairwise differences. Journal of Chemical Information and Modeling, 59(11), pp.4720-4728. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| ============================ | ||
| The ``FEMap`` Data Structure | ||
| ============================ | ||
|
|
||
| The :class:`.FEMap` is the **core abstraction in cinnabar**. | ||
| It provides a unified representation of free energy data, connecting the inputs from relative binding free energy | ||
| calculations with the **analyses** and **visualizations** that cinnabar enables. | ||
|
|
||
| Why a Graph? | ||
| ------------ | ||
|
|
||
| Relative free energy calculations are inherently **relational**: they compare two ligands at a time, producing a free | ||
| energy difference (:math:`\Delta\Delta G`) with an associated uncertainty. If we want to reason about a whole series of ligands, which | ||
| may be connected by multiple pairwise comparisons, we need to connect these edges in a logical way. | ||
|
|
||
| A **graph** is a natural way to represent this data, where: | ||
|
|
||
| - **Nodes** represent individual ligands. | ||
| - **Absolute values** (:math:`\Delta G` experimental or calculated) can be atached to nodes as attributes. | ||
| - **Edges** represent pairwise free energy differences (:math:`\Delta\Delta G`) between ligands. | ||
|
|
||
| This graph representation is powerful: it allows integration of relative and absolute data, and it provides the | ||
| foundation for robust statistical analysis and visualization. | ||
|
|
||
|
|
||
| From Relative to Absolute | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Although we calculate relative free energies directly, many applications require absolute binding free energies for | ||
| each ligand in order to rank the ligands for prioritization. | ||
|
|
||
| The :class:`.FEMap` provides convenience methods to support this transformation via a :ref:`maximum likelihood estimation (MLE) <estimators>` [1]_ | ||
| method by default. These methods take advantage of the entire network of relative free energies to infer absolute values. | ||
| Thus they require a graph which is at least weakly connected, with at least one path between any two ligands in the graph. | ||
| The ``FEMap`` provides utilities to check (:meth:`.FEMap.check_weakly_connected`) and visualise (:meth:`.FEMap.draw_graph`) the connectivity of the network. | ||
|
|
||
|
|
||
| References | ||
| ~~~~~~~~~~~ | ||
|
|
||
| .. [1] Xu, H., 2019. Optimal measurement network of pairwise differences. Journal of Chemical Information and Modeling, 59(11), pp.4720-4728. | ||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| Core Concepts | ||
| ============= | ||
|
|
||
| This section covers the core concepts of the software, offering a deep dive into the key abstractions, data structures and theory that underpin its functionality. | ||
| For detailed step-by-step tutorials on using the software, please refer to the tutorials section. | ||
|
|
||
|
|
||
| .. toctree:: | ||
| :maxdepth: 1 | ||
|
|
||
| femap | ||
| estimators | ||
| plotting |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| ======================= | ||
| Plotting Best Practices | ||
| ======================= | ||
|
|
||
| Visualisation plays a crucial role in assessing the accuracy of binding free energy calculations. The same | ||
| raw data can be represented in many different ways, and the choice of plot or analysis metric determines which aspects of performance are | ||
| highlighted. This page explains the plot types available in ``cinnabar``, and the best practices they follow. For more | ||
| detailed information we recommend reading the companion article [1]_. | ||
|
|
||
| Edgewise :math:`\Delta\Delta G` Plots | ||
|
IAlibay marked this conversation as resolved.
|
||
| ------------------------------------- | ||
|
|
||
| Edgewise relative free energy plots allow direct comparison of calculated and experimental :math:`\Delta\Delta G` values. | ||
| These plots are best for method developers, as they show how well individual transformations | ||
| are predicted and highlight outliers. These plots can be generated from an :class:`.FEMap` using the :meth:`plot_DDGs <cinnabar.plotting.plot_DDGs>` method. | ||
|
|
||
| Best practices: | ||
|
|
||
| - **Only error statistics (RMSE, MUE)** are shown by default. Correlation measures (:math:`R^{2}`, :math:`\rho`) are not meaningful, as the direction of a relative transformation is arbitrary. | ||
| - **Error bars** are shown on to represent uncertainty in both calculated and experimental values. | ||
| - **Statistics uncertainty estimates** are bootstrapped (1000 samples with replacement) to provide 95% confidence intervals. | ||
|
|
||
|
|
||
| Absolute :math:`\Delta G` Plots | ||
| ------------------------------- | ||
|
|
||
| Absolute free energy plots compare calculated and experimental :math:`\Delta G` values for each ligand. Absolute | ||
| :math:`\Delta G` predictions can be obtained from a connected network of relative free energies via an :ref:`estimator <estimators>`. | ||
| These plots are useful for both method developers and users, as they show how well the overall ranking of ligands is predicted. | ||
| As they depend on the entire network they accumulate errors from multipule aspects of the calculation, including | ||
| edge accuracy, uncertainty quantification, perturbation network design and estimator, giving a complete picture of protocol performance. | ||
| These plots can be generated from an :class:`.FEMap` using the :meth:`plot_DGs <cinnabar.plotting.plot_DGs>` method. | ||
|
|
||
| Best practices: | ||
|
|
||
| - **Both error (RMSE, MUE) and correlation** (:math:`R^{2}`, :math:`\rho`) statistics are shown by default, as absolute data are directional and correlation is meaningful. | ||
| - **Error bars** are shown on to represent uncertainty in both calculated and experimental values. | ||
| - **Statistics uncertainty estimates** are bootstrapped (1000 samples with replacement) to provide 95% confidence intervals. | ||
| - **Mean centering** is applied by default to align the mean of calculated and experimental values. This is needed when analyzing the outcomes of relaltive free energy simulations as the absolute scale is arbitrary. | ||
|
|
||
| Pairwise (all-to-all) :math:`\Delta\Delta G` Plots | ||
| -------------------------------------------------- | ||
|
|
||
| Pairwise or all-to-all relative free energy plots compare all possible pairwise :math:`\Delta\Delta G` values between ligands in a dataset. | ||
| These are generated by calculating pairwise differences from the estimated absolute :math:`\Delta G` values. | ||
| These plots are especially useful for method developers because they remove analysis biases introduced by the perturbation network design: | ||
| every method is compared on the same set of pairwise values, even if the underlying networks differ. These plots can be generated from an :class:`.FEMap` using the :meth:`plot_all_DDGs <cinnabar.plotting.plot_all_DDGs>` method. | ||
|
|
||
| Best practices: | ||
|
|
||
| - Follow the same guidelines as for edgewise plots (error statistics only, with uncertainties represented). | ||
| - Use these plots for fair, network-independent comparisons between methods. | ||
|
|
||
|
|
||
| Summary of Best Practices | ||
| ------------------------- | ||
|
|
||
| - Use :math:`\Delta\Delta G` plots for edge-level diagnostics. | ||
| - Use absolute :math:`\Delta G` plots for global performance and ranking. | ||
| - Always represent uncertainties, both at the individual estimate level and on reported statistics (e.g. RMSE). | ||
| - When comparing across methods, use all-to-all pairwise :math:`\Delta\Delta G` to enable fair comparisons. | ||
|
|
||
|
|
||
| References | ||
| ~~~~~~~~~~~ | ||
|
|
||
| .. [1] Hahn, D.F., Bayly, C.I., Boby, M.L., Macdonald, H.E.B., Chodera, J.D., Gapsys, V., Mey, A.S., Mobley, D.L., Benito, L.P., Schindler, C.E. and Tresadern, G., 2022. Best practices for constructing, preparing, and evaluating protein-ligand binding affinity benchmarks [article v1. 0]. Living journal of computational molecular science, 4(1), p.1497. | ||
|
|
||
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -30,7 +30,7 @@ Contents | |
| :maxdepth: 1 | ||
|
|
||
| installation | ||
| getting_started | ||
| concepts/index | ||
|
|
||
|
|
||
| Reference | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.