Skip to content

Commit 5819e3e

Browse files
committed
enh: save stat values in a pickle file
1 parent 1c1f484 commit 5819e3e

File tree

2 files changed

+9
-0
lines changed

2 files changed

+9
-0
lines changed

README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,11 @@ Each model contains:
164164
to true.
165165
- One figure per metric with performance distribution across splits (with or
166166
without null distribution trained on permuted labels)
167+
- One figure per any metric with the word `score` in it reporting the results of
168+
a Wilcoxon signed rank test. The figure reports one-sided stats values as the
169+
color of each cell and the corresponding `-log10(pvalue)` as the annotation.
170+
Higher numbers indicate stronger effect (color) and lower p-values (annotation).
171+
The actual numeric values are stored in a correspondingly named pkl file.
167172
- `shap-{timestamp}` dir
168173
- SHAP values are computed for each prediction in each split's test set
169174
(e.g., 30 bootstrapping splits with 100 prediction will create (30,100) array).

pydra_ml/report.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -358,6 +358,10 @@ def gen_report(
358358
)
359359
ax.xaxis.set_ticks_position("top")
360360
plt.savefig(f"stats-{name}-{timestamp}.png")
361+
save_obj(
362+
dict(effects=effects, pvalues=pvalues, order=order),
363+
f"stats-{name}-{timestamp}.pkl",
364+
)
361365

362366
# create SHAP summary csv and figures
363367
if gen_shap:

0 commit comments

Comments
 (0)