feat(reporting): add full_report() six-stat bundler (n / CI / method / p / effect size / statistic) - #67
Open
ywatanabe1989 wants to merge 2 commits into
Open
feat(reporting): add full_report() six-stat bundler (n / CI / method / p / effect size / statistic)#67ywatanabe1989 wants to merge 2 commits into
ywatanabe1989 wants to merge 2 commits into
Conversation
… doctrine Encodes the operator's 2026-07-05 six-stat reporting doctrine (n, 95% CI, method, p-value, effect size, test statistic all required; partial reporting is incomplete) as a checked invariant rather than a docs convention. - scitex_stats.reporting.full_report(result, ...) bundles a run_test()/ test_*() result dict into all six fields, deriving a 95% CI when not already present: analytically via scipy.stats.ttest_*().confidence_interval() for parametric t-tests, or via bootstrap (scipy.stats.bootstrap, with a preferred hand-off to scitex_stats.resampling.bootstrap_ci once PR #66 lands) otherwise. Raises IncompleteReportError by default when any of the six fields can't be determined (opt out via strict=False). - Wired into the top-level lazy loader (__init__.py) alongside the other submodules, following the existing _LAZY_ATTRS/__all__ convention. - Added _utils._formatters.fmt_sym_md(): markdown-italic counterpart to the existing matplotlib-mathtext fmt_sym(), for plain-text six-stat strings. Preserves the N (subject-level) / n (window-level) convention verbatim -- it only italicizes the base letter, doesn't decide N vs n. - Tests: tests/scitex_stats/reporting/test__full_report.py (19 cases) and tests/scitex_stats/_utils/test__fmt_sym_md.py (4 cases), one assertion + AAA markers per function per repo convention.
Contributor
|
Please sign the SciTeX CLA before your contribution can be merged. I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
The standalone tests/scitex_stats/_utils/test__fmt_sym_md.py was an orphan test file per PS-204 §2: fmt_sym_md() lives inside the existing _utils/_formatters.py, not a new _fmt_sym_md.py, so its tests belong in the existing test__formatters.py mirror file. Merged the 4 fmt_sym_md test cases into test__formatters.py and deleted the orphan file. `scitex-dev ecosystem audit-all scitex-stats --path .` now reports 0 errors (audit-python-apis: no violations).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Driven by the operator's six-stat reporting doctrine (2026-07-05, tracked
as scitex-todo card
scitex-stats-six-stat-report-doctrine): everyreported statistic must carry all SIX of (1) n, (2) 95% CI, (3)
method/test name, (4) p-value, (5) effect size, (6) test statistic.
Partial reporting is treated as incomplete.
Today's
test_*()/run_test()result dicts already carry 5 of 6(method, statistic, pvalue, effect_size, n) — there was no confidence-
interval field anywhere. This PR adds a
full_report()helper thatbundles all six into one dict + a human-readable string, and makes
"missing a field" a raised error, not a silent gap.
scitex_stats.reporting.full_report(result, *, data=None, data2=None, ci=None, confidence=0.95, n_bootstrap=10_000, random_state=None, strict=True)(
src/scitex_stats/reporting/_full_report.py)run_test()/test_*()-style result dict, plus either analready-computed
ci=(lower, upper)tuple or the rawdata/data2arrays so it can derive one itself.
scipy.stats.ttest_ind/ttest_rel/ttest_1samp(...).confidence_interval()for parametric t-tests (reusing scipy's own machinery rather than
re-deriving the formula); percentile bootstrap via
scipy.stats.bootstrapfor anything without a closed form. Prefersscitex_stats.resampling.bootstrap_ciwhen importable (that moduleis currently on open, unmerged PR feat(resampling): add auc_ci, delta_auc_ci, bootstrap_ci #66 — this hands off to it
once it lands, rather than duplicating it now).
IncompleteReportError(aValueErrorsubclass) when any ofthe six fields can't be determined — pass
strict=Falseto log awarning and return a partial report with
missing_fieldsinstead.method,statistic,stat_symbol,pvalue,effect_size,effect_size_metric,n,ci,ci_level,formatted,missing_fields.src/scitex_stats/__init__.py),following the exact
_LAZY_ATTRS/__all__pattern used for the othersubmodules (
auto,descriptive, etc.) —import scitex_stats as ss; ss.full_report(...)works without eagerly importing scipy/matplotlib.fmt_sym_md()to_utils/_formatters.py: a markdown-italiccounterpart to the existing matplotlib-mathtext
fmt_sym(), forplain-text six-stat strings (e.g.
*t*,*n*_x,*N*_subjects).Preserves the N (subject-level) / n (window-level) convention
verbatim — it only italicizes the base letter, it doesn't decide N vs
n (a generic array-based test function has no way to know which level
its input represents; that judgement stays with the caller).
src/scitex_stats/__init__.py.Example
Notes on scope
test_*()files were rewritten — this is additive. Ispot-checked several
_plot_*call sites (fmt_sym('n')usage acrossshapiro/pearson/kendall/wilcoxon/anova/ttest_rel) and found the
existing matplotlib italics already correct; no bugs to spot-fix
there.
scitex_stats.resampling/effect_size_from_ci(referenced in theoriginal task brief as "already landed") are actually still on open
PR feat(resampling): add auc_ci, delta_auc_ci, bootstrap_ci #66, not merged into
develop— verified viagh pr view 66(
mergedAt: null). This PR's bootstrap fallback usesscipy.stats.bootstrap(already a hard dependency) directly instead,with a guarded
try/except ImportErrorhand-off toscitex_stats.resampling.bootstrap_cifor if/when feat(resampling): add auc_ci, delta_auc_ci, bootstrap_ci #66 merges.Test plan
pytest tests/scitex_stats/reporting/— 19 new tests, all passpytest tests/scitex_stats/_utils/test__fmt_sym_md.py— 4 new tests, all passpytest tests/scitex_stats/_utils/ tests/scitex_stats/test__dispatch.py— 249 passed, no regressionspytest tests/develop/test_audit.py(PA-307audit_all_for_package) — passes; fixed 4 initial STX-TQ002 (AAA-marker) violations in the new test file before thisscitex-dev linter check-fileson all changed files — only pre-existing-pattern warnings (scipy import, matplotlib import — same as e.g.tests/parametric/_test_ttest_ind.py), no new errors