Skip to content

feat(reporting): add full_report() six-stat bundler (n / CI / method / p / effect size / statistic) - #67

Open
ywatanabe1989 wants to merge 2 commits into
developfrom
feat/six-stat-full-report
Open

feat(reporting): add full_report() six-stat bundler (n / CI / method / p / effect size / statistic)#67
ywatanabe1989 wants to merge 2 commits into
developfrom
feat/six-stat-full-report

Conversation

@ywatanabe1989

Copy link
Copy Markdown
Collaborator

Summary

Driven by the operator's six-stat reporting doctrine (2026-07-05, tracked
as scitex-todo card scitex-stats-six-stat-report-doctrine): every
reported statistic must carry all SIX of (1) n, (2) 95% CI, (3)
method/test name, (4) p-value, (5) effect size, (6) test statistic.
Partial reporting is treated as incomplete.

Today's test_*() / run_test() result dicts already carry 5 of 6
(method, statistic, pvalue, effect_size, n) — there was no confidence-
interval field anywhere. This PR adds a full_report() helper that
bundles all six into one dict + a human-readable string, and makes
"missing a field" a raised error, not a silent gap.

  • scitex_stats.reporting.full_report(result, *, data=None, data2=None, ci=None, confidence=0.95, n_bootstrap=10_000, random_state=None, strict=True)
    (src/scitex_stats/reporting/_full_report.py)
    • Accepts a run_test()/test_*()-style result dict, plus either an
      already-computed ci=(lower, upper) tuple or the raw data/data2
      arrays so it can derive one itself.
    • CI derivation: analytic closed-form via
      scipy.stats.ttest_ind/ttest_rel/ttest_1samp(...).confidence_interval()
      for parametric t-tests (reusing scipy's own machinery rather than
      re-deriving the formula); percentile bootstrap via
      scipy.stats.bootstrap for anything without a closed form. Prefers
      scitex_stats.resampling.bootstrap_ci when importable (that module
      is currently on open, unmerged PR feat(resampling): add auc_ci, delta_auc_ci, bootstrap_ci #66 — this hands off to it
      once it lands, rather than duplicating it now).
    • Raises IncompleteReportError (a ValueError subclass) when any of
      the six fields can't be determined — pass strict=False to log a
      warning and return a partial report with missing_fields instead.
    • Returns: method, statistic, stat_symbol, pvalue,
      effect_size, effect_size_metric, n, ci, ci_level,
      formatted, missing_fields.
  • Wired into the top-level lazy loader (src/scitex_stats/__init__.py),
    following the exact _LAZY_ATTRS/__all__ pattern used for the other
    submodules (auto, descriptive, etc.) — import scitex_stats as ss; ss.full_report(...) works without eagerly importing scipy/matplotlib.
  • Added fmt_sym_md() to _utils/_formatters.py: a markdown-italic
    counterpart to the existing matplotlib-mathtext fmt_sym(), for
    plain-text six-stat strings (e.g. *t*, *n*_x, *N*_subjects).
    Preserves the N (subject-level) / n (window-level) convention
    verbatim — it only italicizes the base letter, it doesn't decide N vs
    n (a generic array-based test function has no way to know which level
    its input represents; that judgement stays with the caller).
  • Updated the module docstring "Functionalities" bullets in
    src/scitex_stats/__init__.py.

Example

import numpy as np
from scitex_stats import run_test, full_report

x = np.random.default_rng(0).normal(0, 1, 50)
y = np.random.default_rng(1).normal(0.8, 1, 50)
result = run_test("ttest_ind", data=x, data2=y)
report = full_report(result, data=x, data2=y)
print(report["formatted"])
# Welch's t-test (independent): *t* = 2.34, *p* = .021, *d* = 0.47,
# 95% CI [0.12, 0.89], *n*_x = 50, *n*_y = 50

Notes on scope

  • No existing test_*() files were rewritten — this is additive. I
    spot-checked several _plot_* call sites (fmt_sym('n') usage across
    shapiro/pearson/kendall/wilcoxon/anova/ttest_rel) and found the
    existing matplotlib italics already correct; no bugs to spot-fix
    there.
  • scitex_stats.resampling / effect_size_from_ci (referenced in the
    original task brief as "already landed") are actually still on open
    PR feat(resampling): add auc_ci, delta_auc_ci, bootstrap_ci #66
    , not merged into develop — verified via gh pr view 66
    (mergedAt: null). This PR's bootstrap fallback uses
    scipy.stats.bootstrap (already a hard dependency) directly instead,
    with a guarded try/except ImportError hand-off to
    scitex_stats.resampling.bootstrap_ci for if/when feat(resampling): add auc_ci, delta_auc_ci, bootstrap_ci #66 merges.

Test plan

  • pytest tests/scitex_stats/reporting/ — 19 new tests, all pass
  • pytest tests/scitex_stats/_utils/test__fmt_sym_md.py — 4 new tests, all pass
  • pytest tests/scitex_stats/_utils/ tests/scitex_stats/test__dispatch.py — 249 passed, no regressions
  • pytest tests/develop/test_audit.py (PA-307 audit_all_for_package) — passes; fixed 4 initial STX-TQ002 (AAA-marker) violations in the new test file before this
  • scitex-dev linter check-files on all changed files — only pre-existing-pattern warnings (scipy import, matplotlib import — same as e.g. tests/parametric/_test_ttest_ind.py), no new errors

… doctrine

Encodes the operator's 2026-07-05 six-stat reporting doctrine (n, 95% CI,
method, p-value, effect size, test statistic all required; partial
reporting is incomplete) as a checked invariant rather than a docs
convention.

- scitex_stats.reporting.full_report(result, ...) bundles a run_test()/
  test_*() result dict into all six fields, deriving a 95% CI when not
  already present: analytically via scipy.stats.ttest_*().confidence_interval()
  for parametric t-tests, or via bootstrap (scipy.stats.bootstrap, with a
  preferred hand-off to scitex_stats.resampling.bootstrap_ci once PR #66
  lands) otherwise. Raises IncompleteReportError by default when any of
  the six fields can't be determined (opt out via strict=False).
- Wired into the top-level lazy loader (__init__.py) alongside the other
  submodules, following the existing _LAZY_ATTRS/__all__ convention.
- Added _utils._formatters.fmt_sym_md(): markdown-italic counterpart to
  the existing matplotlib-mathtext fmt_sym(), for plain-text six-stat
  strings. Preserves the N (subject-level) / n (window-level) convention
  verbatim -- it only italicizes the base letter, doesn't decide N vs n.
- Tests: tests/scitex_stats/reporting/test__full_report.py (19 cases) and
  tests/scitex_stats/_utils/test__fmt_sym_md.py (4 cases), one assertion
  + AAA markers per function per repo convention.
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Please sign the SciTeX CLA before your contribution can be merged.
Comment I have read and agree to the SciTeX CLA. to sign.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

The standalone tests/scitex_stats/_utils/test__fmt_sym_md.py was an
orphan test file per PS-204 §2: fmt_sym_md() lives inside the existing
_utils/_formatters.py, not a new _fmt_sym_md.py, so its tests belong in
the existing test__formatters.py mirror file.

Merged the 4 fmt_sym_md test cases into test__formatters.py and deleted
the orphan file. `scitex-dev ecosystem audit-all scitex-stats --path .`
now reports 0 errors (audit-python-apis: no violations).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants