Skip to content

Add G-theory modeling-assumption checks (normality, Tukey nonadditivity)#38

Open
sonnetx wants to merge 4 commits into
aims-foundations:mainfrom
sonnetx:feat/modeling-assumptions
Open

Add G-theory modeling-assumption checks (normality, Tukey nonadditivity)#38
sonnetx wants to merge 4 commits into
aims-foundations:mainfrom
sonnetx:feat/modeling-assumptions

Conversation

@sonnetx
Copy link
Copy Markdown
Contributor

@sonnetx sonnetx commented May 28, 2026

Follow-up to #26. Stacks on #27, #35, #36.

Summary

Adds a new src/torch_measure/metrics/assumptions.py module with two diagnostics for the G-theory variance-component model:

  • normality_check(response_matrix, target=...) — Shapiro-Wilk on a chosen effect family ("subject", "item", "subject_item", or "residual"), plus the sorted sample and theoretical quantiles needed for a Q-Q plot.
  • tukey_nonadditivity_test(response_matrix) — Tukey's 1-df test on cell means. Especially load-bearing when n_reps=1, where subject_item and residual are confounded in variance_components and this test asks whether the combined residual contains structured interaction beyond noise.

Both functions return the test statistic, p-value, and the underlying numbers (Q-Q quantiles for normality, SS components for Tukey). Levene (homogeneity) and balance/missingness checks deferred but happy to add those or other checks as needed.

Test plan

  • 14 new tests in tests/test_metrics/test_assumptions.py:
    • Normality: structure, normal residuals not rejected, heavily-skewed (lognormal) residuals rejected, each target variant works, target="residual" requires replications, validation errors.
    • Tukey: structure, additive synthesis not rejected, multiplicative synthesis rejected (with and without replications), unbalanced design raises.
  • Full tests/test_metrics/ suite green (97 tests)
  • ruff check / ruff format clean

@sangttruong sangttruong marked this pull request as ready for review May 28, 2026 15:37
@sonnetx sonnetx force-pushed the feat/modeling-assumptions branch from c5e81e9 to cf1f305 Compare May 28, 2026 16:18
@nicchiou
Copy link
Copy Markdown
Collaborator

Please include Levene's test for homogeneity of variance to this PR as well.

@sonnetx
Copy link
Copy Markdown
Contributor Author

sonnetx commented May 30, 2026

Added levene_homogeneity_test in d046e6d. Defaults to Brown-Forsythe (median-centered) for robustness; supports group_by="item" (default) or "subject" and returns the per-group sizes and variances alongside the statistic so callers can spot small-group issues. 7 new tests; full assumptions suite is 22 green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants