Skip to content

Add intraclass correlation coefficient (ICC) to generalizability metrics#35

Merged
nicchiou merged 1 commit into
aims-foundations:mainfrom
sonnetx:feat/intraclass-correlation
May 29, 2026
Merged

Add intraclass correlation coefficient (ICC) to generalizability metrics#35
nicchiou merged 1 commit into
aims-foundations:mainfrom
sonnetx:feat/intraclass-correlation

Conversation

@sonnetx
Copy link
Copy Markdown
Contributor

@sonnetx sonnetx commented May 27, 2026

Summary

Adds intraclass_correlation() to src/torch_measure/metrics/generalizability.py. ICC is a ratio of the variance components already estimated by variance_components (subjects = targets, items = raters), so it slots in next to g_coefficient and reuses the same dict.

Supported forms (McGraw & Wong 1996 / Shrout & Fleiss 1979 two-way):

  • ICC2 — two-way random, single rater, absolute agreement
  • ICC3 — two-way mixed, single rater, consistency
  • ICC2k — average of k raters, absolute agreement
  • ICC3k — average of k raters, consistency

The averaged forms are mathematically identical to the existing G-coefficient at n_reps=1 (ICC2k == g_coefficient(type="absolute"), ICC3k == g_coefficient(type="relative")) which is used as the primary correctness test. One-way forms (ICC1/ICC1k) require a different (one-way) decomposition and raise a clear ValueError rather than returning a wrong number.

Test plan

  • pytest tests/test_metrics/test_generalizability.py — 33 tests pass (10 new in TestIntraclassCorrelation: unit-interval, G-coefficient equivalence, consistency ≥ absolute, average ≥ single, n_items defaulting, ICC1/unknown-form/missing-keys errors, zero-components, real-components)
  • pytest tests/test_metrics/ — 74 pass, no regressions
  • ruff check / ruff format --check — clean

Note for reviewers

This stacks on #27 (G-theory base). Once #27 lands I'll rebase onto main and the diff will collapse to just the ICC commit, then flip this out of draft.

First of several planned G-theory follow-ups (bootstrap CIs, REML backend, assumption checks, a walk-through tutorial).

@sonnetx sonnetx force-pushed the feat/intraclass-correlation branch from 88ce9a9 to 4a0740b Compare May 28, 2026 16:16
@sonnetx sonnetx marked this pull request as ready for review May 28, 2026 16:18
@nicchiou nicchiou merged commit 536e042 into aims-foundations:main May 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants