Skip to content

Add Taylor linearization and R-parity features, validated against the R survey package - #330

Open
ayushpatnaikgit wants to merge 40 commits into
mainfrom
wp
Open

ayushpatnaikgit wants to merge 40 commits into
mainfrom
wp

Conversation

@ayushpatnaikgit

Copy link
Copy Markdown
Member

This PR closes most of the everyday-usage gap with R's survey package and adds a test
suite that validates every deterministic estimate against reference values computed by
R (survey 4.5), to ~1e-9 relative tolerance.

Taylor linearization (the foundation)

SurveyDesign estimators now compute standard errors by Taylor linearization — the
default variance method of R's svydesign — including finite population correction
when the design is constructed with popsize, and support for stratified one-stage
cluster designs with nested PSU ids (validated on NHANES):

  • mean, total, ratio on SurveyDesign return SE columns matching
    svymean/svytotal/svyratio to machine precision
  • Domain estimation (mean(:y, :domain, design), etc.) now computes SEs via score
    variables on the full design, matching svyby. The previous subset-based approach
    dropped PSUs without domain members and produced incorrect design-based SEs.
  • degf(design) — degrees of freedom (PSUs − strata), as in R

New features

Julia R equivalent
mean(:stype, d) / total(:stype, d) svymean/svytotal on a factor (proportions/counts)
var(:y, d), std(:y, d) svyvar
confint(est), confint(est; dof=degf(d)) confint, confint(..., df=degf(d))
svytable(d, :a, :b) svytable
svyttest(:y, :group, d) svyttest
svychisq(d, :a, :b) (:F and :Chisq) svychisq (Rao–Scott)
quantile(:y, d, p; ci=true) oldsvyquantile(..., ci=TRUE) (Woodruff CIs)
trimweights(d; lower, upper, strict) trimWeights
poststratify(d, :var, population) postStratify
rake(d, margins, populations) rake
brrweights(d; fay_rho) as.svrepdesign(type="BRR"/"Fay")

poststratify and rake on a ReplicateDesign adjust every set of replicate weights
separately (as in R), so replicate SEs fully reflect the calibration. On a plain
SurveyDesign the adjusted weights match R exactly, but linearized SEs treat them as
fixed (conservative) — documented in the docstrings.

BRR uses a Sylvester Hadamard matrix by default (identical to R's choice for the
tested stratum counts) and accepts a hadamard_matrix argument to reproduce R for any
stratum count. Fay's method is a new BRRReplicates inference type.

Validation against R

  • test/r_reference/generate_fixtures.R computes reference values with R survey 4.5
    and writes them to CSVs (committed), so CI needs no R installation. Regenerate with
    Rscript generate_fixtures.R.
  • test/r_comparison.jl — 527 assertions across 16 testsets. Since the api, scd
    and nhanes datasets shipped here are identical to R's, deterministic results
    (linearized SEs, jackknife, BRR with matching Hadamard matrix, weights from
    trimming/post-stratification/raking) agree to machine precision.

Behaviour changes

  • mean/total/ratio on SurveyDesign gained an SE column.
  • Quantile point estimates now follow R's oldsvyquantile (ties="discrete", linear
    interpolation of the inverse weighted CDF): e.g. the apisrs median is now 658
    (R's answer) instead of 659.
  • New dependency: Distributions (already transitive via GLM).

Fixes along the way

  • Three stale hardcoded bootstrap expectations (test/total.jl, test/reg.jl) that
    fail on Julia 1.11 because the RNG stream changed across Julia versions — verified
    pre-existing on a clean checkout; tolerances corrected.
  • Docstring doctests regenerated and passing under Documenter.doctest.
  • CI and docs workflows pinned to Julia 1.11: the CairoMakie versions allowed by
    current compat bounds fail to precompile on Julia 1.12 (this broke CI before any
    tests ran, independent of this PR's content). Upgrading the Makie stack is left as
    a follow-up.

Note: this branch also contains the working-paper commits (paper/), which predate
the feature work.

🤖 Generated with Claude Code

ayushpatnaikgit and others added 30 commits April 12, 2024 17:58
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
Added co-author
JuliaCon Proceedings Author's Guide says we should only edit these 3 files because other files might be overwritten later:
`paper.tex`
`ref.bib`
`paper.yml`
Co-authored-by: Nadia <76887318+nadiaenh@users.noreply.github.com>
added Shikhar

Co-authored-by: Ayush Patnaik <ayushpatnaik@gmail.com>
ayushpatnaikgit and others added 10 commits May 29, 2024 13:05
Add paper for JuliaCon proceedings
The CairoMakie versions allowed by the current compat bounds fail to
precompile on Julia 1.12 (removed Core.TypeName.mt internal), which broke
every workflow before tests could run. Pin to 1.11 until the Makie stack
is upgraded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts:
#	src/Survey.jl
#	src/mean.jl
#	src/quantile.jl
#	test/quantile.jl
#	test/reg.jl
#	test/runtests.jl
#	test/total.jl
The domain estimation examples printed standard errors that are
mathematically zero, so their floating-point noise differed between
architectures and failed the docs build on CI. Use stype domains (all
meaningfully nonzero) and round the post-stratified totals instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Preview deployment from pull_request events fails with the repository's
deploy key (Permission denied), while regular push deploys work. PR runs
now build the docs and doctests and skip deployment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants