Project Overview: Update TMD national and area data

[donboyd5]

Goals

• Update national TMD data to use 2022 IRS data and 2022 CPS, and area data to use 2022 IRS data
• Update outyears to reflect recent economic history and current economic forecasts
• Produce updated area weights for all states and all Congressional Districts

This is the master issue - showing the anticipated overall project structure and progress. Thanks to @martinholmer for advice on how to do this.

The next comment shows tasks and subtasks. I will update it periodically as I complete tasks and make adjustments to project organization.

[donboyd5]

Tasks and subtasks

TMD infrastructure enhancements and fixes

• Fix SSL certificate verification error preventing CPS data downloads #354 -- addressed by Fix SSL certificate verification for CPS downloads -- cleaner version #357
• Fix pandas ChainedAssignmentError warning in create_taxcalc_growth_factors.py (not essential in short term) #373-- addressed by PR fix pandas chained assignment #374
• Tax expenditure tests fail when using Tax-Calculator 5.2.0 but not 5.1.0 #356 -- addressed by Fix failing taxcalc 5.2.0 tests #362
• Add GPU acceleration of national reweighting when GPU is available, falling back to CPU otherwise #366 -- addressed by Add GPU Acceleration for National Weight Optimization #367
• Ignore local venv -- addressed by ignore local venv #361
• Add claude/ to .gitignore
  -- addressed by Add claude/ to .gitignore #359
• Add .quarto/ directories to .gitignore
  -- addresssed by Add .quarto/ directories to .gitignore #358
• Create reproducible environments for subprojects in R #379 -- to come
• Address deprecation warning in scipy minimize call to L-BFGS-B (low priority) -- to come

---

Update national tmd data to 2022 base year and grow from there

• Prepare initial folder structure for national target development and reproduce 2021 national targets #370 -- addressed by Add potential 2022 targets data to national_targets folder #376
• Ensure that newly-included potential national targets for 2022 are the values that were intended
• Map and document relationships between PUF variables and IRS spreadsheets -- in progress
• Update CPS data to the 2022 income year #369 -- to come
• Update TMD targets and targeting process to reflect the 2022 targets -- to come
• Develop additional comparisons and tests for the new data -- to come
• Update growfactors as appropriate

---

Construct weights for states based on 2022 IRS SOI Historical Table 2 data for states

• Obtain and computerize 2022 IRS state targeting data from SOI Historical Table 2
• Updating and, as appropriate, extend mapping of IRS reporting concepts to PUF concepts (i.e., which IRS table-column-row or combination of these corresponds most closely to a particular PUF variable or combination of PUF variables?)
• Develop proxy linkages as needed, where an SOI HT2 variable, or combination of variables, is used to apportion a related PUF variable, or combination of variables, across states
• Incorporate updated data into the TMD framework
• Improve documentation
• Analyze results
• Tests

---

Construct weights for Congressional Districts based on 2022 IRS SOI historical data

• Obtain and computerize 2022 IRS CD targeting data from SOI historical tables
• Update and, as appropriate, extend mapping of IRS reporting concepts to PUF concepts (i.e., which IRS table-column-row or combination of these corresponds most closely to a particular PUF variable or combination of PUF variables?)
• Develop proxy linkages as needed, where an SOI historical data variable, or combination of variables, is used to apportion a related PUF variable, or combination of variables, across CDs
• Incorporate updated data into the TMD framework
• Improve documentation
• Analyze results
• Tests

[donboyd5]

Background

The current base year for tax microdata benchmarking (tmd) is 2021. The national file is targeted in detail to 2021 levels, and grows from there (with growfactors). State and Congressional District weights were also developed based on 2021 data. The reason for this was data availability: At the time the tmd data were prepared, the greatest amount of detailed data were available for 2021.

Now, detailed IRS targeting data are available nationally, for states, and for Congressional Districts for 2022. This, in concept, is a much better year to use for targeting because 2021 was severely affected by disruptions related to COVID and to economic and policy responses to COVID. 2022 is likely to be far less affected by COVID-related disruptions.

Under an agreement with PSL, I will update the national data to 2022, and growth factors to later years, and then update state and Congressional District weights using these updated national data, allocated to states and CDs relying heavily on the now-available 2022 IRS data for states and CDs.