Student decline: More than pandemic learning loss alone

# Investigate the Decline in U.S. Student Achievement

## Working Documents
- [Issue 174](https://drive.google.com/drive/u/0/folders/1F2l95aaDerpEPVvzIUXDuhDyNfgcmjk8) Google Drive Folder
- [Data Source to Hypothesis Alignment Chart](https://docs.google.com/spreadsheets/d/14ZdoOvpv6nY4ulvS1IUQmn9C2bT0H7QQSPTGqR89wE4/edit?usp=sharing) (Google Sheets)

## Resources
- [NAEP Data Service API](https://www.nationsreportcard.gov/api_documentation.aspx)
- [Education Scorecard](https://educationscorecard.org/)
- Stanford Education Data Archive (SEDA) [Data Downloads](https://edopportunity.org/opportunity/data/downloads/) | [Explorer](https://edopportunity.org/trends/explorer/)
- Urban Institute Education Data Portal [Main Portal](https://educationdata.urban.org/documentation/) | [School Districts](https://educationdata.urban.org/documentation/school-districts.html)
- EdFacts [Initiative](https://www.ed.gov/data/edfacts-initiative) | [Report cards](https://www.ed.gov/birth-to-grade-12-education/elementary-and-secondary-education/where-can-i-find-my-state-report-card-website)
- [Common Core of Data (CCD)](https://nces.ed.gov/ccd/files.asp)

Resources not yet linked/sourced

### Resources
- Civil Rights Data Collection (CRDC)
- School Pulse Panel
- State Chronic-Absenteeism Files
- ESSER Data
- National Teacher and Principal Survey (NTPS)
- Teacher Follow-up Survey
- American Community Survey (ACS)
- SAIPE
- NCES Locale Data
- State curriculum-adoption records
- Literacy-policy trackers
- District board minutes
- Procurement records
- Tutoring contracts
- School phone-policy databases
- Local attendance-program records
- Teacher vacancy postings
- State licensure databases

## Overview

U.S. reading and mathematics performance has deteriorated substantially, with the largest losses concentrated among lower-performing students. This is not exclusively a COVID-19 problem. Recent district-level research identifies a broader “learning recession” beginning around 2013, followed by a much larger pandemic-era shock and an uneven recovery.

The decline matters because foundational reading and mathematics skills affect later educational attainment, employment, earnings, health, civic participation, and the country’s ability to maintain a highly skilled workforce.

The central research question is:

> **Why have U.S. reading and mathematics outcomes declined—particularly among lower-performing students—and which interventions are most effective at reversing the decline?**

The investigation should distinguish among:

* **Pre-pandemic decline:** deterioration beginning before 2020;
* **Pandemic learning loss:** losses associated with school disruption from 2020 onward;
* **Post-pandemic recovery:** gains or continued deterioration after regular instruction resumed;
* **Lower-tail decline:** worsening performance among students near the bottom of the achievement distribution;
* **Achievement inequality:** widening gaps between higher- and lower-performing students;
* **Measurement changes:** changes in assessments, proficiency standards, participation, or reporting that can mimic real improvement or decline.

The recommended primary unit of analysis is the **school-district-year**, with student subgroup and school-level extensions where sufficiently comparable data are available.

---

## Action Items

### Initial Evidence

Several findings establish the need for investigation:

* National and district-level evidence indicates that the current learning recession began around 2013, before COVID-19.
* NAEP results show particularly severe deterioration among lower-performing students, widening the distance between the upper and lower portions of the achievement distribution.
* By spring 2022, the average student in grades 3–8 had lost approximately half a grade level in mathematics and one-third of a grade level in reading relative to 2019.
* By spring 2024, students remained nearly half a grade level below pre-pandemic performance in both subjects, while reading had continued to decline after 2022.
* Recovery has been highly uneven. More than 100 districts surpassed their pre-pandemic performance in both reading and mathematics, demonstrating that recovery is possible under current conditions.
* Chronic absenteeism remains much higher than before the pandemic. On a recent NAEP administration, 22% of fourth graders reported missing at least five days during the previous month—approximately twice the 2019 proportion.
* Districts with higher post-pandemic absenteeism generally experienced slower academic recovery, especially in high-poverty communities.
* Federal pandemic-relief spending appears to have prevented larger losses, particularly in high-poverty districts, but the academic effect varied according to how funds were used. Tutoring and summer learning were associated with stronger recovery in available district-level evidence.
* Early evidence links state adoption of evidence-based reading policies with improved reading trajectories, although policy adoption alone does not prove implementation quality or causation.

These trends establish the decline but do not prove whether attendance, instructional quality, curriculum, technology use, family conditions, school funding, or lowered expectations is the dominant cause.

---

### Hypotheses

#### H1 — Chronic absenteeism is a major driver of continued academic decline

Students cannot benefit from instruction they do not receive, and frequent absence may also disrupt the classroom environment for students who attend regularly.

**Evidence to seek:**

* Increases in absenteeism precede or coincide with score declines;
* Returning attendance toward pre-pandemic levels predicts recovery;
* Attendance effects are strongest among already low-performing students;
* Absenteeism explains part of the widening achievement gap;
* Specific attendance interventions improve both attendance and achievement.

#### H2 — Pandemic disruption accelerated a decline that began earlier

COVID-19 may have intensified, rather than created, the underlying trend.

**Evidence to seek:**

* Change-point analysis identifies deterioration before 2020;
* States or districts declining before COVID continue declining afterward;
* Pre-2013 trends differ materially from 2013–2019 trends;
* Pandemic closures explain the 2020–2022 shock but not the earlier decline.

#### H3 — Reduced instructional rigor or lowered expectations allowed students to advance without mastery

Grades, course completion, and graduation rates may have become less aligned with externally measured achievement.

**Evidence to seek:**

* Course grades increase while standardized scores decline;
* Graduation rates rise without corresponding gains in readiness;
* Promotion and credit-recovery policies predict weaker mastery;
* Districts maintaining stronger mastery requirements experience different lower-tail trends.

This hypothesis must be tested without assuming that retention or stricter grading automatically improves learning.

#### H4 — Weak foundational reading instruction contributed to reading decline

Students who do not acquire decoding, phonemic awareness, fluency, vocabulary, and comprehension early may experience compounding later losses.

**Evidence to seek:**

* States adopting evidence-based literacy policies improve relative to comparable states;
* Teacher training and curriculum implementation predict stronger outcomes;
* Early-grade gains persist into later grades;
* Results differ according to implementation quality and student subgroup.

#### H5 — Smartphone, social-media, and digital-media exposure reduced attention and academic engagement

The timing of widespread smartphone adoption overlaps with the pre-pandemic decline, but correlation alone is insufficient.

**Evidence to seek:**

* Device adoption or use predicts achievement after socioeconomic controls;
* Effects differ by age, subject, and prior achievement;
* School phone restrictions create measurable changes;
* Attention, sleep, homework completion, or attendance mediates the relationship.

#### H6 — Schools have insufficiently targeted students furthest behind

Universal programs may produce small average gains while failing to provide sufficient instructional intensity to students with the largest deficits.

**Evidence to seek:**

* High-dosage tutoring produces larger effects than low-intensity support;
* Effects are larger when tutoring occurs during the school day;
* Programs aligned with the core curriculum outperform generic assistance;
* Participation and dosage explain variation in results.

#### H7 — Teacher shortages and instability reduce instructional quality

Persistent vacancies, long-term substitutes, turnover, and out-of-field teaching may disproportionately affect high-poverty schools.

**Evidence to seek:**

* Teacher turnover predicts later score decline;
* Vacancy effects are largest in mathematics, science, special education, and rural schools;
* Stable staffing predicts better recovery among comparable districts;
* Compensation, working conditions, and leadership explain variation in retention.

#### H8 — Increased school spending has not always been directed toward interventions with strong academic effects

The question is not merely how much districts spend, but how resources are allocated.

**Evidence to seek:**

* Academic recovery differs by spending category;
* Tutoring, extended learning, attendance programs, and instructional materials outperform less targeted expenditures;
* Effects vary by poverty and baseline loss;
* Spending effects persist after temporary programs end.

#### H9 — Student mental health, disengagement, and school climate affect attendance and achievement

Anxiety, depression, bullying, safety concerns, weak belonging, and family stress may contribute to absence and reduced academic engagement.

**Evidence to seek:**

* School-climate measures predict attendance and achievement;
* Mental-health support improves attendance before scores improve;
* Effects vary by age and subgroup;
* School engagement mediates the relationship between mental health and absence.

#### H10 — The apparent national decline consists of several different local problems

Different districts may face fundamentally different combinations of attendance, staffing, curriculum, poverty, migration, language, and public-health challenges.

**Evidence to seek:**

* Districts form distinct and stable problem clusters;
* Predictors differ across rural, suburban, and urban systems;
* Successful interventions are context dependent;
* National averages conceal positive outliers worth replicating.

---

### Research Plan

#### 1. Define the principal outcomes

Do not rely only on the percentage labeled “proficient.”

Measure:

* Mean reading and mathematics scores;
* Scores at the 10th, 25th, 50th, 75th, and 90th percentiles;
* Percentage below NAEP Basic;
* Percentage at or above NAEP Proficient;
* Student growth where longitudinal measures are available;
* Achievement gaps;
* Chronic absenteeism;
* Course failure;
* Graduation;
* College remediation or readiness.

The primary lower-tail metric should be:

A distributional-gap measure should also be calculated:

This prevents an improving average from concealing worsening outcomes among the lowest-performing students.

#### 2. Establish the historical timeline

Separate the analysis into:

* 2003–2012: earlier baseline;
* 2013–2019: pre-pandemic learning recession;
* 2020–2022: pandemic disruption;
* 2022 onward: recovery period.

Use segmented regression or change-point detection to test whether 2013 and 2020 represent statistically meaningful breaks rather than imposing them as assumptions.

#### 3. Build a district-year panel

Recommended variables:

* Reading and mathematics achievement;
* Achievement by percentile and subgroup;
* Chronic absenteeism;
* Enrollment;
* Poverty;
* Race and ethnicity;
* English-learner status;
* Disability status;
* Per-pupil spending;
* Spending categories;
* Teacher vacancies and turnover;
* Student-teacher ratio;
* School closures and remote-learning duration;
* Tutoring and summer-school programs;
* Curriculum adoption;
* Smartphone policies;
* Mental-health staffing;
* Urbanicity;
* Local unemployment and household conditions.

Use NCES district identifiers to join datasets and preserve historical crosswalks for district consolidations and boundary changes.

#### 4. Examine distributional change

Analyze whether:

* The entire score distribution shifted downward;
* Losses were concentrated below the median;
* High-performing students remained stable;
* Achievement inequality increased within the same demographic groups;
* Lower-tail losses differ by subject and grade.

Quantile regression should be used where student-level or suitable grouped data permit it:

Estimate effects at several quantiles instead of assuming the same relationship for all students.

#### 5. Estimate the effect of absenteeism

Model:

Because low achievement can itself increase disengagement and absence, use:

* Lagged attendance measures;
* Attendance shocks caused by weather or illness where defensible;
* Within-district changes;
* Student fixed effects where longitudinal records are available;
* Intervention-based comparisons.

Do not interpret a simple attendance-score correlation as causal.

#### 6. Evaluate evidence-based literacy reforms

Identify states and districts adopting:

* Structured literacy;
* Universal early screening;
* Phonics-aligned curricula;
* Teacher retraining;
* Reading coaches;
* Third-grade intervention;
* Dyslexia screening.

Use event studies and difference-in-differences to compare changes before and after implementation.

Measure:

* Early-grade reading;
* Later-grade persistence;
* Lower-tail improvement;
* Special-education identification;
* Grade retention;
* Subgroup effects.

#### 7. Evaluate tutoring and extended learning

Create an intervention dataset containing:

* Tutoring provider;
* Student eligibility;
* Group size;
* Sessions per week;
* Minutes per session;
* In-school or after-school delivery;
* Tutor qualifications;
* Curriculum alignment;
* Participation;
* Cost per student.

Estimate intention-to-treat and treatment-on-the-treated effects where possible. Dosage must be measured; merely offering tutoring is not equivalent to delivering it.

#### 8. Investigate smartphone and social-media hypotheses

Possible designs include:

* District or state phone-policy changes;
* Staggered policy adoption;
* Student survey data;
* Age-based exposure differences;
* Changes in sleep, attention, discipline, and attendance.

Primary outcomes should include both achievement and plausible mediators. A policy that changes test scores without changing attention, attendance, or engagement would require careful interpretation.

#### 9. Analyze teacher workforce instability

Measure:

* Annual turnover;
* Vacancy rate;
* Emergency credentials;
* Long-term substitutes;
* Teacher experience;
* Out-of-field teaching;
* Salary relative to comparable occupations;
* Class size.

Compare schools exposed to staffing shocks with similar schools experiencing stable staffing.

#### 10. Identify positive-deviant districts

Find districts that:

* Serve high-poverty populations;
* Experienced substantial pandemic losses;
* Improved faster than comparable districts;
* Reduced lower-tail inequality;
* Sustained gains across multiple years.

Use matched comparison, clustering, and case-study validation to determine which practices distinguish them. The Education Scorecard currently identifies more than 100 districts improving faster than their peers in both reading and mathematics.

#### 11. Test spending effectiveness

Estimate:

Separate spending on:

* Tutoring;
* Summer learning;
* Additional instructional time;
* Staffing;
* Technology;
* Facilities;
* Mental health;
* Attendance;
* Administrative operations.

Use cost-effectiveness measures:

[
CostEffectiveness_k
===================

\frac{AchievementGain_k}
{CostPerStudent_k}
]

#### 12. Perform robustness checks

Repeat results:

* By grade;
* By subject;
* By achievement percentile;
* By poverty level;
* By race and ethnicity;
* By disability and English-learner status;
* With and without pandemic years;
* Using NAEP and state assessments;
* Using proficiency and continuous scores;
* Using district and state units;
* With alternative definitions of recovery.

#### 13. Produce the final PowerPoint

Recommended structure:

1. The national learning recession;
2. Why averages conceal lower-performing students;
3. Pre-2013, pre-pandemic, pandemic, and recovery periods;
4. Reading trends;
5. Mathematics trends;
6. Chronic absenteeism;
7. Staffing and instructional conditions;
8. Literacy-policy evidence;
9. Tutoring and extended learning;
10. Districts recovering faster than peers;
11. Which hypotheses survived testing;
12. Policy interventions ranked by evidence and cost-effectiveness.

The final report must distinguish:

* Test performance from intelligence;
* Proficiency labels from continuous achievement;
* Pandemic effects from earlier trends;
* Correlation from causation;
* Program availability from actual participation;
* Average recovery from lower-tail recovery.

---

## Resources

### Data, APIs, and Where to Get Data

#### NAEP Data Service API

The National Assessment of Educational Progress provides an official API for retrieving achievement estimates, percentiles, proficiency levels, jurisdictions, grades, subjects, years, and student subgroups.

Use for:

* National and state trends;
* Percentile analysis;
* Achievement gaps;
* Reading and mathematics;
* Grade 4, grade 8, and long-term trends;
* Survey-variable analysis.

NAEP should establish the nationally comparable trend, but it generally cannot support annual district analysis outside participating large urban districts.

#### Education Scorecard

The Harvard–Stanford–Dartmouth Education Scorecard combines state assessments with NAEP to estimate comparable district-level achievement for roughly 35 million students in grades 3–8 across 43 states. It also provides data on academic recovery and chronic absenteeism.

Use for:

* District-level achievement;
* Pandemic loss and recovery;
* Cross-state comparisons;
* High-poverty district analysis;
* Identification of rapidly improving districts.

#### Stanford Education Data Archive

SEDA provides downloadable, harmonized data on district- and school-level achievement, achievement gaps, educational opportunity, demographics, and geography. Technical documentation and public data files are available through Stanford’s Educational Opportunity Project.

Use for:

* Long-run district comparisons;
* Achievement gaps;
* Cohort trends;
* Geographic opportunity analysis.

#### Urban Institute Education Data Portal API

The Education Data Portal offers a documented public API aggregating data from NCES, EDFacts, CRDC, IPEDS, and other education sources.

Use for:

* School and district characteristics;
* Enrollment;
* staffing;
* finances;
* assessments;
* discipline;
* demographics;
* directory and identifier crosswalks.

The API follows a structured endpoint pattern and can also be accessed through an R package.

#### EDFacts

The U.S. Department of Education publishes state-reported school and district data on:

* Chronic absenteeism;
* Assessments;
* graduation;
* English learners;
* students with disabilities;
* participation;
* school performance.

National chronic-absenteeism download files and data notes are available through Ed Data Express.

The chronic-absenteeism file specification is FS195. Current federal specifications continue to define the required reporting structure.

#### Common Core of Data

Published by NCES.

Use for:

* School and district directories;
* enrollment;
* staffing;
* grade span;
* urbanicity;
* school closures;
* district finances;
* historical identifiers.

#### Civil Rights Data Collection

Use for:

* Discipline;
* course access;
* school counselors;
* nurses;
* psychologists;
* teacher experience;
* advanced coursework;
* restraint and seclusion;
* absenteeism in selected releases.

CRDC is generally collected every two years, which must be considered when building annual panels.

#### School Pulse Panel

Published by NCES.

Use for:

* Attendance challenges;
* staffing vacancies;
* tutoring;
* mental health;
* technology;
* school climate;
* recovery programs.

It is valuable for timely national estimates but not always designed for local causal analysis.

#### State Assessment Portals

State education departments publish:

* School- and district-level test scores;
* growth;
* proficiency;
* attendance;
* graduation;
* staffing;
* accountability indicators.

These data often provide greater local detail than federal sources but require harmonization because assessment scales and proficiency definitions differ across states and years.

At least 13 states changed proficiency definitions after 2019, making direct before-and-after comparisons potentially misleading without NAEP-based linking or scale harmonization.

#### State Chronic-Absenteeism Files

Many states publish annual school-level attendance data. California, for example, provides downloadable 2024–25 chronic-absenteeism files with subgroup measures and suppresses cells of 10 or fewer eligible students for privacy.

#### Every Student Succeeds Act State Report Cards

Use for:

* Assessments;
* attendance;
* graduation;
* teacher qualifications;
* school finance;
* subgroup performance.

Automated extraction may require state-specific scrapers or downloads.

#### Elementary and Secondary School Emergency Relief Data

Use federal and state ESSER transparency files for:

* Allocations;
* expenditures;
* spending categories;
* district-level relief amounts;
* timing of expenditure.

Combine with local budget records to distinguish planned spending from actual spending.

#### National Teacher and Principal Survey

Use for:

* Teacher working conditions;
* experience;
* turnover intentions;
* class size;
* professional development;
* school leadership.

#### Teacher Follow-up Survey

Use for:

* Teacher attrition;
* mobility;
* reasons for leaving;
* occupational transitions.

#### American Community Survey API

Use for district or community controls involving:

* Poverty;
* parental education;
* employment;
* internet access;
* household structure;
* language;
* housing instability;
* transportation.

#### Small Area Income and Poverty Estimates

Use for annual school-district poverty estimates and Title I-related socioeconomic controls.

#### National Center for Education Statistics Locale Data

Use for consistent classification of:

* City;
* suburb;
* town;
* rural district.

#### Optional Intervention Sources

* State curriculum-adoption records;
* Literacy-policy trackers;
* District board minutes;
* Procurement records;
* Tutoring contracts;
* School phone-policy databases;
* Local attendance-program records;
* Teacher vacancy postings;
* State licensure databases.

These sources may require manual collection and policy coding before they can be analyzed.

### Foundational Resources

* Harvard Center for Education Policy Research, *Education Scorecard*.
* *Pivoting from Pandemic Recovery to Long-Term Reform*.
* Harvard Graduate School of Education, *Education Recovery Scorecard Releases Third Report*.
* National Assessment Governing Board, *A Primer on Attendance and Chronic Absenteeism*.
* Stanford Educational Opportunity Project, SEDA data and documentation.
* U.S. Department of Education, EDFacts chronic-absenteeism files.

### Central Scientific Caution

Falling test scores do not automatically prove that schools have become less effective. Scores can also change because of attendance, student population shifts, family conditions, assessment participation, test design, or disrupted learning outside school.

The strongest investigation will triangulate:

1. Nationally comparable NAEP scores;
2. State and district assessments;
3. Attendance;
4. Student demographics;
5. Instructional exposure;
6. Teacher capacity;
7. Program participation;
8. Spending by intervention;
9. Outcomes at multiple achievement percentiles.

The principal success measure should not be whether average scores rise. It should be whether **students furthest behind make sustained gains without reducing progress among other students**.

- If this issue requires access to 311 data, please answer the following questions:
- Do you need a one-time or ongoing dump of the data?
- Do you need subset of data (i.e. certain years) or the entire data set (approx. 4 million rows or 11 GB)?
- If a subset is needed, please define subset characteristics (i.e. date range, etc.)
- Do you need online access via an API or a download of data?

Start with the linked Issue 174 working folder and the Data Source to Hypothesis Alignment Chart, then review the NAEP Data Service API and the listed education data portals. Use the Research Plan to define school-district-year outcomes and historical periods before evaluating the hypotheses. Done means producing the planned evidence-based investigation while distinguishing pre-pandemic decline, pandemic disruption, recovery, and lower-tail outcomes.

Written by the indexing model from the issue text.