hackforla / hackforla/data-science

Decline in productivity in the construction industry

Open
#272 0 comments 0 reactions 0 assignees View on GitHub
complexity: medium CoP: Data Science project duration: one time role: data analysis size: 5pt
Dominant language
Jupyter Notebook
Stars
33
Forks
22
PR merge metrics
No merged PRs in 30d

Description

# Investigate the Decline of U.S. Construction Productivity

## Overview

Construction is the only major U.S. industry to have recorded sustained negative or near-zero productivity growth over several decades. Real construction output per worker fell by more than 30% between 1970 and 2020, while economy-wide productivity more than doubled. Recent research also finds that the decline remains visible when productivity is measured using physical outputs, such as housing units completed per worker, rather than inflation-adjusted dollar values.

This matters because construction supplies the housing, transportation systems, energy infrastructure, factories, schools, hospitals, water systems, and research facilities on which the rest of the economy depends. Low productivity can produce fewer completed projects, higher public costs, worsening housing affordability, and slower deployment of strategically important infrastructure.

The central research question is:

> **Why has U.S. construction productivity stagnated or declined, and which measurable interventions are most likely to improve the number, quality, and speed of projects delivered per unit of labor and capital?**

The investigation should distinguish among:

* **Labor productivity:** real output per labor hour;
* **Multifactor productivity:** output relative to labor, capital, materials, energy, and services;
* **Physical productivity:** completed units, lane-miles, square feet, or other outputs per worker or dollar;
* **Project delivery performance:** time and cost relative to the original plan;
* **Construction capacity:** the amount of work the industry can complete under current labor, regulatory, and supply constraints.

The study should initially focus on residential construction because it offers relatively consistent local data on permits, units, employment, and housing markets. A second module should examine highways, transit, utilities, and other public infrastructure.

---

## Action Items

### Initial Evidence

Several established findings justify the investigation:

* Construction labor productivity in 2023 was approximately where it had been in 1948 and had followed a sustained downward path since around 1970.
* Construction was the only major industry with negative average productivity growth after 1987 in a recent reexamination of the data. The researchers concluded that plausible measurement error could not explain the industry’s poor performance.
* Residential construction productivity declined most in areas with tighter housing-supply constraints, more urban-core construction, and longer permitting times.
* Recent research argues that restrictive land-use rules reduce project size, encourage smaller construction firms, and weaken incentives to invest in technology or standardized processes.
* The decline cannot be assumed to arise entirely from zoning. Construction is unusually fragmented, site-specific, subcontractor-dependent, exposed to weather and material volatility, and difficult to measure consistently.
* Public infrastructure projects add further risks involving procurement, design changes, litigation, domestic-content requirements, interagency coordination, and delayed obligation of funds. Recent reporting has documented projects delayed by sourcing and waiver requirements, although those examples do not by themselves establish the national cause of low productivity.

### Hypotheses

#### H1 — Restrictive land-use and permitting systems reduce project scale

Larger projects are harder or less predictable to approve, causing developers to pursue smaller projects and limiting economies of scale.

**Evidence to seek:**

* Longer permitting times predict fewer units per worker;
* More approval stages predict smaller projects;
* Productivity improves after permitting or zoning reforms;
* The effect remains after controlling for land values and demand.

#### H2 — Firm fragmentation prevents investment in technology and process improvement

Small contractors may lack sufficient project volume to justify prefabrication, automation, specialized software, workforce training, or research and development.

**Evidence to seek:**

* Larger firms complete more units or square feet per worker;
* Regions dominated by small firms have lower productivity;
* Firm growth is weaker in more restrictive markets;
* Technology adoption explains part of the firm-size productivity gap.

#### H3 — Project complexity and quality improvements are being mistaken for productivity decline

Modern structures may contain better insulation, safety systems, accessibility, seismic protection, communications, and mechanical systems than older buildings.

**Evidence to seek:**

* Productivity decline weakens after adjusting for measurable quality;
* Physical output measures diverge from dollar-based productivity;
* Different building types show substantially different trends.

Current research suggests quality adjustment may explain part, but not all, of the stagnation.

#### H4 — Skilled-labor shortages and workforce instability reduce output

Construction may be constrained by an aging workforce, shortages in particular trades, weak training pipelines, and high employee or subcontractor turnover.

**Evidence to seek:**

* Vacancy rates and wages rise before project delays;
* Productivity is lower where licensed-trade shortages are greater;
* Apprenticeship growth predicts later capacity improvements;
* Labor scarcity has different effects across residential and infrastructure work.

#### H5 — Subcontracting and fragmented project management create coordination losses

Multiple firms, contracts, schedules, and information systems can create rework, delays, and unclear accountability.

**Evidence to seek:**

* Projects with more subcontracting layers have more change orders;
* Rework and design changes predict overruns;
* Integrated project-delivery methods outperform conventional contracting;
* Digital coordination tools reduce delay or rework.

#### H6 — Public procurement reduces competition or rewards low initial bids over total project performance

The lowest bid may not minimize final cost when change orders, delays, litigation, or maintenance are included.

**Evidence to seek:**

* Low-bid projects experience larger percentage overruns;
* Fewer qualified bidders predict higher final costs;
* Design-build or standardized procurement produces faster completion;
* Procurement effects differ after adjusting for project complexity.

#### H7 — Input-price volatility is being confused with productivity decline

Rapid increases in materials, financing, insurance, and equipment costs may reduce real measured output even when physical efficiency is stable.

**Evidence to seek:**

* Physical productivity remains stable during periods of cost-index deterioration;
* Material prices explain cost overruns but not labor-hour overruns;
* Results differ depending on the price deflator used.

FHWA warns that its highway cost index alone cannot separate pure price changes from productivity changes without additional labor and input data.

#### H8 — Local market conditions prevent productive firms from expanding

Highly productive firms may be unable to gain market share because construction is geographically local and firms cannot easily move workers, licenses, suppliers, or equipment.

**Evidence to seek:**

* Weak reallocation from low- to high-productivity firms;
* Persistent local productivity differences;
* Regulatory fragmentation reduces cross-jurisdiction expansion;
* Standardized state rules increase firm scale.

#### H9 — Construction technology exists but diffuses slowly

Building information modeling, modular construction, robotics, drones, automated estimating, and digital project management may improve selected firms without changing the industry average.

**Evidence to seek:**

* Technology adopters outperform similar non-adopters;
* Adoption is concentrated among larger projects and firms;
* Benefits appear only when management practices also change;
* Public agencies using standardized digital systems have fewer change orders.

---

### Research Plan

#### 1. Establish the measurement framework

Construct several productivity measures rather than relying on one index:

[
LaborProductivity_{g,t}
=======================

\frac{RealConstructionOutput_{g,t}}
{ConstructionLaborHours_{g,t}}
]

[
PhysicalProductivity_{g,t}
==========================

\frac{CompletedUnits_{g,t}}
{ConstructionWorkers_{g,t}}
]

[
DeliveryEfficiency_i
====================

\frac{PlannedCost_i}{FinalCost_i}
\times
\frac{PlannedDuration_i}{ActualDuration_i}
]

Where (g) is a state, metropolitan area, county, or permitting jurisdiction and (i) is an individual project.

Calculate separate measures for:

* Single-family housing;
* Multifamily housing;
* Commercial construction;
* Manufacturing facilities;
* Highways;
* Transit;
* Utilities and water infrastructure.

#### 2. Create a geographic panel

Recommended unit:

* Metro-year or county-year for residential construction;
* State-year as a national fallback;
* Project-level records for infrastructure analysis.

Suggested period:

* 1980 through the latest comparable year for long-run analysis;
* 2010 onward for detailed local and project-level analysis.

Core variables:

* Real construction GDP;
* Employment and labor hours;
* Permits and completed units;
* Construction-establishment count;
* Firm-size distribution;
* Wages;
* Material prices;
* Land values;
* Population growth;
* Housing demand;
* Permit duration;
* Regulatory intensity;
* Unionization;
* Occupational shortages;
* Project type and scale.

#### 3. Validate the productivity decline

Compare:

* BEA real output per worker;
* BLS labor and multifactor productivity;
* Housing units per construction worker;
* Square footage per worker where available;
* Highway lane-miles or project quantities per real dollar;
* Project completion time and cost.

If all measures deteriorate, confidence in a real productivity decline increases. If only deflated financial measures deteriorate, measurement assumptions require further investigation.

#### 4. Build local permitting and regulation measures

Collect:

* Median application-to-approval time;
* Number of required approvals;
* Public-hearing requirements;
* Environmental-review duration;
* Number of zoning classifications;
* Minimum lot sizes;
* Density limits;
* Parking requirements;
* Development fees;
* Appeal frequency;
* Permit staffing.

Use local open-data portals where available and construct a standardized permitting dataset for a sample of large metropolitan areas.

#### 5. Estimate the role of firm scale

Measure:

* Firms and establishments by employment size;
* Units or revenue per worker;
* Market share of large builders;
* Entry and exit rates;
* Firm survival;
* Technology adoption proxies.

Test whether regulation affects productivity directly or indirectly through project and firm size:

[
Regulation
\rightarrow
ProjectScale
\rightarrow
FirmScale
\rightarrow
TechnologyInvestment
\rightarrow
Productivity
]

Mediation analysis may help describe this pathway, but it should not be treated as causal without a credible research design.

#### 6. Analyze policy changes as natural experiments

Identify jurisdictions that changed:

* Zoning;
* Permit-processing rules;
* Approval deadlines;
* Accessory-dwelling-unit rules;
* Parking requirements;
* Building-code standardization;
* Electronic permitting;
* By-right development;
* Procurement methods.

Use difference-in-differences or synthetic controls:

[
Y_{g,t}
=======

\alpha_g+\gamma_t+
\beta Reform_{g,t}
+
\delta X_{g,t}
+
\epsilon_{g,t}
]

Possible outcomes:

* Permitting time;
* Project size;
* Units permitted;
* Units completed;
* Labor productivity;
* Construction employment;
* Prices and rents;
* Firm entry.

Check pre-treatment trends and possible spillovers into neighboring jurisdictions.

#### 7. Build a public-project delivery dataset

For highways, transit, water, and federal construction, collect:

* Initial cost;
* Final or current cost;
* Planned completion;
* Actual completion;
* Award method;
* Number of bidders;
* Change orders;
* Contractor;
* Funding sources;
* Environmental-review duration;
* Domestic-content requirements;
* Litigation or redesign events.

Use survival analysis to model time to construction and completion. Use regression or matched comparison to identify predictors of cost escalation.

#### 8. Separate price inflation from inefficiency

Deflate project costs using multiple indexes:

* Construction producer-price indexes;
* Engineering News-Record indexes where licensing permits;
* FHWA National Highway Construction Cost Index;
* Regional wage and material measures.

FHWA’s index is built from state-level winning highway bids and associated quantities, making it useful for identifying changes in highway input prices.

#### 9. Perform robustness checks

Repeat the analysis:

* With and without 2008–2012;
* With and without 2020–2022;
* Using workers versus labor hours;
* Using dollar output versus physical output;
* By project type;
* By urban and nonurban geography;
* By firm size;
* By union status;
* By high- and low-regulation jurisdictions.

#### 10. Produce the final PowerPoint

Recommended presentation:

1. Why construction productivity matters;
2. Fifty years of divergence from the wider economy;
3. Measurement and definitions;
4. Residential construction results;
5. Infrastructure project results;
6. Permitting and regulatory findings;
7. Firm scale and industry fragmentation;
8. Labor and skills;
9. Materials, financing, and price effects;
10. Technology adoption;
11. Which hypotheses survived testing;
12. Policy options and expected measurable outcomes.

The final report should identify which explanations are:

* Strongly supported;
* Partially supported;
* Unsupported;
* Not testable with existing public data.

---

## Resources

### Data, APIs, and Where to Get Data

#### Bureau of Labor Statistics Public Data API

Provides programmatic access to employment, wages, producer prices, occupational employment, labor productivity, and other BLS series.

Use for:

* Construction employment;
* Labor hours;
* Occupational wages;
* Producer-price indexes;
* Labor and multifactor productivity;
* Employment by construction subsector.

BLS defines labor productivity as output relative to labor hours and multifactor productivity as output relative to labor, capital, energy, materials, and services.

#### BEA API

Provides national, industry, state, county, and metropolitan economic statistics. Regional API tables include real GDP by state and county and industry-level contributions.

Use for:

* Construction GDP;
* Real output;
* Compensation;
* Employment;
* State and county economic controls.

#### Census Building Permits Survey

Provides housing units authorized by permit for states, metropolitan areas, counties, and permit-issuing places. Final 2025 annual estimates are available.

Use for:

* Permit volume;
* Units per permit;
* Project scale;
* Geographic comparisons;
* Monthly and annual construction pipelines.

Caution: permit valuation may differ materially from actual construction cost.

#### Census Survey of Construction and New Residential Construction

Provides housing starts, completions, construction characteristics, and time from authorization to completion.

Use for:

* Physical output;
* Construction duration;
* Single-family and multifamily comparisons;
* Permit-to-start and start-to-completion times.

#### Census Economic Census and County Business Patterns

Use for:

* Number of construction firms and establishments;
* Employment by establishment size;
* Revenue;
* Payroll;
* Industry composition;
* Local market fragmentation.

#### Census Business Dynamics Statistics

Use for:

* Firm entry;
* Exit;
* job creation;
* job destruction;
* establishment survival;
* changes in business scale.

#### Quarterly Census of Employment and Wages

Available through BLS downloads and series tools.

Use for:

* County-level construction employment;
* Establishments;
* Wages;
* Construction subsectors;
* Longitudinal regional comparisons.

#### American Community Survey API and PUMS

Use for:

* Construction occupations;
* Worker age and education;
* Commutes;
* migration;
* housing costs;
* local demographic and economic controls.

#### Federal Highway Administration National Highway Construction Cost Index

Provides national highway construction cost indexes and component contributions based on winning contract bids and project quantities.

Use for:

* Highway input-cost inflation;
* Cost decomposition;
* Converting nominal highway spending into constant dollars.

#### FHWA Highway Statistics and Project Data

Use for:

* Highway expenditures;
* lane mileage;
* system condition;
* capital projects;
* state comparisons.

#### USAspending API

Use for:

* Federal construction contracts;
* Grants;
* award amounts;
* recipients;
* agencies;
* obligations;
* modifications.

Potential project types include:

* Federal buildings;
* transportation;
* water systems;
* energy infrastructure;
* military construction.

#### Federal Procurement Data System Records

Available through federal procurement and USAspending resources.

Use for:

* Contract type;
* competition;
* bidders where available;
* modifications;
* contractor;
* award dates;
* final value.

#### Federal Transit Administration and National Transit Database

Use for:

* Capital projects;
* transit-agency characteristics;
* expenditures;
* infrastructure condition;
* service outcomes.

#### EPA and State Revolving Fund Data

Use for:

* Drinking-water and wastewater projects;
* project funding;
* construction status;
* community size;
* compliance outcomes.

#### Local Permitting Portals

Many cities and counties provide APIs or downloadable records containing:

* Application dates;
* approval dates;
* project descriptions;
* valuations;
* addresses;
* contractors;
* inspections;
* status.

Potential pilot jurisdictions should include a mix of:

* High- and low-cost metros;
* Fast- and slow-permitting systems;
* High- and low-growth regions;
* Different state regulatory environments.

#### Optional Regulation Indexes

Potential sources include:

* Wharton Residential Land Use Regulatory Index;
* Local zoning atlases;
* National Zoning Atlas;
* Eviction Lab and housing-policy repositories;
* Municipal code databases;
* State housing reform trackers.

These may require manual processing, licensing review, or careful harmonization and should not be treated as equivalent measures.

### Foundational Research

* Federal Reserve Bank of Richmond, *Five Decades of Decline: U.S. Construction Sector Productivity*.
* NBER, *Why Has Construction Productivity Stagnated? The Role of Land-Use Regulation*.
* Garcia and Molloy, *Reexamining Lackluster Productivity Growth in Construction*.
* BLS research on construction productivity measurement and subcontractor inclusion.

### Central Scientific Caution

The project must not use high prices as a synonym for low productivity. Prices can rise because of better quality, scarce land, material inflation, financing costs, or stronger demand. Productivity concerns how effectively inputs are converted into completed output.

The most defensible investigation will triangulate:

1. Real economic output;
2. Labor hours;
3. Physical construction quantities;
4. Project delivery time;
5. Final versus expected cost;
6. Quality and complexity.

Only when several measures point in the same direction should a place, industry, or policy be classified as having a construction-productivity problem.

### Resources/Instructions
REPLACE THIS TEXT -If there is a website which has documentation that helps with this issue provide the link(s) here.

- If this issue requires access to 311 data, please answer the following questions:
- Do you need a one-time or ongoing dump of the data?
- Do you need subset of data (i.e. certain years) or the entire data set (approx. 4 million rows or 11 GB)?
- If a subset is needed, please define subset characteristics (i.e. date range, etc.)
- Do you need online access via an API or a download of data?

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are specified. Start by narrowing the research plan to the residential-construction module and identifying comparable data sources for the listed productivity measures. Done should include a documented measurement framework, geographic panel, validated comparisons, and evidence-based evaluation of the hypotheses.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
analytics, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.