PolicyEngine / PolicyEngine/firm-microsim-paper

Address publication audit: VAT accounting, sensitivity, diagnostics, and reproducibility

Open
#61 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
0
Avg merge
5h 5m
Merged PRs (30d)
10

Description

Summary

Prepare the paper and replication package for submission by clarifying the VAT policy experiments, testing the assumptions that drive local fiscal estimates, fixing the bunching diagnostics, and producing one consistent publication release.

The repository already provides an installable package, locked dependencies, archived inputs, and tests. The manuscript explicitly acknowledges synthetic-data non-identification and several modeling limits. This issue tracks the remaining work needed to make the results and their interpretation reviewable.

Suggested owner: Vahid. Sections can be addressed through separate linked PRs.

Audit baseline: commit e6d5dd6, reviewed on 15 September 2026. All findings and example values below refer to that snapshot. Check subsequent changes before implementing fixes.

1. Specify VAT accounting and the reform definitions

The reduced-rate schedules multiply existing net liability by the ratio of the new rate to 20%. This prices relief against net remittance. An output-VAT rate cut with unchanged credits for tax paid on inputs is a different policy.

For a firm with £100,000 of sales and £60,000 of inputs, both measured excluding VAT:

Scenario Net VAT liability
Baseline: sales and inputs taxed at 20% £8,000
Implemented 10% net-remittance scenario £4,000
Sales taxed at 10%, with inputs still taxed at 20% and fully reclaimable −£2,000
  • State whether each reduced-rate/taper scenario changes invoice rates or provides relief against net remittance; specify input credits and downstream business customers' credits.
  • Define customer expenditure, VAT-exclusive sales, taxable turnover, input expenditure, and producer incidence consistently in the revenue and profit equations. Explain input-VAT treatment before and after registration.
  • Add worked examples and regression tests for the chosen policy definition, and recalculate affected results if that definition changes the implementation.

References: rate schedules, profit model, HMRC VAT-return rules.

2. Report structural sensitivity of the main costings

The £85–100k reform exposure lies inside a calibrated £85–150k liability band. Fine-band firm counts do not identify fine-band liabilities. The paper's smooth-profile alternative changes the threshold-rise cost from £742m to £616m, approximately 17%.

  • Replace the claim that £85–100k liabilities are directly calibrated with the actual broad-band constraint and within-band allocation assumptions.
  • Add a finite, documented sensitivity set for the headline reforms covering within-band liability allocation, registration/retention, and taxable-turnover/input-VAT assumptions. Justify the alternatives and report their calibration fit alongside fiscal effects.
  • Distinguish sensitivity to economic assumptions from random-seed variation. Include the existing £742m/£616m comparison in the main uncertainty discussion.
  • Label threshold-cut estimates by population scope and explain the omitted unregistered businesses outside the ONS frame. Support or remove claims that timing, taxable-share, or upper-tail assumptions have negligible local effects.

References: sub-band costing, turnover and registration assumptions.

3. Align behavioral and distortion claims with the model

The £21,250 interval is a guaranteed dominated region under the maintained profit model. Removing this interval does not establish that all location incentives or welfare losses disappear.

The audit reproduced a counterexample in formulation A: with the constant-50% taper, VAT rate 20%, threshold £85,000, input share 0.6, elasticity 0.17, and ability £178,350.419, two equally profitable turnover choices occur at £135,659.194 and £146,943.246. They straddle the £141,666.667 taper top, despite net revenue being continuous and monotone.

  • Document this counterexample or resolve it mathematically; reserve exactness claims for the result actually proved. Avoid interpreting interval width as an exact welfare or misallocation measure.
  • Consistently describe the behavioral results as sensitivity under the region-confined response rule. Explain that the threshold-rise revenue offset is zero because released firms cannot re-cross the threshold in this exercise.
  • Distinguish local first-order-condition anchoring from a globally optimal baseline, and make the exclusion of taper behavioral responses visible in summaries.

Broader global-choice or welfare modeling can remain future work if the manuscript's claims stay within the demonstrated scope.

References: taper conclusions, response restriction.

4. Repair bunching diagnostics and seed-cache provenance

These are demonstrated implementation issues or code-traced failure paths; their effects on the published headline estimates still require regeneration.

  • Separate observations on either side of the threshold. The histogram currently places £85k at a bin center. Moving 100 weighted firms from £85,100 to £84,900 leaves the histogram unchanged. Align/split bins at the threshold, document endpoint conventions, and add a cross-threshold relocation regression.
  • Fix closing-bin integration and zero excess. With threshold 85, excess mass 10, and first-bin deficit 100, the search returns y_R=84.1, Delta_R=100, uncensored. With zero excess it returns y_R=84, Delta_R=100. Require an above-threshold closing location, correct partial-bin integration, and zero matched deficit when excess is zero; document the location convention for the latter case.
  • Validate reused populations. The seed script accepts any canonical CSV as the default-seed build, including a prior fast or different-seed run. Validate seed, full/fast configuration, input hashes, and code version, or regenerate each requested population.
  • Regenerate the affected bunching, recovery, placebo, and sensitivity outputs and report changes to published values.

References: bin construction, missing-mass search, seed-cache read.

5. Correct source, population, and revenue-base descriptions

  • Distinguish the ONS frame, registered firms within that frame, appended out-of-frame traders, and all registered traders in a small universe table. Make denominators explicit.
  • Qualify the 43% voluntary-registration statistic by its historical corporation sample and distinguish that firm share from a retention probability or liability share for newly released firms.
  • Describe the modeled above-threshold net-liability subtotal using its declared-liability targets. Reconcile exclusions and calibration residuals; remove the VTTL interpretation unless a full-compliance theoretical-liability derivation supports it.
  • Independently reconcile the OBR Chart C fine-band targets and interpolation with the original source, recording table identifiers and checksums. Flag any remaining unavailable source evidence explicitly.
  • Source or derive the counterfactual threshold path 85/85/87/89/92 thousand and the phased-response attribution. If these are modeling reconstructions, label them and assess sensitivity.

References: data universes, revenue-base explanation, HMRC VTTL methodology.

6. Tighten the manuscript

  • Consolidate the repeated tasks, findings, and contributions into one introduction statement; emphasize the open accounting/microsimulation contribution.
  • Present one common-baseline policy table with explicit years, scope, assumptions, and sensitivity. Separate diagnostics from policy comparisons.
  • Use calibration-fit terminology and show important residuals. Qualify unsupported bounds and describe the £5–50m truncated log-uniform assumption precisely.
  • Move detailed bug/migration history into supplementary records, preserving a correction trail for previously public results.
  • Reconcile notation, stale structural-model docstrings, and small table/count discrepancies throughout the active manuscript and documentation.

7. Produce and verify one publication release

On the audit date, the public website reported +£1.46bn for the £90k→£70k scenario and −£876m for £85k→£100k; the reviewed manuscript/results reported +£186.8m and −£742m. The checked PDF also differed from current source in several table cells and rendered dollar signs in some mathematical currency expressions.

  • Run the documented release path, including uv sync --extra dev --locked, make check, make reproduce-heavy, and the PDF/HTML builds. Record actual outcomes and explain material changes to previous results.
  • Generate numeric text/tables, figures, PDF, HTML, and README summaries from one results bundle. Extend consistency checks beyond the current selected LaTeX substring checks.
  • Verify sterling rendering and inspect the final tables/figures visually.
  • Publish synchronized artifacts with a visible release/commit identifier; archive a tagged release with code/input/configuration hashes, seeds, outputs, and a DOI.

Closure criteria

Close after the workstreams are resolved through linked changes or evidence-backed decisions, the full numerical reproduction is documented, and the published artifacts agree. Record any remaining assumptions and source gaps explicitly beside the claims they limit. Modeling concerns may be resolved through a defensible definition or narrower claim; demonstrated code defects require fixes and regression coverage.

The audit verified passing test/Quarto CI at the reviewed commit, 47 selected source-to-artifact claims, four dependency-light tests, ONS extraction checks, and focused numerical diagnostics. It did not independently rerun the full generation/calibration pipeline. Original OBR fine-band provenance and the exact official counterfactual threshold path remained unverified. Those limits should remain visible until the checks above resolve them.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by checking changes after audit commit e6d5dd6 and reading the referenced entry points in src/firm_microsim/bunching/model.py, scripts/seed_sensitivity.py, and paper/Sections/. Run uv sync --extra dev --locked, make check, and the documented reproduction and build commands. Done means the audit workstreams are resolved or documented, regression coverage and provenance checks exist, and one synchronized release reproduces the published artifacts.

Written by the indexing model from the issue text.

Assessment

Tech stack
latex, python
Domain
build-system, data, documentation, release, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.