endolith / endolith/elsim

merrill_1984_table_2 and table_4 don't reproduce the published Merrill (1984) tables

Open
#88 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8
Forks
4
Avg merge
8h 2m
Merged PRs (30d)
2

Description

## Problem
`examples/merrill_1984_table_2.py` and `examples/merrill_1984_table_4.py` produce Condorcet/social-utility efficiencies that deviate from the published Merrill (1984) tables by more than Monte Carlo error.

Measured at 3,000 elections (a single run, so ~1 pp MC noise is included):

**Table 2 (Condorcet efficiencies, 201 voters, 5 candidates)** — max deviations per method:
| Method | max deviation from paper (pp) |
|---|---|
| Hare | 4.9 |
| Approval | 4.5 |
| Plurality | 4.4 |
| Coombs | 3.9 |
| Runoff | 3.8 |
| SU max | 3.3 |
| Borda | 2.0 |

**Table 4 (social-utility efficiencies, 201 voters, 5 candidates)** — max deviations per method:
| Method | max deviation from paper (pp) |
|---|---|
| Plurality | 8.9 |
| Hare | 7.5 |
| Runoff | 6.9 |
| Coombs | 3.1 |

For example, Table 4 Plurality at (disp 1.0, corr 0.5, D=2) is ~73 in the script vs 64 in the paper, and at (disp 0.5, corr 0.5, D=2) is ~4 vs −1.

Note the paper's Table 4 also has negative efficiencies (e.g. Plurality −1, 0), which the script's model never produces.

## Impact
The new example-script tests (PR #87) therefore check these two scripts against their docstring "Typical result" tables rather than the paper values, with a note. Table 1, Table 3, and the Weber scripts do reproduce the papers within ~2 pp and are checked against the papers.

## Possible causes (unverified)
- Tie-breaking or normalization differences (e.g. the paper's "Plurality\*" asterisk in Table 4).
- The random-society vs spatial-model handling of utilities.

## Reference
Merrill, S. (1984). "A Comparison of Efficiency of Multicandidate Electoral Systems." *American Journal of Political Science* 28(1), 23-48.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with examples/merrill_1984_table_2.py and examples/merrill_1984_table_4.py, then run their simulations and compare the outputs with the published Merrill (1984) tables and the docstring tables tested by PR #87. Trace how tie-breaking, normalization, and voter-model utilities are handled; done means the relevant example results reproduce the published values within the stated Monte Carlo variation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.