A relation for "rival answers to one question", so "nobody has compared them" becomes derivable
- Dominant language
- Python
- Stars
- 21
- Forks
- 2
- Avg merge
- 6h 33m
- Merged PRs (30d)
- 91
Description
## The gap
`compared_against` asserts a comparison **was run** — [ADR-011](https://github.com/dmarx/anthology-of-the-sota/blob/main/record/decisions.d/ADR-011.md) in the anthology defines it as "this work evaluates itself against that one". There is no way to say the weaker and much more common thing: *these two answer the same question, and nobody has put them head to head.*
Hit three times in `dmarx/anthology-of-the-sota` in the last week:
- `SOTA-129` (three-stage reasoning recipe) and `SOTA-130` (skip the SFT stage). `SOTA-130`'s `promote_when` asks for exactly the head-to-head that does not exist, so declaring `compared_against` would assert the evidence the practice says is missing.
- `SOTA-063` (use RoPE) and `SOTA-153` (NoPE on the global layers of a hybrid). Rival architectural answers; the comparisons that exist are each lab ablating its own design.
- `SOTA-151` (rescale RoPE to extend context) and the same NoPE practice. Two notes said "alternative answers to the same problem" in prose, in two places, for a week.
In every case the fact was written as a paragraph, in more than one document, because the graph could not hold it.
## Why this is the shape ADR-011 was filed to prevent
ADR-011's own motivating failure, quoted from its context:
> two notes each ended with a sentence saying two rival designs had never been compared. True when written, false three weeks later, corrected in two places — found because one person happened to read both notes the same afternoon.
That is *precisely* this fact, and `compared_against` does not express it. The relation that was built to stop the duplication cannot carry the sentence that was being duplicated.
## Proposal
`alternative_to` — self-converse, like `compared_against`; asserts only that two documents answer the same question, with no claim about evidence.
The payoff is that **the stale sentence becomes derivable**: `alternative_to` present ∧ `compared_against` absent ⇒ "these have not been compared." A view or report can render it, it updates itself the moment someone declares the comparison, and no note has to remember to delete a paragraph.
## Design questions
- **Does it join chain components?** `compared_against` does (see the companion issue), and `alternative_to` would be worse: it is precisely the relation that crosses families. Probably renders as a cross-link on each document without merging their lines.
- **Is it distinct enough from `compared_against`, or is `compared_against` really two relations?** Worth considering the alternative shape: one relation `rivals`, with an optional `compared: true/false`. Rejected on first pass because a boolean beside a reference is the qualifier ADR-011 warns about — but it deserves stating in the decision rather than being silently not chosen.
- **Should the derived absence be a report?** "Rivals nobody has compared" is a genuine research agenda for a record like the anthology, and it is exactly the kind of always-current absence a generated view is good at.
_Generated by [Claude Code](https://claude.ai/code)_
Contributor guide
Research direction
Start by reading ADR-011 and the existing compared_against relation, then inspect the companion issue mentioned for current relation semantics. Resolve whether alternative_to is distinct, how it affects chain components, and whether the derived absence belongs in a generated view or report; record that decision. Done means the chosen behavior is specified and covered by the project's relevant checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100