dmarx / dmarx/luria

A relation for "rival answers to one question", so "nobody has compared them" becomes derivable

Open
#208 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
21
Forks
2
Avg merge
6h 33m
Merged PRs (30d)
91

Description

## The gap

`compared_against` asserts a comparison **was run** — [ADR-011](https://github.com/dmarx/anthology-of-the-sota/blob/main/record/decisions.d/ADR-011.md) in the anthology defines it as "this work evaluates itself against that one". There is no way to say the weaker and much more common thing: *these two answer the same question, and nobody has put them head to head.*

Hit three times in `dmarx/anthology-of-the-sota` in the last week:

- `SOTA-129` (three-stage reasoning recipe) and `SOTA-130` (skip the SFT stage). `SOTA-130`'s `promote_when` asks for exactly the head-to-head that does not exist, so declaring `compared_against` would assert the evidence the practice says is missing.
- `SOTA-063` (use RoPE) and `SOTA-153` (NoPE on the global layers of a hybrid). Rival architectural answers; the comparisons that exist are each lab ablating its own design.
- `SOTA-151` (rescale RoPE to extend context) and the same NoPE practice. Two notes said "alternative answers to the same problem" in prose, in two places, for a week.

In every case the fact was written as a paragraph, in more than one document, because the graph could not hold it.

## Why this is the shape ADR-011 was filed to prevent

ADR-011's own motivating failure, quoted from its context:

> two notes each ended with a sentence saying two rival designs had never been compared. True when written, false three weeks later, corrected in two places — found because one person happened to read both notes the same afternoon.

That is *precisely* this fact, and `compared_against` does not express it. The relation that was built to stop the duplication cannot carry the sentence that was being duplicated.

## Proposal

`alternative_to` — self-converse, like `compared_against`; asserts only that two documents answer the same question, with no claim about evidence.

The payoff is that **the stale sentence becomes derivable**: `alternative_to` present ∧ `compared_against` absent ⇒ "these have not been compared." A view or report can render it, it updates itself the moment someone declares the comparison, and no note has to remember to delete a paragraph.

## Design questions

- **Does it join chain components?** `compared_against` does (see the companion issue), and `alternative_to` would be worse: it is precisely the relation that crosses families. Probably renders as a cross-link on each document without merging their lines.
- **Is it distinct enough from `compared_against`, or is `compared_against` really two relations?** Worth considering the alternative shape: one relation `rivals`, with an optional `compared: true/false`. Rejected on first pass because a boolean beside a reference is the qualifier ADR-011 warns about — but it deserves stating in the decision rather than being silently not chosen.
- **Should the derived absence be a report?** "Rivals nobody has compared" is a genuine research agenda for a record like the anthology, and it is exactly the kind of always-current absence a generated view is good at.

_Generated by [Claude Code](https://claude.ai/code)_

Contributor guide

Open the contributing guide

Research direction

Start by reading ADR-011 and the existing compared_against relation, then inspect the companion issue mentioned for current relation semantics. Resolve whether alternative_to is distinct, how it affects chain components, and whether the derived absence belongs in a generated view or report; record that decision. Done means the chosen behavior is specified and covered by the project's relevant checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.