basecamp / basecamp/basecamp-sdk

Event feed conformance: the tier-2 driver matches arrival-strict actions retroactively, so it can pass an implementation that violates step ordering

Open
#789 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
49
Forks
12
Avg merge
20h 47m
Merged PRs (30d)
89

Description

Raised by Copilot as two **suppressed** comments on #705 (`scenario_harness_test.go`, the
`Save` recorder and `readPeer`), against head `742fd1a9f`. Suppressed comments never become
threads, so this is filed to keep them from being lost.

### The contract

`conformance/event-feed/schema.json` declares the strictness classes normatively, for all
six SDKs:

> Strict-matched actions (mint, connect, poll, outbound frame, client close, checkpoint
> save) must each match their expect step under the per-action-class rules in this family's
> README … **saves and outbound frames are arrival-strict**; mint/poll seam calls are parked
> and matched in order.

### What the Go driver actually enforces

Both arrival-strict classes are **recorded for later matching** rather than checked on
arrival.

`Save` appends to a list with an ordering witness:

```go
h.saves = append(h.saves, saveRecord{position: position, deliveredAt: len(h.delivered)})
```

and `readPeer` queues outbound frames and client closes until `expectSubscribe` /
`expectClientClose` consumes them, with `finally` catching only what remains unmatched.

So an action that arrives during some *other* step is accepted retroactively when its expect
step is eventually reached. A driver that is supposed to reject "the save happened at the
wrong moment" instead accepts "the save happened, and later a step asked for one".

### The part that is already covered, and why it is not the whole thing

`TestScenarioDriverEnforcesDeliveryBeforeCheckpoint` does pin arrival-strictness for the
save-versus-**delivery** ordering, using the `deliveredAt` witness, and it is a real
mutation-proven probe. Copilot's comment anticipates exactly this and is right that it does
not generalize: `deliveredAt` orders a save against *deliveries only*. A save that lands
during an `expectState`, `expectTimers`, `expectBuffered` or `expectSignal` step carries an
unchanged `deliveredAt` and is indistinguishable from one that landed at its proper step. The
outbound-frame class has no equivalent witness at all.

So the gap is narrower than "no strictness" and wider than "already covered": strictness
holds along one axis and is absent along the others.

### Why it matters more than a normal test weakness

This is the tier-2 driver — the thing that decides whether an implementation conforms. A
weakness here does not fail a test; it **passes an implementation whose action ordering
violates the fixture**, in every SDK whose runner shares the shape. It is also self-masking:
the fixtures currently in the tree pass either way, so nothing signals the difference until
an implementation is wrong in the specific way the driver cannot see.

### Shape of the fix

Track the driver's active step in the harness and have the strict-matched recorders consult
it: record an immediate violation unless the active step is the matching expect. That is the
literal reading of the schema sentence and it needs no fixture changes.

Two things to settle while doing it: whether the same treatment is owed to the other
strict-matched classes (mint/connect/poll are documented as *parked*, so they are
deliberately different), and whether the other SDKs' runners share the shape — the schema
sentence is normative for all six, so if the Go driver is loose here it is evidence about
the family, not just about Go.

Related: #602 and #736 covered the conformance case census; this is about the driver's
enforcement strength rather than case coverage.

Contributor guide

Open the contributing guide

Research direction

Start in scenario_harness_test.go, reading Save, readPeer, and TestScenarioDriverEnforcesDeliveryBeforeCheckpoint alongside conformance/event-feed/schema.json and the family README. Trace how the active step is represented and compare the other SDK runners if needed. Done means arrival-strict saves and outbound frames reject actions received outside their matching expect step, while parked actions retain their documented behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.