submitqueue: port internal queue contracts to proto (submitqueue/core/messagequeue)
Nessuno ha ancora preso questa issue.
- Lingua principale
- Go
- Stelle
- 225
- Fork
- 11
- Merge medio
- 3g 8h
- PR unite (30g)
- 61
Descrizione
Problem
The repo's queue-contract convention (doc/rfc/messagequeue-contract.md) is proto3 payloads serialized as protojson, with a contract package owning both the messages and their topic_keys bindings — api/runway/messagequeue/ is the reference example and stovepipe/core/messagequeue/ follows it. The submitqueue domain has not been ported: there is no submitqueue/core/messagequeue/ package, and 11 of its 12 topics still carry hand-rolled JSON via ToBytes/FromBytes methods on entity types.
Current state per topic:
| Topic key | Payload | Serialization |
|---|---|---|
start |
RequestID |
entity JSON |
cancel |
CancelRequest |
entity JSON |
validate |
RequestID |
entity JSON |
batch |
RequestID |
entity JSON |
score |
BatchID |
entity JSON |
speculate |
BatchID |
entity JSON |
prioritize |
QueueID |
entity JSON |
build |
BatchID |
entity JSON |
buildsignal |
BuildID |
entity JSON |
conclude |
BatchID |
entity JSON |
log |
RequestLog |
entity JSON |
merge |
MergeRequest |
✅ protojson (api/runway/messagequeue — correctly borrowed; runway owns that queue's contract) |
Consequences of the split: no additive-evolution guarantees (protojson's unknown-field discard, UPPER_SNAKE enums, int64-as-string are only enforced on merge), no topic_keys binding or contract test for internal topics, and queue-payload serialization concerns living on domain entities (which are supposed to be pure data).
Work
- Create
submitqueue/core/messagequeue/mirroring the reference layout:proto/sources, committedprotopb/, and the generic protojson glue (Marshal/Unmarshal[T]/TopicKeys) — same asstovepipe/core/messagequeue/. - Define proto messages for each payload shape (request-ID carrier, batch-ID carrier, build-ID carrier, queue-ID carrier, cancel request, request log) and bind each topic key to its message via the
topic_keysoption fromapi/base/messagequeue. - Add the contract test: protojson round-trip per message + every topic key bound to exactly one message.
- Migrate producers and consumers topic by topic (gateway
Land/Cancelpublishes, all orchestrator stage controllers,core/request.PublishLog, and the DLQ controllers/republish tooling — anything touchingmsg.Payload). - Once no queue payload uses them, remove
ToBytes/FromBytesfrom the submitqueue entity types (RequestID,BatchID,BuildID,QueueID,CancelRequest,RequestLog, and the full-entity variants) so entities go back to being pure data. - Bazel
visibilitystays domain-scoped per the internal-contract rule.
Caveats
- Wire-format flip: protojson output (snake_case fields, int64-as-string) differs from the current
encoding/jsonentity output, so in-flight messages from before a cutover won't parse after it. Port topic-by-topic with drained queues (fine at the current pre-prod stage), or give consumers a transitional dual-format read if needed. - The
logtopic crosses gateway ↔ orchestrator but both are submitqueue-domain services, so the internal contract location (submitqueue/core/messagequeue/) is correct — it must not go underapi/. - The message-ID convention is a separate concern tracked in #352 (intent-scoped message IDs); porting payloads to proto neither depends on nor resolves it, but call sites will be touched by both — coordinate to avoid churn.
Related
- #352 — intent-scoped message IDs (same call sites)
- #357 — stovepipe counterpart (retire leftover JSON serializers, contracts for upcoming stages)
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Inizia da doc/rfc/messagequeue-contract.md e confronta api/runway/messagequeue/ con stovepipe/core/messagequeue/. Mappa i dodici argomenti tra gateway Land/Cancel, gli stage controllers dell’orchestrator, core/request.PublishLog e i controller DLQ e gli strumenti di republish. Il lavoro è completo quando submitqueue/core/messagequeue/ contiene i contratti proto, il protopb generato, glue, bindings e contract test, tutti gli utenti del payload sono stati migrati e i serializer delle entità elencati sono stati rimossi.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- go
- Ambito
- distributed-systems
- Tipo di issue
- Refactoring
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Tranquilla
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 32/100