submitqueue: port internal queue contracts to proto (submitqueue/core/messagequeue)
Nadie ha tomado este issue todavía.
- Lenguaje dominante
- Go
- Estrellas
- 225
- Forks
- 11
- Merge medio
- 3 d 8 h
- PR fusionados (30 d)
- 61
Descripción
Problem
The repo's queue-contract convention (doc/rfc/messagequeue-contract.md) is proto3 payloads serialized as protojson, with a contract package owning both the messages and their topic_keys bindings — api/runway/messagequeue/ is the reference example and stovepipe/core/messagequeue/ follows it. The submitqueue domain has not been ported: there is no submitqueue/core/messagequeue/ package, and 11 of its 12 topics still carry hand-rolled JSON via ToBytes/FromBytes methods on entity types.
Current state per topic:
| Topic key | Payload | Serialization |
|---|---|---|
start |
RequestID |
entity JSON |
cancel |
CancelRequest |
entity JSON |
validate |
RequestID |
entity JSON |
batch |
RequestID |
entity JSON |
score |
BatchID |
entity JSON |
speculate |
BatchID |
entity JSON |
prioritize |
QueueID |
entity JSON |
build |
BatchID |
entity JSON |
buildsignal |
BuildID |
entity JSON |
conclude |
BatchID |
entity JSON |
log |
RequestLog |
entity JSON |
merge |
MergeRequest |
✅ protojson (api/runway/messagequeue — correctly borrowed; runway owns that queue's contract) |
Consequences of the split: no additive-evolution guarantees (protojson's unknown-field discard, UPPER_SNAKE enums, int64-as-string are only enforced on merge), no topic_keys binding or contract test for internal topics, and queue-payload serialization concerns living on domain entities (which are supposed to be pure data).
Work
- Create
submitqueue/core/messagequeue/mirroring the reference layout:proto/sources, committedprotopb/, and the generic protojson glue (Marshal/Unmarshal[T]/TopicKeys) — same asstovepipe/core/messagequeue/. - Define proto messages for each payload shape (request-ID carrier, batch-ID carrier, build-ID carrier, queue-ID carrier, cancel request, request log) and bind each topic key to its message via the
topic_keysoption fromapi/base/messagequeue. - Add the contract test: protojson round-trip per message + every topic key bound to exactly one message.
- Migrate producers and consumers topic by topic (gateway
Land/Cancelpublishes, all orchestrator stage controllers,core/request.PublishLog, and the DLQ controllers/republish tooling — anything touchingmsg.Payload). - Once no queue payload uses them, remove
ToBytes/FromBytesfrom the submitqueue entity types (RequestID,BatchID,BuildID,QueueID,CancelRequest,RequestLog, and the full-entity variants) so entities go back to being pure data. - Bazel
visibilitystays domain-scoped per the internal-contract rule.
Caveats
- Wire-format flip: protojson output (snake_case fields, int64-as-string) differs from the current
encoding/jsonentity output, so in-flight messages from before a cutover won't parse after it. Port topic-by-topic with drained queues (fine at the current pre-prod stage), or give consumers a transitional dual-format read if needed. - The
logtopic crosses gateway ↔ orchestrator but both are submitqueue-domain services, so the internal contract location (submitqueue/core/messagequeue/) is correct — it must not go underapi/. - The message-ID convention is a separate concern tracked in #352 (intent-scoped message IDs); porting payloads to proto neither depends on nor resolves it, but call sites will be touched by both — coordinate to avoid churn.
Related
- #352 — intent-scoped message IDs (same call sites)
- #357 — stovepipe counterpart (retire leftover JSON serializers, contracts for upcoming stages)
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Línea de trabajo
Empieza con doc/rfc/messagequeue-contract.md y compara api/runway/messagequeue/ con stovepipe/core/messagequeue/. Distribuye los doce temas entre gateway Land/Cancel, los stage controllers del orchestrator, core/request.PublishLog y los controladores de DLQ y las herramientas de republish. Se considera terminado cuando submitqueue/core/messagequeue/ contenga los contratos proto, el protopb generado, glue, bindings y pruebas de contrato, todos los usuarios de payload se hayan migrado y se hayan eliminado los serializadores de entidades indicados.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- go
- Área
- distributed-systems
- Tipo de issue
- Refactorización
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Estado de actividad
- Tranquilo
- Claridad
- Bastante claro
- Aptitud para principiantes
- 32/100