submitqueue: port internal queue contracts to proto (submitqueue/core/messagequeue)
まだ誰も着手していません。
- 主要言語
- Go
- スター
- 225
- フォーク
- 11
- 平均マージ
- 3日 8時間
- マージ済み PR(30日)
- 61
説明
Problem
The repo's queue-contract convention (doc/rfc/messagequeue-contract.md) is proto3 payloads serialized as protojson, with a contract package owning both the messages and their topic_keys bindings — api/runway/messagequeue/ is the reference example and stovepipe/core/messagequeue/ follows it. The submitqueue domain has not been ported: there is no submitqueue/core/messagequeue/ package, and 11 of its 12 topics still carry hand-rolled JSON via ToBytes/FromBytes methods on entity types.
Current state per topic:
| Topic key | Payload | Serialization |
|---|---|---|
start |
RequestID |
entity JSON |
cancel |
CancelRequest |
entity JSON |
validate |
RequestID |
entity JSON |
batch |
RequestID |
entity JSON |
score |
BatchID |
entity JSON |
speculate |
BatchID |
entity JSON |
prioritize |
QueueID |
entity JSON |
build |
BatchID |
entity JSON |
buildsignal |
BuildID |
entity JSON |
conclude |
BatchID |
entity JSON |
log |
RequestLog |
entity JSON |
merge |
MergeRequest |
✅ protojson (api/runway/messagequeue — correctly borrowed; runway owns that queue's contract) |
Consequences of the split: no additive-evolution guarantees (protojson's unknown-field discard, UPPER_SNAKE enums, int64-as-string are only enforced on merge), no topic_keys binding or contract test for internal topics, and queue-payload serialization concerns living on domain entities (which are supposed to be pure data).
Work
- Create
submitqueue/core/messagequeue/mirroring the reference layout:proto/sources, committedprotopb/, and the generic protojson glue (Marshal/Unmarshal[T]/TopicKeys) — same asstovepipe/core/messagequeue/. - Define proto messages for each payload shape (request-ID carrier, batch-ID carrier, build-ID carrier, queue-ID carrier, cancel request, request log) and bind each topic key to its message via the
topic_keysoption fromapi/base/messagequeue. - Add the contract test: protojson round-trip per message + every topic key bound to exactly one message.
- Migrate producers and consumers topic by topic (gateway
Land/Cancelpublishes, all orchestrator stage controllers,core/request.PublishLog, and the DLQ controllers/republish tooling — anything touchingmsg.Payload). - Once no queue payload uses them, remove
ToBytes/FromBytesfrom the submitqueue entity types (RequestID,BatchID,BuildID,QueueID,CancelRequest,RequestLog, and the full-entity variants) so entities go back to being pure data. - Bazel
visibilitystays domain-scoped per the internal-contract rule.
Caveats
- Wire-format flip: protojson output (snake_case fields, int64-as-string) differs from the current
encoding/jsonentity output, so in-flight messages from before a cutover won't parse after it. Port topic-by-topic with drained queues (fine at the current pre-prod stage), or give consumers a transitional dual-format read if needed. - The
logtopic crosses gateway ↔ orchestrator but both are submitqueue-domain services, so the internal contract location (submitqueue/core/messagequeue/) is correct — it must not go underapi/. - The message-ID convention is a separate concern tracked in #352 (intent-scoped message IDs); porting payloads to proto neither depends on nor resolves it, but call sites will be touched by both — coordinate to avoid churn.
Related
- #352 — intent-scoped message IDs (same call sites)
- #357 — stovepipe counterpart (retire leftover JSON serializers, contracts for upcoming stages)
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
調査の方向性
doc/rfc/messagequeue-contract.md から始め、api/runway/messagequeue/ と stovepipe/core/messagequeue/ を比較します。12個のトピックを gateway Land/Cancel、orchestrator stage controllers、core/request.PublishLog、DLQ controllers および republish tooling に対応付けます。完了とは、submitqueue/core/messagequeue/ に proto contracts、生成された protopb、glue、bindings、contract tests があり、すべての payload users が移行され、一覧にある entity serializers が削除されている状態です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- go
- 領域
- distributed-systems
- issue の種類
- リファクタリング
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 活発さ
- 静か
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 32/100