Harmonize EC2Tags peer discovery: controller-side node_id + retire DNSEndpointsSource
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Refactor
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- go, grpc, kubernetes
- Domain
- distributed-systems, infrastructure
Research direction
Start with internal/controller/node/peers.go and resolveLabelPeers, then inspect internal/planner/planner.go to compare the current EC2Tags mapping with the Label-to-Static path. Resolve the out-of-cluster node_id strategy with the listed Kubernetes, platform, and Sei-network experts before deciding whether DNSEndpointsSource can be retired in seictl. Done means EC2Tags produces composed entries in Status.ResolvedPeers, preserves prior entries on transient failures, and uses the agreed planner path without changing Spec.Peers.
Written by the indexing model from the issue text.
Description
Problem
PR #369 (merged 2026-05-29) moved node_id resolution for the LabelPeerSource path from seictl's sidecar (:26657/status query) into the controller, producing fully-composed <node_id>@<host>:<port> strings in Status.ResolvedPeers and feeding them to the planner via sidecar.PeerSourceStatic. The EC2TagsPeerSource path is still split: the controller resolves the host list, but the sidecar still queries each peer's :26657/status for node_id at config-render time, via sidecar.PeerSourceDNSEndpoints.
This is the split shape #368 identified as load-bearing. Now mixed semantics inside one controller: Label peers preserve prior entries on transient failure, EC2Tag peers don't; Label peers are resilient to mass-restart, EC2Tag peers re-render fragility every reconcile.
Impact
Affects sei-infra peer discovery — pacific-1 validator nodes peer with sei-infra-managed peers via EC2 tag selectors, and those peer relationships should benefit from the controller-side resilience story #369 shipped for Labels. While DNSEndpointsSource remains live, the EC2Tag side carries the pre-#369 failure modes:
- Sidecar config-render queries
:26657/statusof each peer at task-execution time; a peer mid-restart drops out of the renderedpersistent_peersand the gap persists until the next config-render task fires. - No prior-entry preservation; each render is a fresh resolution against current peer reachability.
- The EC2 tag query and the DNS query are temporally split, allowing membership drift between "what the controller thinks the peer set is" and "what got written to config.toml".
Proposed approach (to refine)
Mirror the #369 pattern for EC2Tags:
- Controller's
reconcilePeerslearns anEC2TagsPeerSourcebranch alongside the existing Label branch. Resolves EC2-tagged instances tohost:port, then calls the per-peer sidecar gRPCGetNodeIDfor each — same per-peer-best-effort semantics (preserve prior on transient sidecar failure, skip new peer with structured log). - Compose
<node_id>@<host>:<port>intoStatus.ResolvedPeers, same wire format as the Label path. - Planner maps the EC2Tags branch to
sidecar.PeerSourceStatic(same as Labels post-#369), retiringsidecar.PeerSourceDNSEndpointsfrom the resolver→sidecar contract. - Once both Label and EC2Tags use the static path, the seictl-side
DNSEndpointsSourcehandler is dead code — remove it from the seictl repo as a follow-up.
Open question for the experts: EC2-tagged peers that aren't in the same K8s cluster (sei-infra-managed) — the controller can't dial a per-peer sidecar gRPC the way it does for in-cluster SeiNodes. Resolution path options:
- (a) Gate the controller-side
GetNodeIDon "is the peer an in-cluster SeiNode"; fall back to leaving DNSEndpointsSource live for out-of-cluster EC2 peers. - (b) Controller queries
:26657/statusdirectly for out-of-cluster peers. - (c) Require sei-infra peers to also publish
node_idvia a discoverable surface (tag, S3, on-chain).
Worth a coral round before implementation.
Out of scope
- Anything that changes the
Spec.Peersuser surface (ec2Tags,static,labelunion stays). - Genesis ceremony peer logic (controller-side genesis assembly is a different code path).
- Drain policy for stale
Status.ResolvedPeersentries (separate concern, deferred until prod signal warrants).
Relevant experts
- kubernetes-specialist — extending
reconcilePeerswith a new branch; reusing the per-peer-best-effort + prior-preserve pattern from #369. - platform-engineer — DNSEndpointsSource retirement on the seictl side; the out-of-cluster identity-resolution question.
- sei-network-specialist — CometBFT
node_idresolution for out-of-cluster (sei-infra) peers; whether:26657/statusdirect-query is acceptable or if identity should come from a more authoritative surface.
References
- #368 — parent tracking issue, broader harmonization story
- #369 — merged 2026-05-29; the Label-flow slice that established the per-peer-best-effort pattern and the composed-string wire format in
Status.ResolvedPeers internal/controller/node/peers.go—resolveLabelPeers; the pattern to mirror for EC2Tagsinternal/planner/planner.go— Label→Static branch; the EC2Tags branch currently maps toPeerSourceDNSEndpointsseictlrepo —DNSEndpointsSourcehandler (to retire once both controller branches use Static)
- Dominant language
- Go
- Stars
- 1
- Forks
- 2
- Avg merge
- 2h 29m
- Merged PRs (30d)
- 56
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from sei-protocol/sei-k8s-controller
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
sei-protocol/sei-k8s-controller#457 · 2 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 3/5 1-2 days Newbie friendliness 65/100
All issues in sei-protocol/sei-k8s-controller
Similar issues
-
optimization optimization:agents-md-curator
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
githubnext/gh-aw-cao#13143 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
blinklabs-io/bursa#904 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
yanet-platform/ipfw-go#129 ·
-
bug confmap/provider/googlesecretmanagerprovider needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
open-telemetry/opentelemetry-collector-contrib#51273 · 2 comments ·
-
bug: AI Gateway client filter lists "Unknown" twice when NULL and literal Unknown clients coexist Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100