chore(security): network + opsec hardening tracking for prod-readiness

This is a broad hardening tracker spanning config/manager/manager.yaml, config/network-policy/allow-metrics-traffic.yaml, internal/controller/nodedeployment/networking.go, and Terraform modules under validators, state-syncer, and related deployments. Start by selecting one suggested first move and reading the referenced design or configuration files; completion should be defined by a focused PR with the relevant validation or deployment evidence.

Context

Audit conducted 2026-04-21 by Tide kubernetes-specialist + platform-engineer across sei-k8s-controller (operator-world design) and sei-infra (legacy EC2 world still running prod). Captures network/opsec findings that must be addressed — or explicitly accepted with compensating controls — before onboarding customer-facing validators or public RPC under the operator.

The operator-side design is sound (Istio Gateway, internal ClusterIP services, designed mTLS, no node-level LoadBalancer). The concerns below are a mix of (a) incomplete operator plumbing, (b) legacy-world posture that will poison a migration if carried forward, and (c) platform-repo dependencies this repo can't see.

Ranked concerns

HIGH — address before any external-facing prod workload under the operator

1. Dev S3 buckets hardcoded as operator defaults — config/manager/manager.yaml:65,69,75 sets dev-sei-snapshots, dev-sei-k8s-genesis-artifacts, dev-sei-shadow-results. Applying manager.yaml to a prod cluster without per-env overrides writes prod node data to dev buckets. Strip defaults or swap for // required, no default.

2. No NetworkPolicy covers SeiNode pods — only config/network-policy/allow-metrics-traffic.yaml ships, scoped to the operator's own metrics port. No default-deny at the node-pod namespace. Combined with (7) below, a compromised pod has broad lateral reach. Add a default-deny + explicit allows: Istio mesh, headless-Service p2p peer DNS, S3/ECR egress endpoints, metrics scrape.

3. AuthorizationPolicy referenced in two design docs but not implemented — docs/design-networking-monitoring.md:41,327-334 and .tide/designs/internal-networking-use-cases.md:18. internal/controller/nodedeployment/controller.go:49-57 RBAC has no Istio groups. At minimum, seid pods must refuse RPC traffic that didn't arrive through the Istio sidecar.

4. IRSA wiring not checked in — config/rbac/service_account.yaml:1-9 has no eks.amazonaws.com/role-arn annotation. seid-node SA referenced in manager.yaml:44-45 has no definition. Either (a) the controller and workload SAs fall back to node IAM (too broad), or (b) IRSA lives in sei-protocol/platform and this repo needs a documented cross-link. Decide and encode.

5. Legacy validator SG is 0.0.0.0/0 on all ports — validators/deploy/terraform/modules/ec2/main.tf:11-24 with an explicit "# needs to be tightened later" comment. No LB, associate_public_ip_address=true, all seid ports reachable from anywhere. Any operator-managed validator that inherits this posture via EC2-tag peer discovery or a LoadBalancer Service on 26656/26657 without an AuthorizationPolicy continues the exposure.

MEDIUM — address on a normal hardening cadence

6. Plaintext RPC on state-syncer ALB — state-syncer/deploy/terraform/modules/alb/main.tf:82-102 serves HTTP:26657 and HTTP:1317 with no TLS and no WAF. Public NLB on TCP:26656. If any client has cached these URLs, traffic is observable in transit.

7. Default VPC across legacy components — aws_default_vpc in validators/standalone-rpc/state-syncer EC2 modules; default=true VPC in webapp/state-syncer ALB modules. No private subnets, no NAT, no tier separation, no S3/DynamoDB gateway endpoints for any node (only indexer has one, indexer/.../dynamodb/main.tf:417). Any instance in the account's default VPC can reach validator instances on any port.

8. Istio public-DNS cert strategy designed but unshipped — .tide/designs/public-dns-platform-sei-io.md:30-49 notes the current wildcard does not cover the new 2-level subdomains the operator will create. HTTPRoutes under SEI_GATEWAY_DOMAIN=prod.platform.sei.io (set in manager.yaml:83) will fail TLS until the per-namespace cert plumbing lands.

9. WAF only on snapshotter + webapp ALBs, with permissive rate limits — snapshotter/deploy/terraform/modules/waf/main.tf:66-69 caps at 300 req / 5 min / IP. No WAF on state-syncer ALB/NLB or direct validator IPs. Rate limit is IP-keyed → trivially bypassed from multiple sources and painful for legitimate shared-NAT clients.

10. No AWS Shield / DDoS posture documented — absent from both repos. Decide whether Shield Standard (free, on by default) is sufficient or whether public-facing validators/RPC need Advanced.

LOW — nice-to-haves

11. Istio sidecar injection on SeiNode namespaces — no istio-injection label in any sample; docs/design-rpc-migration-istio.md:108 lists it as a prerequisite but doesn't enforce. Operator could fail a Validate(node) check if namespace labels are missing, or the platform layer could set it uniformly.

12. P2P double-encryption consideration — tendermint's own authenticated encryption on 26656 runs inside the Istio mTLS envelope for pods with sidecars injected. Not insecure, but wastes CPU on an already-hot path. Decide whether validators/archives opt out of injection on p2p ports.

13. VPC endpoints for S3/ECR — from the EKS VPC, absence costs money on every snapshot upload and widens the egress surface. Cheap to add if not already present (depends on platform-repo).

Data we can't see from this repo

These belong to sei-protocol/platform or equivalent infra-repo. Need to be verified out-of-band:

• EKS cluster Terraform — VPC/subnet layout, NAT, IGW, Karpenter NodePool definitions (operator refs SEI_NODEPOOL_NAME=sei-node, SEI_NODEPOOL_ARCHIVE=sei-archive, SEI_TOLERATION_KEY=sei.io/workload in manager.yaml:38-45 without defining them)
• IRSA role ARNs for controller-manager and seid-node service accounts
• Istio install (version, mesh-wide mTLS mode, global AuthorizationPolicy)
• cert-manager ClusterIssuer + external-dns + Route53 zone wiring for prod.platform.sei.io
• Actual live security group rules (Terraform state may have drifted from source)
• Org-level SCPs, IAM boundaries, AWS Shield subscription
• KMS / consensus-key management for validators (no key module in validators/deploy/terraform/)

Suggested first moves

• Land a "no default bucket" PR — strip the dev-sei-* values from manager.yaml; manager startup fails loudly if any required bucket env var is unset
• Add a default-deny NetworkPolicy for SeiNode namespaces with explicit allows (separate PR per namespace pattern)
• Implement the AuthorizationPolicy reconciler for SeiNodeDeployment, scoped to sei.io/nodedeployment selector
• Document (in-repo) the IRSA roles the operator expects to exist, with a .tide/designs/irsa-dependencies.md or similar
• Cross-link from docs/production-deployment-analysis.md to the platform-repo Terraform for the EKS cluster, so this repo's readers can find the rest of the picture

References

• Audit findings memos (Tide council, 2026-04-21) — available in session context if this issue is picked up soon
• docs/design-rpc-migration-istio.md — current Istio migration design
• docs/design-networking-monitoring.md — networking design with the aspirational AuthorizationPolicy
• .tide/designs/public-dns-platform-sei-io.md — public DNS + cert strategy
• docs/production-deployment-analysis.md — prior analysis of migration gaps
• config/manager/manager.yaml — where the env-var defaults live
• internal/controller/nodedeployment/networking.go, internal_service.go — what the operator creates today