Phase 2: MinIO/S3-Compatible Artifact Storage to Replace NFS Bottleneck

Problem

The single NFS server provisioner pod with 5.1Ti Cinder backing is a SPOF with ceiling at ~1k-2k workers. At 10k workers writing simulation results concurrently:

1. NFS server pod CPU saturates (kernel NFS daemon)
2. Backing Cinder volume hits IOPS quota
3. Network bandwidth on the NFS pod node saturates
4. Kernel NFS client timeouts on worker pods cause job failures

What Phase 1 Did

• Nothing — NFS architecture unchanged. emptyDir: {} in worker pods means results go to NFS via application logic.

What Should Be Done (This Issue)

1. MinIO StatefulSet template: Create a 4-pod MinIO deployment with distributed mode (erasure coding) for high availability
2. S3-compatible endpoint: Expose via ClusterIP service + optional NodePort for external access
3. Mount simlink or app config: Update the app to use S3-compatible storage instead of NFS paths for simulation artifacts
4. NFS→MinIO migration path: Document how to migrate existing artifacts
5. PVC sizing: 4 × independent PVCs for MinIO data, with recommended size per workload
6. TLS support: Optional cert-manager integration for MinIO API endpoint
7. Fallback option: Gate behind a storage.backend: nfs|s3 value toggle

Design Notes

• MinIO distributed mode requires 4+ pods with independent PVCs (erasure coding = n/2 failure tolerance)
• Must update the OpenStudio Server application to use aws-sdk-s3 or MinIO SDK instead of filesystem paths
• NFS remains default to maintain backward compat; S3 is opt-in
• Worker pods write results via S3 API, not NFS mount
• Chart should bundle MinIO as a subchart or separate template set
• Consider existing nfs_pvc → artifact_storage semantic rename

References

• .agents/results/result-tf-infra.md — B3: NFS Shared Storage (CRITICAL)
• openstudio-server/values.yaml — nfs_pvc.*, nfs-server-provisioner.*
• openstudio-server/templates/worker/worker-deploy.yaml — NFS mount on workers

Acceptance Criteria

• MinIO StatefulSet renders with correct erasure coding config
• Worker pods can write simulation results via S3 API
• Application recognizes storage.backend: s3 config
• Existing NFS artifacts can be migrated
• MinIO console accessible for admin
• All existing tests pass with NFS backend unchanged
• Document stress test: 10k concurrent S3 PUT operations

Start with openstudio-server/values.yaml and openstudio-server/templates/worker/worker-deploy.yaml, then read .agents/results/result-tf-infra.md for the existing NFS design. Map the requested storage toggle, MinIO templates, worker configuration, migration documentation, and TLS or external-access options before changing anything. Done means the listed acceptance checks pass while the NFS backend remains unchanged.