influxdata / influxdata/influxdb
[2.X] Implement Scalable Multi-Tenant High-Availability Pattern for InfluxDB 2.x OSS
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
Use case:
Currently, InfluxDB 2.x OSS users facing high-growth or multi-tenant SaaS requirements hit a "scale wall." Because the OSS version is natively single-node, users are forced to choose between expensive Enterprise licenses or manual, fragile sharding logic. This proposal provides a standardized Kubernetes blueprint to achieve horizontal ingest scaling and storage redundancy using only OSS components.
Proposal:
Introduce a "Shared-Nothing" Cluster pattern for Kubernetes deployments. This feature-set leverages Telegraf as a dynamic path-based router and the Edge Data Replication API to decouple compute-heavy ingestion from long-term storage. It allows InfluxDB OSS to scale horizontally at the ingest tier while maintaining a mirrored, high-availability storage "Hub" for multiple tenants.
Current behaviour:
The standard deployment is a single StatefulSet.
-
No Redundancy: A pod crash or node maintenance results in total downtime.
-
No Multi-Tenancy: All tenants share a single compute boundary; a "noisy neighbor" in one organization can crash the pod for all others.
-
Vertical Scaling Only: Users must increase CPU/RAM on a single pod indefinitely.
Desired behaviour:
A distributed architecture where:
-
Ingress: A Telegraf gateway captures tenant names from the URL path (e.g., /tenant-a/v2/write) and routes data to specific buckets.
-
Ingest Tier (Auto-Scaling): A stateless-like tier of InfluxDB pods that handles the "heat" of parsing and compression, governed by a Horizontal Pod Autoscaler (HPA).
-
Edge Replication: Automated background streaming from Ingest pods to a redundant Hub Tier.
-
Storage Hubs: A mirrored pair (hub-0, hub-1) acting as the "Source of Truth," allowing for zero-downtime upgrades and persistent data safety.
Alternatives considered:
-
InfluxDB 3.0 (IOx): While powerful, the transition path for existing v2.x Flux users is non-trivial, and the resource requirements for the 3.0 engine are significantly higher for small-to-medium SaaS startups.
-
Manual Sidecars: Users currently script their own replication logic, which is error-prone and lacks the standardized monitoring (queue tracking, lag metrics) provided by a formal Helm/Kubernetes pattern.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by mapping the existing single StatefulSet deployment and the proposed Telegraf, HPA, Edge Data Replication, and mirrored hub components; done would require an agreed Kubernetes/Helm blueprint covering tenant routing, replication, scaling, redundancy, and monitoring.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- databases, distributed-systems, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100