Optimize ANA state transitions using a batch RPC to eliminate high IPC latency at scale
- Dominant language
- Python
- Stars
- 132
- Forks
- 69
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 52
Description
**Problem**
In multi-gateway Ceph NVMe-oF deployments operating at scale (e.g., 100+ subsystems, 8 gateways so 8 ANA groups per gateway ), setting or updating ANA states triggers an excessive number of JSON-RPC requests over the management socket to SPDK.
Currently, ANA states are updated on a per-subsystem, per-ANA-group basis. As a result, changing ANA states across 100 subsystems and 8 ANA groups requires 800 individual JSON-RPC invocations per failover/fencing event.
**Impact & Bottlenecks**
Severe Management Socket Overhead: Executing hundreds/thousands of sequential JSON-RPC calls introduces significant Unix domain socket context switches, JSON serialization/parsing overhead, and round-trip IPC delays.
Failover Timeout Risk: The IPC processing time alone can take 1 to 2+ seconds, consuming a massive portion of the time-sensitive failover window and increasing the risk of host NVMe io_timeout expirations.
**Proposed Solution**
Introduce a Batch SPDK RPC API:
Add a single batch RPC (e.g., nvmf_set_ana_states_all_subsystems) that accepts an array or map of ANA groups and target states in a single JSON-RPC payload.
Optimizations:
Inside the C implementation of the batch handler, check the current state against the requested state before initiating a transition
Iterate through all target subsystems internally in C memory during a single socket invocation
Contributor guide
Assessment
This issue has not been assessed yet.