Gluster FS possibly causing high latency on high workload
- Dominant language
- Shell
- Stars
- 184
- Forks
- 186
- PR merge metrics
- No merged PRs in 30d
Description
We've been experiencing high latency with Gluster FS on the [time-gated exam scenario](https://github.com/Azure/Moodle/blob/hs-loadtest-time-gated-exam/loadtest/time-gated-exam-test.jmx). We don't know exactly if Gluster is the issue and why in that case, but replacing Gluster with NFS makes the high latency issue go away, so naturally suspecting Gluster.
We'll need to dive deeper on how Gluster works and why it might cause perf bottleneck like we've been experiencing. In the meantime, we might need to provide an alternative, like HA NFS that's described in places like the following:
- https://www.howtoforge.com/high_availability_nfs_drbd_heartbeat
- https://help.ubuntu.com/community/HighlyAvailableNFS
These are pretty old, and still the top results I get from my related web search, making me think that this is not a widely used solution, but we should evaluate the option. Any PR of an ARM template deploying an HA NFS 2-VM cluster would be highly appreciated.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with loadtest/time-gated-exam-test.jmx and compare the high-latency behavior when using Gluster FS versus NFS. Investigate whether Gluster is the bottleneck, then evaluate whether an ARM template for an HA NFS 2-VM cluster is an appropriate alternative; the work is done when the cause or alternative is documented with supporting results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cloud, infrastructure, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100