Changing locality of processes
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
We managed to lose data on one of our test clusters by executing the following steps:
1. We had 5 machines running FDB 6.2.1 - running 12 procs each
1. Additionally we had one machine that ran an older version (the processes there couldn't join the cluster)
1. The cluster was configured with `configure new triple ssd` and afterwards we populated some generated test data (TPC-C)
1. We changed the config with `configure ssd three_data_hall` which caused the cluster to go down (this is expected as we didn't set the locality_data_hall)
1. We changed the config on all machines and set `locality_data_hall` so that two machines would always share the same value.
1. We restarted fdb on all machines through ansible.
After that, all data was gone.
This being said: I am not 100% convinced that this is the full story and I am not sure this was an fdb bug. However, it seems that changing localities on processes isn't super well tested in simulation. Because of that I think we should mention in the documentation that changing localities on processes is not supported.
Contributor guide
Research direction
No file or test is identified in the issue. Review the documentation covering `configure`, `locality_data_hall`, and process localities, then verify the supported behavior with maintainers or existing tests; done means clearly documenting whether changing process localities is supported and any required procedure or warning.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases, distributed-systems, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100