Support running on non-empty database
- Dominant language
- Go
- Stars
- 52.3k
- Forks
- 10.5k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 42
Description
### What would you like to be added?
This task is a little harder as it requires diving into robustness linearization model, interested contributor beware!
Currently we are validating that database is empty before running tests https://github.com/etcd-io/etcd/blob/49767381e803cc45c308fabea3c636ebb20f1d2f/tests/antithesis/test-template/robustness/main.go#L96-L99.
This is done by reading revision of etcd before any traffic is sent and checking if revision is equal `1` https://github.com/etcd-io/etcd/blob/49767381e803cc45c308fabea3c636ebb20f1d2f/tests/robustness/traffic/traffic.go#L207-L227. Revision in etcd is a global logical clock for operations that start at 1. This ensures that there were no transactions executed on etcd before so etcd is empty.
This is also checked as first step of validation https://github.com/etcd-io/etcd/blob/49767381e803cc45c308fabea3c636ebb20f1d2f/tests/robustness/validate/validate.go#L113-L127. This validation is because this is an implicit assumption of the linearization code, that without this assertion would return hard to discern error.
We can try to improve the linearization to allow non empty code. So why linearization doesn't support non-empty database? Because it needs to a complete view of database, so it can simulate it. Assuming it's empty was just easier to implement. https://github.com/etcd-io/etcd/blob/49767381e803cc45c308fabea3c636ebb20f1d2f/tests/robustness/model/deterministic.go#L99-L108
To improve it we could just replace the Read of revision 1 at the beginning with operation to download the state of database and change the model to accept this state as initial one.
Steps;
1. We still need to make a special request at before all other requests. But now instead of checking `revision == 1`, we just need to read the database state. Changes in https://github.com/etcd-io/etcd/blob/49767381e803cc45c308fabea3c636ebb20f1d2f/tests/antithesis/test-template/robustness/main.go#L96-L99.
2. In validation we still need to confirm if the special Read request is present. However the conditions are different, we need to make sure it precedes all other requests that reads the whole database contents (or at least the prefix of all keys used in the robustness test). Changes in https://github.com/etcd-io/etcd/blob/49767381e803cc45c308fabea3c636ebb20f1d2f/tests/robustness/validate/validate.go#L113-L127.
3. We would want to add a way in [EtcdState](https://github.com/etcd-io/etcd/blob/49767381e803cc45c308fabea3c636ebb20f1d2f/tests/robustness/model/deterministic.go#L69-L75) to represent a state that etcd is not initialized, during which the [Step](https://github.com/etcd-io/etcd/blob/49767381e803cc45c308fabea3c636ebb20f1d2f/tests/robustness/model/deterministic.go#L111) method will only allow a read requests. We could hardcode the same request as in step 2.
### Why is this needed?
Allow multiple runs of antithesis locally.
/cc @henrybear327 @nwnt
Contributor guide
Research direction
Start with tests/antithesis/test-template/robustness/main.go and tests/robustness/validate/validate.go to trace the initial database read and its validation. Then study EtcdState and Step in tests/robustness/model/deterministic.go to understand how the initial state is represented. Done means robustness validation and linearization accept a captured non-empty database state while preserving the required ordering of the special read.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100