etcd-io / etcd-io/etcd

Improve robustness coverage of revision window for Watch and other read methods.

Open
#20,349 10 comments 0 reactions 1 assignee Claimed by @henrybear327 View on GitHub
area/robustness-testing type/feature
Dominant language
Go
Stars
52.3k
Forks
10.5k
Avg merge
2d 21h
Merged PRs (30d)
43

Description

### What would you like to be added?

Improving revision coverage with regards to reads
Recently, a correctness issue https://github.com/etcd-io/etcd/issues/20221 was discovered related to watch operations regarding future revisions. The issue was not reproducible in robustness test because we lack dedicated testing for future revision. This means that the issue is very rare, that robustness cannot reproduce but Antithesis can thanks to dedicated assertion that guide exploration.

We should improve coverage of revision for watch and other read methods, as it’s currently lacking.

Problem:

Revisions represent history of changes, so covering them means we are testing a moving window of etcd history. As etcd receives writes and compacts the window changes. At any point there are 3 areas:
* Future revisions, ask about changes that etcd cluster or etcd member has not yet observed.
* Available revisions, ask about history of changes that etcd is still storing and can respond to
* Compacted revisions, ask about history so old that etcd has already purged.

During the test we are doing concurrent reads and writes, so the window of future,available,compacted revisions is shifting. We should propose a method that covers all those thee cases well enough and has some way to guarantee that.

### Why is this needed?

Be able to prevent issues like https://github.com/etcd-io/etcd/issues/20221 in the future

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.