cockroachdb / cockroachdb/cockroach
roachtest: enable follower-reads/mixed-version/survival=region/locality=global/reads=strong test in shared mode
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
Roach test for follower reads on global tables has been failing for some time for shared tenants. The test expects follower reads to be served from the nearest replica based on latency, but in some cases, requests are routed to a different replica. A code walkthrough suggests this behavior could result from failed health checks, though no such evidence was observed in the logs. Attempts to reproduce the issue were unsuccessful. As next steps, we plan to disable the test for secondary tenants, add additional logging (maybe using a test knob), and re-enable the test once additional logging is available.
The test is temporarily disabled for shared tenants to avoid noise. This ticket tracks re-enabling it once the issue is fixed.
**To Reproduce**
It wasn't always reproducible but the best possible way to reproduce the issue is to run the test concurrently multiple times.
```
COCKROACH_RANDOM_SEED= roachtest run 'follower-reads/mixed-version/survival=region/locality=global/reads=strong' --count 10
```
**Expected behavior**
The request should be routed to the nearest replica based on latency.
**Environment:**
Roach test on master branch.
**Additional context**
Analysis on the previous failures is available here:
- https://github.com/cockroachdb/cockroach/issues/142194#issuecomment-2801710182
- https://github.com/cockroachdb/cockroach/issues/142194#issuecomment-2802173609
- https://github.com/cockroachdb/cockroach/issues/142194#issuecomment-3452619023
- https://github.com/cockroachdb/cockroach/issues/142194#issuecomment-3478121416
- https://github.com/cockroachdb/cockroach/issues/142194#issuecomment-3480832574
Jira issue: CRDB-56389
Contributor guide
Assessment
This issue has not been assessed yet.