cockroachdb / cockroachdb/cockroach
roachtest: regression.diffs is overly broad in asserting implementation internals
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Currently cmd/roachtest/testdata/regression.diffs contains some diffs that appear to be updating the expected result sets of some introspection queries such that they now expect results sets which enumerate the set of (cockroach-specific) internal or system tables.
For example, this file currently references `cluster_replication_node_stream_checkpoints` which is an LDR-specific internally debugging view that is not relevant to tests and features outside of LDR, as well as `system_jobs_status_idx` is the name of a (virtual) index specific to the jobs system that is not relevant to this test.
As described in https://cockroachlabs.atlassian.net/wiki/spaces/CRDB/pages/181371303/Go+Golang+coding+guidelines#Scope-of-Tested-Behavior, we generally do not want tests to enumerate unrelated details of the system like this as it results in brittle and high toil tests.
This test should ideally find a way to adjust the query being executed rather than the expected results of executing it, such that we just do not enumerate these names/oids/etc rather than needing to assert the cockroach-specific results that are enumerated.
Jira issue: CRDB-45362
Contributor guide
Assessment
This issue has not been assessed yet.