cockroachdb / cockroachdb/copyist

Query arguments are not stored or verified

Open
#14 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
850
Forks
21
PR merge metrics
No merged PRs in 30d

Description

The arguments to a query are not stored when creating a recording, and not verified when playing back recording (obviously because there is nothing to verify with). This means that it is possible to change the arguments to a query without failing existing tests, as long as the sequence of queries does not change.
As an example, consider this line in the simple query test: https://github.com/cockroachdb/copyist/blob/0d1c75493bd277528140450fc7b3c9fa0ab77a87/drivertest/commontest/common.go#L115
If we change the `id` argument to 2 or 50 the result should change to a different name or no rows in the result set respectively. However running the test with playback (I used `pqtest` but I think it should be the same for all) results in a pass with no warning that the tests need to recorded again. If you update the test to look for the correct name or error then the test fails and gives a message suggesting you might need to redo the recording.
I know this example is a bit silly because you are changing the test without changing the expected result or redoing the recording, but the same thing could happen if you make changes in your application logic which ends up affecting the arguments to a query.

Is there a reason to not include the query arguments in the recording? I don't think serialization should be an issue because the library already handles that for the results.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.