citusdata / citusdata/citus

Write regression test guidelines

Open
#1,395 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

It's hard to remember all the rules that make for good regression tests. We should add a document to the wiki that succinctly describes the rules to follow.

Some basic ideas:

If applicable, test whether a new feature:
- shows the expected error messages when used incorrectly
- works with prepared statements
- works in a transaction block
- works in the same transaction as other commands (e.g. DDL)
- works with reference tables
- works with MX
- locks tables and shards appropriately
- ...

(don't need to have all of these all the time, but just good to remember pitfalls)

Do *not*:
- Show output that depends on prior or concurrent tests
- Use COPY with files
- Show unordered SELECT results
- Show timing
- Show shard IDs
- EXPLAIN without (COSTS OFF)
- Use the same tables across different tests
- Use DEBUG output unless
- ...

(unless absolutely necessary, e.g. EXPLAIN always shows shard IDs)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.