dlt-hub / dlt-hub/dlt

repo(tests): Reduce number of parameterized tests

Open
#2,883 0 comments 0 reactions 0 assignees View on GitHub
tech-debt
Dominant language
Python
Stars
5.9k
Forks
600
Avg merge
1d 14h
Merged PRs (30d)
38

Description

One of the development bottlenecks is slow tests (locally and CI). In particular, there's an explosion of cases with parameterization as we add features. Some common parameters in the test suite:

- backend: duckdb, postgres, filesystem, etc.
- backend flavor: duckdb/motherduck, filesystem/iceberg, filesystem/s3
- file loader format: jsonl, csv, parquet
- dlt data type: bigint, float, timestamp
- source type: source, resource within a source, standalone resource
- reflection level: minimal, full, full with precision
- incremental configuration (multiple params): row order, last value function
- ... and many more

It would be highly valuable to manually review our entire test suite and reduce parameterization. This would involve writing a few more separate test that checks the condition. For example, `test_sql_database_source.py::test_reflect_foreign_keys_as_table` probably shouldn't be parameterized on `backend` (all assertions are about the dlt schema). Expected differences between `backend` should be tested explicitly in their own tests

Contributor guide

Open the contributing guide

Research direction

Start with test_sql_database_source.py::test_reflect_foreign_keys_as_table and inspect which parameterized cases affect its assertions. Review the broader test suite for parameters that do not change the behavior under test; done means redundant parameterization is reduced and backend-specific differences have explicit tests where needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.