Create tool that can infer missing test fixtures
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Coming up with test fixtures (e.g. [builtins ...]) adds a lot of friction, especially for new contributors. Write a tool to automatically infer and add a missing fixture to newly added test cases. It doesn't need to work perfectly, but we can perhaps implement something that works for, say, 80-90% of test cases.
The tool could work like this:
- Use git commands to determine newly added test cases by comparing against an ancestor master branch commit.
- For each new test case that doesn't have a fixture yet, look at the code being tested and guess a suitable fixture. For example, if a test case uses tuples, suggest
fixtures/tuple.pyi.
The inference might need some pretty ad hoc heuristics (e.g. which fixture to pick if multiple features are used), but that's okay.
This would need to be documented prominently in the developer docs.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or entry points are named. Start by locating the existing test-case and fixture conventions, then inspect how developer documentation is organized and how git identifies changes from an ancestor master commit. Done means a documented tool can detect newly added tests without fixtures, suggest fixtures from the tested code, and handle the expected common cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- developer-experience, documentation, testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100