`sort_fuzz` testing DX improvements
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
`datafusion/core/tests/fuzz_cases` is a very cool tool! It helped to spot https://github.com/apache/datafusion/issues/16228. However, once the issue was discovered, it tool several hours to understand a problem and create a reproducible example
Partially that's because debug info to reproduce the problem right now is very big and not very useful:
### Describe the solution you'd like
Ideas I have to improve the DX:
- [ ] Store dataset in a file instead of printing to stdout. The current version is too big and a bit messy - and it also makes reproducing issues harder.
- [ ] Streamline seeds. There are some hardcoded seeds, but it's unclear how to call a function with arguments to get the same error-inducing dataset. It'd help to include a hint on how to generate a test batch for debugging later.
- [ ] Add docs on how to extend, use, and maintain the suite (including the modules above).
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start in datafusion/core/tests/fuzz_cases and inspect the sort_fuzz testing workflow, including how datasets and seeds are currently reported. Compare the requested options for storing reproducible datasets, documenting seed or test-batch generation, and explaining how to extend and maintain the suite. Done means the selected improvements make failures easier to reproduce and the suite has the requested usage documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- developer-experience, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100