Add fuzz regression testing to parquet/arrow/csv readers
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
The arrow-testing repository has several data files that caused issues with the C/C++ implementation
* https://github.com/apache/arrow-testing/tree/master/data/arrow-ipc-stream
* https://github.com/apache/arrow-testing/tree/master/data/arrow-ipc-file
* https://github.com/apache/arrow-testing/tree/master/data/parquet/fuzzing
* https://github.com/apache/arrow-testing/tree/master/data/csv/fuzzing
It would be nice to add tests in this repository that ensure the parquet/arrow/csv readers behave "nicely" when reading such files.
The definition of "nice" would be "no panics, and errors when appropriate"
**Describe the solution you'd like**
1. Add tests to try and read all the above mentioned invalid files
2. If any cause panics, temporarily skip them in the tests and file a ticket to track fixing the panics
**Describe alternatives you've considered**
**Additional context**
This was inspired while reviewing this doc from @pitrou on arrow security guidelines:
- https://github.com/apache/arrow/pull/48870
Contributor guide
Research direction
Review the arrow-testing paths data/arrow-ipc-stream, data/arrow-ipc-file, data/parquet/fuzzing, and data/csv/fuzzing, then locate the repository's parquet, Arrow, and CSV reader test entry points. Exercise every listed invalid file and verify that readers do not panic and return errors when appropriate; temporarily skip panic cases and file tracking tickets for them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100