Question: does datafusion-cli being tested on all files in apache/parquet-testing?
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
I found a parquet file that datafusion-cli failed to read and opened a bug in:
- https://github.com/apache/arrow-rs/issues/8495
and added the file in:
- https://github.com/apache/parquet-testing/pull/96
I can't find it but does the CI run basic sanity on all the files in https://github.com/apache/parquet-testing data directory (https://github.com/apache/parquet-testing/tree/a3d96a65e11e2bbca7d22a894e8313ede90a33a3/data)
like simple:
```bash
find parquet-testing/data/ -name "*.parquet" -type f | while read file; do
echo "Processing: $file"
datafusion-cli --command "select * from '$file'"
done
```
for sanity test
Contributor guide
Research direction
Start by checking the existing CI and datafusion-cli entry points for any use of the apache/parquet-testing data directory. Compare the current coverage with the proposed find loop and determine whether a sanity run over all .parquet files is practical. Done means the CI behavior and failure reporting are explicitly established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100