Run tpch q5 with 100G data got error
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
**Describe the bug**
```
Error: ArrowError(ExternalError(ArrowError(ExternalError(Execution("Arrow error: External error: Arrow error: External error: Execution error: Arrow error: External error: Arrow error: External error: Execution error: Arrow error: External error: Arrow error: External error: Execution error: Arrow error: External error: Arrow error: External error: Execution error: Arrow error: External error: Arrow error: External error: Execution error: Arrow error: External error: Execution error: Arrow error: Parser error: Error parsing line 14999998: Error(UnequalLengths { pos: Some(Position { byte: 2463490006, line: 14999999, record: 14999998 }), expected_len: 9, len: 3 })")))))
```
**To Reproduce**
command:
```
./tpch-gen.sh 100
cargo run --release --bin tpch -- benchmark datafusion --iterations 1 --path ./data --format tbl --query 5 --batch-size 4096
```
**Expected behavior**
A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
Contributor guide
Research direction
Start by running tpch-gen.sh 100 and the provided cargo run --release --bin tpch command to reproduce the parser error. Inspect the generated data around line 14999999 and the tpch benchmark entry point to determine why the record has three fields instead of nine. Done means query 5 completes successfully on the 100G dataset.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100