Error when generating metadata of file with 155 columns
- Dominant language
- Java
- Stars
- 2k
- Forks
- 990
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 5
Description
Hello!
I have problems to generate the metadata of a Parquet file in S3 with 156 columns (some with long names). Drill version is 1.21.1
In my IDE I see the error:
`"SQL Error: INTERNAL_ERROR ERROR: Code generation error - likely code error.
Fragment: 0:0"`
I attach the complete log (error_drill_metadata.log) where you can see much more detail.
For example:
`"Caused by: java.lang.Exception: File 'org.apache.drill.exec.compile.DrillJavaFileObject[StreamingAggregatorGen1872.java]', Line 13982, Column 18: StreamingAggregatorGen1872.java:13982: error: code too large".`
[error_drill_metadata.log](https://github.com/apache/drill/files/13407810/error_drill_metadata.log)
Step to reproduce:
1. Upload the empty_parquet.parquet ( inside [empty_parquet.zip](https://github.com/apache/drill/files/13407880/empty_parquet.zip)) file to an S3 bucket. For example. "s3://your-bucker/empty_parquet/empty_parquet.parquet
2. Configure Drill to access the bucket
3. Execute the command "ANALYZE TABLE s3.empty_parquet refresh METADATA;".
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the failure with empty_parquet.parquet in S3 and the ANALYZE TABLE s3.empty_parquet refresh METADATA command, then inspect error_drill_metadata.log for the generated-source failure. The work is done when metadata generation succeeds for the 156-column file without the Java "code too large" error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100