apache / apache/drill

Error when generating metadata of file with 155 columns

Open
#2,847 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
2k
Forks
990
Avg merge
1d 8h
Merged PRs (30d)
5

Description

Hello!
I have problems to generate the metadata of a Parquet file in S3 with 156 columns (some with long names). Drill version is 1.21.1
In my IDE I see the error:
`"SQL Error: INTERNAL_ERROR ERROR: Code generation error - likely code error.
Fragment: 0:0"`

I attach the complete log (error_drill_metadata.log) where you can see much more detail.
For example:
`"Caused by: java.lang.Exception: File 'org.apache.drill.exec.compile.DrillJavaFileObject[StreamingAggregatorGen1872.java]', Line 13982, Column 18: StreamingAggregatorGen1872.java:13982: error: code too large".`
[error_drill_metadata.log](https://github.com/apache/drill/files/13407810/error_drill_metadata.log)

Step to reproduce:

1. Upload the empty_parquet.parquet ( inside [empty_parquet.zip](https://github.com/apache/drill/files/13407880/empty_parquet.zip)) file to an S3 bucket. For example. "s3://your-bucker/empty_parquet/empty_parquet.parquet
2. Configure Drill to access the bucket
3. Execute the command "ANALYZE TABLE s3.empty_parquet refresh METADATA;".

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure with empty_parquet.parquet in S3 and the ANALYZE TABLE s3.empty_parquet refresh METADATA command, then inspect error_drill_metadata.log for the generated-source failure. The work is done when metadata generation succeeds for the 156-column file without the Java "code too large" error.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.