Issues when using the head on a parquet file
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
```
**$** parquet head train-00000-of-00001-15a05aeec7726f9d.parquet
Unknown error
shaded.parquet.org.apache.avro.SchemaParseException: Illegal character in: original-instruction
at shaded.parquet.org.apache.avro.Schema.validateName(Schema.java:1607)
at shaded.parquet.org.apache.avro.Schema.access$400(Schema.java:92)
at shaded.parquet.org.apache.avro.Schema$Field.(Schema.java:556)
at shaded.parquet.org.apache.avro.Schema$Field.(Schema.java:595)
at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:295)
at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:279)
at org.apache.parquet.cli.util.Schemas.fromParquet(Schemas.java:89)
at org.apache.parquet.cli.BaseCommand.getAvroSchema(BaseCommand.java:405)
at org.apache.parquet.cli.commands.CatCommand.run(CatCommand.java:66)
at org.apache.parquet.cli.Main.run(Main.java:163)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.parquet.cli.Main.main(Main.java:193)
```
this is the dataset in question: https://huggingface.co/datasets/argilla/databricks-dolly-15k-curated-en/tree/main/data
**Reporter**: [Rémy Léone](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=remyleone) / @remyleone
**Note**: *This issue was originally created as [PARQUET-2377](https://issues.apache.org/jira/browse/PARQUET-2377). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the failure with the linked Parquet file using the parquet head command, then trace the stack through CatCommand, BaseCommand.getAvroSchema, and Schemas.fromParquet into AvroSchemaConverter. Done means head can process this file despite the original-instruction field name, with a regression test covering the behavior if the existing test structure identifies a suitable location.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100