Issues when using the head on a parquet file
- 主要语言
- Java
- 星标
- 3.1k
- 派生
- 1.6k
- 平均合并
- 3 天 12 小时
- 30 天内合并 PR
- 33
描述
```
**$** parquet head train-00000-of-00001-15a05aeec7726f9d.parquet
Unknown error
shaded.parquet.org.apache.avro.SchemaParseException: Illegal character in: original-instruction
at shaded.parquet.org.apache.avro.Schema.validateName(Schema.java:1607)
at shaded.parquet.org.apache.avro.Schema.access$400(Schema.java:92)
at shaded.parquet.org.apache.avro.Schema$Field.(Schema.java:556)
at shaded.parquet.org.apache.avro.Schema$Field.(Schema.java:595)
at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:295)
at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:279)
at org.apache.parquet.cli.util.Schemas.fromParquet(Schemas.java:89)
at org.apache.parquet.cli.BaseCommand.getAvroSchema(BaseCommand.java:405)
at org.apache.parquet.cli.commands.CatCommand.run(CatCommand.java:66)
at org.apache.parquet.cli.Main.run(Main.java:163)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.parquet.cli.Main.main(Main.java:193)
```
this is the dataset in question: https://huggingface.co/datasets/argilla/databricks-dolly-15k-curated-en/tree/main/data
**Reporter**: [Rémy Léone](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=remyleone) / @remyleone
**Note**: *This issue was originally created as [PARQUET-2377](https://issues.apache.org/jira/browse/PARQUET-2377). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
贡献指南
这个仓库没有索引到贡献指南
调研方向
使用 parquet head 命令通过链接的 Parquet 文件重现该故障,然后依次跟踪经过 CatCommand、BaseCommand.getAvroSchema 和 Schemas.fromParquet 进入 AvroSchemaConverter 的调用栈。如果现有测试结构能确定合适的位置,则添加覆盖此行为的回归测试;当 head 能够处理此文件(即使字段名为 original-instruction)时,即视为完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java
- 领域
- cli
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100