ProtoSchemaConverter renders invalid schema for oneof in unwrap mode
- 主要语言
- Java
- 星标
- 3.1k
- 派生
- 1.6k
- 平均合并
- 3 天 12 小时
- 30 天内合并 PR
- 33
描述
### Describe the bug, including details regarding any error messages, version, and platform.
When unwrap is enabled all fields of the `TestProto3.OneOfTestMessage` will be required.
```
message TestProto3.OneOfTestMessage {
required int32 first = 1;
required int32 second = 2;
}
```
https://github.com/apache/parquet-java/blob/73a4430af6c40f8eb246ad4911eb6d103c9a2abe/parquet-protobuf/src/test/resources/TestProto3.proto#L116
This will never work but tests are missing for unwrap of `TestProto3.OneOfTestMessage`
The required repetition is added here:
https://github.com/apache/parquet-java/blob/73a4430af6c40f8eb246ad4911eb6d103c9a2abe/parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java#L278
This could be caught early by adding `withValidation(true)` here:
https://github.com/apache/parquet-java/blob/73a4430af6c40f8eb246ad4911eb6d103c9a2abe/parquet-protobuf/src/test/java/org/apache/parquet/proto/TestUtils.java#L221
If validation is disabled it will fail when trying to read one of the non-existing fields in the `oneof`.
Is there really a need for setting all primitive fields in unwrap to required?
### Component(s)
_No response_
贡献指南
这个仓库没有索引到贡献指南
调研方向
从 parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java 中 unwrap repetition 的处理开始,然后检查 TestProto3.OneOfTestMessage fixture 及其测试。在 parquet-protobuf/src/test/java/org/apache/parquet/proto/TestUtils.java 中启用 validation,并为 unwrap mode 添加 coverage。完成的标准是 oneof conversion 生成有效的 schema,并且相关的 read path 得到测试。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java
- 领域
- data
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 45/100