Pulsar Java client's RecordSchemaBuilder doesn't provide any APIs to add nested records or Arrays for Avro and JSON type
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
Java Client(2.11.0)-->broker(2.11.0)
Broker instance OS: Amazon Linux 2023
Java App running OS : Windows 10
### Minimal reproduce step
- Using the RecordSchemaBuilder API I couldn't add any nested elements.
- Refer to the following code snippet which has nested records and array elements in the Schema definition.
- Using the record schema builder I couldn't find a way to add nested records or arrays.
- This applicable for both Avro as well as JSON. Hence request you to provide recommendations on how to achieve such use cases.
```
String avroSchemaDef= "{
"namespace": "my.com.ns",
"name": "myrecord",
"type": "record",
"fields": [{
"name": "uid",
"type": "int"
},
{
"name": "Product",
"type": "string"
},
{
"name": "options",
"type": {
"type": "array",
"items": {
"type": "record",
"name": "lvl2_record",
"fields": [{
"name": "item1_lvl2",
"type": "string"
},
{"name": "additional",
"type": {"type": "map", "values": "string"}},
{
"name": "item2_lvl2",
"type": {
"type": "array",
"items": {
"type": "record",
"name": "lvl3_record",
"fields": [{
"name": "item1_lvl3",
"type": "string"
},
{
"name": "item2_lvl3",
"type": ["string", "null"]
}, {
"name": "item_status",
"type": {
"name": "item_status",
"type": "enum",
"symbols": ["AVAILABLE", "OUT_OF_STOCK", "ONLY_FEW_LEFT"]
},
"default": "AVAILABLE"
}
]
}
}
}
]
}
}
}
]
}";
JSONObject schemaDefObj = new JSONObject(avroSchemaDef);
JSONArray fArray = schemaDefObj.getJSONArray("fields");
RecordSchemaBuilder recordSchemaBuilder = SchemaBuilder.record(schemaName);
for(int i=0;i
Contributor guide
Research direction
Start with the RecordSchemaBuilder and SchemaBuilder.record entry points mentioned in the issue, then trace how SchemaType and build(SchemaType) represent fields for Avro and JSON. Done should mean the builder offers a documented way to represent nested records and arrays for both schema types, with coverage for the nested example described here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, json
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100