druid 35: protobuf desc loading fails
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Affected Version
druid 35.0.0, dockerhub official containers
### Description
I'm running a druid cluster over managed k8s. A kafka+protobuf spec is working in 34 but failing in 35 (yes I know using 35 was maybe a bit early), due to the protobuf loading:
"protoBytesDecoder": {
"type": "file",
"descriptor": "http://processor-schema/flow.pb.desc",
"protoMessageType": "FlowMessage"
}
I tried moving to in-container files (using file:///), but it also fails or inline with base64 (gets a null pointer)
Cannot construct instance of org.apache.druid.data.input.protobuf.FileBasedProtobufBytesDecoder, problem: Descriptor not found in class path [file:///opt/druid/flow.pb.desc]
with inline:
Cannot construct instance of org.apache.druid.data.input.protobuf.InlineDescriptorProtobufBytesDecoder, problem: java.lang.NullPointerException
So there's something quite not right that looks like a regression.
No issues up to druid-34, I reverted and everthing worked appropriately.
I don't *need* druid 35 so no hurry for me but I thought you may want to know.
Contributor guide
Research direction
Start with org.apache.druid.data.input.protobuf.FileBasedProtobufBytesDecoder and InlineDescriptorProtobufBytesDecoder, comparing descriptor loading between Druid 34 and 35. Reproduce the HTTP, file, and inline configurations, then verify that protobuf descriptors load successfully without the reported classpath error or null pointer exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, java, kubernetes
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100