apache / apache/druid

druid 35: protobuf desc loading fails

Open
#18,730 2 comments 0 reactions 0 assignees View on GitHub
Uncategorized problem report
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version

druid 35.0.0, dockerhub official containers

### Description

I'm running a druid cluster over managed k8s. A kafka+protobuf spec is working in 34 but failing in 35 (yes I know using 35 was maybe a bit early), due to the protobuf loading:

"protoBytesDecoder": {
"type": "file",
"descriptor": "http://processor-schema/flow.pb.desc",
"protoMessageType": "FlowMessage"
}

I tried moving to in-container files (using file:///), but it also fails or inline with base64 (gets a null pointer)

Cannot construct instance of org.apache.druid.data.input.protobuf.FileBasedProtobufBytesDecoder, problem: Descriptor not found in class path [file:///opt/druid/flow.pb.desc]

with inline:
Cannot construct instance of org.apache.druid.data.input.protobuf.InlineDescriptorProtobufBytesDecoder, problem: java.lang.NullPointerException

So there's something quite not right that looks like a regression.

No issues up to druid-34, I reverted and everthing worked appropriately.

I don't *need* druid 35 so no hurry for me but I thought you may want to know.

Contributor guide

Open the contributing guide

Research direction

Start with org.apache.druid.data.input.protobuf.FileBasedProtobufBytesDecoder and InlineDescriptorProtobufBytesDecoder, comparing descriptor loading between Druid 34 and 35. Reproduce the HTTP, file, and inline configurations, then verify that protobuf descriptors load successfully without the reported classpath error or null pointer exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java, kubernetes
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.