Producer with JSON schema type with RecordSchemaBuilder doesn't throw an error if any extra parameter is passed in the message payload.
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
Java Client(2.11.0)-->broker(2.11.0)
Broker instance OS: Amazon Linux 2023
Java App running OS : Windows 10
### Minimal reproduce step
- Enable **isSchemaValidationEnforced** on the pulsar instance.
- Use the following code snippet to build schema definition and send the message.
```
String jsonSchemaDef= "{"name" : "Employee", "fields" : [{ "name" : "Name" , "type" : "string" }, { "name" : "Age" , "type" : "int" }]}";
JSONObject schemaDefObj = new JSONObject(jsonSchemaDef);
JSONArray fArray = schemaDefObj.getJSONArray("fields");
RecordSchemaBuilder recordSchemaBuilder = SchemaBuilder.record(schemaName);
for(int i=0;i producerJson = pulsarclient.newProducer(Schema.generic(schemaInfo))
.topic(topicName)
.create();
GenericSchemaImpl schema = GenericJsonSchema.of(schemaInfo);
GenericRecordBuilder recordbuilder =schema.newRecordBuilder();
String message ="{"Employee":{"Age" :35,"Name" :"TestUser","Mobile":123469870}}"; // Added an extra field Mobile in the payload.
JSONObject payloadObj = new JSONObject(message);
Set fieldSet =payloadObj.keySet();
for (String key1 : fieldSet) {
recordbuilder.set(key1, payloadObj.getJSONObject(key1).toMap()); //Building the payload using GenericRecord builder to send the message
}
GenericRecord fieldrecord= rcbuilder.build();
MessageId msgId =producerAvro.newMessage().value(fieldrecord).send();
```
### What did you expect to see?
As the message payload contains extra parameter Mobile the pulsar broker should reject the message and throw an error.
### What did you see instead?
The message is acknowledged by the broker and send operation is successful.
### Anything else?
We tried the same case with Avro schema type and it throws an error if any extra parameter is passed in the message payload.
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Start by tracing RecordSchemaBuilder, GenericJsonSchema, and GenericRecordBuilder with isSchemaValidationEnforced enabled, then compare JSON handling with the Avro schema path. Reproduce the payload containing the extra Mobile field and determine whether the completed behavior rejects it during record construction or send validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100