apache / apache/pulsar

Producer with JSON schema type with RecordSchemaBuilder doesn't throw an error if any extra parameter is passed in the message payload.

Open
#20,274 4 comments 0 reactions 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Version

Java Client(2.11.0)-->broker(2.11.0)
Broker instance OS: Amazon Linux 2023
Java App running OS : Windows 10

### Minimal reproduce step

- Enable **isSchemaValidationEnforced** on the pulsar instance.
- Use the following code snippet to build schema definition and send the message.

```
String jsonSchemaDef= "{"name" : "Employee", "fields" : [{ "name" : "Name" , "type" : "string" }, { "name" : "Age" , "type" : "int" }]}";
JSONObject schemaDefObj = new JSONObject(jsonSchemaDef);
JSONArray fArray = schemaDefObj.getJSONArray("fields");
RecordSchemaBuilder recordSchemaBuilder = SchemaBuilder.record(schemaName);
for(int i=0;i producerJson = pulsarclient.newProducer(Schema.generic(schemaInfo))
.topic(topicName)
.create();
GenericSchemaImpl schema = GenericJsonSchema.of(schemaInfo);
GenericRecordBuilder recordbuilder =schema.newRecordBuilder();
String message ="{"Employee":{"Age" :35,"Name" :"TestUser","Mobile":123469870}}"; // Added an extra field Mobile in the payload.
JSONObject payloadObj = new JSONObject(message);
Set fieldSet =payloadObj.keySet();
for (String key1 : fieldSet) {
recordbuilder.set(key1, payloadObj.getJSONObject(key1).toMap()); //Building the payload using GenericRecord builder to send the message
}
GenericRecord fieldrecord= rcbuilder.build();
MessageId msgId =producerAvro.newMessage().value(fieldrecord).send();

```

### What did you expect to see?

As the message payload contains extra parameter Mobile the pulsar broker should reject the message and throw an error.

### What did you see instead?

The message is acknowledged by the broker and send operation is successful.

### Anything else?

We tried the same case with Avro schema type and it throws an error if any extra parameter is passed in the message payload.

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by tracing RecordSchemaBuilder, GenericJsonSchema, and GenericRecordBuilder with isSchemaValidationEnforced enabled, then compare JSON handling with the Avro schema path. Reproduce the payload containing the extra Mobile field and determine whether the completed behavior rejects it during record construction or send validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.