apache / apache/pulsar

[Bug] Pulsar client AUTO_CONSUME "double" as "BigDecimal" in JSON schemas

Open
#20,092 8 comments 0 reactions 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Version

Pulsar client 2.10.3, 2.10.2, 2.10.1. Please note the behavior was correct with 2.8.3 and 2.10.0 and regressed starting with 2.10.1

### Minimal reproduce step

Run the below program to create a topic with a JSON schema that has an optional double field. **Please note the behavior is correct if the field is not optional**

```
public static void main(String[] args) throws PulsarClientException {
final PulsarClient client = PulsarClient.builder()
//.serviceUrl("pulsar://localhost:65093")
.serviceUrl("pulsar://localhost:6650")
.build();

RecordSchemaBuilder schemaBuilder = SchemaBuilder.record("myrecord");
schemaBuilder.field("xdouble").type(SchemaType.DOUBLE).optional();
SchemaInfo schemaInfo = schemaBuilder.build(SchemaType.JSON);
GenericSchema schema = Schema.generic(schemaInfo);
GenericRecordBuilder builder = schema.newRecordBuilder();
builder.set("xdouble", 1.0d);
GenericRecord record = builder.build();

Producer producer = client.newProducer(schema)
.topic("persistent://public/default/json-topic3")
.create();
producer.send(record);

Consumer consumer = client.newConsumer(Schema.AUTO_CONSUME())
.topic("persistent://public/default/json-topic3")
.subscriptionName("my-subscription2")
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
.subscribe();

var message = consumer.receive();
var value = ((JsonNode)message.getValue().getNativeObject()).get("xdouble").numberValue();
assertTrue("expected double, got " + value.getClass() , value instanceof Double);
//consumer.acknowledge(message);

client.close();
}
```

### What did you expect to see?

The assertion should pass

### What did you see instead?

```
Exception in thread "main" java.lang.AssertionError: expected double, got class java.math.BigDecimal
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.example.SimpleConsumer.main(SimpleConsumer.java:63)
```

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the Java reproduction using Schema.AUTO_CONSUME(), an optional DOUBLE JSON field, and the GenericRecord value; compare the behavior across the listed Pulsar client versions. Done means the assertion receives a Double for the optional field, while preserving the correct behavior for non-optional fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.