Avro schema incompatible with schema upload vs python producer
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 142
Description
**Describe the bug**
Hi i just want to know if i do something wrong or if someone has identify following behavior (and found some solution for fixing it)
I create a topic and upload the Avro Schema as Yaml
```
pulsar/bin/pulsar-admin topics create persistent://schema/test3/foo22
/pulsar/bin/pulsar-admin schemas upload persistent://schema/test3/foo22 -f /git_files/schema/schema-validation.avro
```
Content of Avro Yaml Schema:
`{"type": "AVRO","schema":"{\"name\":\"Validation\",\"type\":\"record\",\"fields\":[{\"name\":\"messagetimestamp\",\"type\":\"string\"}]}","properties": {}}`
Schema looks like that with schemas get
```/pulsar/bin/pulsar-admin schemas get schema/test3/foo22
{
"version": 0,
"schemaInfo": {
"name": "foo22",
"schema": {
"name": "Validation",
"type": "record",
"fields": [
{
"name": "messagetimestamp",
"type": "string"
}
]
},
"type": "AVRO",
"properties": {}
}
}
```
set-is-allow-auot-update-schema is set to disable(!) to cover that nowone is “destroying” or topic
but if i know use a python producer the schema is failing with “incompatible schema”
python code:
```
class Validation(Record):
messagetimestamp = String(required=True)
producer = client.create_producer(topic, schema=AvroSchema(Validation))
inputString='Test55'
producer.send(Validation(
messagetimestamp = inputString
))
print("Message send")
client.close()
```
same behaviour with python pulsar functions .
if i turn on the auto schema update a new schema is created but total same structure:
```
{
"version": 1,
"schemaInfo": {
"name": "foo22",
"schema": {
"name": "Validation",
"type": "record",
"fields": [
{
"name": "messagetimestamp",
"type": "string"
}
]
},
"type": "AVRO",
"properties": {}
}
}
```
Any idea why that happens and how can i avoid that multiple producer (written in Java, Python) are changing the schema all time?!
**To Reproduce**
Steps to reproduce the behavior:
1. create topic
2. upload avro Schema (with yml reference -f yml file)
3. deactivate auto schema update
4. create python consumer (incompatible schema)
5. enable auto schema update
6. check schema (schema version changed)
**Expected behavior**
Schema version shouldn't be updated if python schema is the same as given in the schema upload (could it be that there is some hidden character given?!)
Contributor guide
Research direction
Start by reproducing the listed steps with pulsar-admin schemas upload/get and the Python AvroSchema producer, comparing the uploaded schema with the producer-generated schema. Check whether the schemas are treated as incompatible and why auto-update creates a new version; done means equivalent Java and Python producer schemas are accepted without an unnecessary version change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100