apache / apache/pulsar

schema registry / pulsar-admin: Schema type BYTES is returned as Schema type NONE

Open
#10,271 2 comments 0 reactions 0 assignees View on GitHub
lifecycle/stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 22h
Merged PRs (30d)
142

Description

**Describe the bug**

Schema type BYTES set for the topic is returned as Schema type NONE.

I hit this issue while working on the sink that uses GenericObject / relies on AutoConsumeSchema returning correct schema.
While there is a workaround there to make AutoConsumeSchema handle schema type NONE as BYTES this whole behavior is rather unexpected and illogical.

**To Reproduce**

```
$ bin/pulsar-admin schemas delete my-topic
$ bin/pulsar-admin schemas get my-topic
HTTP 404 Not Found
Reason: HTTP 404 Not Found
```
This is ok.

```
$ cat ~/schema2.json
{
"type": "BYTES",
"schema": "",
"properties": {}
}

$ bin/pulsar-admin schemas upload --filename ~/schema2.json my-topic
$ bin/pulsar-admin schemas get my-topic
{
"version": 8,
"schemaInfo": {
"name": "my-topic",
"schema": "",
"type": "NONE",
"properties": {}
}
}
```

**Expected behavior**

Expect to get the same schema as the one I set (BYTES)

**Desktop (please complete the following information):**
- OS: macOS

**Additional context**

version 2.8.0-snapshot

I think it is related to SchemaRegistryServiceImpl convertToDomainType/convertFromDomainType where any type with type.getValue() < 0 is treated as NONE.
src/main/proto/SchemaRegistryFormat.proto doesn’t even have SchemaType.BYTES
SchemaType.BYTES is -1.
Other things with negative values there are AUTO/AUTO_CONSUME/AUTO_PUBLISH.

I haven't looked at what else handles BYTES as a special case compared with other schemas (STRING, INT.., ...)

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the schema2.json payload using bin/pulsar-admin schemas upload and get. Start in SchemaRegistryServiceImpl, especially convertToDomainType and convertFromDomainType, and inspect SchemaRegistryFormat.proto for the schema type mapping. Done means a BYTES schema uploaded for a topic is returned as BYTES rather than NONE.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.