Schema.json: Type not supported: object
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### **Describe the bug**
We use schema to send messages, and the creation fails during the process of creating the producer. The error log is: caused by org.apache.avro.SchemaParseException: Type not supported: object. We suspect that there is a problem with Schema.json(). When we change a topic, the log no longer reports an error.
Pulsar: v2.8.2
### **To Reproduce**
Steps to reproduce the behavior:
1. Schema class
```java
@Data
public class StudentUploadQuestionResultMsg {
private String studentId;
private String classId;
private String schoolId;
private Integer subjectId; // SubjectEnum.pushValue
private Long uploadTime;
private List uploadQuestions;
private String dataSource;
@Data
public static class UploadQuestion {
private String answerId;
private String questionId;
private Integer answerResult;
private List tag;
}
}
```
2. Producer Code
```java
producer = client.newProducer(Schema.JSON(StudentUploadQuestionResultMsg.class)).topic(studentUploadQuestionTopic).create();
```
3. see error
```
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.pulsar.client.api.Producer]: Factory method 'studentUploadQuestionProducer' threw exception; nested exception is org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.ExecutionException: org.apache.pulsar.client.api.PulsarClientException: org.apache.avro.SchemaParseException: Type not supported: object caused by org.apache.avro.SchemaParseException: Type not supported: object
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:622)
... 66 common frames omitted
Caused by: org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.ExecutionException: org.apache.pulsar.client.api.PulsarClientException: org.apache.avro.SchemaParseException: Type not supported: object caused by org.apache.avro.SchemaParseException: Type not supported: object
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1027)
at org.apache.pulsar.client.impl.ProducerBuilderImpl.create(ProducerBuilderImpl.java:95)
at com.youdao.correctionReport.config.mq.PulsarConfig.studentUploadQuestionProducer(PulsarConfig.java:80)
at com.youdao.correctionReport.config.mq.PulsarConfig$$EnhancerBySpringCGLIB$$34e7bea7.CGLIB$studentUploadQuestionProducer$11()
at com.youdao.correctionReport.config.mq.PulsarConfig$$EnhancerBySpringCGLIB$$34e7bea7$$FastClassBySpringCGLIB$$60fba01a.invoke()
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363)
at com.youdao.correctionReport.config.mq.PulsarConfig$$EnhancerBySpringCGLIB$$34e7bea7.studentUploadQuestionProducer()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 67 common frames omitted
Caused by: java.util.concurrent.ExecutionException: org.apache.pulsar.client.api.PulsarClientException: org.apache.avro.SchemaParseException: Type not supported: object caused by org.apache.avro.SchemaParseException: Type not supported: object
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at org.apache.pulsar.client.impl.ProducerBuilderImpl.create(ProducerBuilderImpl.java:93)
... 78 common frames omitted
Caused by: org.apache.pulsar.client.api.PulsarClientException: org.apache.avro.SchemaParseException: Type not supported: object caused by org.apache.avro.SchemaParseException: Type not supported: object
```
### **Additional context**
Two things strike us as strange:
1. We have been producing data to this topic for a while. The behavior is normal. This error is originated when we updated a the schema class.
```java
// old schema class
@Data
public class StudentUploadQuestionResultMsg {
private String studentId;
private String classId;
private String schoolId;
private Integer commonSubjectId;
private Long uploadTime;
private List uploadQuestions;
@Data
public static class UploadQuestion {
private String answerId;
private String questionId;
private Integer answerResult;
}
}
```
2. We used restapi to check the schema infomation. We found a strange schema which VERSION is 4. The autoupdateschema is true and Schema compatibility check strategy is FULL.
```json
// strange schema. It has no name and type is object.
{
"type":"object",
"id":"urn:jsonschema:com:youdao:correctionReport:answer:mq:StudentUploadQuestionResultMsg",
"properties":{
"studentId":{
"type":"string"
},
"classId":{
"type":"string"
},
"schoolId":{
"type":"string"
},
"commonSubjectId":{
"type":"integer"
},
"uploadTime":{
"type":"integer"
},
"uploadQuestions":{
"type":"array",
"items":{
"type":"object",
"id":"urn:jsonschema:com:youdao:correctionReport:answer:mq:StudentUploadQuestionResultMsg:UploadQuestion",
"properties":{
"answerId":{
"type":"string"
},
"questionId":{
"type":"string"
},
"answerResult":{
"type":"integer"
}
}
}
}
}
}
// nomal schema
{
"type":"record",
"name":"StudentUploadQuestionResultMsg",
"namespace":"com.youdao.correctionReport.answer.mq",
"fields":[
{
"name":"classId",
"type":[
"null",
"string"
],
"default":null
},
{
"name":"commonSubjectId",
"type":[
"null",
"int"
],
"default":null
},
{
"name":"schoolId",
"type":[
"null",
"string"
],
"default":null
},
{
"name":"studentId",
"type":[
"null",
"string"
],
"default":null
},
{
"name":"uploadQuestions",
"type":[
"null",
{
"type":"array",
"items":{
"type":"record",
"name":"UploadQuestion",
"namespace":"com.youdao.correctionReport.answer.mq.StudentUploadQuestionResultMsg",
"fields":[
{
"name":"answerId",
"type":[
"null",
"string"
],
"default":null
},
{
"name":"answerResult",
"type":[
"null",
"int"
],
"default":null
},
{
"name":"questionId",
"type":[
"null",
"string"
],
"default":null
}
]
},
"java-class":"java.util.List"
}
],
"default":null
},
{
"name":"uploadTime",
"type":[
"null",
"long"
],
"default":null
}
]
}
```
Contributor guide
Research direction
Start with Schema.JSON generation and the schema registration path reached by ProducerBuilderImpl.create; compare the reported object schema with the normal Avro schema and the model changes that preceded it. Reproduce the producer creation failure with the supplied StudentUploadQuestionResultMsg classes and determine why the object schema is registered or consumed; done means the cause and a verified correction or documented limitation are established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100