camel-quarkus-jackson-avro - Cannot use FormatSchema of type com.fasterxml.jackson.dataformat.avro.AvroSchema for format JSON
- Dominant language
- Java
- Stars
- 302
- Forks
- 232
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 114
Description
### Bug description
These days I suffered from an error described as:
```
com.fasterxml.jackson.core.JsonParseException: No AvroSchema set, can not parse at com.fasterxml.jackson.dataformat.avro.deser.MissingReader._checkSchemaSet(MissingReader.java:68)
```
This error, along with the lack of examples of how to apply serialization/deserialization with camel quarkus avro in the java DSL format, made me try to create a ZulipChat topic to find some guidance on the problem: [how to marshall/unmarshall AVRO format](https://camel.zulipchat.com/#narrow/stream/257302-camel-quarkus/topic/how.20to.20marshal.2Funmarshal.20avro.20format)
With support from **James Netherton** and his example [repository](https://github.com/apache/camel-quarkus/blob/main/integration-tests/jackson-avro/src/main/java/org/apache/camel/quarkus/component/jackson/avro/it/JacksonAvroResource.java#L96-L99), I adjusted the code like this:
- A RouteBuilder for building REST:
```
@ApplicationScoped
public class EndpointRest extends RouteBuilder {
@Override
public void configure() throws Exception {
super.configure();
restConfiguration()
.bindingMode(RestBindingMode.off)
.skipBindingOnErrorCode(false)
.component("platform-http")
.dataFormatProperty("prettyPrint", "true")
.contextPath("/").port(8080)
.apiContextPath("/q/openapi")
.apiProperty("api.title", "{{openApi.apiTitle}}")
.apiProperty("api.description", "{{openApi.apiDescription}}")
.dataFormatProperty("json.in.disableFeatures", "FAIL_ON_UNKNOWN_PROPERTIES")
.apiProperty("api.version", "0.0.1")
.apiProperty("cors", "true");
rest("/cargo")
.tag("cargo")
.produces(MediaType.APPLICATION_JSON)
.consumes(MediaType.APPLICATION_JSON)
.post()
.route()
.process(tempoTracingRota)
.routeId("recebeCargoPost")
.setHeader(UtilConstante.EVENT, constant("Recebe solicitação"))
.process(loggingProcessor)
.to(Route.RECEBE)
.end()
.endRest();
}
}
```
- A RouteBuilder for building Camel Routes:
```
public class Route extends RouteBuilder {
public static final String RECEBE = "direct:receive-message";
@Override
public void configure() throws Exception {
from(RECEIVE)
.routeId("receive-id")
.log("Body before ${body}")
.unmarshal().json(JsonLibrary.Jackson, MensagemEntrada.class)
// .convertBodyTo(InputStream.class)
.log("Body convert ${body}")
// .marshal().json(JsonLibrary.Jackson, MensagemEntrada.class)
.marshal().avro(
AvroLibrary.Jackson,
InputMessage.class,
"serializeSchema"
)
.log("Body after ${body}")
;
}
}
```
- A Utility Class to be used as a CDI bean:
```
public class AvroMarshalUnmarshal {
private final AvroMapper MAPPER = AvroMapper.builder().build();
@Named
private SchemaResolver serializeSchema() throws IOException {
AvroSchemaGenerator gen = new AvroSchemaGenerator();
MAPPER.acceptJsonFormatVisitor(MensagemEntrada.class, gen);
AvroSchema schema = gen.getGeneratedSchema();
return ex -> schema;
}
}
```
Thus, managing to resolve the previous error, but starting to suffer another error:
```
java.lang.IllegalArgumentException: Cannot use FormatSchema of type com.fasterxml.jackson.dataformat.avro.AvroSchema for format JSON
```
And as described in the topic, I tried to use some approaches both from this repository that was given as an example, and others found in searches I did on the internet, but nothing resolved this specific error. Occurring both trying to marshall/unmarshall the avro format, and also trying to marshall/unmarshall a json (which only worked normally again using the java DSL when commenting out the section of the serializeSchema function in the utility class)
So, after that, I started debugging camel flow to better understand what could be changed in the code to finally have the expected result, which was marshall/unmarshall for the AVRO format.
So, I discovered that using in Camel Route Class
```
.marshal().avro(
AvroLibrary.Jackson,
InputMessage.class,
"serializeSchema"
)
```
The camel in the class `package org.apache.camel.builder.DataFormatClause` on line 155 does not make the appropriate change to the `JsonFactory` (which should be used the `AvroFactory` which is configured with `JacksonAvroDataFormat`) causing an exception on line 186 of `com. fasterxml.jackson.core.JsonFactory` where it compares the `schemaType` of `AvroSchema` which is "avro" with the schema expected by `JsonFactory` (which is "JSON").
I was able to solve this impasse by configuring the utility class as follows:
```
@ApplicationScoped
public class AvroMarshalUnmarshal {
@Inject
private final AvroMapper MAPPER = AvroMapper.builder().build();
@Inject
private final AvroSchemaGenerator gen = new AvroSchemaGenerator();
@Inject
final JacksonAvroDataFormat dataFormat = new JacksonAvroDataFormat();
@Named
private JacksonAvroDataFormat serializeSchema() throws IOException {
MAPPER.acceptJsonFormatVisitor(MensagemEntrada.class, gen);
dataFormat.setUseDefaultObjectMapper(false);
dataFormat.setAutoDiscoverObjectMapper(false);
dataFormat.setObjectMapper(MAPPER);
dataFormat.setAutoDiscoverSchemaResolver(false);
dataFormat.setSchemaResolver(ex -> gen.getGeneratedSchema());
dataFormat.setUnmarshalType(MensagemEntrada.class);
return dataFormat
}
}
```
And changing the Camel Route class like this:
```
public class Route extends RouteBuilder {
public static final String RECEBE = "direct:recebe-message";
@Override
public void configure() throws Exception {
super.configure();
final String enviaProcessamento = "direct:enviaTopicoProcessamento";
final String persistUri = "seda:persist";
from(RECEBE)
.routeId("recebe-id")
.log("Body before ${body.length} ${body}")
.unmarshal().json(JsonLibrary.Jackson)
.marshal(new AvroMarshalUnmarshal().serializeSchema())
.log("Body convert AVRO ${body.length} ${body}")
.unmarshal(new AvroMarshalUnmarshal().serializeSchema())
.log("Body convert Output ${body}")
.marshal().json(JsonLibrary.Jackson, MensagemEntrada.class)
.log("Body after ${body.length} ${body}")
;
}
}
```
NOTE: Gradle was used to manage the dependencies and I inserted the following artifact
`
implementation 'org.apache.camel.quarkus:camel-quarkus-jackson-avro'
`
which has the same jars expected by the dependency described in the [camel documentation](https://camel.apache.org/components/3.14.x/dataformats/avro-jackson-dataformat.html#_dependencies).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Java DSL calls in the reported RouteBuilder and inspect org.apache.camel.builder.DataFormatClause around line 155, then compare them with JacksonAvroDataFormat and the camel-quarkus jackson-avro integration-test example mentioned in the report. Reproduce the JSON and Avro marshal/unmarshal flows with the listed Gradle dependency and determine whether the expected format-specific mapper is selected without the workaround; done means both flows avoid the reported FormatSchema exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100