[destination-gcs] version betwen 0.4.4 and 0.4.8 introduces JsonMappingException
- Lenguaje dominante
- Python
- Estrellas
- 22.1k
- Forks
- 5.3k
- Métricas de merge de PR
- Métricas de PR pendientes
Descripción
### Connector Name
destination-gcs
### Connector Version
v0.4.8
### What step the error happened?
During the sync
### Relevant information
Hello,
We have Airbyte `v1.3.0` deployed on GKE with Helm chart `v1.3.1`.
We recently upgraded our GCS destination connector from `v0.4.4` to `v0.4.8`.
Our sources use Cloud SQL connector in `v3.6.28`.
Majority of sync work well, but for one of them we observed below blocking error:
```
2025-01-15 14:15:56 destination WARN i.a.c.j.Jsons(handleDeserThrowable):424 Failed to deserialize json due to class com.fasterxml.jackson.databind.JsonMappingException
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:402)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:361)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1937)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:312)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3846)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3814)
at io.airbyte.commons.json.Jsons.tryDeserialize(Jsons.java:122)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:113)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)
at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)
at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
at io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)
Caused by class com.fasterxml.jackson.core.exc.StreamConstraintsException
at com.fasterxml.jackson.core.StreamReadConstraints._constructException(StreamReadConstraints.java:549)
at com.fasterxml.jackson.core.StreamReadConstraints.validateStringLength(StreamReadConstraints.java:484)
at com.fasterxml.jackson.core.util.ReadConstrainedTextBuffer.validateStringLength(ReadConstrainedTextBuffer.java:27)
at com.fasterxml.jackson.core.util.TextBuffer.finishCurrentSegment(TextBuffer.java:939)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString2(ReaderBasedJsonParser.java:2241)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString(ReaderBasedJsonParser.java:2207)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.getText(ReaderBasedJsonParser.java:327)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:572)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:100)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:25)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3846)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3814)
at io.airbyte.commons.json.Jsons.tryDeserialize(Jsons.java:122)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:113)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)
at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)
at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
at io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)
2025-01-15 14:15:56 destination ERROR i.a.c.i.b.Destination$ShimToSerializedAirbyteMessageConsumer(consumeMessage):120 Received invalid message:**********
```
If we rollback to GCS destination connector `v0.4.4`, the issue disappears.
Please could you investigate and fix the issue ? 🙏
Complementary question: when do you plan to release an official first version of [GCS](https://docs.airbyte.com/integrations/destinations/gcs) destination connector like [Amazon S3](https://docs.airbyte.com/integrations/destinations/s3) ?
Thanks !
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.