airbytehq / airbytehq/airbyte

[destination-gcs] version betwen 0.4.4 and 0.4.8 introduces JsonMappingException

Abierto
#51,592 4 comentarios 0 reacciones 0 asignados Ver en GitHub
area/connectors community connectors/destination/gcs team/destinations type/bug
Lenguaje dominante
Python
Estrellas
22.1k
Forks
5.3k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

### Connector Name

destination-gcs

### Connector Version

v0.4.8

### What step the error happened?

During the sync

### Relevant information

Hello,

We have Airbyte `v1.3.0` deployed on GKE with Helm chart `v1.3.1`.

We recently upgraded our GCS destination connector from `v0.4.4` to `v0.4.8`.
Our sources use Cloud SQL connector in `v3.6.28`.

Majority of sync work well, but for one of them we observed below blocking error:
```
2025-01-15 14:15:56 destination WARN i.a.c.j.Jsons(handleDeserThrowable):424 Failed to deserialize json due to class com.fasterxml.jackson.databind.JsonMappingException
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:402)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:361)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1937)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:312)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3846)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3814)
at io.airbyte.commons.json.Jsons.tryDeserialize(Jsons.java:122)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:113)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)
at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)
at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
at io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)
Caused by class com.fasterxml.jackson.core.exc.StreamConstraintsException
at com.fasterxml.jackson.core.StreamReadConstraints._constructException(StreamReadConstraints.java:549)
at com.fasterxml.jackson.core.StreamReadConstraints.validateStringLength(StreamReadConstraints.java:484)
at com.fasterxml.jackson.core.util.ReadConstrainedTextBuffer.validateStringLength(ReadConstrainedTextBuffer.java:27)
at com.fasterxml.jackson.core.util.TextBuffer.finishCurrentSegment(TextBuffer.java:939)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString2(ReaderBasedJsonParser.java:2241)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString(ReaderBasedJsonParser.java:2207)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.getText(ReaderBasedJsonParser.java:327)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:572)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:100)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:25)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3846)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3814)
at io.airbyte.commons.json.Jsons.tryDeserialize(Jsons.java:122)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:113)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)
at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)
at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
at io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)
2025-01-15 14:15:56 destination ERROR i.a.c.i.b.Destination$ShimToSerializedAirbyteMessageConsumer(consumeMessage):120 Received invalid message:**********
```

If we rollback to GCS destination connector `v0.4.4`, the issue disappears.

Please could you investigate and fix the issue ? 🙏

Complementary question: when do you plan to release an official first version of [GCS](https://docs.airbyte.com/integrations/destinations/gcs) destination connector like [Amazon S3](https://docs.airbyte.com/integrations/destinations/s3) ?

Thanks !

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.