airbytehq / airbytehq/airbyte

[destination-gcs] version betwen 0.4.4 and 0.4.8 introduces JsonMappingException

未關閉
#51,592 4 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
area/connectors community connectors/destination/gcs team/destinations type/bug
主要語言
Python
星號
22.1k
分支
5.4k
平均合併
5 小時
30 天內合併 PR
671

描述

### Connector Name

destination-gcs

### Connector Version

v0.4.8

### What step the error happened?

During the sync

### Relevant information

Hello,

We have Airbyte `v1.3.0` deployed on GKE with Helm chart `v1.3.1`.

We recently upgraded our GCS destination connector from `v0.4.4` to `v0.4.8`.
Our sources use Cloud SQL connector in `v3.6.28`.

Majority of sync work well, but for one of them we observed below blocking error:
```
2025-01-15 14:15:56 destination WARN i.a.c.j.Jsons(handleDeserThrowable):424 Failed to deserialize json due to class com.fasterxml.jackson.databind.JsonMappingException
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:402)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:361)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1937)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:312)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3846)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3814)
at io.airbyte.commons.json.Jsons.tryDeserialize(Jsons.java:122)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:113)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)
at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)
at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
at io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)
Caused by class com.fasterxml.jackson.core.exc.StreamConstraintsException
at com.fasterxml.jackson.core.StreamReadConstraints._constructException(StreamReadConstraints.java:549)
at com.fasterxml.jackson.core.StreamReadConstraints.validateStringLength(StreamReadConstraints.java:484)
at com.fasterxml.jackson.core.util.ReadConstrainedTextBuffer.validateStringLength(ReadConstrainedTextBuffer.java:27)
at com.fasterxml.jackson.core.util.TextBuffer.finishCurrentSegment(TextBuffer.java:939)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString2(ReaderBasedJsonParser.java:2241)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString(ReaderBasedJsonParser.java:2207)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.getText(ReaderBasedJsonParser.java:327)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:572)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:100)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:25)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4899)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3846)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3814)
at io.airbyte.commons.json.Jsons.tryDeserialize(Jsons.java:122)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.consumeMessage(Destination.java:113)
at io.airbyte.cdk.integrations.base.Destination$ShimToSerializedAirbyteMessageConsumer.accept(Destination.java:91)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:321)
at io.airbyte.cdk.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:301)
at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:190)
at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
at io.airbyte.integrations.destination.gcs.GcsDestination.main(GcsDestination.java:16)
2025-01-15 14:15:56 destination ERROR i.a.c.i.b.Destination$ShimToSerializedAirbyteMessageConsumer(consumeMessage):120 Received invalid message:**********
```

If we rollback to GCS destination connector `v0.4.4`, the issue disappears.

Please could you investigate and fix the issue ? 🙏

Complementary question: when do you plan to release an official first version of [GCS](https://docs.airbyte.com/integrations/destinations/gcs) destination connector like [Amazon S3](https://docs.airbyte.com/integrations/destinations/s3) ?

Thanks !

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。