awslabs / awslabs/amazon-kinesis-client-python

MultiLang Daemon for KCL 2.0 for python - how to increase the SDK timeouts?

Open
#174 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
377
Forks
228
PR merge metrics
No merged PRs in 30d

Description

Hello, I am using the Multilang Daemon for KCL 2.0 using python and seeing the following errors frequently. But I am not seeing any parameters in the properties file. How do I increase the SDK timeout value?

I see that this is discussed on how to do it on KCL using java.
https://github.com/awslabs/amazon-kinesis-client/issues/840

```
2022-05-17 11:21:33,867 [ShardRecordProcessor-0003] WARN s.a.k.l.ShardConsumerSubscriber [NONE] - shardId-000000000003: onError().

Cancelling subscription, and marking self as failed. KCL will recreate the subscription as neccessary to continue processing. If you are seeing this warning frequently consider increasing the SDK timeouts by providing an OverrideConfiguration to the kinesis client. Alternatively youcan configure LifecycleConfig.readTimeoutsToIgnoreBeforeWarning to suppressintermittent ReadTimeout warnings. Last successful request details -- request id - cd29a27d-3453-0c20-97d4-1d508108a749, timestamp - 2022-05-17T15:19:09.298Z

software.amazon.kinesis.retrieval.RetryableRetrievalException: ReadTimeout
at software.amazon.kinesis.retrieval.fanout.FanOutRecordsPublisher.errorOccurred(FanOutRecordsPublisher.java:343)
at software.amazon.kinesis.retrieval.fanout.FanOutRecordsPublisher.access$800(FanOutRecordsPublisher.java:68)
at software.amazon.kinesis.retrieval.fanout.FanOutRecordsPublisher$RecordFlow.executeExceptionOccurred(FanOutRecordsPublisher.java:802)
at software.amazon.kinesis.retrieval.fanout.FanOutRecordsPublisher$RecordFlow.lambda$exceptionOccurred$0(FanOutRecordsPublisher.java:781)
at software.amazon.kinesis.retrieval.fanout.FanOutRecordsPublisher$RecordsRetrievedContext.lambda$executeEventAction$3(FanOutRecordsPublisher.java:259)
at java.util.Optional.ifPresent(Optional.java:159)
at software.amazon.awssdk.utils.Either.apply(Either.java:84)
at software.amazon.kinesis.retrieval.fanout.FanOutRecordsPublisher$RecordsRetrievedContext.executeEventAction(FanOutRecordsPublisher.java:258)
at software.amazon.kinesis.retrieval.fanout.FanOutRecordsPublisher.evictAckedEventAndScheduleNextEvent(FanOutRecordsPublisher.java:200)
at software.amazon.kinesis.retrieval.fanout.FanOutRecordsPublisher.notify(FanOutRecordsPublisher.java:156)
at software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:55)
at software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:27)
at io.reactivex.internal.util.HalfSerializer.onNext(HalfSerializer.java:45)
at io.reactivex.internal.subscribers.StrictSubscriber.onNext(StrictSubscriber.java:97)
at io.reactivex.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:407)
at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:176)
at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:288)
at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:253)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2022-05-17 11:21:34,384 [ShardRecordProcessor-0002] WARN s.a.k.r.f.FanOutRecordsPublisher [NONE] - sha

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.