apache / apache/pulsar

Flaky-test: PulsarSinksTest.testKinesis / KinesisSinkTester.addMoreRecordsAndGetMillisBehindLatest

Open
#15,364 2 comments 0 reactions 0 assignees View on GitHub
area/test Stale type/flaky-tests
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

KinesisSinkTester.addMoreRecordsAndGetMillisBehindLatest is flaky. It fails sporadically.

[example failure](https://github.com/apache/pulsar/runs/6205142531?check_suite_focus=true#step:12:20926)

```
Error: testKinesis(org.apache.pulsar.tests.integration.io.sinks.PulsarSinksTest) Time elapsed: 62.576 s <<< FAILURE!
java.io.UncheckedIOException: Cannot encode string.
at software.amazon.awssdk.utils.StringUtils.fromBytes(StringUtils.java:578)
at software.amazon.awssdk.core.BytesWrapper.asString(BytesWrapper.java:82)
at org.apache.pulsar.tests.integration.io.sinks.KinesisSinkTester.addMoreRecordsAndGetMillisBehindLatest(KinesisSinkTester.java:210)
at org.apache.pulsar.tests.integration.io.sinks.KinesisSinkTester.lambda$internalValidateSinkResult$2(KinesisSinkTester.java:195)
at org.awaitility.core.CallableCondition$ConditionEvaluationWrapper.eval(CallableCondition.java:99)
at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:248)
at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:235)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.nio.charset.MalformedInputException: Input length = 3
at java.base/java.nio.charset.CoderResult.throwException(CoderResult.java:274)
at java.base/java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:820)
at software.amazon.awssdk.utils.StringUtils.fromBytes(StringUtils.java:576)
... 10 more
```

Contributor guide

Open the contributing guide

Research direction

Start with KinesisSinkTester.java at addMoreRecordsAndGetMillisBehindLatest (line 210) and the PulsarSinksTest.testKinesis integration test. Reproduce the example failure and inspect the reported AWS SDK MalformedInputException during the Awaitility polling path. Done means the Kinesis sink test no longer fails sporadically and still validates the sink result.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
stream-processing, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.