apache / apache/fluss

[Feature] Enable Fluss to run in github-hosted runners

Open
#109 0 comments 0 reactions 0 assignees View on GitHub
component=test
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.

### Motivation

We hope Fluss can run test in the github-hosted runners so that the contributor can verify their changes in their own github fork. But there still some issue in testing, there may be other issues, I'll add them if I meet new ones:

- [ ] #110
```
Caused by: java.util.concurrent.ExecutionException: com.alibaba.fluss.exception.NetworkException: Disconnected from node 127.0.0.1:45007 (id: ts-1)
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at com.alibaba.fluss.server.testutils.FlussClusterExtension.lambda$waitUtilAllGatewayHasSameMetadata$5(FlussClusterExtension.java:364)
at com.alibaba.fluss.testutils.common.CommonTestUtils.retry(CommonTestUtils.java:141)
... 32 more
```

- [ ] #116
When run `FlussTableITCase.testAppendWithSmallBuffer`, it's easy to fall into hang forever, and the thread stack is
```
at com.alibaba.fluss.client.write.WriterMemoryBuffer.allocate(WriterMemoryBuffer.java:187)
at com.alibaba.fluss.client.write.RecordAccumulator.allocateMemorySegment(RecordAccumulator.java:412)
at com.alibaba.fluss.client.write.RecordAccumulator.append(RecordAccumulator.java:210)
at com.alibaba.fluss.client.write.WriterClient.doSend(WriterClient.java:180)
at com.alibaba.fluss.client.write.WriterClient.send(WriterClient.java:119)
at com.alibaba.fluss.client.table.writer.TableWriter.send(TableWriter.java:71)
at com.alibaba.fluss.client.table.writer.AppendWriter.append(AppendWriter.java:64)
at com.alibaba.fluss.client.table.FlussTableITCase.testAppendWithSmallBuffer(FlussTableITCase.java:126)
```
By add `@RepeatedTest(10)` in this test, it should be reproduced

- [x] #117
When running `ReplicaFetcherITCase.testPutKvNeedAck` , the assertion will fail with the following error
```
ERROR] com.alibaba.fluss.server.replica.fetcher.ReplicaFetcherITCase.testPutKvNeedAck Time elapsed: 0.257 s <<< FAILURE!
org.opentest4j.AssertionFailedError:

expected: 8
but was: 0
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.alibaba.fluss.testutils.DataTestUtils.assertLogRecordsEqualsWithRowKind(DataTestUtils.java:508)
at com.alibaba.fluss.server.replica.fetcher.ReplicaFetcherITCase.testPutKvNeedAck(ReplicaFetcherITCase.java:244)
```
`ReplicaFetcherITCase.testProduceLogNeedAck` will also throw exception:
```
[ERROR] com.alibaba.fluss.server.replica.fetcher.ReplicaFetcherITCase.testProduceLogNeedAck Time elapsed: 1.561 s <<< FAILURE!
org.opentest4j.AssertionFailedError:

expected: 10
but was: 0
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.alibaba.fluss.testutils.DataTestUtils.assertLogRecordsEqualsWithRowKind(DataTestUtils.java:508)
at com.alibaba.fluss.testutils.DataTestUtils.assertLogRecordsEquals(DataTestUtils.java:484)
at com.alibaba.fluss.server.replica.fetcher.ReplicaFetcherITCase.testProduceLogNeedAck(ReplicaFetcherITCase.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
```

- [ ] #144
`Error: Failures:
Error: TableManagerITCase.testPartitionedTableManagement:379->lambda$testPartitionedTableManagement$11:381
Expecting an empty Optional but was containing value: {0=[0], 1=[1], 2=[2], 3=[0], 4=[1], 5=[2], 6=[0], 7=[1], 8=[2], 9=[0], 10=[1], 11=[2], 12=[0], 13=[1], 14=[2], 15=[0]}
`

### Solution

_No response_

### Anything else?

_No response_

### Willingness to contribute

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the named integration tests on a GitHub-hosted runner and compare the failures in FlussClusterExtension, WriterMemoryBuffer, ReplicaFetcherITCase, and TableManagerITCase. Check issues #110, #116, #117, and #144 for the reported reproductions. Done means the relevant test suite runs reliably on a contributor's fork without the listed disconnects, hangs, or assertion failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, java
Domain
ci-cd, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.