[Feature] Enable Fluss to run in github-hosted runners
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.
### Motivation
We hope Fluss can run test in the github-hosted runners so that the contributor can verify their changes in their own github fork. But there still some issue in testing, there may be other issues, I'll add them if I meet new ones:
- [ ] #110
```
Caused by: java.util.concurrent.ExecutionException: com.alibaba.fluss.exception.NetworkException: Disconnected from node 127.0.0.1:45007 (id: ts-1)
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at com.alibaba.fluss.server.testutils.FlussClusterExtension.lambda$waitUtilAllGatewayHasSameMetadata$5(FlussClusterExtension.java:364)
at com.alibaba.fluss.testutils.common.CommonTestUtils.retry(CommonTestUtils.java:141)
... 32 more
```
- [ ] #116
When run `FlussTableITCase.testAppendWithSmallBuffer`, it's easy to fall into hang forever, and the thread stack is
```
at com.alibaba.fluss.client.write.WriterMemoryBuffer.allocate(WriterMemoryBuffer.java:187)
at com.alibaba.fluss.client.write.RecordAccumulator.allocateMemorySegment(RecordAccumulator.java:412)
at com.alibaba.fluss.client.write.RecordAccumulator.append(RecordAccumulator.java:210)
at com.alibaba.fluss.client.write.WriterClient.doSend(WriterClient.java:180)
at com.alibaba.fluss.client.write.WriterClient.send(WriterClient.java:119)
at com.alibaba.fluss.client.table.writer.TableWriter.send(TableWriter.java:71)
at com.alibaba.fluss.client.table.writer.AppendWriter.append(AppendWriter.java:64)
at com.alibaba.fluss.client.table.FlussTableITCase.testAppendWithSmallBuffer(FlussTableITCase.java:126)
```
By add `@RepeatedTest(10)` in this test, it should be reproduced
- [x] #117
When running `ReplicaFetcherITCase.testPutKvNeedAck` , the assertion will fail with the following error
```
ERROR] com.alibaba.fluss.server.replica.fetcher.ReplicaFetcherITCase.testPutKvNeedAck Time elapsed: 0.257 s <<< FAILURE!
org.opentest4j.AssertionFailedError:
expected: 8
but was: 0
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.alibaba.fluss.testutils.DataTestUtils.assertLogRecordsEqualsWithRowKind(DataTestUtils.java:508)
at com.alibaba.fluss.server.replica.fetcher.ReplicaFetcherITCase.testPutKvNeedAck(ReplicaFetcherITCase.java:244)
```
`ReplicaFetcherITCase.testProduceLogNeedAck` will also throw exception:
```
[ERROR] com.alibaba.fluss.server.replica.fetcher.ReplicaFetcherITCase.testProduceLogNeedAck Time elapsed: 1.561 s <<< FAILURE!
org.opentest4j.AssertionFailedError:
expected: 10
but was: 0
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.alibaba.fluss.testutils.DataTestUtils.assertLogRecordsEqualsWithRowKind(DataTestUtils.java:508)
at com.alibaba.fluss.testutils.DataTestUtils.assertLogRecordsEquals(DataTestUtils.java:484)
at com.alibaba.fluss.server.replica.fetcher.ReplicaFetcherITCase.testProduceLogNeedAck(ReplicaFetcherITCase.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
```
- [ ] #144
`Error: Failures:
Error: TableManagerITCase.testPartitionedTableManagement:379->lambda$testPartitionedTableManagement$11:381
Expecting an empty Optional but was containing value: {0=[0], 1=[1], 2=[2], 3=[0], 4=[1], 5=[2], 6=[0], 7=[1], 8=[2], 9=[0], 10=[1], 11=[2], 12=[0], 13=[1], 14=[2], 15=[0]}
`
### Solution
_No response_
### Anything else?
_No response_
### Willingness to contribute
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the named integration tests on a GitHub-hosted runner and compare the failures in FlussClusterExtension, WriterMemoryBuffer, ReplicaFetcherITCase, and TableManagerITCase. Check issues #110, #116, #117, and #144 for the reported reproductions. Done means the relevant test suite runs reliably on a contributor's fork without the listed disconnects, hangs, or assertion failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, java
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100