datafusion-contrib / datafusion-contrib/StreamFusion
Flink 1.18: dual-line CI, upstream suite and release validation
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 120
- Forks
- 8
- Avg merge
- 7h 29m
- Merged PRs (30d)
- 76
Description
Current behavior
CI and the upstream suite already run both Flink lines. What remains is the release half: the
release workflow publishes one line. The Flink 1.18 benchmark is already recorded in #198.
Shipped
- The CI Java matrix runs the full suite on both lines, and uploads per-line portable SQL audit
outcomes. - All nine tagged connector and format module suites run on both lines.
- Lake module CI runs Paimon Parquet and ORC on both lines; Delta stays on 2.2.
- Image integration builds the optimized payload and runs the Session-cluster smoke test on both
lines, includingbin/check-artifacts.sh --flink-line. - The upstream suite matrix runs
runtime,formats,parquet,orc,kafka,paimonand
stateon both lines, withdeltaexcluded on 1.18 and a separate Delta 3.3 / Flink 1.18 host
audit job. Each run records result totals and separates native execution, expected fallback and
cases with no execution contract. - The loader is tested on both lines with real planner installation.
The optimized release+mimalloc Flink 1.18 Nexmark comparison shipped in #198 and is linked from
docs/benchmarks.md: recorded method and results.
It keeps the unchanged rowwise generator, blackhole sink and both transposes. All ten measured
queries are slower than stock (0.28–0.88x); the existing Kafka headline table is unchanged.
Remaining
- Publish and validate both artifact sets from the release workflow: separate Maven staging,
line-qualified archives and image tags, per-line source and Javadoc artifacts, and
bin/check-artifacts.shagainst each line's artifact set. The workflow currently performs one
deployment and assembles unsuffixed archives. - Update
docs/releasing.mdwith the completed automated dual-line publication process. The
experimental build/package commands and benchmark methodology are already documented.
Publication of the 1.18 line stays blocked on #188 regardless of this issue; passing the CI and
upstream matrix does not establish real-cluster cross-line upgrade support.
Done when
The release workflow publishes and validates both lines' artifact sets. The required release-build
1.18 benchmark and its methodology are already recorded.
Split from #182.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the release workflow and compare its current single deployment and unsuffixed archive assembly with the dual-line requirements in the issue. Trace bin/check-artifacts.sh for validation of each artifact set, then update docs/releasing.md with the completed process. Done means both lines publish separate Maven, archive, image, source, and Javadoc artifacts and each set is validated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- ci-cd, documentation, release
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100