[Umbrella] Lack of standardized rules for splitting long-running connector E2E tests in CI
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 204
Description
### Background
Currently, SeaTunnel CI suffers from long execution time and high timeout risk. A single `all-connectors-it` module can take up to **4.5 hours** to complete. If any submodule times out due to network flakiness or resource contention, the whole job gets re-run for another 4.5 hours, seriously delaying PR merge.
Some connector E2E tests are already very time-consuming:
| Module | Observed duration |
|--------|-------------------|
| E2Elasticsearch | ~62 min |
| Hbase | ~44 min |
| Clickhouse | ~42 min |
| Mongodb | ~35 min |
| CDC MySQL | ~32 min |
| Http | ~30 min |
### Problem identified from previous attempt (PR #9976)
In PR #9976, which attempted to split long-running modules into standalone jobs, we found a key issue:
**No standardized splitting rules** – There is no documented guideline on:
- **Trigger conditions** – When should a module be considered "too long" and be split out?
- **Splitting method** – How exactly should the split be done (e.g., modifying `backend.yml` and `update_modules_check.py`)?
- **Responsibility** – Who should perform the splitting? Is it suitable for new contributors, or should it be done by experienced ones?
Without such standards, different contributors may handle similar situations inconsistently, increasing review complexity and risk of configuration errors.
### Proposal
We propose to establish a **written standard for splitting long-running connector E2E tests** in CI. The standard should cover:
1. **Trigger condition** – e.g., any connector E2E module that consistently runs > 60 minutes, or any aggregated job that exceeds 2.5 hours, should be considered for splitting.
2. **Splitting method** – a step-by-step template or checklist (e.g., copy an existing standalone job like `elasticsearch-connector-it`, adjust module names, and update both `backend.yml` and `update_modules_check.py` accordingly).
3. **Responsibility** – clarify whether this should be done by core contributors (due to CI complexity) or can be done by anyone with proper guidance, and who should review such changes.
### Next steps / discussion points
- What should the exact thresholds be (e.g., 60 min per module, 2.5h per parent job)?
- Should we write a short guide / PR template checklist?
- Who will lead the effort to draft and document the standard?
Contributor guide
No contributing guide indexed for this repository
Research direction
Read the existing standalone job such as elasticsearch-connector-it, then compare the CI configuration in backend.yml with the module handling in update_modules_check.py and the approach from PR #9976. Done means a written standard covering split thresholds, the step-by-step update process, contributor responsibility, and review ownership.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100