apache / apache/texera

Extract shared integration-job infra provisioning into a composite action (DRY amber-integration + platform-integration)

Open
#6,275 1 comment 0 reactions 1 assignee Claimed by @mengw15 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Task Summary

#6274 added a `platform-integration` job that provisions postgres + MinIO + LakeFS, alongside `amber-integration`'s postgres + MinIO + Lakekeeper. The two jobs now **duplicate** a lot of infra-provisioning shell (Start Postgres / Create Databases / Start MinIO / Start LakeFS+Lakekeeper), which will drift as image versions, ports, or setup steps change.

Factor the common provisioning into a reusable **composite action** (`.github/actions/setup-texera-infra/action.yml`) that both jobs `uses:`, parameterized by which services to bring up (postgres / minio / lakefs / lakekeeper) and by OS — one source of truth.

### Current state

- `amber-integration`: inlines Start Postgres / Create Databases / Start MinIO / Start Lakekeeper + warehouse-init, with per-OS (docker / brew) branches.
- `platform-integration` (#6274): inlines postgres (services container) / Create Databases / Start MinIO / Start LakeFS, ubuntu-only.
- The object-store + schema bring-up logic is duplicated across the two.

### Suggested direction

- Add `.github/actions/setup-texera-infra` (composite), handling the docker / brew OS branching internally.
- Inputs to toggle each service (postgres / minio / lakefs / lakekeeper) and pass OS.
- Refactor both `amber-integration` and `platform-integration` to call it, so infra bumps / fixes land in one place.

Follow-up to #6274; related to #6047 (which generalizes the sbt-side `@IntegrationTest` mechanism — this is the CI-infra counterpart).

### Task Type

- [x] Refactor / Cleanup
- [x] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.