apache / apache/arrow

[CI][Crossbow] Use apache/arrow instead of separated repository (e.g. ursacomputing/crossbow)

Open
#46,014 32 comments 3 reactions 0 assignees View on GitHub
Component: Continuous Integration Component: Developer Tools Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

We didn't have enough GitHub Actions capacity when we develop Crossbow. So using separated repository (and separated CI services such as Azure Pipelines and Travis CI) was reasonable.

But we have enough GitHub Actions capacity recently. Can we reconsider the Crossbow approach?

We may be able to use [`on.workflow_dispatch`](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#workflow_dispatch) and [`gh wokrflow run --ref`](https://docs.github.com/en/actions/managing-workflow-runs-and-deployments/managing-workflow-runs/manually-running-a-workflow?tool=cli#running-a-workflow) as an alternative of the current Crossbow approach.

Some inconvenient points for the current Crossbow approach:
* CI results use a GitHub comment ([example](https://github.com/apache/arrow/pull/46006#issuecomment-2772173395)) instead of GitHub [status checks](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks).
* It may cause a long history.
* No direct link to artifacts.
* Crossbow creates a new tag for each job and uses GitHub Release associated with the tag for artifacts. There is no direct link for the associated GitHub Release.
* We can't use Dependabot for `dev/tasks/*/*.yml`
* Difficult to automate job submission.
* For example, we want to submit binary artifacts build jobs after source archive build is completed automatically in release process.
* FYI: We already have `secrets.GITHUB_CROSSBOW_TOKEN` in apache/arrow GitHub Actions. So we will be able to implement it with the current Crossbow approach.
* ...

### Component(s)

Continuous Integration, Developer Tools

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.