apache / apache/sedona-db

ci: Make it easier to reproduce CI jobs with local scripts

Open
#1,201 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
503
Forks
61
Avg merge
2d 4h
Merged PRs (30d)
90

Description

Yes that sounds good.

Here is some additional information: my final goal is to have an easy way to reproduce the full CI locally, so for small fixes we don’t have to go back and forth checking the GitHub CI runner status.

I usually use plain shell scripts for this purpose, as I find them much more flexible than pre-commit configuration files.

How about maintaining CI jobs like:

```
cargo-fmt.sh
cargo-clippy.sh
test.sh
```

Each one can support different options, for example:

```
./cargo-clippy.sh # check only
./cargo-clippy.sh --fix # auto-fix and then check
./test.sh --fast # quick tests, finish in minutes
./test.sh # full test suite, takes longer
```

Then local CI scripts, GitHub CI configuration, and pre-commit hooks can all call these shared scripts, while still allowing us to customize our local development workflow based on personal preferences.

I’m trying to set this up and hopefully sync it with upstream later.

_Originally posted by @2010YOUY01 in https://github.com/apache/sedona-db/issues/580#issuecomment-3882020292_

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing GitHub CI configuration and pre-commit hooks, then compare their jobs with the proposed cargo-fmt.sh, cargo-clippy.sh, and test.sh scripts. Define how shared shell scripts reproduce the full CI locally, including the stated options, and consider how the CI configuration and hooks would call them; done means local runs match the CI jobs.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, rust, shell
Domain
ci-cd, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.