google / google/comprehensive-rust
Speed up publish CI pipeline
- Dominant language
- Rust
- Stars
- 33.4k
- Forks
- 2.1k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 10
Description
Currently a run of the [publish pipeline](https://github.com/google/comprehensive-rust/actions/workflows/publish.yml) is taking around 30 minutes.
Most of the time is spent in the "build all translation" step.
https://github.com/google/comprehensive-rust/blob/c1773a7fdbfc493f29005af6f3713745c6674d5d/.github/workflows/publish.yml#L55-L60
This is the main loop for building the translations. This is currently writing files into the `book//html` folder, then moved into `book/html/`. The build.sh script is heavily modifying the current state of the repository so this is not trivially parallelized in the same step.
Options:
- separate steps by using Githubs [job matrix](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow#using-a-matrix-strategy) to run each translation as a separate job. Note: this requires all other steps in that job to run multiple times which is wasting resources. Installing mdbook in that environment should be cached properly. The English translation should then also be handled like other languages as this is currently a special case.
- This also needs to take into account that the generated artifacts need to be used in later steps
- Alternative: Split the translation step into two steps (likely more efficient)
- (for each language) restore the entire book source as currently and copy all necessary files into a new `source//` directory
- With all sources in each `source/` folder, this can be easily parallelized after modifying the build script.
https://github.com/google/comprehensive-rust/blob/c1773a7fdbfc493f29005af6f3713745c6674d5d/.github/workflows/build.sh#L27-L28

Contributor guide
Assessment
This issue has not been assessed yet.