rapidsai / rapidsai/shared-workflows
Implement "dispatch" pattern from telemetry shared-actions
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 8
- Forks
- 33
- Avg merge
- 23h 51m
- Merged PRs (30d)
- 19
Description
One thing useful that came out of the telemetry work was the "dispatch" pattern, which is described in the shared-actions repo readme. The general idea is that instead of calling an action directly with a repo/branch reference, you should first manually clone the shared-actions repo at that branch, and then call the desired action with a relative path. The advantages to doing this are:
- It's much easier to include code outside of actions, such as native Python files. When a shared action is called directly, any other files in the repo do not come along for the ride. Using native files facilitates use of linters and other code helper tools.
- It's much easier to change branch references for several actions at once. In the shared-actions workflow, we run a
telemetry-stash-base-env-varsaction that captures several environment variables, two of which are the SHARED_ACTIONS_REPO and SHARED_ACTIONS_REF variables. This are written to a file, and stored as a github artifact. Subsequent actions (which use very stable logic on themainbranch) load that file, read those env vars, clone the requested repo/ref, and call the requested shared action with relative path. - This is a hack around the restricted list of actions that rapids jobs can clone, allowing us to point to our user forks instead of pushing branches directly to the rapidsai org repo.
For example, instead of changing multiple branch references to test a shared-workflows branch, like:
One could have:
# This is a project's pr.yaml
jobs:
shared-vars-setup:
runs-on: ubuntu-latest
continue-on-error: true
env:
SHARED_WORKFLOWS_REPO: rapidsai/shared-workflows
SHARED_WORKFLOWS_ACTION: cuda-12.9.0
steps:
- name: Stash shared variables setup
uses: rapidsai/shared-actions/telemetry-dispatch-stash-base-env-vars@main
cpp-build:
needs: [shared-vars-setup]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/dispatch-conda-cpp-build.yaml@main
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
and dispatch-conda-cpp-build.yaml would be:
inputs:
build_type: ...
branch: ...
jobs:
shared-vars-load-or-default:
steps:
# Downloads the env var file, loads it, then clones shared-workflows according to env var values
- name: Load shared variables setup
uses: rapidsai/shared-actions/load-env-vars-then-clone-workflows@main
cpp-build:
needs: [shared-vars-load-or-default]
uses: ./shared-workflows/.github/workflows/conda-cpp-build.yaml
with:
build_type: ${{inputs.build_type}}
branch: ${{inputs.branch}}
This adds overhead for the indirection, but when there are multiple shared-workflows being called, the developer need only set the values in the base parameter setup:
# This is a project's pr.yaml
jobs:
shared-vars-setup:
runs-on: ubuntu-latest
env:
SHARED_WORKFLOWS_REPO: rapidsai/shared-workflows
SHARED_WORKFLOWS_ACTION: cuda-12.9.0
steps:
- name: Stash shared variables setup
uses: rapidsai/shared-actions/telemetry-dispatch-stash-base-env-vars@main
All references to @main or @branch-25.08 in the project's pr.yaml file would not need to change, unless you needed to point to a new or altered dispatch action/workflow.
Implementation note:
Stashing and loading shared variables can utilize the existing shared-actions code. We would need to generalize the clone or copy the script to differentiate destination path of the cloned folder.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the shared-actions README and its existing code for stashing/loading shared variables and cloning repositories. Define the dispatch workflow changes around the examples in this issue, then verify that shared workflows can clone the requested repository and invoke actions or workflows through relative paths without changing each reference.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, github-actions, shell
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100