Handle GitHub runners possibly running out of space
Open
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 429
- Forks
- 1.1k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 153
Description
sorry this feels very draft status. move it into a another PR before merging this one
Originally posted by @mashehu in https://github.com/nf-core/modules/pull/6286#discussion_r1846901512
# get-number-of-shards:
# runs-on: ubuntu-latest
# outputs:
# # Needs to be a json array
# shards: ${{ steps.shards.outputs.shards }}
# total_shards: ${{ steps.shards.outputs.total_shards }}
# steps:
# - name: Install nf-test
# uses: nf-core/setup-nf-test@v1
# with:
# version: ${{ env.NFT_VER }}
# - id: shards
# run: |
# nftest_output=$(nf-test test --dry-run --changed-since HEAD^ --filter process --follow-dependencies)
# number_of_shards=$(echo $nftest_output | grep -o 'Found [0-9]* related test' | tail -1 | awk '{print $2}')
# three_tests_per_shard=$(echo $(($number_of_shards / 3)) | awk '{print int($1+0.5)}')
# shards_array=$(for shard in $(seq 1 $number_of_shards); do echo $shard; done | tr ' ' '\n' | jq -R . | jq -s .)
# echo "shards=${shards_array}" >> $GITHUB_OUTPUT
# echo "total_shards=${number_of_shards}" >> $GITHUB_OUTPUT
WIP Code
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the commented get-number-of-shards GitHub Actions job in the issue and the referenced nf-test commands, including --dry-run, --changed-since, and --follow-dependencies. Determine the intended workflow change for handling runner space, then verify that the resulting workflow performs the expected test sharding.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, shell, yaml
- Domain
- ci-cd, devops, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100