aws / aws/aws-encryption-sdk-python

CI: Avoid GH throttling via Consolidation

Open
#565 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
255
Forks
92
Avg merge
2d 17h
Merged PRs (30d)
2

Description

#### Issue:
Our CI can fail due to GitHub throwing a 429.
i.e: Too many CI actions tried to pull from the repo and GitHub throttled us.

There are two causes of this:
1. We use GH Workflow Matrices, which spawn a job per entry
2. Every Dependabot PR updates on a merge

Thus the requests to pull the Repo from GitHub is:
```
|dependabot_prs| x 79
```
Where 79 is the current number of individual GitHub actions created through our matrix use.

#### A Solution:
Too prevent this in the future, we should consolidate are GitHub Workflow Jobs.

Technically, a Job could run all the tox envs for a OS Version,
even across Python versions.

That would dramatically reduce the requests to pull the repo down.

This reduce the number GitHub Jobs and increase the Steps per Job.

We would loose some of our high level test reporting,
but I believe that is acceptable.

_Originally posted by @texastony in https://github.com/aws/aws-encryption-sdk-python/issues/564#issuecomment-1441033254_

Contributor guide

Open the contributing guide

Research direction

Start by locating the GitHub Workflow matrix definitions and reviewing how tox environments are distributed across operating-system and Python-version jobs. Compare the current job count with a consolidated arrangement, then verify that CI still runs all tox environments and reports failures acceptably without triggering GitHub throttling.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.