digitalocean / digitalocean/action-doctl
New API rate limits hit by GitHub Actions node IP with 'no_auth: true'
- Dominant language
- JavaScript
- Stars
- 520
- Forks
- 51
- PR merge metrics
- No merged PRs in 30d
Description
An API limit is hit far too easily when using `digitalocean/action-doctl` with the `no_auth: 'true'` option to download a newer version of `doctl` than is available on the runner node.
The `no_auth: 'true'` option is important for security reasons when running DigitalOcean app schema validation for untrusted Pull Requests from forked repos, or from automations like `@dependabot`.
This option was added for use with offline app schema validation using `doctl app spec validate .do/app.yaml --schema-only`. A `pre-commit` hook exists here [`LyraPhase/pre-commit-digitalocean` - `doctl-app-spec-validate-offline`][1] which uses this functionality to run schema validation in an "offline" mode without requiring to expose a `DIGITALOCEAN_ACCESS_TOKEN` secret to an untrusted Pull Request or fork.
However, it now appears that there is a new API limit for downloading a version of `doctl` that supports the `--schema-only` option (`doctl >= v1.101.0` digitalocean/doctl#1450). GitHub Actions node seems to only have an old fallback version by default, version `1.98.1`, which does not support the `--schema-only` offline validation flag.
The API limit appears extremely easy to hit, perhaps due to the nature of GitHub Actions runners being shared infrastructure, and the API limit being tied to IP address. In fact, I just ran into this by merging a `@dependabot` PR on a project using `digitalocean/action-doctl` which had not had any changes or GitHub actions runs for over 5 months. On the first run of `digitalocean/action-doctl` after merge, it failed with the following error:
```console
Warning: API rate limit exceeded for 52.238.24.37. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Failed to retrieve latest version; falling back to: 1.98.1
```
Expand for example pre-commit.yaml GitHub Actions workflow using no_auth: 'true'
```yaml
name: pre-commit
on:
pull_request:
branches:
- master
- main
- develop
push:
branches:
- master
- main
- develop
jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install doctl
uses: digitalocean/action-doctl@v2.5.1
with:
no_auth: 'true'
- uses: pre-commit/action@v3.0.1
```
Expand for .pre-commit-config.yaml using LyraPhase/pre-commit-digitalocean doctl-app-spec-validate-offline
```yaml
# .pre-commit-config.yaml
---
exclude: |
(?x)^(
.*\.svg|
\.vscode/.*|
assets/.*|
target/.*|
go.mod|
go.sum|
vendor/.*|
tmp/.*
)$
repos:
- repo: https://github.com/LyraPhase/pre-commit-digitalocean.git
rev: v0.1.1
hooks:
- id: doctl-app-spec-validate-offline
args: ['--verbose']
```
Logs for the failed GitHub Actions workflow can be [viewed here][2].
Expand for relevant log sections from the failed GitHub Actions workflow
```console
Run digitalocean/action-doctl@v2.5.1
with:
no_auth: true
version: latest
env:
pythonLocation: /opt/hostedtoolcache/Python/3.11.13/x64
PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.13/x64/lib/pkgconfig
Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.13/x64
Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.13/x64
Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.13/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.13/x64/lib
Warning: API rate limit exceeded for 52.238.24.37. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Failed to retrieve latest version; falling back to: 1.98.1
/usr/bin/tar xz --warning=no-unknown-keyword -C /home/runner/work/_temp/164d3c69-a8c9-404e-8397-915de9b01fe8 -f /home/runner/work/_temp/de6a5c5e-4ddf-4bb5-88c1-25eda27a8815
>>> doctl version v1.98.1 installed to /opt/hostedtoolcache/doctl/1.98.1/x64
>>> Skipping doctl auth
```
```console
Run pre-commit/action@v3.0.1
Run python -m pip install pre-commit
Collecting pre-commit
Downloading pre_commit-4.3.0-py2.py3-none-any.whl.metadata (1.2 kB)
Collecting cfgv>=2.0.0 (from pre-commit)
Downloading cfgv-3.4.0-py2.py3-none-any.whl.metadata (8.5 kB)
Collecting identify>=1.0.0 (from pre-commit)
Downloading identify-2.6.13-py2.py3-none-any.whl.metadata (4.4 kB)
Collecting nodeenv>=0.11.1 (from pre-commit)
Downloading nodeenv-1.9.1-py2.py3-none-any.whl.metadata (21 kB)
Collecting pyyaml>=5.1 (from pre-commit)
Downloading PyYAML-6.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting virtualenv>=20.10.0 (from pre-commit)
Downloading virtualenv-20.34.0-py3-none-any.whl.metadata (4.6 kB)
Collecting distlib<1,>=0.3.7 (from virtualenv>=20.10.0->pre-commit)
Downloading distlib-0.4.0-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting filelock<4,>=3.12.2 (from virtualenv>=20.10.0->pre-commit)
Downloading filelock-3.19.1-py3-none-any.whl.metadata (2.1 kB)
Collecting platformdirs<5,>=3.9.1 (from virtualenv>=20.10.0->pre-commit)
Downloading platformdirs-4.3.8-py3-none-any.whl.metadata (12 kB)
Downloading pre_commit-4.3.0-py2.py3-none-any.whl (220 kB)
Downloading cfgv-3.4.0-py2.py3-none-any.whl (7.2 kB)
Downloading identify-2.6.13-py2.py3-none-any.whl (99 kB)
Downloading nodeenv-1.9.1-py2.py3-none-any.whl (22 kB)
Downloading PyYAML-6.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (762 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 763.0/763.0 kB 93.1 MB/s 0:00:00
Downloading virtualenv-20.34.0-py3-none-any.whl (6.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 156.5 MB/s 0:00:00
Downloading distlib-0.4.0-py2.py3-none-any.whl (469 kB)
Downloading filelock-3.19.1-py3-none-any.whl (15 kB)
Downloading platformdirs-4.3.8-py3-none-any.whl (18 kB)
Installing collected packages: distlib, pyyaml, platformdirs, nodeenv, identify, filelock, cfgv, virtualenv, pre-commit
Successfully installed cfgv-3.4.0 distlib-0.4.0 filelock-3.19.1 identify-2.6.13 nodeenv-1.9.1 platformdirs-4.3.8 pre-commit-4.3.0 pyyaml-6.0.2 virtualenv-20.34.0
Run python -m pip freeze --local
cfgv==3.4.0
distlib==0.4.0
filelock==3.19.1
identify==2.6.13
nodeenv==1.9.1
platformdirs==4.3.8
pre_commit==4.3.0
PyYAML==6.0.2
virtualenv==20.34.0
Run actions/cache@v4
Cache not found for input keys: pre-commit-3|/opt/hostedtoolcache/Python/3.11.13/x64|8edc02a8a73aedc615930d023e83036e854d0e6ab9c7f7c32a4e2647097b9c2b
Run pre-commit run --show-diff-on-failure --color=always --all-files
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.git.
[INFO] Initializing environment for https://github.com/igorshubovych/markdownlint-cli.
[INFO] Initializing environment for https://github.com/detailyang/pre-commit-shell.git.
[INFO] Initializing environment for https://github.com/LyraPhase/pre-commit-digitalocean.git.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.git.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/igorshubovych/markdownlint-cli.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Check for added large files................................................Passed
Check for merge conflicts..................................................Passed
Check for broken symlinks..............................(no files to check)Skipped
Check Yaml.................................................................Passed
Fix End of Files...........................................................Passed
Trim Trailing Whitespace...................................................Passed
markdownlint...............................................................Passed
Shell Syntax Check.....................................(no files to check)Skipped
DigitalOcean App Spec Validate Offline (schema-only).......................Failed
- hook id: doctl-app-spec-validate-offline
- exit code: 1
Error: Unable to initialize DigitalOcean API client: access token is required. (hint: run 'doctl auth init')
Error: Process completed with exit code 1.
```
An old version of `doctl` was used as fallback.
Thus, the `doctl app spec validate [...] --schema-only` flag did not exist, resulting in job failure.
[1]: https://github.com/LyraPhase/pre-commit-digitalocean/blob/5bc04eab049e71925723eb426b9961ce8a797f17/hooks/doctl-app-spec-validate-offline.sh
[2]: https://github.com/LyraPhase/docker-phasik.tv/actions/runs/17109231308/job/48526154689#step:4:13
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.