canonical / canonical/sphinx-stack
Link check rate limit causing delay
- Dominant language
- No language data
- Stars
- 37
- Forks
- 77
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 3
Description
I've been asked by an engineer from a big project on what can cause a very long duration (>1 hour) of documentation testing.
It seems like most time is used by the link check test: [example](https://github.com/canonical/charmed-kubeflow-documentation/actions/runs/19890125455/job/57006627693?pr=26).
Tests duration:
Timestamps for the link check:
Looks like it could be that the rate limit is causing a 1-hour sleep.
I'm not sure what caused the rate limit, most likely - an abundance of links to GitHub resources, but I have an idea of what could be done to decrease the load during testing.
If we use the `sphinx-build` command to test links, we can limit its scope to specified files.
```shell
sphinx-build -b linkcheck srcdir outdir path/to/target1.rst path/to/target2.rst ...
```
See the [Run make linkcheck on only changed files](https://github.com/sphinx-doc/sphinx/issues/10643#top) Sphinx issue discussion for more context.
**We can limit it to only check the changed files in a PR**. That alone should improve the situation a lot.
As for checking the rest of the links (because they can become broken without changing in our docs) - we can do a full sweep on schedule, once a week, or month. We can even allocate weekdays and time slots to distribute it to avoid our line of products/repositories to avoid doing it at the same time (assuming that can create some problems).
I think this double approach is more reasonable than doing a full sweep for each commit to each PR with docs, given that we are experiencing rate limits.
Contributor guide
Assessment
This issue has not been assessed yet.