canonical / canonical/starbase

Docs: Implement an alternative link checker

Open
#417 1 comment 0 reactions 1 assignee Claimed by @medubelko View on GitHub
Status: Triaged
Dominant language
Makefile
Stars
13
Forks
6
Avg merge
23h 56m
Merged PRs (30d)
9

Description

In a recent [Rockcraft build](https://github.com/canonical/rockcraft/actions/runs/16355474142/job/46217812410?pr=921#step:6:5734), the sphinx-build link checker spent six hours trying to reach a link to [gnu.org](https://gnu.org). I've tried every timeout-related config setting available in the configuration, but none seem to stop it from infinitely looping.

I think it's caused by the FSF's recent work walling themselves off from [DDoSing by LLM agents](https://www.fsf.org/blogs/sysadmin/our-small-team-vs-millions-of-bots). Something about the response causes it to keep trying, possibly because of a limitation with [`Retry-After`](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-linkcheck_rate_limit_timeout). Or, Sphinx's user agent might be blocked now. Or, FSF is sending bots into a maze.

Regardless of what other domains are doing, it shouldn't bog our builds. If other savvy sites do likewise in the future, we don't want to be surprised by blocked builds.

Sphinx's link checker has always been slow and its capabilities meagre. I'd like to pursue other link checking solutions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.