solo-io / solo-io/docs-link-checking
Separate links from autogenerated reference pages in the link report
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 0
- Forks
- 0
- Avg merge
- 16h 48m
- Merged PRs (30d)
- 1
Description
The weekly link report lists every broken link the same way: the URL, the status, and Found on: <page>. Nothing distinguishes a link a writer typed from a link that fell out of crd-ref-docs, helm-docs, or a CLI doc generator.
That matters because the two need different people. A writer triaging the report cannot fix a generated link at the source. The fix lives in a Go doc comment or a chart template in a product repo, or it is a placeholder URL that was never meant to resolve. For the writer working through their own PRs, those entries are noise they can neither act on nor close.
PR #2 is the shape of the problem. Four URL patterns, all from generated reference pages, and the PR's opening line is that "no content edit can fix them at the source." The only available resolution was to exclude them globally.
Where these come from
Generated reference content reaches the published pages several different ways across our sites:
- Helm value tables land in
assets/<product>-docs/pages/reference/helm/<version>/*.mdand are pulled into a hand-written wrapper page via{{< reuse >}}. For example,content/docs/standalone/latest/reference/helm/_index.mdin agentgateway-oss-website is three lines of front matter and tworeusecalls. - CRD reference from
crd-ref-docs, which is where thekubernetes.io/docs/reference/generated/kubernetes-api/...anchors in PR #2 come from. Every field documenting anObjectMetaor aDurationlinks there. - CLI reference, which
solo-projects'push-docs.yamlPRs straight into the docs hub underreference/cli/. - Endpoint examples quoted in Go doc comments, which lychee auto-links. PR #2's Azure and SAP patterns are both this.
The blocker: nothing marks generated content today
I checked the generated Helm snippets in agentgateway-oss-website and kgateway-oss. They carry no DO NOT EDIT header, no generated by line, and no front matter flag. So the first decision is how to recognize this content at all, and the three options are not equivalent:
a. Path patterns in this repo. A generated-path-patterns.txt alongside curl-retry-patterns.txt, matched against the source page path. Cheapest, changes nothing in the product or site repos.
Downside: it is per-page, and the wrapper pages are mixed. kgateway-oss/content/docs/envoy/main/reference/helm/_index.md is hand-written prose with a {{< link-hextra >}} link in it, sitting directly above a generated table. Path matching attributes the writer's own broken link to the generator.
b. Front matter flag on the wrapper page, surfaced as a meta tag in the built HTML so the report can read it. More honest than a path guess and docs-theme-extras could emit the tag centrally for every site at once. Still per-page, so it has the same mixed-wrapper problem, and it needs writers to remember to set it.
c. A marker in the generated snippet itself, rendered as an HTML comment that brackets the generated block. This is the only option that is per-link rather than per-page, so the intro paragraph stays the writer's and the table stays the generator's. It is also the most work, since the markers have to be added by generators living in several product repos.
I lean toward (a) as a first cut, because it is the only one that can land entirely in this repo and it would already cover the fully generated pages, which is most of the volume. Worth going in knowing it misattributes the wrapper pages, and treating (c) as the version to graduate to if the split proves useful.
What to do with them once identified
- A separate
## Generated reference pagessection in the report, placed below the writer-facing Errors, Warnings, and Redirects sections. - Counted on its own row in the summary table rather than folded into Errors, so the number a writer sees is the number they can act on.
- Ideally routed rather than just relocated: its own issue, its own label, or an assignee per generator.
PR-mode reports are mostly fine already, since generate-link-report.sh scopes those to changed files. This is a weekly-full-run problem.
Reasons this could be the wrong call
- A separate list that nobody owns is a list that rots. This is my main worry. Moving generated links into their own section makes the writer's list clean whether or not anyone ever fixes them, which removes the pressure that currently gets them fixed. I would not land the split without naming an owner per generator, otherwise this is a relabeling exercise.
- "Generated" does not mean "not the writer's problem." A typo'd URL in a Go doc comment is a real bug and a small PR to the product repo. The section should be framed as different ownership, not as a suppression list.
- It may overlap with #2's fix. If the exclude-list audit moves the blocked-host and redirect classes out of
excludeand into verification tiers, some of what looks like generated-page noise today resolves on its own, and the remainder may be small enough that a section header is more machinery than it is worth. Worth measuring the actual generated-page share of a weekly report before building anything. - A cheaper version exists: keep one list, and just annotate each entry with
(generated)when the source path matches. That gets a writer scanning the report the same signal for a fraction of the work, and it does not require deciding where the entries go or who owns them.
Related: #2.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading generate-link-report.sh and the existing curl-retry-patterns.txt, then compare the weekly full-run behavior with PR #2. Measure how many report entries come from generated reference pages before choosing between path patterns, wrapper-page metadata, generated-block markers, or annotations. Done means generated links are visibly separated or labeled, counted independently, and have a named ownership path without disrupting PR-mode reports.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell
- Domain
- documentation, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100