solo-io / solo-io/docs-theme-extras
selectedPages() is never called, so 17 specs silently no-op for any consumer without [[pages]]
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 0
- Avg merge
- 18h 13m
- Merged PRs (30d)
- 20
Description
Summary
selectedPages() in tests/helpers/crawl.ts exists to fall back to a crawl of builtRoot when a consumer declares no [[pages]] entries. It is exported, documented, and never called by anything:
$ grep -rn "selectedPages" . --exclude-dir=node_modules
tests/helpers/crawl.ts:56:export function selectedPages(): CrawledPage[] {
Its own comment names the specs that are supposed to use it:
Return only the explicitly listed pages from CONFIG, falling back to the crawl when no
[[pages]]entries are declared. Specs that test specific behaviors against representative pages (contrast, viewport, browser specs) use this; specs that scan everything (content, static) callcrawlBuiltRoot.
All three named specs read target.pages raw instead. So do 14 others — 17 in total:
alert-body, back-to-top, brand, browser, contrast, cross-browser, mermaid-render, mobile-drawer, openapi-example-contrast, openapi-method-colors, page-feedback, reuse-image-dark-pair, sidebar-rail, theme-toggle, toc-layout, version-banner-link, viewport.
Impact
For a consumer with no [[pages]] in its config, target.pages is [] (config.ts:295 maps over pagesData ?? []). Every one of those 17 specs then either hits a skip guard or runs against an empty sample. The skip is worded as a benign configuration note, so it reads as "not applicable here" rather than "this check is off":
// tests/alert-body.spec.ts
const SAMPLE = target.pages.filter((p) => target.versionOf(p.url) !== null);
const PAGE = SAMPLE[0]?.url ?? null;
test.skip(PAGE === null, "no non-landing pages configured");
kgateway.dev is in exactly this state — its .docs-test.toml declares zero [[pages]], so all 17 have been skipping since the harness was adopted there. I would guess other consumers are too; the config is easy to adopt without the optional block.
The fallback was clearly written to prevent precisely this. It just never got wired in.
Suggested fix
Swap the raw reads for selectedPages() in the specs whose comments already say they use it, at minimum the three it names. That makes the no-[[pages]] case degrade to "sample from the crawl" instead of "silently do nothing", and it needs no consumer-side change.
Worth considering alongside it: a guard that fails when a spec's page sample comes back empty, in the spirit of tests/scan-roots.spec.ts treating a scan root that reads nothing as an error rather than a pass. A skip that means "the harness found no input" and a skip that means "this consumer opted out" should not look the same in the report.
How this surfaced
Found while reviewing kgateway-dev/kgateway.dev#978, which fixes a related class of silent skip: .docs-test.toml had drifted from versions.json, so the per-version specs never checked 2.3.x and the version dropdown spec chased a 2.0.x tree that no longer builds.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/helpers/crawl.ts and the three specs named in its comment, then trace config.ts:295 to understand the empty target.pages case. Review the listed specs for raw page reads and compare their behavior with selectedPages(). Done means consumers without [[pages]] obtain crawled sample pages instead of silently skipping or testing an empty sample.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing-qa, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100