ibm-granite / ibm-granite/docs
chore: ensure non-production builds emit noindex signals
Nobody has claimed this yet.
- Dominant language
- MDX
- Stars
- 0
- Forks
- 3
- Avg merge
- 1h 24m
- Merged PRs (30d)
- 1
Description
Background
The static robots.txt is a production artefact (Allow: /) that gets served on all builds — staging and forks alike. This is fine in practice today:
- Staging (
wwwstage.ibm.com) — Akamai injectsx-robots-tag: noindex, nofollowon every response, which takes precedence overrobots.txtfor all major crawlers - Forks (GitHub Pages) — no Akamai protection, so
Allow: /technically permits crawling of fork preview URLs, though in practice they won't be discovered
The subtle risk is that the noindex protection on staging is entirely Akamai-side and invisible in the repo. If the Akamai rule ever changes or a new non-IBM hosting environment is added (e.g. a partner staging domain), the Docusaurus build itself provides no protection.
Suggested fix
Inject a <meta name="robots" content="noindex"> tag for non-production builds via docusaurus.config.ts:
customFields: { noIndex: process.env.DOCS_SITE_URL !== 'https://www.ibm.com' },
And consume it in a swizzled <Head> or via the noIndex Docusaurus config option:
noIndex: process.env.DOCS_SITE_URL !== 'https://www.ibm.com',
This makes the noindex intent explicit in the codebase and independent of Akamai, with zero impact on production.
Related
- Raised during review of #67
- Ties into the staging strategy discussion noted in that PR review
🤖 Generated with Claude Code
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with docusaurus.config.ts and inspect how the site URL is provided to builds. Determine whether the Docusaurus noIndex option or a swizzled Head component fits the existing setup, then verify generated pages for production and non-production builds. Done means non-production pages emit a noindex signal while the production site does not.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- build-system, documentation
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100