influxdata / influxdata/docs-v2
AI visibility: legacy-URL hygiene — ~507 stale paths in GSC 'Crawled, currently not indexed'
- Dominant language
- JavaScript
- Stars
- 82
- Forks
- 326
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 82
Description
## Goal
Resolve the legacy/stale URLs that dominate the Google Search Console **"Crawled – currently not indexed"** report, so Googlebot stops spending crawl budget on dead paths and the report reflects only current content.
Parent: #7230 (Phase 0 — AI visibility)
Related: #7245 (canonical audit — covers the *duplicate current-content* subset, distinct from this legacy-URL subset)
## Context
A GSC coverage drilldown export (2026-05-28, **1,006 affected URLs** under "Crawled – currently not indexed") breaks down so that the single largest driver is **legacy URLs from past restructures**, not current-content problems. ~507 of the 1,006 are stale paths:
| Legacy category | Count | Example |
| --- | ---: | --- |
| `/influxdb/v2.x/` point-version paths (current: `/influxdb/v2/`) | 196 | `/influxdb/v2.1/tags/schema` |
| pre-v3 `/influxdb//` (moved to `/influxdb3/...`) | 172 | `/influxdb/cloud-dedicated/api/v2/` |
| `/influxdb/v1.x/` point-version paths (current: `/influxdb/v1/`) | 74 | `/influxdb/v1.4/troubleshooting/statistics/` |
| top-level `/v2.0/` (pre-`/influxdb/` namespace) | 51 | `/v2.0/reference/flux/functions/built-in/transformations/aggregates/difference/` |
| `/influxdb/cloud-iox/` (renamed to `cloud-serverless`) | 14 | `/influxdb/cloud-iox/reference/cli/influx/user/password/` |
| **Total legacy** | **~507** | |
(The remainder of the 1,006: ~435 current pages — handled by #7245 plus normal discretionary "crawled, not indexed"; ~81 thin `tags/` taxonomy pages; ~28 non-HTML assets `.yml`/`.json`/`/downloads/` that shouldn't be indexed.)
## Findings from the repo
- **None of these legacy paths exist in current content** (`content/influxdb/cloud-serverless/`, `content/influxdb/v2.0/`, `content/influxdb/cloud-iox/`, etc. are all absent).
- **Alias coverage is minimal**: ~1 current file aliases the pre-v3 distributed paths, **0** alias `/v2.0/`, **0** alias `cloud-iox`, ~1 aliases `/influxdb/v2.x/` point versions.
- There is **no `netlify.toml` or `_redirects`** in the repo; redirects are handled entirely by Hugo `aliases:` frontmatter (736 pages currently use it).
- Conclusion: these URLs most likely **404 today** (no content + no alias → Hugo emits nothing). They linger in this report because Google crawled them previously; they need either a 301 to a live equivalent or an explicit 410, plus removal from the sitemap.
## Acceptance criteria
- [ ] Confirm the live HTTP status (200 / 301 / 404 / 410) for a representative sample from each category above — this determines per-category remediation.
- [ ] For each category, decide the target:
- **301** to the current equivalent where one exists (e.g. `/influxdb/cloud-dedicated/...` → `/influxdb3/cloud-dedicated/...`; `/influxdb/cloud-iox/...` → `/influxdb/cloud-serverless/...` *(verify — cloud-iox predates the v3 move)*; `/influxdb/v2.7/...` → `/influxdb/v2/...`)
- **410 Gone** where no equivalent exists (truly removed pages)
- [ ] Implement the redirects. Note the repo has no blanket redirect mechanism today — decide between (a) bulk Hugo `aliases:` on current pages, or (b) introducing a server/CDN redirect-rule layer for version-prefix patterns (cleaner for the hundreds of point-version paths than per-page aliases).
- [ ] Prune dead/legacy URLs from the generated sitemap so they stop being submitted for crawl.
- [ ] Re-run the GSC export after deploy and confirm the legacy bucket shrinks.
## Out of scope
- Duplicate **current** content / canonical consolidation of v3 query guides — tracked in #7245.
- Discretionary "crawled, currently not indexed" on healthy current pages (e.g. `/influxdb/v2/api/query-data/`, which GSC self-canonicals and fetches successfully) — monitor; no code fix.
- Source export: `https___docs.influxdata.com_-Coverage-Drilldown-2026-05-28` (GSC, issue = "Crawled - currently not indexed", sitemap = "All known pages").
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by checking the current Hugo content frontmatter, existing aliases, generated sitemap, and representative live HTTP statuses for each legacy URL category. Decide per category between redirects and 410 responses, then implement the chosen redirect mechanism and remove dead paths from the sitemap. Done means the deployed statuses and sitemap are verified, followed by a GSC export confirming that the legacy bucket has shrunk.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hugo
- Domain
- documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100