TanStack / TanStack/tanstack.com

fix(seo): make the sitemap canonical-only, healthy, and testable

Open
#1,153 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
TypeScript
Stars
1.1k
Forks
401
Avg merge
18h 14m
Merged PRs (30d)
56

Description

Problem

The production sitemap currently contains 3,245 URLs. A live audit found several categories of non-canonical or unhealthy entries:

  • 189 Charts catalog URLs use trailing slashes and immediately redirect to the no-trailing-slash canonical.
  • /showcase redirects to a default query-string URL before canonicalizing back to /showcase.
  • Four /drafts/ URLs and two README URLs are included.
  • Draft and README examples currently return 200 pages headed Content temporarily unavailable.
  • A stratified check of 160 sitemap entries found 23 redirects and one 404, including stale DB reference paths that redirect to a parent reference page or no longer resolve.
  • Only 77 entries currently contain lastmod values; docs and most other content do not.
  • Approved showcase detail pages are not included even though later showcase pagination is currently noindex/nofollow.

Google says sitemap entries should be the canonical URLs the site wants indexed: https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap

Relevant generation code: src/utils/sitemap.ts

Required changes

  • Emit only URLs that return 200, are indexable, and self-canonicalize.
  • Normalize Charts catalog entries to the production no-trailing-slash form.
  • Exclude drafts, *.draft paths, README files, temporary-unavailable placeholders, hidden libraries, and other non-public docs artifacts.
  • Apply docs redirect resolution before adding manifest paths to the sitemap.
  • Remove stale or missing generated-reference paths.
  • Reconcile /showcase so the sitemap entry does not require an avoidable redirect.
  • Add approved showcase detail pages if they are intended for organic discovery.
  • Keep query-string URLs only when they represent an intentionally indexable canonical page, such as the previous-partners directory.
  • Add accurate lastmod values where the source provides a real meaningful modification date; do not synthesize timestamps on every request.

Automated validation

Add a sitemap contract checker that can validate generated entries and optionally check a deployed base URL.

For every sitemap URL, verify:

  • Final status is 200 without following a redirect.
  • Exactly one canonical link exists.
  • Canonical equals the sitemap URL after consistent URL normalization.
  • No robots noindex directive is present.
  • The page is not a temporary-unavailable or error placeholder.
  • The URL has no disallowed draft, README, embed, account, or admin pattern.

Acceptance criteria

  • Production sitemap contains zero redirecting URLs.
  • Production sitemap contains zero 4xx or 5xx URLs.
  • Production sitemap contains zero noindex URLs.
  • Every entry is self-canonical.
  • Drafts and repository-maintenance documents are absent.
  • Charts catalog URLs use the same trailing-slash policy as their canonical pages.
  • Unit tests cover path filtering and canonical normalization.
  • A CI or release-time check prevents regressions in the sitemap contract.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/utils/sitemap.ts and review how manifest paths, redirects, showcase pages, and lastmod values enter the generated sitemap. Add the requested sitemap contract validation and unit coverage for path filtering and canonical normalization; done means production entries are 200, indexable, self-canonical, non-placeholder URLs with no disallowed artifacts.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
documentation, testing, web-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.