TanStack / TanStack/router

Static prerendering: Duplicate crawled/sitemap entries for nested index pages

Open
#6,978 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
15.1k
Forks
1.9k
Avg merge
1d 20h
Merged PRs (30d)
143

Description

Which project does this relate to?

Start

Describe the bug

Example route structure:

├── blog
│   ├── $slug.tsx
│   └── index.tsx
├── contact
│   └── index.tsx
├── index.tsx
└── __root.tsx

For /blog/index.tsx and /contact/index.tsx page routes, when building with static prerendering and sitemaps enabled, they are crawled twice and result in duplicate sitemap entries.

[prerender] Prerendering pages...
[prerender] Concurrency: 12
...
[prerender] Crawling: /blog/
[prerender] Crawling: /contact/
[prerender] Crawling: /blog
[prerender] Crawling: /contact
...
[prerender] Prerendered 8 pages:
...
[prerender] - /blog/
[prerender] - /contact/
[prerender] - /blog
[prerender] - /contact
...
[sitemap] Building Sitemap...

These routes do not have any layout routes, so only the page routes with no trailing slashes must be crawled and added to the sitemap. This trailing-slash consistency is important for SEO.

Router is configured for no trailing slashes (which is also the default afaik) so this should be followed for the output:
https://github.com/dotnize/start-static-test/blob/main/src/router.tsx#L19

Same behavior with

blog.index.tsx
contact.index.tsx

but this route structure works fine. These will only be crawled once, but only works well for routes that don't have child routes:

blog.tsx // breaks /blog/$slug
contact.tsx // OK for /contact, since this has no child routes

With that structure, I assume blog.tsx is treated as a layout route, so blog/$slug.tsx contents are never rendered since blog.tsx doesn't have an Outlet.

Your Example Website or App

https://github.com/dotnize/start-static-test

Steps to Reproduce the Bug or Issue
  1. pnpm install
  2. pnpm build
  3. See build logs, and dist/client/sitemap.xml. Duplicate entries for /blog and /contact are generated, one with a trailing slash, and one without
Expected behavior

Only one entry should be generated for each page, for structures like blog.index.tsx or blog/index.tsx

Screenshots or Videos

No response

Platform
  • Router / Start Version: 1.167.5 / 1.166.17
  • OS: Fedora Linux 43
  • Bundler: Vite
  • Bundler Version: 8.0.1
Additional context

Workarounds:

  • As mentioned above, a flat route structure like contact.tsx works fine if it doesn't have child routes, otherwise it will get treated like a layout route.
  • For routes that have child routes like /blog, These pages can be manually excluded in the Start Vite plugin
    {
      // Workaround to exclude "/blog/" from sitemap, so we only get "/blog"
      path: "/blog/",
      prerender: { enabled: false },
      sitemap: { exclude: true },
    },
    
    ref: https://github.com/dotnize/start-static-test/blob/main/vite.config.ts#L35-L40

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with pnpm install and pnpm build in the linked start-static-test example, then inspect src/router.tsx, vite.config.ts, and dist/client/sitemap.xml. Compare the /blog and /contact entries for both trailing-slash forms; done means each page is crawled once and the sitemap contains only the configured no-trailing-slash URL.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript, vite
Domain
build-system, web-dev
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.