Expand prerender crawling to improve DX

Describe the problem

Some common routes are not considered during crawling for pre-rendering, leading to avoidable errors for developers that they need to manually resolve by updating their svelte.config.js entries array.

Reducing these errors would smooth the developer experience and help things "just work", for more common scenarios.

Describe the proposed solution

1. If link[rel=alternate] exists with the page HTML, include this href as a route to potentially prerender (e.g.<link rel="alternate" href="/blog/atom.xml" type="application/atom+xml" />). (Update: this one is good and crawled already.)
2. If /robots.txt exists during the crawl, split lines and get the URL from: Sitemap: https://example.com/sitemap.xml. A valid sitemap line robots.txt will always use this exact format of Site: <fully qualified URL>. 1.) prerender the sitemap path itself if marked for prerendering, 2.) parse sitemap.xml urls to evaluate for prerendering. (/robots.txt itself is already prerendered even without updating svelte.config.js.)

<?xml version="1.0" encoding="UTF-8" ?>
  <urlset xmlns="https://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <changefreq>daily</changefreq>
    <priority>0.7</priority>
  </url>
  <url>
    <loc>https://example.com/about</loc>
    <changefreq>daily</changefreq>
    <priority>0.7</priority>
  </url>
  <url>
    <loc>https://example.com/contact</loc>
    <changefreq>daily</changefreq>
    <priority>0.7</priority>
  </url>
</urlset>

Alternatives considered

No response

Importance

nice to have

Additional Information

No response

Start by tracing the prerender crawl behavior described in the issue, then inspect how svelte.config.js entries, /robots.txt, and sitemap.xml are handled. Done means sitemap URLs discovered through a valid robots.txt Sitemap line are evaluated for prerendering, with the sitemap path itself handled as specified.