cloudfour / cloudfour/lighthouse-parade
Why does sitemap.xml get crawled before the passed URL
- Dominant language
- TypeScript
- Stars
- 373
- Forks
- 15
- Avg merge
- 13h 36m
- Merged PRs (30d)
- 34
Description
Ex:
```
$ lighthouse-parade htts://www.baptistjax.com
Created CSV file
Starting the crawl...
Crawled https://www.baptistjax.com/sitemap.xml [text/xml] (646288 bytes)
Crawled https://www.baptistjax.com/ [text/html; charset=utf-8] (289135 bytes)
Report is done for https://www.baptistjax.com/
Wrote report for https://www.baptistjax.com/
Crawled https://www.baptistjax.com/services [text/html; charset=utf-8] (246368 bytes)
Report is done for https://www.baptistjax.com/services
Wrote report for https://www.baptistjax.com/services
Crawled https://www.baptistjax.com/site-search [text/html; charset=utf-8] (244996 bytes)
```
Notice that `sitemap.xml' is crawled before the url I requested. Why? Maybe this is some internal logic of simplecrawler?
Possibly related to #3
Contributor guide
Assessment
This issue has not been assessed yet.