cloudfour / cloudfour/lighthouse-parade
add option to limit number of pages crawled?
- Dominant language
- TypeScript
- Stars
- 373
- Forks
- 15
- Avg merge
- 13h 36m
- Merged PRs (30d)
- 34
Description
Hey this is a cool tool. Here's a feature idea assuming you're open to it.
Currently one can use:
```
--max-crawl-depth 2
```
to get, say, index page + 1st linked pages.
But maybe there are a lot of linked pages, and you just need a representative sample which is more than 1 page but not tons of pages.
So maybe another option like:
```
--max-crawled-pages N # or just --max-pages ?
```
and the crawler stops after it exhausts all pages allowed by other options, or N, whichever is smaller.
One might then use it like:
```
lighthouse-parade --max-crawl-depth 2 --max-crawled-pages 20 example.com
```
Contributor guide
Research direction
Start by locating the existing --max-crawl-depth option and the crawler loop that applies its limits. Add a page-count option whose value is the smaller of the requested limit and the pages allowed by other options, then verify that crawling stops at that limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100