carbon-design-system / carbon-design-system/devtools
Web crawler for reporting
Open
enhancement
- Dominant language
- JavaScript
- Stars
- 46
- Forks
- 23
- PR merge metrics
- No merged PRs in 30d
Description
If we can build a web crawler to validate and report back we can provide some major value to the carbon teams. We've already built a web crawler, but the issue I came across was data. I think having Carbon telemetry #11 will help us in this area. Then we need a way to add new links and manage the queue of links to validate.
Assuming IBM.com has 30 million pages and we'd like to roughly check each URL once a month then we need to be able to process roughly 12 URLs per second give or take.
Things to consider...
- [ ] Database need to upload reports #11
- [ ] URL Queue
- [ ] Interface to add and manage URL Queue
Contributor guide
Assessment
This issue has not been assessed yet.