ampproject / ampproject/amp-toolbox
[linter] Issues when scripting linter for batch processing
- Dominant language
- HTML
- Stars
- 459
- Forks
- 242
- PR merge metrics
- No merged PRs in 30d
Description
Attempting to run linter as a batch process, I am running into a couple of errors,
1. Consistently running into the following error
URL: https://sports.ndtv.com/webstories/sports/teams-players-that-created-history-in-premier-league-93
Status: FAIL
Message: [publisher-logo-src] (https://c.ndtvimg.com/gws/93/assets/14.png) error: {"errno":-110,"code":"ETIMEDOUT","syscall":"connect","address":"184.26.81.182","port":443}
Status: FAIL
Message: [poster-portrait-src] (https://c.ndtvimg.com/gws/93/assets/16.jpeg) error: {"errno":-110,"code":"ETIMEDOUT","syscall":"connect","address":"184.26.81.182","port":443}
If I were to run this individually, I am able to successfully retrieve both URLs. This happens after about 10-12 URLs are successfully processed so I wonder if there is some socket issue?
2. Once again, after a few URLs are processed, one of the calls fails with,
couldn't load [https://khabar.ndtv.com/webstories/entertainment/ticket-to-bollywood-tv-actor-to-bollywood-stars-shah-rukh-khan-irrfan-hina-138] [debug: curl -sS -i -H 'user-agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' 'https://khabar.ndtv.com/webstories/entertainment/ticket-to-bollywood-tv-actor-to-bollywood-stars-shah-rukh-khan-irrfan-hina-138']
The curl run manually seems to work just fine as does running the URL by itself through the linter. This could potentially be some socket issue as well.
Any suggestions / recommendations would greatly help!
Contributor guide
Research direction
No source files or tests are named. Start by reproducing batch and individual linter runs with the two reported URLs and compare them with the successful curl command; done means batch processing no longer produces the timeout or load failures.
Written by the indexing model from the issue text.
Assessment
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100