ampproject / ampproject/amp-toolbox

[linter] Issues when scripting linter for batch processing

Open
#775 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
459
Forks
242
PR merge metrics
No merged PRs in 30d

Description

Attempting to run linter as a batch process, I am running into a couple of errors,

1. Consistently running into the following error

URL: https://sports.ndtv.com/webstories/sports/teams-players-that-created-history-in-premier-league-93
Status: FAIL
Message: [publisher-logo-src] (https://c.ndtvimg.com/gws/93/assets/14.png) error: {"errno":-110,"code":"ETIMEDOUT","syscall":"connect","address":"184.26.81.182","port":443}
Status: FAIL
Message: [poster-portrait-src] (https://c.ndtvimg.com/gws/93/assets/16.jpeg) error: {"errno":-110,"code":"ETIMEDOUT","syscall":"connect","address":"184.26.81.182","port":443}

If I were to run this individually, I am able to successfully retrieve both URLs. This happens after about 10-12 URLs are successfully processed so I wonder if there is some socket issue?

2. Once again, after a few URLs are processed, one of the calls fails with,

couldn't load [https://khabar.ndtv.com/webstories/entertainment/ticket-to-bollywood-tv-actor-to-bollywood-stars-shah-rukh-khan-irrfan-hina-138] [debug: curl -sS -i -H 'user-agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' 'https://khabar.ndtv.com/webstories/entertainment/ticket-to-bollywood-tv-actor-to-bollywood-stars-shah-rukh-khan-irrfan-hina-138']

The curl run manually seems to work just fine as does running the URL by itself through the linter. This could potentially be some socket issue as well.

Any suggestions / recommendations would greatly help!

Contributor guide

Open the contributing guide

Research direction

No source files or tests are named. Start by reproducing batch and individual linter runs with the two reported URLs and compare them with the successful curl command; done means batch processing no longer produces the timeout or load failures.

Written by the indexing model from the issue text.

Assessment

Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.