Prevent being blacklisted and/or report exceptions
- Dominant language
- JavaScript
- Stars
- 17
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
> As the respective page layout may change, this approach is pretty fragile. Keep in mind that the respective platform may rate-limit / blacklist you in case of "excessive" usage.
Source: [readme](https://github.com/lipp/likes/blob/master/README.md)
I create unofficial APIs using _request_ and _cheerio_ all the time in the exact same way. What you can do to prevent getting blacklisted is cache the response body to the local file system and set a minimum delay time required before fetching it again. If cache is less than x period old, resolve local file and otherwise fetch it again and overwrite. This also protects the user if he or she is stupid and the script accidentally gets executed in a loop.
You don't want to periodically check if all of your modules like this 'still work'. You can implement a sort of event handler/callback at the end of your HTTP request logic that will report back to you if something failed. In other words, if successful do nothing but otherwise send the error somewhere using webhooks or websockets or something. Whether you want this depends on the situation or should be optional I guess but in this way when something changes that breaks your script, you will know about it in seconds from all users and applications using it.
Contributor guide
No contributing guide indexed for this repository
Research direction
Review the README and the existing request/cheerio HTTP path first. Clarify whether response caching with a minimum age and optional failure reporting are both required, then define how success and exceptions should be exposed. Done should protect against repeated fetches and provide a documented way to notice failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100