lipp / lipp/likes

Prevent being blacklisted and/or report exceptions

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
17
Forks
5
PR merge metrics
No merged PRs in 30d

Description

> As the respective page layout may change, this approach is pretty fragile. Keep in mind that the respective platform may rate-limit / blacklist you in case of "excessive" usage.
Source: [readme](https://github.com/lipp/likes/blob/master/README.md)

I create unofficial APIs using _request_ and _cheerio_ all the time in the exact same way. What you can do to prevent getting blacklisted is cache the response body to the local file system and set a minimum delay time required before fetching it again. If cache is less than x period old, resolve local file and otherwise fetch it again and overwrite. This also protects the user if he or she is stupid and the script accidentally gets executed in a loop.

You don't want to periodically check if all of your modules like this 'still work'. You can implement a sort of event handler/callback at the end of your HTTP request logic that will report back to you if something failed. In other words, if successful do nothing but otherwise send the error somewhere using webhooks or websockets or something. Whether you want this depends on the situation or should be optional I guess but in this way when something changes that breaks your script, you will know about it in seconds from all users and applications using it.

Contributor guide

No contributing guide indexed for this repository

Research direction

Review the README and the existing request/cheerio HTTP path first. Clarify whether response caching with a minimum age and optional failure reporting are both required, then define how success and exceptions should be exposed. Done should protect against repeated fetches and provide a documented way to notice failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.