clearlydefined / clearlydefined/service
Cache unsuccessful harvests for a short period of time
- Dominant language
- TypeScript
- Stars
- 51
- Forks
- 46
- PR merge metrics
- No merged PRs in 30d
Description
## Context
We have been consuming Clearly Defined and noticed it takes longer than 1.5 min to process unharvested/invalid packages when we call both "notices" and "definitions" endpoints. Whenever a request exceeds 90 seconds, Cloudflare returns a 524 to client, this has some undesirable consequences for both client and ClearlyDefined service. Client will likely retry due to timeout in Clearly Defined, overwhelming more the service since it may still be processing the previous request.
## Solution
- If a package is not found in a public registry it should not harvest it (cache registry response when true for a long period of time, cache for a few minutes (10?) when response is false). Client retries for unknown packages should be quick, since ClearlyDefined will use the original cached value.
Contributor guide
Assessment
This issue has not been assessed yet.