clearlydefined / clearlydefined/service

Cache unsuccessful harvests for a short period of time

Open
#877 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
51
Forks
46
PR merge metrics
No merged PRs in 30d

Description

## Context
We have been consuming Clearly Defined and noticed it takes longer than 1.5 min to process unharvested/invalid packages when we call both "notices" and "definitions" endpoints. Whenever a request exceeds 90 seconds, Cloudflare returns a 524 to client, this has some undesirable consequences for both client and ClearlyDefined service. Client will likely retry due to timeout in Clearly Defined, overwhelming more the service since it may still be processing the previous request.

## Solution

- If a package is not found in a public registry it should not harvest it (cache registry response when true for a long period of time, cache for a few minutes (10?) when response is false). Client retries for unknown packages should be quick, since ClearlyDefined will use the original cached value.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.