Server as client - cache invalidation policy
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.6k
- Forks
- 82
- Avg merge
- 10h 34m
- Merged PRs (30d)
- 205
Description
The server retrieves and stores resources in (at least) the following scenarios
- Agent key validation
- Property & Class validation
/pathendpoint explicitly finding specific version
But this can result in a couple of errors:
- Agent key has changed, but server gives error due to having old key
- Property datatype has changed (don't do this, though!)
How do we deal with cache invalidation on server-as-a-client resources? A couple of ideas:
Store fetched_at in Resource as property
- Changes contents of resource (is that OK?). Users might manually set this property to some time in the future, which would perhaps always trigger. Maybe that opens up attacks. I dont know
- More information to end user
Store fetched_at in Resource struct as bytes
- Not available in JSON-AD serialization
- Requires migration of store
Store fetched_at in new Tree
- Does not require a migration
- Requires an additional lookup if the
fetched_atis needed.
Have /invalidate endpoint
- Client determines when fetching again is needed
- Is slower, because it now takes a bunch of round trips
Add max_age param to get_resource
- If the
max_ageis higher then the difference infetched_at, we re-fetch. - Could still result in errors staying quite a while if we use a high
max_age, which should probably be default for many types ofgets.
Re-fetch on specific errors
- e.g. property of class validation
- To prevent doing this 1000s of times, we set a
fetched_atand setmax_ageto something like 10 minutes.
Current preference
Store fetched_at in Resource as bytes
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the Resource storage model and the get_resource flow, including the /path endpoint and the listed validation cases. Compare the proposed fetched_at approaches, including storage as bytes, and define how stale resources should be detected and refreshed. Done requires an agreed policy that addresses changed agent keys and property datatypes without excessive repeated fetching.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100