ontola / ontola/atomic-server

Server as client - cache invalidation policy

Open
#583 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
1.6k
Forks
82
Avg merge
10h 34m
Merged PRs (30d)
205

Description

The server retrieves and stores resources in (at least) the following scenarios

  • Agent key validation
  • Property & Class validation
  • /path endpoint explicitly finding specific version

But this can result in a couple of errors:

  • Agent key has changed, but server gives error due to having old key
  • Property datatype has changed (don't do this, though!)

How do we deal with cache invalidation on server-as-a-client resources? A couple of ideas:

Store fetched_at in Resource as property

  • Changes contents of resource (is that OK?). Users might manually set this property to some time in the future, which would perhaps always trigger. Maybe that opens up attacks. I dont know
  • More information to end user

Store fetched_at in Resource struct as bytes

  • Not available in JSON-AD serialization
  • Requires migration of store

Store fetched_at in new Tree

  • Does not require a migration
  • Requires an additional lookup if the fetched_at is needed.

Have /invalidate endpoint

  • Client determines when fetching again is needed
  • Is slower, because it now takes a bunch of round trips

Add max_age param to get_resource

  • If the max_age is higher then the difference in fetched_at, we re-fetch.
  • Could still result in errors staying quite a while if we use a high max_age, which should probably be default for many types of gets.

Re-fetch on specific errors

  • e.g. property of class validation
  • To prevent doing this 1000s of times, we set a fetched_at and set max_age to something like 10 minutes.

Current preference

Store fetched_at in Resource as bytes

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the Resource storage model and the get_resource flow, including the /path endpoint and the listed validation cases. Compare the proposed fetched_at approaches, including storage as bytes, and define how stale resources should be detected and refreshed. Done requires an agreed policy that addresses changed agent keys and property datatypes without excessive repeated fetching.

Written by the indexing model from the issue text.

Assessment

Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.