developmentseed / developmentseed/titiler-multidim

Is the cache for opened datasets safe for dynamic mutable datasets?

Open
#136 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
88
Forks
13
Avg merge
2d 53m
Merged PRs (30d)
10

Description

We have been caching the opened dataset view with redis - is this a safe thing to do if the dataset is changing over time? e.g. if we append a new time step to an Icechunk store. cc @maxrjones @abarciauskas-bgse @sharkinsspatial

https://github.com/developmentseed/titiler-multidim/blob/12337ae30e58600a183adfaee5f7e01fdce829cd/src/titiler/multidim/reader.py#L211-L218

It could make sense to rip the cache out for now if we can't safely invalidate it when a dataset gets updated.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at src/titiler/multidim/reader.py lines 211-218 and trace how the Redis cache stores the opened dataset view. Test the behavior when an Icechunk store gains a time step, then determine whether safe invalidation is possible or whether removing the cache is required. Done means the issue has a documented, verified resolution for changing datasets.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, redis
Domain
backend, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.