developmentseed / developmentseed/titiler-multidim
Is the cache for opened datasets safe for dynamic mutable datasets?
- Dominant language
- Python
- Stars
- 88
- Forks
- 13
- Avg merge
- 2d 53m
- Merged PRs (30d)
- 10
Description
We have been caching the opened dataset view with redis - is this a safe thing to do if the dataset is changing over time? e.g. if we append a new time step to an Icechunk store. cc @maxrjones @abarciauskas-bgse @sharkinsspatial
https://github.com/developmentseed/titiler-multidim/blob/12337ae30e58600a183adfaee5f7e01fdce829cd/src/titiler/multidim/reader.py#L211-L218
It could make sense to rip the cache out for now if we can't safely invalidate it when a dataset gets updated.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at src/titiler/multidim/reader.py lines 211-218 and trace how the Redis cache stores the opened dataset view. Test the behavior when an Icechunk store gains a time step, then determine whether safe invalidation is possible or whether removing the cache is required. Done means the issue has a documented, verified resolution for changing datasets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, redis
- Domain
- backend, data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100