Warped chunk caching

We recently had to give some presentations to members of the GIBs and EGIS teams talking about edge caching for improving the performance and scalability of some our tiling services. Two main points we noted are that caching dynamic, mosaic tiles is highly inefficient for 2 reasons

• The very large numbers of tile permutations based on query, band ordering and colormap application.
• The difficulty of cache invalidation when upstream granules are added or altered.

You can read more detail about these issues in https://github.com/developmentseed/chunk_caching_presentation.

In discussions about caching for titiler-cmr @hrodmn had proposed the idea of data layer caches using serialized numpy arrays to address the tile permutation issue. The warped chunk cache is the natural Zarrified extension of this concept.

Let's use a collection like HLS as an example. Granules use a variety of UTM CRSs so for almost all visualization use cases (and analysis use cases which cross UTM zones) the source chunks will need to be warped to a CRS. Currently when we dynamically tile data we pay this warping cost for every request. Instead, we want the first user to access a chunk to pay the warping penalty but save the warped result so that subsequent requesters don't need to.

We start with a chunk cache Zarr array that uses the target CRS (web mercator), covers the full extent of the collection and uses a smaller reasonably aggregated time dimension (which we can incrementally extend going forward). This array is initialized with null values. When a tiling request fetches a chunk, and the chunk is null it attempts to use an index (zarr-datafusion-search, CMR, STAC) to locate the appropriate source chunks. It then applies the warping operation and writes the chunk as a native chunk to the chunk cache array.

In this way, we incrementally build a set of warped, native chunks based on request traffic. This same concept can also be applied to building multi-resolution pyramids. Rather than exhaustively pre-generating pyramids for data which might not ever be accessed, we can have pyramid chunks calculated on demand. Obviously there are scaling and latency limitations for this approach and we will likely still need to pre-generate pyramids that require many input source chunks for each target chunk.

From a cache invalidation perspective, if we have virtual chunk representations of all our source data it should be relatively simple to invert the process we use for looking up source chunks. When a new virtual chunk is created or an existing one is updated it broadcasts a notification and the warped chunk cache writer can respond by updating the warped chunk cache.

The issue names no repository files, tests, or entry points. Start by reviewing the linked chunk-caching presentation and locating the existing tiling, Zarr, and source-chunk lookup code. Done would require an agreed design and an implemented warped chunk cache with cache population, invalidation, and scaling behavior defined.