dandi / dandi/dandi-archive

evaluate kerchunk as streamable API endpoint

Open
#918 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
26
Forks
21
Avg merge
4d 23h
Merged PRs (30d)
15

Description

it would be sweet if we can use [kerchunk](https://fsspec.github.io/kerchunk/) to provide a single streamable API endpoint for any given data type. this could open up significant web viz capabilities.

caveat: like our zarr endpoint we need to ensure that the bytes are sent directly from s3 rather than our API server.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the existing zarr endpoint and the linked kerchunk documentation. Determine the endpoint scope, supported data types, and how S3 can serve bytes directly rather than routing them through the API server. Done means a defined streamable endpoint design that supports the intended web-visualization use case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.