source-cooperative / source-cooperative/data.source.coop
Support suffix byte range requests
Open
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 24
- Forks
- 6
- Avg merge
- 1h 32m
- Merged PRs (30d)
- 1
Description
The HTTP Range header has a form which allows clients to request the last N elements/bytes of a file without knowing ahead of time how large that file is. Suffix requests are required to read zarr v3 files which are "sharded".
Test case:
curl -H "Range: bytes=-10" https://data.source.coop/dynamical/noaa-gefs-forecast-35-day/v0.2.0.zarr/longitude/c/0 --silent | dd > /dev/null
This prints that 1374 bytes were copied but it should only return 10 bytes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the provided curl command and inspect the HTTP Range request handling in the Rust proxy. Focus on the suffix form, bytes=-10, and verify that the response for the given zarr URL contains only the final 10 bytes rather than 1374 bytes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100