developmentseed / developmentseed/zarrista
Extend `EncodedChunk` abstraction to write side
- Dominant language
- Python
- Stars
- 45
- Forks
- 2
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 7
Description
Have
```py
Array.encode_chunk() -> EncodedChunk
```
which takes in chunk indices and a typed, decoded array (#47).
Or rather, because chunk encoding will be either sync (current thread) or async (on a Rust thread pool), maybe have a separate `ChunkEncoder` class.
Or, just put encoding onto `CodecChain`, so we have `CodecChain.encode` and `CodecChain.encode_async`.
- We probably want a way to encode/decode chunks that is independent of a store.
- It seems a little messy to put sync/async methods for chunk encoding onto `Array`/ `AsyncArray`, when the sync/async bridge of chunk encoding describes _where the CPU bound work is happening_, not what kind of IO is happening
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing the mentioned Array, AsyncArray, EncodedChunk, ChunkEncoder, and CodecChain abstractions and how chunk encoding should remain independent of a store. Resolve whether encoding belongs on a separate encoder or CodecChain, and define the sync and async API that satisfies the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- backend-api-design, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100