developmentseed / developmentseed/zarrista
Is it a problem to reconstruct & have two icechunk sessions
- Dominant language
- Python
- Stars
- 45
- Forks
- 2
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 7
Description
It's not possible to share a Rust object across two Rust-Python libraries that were compiled independently.
This means that for Zarrista to natively work with icechunk as a store (where zarrista manages icechunk from Rust without going through Python) Zarrista has to serialize the icechunk state from the icechunk python library and create a new icechunk session in the statically-compiled icechunk version in zarrista.
Unlike obstore, Icechunk is not stateless, so perhaps there could be bad things that happen if two different sessions are trying to work on the same "repo" at the same time?
I think this session serialization may be the same process done when an icechunk session is pickled, so _hopefully_ it won't be a problem, but I don't know.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Begin by tracing the session serialization used between the icechunk Python library and zarrista's statically compiled Rust version, then compare it with icechunk session pickling. Done means establishing whether reconstructed sessions can safely operate on the same repository concurrently and recording the conclusion or required follow-up.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100