developmentseed / developmentseed/zarrista
Add free-threaded tests with pytest-run-parallel
- Dominant language
- Python
- Stars
- 45
- Forks
- 2
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 7
Description
Follow up from https://github.com/developmentseed/zarrista/pull/69
----
Claude:
## Follow-up: free-threaded (3.14t) CI testing
PR #69 adds the **wheel builds** for free-threaded CPython 3.14 (`cp314-cp314t`)
and scaffolds the abi3t (3.15+) wheel. It intentionally does **not** add CI that
*tests* zarrista on a free-threaded interpreter — this issue tracks that work,
which turned out to need a test-suite restructure rather than a CI one-liner.
### Why it was deferred
Spiking the 3.14t test job surfaced three blockers:
1. **`numcodecs` re-enables the GIL.** `numcodecs.blosc` hasn't declared that it
can run without the GIL, so importing it on 3.14t silently turns the GIL back
on:
> `RuntimeWarning: The global interpreter lock (GIL) has been enabled to load
> module 'numcodecs.blosc' ...`
Since nearly all tests pull in zarr-python (→ numcodecs), running the current
suite on 3.14t gives a **false** free-threaded signal — the GIL is on.
2. **Missing free-threaded wheels → slow / degraded.** `icechunk` and
`numcodecs` compile from source on 3.14t (no `cp314t` wheels yet), and
`google-crc32c` (via icechunk) falls back to its pure-Python implementation:
> `As the c extension couldn't be imported, google-crc32c is using a pure
> python implementation that is significantly slower.`
3. **Tests use zarr-python as the fixture *writer*.** 8 of 10 test files call
`zarr.create_array` / `zarr.open_group` to produce fixtures, then read them
back with zarrista. These are really *interop/integration* tests. We have
almost no coverage that exercises zarrista's **own** read+write round-trip
(`ArrayBuilder.create` / `store_chunk` / `store_metadata` → `Array` read).
### Proposed plan
- [ ] Split the suite into **own** tests (zarrista-only read/write) vs
**integration** tests (zarr-python / icechunk / arro3 interop), via markers
or a `tests/integration/` directory.
- [ ] Add zarrista-only round-trip tests (create + store chunks + read back,
compared against numpy) so there's real own-code coverage.
- [ ] Split dependency groups so the free-threaded job installs only GIL-safe
deps (avoid zarr/numcodecs/icechunk).
- [ ] Add a `3.14t` CI job that runs **only** the own tests on the free-threaded
interpreter.
- [ ] Revisit `pytest-run-parallel` for genuine thread-safety stress — note it
runs the *same* test in N threads sharing one fixture, so any test that
mutates shared store state flakes; concurrency tests must use per-thread
isolated stores.
- [ ] Re-enable integration tests on 3.14t once `numcodecs`/`icechunk` ship
free-threaded wheels and declare GIL-safety.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the tests that use zarr.create_array and zarr.open_group, then inspect the zarrista-only entry points ArrayBuilder.create, store_chunk, store_metadata, and Array reads. Separate own tests from integration tests, add isolated round-trip coverage compared with numpy, and verify a 3.14t CI job runs only GIL-safe tests without shared-store concurrency flakes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100