Pyodide / Emscripten support (PEP 783 pyemscripten wheels)

Written by claude:

tl;dr: compiling async Rust to Python for emscripten is still _really hard_

---

## Goal

Make async-tiff usable from Pyodide (Python-in-the-browser, `wasm32-unknown-emscripten`), shipping PEP 783 `pyemscripten_*` wheels to PyPI — while keeping the Python API unchanged: `TIFF.open(store)` stays async and keeps accepting any generic `ObspecInput` (`GetRangeAsync` + `GetRangesAsync` protocol), so a jsfetch-backed store can be swapped in at runtime without user code changes.

Packaging is solved upstream (PEP 783 accepted; maturin ≥ ~1.13 emits `pyemscripten` tags; pydantic-core and arro3 already ship these wheels). Async runtimes are **not**: Pyodide is single-threaded, tokio's reactor doesn't exist on emscripten, and pyo3-async-runtimes spawns an OS thread to drive tokio — it panics at init on this target.

## Compile experiment results (2026-06-11, validated locally)

Targeted the Python 3.14 line (`pyemscripten_2026_0`, Emscripten 5.0.3, **stable Rust 1.93** — no nightly). Toolchain: Pyodide's *patched* emsdk via `pyodide xbuildenv install-emscripten` (not stock emsdk), config from `pyodide config get`.

**Everything compiles, links, and runs.** A throwaway pyo3 wheel depending on `async-tiff --no-default-features --features lerc,jpeg2k,webp,lzma,tokio` built with `uvx maturin build --release --target wasm32-unknown-emscripten -i python3.14`, then **imported and ran inside Pyodide** (node, via `pyodide venv`), constructing `DecoderRegistry::default()` with all codecs registered. All four C/C++ sys crates compiled and linked: zstd-sys, openjpeg-sys, libwebp-sys, lerc-sys.

`cargo check --target wasm32-unknown-emscripten` feature matrix on the core crate:

| Config | Result |
|---|---|
| `--no-default-features` (zstd, flate2, weezl, jpeg baked in) | ✅ |
| `tokio` (only `tokio/sync` + `io-util` — runtime-independent) | ✅ |
| `jpeg2k` (OpenJPEG C), `webp`, `lzma`, `ndarray`, `object_store` | ✅ |
| `lerc` | ✅ with 2 env vars (below) |
| `reqwest` | ❌ genuinely incompatible |
| default features | ❌ (via reqwest) |

Required env (beyond pyodide's canonical rustflags/cflags):

```bash
export CARGO_TARGET_WASM32_UNKNOWN_EMSCRIPTEN_RUSTFLAGS="-C link-arg=-sSIDE_MODULE=2"
export CFLAGS_wasm32_unknown_emscripten="-O2 -g0 -fPIC -fwasm-exceptions -sSUPPORT_LONGJMP=wasm -Oz"
export CXXFLAGS_wasm32_unknown_emscripten="-O2 -g0 -fPIC -fexceptions -fwasm-exceptions -sSUPPORT_LONGJMP=wasm -Oz"
export BINDGEN_EXTRA_CLANG_ARGS_wasm32_unknown_emscripten="-fvisibility=default"
export MATURIN_PYEMSCRIPTEN_PLATFORM_VERSION=2026_0
```

Findings worth remembering:

- **lerc needs two env-var workarounds** (no upstream patches): (1) the `cc` crate injects `-fno-exceptions` for emscripten C++ but LERC uses `try/catch` — an explicit `-fexceptions` in `CXXFLAGS_wasm32_unknown_emscripten` fixes it (`-fwasm-exceptions` alone does not re-enable exceptions); (2) bindgen on wasm emits constants but **zero functions** (clang hidden-visibility default) → `BINDGEN_EXTRA_CLANG_ARGS_wasm32_unknown_emscripten="-fvisibility=default"`.
- **reqwest can't work**: on wasm32 it switches to its wasm-bindgen backend whose futures are `!Send`, conflicting with `AsyncFileReader: Send + Sync` (`src/reader.rs`); wasm-bindgen JS glue isn't present in an emscripten module anyway. wasm builds use `--no-default-features`.
- **emcc is required even for `cargo check`** — zstd-sys (non-optional via core `zstd`) compiles C in its build script.
- **py-async-tiff is blocked as-is**: tokio's own guard fires (`Only features sync,macros,io-util,rt,time are supported on wasm`), pulled in by `pyo3-async-runtimes` + `pyo3-object_store` (`rt-multi-thread`) and reqwest/hyper (`net`) via object_store cloud features. `rayon`/`tokio-rayon` thread pools also can't run (no threads in Pyodide).
- Not yet exercised: actually decoding a tile inside Pyodide (esp. LERC C++ exception handling under wasm EH). Good first runtime test once bindings exist.

## The design question (open)

The breakage is **not** the ObspecInput protocol — it's where it gets consumed. Today `python/src/reader.rs` wraps `store.get_range(...)` coroutines via `pyo3_async_runtimes::tokio::into_future` and tokio drives the futures on a background thread. Options to keep the public API identical:

**A — Python-async facade over a sync sans-IO Rust core.** Rust gains sync entry points: restartable metadata parse over a sparse buffer (`NeedsBytes {range}` error → host fetches → retry, precedent: arrow-rs `ParquetMetaDataReader::try_parse_sized` / `NeedMoreData`), `tile_byte_ranges`, decode-from-bytes. A thin pure-Python async wrapper implements today's API on top, calling `await store.get_ranges(...)` directly in Python — works on any event loop including Pyodide's. ~100–200 lines of Python.

**B — Keep Rust async everywhere, swap the executor on wasm.** PyO3 `experimental-async` (`Coroutine` polls Rust futures inline on the Python event loop — the right model for Pyodide, no threads) + a hand-rolled "await a Python awaitable from Rust" via `asyncio.ensure_future` + done-callback waker. Single Rust codebase, but unstable PyO3 API, untested on emscripten.

**C — Hybrid (was leaning this way):** build the sync sans-IO core (useful regardless — free sync API, future wasm-bindgen/JS build), use the Python facade from A as the **wasm implementation** selected at import time (`try/except ImportError`, arro3 pattern), leave the native tokio path untouched. If PyO3 native async stabilizes, B becomes a unification path later. Bonus: the facade works with any obspec store on native too, so unifying downward (dropping pyo3-async-runtimes entirely) stays an option pending benchmarks.

Either way the py-async-tiff crate needs the arro3-io structural pattern: a default-on `async` cargo feature gating the entire async stack (pyo3-async-runtimes, pyo3-object_store, object_store, reqwest, rayon/tokio-rayon, all `future_into_py` methods), with the wasm wheel built `--no-default-features`.

## Suggested staging (when resumed)

1. Sync sans-IO core: sparse-buffer restartable metadata parse + bytes→`Tile` decode path (`TileByteRange`/`TilesByteRanges` and `Tile::decode` are already public and sync)
2. `async` feature split in py-async-tiff + sync `#[pyfunction]`s
3. Python async facade implementing the existing API over the sync core; jsfetch/`pyodide.http.pyfetch` obspec store impl
4. CI: copy arro3's `wheels.yml` emscripten job (xbuildenv install, `pyodide config get` outputs, maturin-action) + the two lerc env vars; runtime-test with `pyodide venv` under node
5. Runtime decode tests in Pyodide (all codecs, esp. LERC)

## References

- arro3 shipped this end-to-end: kylebarron/arro3#502, kylebarron/arro3#504 (working CI: `.github/workflows/wheels.yml` emscripten job, `DEVELOP.md`)
- https://pydantic.dev/articles/emscripten-wheels-pydantic — maturin + PyEmscripten CI guide
- https://peps.python.org/pep-0783/ · https://blog.pyodide.org/posts/314-release/
- https://pyo3.rs/latest/async-await — PyO3 `experimental-async` · https://github.com/PyO3/pyo3/issues/1632
- https://blog.pyodide.org/posts/jspi/ — JSPI / `run_sync` escape hatch (out of scope for first pass)
- arrow-rs `ParquetMetaDataReader::try_parse_sized` — sans-IO restartable-parse precedent

Start with python/src/reader.rs and src/reader.rs, then run the documented cargo check --target wasm32-unknown-emscripten feature matrix to understand the current boundaries. Compare the proposed sans-IO core, wasm executor, and hybrid approaches, using the existing Tile::decode and tile-byte-range entry points; done is an agreed implementation path that preserves the Python API and supports a runtime-tested Pyodide wheel.

Written by the indexing model from the issue text.