petercorke / petercorke/robotics-toolbox-python

JupyterLite: upstream piplite index bug needs a real pure-Python wheel, not just the CI stopgap

Open
#579 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

tech-debt
Dominant language
C++
Stars
3.5k
Forks
624
Avg merge
2d 4h
Merged PRs (30d)
53

Description

Migrated from tech-debt.md (deleted, see repo history via git log -- tech-debt.md).

The live JupyterLite "Try it Now" deployment's docs/lite/pypi/all.json package index only listed a cp313 wasm wheel for roboticstoolbox-python, even though the pinned jupyterlite-pyodide-kernel==0.6.1 embeds Pyodide 0.27.6/CPython 3.12 and needs a cp312 wheel. Both wheel files were present at docs/lite/pypi/, but only cp313 made it into all.json -- piplite.install(["roboticstoolbox-python"]) failed with ValueError: Can't find a pure Python 3 wheel for 'roboticstoolbox-python'.

Root cause is upstream, in jupyterlite_pyodide_kernel/addons/piplite.py's get_wheel_index(): all_json[normalized_name]["releases"][version] = [release] assigns a fresh single-item list per (name, version) key instead of appending -- so when a release carries wasm wheels for more than one CPython version (as ours do, cp312 and cp313 both tagged the same version), whichever sorts alphabetically last silently overwrites the others. No build error, just a runtime failure in the browser. Confirmed not fixed in jupyterlite_pyodide_kernel 0.7.1 either (latest as of the last check) -- bumping the pin doesn't resolve it.

Fix applied (stopgap): narrowed the gh release download --pattern in ci.yml's "Fetch pyodide wheel for JupyterLite" step from *pyodide* to *cp312*pyodide*, so only the one wheel the pinned kernel actually needs ever reaches docs/lite/pypi/, sidestepping the upstream bug entirely.

Proposed real fix: the whole fetch-a-prebuilt-wasm-wheel-from-a-GitHub-release mechanism (and the CPython-tag pinning it depends on) is a maintenance burden in its own right. The real fix is a genuine py3-none-any pure-Python wheel published to real PyPI (no compiled _fknm_c/_frne_c) -- Pyodide's micropip resolves that automatically via its own standard fallback logic, no custom JupyterLite piplite index or GitHub-release-asset fetching required at all. Once that lands (tracked as a follow-on to the fknm/frne nanobind refactor work), this stopgap and the "Fetch pyodide wheel"/"Build JupyterLite site" CI steps should be revisited -- possibly removable outright.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with jupyterlite_pyodide_kernel/addons/piplite.py and the Fetch pyodide wheel for JupyterLite and Build JupyterLite site steps in ci.yml; review the current stopgap and the fknm/frne nanobind refactor work. Done means a genuine py3-none-any wheel is published to real PyPI, after which the workaround and related CI steps are revisited.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, github-actions, python
Domain
build-system, ci-cd
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.