Design CLI API

We need to design the CLI API for this package.

In https://github.com/pyodide/pyodide/issues/3573#issue-1580936524 @bollwyvl proposed,

$> pyodide-index path/to/wheels/folder
Wrote 200 packages to path/to/wheels/folder/repodata.json

and I agree this is the direction. Though given the current name of this package, it would also be more logical to call it pyodide lock IMO.

Also we need to keep in mind that the resulting lockfile would need to include information about the unvendored stdlib modules (and Python version). So it needs to have access to the original pyodide-lock.json (either via the pyodide version and looking at the CDN or by providing a path/URL to it). The difference with respect to conda index producing the repodata.json is that there,

• they can have multiple Python versions
• as far as I understand, they don't care that the included files may have some conflicts. The dependency resolution would really happen only at runtime when running conda install and combining indexes from different channels.. While in our case, by design once pyodide-lock.json is generated the dependency resolution is already done. So any package would be guaranteed to be installable with trivial dependency resolution (there is a single version for each package, and we can just ignore versions).

There are two use cases,

1. Adding/updating packages with actual files being stored on some remote CDN. In this case, extra entries in the pyodide-lock.json don't matter, since they would only load if explicitly imported, and we don't necessarily need to download all the included files locally IMO. Here I was thinking of taking something like a requirements.in as input (as in pip-tools) which would compute a consistent dependency graph merging the original pyodide-lock.json with the requirements in requirements.in and combine both (not easy).
2. Including only a subset of packages for a given application, and ship them alongside pyodide-lock.json for reproducibility. This is closer to the use case of https://github.com/pyodide/pyodide-pack: BTW I'm changing a bit the focus of that package away from rather an experimental module stripping via runtime detection to any kind of package/wheel minification. So the wheel files would be modified by that tool, but the end pyodide-lock.json would be still generated by this project.
   The challenge with this use case is that even given a list of wheels in some folder, we still need to verify that there are no missing requirements and that the dependency graph is consistent, so we need a resolver that would understand the wasm platform for finding compatible wheels.

Anyway, it's still early day, this needs more discussion. My current idea is to iterate on the implementation that would work well in practice for these use cases, while only pushing alpha releases to PyPI. Any API in this package is considered unstable and can be completely changed.

Please let me know if you have any other ideas about how this should work.

@hoodmane @ryanking13

Start by reading the proposed pyodide-index and pyodide lock command designs, then compare the two package-selection use cases described in the issue. Review pyodide-lock.json, requirements.in, and the relationship with pyodide-pack. The work is not yet defined; completion would require an agreed CLI and resolver design that accounts for Python versions, unvendored standard-library modules, and compatible wheels.