micropython / micropython/micropython

RFC: Distribute native modules for easy installation

Open
#19,479 31 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement tools
Dominant language
C
Stars
22.1k
Forks
9k
Avg merge
6d 4h
Merged PRs (30d)
16

Description

Description

One of MicroPython's superpowers is the ability to install and use additional libraries without needing to rebuild and flash the device. The native module support in MicroPython makes installing at runtime also possible for modules that are implemented in C.

There are a couple of C modules in MicroPython repo, and in micropython-lib that can or would be very useful to distribute as native modules. To make that happen, we would need build them (for relevant architectures), and to publish them on HTTP server. It would then be possible to install them using mip - and this could be documented.

The immediate candidates would be btree, deflate, re from examples/natmod/. But in general this mechanism is useful for any module which is useful to have optional, and which can be implemented within the restrictions of natmod.

I propose adding the build and distribution setup to Github Actions. Such that on every new tag of MicroPython, natmod .mpy files would be placed onto HTTP file-server (Github Pages etc), in a versioned manner. Might also want to have "latest" (latest release) and "nightly" folders.

If this addition is considered welcome, then I can provide a pull request.

Related: PRs for making mip install for natmods more ergonomic:

Code Size

This feature could be used to enable smaller code size of default images, by having more modules be installable at runtime instead of being baked into firmware image.

Implementation

I intend to implement this feature and would submit a Pull Request if desirable

Code of Conduct

Yes, I agree

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the native-module examples in examples/natmod/, especially btree, deflate, and re, and review the related mip installation pull requests. Then inspect how GitHub Actions handles tagged builds and how artifacts could be published to an HTTP file server. Done means relevant architectures are built, versioned native-module files are distributed, and the mip installation path is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, github-actions
Domain
build-system, ci-cd, embedded-iot, release
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.