pytest-dev / pytest-dev/pytest-xdist

Question: running pytest worker in WebAssembly

Open
#336 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
287
Avg merge
9h 30m
Merged PRs (30d)
2

Description

Sorry if this issue is a bit out of scope for pytest-xdist, however as this project deals with running pytest remotely and results serialization I though there might be some intersection.

I'm trying to run unit tests with Pytest in a WebAssembly environment as part of the pyodide project. For instance to run the numpy test suite, the test would conceptually look as follows,

test_numpy_suite.py

def test_numpy_suite(selenium):
    selenium.load_package(['numpy', 'pytest', 'nose'])
    exit_code = selenium.run(
       """
       import pytest

       pytest.run(['--pyargs', 'numpy'])
       """)
    assert exit_code == 0
    print(selenium.logs)  # stdout with test results would be here

which would be run with,

pytest -s test_numpy_suite.py

(a more concrete example can be found here). The selenium fixture is not created with pytest-selenium, but it's close enough; the main points is that it yields a browser instance.

So we have two pytest instances, one in the host system (master in pytest-xdist terminology), and one running in WebAssembly (worker). Currently the output of the second one, are just printed at stdout of the host system. Ideally I would like to do something like,

pytest --wasm --pyargs numpy   # e.g. assuming pytest-xdist is installed on host

in which the master pytest instance passes the argument to the worker, that does test collection and execution, and sends the results back to the master. (If I understand correctly pytest-xdist works the other way around with master doing collection by default).

Another point I was wondering is about the communication between the two instances. After superficially reading the execnet docs, from what I understood it communicates via "gateways", but the current limitation of the WebAssembly environment is that it's not possible to create new processes, threads, open sockets (or use SSH), and I am not sure what would be the best workaround. It is possible to send Python basic objects (in the main thread) from the WASM environment back to the host python process, though. Or create some kind of HTTP server on the master to which the the worker would connect to.

Any suggestions would be very much appreciated! For more context see e.g. https://github.com/iodide-project/pyodide/issues/69

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the concrete example in pyodide's test/test_testing.py and compare its host and WebAssembly pytest instances with pytest-xdist's master/worker model. Read the execnet gateway documentation and the linked Pyodide issue to determine whether the proposed execution and communication model is feasible. Done would require a settled implementation scope, not just advice on possible workarounds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, wasm
Domain
distributed-systems, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.