scikit-hep / scikit-hep/vector

Replace uncompyle6 test with `inspect` and `ast.parse` or just drop it (Python 3.8 is end-of-lifed)

Open
#531 0 comments 2 reactions 0 assignees View on GitHub

A pull request for this has already been merged.

  • #643 by @henryiii — merged
tests
Dominant language
Python
Stars
99
Forks
42
Avg merge
1d 13h
Merged PRs (30d)
6

Description

Although we can keep checking uncompyle6 releases, I think the state of the art is fixed at

Although we do not decompile bytecode greater than 3.8, code supports running from up to 3.12.

Our test_compute_features.py test uses uncompyle6 to get an AST of all the functions in the compute submodule and verify that they restrict themselves to a small set of features, so that there are fewer limitations on new backends. The new SymPy backend is also helping to keep the compute functions general, since the SymPy backend makes fewer assumptions than anything else (vector components aren't even numerical!), but it would be nice to keep directly checking.

Python 3.8 support ended over a month ago. But, rather than dropping the AST checks, what if we use Python's inspect module to find the source code lines and re-parse them with ast.parse?

Aha! It's because many of them are auto-generated. For instance, this for loop generates 48 functions via composition. Python's inspect can't find the source code because there never was any source code for these functions—they're made on-the-fly.

So if we want to preserve the test_compute_features.py tests (past the point when Python 3.8 can no longer be accessed in GitHub Actions), we'd have to generate source code text for these functions. One way to do that would be to use uncompyle6 to get AST and then use ast.unparse (or similar functionality from uncompyle6) to make source code text that we'll check into git instead of the generating functions. Or modify the generating functions to produce source code text directly. But, either way, we'd switch to managing 48 (not a big number) functions, rather than the for loop that generates them. As a side-bonus, error messages would report line numbers that exist in source code that you can look up.

Alternatively, we'll just remove test_compute_features.py when it becomes impossible to install Python 3.8 (in an easy way, without deadsnakes...)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/test_compute_features.py and the generated functions in src/vector/_compute/lorentz/add.py. Review how uncompyle6 currently supports the AST checks and evaluate the proposed inspect/ast.parse or source-generation alternatives. Done means either preserving equivalent checks without Python 3.8 or removing the test with the project’s decision documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.