Instagram / Instagram/LibCST

Supported Python versions

Open
#776 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
1.9k
Forks
229
PR merge metrics
No merged PRs in 30d

Description

I've been thinking of tightening up the supported Python versions in LibCST.

## Problem

Currently, LibCST allows for handling any Python version since 3.0. This is problematic on many fronts:
- Maintaining the parser is a larger and larger burden going forward
- Figuring out how to represent multiple versions within the CST is both challenging to implement and awkward to understand when using (e.g. "is this a `Decorator` node that supports any expression or just a few of them?")
- Making sure a CST is valid for a certain Python version is non-trivial
- No real motivation for fixing bugs and inaccuracies in old Python versions (good luck getting a PR about Python 3.3 support reviewed)

So it's clear we should limit the amount of Python versions supported. Here are the two obvious (to me anyway) ways forward:

## Proposal 1: Strictly One Version

In this world, a particular version of LibCST would only ever support one Python version. Whenever a new CPython gets released (currently at most once a year) with a new grammar, a new major release of LibCST will come out shortly after, dropping support for the previous grammar.

- LibCST would start shipping codemods to future-proof code (e.g. `libcst==2.0` would have a codemod to transform code that `libcst==3.0` would be guaranteed to support)
- (+) this is simple from the parsing point of view: if the targeted CPython parses your code, LibCST will, too. If the targeted CPython doesn't, LibCST won't either.
- (+) simple to represent: the CST classes can more closely resemble the structure of the official `ast` module. It will be clear what can go into a `Decorator` node, given a particular LibCST version. There will never be a node representing the `print` statement.
- (+) simple to validate: we could just round-trip the generated code through LibCST's own parser (this is probably impractical, but useful for testing the validation code)
- long term, this option should motivate people to future-proof their codebases if they want to stay on the latest LibCST version
- (-) tools using LibCST to parse Python code will likely have to pin the major version of LibCST or introduce some kind of facade to manage multiple versions
- (-) we will likely have to maintain multiple concurrent versions of LibCST for the most actively used Python versions

## Proposal 2: Officially Supported CPython Versions

In this world, only the actively supported Python versions (to be more precise, the actively supported CPython releases implementing a Python version) would work with LibCST. As of this writing (2022-09-15) that would mean 3.7-3.10: four versions. There will be a brief period of having to support 5 versions until 3.7 reaches end-of-life.

Whenever a Python version is dropped from support, LibCST would bump its major release number.

This is a less pure, more practical version of proposal 1, that compromises on complexity: the grammar, validation rules, and CST nodes would be more complex and ambiguous (user code will have to make sure they only put the correct type of expression into a `Decorator`, or avoid using a `Name` that becomes a keyword in later Python versions). But at least tools using LibCST will have a clearer path forward.

The complexity in the grammar can be solved by shipping multiple copies of the parser, each being capable of parsing a single Python version. This means that parsing would have to try each grammar in sequence, unless a requirement for a particular version is passed in (#740)

## Conclusion

I don't have a conclusion yet. I'm slightly leaning towards Proposal 1, but am open to be convinced otherwise.

Does anyone have a better idea?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.