python / python/cpython

refactor the implementation of inlined comprehensions

Open
#124,697 5 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core type-refactor
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

The current implementation of comprehension inlining is a pile of special cases that is quite difficult to understand and debug. The root cause of this is that it attempts to flatten the inlined scope into the enclosing scope in the symbol table, to avoid the compiler needing to generally understand the concept of inlined sub-scopes, but this flattening loses information and requires a number of special cases. And then the compiler implementation of inlining modifies the flattened scope as it traverses the inlined comprehension(s) in order to ensure correct handling of the names within the inlined sub-scope; ideally the compiler would not modify the symbol table as it generates code.

A better implementation would leave the scopes separate in the symbol table, losing no information but just marking the comprehension scope as an inlined sub-scope, and then add explicit support for inlined sub-scopes in the compiler (with appropriate query methods in the symtable.)

Linked PRs
  • gh-156819

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review linked PR gh-156819 first, since the issue indicates that work has moved there. Otherwise, trace comprehension inlining through CPython's symbol-table and compiler implementation; done means preserving separate inlined sub-scopes in the symbol table and adding explicit compiler support without modifying the symbol table during code generation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.