Include `Pdb._exec_in_closure()` as code-path in `exec()` itself, or make it a public utility otherwise?

Feature or enhancement

Proposal:

When IPython is embedded into a non-IPython interpreter, it evaluates code using frame locals (FrameLocalsProxy) in the same way as pdb does in Python stdlib.

This does not work with objects which require access to the closure scope:

• comprehensions in Python <3.13 (fixed for 3.13+ by inlining comprehensions introduced in PEP 709)
• generators

This bug affected pdb module too (https://github.com/python/cpython/issues/65360) but it was fixed/worked around by:

• https://github.com/python/cpython/pull/111094

For example, the following does not work:

import sys
call_frame = sys._getframe(0).f_back
local_ns = call_frame.f_locals
exec('x = 1; sum(x * i for i in range(5))', locals=local_ns)

Produces:

Traceback (most recent call last):
  File "<python-input-0>", line 4, in <module>
    exec('x = 1; sum(x * i for i in range(5))', locals=local_ns)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <genexpr>
NameError: name 'x' is not defined

I know that FrameLocalsProxy has a few undocumented limitations, I am not sure if those are relevant here - I am linking the relevant issue just in case if that helps maintainers to refresh the context:

• https://github.com/python/cpython/issues/125731

I am coming here from IPython, to explore if the solution belongs in IPython, or in CPython repo:

• a) could exec handle FrameLocalsProxy specially in some way?
• b) as one option, could the same code as added for pdb in https://github.com/python/cpython/pull/111094 (_exec_in_closure method) be included in the default exec implementation for when FrameLocalsProxy is passed?
• c) if adding _exec_in_closure into exec proper does not make sense, is it a good idea to explore moving _exec_in_closure out of pdb and making it public?

The fix/workaround from #111094 is not perfect yet, though I think most of the problems could be resolved with a little bit more work. These issues were reported in:

• https://github.com/python/cpython/issues/126958

If _exec_in_closure were exposed as public, there would be a bigger incentive for community to fix issues outlined in #126958, basically centralising the effort to make that work well.

On the other hand, if there are plans to inline generators in the future (I saw that PEP 709 leaved that as a possibility), maybe _exec_in_closure would no longer be necessary in the first place.

As an alternative to reusing _exec_in_closure, IPython could use locals() call to populate locals_ns which get passed down to exec via locals argument when in embed mode; I am somewhat apprehensive to make this change as I worry it might break downstream code.

However, if you all advise that _exec_in_closure shall remain a private pdb utility, this would likely tilt the the trade-off towards using the locals() call, as otherwise IPython would need to maintain its own version of _exec_in_closure adding to maintenance cost on already over-stretched team.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

Not directly, but this is the third oldest unresolved issue in IPython dating back over 15 years:

• https://github.com/ipython/ipython/issues/62
• https://github.com/ipython/ipython/issues/136
• https://github.com/ipython/ipython/issues/12199

CC @gaogaotiantian if I may, as author of #111094 and assignee on #126958

Start by reading the exec handling of FrameLocalsProxy and pdb.Pdb._exec_in_closure, then compare the approach in CPython pull request 111094 with issues 126958 and 125731. Determine whether exec should support this directly or whether _exec_in_closure should become public; done requires a reviewed, agreed scope and corresponding tests.