Nested replacement fields switch f-string lexing out of format-spec mode

Bug report

Summary

Closing a nested replacement field inside an f-string format specifier switches the lexer to middle-string mode even though the outer replacement field remains open. Braces that follow the nested field are then parsed as escaped literal content instead of expressions in the format specifier. The shared lexer path also affects t-strings and changes the diagnostic produced for a newline after a nested field.

Reproduction Code

class X:
    def __format__(self, spec):
        return repr(spec)


x = X()
y = "Y"
z = "Z"

print(f'{x:{{y}}}')
print(f'{x:{y}{{z}}}')

Actual Behavior

"{'Y'}"
'Y{z'}

The second f-string passes Y{z to X.__format__(). repr() returns 'Y{z', and the remaining } is appended as literal f-string content. The corresponding t-string, t'{x:{y}{{z}}}', records Y{z as its interpolation's format specifier and places the remaining } in the following literal string.

Expected Behavior

The second f-string should treat {{z}} as a nested replacement field whose expression is the set literal {z}. It should pass Y{'Z'} to X.__format__() and print:

"{'Y'}"
"Y{'Z'}"

The t-string interpolation should likewise record Y{'Z'} as its format specifier.

Root Cause

begin_ftstring_expr() increments replacement_depth when a replacement field starts. A nested replacement field within a format specifier therefore has a greater replacement depth than its enclosing field.

When the matching } is processed, _PyLexer_close_ftstring_expr() decrements replacement_depth and unconditionally selects FTSTRING_MODE_MIDDLE. After a nested field closes, the remaining non-zero replacement depth represents the still-open outer field, whose format specifier is still being scanned. Losing that state changes how _PyLexer_get_ftstring() handles subsequent braces and
newlines.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-157492

Run the two f-string examples and compare their output with the expected behavior, including the corresponding t-string case. Then read begin_ftstring_expr() and _PyLexer_close_ftstring_expr(), which the report identifies as the relevant lexer entry points; done means nested fields remain correctly handled while the outer format specifier is open.