Improve PyREPL auto-indentation

Bug report

On second thought this issue should be an enhancement instead of a bug report. Sorry for the wrong template.

Bug description:

There are some cases where _pyrepl auto-indentation works not well.

Cases

1. A line ending with : in a multi-line string is wrongly indented.

Observed

>>> s = '''
... Note:
... ␣␣␣␣|

Expected

>>> s = '''
... Note:
... |

2. # inside strings is seen as a comment, the following : is ignored.

Observed

>>> if ' ' == '#':
... |

Expected

>>> if ' ' == '#':
... ␣␣␣␣|

3. When the entire cursor line is a comment and is already indented, pressing Enter gives a further indent.

Observed

>>> def f():
...     # foo⤶
... ␣␣␣␣␣␣␣␣|

Expected

>>> def f():
...     # foo⤶
... ␣␣␣␣|

Possible solution

Currently _should_auto_indent() parses the buffer from right to left and stops at the first newline it encounters. Only the last line that is not a comment line of the buffer is parsed.

But by parsing from right to left we can't tell if a # starts a comment or is part of a string. For example if we, from right to left, encounter a " first and then a #, we don't know if the # is a comment. To know that, we need to know if the " is a string boundary, but the # might comment out the ", so we can't be sure. There is a information dependency cycle.

To fix this I made a change to parse the buffer from left to right, keeping track of whether current char is inside a string or a comment. This approach solves the above three cases. However the whole buffer is parsed on every call of _should_auto_indent(), with very long buffer, there might be noticeable delay when pressing Enter.

I think this is a big change. It affects how _should_auto_indent() works as a whole. I am hesitated to create a PR and just put it here first to hopefully get feedback.

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Linked PRs

• gh-140710

从 Lib/_pyrepl/readline.py 中的 _should_auto_indent() 开始，然后检查它如何从右到左解析 buffer。重现 issue 中描述的 multiline-string、string-comment 和 indented-comment 三种情况，并将其行为与建议的从左到右解析方法进行比较。当自动缩进与预期示例一致，且不会为较长的 buffer 引入明显延迟时，即视为完成。