UART.readline() reads the buffer even when it does not end in a newline character

The UART documentation says:

UART.readline()
    Read a line, ending in a newline character.
    Return value: the line read or None on timeout.

If I connect a wire between TX2 and RX2 on the ESP32, and go:

from machine import UART
import time
u = UART(2)
s = u.write("begin1 end1\nbegin2 ");
# s = u.write("end2\nbegin3 ");
time.sleep(0.1)
print(u.any(), u.readline())
print(u.any(), u.readline())
print(u.any(), u.readline())
print(u.any(), u.readline())

The output comes out as:

19 b'begin1 end1\n'
7 b'begin2 '
None
None

You can uncomment the second u.write() to prove that the strings will be concatenated in the buffer prior to the readline().

(The timeout value in the constructor is not documented, but I've worked out that it is in milliseconds.)

With quite a bit of fiddling I can prove the same effect on the ESP8266 with the following code loaded into main.py:

from machine import UART
import time, os

u2 = UART(1, baudrate=115200)
print("startup")
u2.write("startup\n")
time.sleep(3)  # allow for interrupt
print("3seconds")
u2.write("3seconds\n")

os.dupterm(None, 1)
u = UART(0, baudrate=115200, rxbuf=240)

s = u.write("begin1 end1\nbegin2 ");
s = u.write("end2\nbegin3 ");
time.sleep(0.1)
for i in range(5):
    u2.write("%d %d %s\r\n" % (i, u.any(), str([u.readline()])))

Then you can monitor the output of the TX1 from the ESP8266 (which goes out of the GPIO2) by connecting it to RX2 of a spare ESP32 which is running the program:

Then on the ESP32 you run:

from machine import UART
u = UART(2, timeout=5000)
while True:
    print(u.readline())

(Don't forget to connect the RX to TX of the ESP32 and the VIN and G pins from the ESP32 to the ESP8266 to send it the power.)

The result is:

b'...\x02startup\n'
b"0 1 [b'3seconds\\r\\n']\r\n"
b"1 1 [b'begin1 end1\\n']\r\n"
b"2 1 [b'begin2 end2\\n']\r\n"
b"3 1 [b'begin3 ']\r\n"
b'4 0 [None]\r\n'

The fact that UART.any() can return 1 even when there are >1 characters in the buffer is at least documented. However, I can't tell how to use select.poll() for a sophisticated way of querying the available characters.

Normally I'm not daft enough to waste so much time trying to use UART0 on the ESP8266 (owing to the problem that it blocks the Repl), but I'm trying to do something with a Wemos Arduino Megas that -- with the right selection of dip-switches -- wires the atmega to the esp8266 on its USB0, so I have no choice.

Now that I have identified this issue (which was very hard to see when you can't print debug messages on a repl blocked esp8266), I can make some code to concatenate the incomplete lines. But it would be much more useful if this was done properly inside the UART buffer itself.

Start with the UART.readline documentation and reproduce the ESP32 and ESP8266 examples, including the main.py code shown in the issue. Compare the observed handling of incomplete buffered lines with the documented newline and timeout behavior, then verify that the behavior is consistent across both examples and that the resulting UART behavior is covered by appropriate tests.