Strange new line behavior and missing color in the iOS testbed logs

Bug report

Bug description:

When using cibuildwheel to build and test a Python package for iOS, pytest output captured through the Apple testbed runner is difficult to read. Two distinct problems occur simultaneously:

1. Progress output is split across individual lines.

pytest emits progress characters (., F, E, s, …) without a trailing newline and flushes after each one. In the captured log each character appears on its own line instead of forming a compact progress line:

============================= test session starts ==============================
 platform ios -- Python 3.13.11, pytest-9.0.3, pluggy-1.6.0
 rootdir: /Users/timrid/Library/Developer/CoreSimulator/Devices/84EFB4EB-1C63-49FF-972C-D7276CB71115/data/Containers/Bundle/Application/3CA6642E-C9F4-4388-B444-2FA31C2AA074/iOSTestbed.app/app
 configfile: setup.cfg
 testpaths: tests/
 plugins: mock-3.15.1, cov-7.1.0, timeout-2.4.0, asyncio-1.4.0
 timeout: 120.0s
 timeout method: signal
 timeout func_only: False
 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
 collected 4612 items / 24 deselected / 14 skipped / 4588 selected
 
 tests/test_base_protocol.py
 .
 .
 .
 .
 .
 .
 .
 .
                           [  0%]
                      [  0%]
 
 tests/test_circular_imports.py
 s
 s
 s
 s
 s [  1%]
 s
 s
 s
 s
 s
                                                             [  1%]
 
 tests/test_classbasedview.py
 .
 .
 .
 .
                                         [  1%]
 
...

The same problem affects any output that is deliberately written without a newline — e.g. print(".", end="", flush=True).

2. ANSI color codes are shown as raw escape sequences.

When color is explicitly enabled in pytest via --color=yes, pytest's colored output is not rendered. Instead, raw escape sequences appear as unreadable caret-encoded text:

\^[[1m============================= test session starts ==============================\^[[0m
 platform ios -- Python 3.13.11, pytest-9.0.3, pluggy-1.6.0
 rootdir: /Users/timrid/Library/Developer/CoreSimulator/Devices/84EFB4EB-1C63-49FF-972C-D7276CB71115/data/Containers/Bundle/Application/F9002FB8-67E9-4424-AC42-8F5807D4DB47/iOSTestbed.app/app
 configfile: setup.cfg
 testpaths: tests/
 plugins: mock-3.15.1, cov-7.1.0, timeout-2.4.0, asyncio-1.4.0
 timeout: 120.0s
 timeout method: signal
 timeout func_only: False
 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
 collected 4612 items / 24 deselected / 14 skipped / 4588 selected
 
 tests/test_base_protocol.py
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m                          [  0%]\^[[0m
 
 tests/test_circular_imports.py
 \^[[33ms\^[[0m
 \^[[33ms\^[[0m
...

Together these two issues make it very hard to interpret test results, especially in CI in combination with bad Github Actions performance on huge test outputs.

I think that this is an upstream CPython issue because the root cause is in Python's Apple system-logger redirection (Lib/_apple_support.py), which is used by the testbed runner to capture output from iOS builds.

Maybe a possible solution for problem 1 is to modify the Apple system-logger redirection in Lib/_apple_support.py to append a marker byte (e.g. 0x1f, ASCII Unit Separator) to any log message that does not end with \n. The testbed runner can then detect the marker, suppress the line break, and join the message with the next one. This would fix it without buffering output or breaking the flush() contract. To solve problem 2, the testbed runner can reverse the system log's caret encoding (\^[ etc.) back into the original control characters, restoring colored output.

@freakboy3742 What do you think?

CPython versions tested on:

3.13

Operating systems tested on:

Other

Linked PRs

• gh-150983

從 Lib/_apple_support.py 和 Apple 測試平台 runner 開始，接著使用 --color=yes 和不換行的 print 重現 pytest 輸出。將擷取的日誌與預期的精簡進度輸出及呈現的 ANSI 色彩進行比較；開始前檢查已連結的 PR gh-150983，因為相關工作可能已在進行中。