tarantool / tarantool/tarantool
macOS: make tests stable and portable
Nobody has claimed this yet.
- Dominant language
- Lua
- Stars
- 3.7k
- Forks
- 419
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 88
Description
While running the test suite on macOS, I encountered several tests that either consistently fail or flaky. This indicates that some tests rely on platform-specific assumptions or are not fully stable across environments.
It would be great to improve test stability and make them platform-independent where possible.
Environment:
Tarantool 3.7.0-entrypoint-218-g4e6123983f
Target: Darwin-arm64-Debug
Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=TRUE
Compiler: AppleClang-17.0.0.17000013
C_FLAGS: -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=<...>=. -std=c11 -Wall -Wextra -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror -g -ggdb -O0
CXX_FLAGS: -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=<...>=. -std=c++17 -Wall -Wextra -Wno-invalid-offsetof -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror -g -ggdb -O0
System & Hardware
$ sw_vers
ProductName: macOS
ProductVersion: 15.6.1
BuildVersion: 24G90
$ sysctl -n machdep.cpu.brand_string
Apple M4 Pro
List of problematic tests
Note: the list below is not exhaustive — it only includes the issues I was able to identify so far.
Fails
-
https://github.com/tarantool/tarantool/pull/12561app/socket.test.lua
SO_REUSEPORTvalue differs from Linux:getsockopt()may return512instead of1 -
https://github.com/tarantool/metrics/pull/542metrics-luatest/tarantool_memory_metrics_test.lua
tnt_memory_virtis expected to be positive, but is0on non-Linux platforms -
https://github.com/tarantool/tarantool/pull/12562app-luatest/popen_test.lua
popen.shell(...):close()may returnnil, erreven though the handle is already closed -
box-luatest/gh_11803_interface_option_test.luahttps://github.com/tarantool/tarantool/pull/12563
test assumes loopback interface name islo, while macOS useslo0 -
box-luatest/iproto_export_test.luahttps://github.com/tarantool/tarantool/pull/12586
server crashes during startup in application threads. Lua module resolution is broken (http.client.libis attempted to be loaded from/, leading tocannot read /: Is a directory)server.log
server | 2026-04-18 19:34:16.534 [88422] app1 init.c:618 C> builtin/http.client.lua:32: error loading module 'http.client.lib' from file '/': server | cannot read /: Is a directory server | 2026-04-18 19:34:16.534 [88422] app1 init.c:626 C> #2 (null) (), builtin/internal.loaders.lua:245 server | 2026-04-18 19:34:16.534 [88422] app1 init.c:626 C> #3 require (global), [C]:-1 server | 2026-04-18 19:34:16.534 [88422] app1 init.c:626 C> #4 (null) (), builtin/http.client.lua:32 -
box-luatest/app_threads_test.luahttps://github.com/tarantool/tarantool/pull/12586
application threads fail to load standard Lua modules (e.g. metrics.tarantool).server.log
server | 2026-04-18 19:34:15.896 [88396] app2 say.c:88 F> Error loading Lua module metrics.tarantool...: (null) -
box-luatest/app_threads_module_test.luahttps://github.com/tarantool/tarantool/pull/12586
same issue as inbox-luatest/app_threads_test.lua -
— fixed by https://github.com/tarantool/tarantool/pull/12543config-luatest/roles_test.lua
Shutdown log is matched too strictly: test expects line ending atTerminated,
but macOS may add a suffix (e.g.Terminated: 15). -
https://github.com/tarantool/tarantool/pull/12569replication-luatest/gh_10506_spam_in_netbox_logs_test.lua
The test useslocalhost:0as a stand-in for a refused connection, but on macOS that address may produce a different socket error instead ofConnection refused(Can't assign requested address).
Flaky / may hang
-
box-luatest/session_test.lua— flaky under parallel runs https://github.com/tarantool/tarantool/pull/12565
box.cfg({})in an interactive child may fail withALREADY_RUNNING: Failed to lock WAL directory . -
app-luatest/http_client_http_version_test.luahttps://github.com/tarantool/tarantool/pull/12587
intermittentCould not connect to serverbefore reaching the HTTP/ALPN assertion -
app/fio.test.lua— flaky under parallel runs https://github.com/tarantool/tarantool/pull/12594
Thegh-4794section uses a fixed directory name undercwdas a customTMPDIR. Under repeated runs, may fails withfio: File exists -
metrics-luatest/tarantool_config_metrics_test.lua— flaky under parallel runs
Fixed port (localhost:3301) causes conflicts; metrics are read too early afterconfig:reload(). -
replication-py/cluster.test.py
Reuses iproto connection after failedSUBSCRIBE, which may leave it broken =>Connection reset by peer.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the listed luatest, Lua, and Python tests on the Darwin-arm64 configuration, beginning with the unchecked cases: box-luatest/gh_11803_interface_option_test.lua, iproto_export_test.lua, app_threads_test.lua, session_test.lua, and replication-py/cluster.test.py. Review the linked pull requests and failure logs before choosing a case. Done means the affected tests pass reliably on macOS and remain stable under parallel runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua, macos, python
- Domain
- operating-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100