lightly-ai / lightly-ai/lightly

[BUG] macOS: `make test-fast` hangs indefinitely

Open
#1,917 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.8k
Forks
367
Avg merge
3d 22h
Merged PRs (30d)
5

Description

Description

Running make test-fast on macOS hangs indefinitely during tests/utils/test_dist__gather__losses.py. The pytest process enters a sleeping state and never makes further progress. I left it running for over 70 minutes and observed only about 1 minute of cumulative CPU time.

There is no exception and no traceback, so nothing is printed when the hang occurs. Nothing in the terminal indicates which file is responsible.

Reproduction

On macOS:

git clone https://github.com/lightly-ai/lightly.git
cd lightly
uv venv
source .venv/bin/activate
make install-dev
make test-fast

Or isolated to the offending file:

pytest tests/utils/test_dist__gather__losses.py -v

Expected: the suite completes in a few minutes.

Actual: hangs after 1 to 2 minutes of apparent progress, at or during tests/utils/test_dist__gather__losses.py. The hang is presistant on macOS across repeated runs.

Orphaned child processes after kill

If the hung pytest is terminated (Ctrl-C or kill), the spawned child process is frequently orphaned. The orphan remains in a sleeping state with roughly 500 MB to 1 GB resident memory and does not exit on its own, it must be cleaned up manually with kill -9 <pid>. Observed repeatedly during investigation.

Not observed in CI

The four tests in TestGatherLayer_Losses are guarded by a class-level pytest.mark.skipif decorator gated on the GITHUB_ACTIONS environment variable, so they are always skipped on GitHub Actions runners. The existing skip reason attributes the skip to a codecov compatibility concern, not a platform compatibility concern. Local macOS contributors have no equivalent skip applied.

Environment

  • OS: macOS 26.4.1 (arm64)
  • Python: 3.12.13
  • torch: 2.11.0
  • pytorch-lightning: 2.6.1
  • Installed via: make install-dev at commit c792812 on master
  • Invocation: make test-fast

Impact

  • Affected: contributors running the full test suite on macOS locally.
  • Not affected: GitHub Actions CI and Linux contributors.
  • Severity: high friction for new and returning macOS contributors. There is no error, no progress indication, and the hang is easy to mistake for a slow test. Cleanup after a kill also requires manual intervention to remove orphaned child processes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running pytest tests/utils/test_dist__gather__losses.py -v on macOS and inspect the four tests in TestGatherLayer_Losses, including their class-level GITHUB_ACTIONS skipif. Compare the local process behavior with the existing CI skip and verify cleanup after interruption. Done means make test-fast completes on macOS without hanging or leaving orphaned child processes.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, python, pytorch
Domain
operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.