ml-explore / ml-explore/mlx

ParallelFileReader: uncatchable fatal error when pread lands at EOF

Open
#4,526 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
28.5k
Forks
2.3k
Avg merge
3d 8h
Merged PRs (30d)
62

Description

Summary

Loading .safetensors files through the parallel file reader can abort the process with an uncatchable fatal error when a pread lands at/after EOF. The std::runtime_error thrown by ParallelFileReader crosses the C API boundary and becomes a hard crash instead of a catchable error.

Reproduction

Any multi-reader safetensors load where a reader's pread returns 0 (read at/after EOF):

// mlx/io/load.cpp  (ParallelFileReader)
// load.cpp:345  static ThreadPool* thread_pool = new ThreadPool{4};  // shared singleton
// load.cpp:378  throw std::runtime_error("[read] Unable to read from file.");  // sync tail read
// load.cpp:392  throw std::runtime_error("[read] Unable to read from file.");  // batched future

When this exception is thrown from a worker thread and propagated across the C API, it is not caught and the process aborts (observed as a Swift fatal error on the consumer side).

Behavior

  • Non-deterministic: the crash fires only when a reader's pread returns 0 at the EOF boundary (a race with the shared ThreadPool{4} work distribution). Re-running the same load usually succeeds.
  • The crash is a process abort, not a catchable exception — no Swift/C caller can recover.

Affected versions

  • C++ MLX ce45c52 (v0.31.x) through 1f8e74e (v0.32.2) — the static ThreadPool* shared reader and the throw std::runtime_error on pread == 0 are present in both.

Smallest failing test

MixedPrecisionQuantLoadTests.perModuleOverridesApplied and MixedPrecisionQuantLoadTests.globalOnlyQuantizationFailsLoudly (and RerankerTests) trigger the multi-reader safetensors load and intermittently abort the test process at the C API boundary (mlx_c/transforms.cpp:73 / array.cpp:352).

Suggested fix

Catch/convert the std::runtime_error from ParallelFileReader before it crosses the C API (return an mlStatus/mlxError instead of throwing), and/or guard the EOF pread == 0 case so it does not throw on the tail read.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with mlx/io/load.cpp around ParallelFileReader, the shared ThreadPool, and the two reported pread failure sites; then trace the C API boundary through mlx_c/transforms.cpp:73 and array.cpp:352. Reproduce with the named MixedPrecisionQuantLoadTests or RerankerTests. Done means the EOF-triggered failure no longer aborts the process and is observable as a catchable C or Swift-side error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.