con / con/mechababs

fMRIPrep: mris_expand aborts with a buffer overflow when the nipype workdir path is long (FreeSurfer)

Open
#150 3 comments 0 reactions 0 assignees View on GitHub
decision pipeline:fmriprep upstream upstream-NOT-FILED
Dominant language
Python
Stars
1
Forks
4
Avg merge
15h 39m
Merged PRs (30d)
24

Description

`sub-010001 ses-02` ran 3h54m on the 2026-09-04 OpenNeuroStudies shakeout and failed. FreeSurfer's `mris_expand` aborted while building the right-hemisphere midthickness surface; `--stop-on-first-crash` ended the run there.

```
Node: _midthickness0
Cmdline: mris_expand -pial .../rh.pial -thickness -thickness_name .../rh.thickness .../rh.white 0.5 .../rh.midthickness
Stdout: reading pial surface ... using distance as a % of thickness ... reading thickness...
Stderr: *** buffer overflow detected ***: terminated
Aborted
Traceback: RuntimeError: subprocess exited with code 134.
```

## Facts

| | |
|---|---|
| Job | `63990909_1`, 8 CPUs, 40 G grant, `--mem-mb 32000` |
| `max_rss` | `14539284K` = 13.9 GiB (**of a run that died partway** — a lower bound, not a completion figure) |
| `time_elapsed_raw` | `14037` s = 3:53:57 |
| `exit_code` | `1:0` (babs's zip script; the `134` is `mris_expand` inside it) |
| `has_results` | `False` |
| Anatomicals present | `acq-mp2rage_T1w`, `inv-1_mp2rage`, `inv-2_mp2rage`, `T1map`, `acq-lowres_FLAIR`, `defacemask` |
| Stage | `--level minimal`, unchained, so it ran its own FreeSurfer recon from raw |

`mechababs iterate` routed the cell correctly: `1 job(s): 1 submitted, 0 done, 1 failed -> fail`, flagged and not merged.

## What is and isn't known

**Known:** recon-all got far enough to produce `rh.white`, `rh.pial` and `rh.thickness` — `midthickness` sits downstream of it in `anat_fit_wf/surface_recon_wf`. So the failure is not "recon-all fell over"; it is `mris_expand` aborting while reading surfaces recon-all had already written.

**A glibc fortify abort is a memory-safety bug in `mris_expand` regardless of its input.** Something about these surfaces triggers it; a well-behaved program would have errored, not overflowed a buffer.

**Not known:** whether the input surfaces are pathological. The dataset's only T1w is MP2RAGE-derived (no combined/denoised UNI), and uncombined MP2RAGE carries strong background noise that can yield poor surfaces — but that is a hypothesis fitting the file list, not something this run establishes. It also ran with `--skull-strip-t1w force`, which is worth re-examining for this anatomical.

**The evidence to settle it no longer exists.** babs's `participant_job.sh` traps `EXIT` and `rm -rf`s the per-job scratch on every exit path, so the fmriprep crash file, the recon-all log and the surfaces themselves went with the failed job. Only the SLURM `.o`/`.e` survive. Filed separately.

## Next

- [ ] Re-run one subject with the scratch preserved, then `mris_info` the surfaces — that is what separates "degenerate surface" from "`mris_expand` bug on a valid one". Blocked on the scratch-preservation issue.
- [ ] Decide the MP2RAGE anatomical handling. Shares a decision with #48 (ds004146, MP2RAGE-only, use `UNIT1_denoised`), though the failure mode differs: #48 has no `_T1w` at all, here one exists and fmriprep gets far enough to crash on it.
- [ ] If it turns out to be a FreeSurfer bug, report upstream.

Not an OpenNeuro platform problem — the content fetched fine and the files are valid; this is a tool failure on unusual-but-legal data, so it stays ours.

Found on the 2026-09-04 OpenNeuroStudies shakeout (campaign `c3`, Unity). MRIQC on the same subject/session succeeded and merged (30.2 GiB, 40.5 min).

TODO before close:
- [ ] should be filed upstream freesurfer

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with participant_job.sh and the separate scratch-preservation issue; rerun one subject with per-job scratch retained, then inspect rh.white, rh.pial, and rh.thickness with mris_info. Done means determining whether the surface input is pathological or mris_expand is faulty, then filing upstream if the FreeSurfer bug is confirmed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.