stablediffusion-ggml backend segfaults instead of returning an error when a Metal pipeline fails to compile, and its stderr never reaches the LocalAI log
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 49.2k
- Forks
- 4.5k
- Avg merge
- 1d 3m
- Merged PRs (30d)
- 239
Description
Summary
When a Metal pipeline fails to compile, the backend logs the failure and then
uses the pipeline anyway. The null dereference takes SIGSEGV inside cgo and the
process dies. LocalAI sees only exitCode=2 and rpc error: EOF.
Compounding it, the backend's stderr is not attached to anything LocalAI logs, so
the underlying Metal error is invisible from the server side. Between the two,
a precise and well-reported error from Metal reaches the operator as an opaque
crash.
Environment
- LocalAI 4.7.1
- Backend
metal-stablediffusion-ggml, digest
sha256:9cafbf0956c55791cac472f20c73f55f425eeaefeb9d9a9bb22c9c5b8e8d7258 - macOS 26.5.2 (build 25F84), Apple M3 Ultra, 256 GB unified memory
What the operator sees
Nothing useful. A /video request fails, and the LocalAI log contains only:
exitCode=2
rpc error: code = Unavailable desc = error reading from server: EOF
No Metal error, no kernel name, no indication that anything compiled or failed to
compile. Two earlier attempts against this backend produced exactly this and were
abandoned undiagnosed, because there was nothing to go on.
What is actually happening
Metal reports the failure correctly and precisely:
compiling pipeline: kernel_mul_mv_ext_bf16_f32_r1_5
[ERROR] MTLLibraryErrorDomain Code=5 "Function kernel_mul_mv_ext_bf16_f32_r1_5
was not found in the library"
SIGSEGV: segmentation violation, signal arrived during cgo execution
The [ERROR] line shows the failure is detected and logged. The SIGSEGV
immediately after shows the result is then used without a nil check. A returned
error at that point would have made this a one-line diagnosis.
How to see it at all, which is the second half of the report
The backend's stderr does not reach LocalAI's log. To capture it we had to edit
the backend's own launcher, appending a redirect to the exec line in
run.sh:
exec "$CURDIR"/stablediffusion-ggml "$@" 2>>/tmp/sd-ggml-stderr.log
then reproduce, then restore the file. That is not a reasonable diagnostic path
for an operator, and it is the only reason the missing kernel in the companion
report was ever identified (companion issue: #AAA).
What would help
- Check the pipeline before using it. A failed
newComputePipelineState(or
equivalent) should propagate an error rather than being dereferenced. This is
the fix that matters; it converts an opaque crash into an actionable message
for every future missing-kernel case, not just this one. - Surface backend stderr in the LocalAI log, or document how to capture it.
A backend that dies withexitCode=2and no output leaves the operator with
nothing. - Optionally, include the Metal error text in the gRPC error returned to
LocalAI, sorpc error: EOFcarries a cause.
Why this is worth fixing independently of the missing kernel
The companion report is one absent kernel in one build, and a rebuild will fix
it. This defect turns any future pipeline-compile failure, from any cause,
into an uninformative segfault with no log output. The next occurrence will cost
the next person the same days.
Notes
Happy to run diagnostics or test a patched build on request. The model and
hardware stay available specifically to re-test.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure through a /video request and inspect the stablediffusion-ggml pipeline-creation path alongside the launcher in run.sh. Confirm that a failed Metal pipeline returns an error instead of being used, and that backend stderr or its cause reaches the LocalAI log; rerun the reproduction to verify there is no segfault.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, shell
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100