mudler / mudler/LocalAI

stablediffusion-ggml backend segfaults instead of returning an error when a Metal pipeline fails to compile, and its stderr never reaches the LocalAI log

Open
#11,529 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug unconfirmed
Dominant language
Go
Stars
49.2k
Forks
4.5k
Avg merge
1d 3m
Merged PRs (30d)
239

Description

Summary

When a Metal pipeline fails to compile, the backend logs the failure and then
uses the pipeline anyway. The null dereference takes SIGSEGV inside cgo and the
process dies. LocalAI sees only exitCode=2 and rpc error: EOF.

Compounding it, the backend's stderr is not attached to anything LocalAI logs, so
the underlying Metal error is invisible from the server side. Between the two,
a precise and well-reported error from Metal reaches the operator as an opaque
crash.

Environment

  • LocalAI 4.7.1
  • Backend metal-stablediffusion-ggml, digest
    sha256:9cafbf0956c55791cac472f20c73f55f425eeaefeb9d9a9bb22c9c5b8e8d7258
  • macOS 26.5.2 (build 25F84), Apple M3 Ultra, 256 GB unified memory

What the operator sees

Nothing useful. A /video request fails, and the LocalAI log contains only:

exitCode=2
rpc error: code = Unavailable desc = error reading from server: EOF

No Metal error, no kernel name, no indication that anything compiled or failed to
compile. Two earlier attempts against this backend produced exactly this and were
abandoned undiagnosed, because there was nothing to go on.

What is actually happening

Metal reports the failure correctly and precisely:

compiling pipeline: kernel_mul_mv_ext_bf16_f32_r1_5
[ERROR] MTLLibraryErrorDomain Code=5 "Function kernel_mul_mv_ext_bf16_f32_r1_5
        was not found in the library"
SIGSEGV: segmentation violation, signal arrived during cgo execution

The [ERROR] line shows the failure is detected and logged. The SIGSEGV
immediately after shows the result is then used without a nil check. A returned
error at that point would have made this a one-line diagnosis.

How to see it at all, which is the second half of the report

The backend's stderr does not reach LocalAI's log. To capture it we had to edit
the backend's own launcher, appending a redirect to the exec line in
run.sh:

exec "$CURDIR"/stablediffusion-ggml "$@" 2>>/tmp/sd-ggml-stderr.log

then reproduce, then restore the file. That is not a reasonable diagnostic path
for an operator, and it is the only reason the missing kernel in the companion
report was ever identified (companion issue: #AAA).

What would help

  1. Check the pipeline before using it. A failed newComputePipelineState (or
    equivalent) should propagate an error rather than being dereferenced. This is
    the fix that matters; it converts an opaque crash into an actionable message
    for every future missing-kernel case, not just this one.
  2. Surface backend stderr in the LocalAI log, or document how to capture it.
    A backend that dies with exitCode=2 and no output leaves the operator with
    nothing.
  3. Optionally, include the Metal error text in the gRPC error returned to
    LocalAI, so rpc error: EOF carries a cause.

Why this is worth fixing independently of the missing kernel

The companion report is one absent kernel in one build, and a rebuild will fix
it. This defect turns any future pipeline-compile failure, from any cause,
into an uninformative segfault with no log output. The next occurrence will cost
the next person the same days.

Notes

Happy to run diagnostics or test a patched build on request. The model and
hardware stay available specifically to re-test.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure through a /video request and inspect the stablediffusion-ggml pipeline-creation path alongside the launcher in run.sh. Confirm that a failed Metal pipeline returns an error instead of being used, and that backend stderr or its cause reaches the LocalAI log; rerun the reproduction to verify there is no segfault.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, shell
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.