canonical / canonical/inference-snaps

Gemma4 restart loop on AMD gfx1103

Open
#337 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
50
Forks
6
Avg merge
1d 20h
Merged PRs (30d)
7

Description

### Bug description

```
Aug 20 16:47:40 anubis gemma4.server[55437]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x378)[0x77f680b09d58]
Aug 20 16:47:40 anubis gemma4.server[55437]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(llama_decode+0xf)[0x77f680b0bb6f]
Aug 20 16:47:40 anubis gemma4.server[55437]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-common.so.0(_Z25common_context_can_seq_rmP13llama_context+0xd9)[0x77f6810a4839]
Aug 20 16:47:40 anubis gemma4.server[55437]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_ZN19server_context_impl10load_modelER13common_params+0xc05)[0x77f68198b515]
Aug 20 16:47:40 anubis gemma4.server[55437]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_Z12llama_serveriPPc+0x2f0e)[0x77f6818d443e]
Aug 20 16:47:40 anubis gemma4.server[55437]: /usr/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x77f6813431ca]
Aug 20 16:47:40 anubis gemma4.server[55437]: /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x77f68134328b]
Aug 20 16:47:40 anubis gemma4.server[55437]: llama-server(+0x1315)[0x5f0c3c424315]
Aug 20 16:47:42 anubis gemma4.server[55173]: Error: signal: aborted (core dumped)
Aug 20 16:47:42 anubis systemd[1]: snap.gemma4.server.service: Main process exited, code=exited, status=1/FAILURE
Aug 20 16:47:42 anubis systemd[1]: snap.gemma4.server.service: Failed with result 'exit-code'.
Aug 20 16:47:42 anubis systemd[1]: snap.gemma4.server.service: Consumed 3.178s CPU time over 4.612s wall clock time, 449.3M memory peak.
Aug 20 16:47:42 anubis systemd[1]: snap.gemma4.server.service: Scheduled restart job, restart counter is at 25.
Aug 20 16:47:42 anubis systemd[1]: snap.gemma4.server.service: Ignoring invalid environment assignment 'export LIBVIRT_DEFAULT_URI=qemu+ssh:///system?no_tty=1': /etc/environment
Aug 20 16:47:42 anubis systemd[1]: Started snap.gemma4.server.service - Service for snap application gemma4.server.
Aug 20 16:47:42 anubis kernel: audit: type=1326 audit(1787258862.854:551): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55474 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:42 anubis kernel: audit: type=1326 audit(1787258862.936:552): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55542 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:43 anubis kernel: audit: type=1326 audit(1787258863.065:553): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55449 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:43 anubis kernel: audit: type=1326 audit(1787258863.114:554): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55611 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:43 anubis kernel: audit: type=1326 audit(1787258863.144:555): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55634 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:43 anubis kernel: audit: type=1326 audit(1787258863.175:556): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55657 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:43 anubis gemma4.server[55610]: + exec llama-server --model /snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf --alias gemma4-e4b-q4-k-m --mmproj /snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf --port 8336 --host 127.0.0.1 --no-warmup --sleep-idle-seconds 600
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.348 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.351 I device_info:
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.417 I - ROCm0 : AMD Radeon 780M Graphics (63285 MiB, 120931 MiB free)
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.419 I - CPU : AMD Ryzen 9 8945HS w/ Radeon 780M Graphics (126571 MiB, 126571 MiB free)
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.445 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | REPACK = 1 |
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.448 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.468 I srv init: running without SSL
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.517 I srv init: using 15 threads for HTTP server
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.120.597 I srv start: binding port with default address family
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.121.774 I srv llama_server: loading model
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.121.785 I srv load_model: loading model '/snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf'
Aug 20 16:47:43 anubis kernel: audit: type=1400 audit(1787258863.363:557): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=55610 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.308.405 I srv load_model: [mtmd] estimated worst-case memory usage of mmproj is 536.08 MiB
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.308.431 I common_init_result: fitting params to device memory ...
Aug 20 16:47:43 anubis gemma4.server[55610]: 0.00.308.432 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
Aug 20 16:47:44 anubis gemma4.server[55610]: 0.01.188.798 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:47:44 anubis gemma4.server[55610]: 0.01.189.304 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:47:44 anubis gemma4.server[55610]: 0.01.213.170 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list
Aug 20 16:47:45 anubis gemma4.server[55610]: 0.02.167.139 W init_audio: audio input is in experimental stage and may have reduced quality:
Aug 20 16:47:45 anubis gemma4.server[55610]: https://github.com/ggml-org/llama.cpp/discussions/13759
Aug 20 16:47:45 anubis gemma4.server[55610]: 0.02.167.188 I srv load_model: loaded multimodal model, '/snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf'
Aug 20 16:47:45 anubis gemma4.server[55610]: 0.02.167.199 I srv load_model: initializing slots, n_slots = 4
Aug 20 16:47:45 anubis gemma4.server[55610]: /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/ggml-cuda.cu:103: ROCm error
Aug 20 16:47:45 anubis gemma4.server[55610]: 0.02.200.770 E ROCm error: device kernel image is invalid
Aug 20 16:47:45 anubis gemma4.server[55610]: 0.02.200.773 E current device: 0, in function ggml_cuda_kernel_launch at /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/common.cuh:1639
Aug 20 16:47:45 anubis gemma4.server[55610]: 0.02.200.773 E hipGetLastError()
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(+0x1b136)[0x767929cfd136]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_print_backtrace+0x21a)[0x767929cfd5ba]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_abort+0x15b)[0x767929cfd79b]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e12b4c2)[0x76792370a4c2]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(_Z18ggml_cuda_op_scaleR25ggml_backend_cuda_contextP11ggml_tensor+0x22a)[0x76792382556a]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e135cb6)[0x767923714cb6]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e1321b6)[0x7679237111b6]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x82f)[0x767929d1b52f]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa1)[0x767929ea2e91]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0x118)[0x767929ea6f98]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x378)[0x767929eacd58]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(llama_decode+0xf)[0x767929eaeb6f]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-common.so.0(_Z25common_context_can_seq_rmP13llama_context+0xd9)[0x76792a447839]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_ZN19server_context_impl10load_modelER13common_params+0xc05)[0x76792ad2e515]
Aug 20 16:47:45 anubis gemma4.server[55703]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_Z12llama_serveriPPc+0x2f0e)[0x76792ac7743e]
Aug 20 16:47:45 anubis gemma4.server[55703]: /usr/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x76792a6e61ca]
Aug 20 16:47:45 anubis gemma4.server[55703]: /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x76792a6e628b]
Aug 20 16:47:45 anubis gemma4.server[55703]: llama-server(+0x1315)[0x642932523315]
Aug 20 16:47:47 anubis gemma4.server[55449]: Error: signal: aborted (core dumped)
Aug 20 16:47:47 anubis systemd[1]: snap.gemma4.server.service: Main process exited, code=exited, status=1/FAILURE
Aug 20 16:47:47 anubis systemd[1]: snap.gemma4.server.service: Failed with result 'exit-code'.
Aug 20 16:47:47 anubis systemd[1]: snap.gemma4.server.service: Consumed 3.059s CPU time over 4.608s wall clock time, 449.4M memory peak.
Aug 20 16:47:47 anubis systemd[1]: snap.gemma4.server.service: Scheduled restart job, restart counter is at 26.
Aug 20 16:47:47 anubis systemd[1]: snap.gemma4.server.service: Ignoring invalid environment assignment 'export LIBVIRT_DEFAULT_URI=qemu+ssh:///system?no_tty=1': /etc/environment
Aug 20 16:47:47 anubis systemd[1]: Started snap.gemma4.server.service - Service for snap application gemma4.server.
Aug 20 16:47:47 anubis kernel: audit: type=1326 audit(1787258867.356:558): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55748 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:47 anubis kernel: audit: type=1326 audit(1787258867.439:559): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55816 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:47 anubis kernel: audit: type=1326 audit(1787258867.574:560): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55724 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:47 anubis kernel: audit: type=1326 audit(1787258867.623:561): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55884 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:47 anubis kernel: audit: type=1326 audit(1787258867.652:562): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55907 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:47 anubis kernel: audit: type=1326 audit(1787258867.681:563): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55932 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:47 anubis gemma4.server[55883]: + exec llama-server --model /snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf --alias gemma4-e4b-q4-k-m --mmproj /snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf --port 8336 --host 127.0.0.1 --no-warmup --sleep-idle-seconds 600
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.295 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.299 I device_info:
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.367 I - ROCm0 : AMD Radeon 780M Graphics (63285 MiB, 121054 MiB free)
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.376 I - CPU : AMD Ryzen 9 8945HS w/ Radeon 780M Graphics (126571 MiB, 126571 MiB free)
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.404 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | REPACK = 1 |
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.407 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.475 I srv init: running without SSL
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.514 I srv init: using 15 threads for HTTP server
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.122.599 I srv start: binding port with default address family
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.123.750 I srv llama_server: loading model
Aug 20 16:47:47 anubis gemma4.server[55883]: 0.00.123.761 I srv load_model: loading model '/snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf'
Aug 20 16:47:48 anubis gemma4.server[55883]: 0.00.310.648 I srv load_model: [mtmd] estimated worst-case memory usage of mmproj is 536.08 MiB
Aug 20 16:47:48 anubis gemma4.server[55883]: 0.00.310.669 I common_init_result: fitting params to device memory ...
Aug 20 16:47:48 anubis gemma4.server[55883]: 0.00.310.669 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
Aug 20 16:47:48 anubis gemma4.server[55883]: 0.01.204.515 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:47:48 anubis gemma4.server[55883]: 0.01.204.965 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:47:48 anubis gemma4.server[55883]: 0.01.229.477 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list
Aug 20 16:47:49 anubis gemma4.server[55883]: 0.02.183.091 W init_audio: audio input is in experimental stage and may have reduced quality:
Aug 20 16:47:49 anubis gemma4.server[55883]: https://github.com/ggml-org/llama.cpp/discussions/13759
Aug 20 16:47:49 anubis gemma4.server[55883]: 0.02.183.140 I srv load_model: loaded multimodal model, '/snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf'
Aug 20 16:47:49 anubis gemma4.server[55883]: 0.02.183.152 I srv load_model: initializing slots, n_slots = 4
Aug 20 16:47:49 anubis gemma4.server[55883]: /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/ggml-cuda.cu:103: ROCm error
Aug 20 16:47:49 anubis gemma4.server[55883]: 0.02.217.178 E ROCm error: device kernel image is invalid
Aug 20 16:47:49 anubis gemma4.server[55883]: 0.02.217.182 E current device: 0, in function ggml_cuda_kernel_launch at /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/common.cuh:1639
Aug 20 16:47:49 anubis gemma4.server[55883]: 0.02.217.182 E hipGetLastError()
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(+0x1b136)[0x73a68f383136]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_print_backtrace+0x21a)[0x73a68f3835ba]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_abort+0x15b)[0x73a68f38379b]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e12b4c2)[0x73a68770a4c2]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(_Z18ggml_cuda_op_scaleR25ggml_backend_cuda_contextP11ggml_tensor+0x22a)[0x73a68782556a]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e135cb6)[0x73a687714cb6]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e1321b6)[0x73a6877111b6]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x82f)[0x73a68f3a152f]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa1)[0x73a68f528e91]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0x118)[0x73a68f52cf98]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x378)[0x73a68f532d58]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(llama_decode+0xf)[0x73a68f534b6f]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-common.so.0(_Z25common_context_can_seq_rmP13llama_context+0xd9)[0x73a68facd839]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_ZN19server_context_impl10load_modelER13common_params+0xc05)[0x73a6903b4515]
Aug 20 16:47:49 anubis gemma4.server[55979]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_Z12llama_serveriPPc+0x2f0e)[0x73a6902fd43e]
Aug 20 16:47:49 anubis gemma4.server[55979]: /usr/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x73a68fd6c1ca]
Aug 20 16:47:49 anubis gemma4.server[55979]: /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x73a68fd6c28b]
Aug 20 16:47:49 anubis gemma4.server[55979]: llama-server(+0x1315)[0x64204ef8e315]
Aug 20 16:47:51 anubis gemma4.server[55724]: Error: signal: aborted (core dumped)
Aug 20 16:47:51 anubis systemd[1]: snap.gemma4.server.service: Main process exited, code=exited, status=1/FAILURE
Aug 20 16:47:51 anubis systemd[1]: snap.gemma4.server.service: Failed with result 'exit-code'.
Aug 20 16:47:51 anubis systemd[1]: snap.gemma4.server.service: Consumed 3.102s CPU time over 4.535s wall clock time, 449.7M memory peak.
Aug 20 16:47:51 anubis systemd[1]: snap.gemma4.server.service: Scheduled restart job, restart counter is at 27.
Aug 20 16:47:51 anubis systemd[1]: snap.gemma4.server.service: Ignoring invalid environment assignment 'export LIBVIRT_DEFAULT_URI=qemu+ssh:///system?no_tty=1': /etc/environment
Aug 20 16:47:51 anubis systemd[1]: Started snap.gemma4.server.service - Service for snap application gemma4.server.
Aug 20 16:47:51 anubis kernel: audit: type=1326 audit(1787258871.852:565): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56018 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:51 anubis kernel: audit: type=1326 audit(1787258871.935:566): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56086 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:52 anubis kernel: audit: type=1326 audit(1787258872.065:567): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=55991 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:52 anubis kernel: audit: type=1326 audit(1787258872.113:568): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56155 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:52 anubis kernel: audit: type=1326 audit(1787258872.142:569): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56178 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:52 anubis kernel: audit: type=1326 audit(1787258872.170:570): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56201 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:52 anubis gemma4.server[56154]: + exec llama-server --model /snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf --alias gemma4-e4b-q4-k-m --mmproj /snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf --port 8336 --host 127.0.0.1 --no-warmup --sleep-idle-seconds 600
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.250 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.252 I device_info:
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.319 I - ROCm0 : AMD Radeon 780M Graphics (63285 MiB, 121128 MiB free)
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.321 I - CPU : AMD Ryzen 9 8945HS w/ Radeon 780M Graphics (126571 MiB, 126571 MiB free)
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.348 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | REPACK = 1 |
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.351 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.400 I srv init: running without SSL
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.436 I srv init: using 15 threads for HTTP server
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.120.524 I srv start: binding port with default address family
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.121.684 I srv llama_server: loading model
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.121.693 I srv load_model: loading model '/snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf'
Aug 20 16:47:52 anubis kernel: audit: type=1400 audit(1787258872.358:571): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=56154 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.308.717 I srv load_model: [mtmd] estimated worst-case memory usage of mmproj is 536.08 MiB
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.308.742 I common_init_result: fitting params to device memory ...
Aug 20 16:47:52 anubis gemma4.server[56154]: 0.00.308.742 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
Aug 20 16:47:53 anubis gemma4.server[56154]: 0.01.214.202 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:47:53 anubis gemma4.server[56154]: 0.01.214.677 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:47:53 anubis gemma4.server[56154]: 0.01.239.282 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list
Aug 20 16:47:54 anubis gemma4.server[56154]: 0.02.214.808 W init_audio: audio input is in experimental stage and may have reduced quality:
Aug 20 16:47:54 anubis gemma4.server[56154]: https://github.com/ggml-org/llama.cpp/discussions/13759
Aug 20 16:47:54 anubis gemma4.server[56154]: 0.02.214.858 I srv load_model: loaded multimodal model, '/snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf'
Aug 20 16:47:54 anubis gemma4.server[56154]: 0.02.214.870 I srv load_model: initializing slots, n_slots = 4
Aug 20 16:47:54 anubis gemma4.server[56154]: /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/ggml-cuda.cu:103: ROCm error
Aug 20 16:47:54 anubis gemma4.server[56154]: 0.02.248.061 E ROCm error: device kernel image is invalid
Aug 20 16:47:54 anubis gemma4.server[56154]: 0.02.248.064 E current device: 0, in function ggml_cuda_kernel_launch at /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/common.cuh:1639
Aug 20 16:47:54 anubis gemma4.server[56154]: 0.02.248.064 E hipGetLastError()
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(+0x1b136)[0x7a7f34fb8136]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_print_backtrace+0x21a)[0x7a7f34fb85ba]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_abort+0x15b)[0x7a7f34fb879b]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e12b4c2)[0x7a7f2b70a4c2]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(_Z18ggml_cuda_op_scaleR25ggml_backend_cuda_contextP11ggml_tensor+0x22a)[0x7a7f2b82556a]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e135cb6)[0x7a7f2b714cb6]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e1321b6)[0x7a7f2b7111b6]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x82f)[0x7a7f34fd652f]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa1)[0x7a7f3515de91]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0x118)[0x7a7f35161f98]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x378)[0x7a7f35167d58]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(llama_decode+0xf)[0x7a7f35169b6f]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-common.so.0(_Z25common_context_can_seq_rmP13llama_context+0xd9)[0x7a7f35702839]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_ZN19server_context_impl10load_modelER13common_params+0xc05)[0x7a7f35fe9515]
Aug 20 16:47:54 anubis gemma4.server[56265]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_Z12llama_serveriPPc+0x2f0e)[0x7a7f35f3243e]
Aug 20 16:47:54 anubis gemma4.server[56265]: /usr/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7a7f359a11ca]
Aug 20 16:47:54 anubis gemma4.server[56265]: /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7a7f359a128b]
Aug 20 16:47:54 anubis gemma4.server[56265]: llama-server(+0x1315)[0x5e5bf7c3a315]
Aug 20 16:47:56 anubis gemma4.server[55991]: Error: signal: aborted (core dumped)
Aug 20 16:47:56 anubis systemd[1]: snap.gemma4.server.service: Main process exited, code=exited, status=1/FAILURE
Aug 20 16:47:56 anubis systemd[1]: snap.gemma4.server.service: Failed with result 'exit-code'.
Aug 20 16:47:56 anubis systemd[1]: snap.gemma4.server.service: Consumed 3.100s CPU time over 4.530s wall clock time, 449.2M memory peak.
Aug 20 16:47:56 anubis systemd[1]: snap.gemma4.server.service: Scheduled restart job, restart counter is at 28.
Aug 20 16:47:56 anubis systemd[1]: snap.gemma4.server.service: Ignoring invalid environment assignment 'export LIBVIRT_DEFAULT_URI=qemu+ssh:///system?no_tty=1': /etc/environment
Aug 20 16:47:56 anubis systemd[1]: Started snap.gemma4.server.service - Service for snap application gemma4.server.
Aug 20 16:47:56 anubis kernel: audit: type=1326 audit(1787258876.348:572): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56303 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:56 anubis kernel: audit: type=1326 audit(1787258876.430:573): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56371 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:56 anubis kernel: audit: type=1326 audit(1787258876.580:574): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56277 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:47:56 anubis gemma4.server[56440]: + exec llama-server --model /snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf --alias gemma4-e4b-q4-k-m --mmproj /snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf --port 8336 --host 127.0.0.1 --no-warmup --sleep-idle-seconds 600
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.021 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.024 I device_info:
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.091 I - ROCm0 : AMD Radeon 780M Graphics (63285 MiB, 121136 MiB free)
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.093 I - CPU : AMD Ryzen 9 8945HS w/ Radeon 780M Graphics (126571 MiB, 126571 MiB free)
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.119 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | REPACK = 1 |
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.122 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.142 I srv init: running without SSL
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.191 I srv init: using 15 threads for HTTP server
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.121.273 I srv start: binding port with default address family
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.122.421 I srv llama_server: loading model
Aug 20 16:47:56 anubis gemma4.server[56440]: 0.00.122.430 I srv load_model: loading model '/snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf'
Aug 20 16:47:56 anubis kernel: audit: type=1400 audit(1787258876.867:578): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=56440 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 20 16:47:57 anubis gemma4.server[56440]: 0.00.307.293 I srv load_model: [mtmd] estimated worst-case memory usage of mmproj is 536.08 MiB
Aug 20 16:47:57 anubis gemma4.server[56440]: 0.00.307.315 I common_init_result: fitting params to device memory ...
Aug 20 16:47:57 anubis gemma4.server[56440]: 0.00.307.316 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
Aug 20 16:47:57 anubis gemma4.server[56440]: 0.01.188.785 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:47:57 anubis gemma4.server[56440]: 0.01.189.211 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:47:57 anubis gemma4.server[56440]: 0.01.212.063 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list
Aug 20 16:47:58 anubis gemma4.server[56440]: 0.02.169.047 W init_audio: audio input is in experimental stage and may have reduced quality:
Aug 20 16:47:58 anubis gemma4.server[56440]: https://github.com/ggml-org/llama.cpp/discussions/13759
Aug 20 16:47:58 anubis gemma4.server[56440]: 0.02.169.095 I srv load_model: loaded multimodal model, '/snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf'
Aug 20 16:47:58 anubis gemma4.server[56440]: 0.02.169.107 I srv load_model: initializing slots, n_slots = 4
Aug 20 16:47:58 anubis gemma4.server[56440]: /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/ggml-cuda.cu:103: ROCm error
Aug 20 16:47:58 anubis gemma4.server[56440]: 0.02.202.100 E ROCm error: device kernel image is invalid
Aug 20 16:47:58 anubis gemma4.server[56440]: 0.02.202.104 E current device: 0, in function ggml_cuda_kernel_launch at /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/common.cuh:1639
Aug 20 16:47:58 anubis gemma4.server[56440]: 0.02.202.104 E hipGetLastError()
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(+0x1b136)[0x7956f08dc136]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_print_backtrace+0x21a)[0x7956f08dc5ba]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_abort+0x15b)[0x7956f08dc79b]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e12b4c2)[0x7956e770a4c2]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(_Z18ggml_cuda_op_scaleR25ggml_backend_cuda_contextP11ggml_tensor+0x22a)[0x7956e782556a]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e135cb6)[0x7956e7714cb6]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e1321b6)[0x7956e77111b6]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x82f)[0x7956f08fa52f]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa1)[0x7956f0a81e91]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0x118)[0x7956f0a85f98]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x378)[0x7956f0a8bd58]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(llama_decode+0xf)[0x7956f0a8db6f]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-common.so.0(_Z25common_context_can_seq_rmP13llama_context+0xd9)[0x7956f1026839]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_ZN19server_context_impl10load_modelER13common_params+0xc05)[0x7956f190d515]
Aug 20 16:47:58 anubis gemma4.server[56535]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_Z12llama_serveriPPc+0x2f0e)[0x7956f185643e]
Aug 20 16:47:58 anubis gemma4.server[56535]: /usr/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7956f12c51ca]
Aug 20 16:47:58 anubis gemma4.server[56535]: /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7956f12c528b]
Aug 20 16:47:58 anubis gemma4.server[56535]: llama-server(+0x1315)[0x5a31f32a9315]
Aug 20 16:48:00 anubis gemma4.server[56277]: Error: signal: aborted (core dumped)
Aug 20 16:48:00 anubis systemd[1]: snap.gemma4.server.service: Main process exited, code=exited, status=1/FAILURE
Aug 20 16:48:00 anubis systemd[1]: snap.gemma4.server.service: Failed with result 'exit-code'.
Aug 20 16:48:00 anubis systemd[1]: snap.gemma4.server.service: Consumed 3.001s CPU time over 4.454s wall clock time, 449.7M memory peak.
Aug 20 16:48:00 anubis systemd[1]: snap.gemma4.server.service: Scheduled restart job, restart counter is at 29.
Aug 20 16:48:00 anubis systemd[1]: snap.gemma4.server.service: Ignoring invalid environment assignment 'export LIBVIRT_DEFAULT_URI=qemu+ssh:///system?no_tty=1': /etc/environment
Aug 20 16:48:00 anubis systemd[1]: Started snap.gemma4.server.service - Service for snap application gemma4.server.
Aug 20 16:48:00 anubis kernel: audit: type=1326 audit(1787258880.790:579): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56608 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:48:00 anubis kernel: audit: type=1326 audit(1787258880.873:580): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56676 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:48:01 anubis kernel: audit: type=1326 audit(1787258881.003:581): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56576 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:48:01 anubis kernel: audit: type=1326 audit(1787258881.052:582): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56745 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:48:01 anubis kernel: audit: type=1326 audit(1787258881.081:583): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56768 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:48:01 anubis kernel: audit: type=1326 audit(1787258881.109:584): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=56791 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
Aug 20 16:48:01 anubis gemma4.server[56744]: + exec llama-server --model /snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf --alias gemma4-e4b-q4-k-m --mmproj /snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf --port 8336 --host 127.0.0.1 --no-warmup --sleep-idle-seconds 600
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.407 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.410 I device_info:
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.476 I - ROCm0 : AMD Radeon 780M Graphics (63285 MiB, 120965 MiB free)
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.478 I - CPU : AMD Ryzen 9 8945HS w/ Radeon 780M Graphics (126571 MiB, 126571 MiB free)
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.504 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | REPACK = 1 |
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.508 I srv llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.573 I srv init: running without SSL
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.609 I srv init: using 15 threads for HTTP server
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.118.690 I srv start: binding port with default address family
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.119.868 I srv llama_server: loading model
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.119.877 I srv load_model: loading model '/snap/gemma4/components/252/model-e4b-q4-k-m-gguf/gemma-4-E4B-it-Q4_K_M.gguf'
Aug 20 16:48:01 anubis kernel: audit: type=1400 audit(1787258881.294:585): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=56744 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.305.098 I srv load_model: [mtmd] estimated worst-case memory usage of mmproj is 536.08 MiB
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.305.120 I common_init_result: fitting params to device memory ...
Aug 20 16:48:01 anubis gemma4.server[56744]: 0.00.305.120 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
Aug 20 16:48:02 anubis gemma4.server[56744]: 0.01.183.607 W load: control-looking token: 212 '' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:48:02 anubis gemma4.server[56744]: 0.01.184.046 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
Aug 20 16:48:02 anubis gemma4.server[56744]: 0.01.207.284 W load: special_eog_ids contains '<|tool_response>', removing '' token from EOG list
Aug 20 16:48:03 anubis gemma4.server[56744]: 0.02.166.658 W init_audio: audio input is in experimental stage and may have reduced quality:
Aug 20 16:48:03 anubis gemma4.server[56744]: https://github.com/ggml-org/llama.cpp/discussions/13759
Aug 20 16:48:03 anubis gemma4.server[56744]: 0.02.166.707 I srv load_model: loaded multimodal model, '/snap/gemma4/components/252/mmproj-e4b-q8-0-gguf/mmproj-gemma-4-E4B-it-Q8_0.gguf'
Aug 20 16:48:03 anubis gemma4.server[56744]: 0.02.166.720 I srv load_model: initializing slots, n_slots = 4
Aug 20 16:48:03 anubis gemma4.server[56744]: /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/ggml-cuda.cu:103: ROCm error
Aug 20 16:48:03 anubis gemma4.server[56744]: 0.02.201.617 E ROCm error: device kernel image is invalid
Aug 20 16:48:03 anubis gemma4.server[56744]: 0.02.201.620 E current device: 0, in function ggml_cuda_kernel_launch at /home/runner/work/llama.cpp-builds/llama.cpp-builds/ggml/src/ggml-cuda/common.cuh:1639
Aug 20 16:48:03 anubis gemma4.server[56744]: 0.02.201.620 E hipGetLastError()
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(+0x1b136)[0x7b41d176c136]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_print_backtrace+0x21a)[0x7b41d176c5ba]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_abort+0x15b)[0x7b41d176c79b]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e12b4c2)[0x7b41cb70a4c2]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(_Z18ggml_cuda_op_scaleR25ggml_backend_cuda_contextP11ggml_tensor+0x22a)[0x7b41cb82556a]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e135cb6)[0x7b41cb714cb6]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/mnt/llamacpp-rocm/251/bin/libggml-hip.so(+0x2e1321b6)[0x7b41cb7111b6]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x82f)[0x7b41d178a52f]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa1)[0x7b41d1911e91]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0x118)[0x7b41d1915f98]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(_ZN13llama_context6decodeERK11llama_batch+0x378)[0x7b41d191bd58]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama.so.0(llama_decode+0xf)[0x7b41d191db6f]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-common.so.0(_Z25common_context_can_seq_rmP13llama_context+0xd9)[0x7b41d1eb6839]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_ZN19server_context_impl10load_modelER13common_params+0xc05)[0x7b41d279d515]
Aug 20 16:48:03 anubis gemma4.server[56837]: /snap/gemma4/components/252/llamacpp-rocm/lib/libllama-server-impl.so(_Z12llama_serveriPPc+0x2f0e)[0x7b41d26e643e]
Aug 20 16:48:03 anubis gemma4.server[56837]: /usr/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7b41d21551ca]
Aug 20 16:48:03 anubis gemma4.server[56837]: /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7b41d215528b]
Aug 20 16:48:03 anubis gemma4.server[56837]: llama-server(+0x1315)[0x61ce35a6f315]
```

```
[Thu Aug 20 16:48:55 2026] audit: type=1326 audit(1787258935.064:665): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=59821 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:48:55 2026] audit: type=1326 audit(1787258935.114:666): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=59984 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:48:55 2026] audit: type=1326 audit(1787258935.141:667): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60007 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:48:55 2026] audit: type=1326 audit(1787258935.170:668): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60030 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:48:55 2026] audit: type=1400 audit(1787258935.357:669): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=59983 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:48:59 2026] audit: type=1326 audit(1787258939.350:670): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60114 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:48:59 2026] audit: type=1326 audit(1787258939.433:671): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60182 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:48:59 2026] audit: type=1326 audit(1787258939.583:672): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60088 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:48:59 2026] audit: type=1326 audit(1787258939.633:673): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60252 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:00 2026] audit: type=1326 audit(1787258939.661:674): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60275 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:00 2026] audit: type=1326 audit(1787258939.691:675): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60298 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:00 2026] audit: type=1400 audit(1787258939.879:676): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=60251 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:49:04 2026] audit: type=1326 audit(1787258943.853:677): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60413 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:04 2026] audit: type=1326 audit(1787258943.938:678): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60482 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:04 2026] audit: type=1326 audit(1787258944.070:679): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60390 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:04 2026] kauditd_printk_skb: 3 callbacks suppressed
[Thu Aug 20 16:49:04 2026] audit: type=1400 audit(1787258944.358:683): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=60550 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:49:08 2026] audit: type=1326 audit(1787258948.349:684): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60727 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:08 2026] audit: type=1326 audit(1787258948.436:685): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60795 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:08 2026] audit: type=1326 audit(1787258948.571:686): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60703 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:08 2026] audit: type=1326 audit(1787258948.621:687): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60866 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:08 2026] audit: type=1326 audit(1787258948.648:688): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60889 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:09 2026] audit: type=1326 audit(1787258948.677:689): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60912 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:09 2026] audit: type=1400 audit(1787258948.863:690): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=60865 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:49:13 2026] audit: type=1326 audit(1787258952.849:691): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60995 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:13 2026] audit: type=1326 audit(1787258952.930:692): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61063 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:13 2026] audit: type=1326 audit(1787258953.062:693): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=60969 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:13 2026] audit: type=1326 audit(1787258953.110:694): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61133 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:13 2026] audit: type=1326 audit(1787258953.138:695): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61156 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:13 2026] audit: type=1326 audit(1787258953.168:696): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61180 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:13 2026] audit: type=1400 audit(1787258953.357:697): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=61132 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:49:17 2026] audit: type=1326 audit(1787258957.351:698): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61278 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:17 2026] audit: type=1326 audit(1787258957.433:699): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61346 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:17 2026] audit: type=1326 audit(1787258957.563:700): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61255 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:18 2026] kauditd_printk_skb: 3 callbacks suppressed
[Thu Aug 20 16:49:18 2026] audit: type=1400 audit(1787258957.851:704): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=61414 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:49:22 2026] audit: type=1326 audit(1787258961.854:705): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61570 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:22 2026] audit: type=1326 audit(1787258961.938:706): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61638 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:22 2026] audit: type=1326 audit(1787258962.086:707): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61545 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:22 2026] audit: type=1326 audit(1787258962.134:708): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61710 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:22 2026] audit: type=1326 audit(1787258962.161:709): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61733 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:22 2026] audit: type=1326 audit(1787258962.187:710): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61756 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:22 2026] audit: type=1400 audit(1787258962.373:711): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=61709 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:49:26 2026] audit: type=1326 audit(1787258966.353:712): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61839 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:26 2026] audit: type=1326 audit(1787258966.434:713): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61907 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:26 2026] audit: type=1326 audit(1787258966.560:714): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61815 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:26 2026] audit: type=1326 audit(1787258966.603:715): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=61977 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:26 2026] audit: type=1326 audit(1787258966.627:716): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62000 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:26 2026] audit: type=1326 audit(1787258966.653:717): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62023 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:27 2026] audit: type=1400 audit(1787258966.837:718): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=61976 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:49:31 2026] audit: type=1326 audit(1787258970.853:719): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62123 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:31 2026] audit: type=1326 audit(1787258970.936:720): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62191 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:31 2026] audit: type=1326 audit(1787258971.067:721): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62099 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:31 2026] kauditd_printk_skb: 3 callbacks suppressed
[Thu Aug 20 16:49:31 2026] audit: type=1400 audit(1787258971.356:725): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=62260 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[Thu Aug 20 16:49:35 2026] audit: type=1326 audit(1787258975.351:726): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62390 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:35 2026] audit: type=1326 audit(1787258975.433:727): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62458 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:35 2026] audit: type=1326 audit(1787258975.561:728): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62366 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:35 2026] audit: type=1326 audit(1787258975.608:729): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62530 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:35 2026] audit: type=1326 audit(1787258975.635:730): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62553 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:36 2026] audit: type=1326 audit(1787258975.665:731): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.gemma4.server pid=62576 comm="modelctl" exe="/snap/gemma4/252/bin/modelctl" sig=0 arch=c000003e syscall=424 compat=0 ip=0x40b7ce code=0x50000
[Thu Aug 20 16:49:36 2026] audit: type=1400 audit(1787258975.850:732): apparmor="DENIED" operation="open" class="file" profile="snap.gemma4.server" name="/opt/" pid=62529 comm="llama-server" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
```

### To reproduce

tbc

### Snap version

```shell
latest/stable
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the snap.gemma4.server restart loop on AMD gfx1103 and review the llama-server launch shown in the logs. Focus on the ROCm error, “device kernel image is invalid,” and the packaged llamacpp-rocm component; done means the Gemma4 service starts without aborting and repeatedly restarting.

Written by the indexing model from the issue text.

Assessment

Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.