canonical / canonical/inference-snaps-dev

Smoke tests crash when rapidly switching between engines

Open
#59 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
1
Forks
0
Avg merge
1d 13m
Merged PRs (30d)
6

Description

```
$ sudo ./smoke-test.sh gemma4 intel-cpu
...
[INFO] Testing engine switch with retry logic...
[INFO] Switching to engine: intel-cpu
Engine changed to "intel-cpu".
[INFO] ✓ Successfully switched to engine: intel-cpu
[INFO] Verifying engine switch via status command...
[INFO] ✓ Engine switch verified via status command

=== All Smoke Tests Completed Successfully! ===
$
$ sudo ./smoke-test.sh gemma4 intel-gpu

=== Starting Smoke Tests ===
[INFO] Running tests against snap: gemma4
[INFO] Selected engine: intel-gpu
[INFO] Attempt 1/50: Checking whether port 8336 is listening
[INFO] Port 8336 is listening

=== API Endpoint Tests ===
[INFO] Testing List available models: http://127.0.0.1:8336/v3/models
[INFO] Attempt 1/50: List available models
{"data":[],"object":"list"}
[INFO] ✓ List available models: Pass
[INFO] Testing chat completion endpoint...
Chat payload:
{
"model": "gemma4-e4b-it-int4-ov",
"messages": [
{
"role": "user",
"content": "Hello!"
}
],
"temperature": 0,
"max_tokens": 5
}
[INFO] Attempt 1/3: Chat completion
++ curl -X POST http://127.0.0.1:8336/v3/chat/completions -H 'Content-Type: application/json' --max-time 10 --retry 0 -d '{"model":"gemma4-e4b-it-int4-ov","messages":[{"role":"user","content":"Hello!"}],"temperature":0,"max_tokens":5}' --fail-with-body
+++ error_handler 235 271 'curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null' run_api_tests
+++ local exit_code=22
+++ local line_no=235
+++ local bash_lineno=271
+++ local 'last_command=curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null'
+++ local func_name=run_api_tests
+++ log_error 'Script failed with exit code 22'
+++ echo -e '\033[0;31m[ERROR]\033[0m Script failed with exit code 22'
[ERROR] Script failed with exit code 22
+++ log_error 'Error occurred in function: run_api_tests'
+++ echo -e '\033[0;31m[ERROR]\033[0m Error occurred in function: run_api_tests'
[ERROR] Error occurred in function: run_api_tests
+++ log_error 'Failed command: curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null'
+++ echo -e '\033[0;31m[ERROR]\033[0m Failed command: curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null'
[ERROR] Failed command: curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null
+++ log_error 'Line number: 235'
+++ echo -e '\033[0;31m[ERROR]\033[0m Line number: 235'
[ERROR] Line number: 235
+++ log_error 'Bash line number: 271'
+++ echo -e '\033[0;31m[ERROR]\033[0m Bash line number: 271'
[ERROR] Bash line number: 271
+++ log_error 'Call stack:'
+++ echo -e '\033[0;31m[ERROR]\033[0m Call stack:'
[ERROR] Call stack:
+++ local frame=0
+++ caller 0
+++ local caller_info
++++ caller 0
+++ caller_info='235 test_chat_completion /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
+++ log_error ' [0] 235 test_chat_completion /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
+++ echo -e '\033[0;31m[ERROR]\033[0m [0] 235 test_chat_completion /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
[ERROR] [0] 235 test_chat_completion /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh
+++ (( frame++ ))
+++ caller 1
+++ local caller_info
++++ caller 1
+++ caller_info='271 run_api_tests /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
+++ log_error ' [1] 271 run_api_tests /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
+++ echo -e '\033[0;31m[ERROR]\033[0m [1] 271 run_api_tests /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
[ERROR] [1] 271 run_api_tests /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh
+++ (( frame++ ))
+++ caller 2
+++ local caller_info
++++ caller 2
+++ caller_info='439 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
+++ log_error ' [2] 439 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
+++ echo -e '\033[0;31m[ERROR]\033[0m [2] 439 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
[ERROR] [2] 439 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh
+++ (( frame++ ))
+++ caller 3
+++ local caller_info
++++ caller 3
+++ caller_info='464 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
+++ log_error ' [3] 464 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
+++ echo -e '\033[0;31m[ERROR]\033[0m [3] 464 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
[ERROR] [3] 464 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh
+++ (( frame++ ))
+++ caller 4
+++ exit 22
+ api_response='{"error":"Mediapipe graph definition with requested name is not found"}'
++ error_handler 235 271 'api_response=$(curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null)' run_api_tests
++ local exit_code=22
++ local line_no=235
++ local bash_lineno=271
++ local 'last_command=api_response=$(curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null)'
++ local func_name=run_api_tests
++ log_error 'Script failed with exit code 22'
++ echo -e '\033[0;31m[ERROR]\033[0m Script failed with exit code 22'
[ERROR] Script failed with exit code 22
++ log_error 'Error occurred in function: run_api_tests'
++ echo -e '\033[0;31m[ERROR]\033[0m Error occurred in function: run_api_tests'
[ERROR] Error occurred in function: run_api_tests
++ log_error 'Failed command: api_response=$(curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null)'
++ echo -e '\033[0;31m[ERROR]\033[0m Failed command: api_response=$(curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null)'
[ERROR] Failed command: api_response=$(curl -X POST "$base_url/chat/completions" -H "Content-Type: application/json" --max-time "$CURL_TIMEOUT" --retry 0 -d "$compact_json_body" --fail-with-body 2> /dev/null)
++ log_error 'Line number: 235'
++ echo -e '\033[0;31m[ERROR]\033[0m Line number: 235'
[ERROR] Line number: 235
++ log_error 'Bash line number: 271'
++ echo -e '\033[0;31m[ERROR]\033[0m Bash line number: 271'
[ERROR] Bash line number: 271
++ log_error 'Call stack:'
++ echo -e '\033[0;31m[ERROR]\033[0m Call stack:'
[ERROR] Call stack:
++ local frame=0
++ caller 0
++ local caller_info
+++ caller 0
++ caller_info='235 test_chat_completion /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
++ log_error ' [0] 235 test_chat_completion /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
++ echo -e '\033[0;31m[ERROR]\033[0m [0] 235 test_chat_completion /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
[ERROR] [0] 235 test_chat_completion /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh
++ (( frame++ ))
++ caller 1
++ local caller_info
+++ caller 1
++ caller_info='271 run_api_tests /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
++ log_error ' [1] 271 run_api_tests /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
++ echo -e '\033[0;31m[ERROR]\033[0m [1] 271 run_api_tests /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
[ERROR] [1] 271 run_api_tests /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh
++ (( frame++ ))
++ caller 2
++ local caller_info
+++ caller 2
++ caller_info='439 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
++ log_error ' [2] 439 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
++ echo -e '\033[0;31m[ERROR]\033[0m [2] 439 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
[ERROR] [2] 439 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh
++ (( frame++ ))
++ caller 3
++ local caller_info
+++ caller 3
++ caller_info='464 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
++ log_error ' [3] 464 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
++ echo -e '\033[0;31m[ERROR]\033[0m [3] 464 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh'
[ERROR] [3] 464 main /home/jpmeijers/inference-snaps-workspace/gemma4-snap/dev/smoke-test.sh
++ (( frame++ ))
++ caller 4
++ exit 22
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.