BotBlake / BotBlake/jellybench_py

Does not detect improper HWA setup on Linux for GPUs

Open
#60 4 comments 0 reactions 1 assignee Claimed by @BotBlake View on GitHub
bug
Dominant language
Python
Stars
28
Forks
12
Avg merge
3d 9h
Merged PRs (30d)
1

Description

### Classifications

- [x] Running `python ./jellybench_py/hwi.py` manually (within the Poetry shell) works and produces correct results.
- [ ] This happens on a Virtual Hardware (VM, WSL, ...)
- [x] The issue occurs during the benchmark section of the script (after `Starting Benchmark...`).
- [ ] A server issue or faulty server response is causing the problem.
- [ ] jellybench was run in `--debug` mode (recommended for issue reports).
- [x] I am using the latest version (branch `develop`) of jellybench. I understand older versions are not supported.

### Description

When running on Linux, the following things might happen:
- The system does not have the correct GPU firmware installed
- The system does not have a new enough version of the kernel, mesa etc.
- The user running the benchmark tool might not have permission to use the GPU

In many of these cases, the program can still detect the presence of the GPUs, but will fail to run benchmarks on them.
The program should run a test before starting benchmarks to detect if things are setup properly.
There are commands in the Jellyfin HWA docs on what ffmpeg commands can be used to test for such cases.

### Error Code or Output

```shell

```

### Steps to Reproduce

1. Start Jellybench on a Linux system with improper hardware acceleration setup (eg. user without permissions to use GPU)
2. Run benchmarks
3. No errors are reported, benchmarks silently fail

### Operating System

Linux

### System Architecture

x86_64 (64-bit)

### Python Version

3.12

### CPU[s]

Intel Core i5-10400F

### GPU[s]

Intel Arc A750

### Additional Context

_No response_

### Possible Solution

1. Run a test command to test if HWA was actually properly setup (commands can be found in Jellyfin HWA docs)
2. Print out a message every time a test fails

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.