BotBlake / BotBlake/jellybench_py

Crash if ffmpeg reports N/A for speed

Open
#70 0 comments 2 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
28
Forks
12
Avg merge
3d 9h
Merged PRs (30d)
1

Description

### Classifications

- [ ] Running `python ./jellybench_py/hwi.py` manually (within the Poetry shell) works and produces correct results.
- [ ] This happens on a Virtual Hardware (VM, WSL, ...)
- [x] The issue occurs during the benchmark section of the script (after `Starting Benchmark...`).
- [ ] A server issue or faulty server response is causing the problem.
- [x] jellybench was run in `--debug` mode (recommended for issue reports).
- [x] I am using the latest version (branch `develop`) of jellybench. I understand older versions are not supported.

### Description

If ffmpeg returns N/A for speed for whatever reason, the program throws an exception and exits.

If N/A is returned for speed the same test should be re-ran

### Error Code or Output

```shell
Status: Done Workers: 1Last Speed: 01.12 0% [ ] ETA: --:--:--> > Current Test: 720p - 1080p
> > > Current Device: cpu
> > > > Workers: 1, Last Speed: -0.5
> > > > Workers: 4, Last Speed: 3.4215384615384616
> > > > Scaling back to: 3, Last Speed: 0.8399387755102041

Traceback (most recent call last):
File "", line 1, in
File "/Users/user/Documents/GitHub/pyTAB/jellybench_py/core.py", line 825, in main
return cli()
^^^^^
File "/Users/user/Documents/GitHub/pyTAB/jellybench_py/core.py", line 792, in cli
valid, runs, result = benchmark(
^^^^^^^^^^
File "/Users/user/Documents/GitHub/pyTAB/jellybench_py/core.py", line 197, in benchmark
output = worker.workMan(total_workers, ffmpeg_cmd)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user/Documents/GitHub/pyTAB/jellybench_py/worker.py", line 131, in workMan
speeds.append(float(new_line[6].split("=")[-1].replace("x", "")))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: could not convert string to float: 'N/A'
```

### Steps to Reproduce

1. Start benchmark
2. Hope it happens
3. If it doesn't happen then do it enough times until it happens

### Operating System

MacOS

### System Architecture

ARM64 (64-bit)

### Python Version

3.12.8

### CPU[s]

Apple M1 Pro

### GPU[s]

Apple M1 Pro

### Additional Context

_No response_

### Possible Solution

_No response_

Contributor guide

Open the contributing guide

Research direction

Start in jellybench_py/worker.py at workMan, especially the speed parsing shown at line 131, and trace its call from jellybench_py/core.py. Check how the benchmark handles an ffmpeg speed of N/A; done means that case causes the same test to run again instead of raising a float-conversion exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.