benchmark-action / benchmark-action/github-action-benchmark
Error: No benchmark found for bench suite. Possibly mangled output from Catch2:
- Lingua principale
- TypeScript
- Stelle
- 1.3k
- Fork
- 184
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
I'm about to set up continuous benchmarking for our project. We benchmark C++ code in a combined R/C++ project with catch2 in a docker container.
When I try to deploy the benchmark results, I get an error that I struggle to approach:
`Error: No benchmark found for bench suite. Possibly mangled output from Catch2:`
The benchmark results look fine, however:
```
Run leoholz/github-action-benchmark@v1
with:
name: Spectre Benchmark
tool: catch2
output-file-path: benchmark_result.txt
github-token: ***
auto-push: true
alert-threshold: 150%
comment-on-alert: true
fail-on-alert: false
alert-comment-cc-users: @bitbacchus
gh-pages-branch: gh-pages
benchmark-data-dir-path: dev/bench
skip-fetch-gh-pages: false
comment-always: false
save-data-file: true
/usr/bin/docker exec a9f0ac018dd04072460f5c72e60419a21237b5e40c9c55dd861053ffe2aeba52 sh -c "cat /etc/*release | grep ^ID"
Error: No benchmark found for bench suite. Possibly mangled output from Catch2:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
benchmark is a Catch v2.13.4 host application.
Run with -? for options
-------------------------------------------------------------------------------
MinConf benchmarks
-------------------------------------------------------------------------------
../../tests/benchmark/benchmark-minconf.cpp:6
...............................................................................
benchmark name samples iterations estimated
mean low mean high mean
std dev low std dev high std dev
-------------------------------------------------------------------------------
MinConf optimize 3 sites, 3
species, <<5k iterations 100 1 146.934 ms
1.42295 ms 1.42224 ms 1.42373 ms
3.80964 us 3.33283 us 4.54339 us
MinConf optimize 100 sites, 139
species, 5 iterations 100 1 22.8952 s
227.471 ms 227.076 ms 227.933 ms
2.18339 ms 1.78353 ms 3.1476 ms
Calculate commonness for a 100x100
solution 100 1 35.275 ms
353.03 us 352.65 us 353.71 us
2.52869 us 1.67274 us 3.89837 us
===============================================================================
test cases: 1 | 1 passed
assertions: - none -
```
Do you have an idea or suggestions?
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Riprodurre il workflow usando leoholz/github-action-benchmark@v1, con tool impostato su catch2 e output-file-path su benchmark_result.txt. Esaminare come l’action analizza l’output mostrato di Catch2 v2.13.4, in particolare i nomi dei benchmark e l’intestazione della suite. Il lavoro è completato quando l’output del benchmark esistente viene accettato e il deployment termina senza l’errore «No benchmark found».
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- cpp, docker, typescript
- Ambito
- ci-cd, testing-qa, tooling
- Tipo di issue
- Bug
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100