benchmark-action / benchmark-action/github-action-benchmark

Error: No benchmark found for bench suite. Possibly mangled output from Catch2:

Open
#55 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.3k
Forks
184
PR merge metrics
No merged PRs in 30d

Description

I'm about to set up continuous benchmarking for our project. We benchmark C++ code in a combined R/C++ project with catch2 in a docker container.

When I try to deploy the benchmark results, I get an error that I struggle to approach:

`Error: No benchmark found for bench suite. Possibly mangled output from Catch2:`

The benchmark results look fine, however:

```
Run leoholz/github-action-benchmark@v1
with:
name: Spectre Benchmark
tool: catch2
output-file-path: benchmark_result.txt
github-token: ***
auto-push: true
alert-threshold: 150%
comment-on-alert: true
fail-on-alert: false
alert-comment-cc-users: @bitbacchus
gh-pages-branch: gh-pages
benchmark-data-dir-path: dev/bench
skip-fetch-gh-pages: false
comment-always: false
save-data-file: true
/usr/bin/docker exec a9f0ac018dd04072460f5c72e60419a21237b5e40c9c55dd861053ffe2aeba52 sh -c "cat /etc/*release | grep ^ID"
Error: No benchmark found for bench suite. Possibly mangled output from Catch2:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
benchmark is a Catch v2.13.4 host application.
Run with -? for options

-------------------------------------------------------------------------------
MinConf benchmarks
-------------------------------------------------------------------------------
../../tests/benchmark/benchmark-minconf.cpp:6
...............................................................................

benchmark name samples iterations estimated
mean low mean high mean
std dev low std dev high std dev
-------------------------------------------------------------------------------
MinConf optimize 3 sites, 3
species, <<5k iterations 100 1 146.934 ms
1.42295 ms 1.42224 ms 1.42373 ms
3.80964 us 3.33283 us 4.54339 us

MinConf optimize 100 sites, 139
species, 5 iterations 100 1 22.8952 s
227.471 ms 227.076 ms 227.933 ms
2.18339 ms 1.78353 ms 3.1476 ms

Calculate commonness for a 100x100
solution 100 1 35.275 ms
353.03 us 352.65 us 353.71 us
2.52869 us 1.67274 us 3.89837 us

===============================================================================
test cases: 1 | 1 passed
assertions: - none -

```

Do you have an idea or suggestions?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.