envoyproxy / envoyproxy/nighthawk
Reconsider failure predicate defaults and/or consider shortcuts for setting them
- Dominant language
- C++
- Stars
- 414
- Forks
- 95
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 11
Description
Should Nighthawk exit by default after a single 5xx? We were suprised to discover an benchmark had been exiting after less than the specified time because of errors, and it was hard to know what was happening. We found that a failure mode of our system under test is elevated 5xx rates, and it would be good to measure them.
The only reason I know of for exiting on the first error (which I think is the default failure predicate, but I could be wrong) is that if you misconfigure Nighthawk, you don't have to wait till the end of the requested duration to find out.
Spinning off offline discussion with dubious90:
- Potentially adding a shorthand for this kind of thing?
- Maybe something like "default_failure_predicate"?
Contributor guide
Assessment
This issue has not been assessed yet.