envoyproxy / envoyproxy/nighthawk
Consider generalizing the adaptive load "testing stage"
- Dominant language
- C++
- Stars
- 414
- Forks
- 95
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 11
Description
Today we have to deal with the adjusting stage and testing stage having separate durations, and the fact that the traffic template is a full `CommandLineOptions` with a duration field that is ignored.
One basic idea I had was to make `StepController::GetLatestCommandLineOptions()` take a duration input, so the `CommandLineOptions` is completely ready to send. But in some sense that just pushes the weird treatment of duration to another place.
I'm not sure that the idea of a dedicated testing stage really needs to be baked into the framework. During the design review we talked about exploring the input space, rather than just searching for a single target RPS. After such an exploration, the adaptive load framework would force you to do a testing stage you aren't interested in, and you would discard the result.
More generic would be to have the `StepController` decide everything, with duration being no different from anything else in `CommandLineOptions`. Some `StepController`s, like the basic one, could have a "search and then test" behavior where they run one longer benchmark at the end, but it would be the `StepController` deciding it. The main loop could just do whatever the `StepController` wants until it says it's time to exit, then not have any testing stage.
Concretely, this would mean:
- Deleting the durations from the `AdaptiveLoadSessionSpec` proto
- Adding duration settings to each `StepController`
- Deleting the testing stage code that follows the main loop
- Deleting the testing stage from the output proto
Duration is certainly an `InputVariableSetter` plugin we could trivially support, as suggested in https://github.com/envoyproxy/nighthawk/pull/535#pullrequestreview-488483737
Contributor guide
Assessment
This issue has not been assessed yet.