Performance results

Unlike binary pass/fail results, performance tests provide measurements which then need some post-processing to determine whether the test passed or failed. This can be done by comparing the measurements with various things: fixed threshold, previous runs on the same platform, same run on other platforms, other branches on same platform. Several runs may also be required if the results aren't stable enough to provide more data for statistics.

Part of this could be determined directly in the test job, if all the relevant data is provided (previous results, absolute thresholds etc.). Then the job could be directly sending pass/fail results to the API. But this is not always possible, in which case the test would submit the measurements but the status would still not be pass/fail. An event would be generated to signal the fact that the measurements have arrived and need some processing, then a pipeline service would be doing the work of translating it into a pass/fail value and "resolve" the result as such by submitting it to the API.

> Note: The data schema for results needs to be adjusted on the API side to include measurements. To handle the link between measurements and pass/fail results, either a new status value could be added or the results could be split into 2 nodes: one with just the measurement and status set to "complete" if the test was run successfully but no pass/fail result was produced, and then another node with the pass/fail status created by the post-processing pipeline service.

Start by reviewing the API result schema and the pipeline service described in the issue. Define how measurements, incomplete pass/fail results, and post-processing events should be represented; done means the API and processing flow support resolving performance measurements into pass/fail results.

Written by the indexing model from the issue text.