Establish Performance Regression Test for Lite Edition

Description

For the distributed edition, there is a performance verification job: https://ob-jenkins.alibaba-inc.com/job/perf_verification/

The Lite edition also needs a similar task established.

Use Cases / Scenarios

1. As a checkpoint for pre-release testing.
2. For daily bug fixes, developers may need to check if performance is impacted.
3. Historical version data needs to be traceable for future reference.

Requirements

1. Comprehensive Scenarios: Should include TP, AP, vector, hybrid search, etc. These can be gradually enriched over time.
2. Strict for Vector, Lenient for Others: Vector scenarios should have strict checks. For TP, AP, and other scenarios, the checks can be more lenient, allowing them to pass as long as there is no major regression. The key is to have records for traceability.
3. Minimize Execution Time: The faster the tests run, the faster the iteration speed. For example, AP tests could run only TPCDS, and TPCDS could be run with only a 1G dataset.

Start by reviewing the existing distributed-edition performance verification job at https://ob-jenkins.alibaba-inc.com/job/perf_verification/ and identify how a Lite-edition job would fit. Define the initial TP, AP, vector, and hybrid-search scenarios, using the proposed smaller datasets and stricter vector checks. Done means the Lite job runs as a pre-release or daily check and preserves results for historical comparison.