apache / apache/pinot

[Feature] improve testing framework

Open
#8,745 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Status Quo
===
Currently,
1. all integration test classes instantiate their own cluster.
2. all integration test classes loads/unloads their own data from avro files.
This would be inefficient because the more test classes we add the more time spent in cluster setup / data loading / etc.

Proposal
===
This proposal is similar to pinot-controller test: a set of stateless test class that instantiates their own test clusters, and a stateful set where only one cluster is instantiated and shared across different test classes.

With the new test group setup (#8706), we now propose to restructure the testing framework
- Test framework structure
- merge `ControllerTestUtils` with `ControllerTest` and uses a singleton default instance for shared access across different test classes. (to avoid duplicating various static vs. non-static methods)
- [ ] (#8744)
- make `ClusterTest` from pinot-integration-test follow the same pattern as `ControllerTest` but now support other cluster components
- make `BaseClusterIntegrationTest` from pinot-integration-test follow the same pattern as `ClusterTest` but now support table/schema/test-cases setup.

- Test framework usage,
- shared cluster setup: for each concrete test class, it should either have a `@BeforeClass` method to setup by starting its own `BaseClusterIntegrationTest` instance; or reuse the `BaseClusterIntegraionTest` from the `@BeforeGroups` base cluster setup class - in this case, the base cluster setup class should instantiate a static test base object.
- shared data setup: for each concrete test class, it should either have `@BeforeClass/@AfterClass` to load/unload data for testing; or reuse the `BaseClusterIntegrationTest` from the `@BeforeGroups` base setup class

Contributor guide

Open the contributing guide

Research direction

Start by reading the mentioned ControllerTestUtils, ControllerTest, ClusterTest, and BaseClusterIntegrationTest patterns, along with the related work in #8706 and #8744. Trace how integration tests currently create clusters and load Avro data. Done means the proposed shared and stateless test setup is implemented across the stated framework components and usage patterns.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.