[Feature] improve testing framework
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
Status Quo
===
Currently,
1. all integration test classes instantiate their own cluster.
2. all integration test classes loads/unloads their own data from avro files.
This would be inefficient because the more test classes we add the more time spent in cluster setup / data loading / etc.
Proposal
===
This proposal is similar to pinot-controller test: a set of stateless test class that instantiates their own test clusters, and a stateful set where only one cluster is instantiated and shared across different test classes.
With the new test group setup (#8706), we now propose to restructure the testing framework
- Test framework structure
- merge `ControllerTestUtils` with `ControllerTest` and uses a singleton default instance for shared access across different test classes. (to avoid duplicating various static vs. non-static methods)
- [ ] (#8744)
- make `ClusterTest` from pinot-integration-test follow the same pattern as `ControllerTest` but now support other cluster components
- make `BaseClusterIntegrationTest` from pinot-integration-test follow the same pattern as `ClusterTest` but now support table/schema/test-cases setup.
- Test framework usage,
- shared cluster setup: for each concrete test class, it should either have a `@BeforeClass` method to setup by starting its own `BaseClusterIntegrationTest` instance; or reuse the `BaseClusterIntegraionTest` from the `@BeforeGroups` base cluster setup class - in this case, the base cluster setup class should instantiate a static test base object.
- shared data setup: for each concrete test class, it should either have `@BeforeClass/@AfterClass` to load/unload data for testing; or reuse the `BaseClusterIntegrationTest` from the `@BeforeGroups` base setup class
Contributor guide
Research direction
Start by reading the mentioned ControllerTestUtils, ControllerTest, ClusterTest, and BaseClusterIntegrationTest patterns, along with the related work in #8706 and #8744. Trace how integration tests currently create clusters and load Avro data. Done means the proposed shared and stateless test setup is implemented across the stated framework components and usage patterns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100