apache / apache/datasketches-bigquery
Perform Scale Tests for all sketch functions across different BQ regions
- Dominant language
- JavaScript
- Stars
- 21
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
Since there is no good way to automate integration tests at scale at the moment that could potentially have caught some of the recent issues we have seen, can we try the following to ensure we are not missing other potential bugs :
1) Come up with a large enough ( ~ x Billion records ) sample table
2) Create sample queries for each of the sketch functions ( with group by clauses in relevant cases )
3) Execute these queries in each of the BQ regions
Since the sketch functions are deployed in each regional BQ public dataset, performing #3 should be trivial.
Contributor guide
No contributing guide indexed for this repository
Research direction
No files or tests are named. Start by inventorying all sketch functions and the regional BigQuery public datasets, then define a sufficiently large sample table and representative grouped queries. Done means each query has been executed in every region and the results have been checked for potential bugs.
Written by the indexing model from the issue text.
Assessment
- Domain
- data, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100