Fix incorrect NDV (1) for VARCHAR columns in DBA_TAB_COL_STATISTICS during online statistics gathering with online_osg_sample / ONLINE_ESTIMATE_PERCENT
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 2.9k
- Forks
- 339
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 52
Description
Description
During online statistics gathering (specifically in the online_osg_sample test case related to the ONLINE_ESTIMATE_PERCENT preference), the NDV (Number of Distinct Values) for VARCHAR columns in the DBA_TAB_COL_STATISTICS view is incorrectly reported as 1.
Actual Behavior:
The NDV for VARCHAR columns is 1.
Expected Behavior:
The NDV should reflect the actual number of distinct values in the column.
Steps to Reproduce
-
Log into the test machine.
-
Navigate to the directory:
/obdata/data/AndroidTest/ob_opensource_test/tools/deploy. -
Execute the following commands to set up the test environment and run the specific mysqltest case:
export OCEANBASE_HOME=/obdata/data/AndroidTest/ob_opensource_test/ cd ${OCEANBASE_HOME}/tools/deploy source activate_obd.sh export OBD_SEEKDB_STATUS_FALLBACK_TCP=1 obd test mysqltest test_env --mysqltest-bin=${OCEANBASE_HOME}/deps/3rd/u01/obclient/bin/mysqltest --obclient-bin=${OCEANBASE_HOME}/deps/3rd/u01/obclient/bin/obclient --init-only --init-sql-dir=${OCEANBASE_HOME}/tools/deploy --init-sql-files='init.sql,init_user.sql|root@sys|test' obd test mysqltest test_env --mysqltest-bin=${OCEANBASE_HOME}/deps/3rd/u01/obclient/bin/mysqltest --obclient-bin=${OCEANBASE_HOME}/deps/3rd/u01/obclient/bin/obclient --disable-reboot --test-set=online_opt_stat_gather.online_osg_sample -
The test case executes the following SQL sequence which reproduces the issue:
USE test; DROP TABLE IF EXISTS t1, t2, t3, t4, t5; DROP PROCEDURE IF EXISTS batch_insert; CREATE TABLE t1 (c1 BIGINT PRIMARY KEY, c2 VARCHAR(256), c3 VARCHAR(256), c4 DATETIME, c5 DATETIME) WITH COLUMN GROUP (each column); CREATE TABLE t2 LIKE t1; CREATE TABLE t3 LIKE t1; CREATE TABLE t4 LIKE t1; CREATE TABLE t5 LIKE t1; DELIMITER $$ CREATE PROCEDURE batch_insert(IN args INT) BEGIN DECLARE i INT; SET i = 0; START TRANSACTION; WHILE i < args DO INSERT INTO t1(c1,c2,c3,c4,c5) VALUES ( i, CONCAT('C2_MKKHJHHS_XXXSDSDSDDSDSSJJJJDSHH_', i), CONCAT('C3_MKKHJHHS_XXXMMMMMMJJJDSHH_', i), '2023-01-01 01:11:11', '2023-01-01 01:11:11' ); SET i = i + 1; IF i % 10000 = 0 THEN COMMIT; END IF; END WHILE; COMMIT; END$$ DELIMITER ; CALL batch_insert(100000); CALL dbms_stats.set_global_prefs('ONLINE_ESTIMATE_PERCENT', '1'); INSERT /*+ query_timeout(100000000) append parallel(2) enable_parallel_dml */ INTO t2 SELECT c1, c2, c3, c4, c5 FROM t1; INSERT /*+ query_timeout(100000000) append parallel(2) enable_parallel_dml */ INTO t3 SELECT c1, c2, c3, c4, c5 FROM t1; CALL dbms_stats.set_global_prefs('ONLINE_ESTIMATE_PERCENT', '50'); INSERT /*+ query_timeout(100000000) append parallel(2) enable_parallel_dml */ INTO t4 SELECT c1, c2, c3, c4, c5 FROM t1; INSERT /*+ query_timeout(100000000) append parallel(2) enable_parallel_dml */ INTO t5 SELECT c1, c2, c3, c4, c5 FROM t1; CALL dbms_stats.set_global_prefs('ONLINE_ESTIMATE_PERCENT', '1'); SELECT TABLE_NAME, NUM_ROWS, SAMPLE_SIZE FROM oceanbase.DBA_TAB_STATISTICS WHERE table_name IN ('t2','t3','t4','t5') ORDER BY table_name; SELECT TABLE_NAME, COLUMN_NAME, NUM_DISTINCT, LOW_VALUE, HIGH_VALUE FROM oceanbase.DBA_TAB_COL_STATISTICS WHERE table_name IN ('t2','t3','t4','t5') ORDER BY table_name, column_name;
The final queries will show that the NUM_DISTINCT for the VARCHAR columns (c2, c3) is 1, which is incorrect.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the online_opt_stat_gather.online_osg_sample mysqltest case with the supplied setup commands, then inspect the online statistics gathering implementation for VARCHAR NDV handling. Confirm the fix by querying oceanbase.DBA_TAB_COL_STATISTICS for c2 and c3 in t2 through t5 and verifying NUM_DISTINCT reflects the inserted distinct values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- databases, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100