TiDB TTL Bug Issue
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
### 1. Minimal reproduce step (Required)
Step 1: Create a partitioned table with TTL on stop_time
```sql
CREATE TABLE `xxx` (
`key_id` bigint NOT NULL AUTO_INCREMENT,
`start_time` datetime(3) NOT NULL COMMENT 'call start time',
`stop_time` datetime(3) DEFAULT NULL COMMENT 'call stop time',
PRIMARY KEY (`key_id`,`start_time`) CLUSTERED,
KEY `idx_start_time` (`start_time`)
) ENGINE=InnoDB
PARTITION BY RANGE (TO_DAYS(`start_time`))
(PARTITION `p20250512` VALUES LESS THAN (739749),
-- ... 273 partitions total ...
)
/*T![ttl] TTL=`stop_time` + INTERVAL 6 MONTH */ -- ⚠️ Initially configured on stop_time
/*T![ttl] TTL_ENABLE='ON' */
/*T![ttl] TTL_JOB_INTERVAL='1h' */;
```
Step 2: Modify TTL column from stop_time to start_time
```sql
ALTER TABLE `xxx`
TTL = `start_time` + INTERVAL 6 MONTH -- ✅ Changed to start_time
TTL_ENABLE = 'ON'
TTL_JOB_INTERVAL = '1h';
Step 3: Verify the configuration is updated
mysql> SHOW CREATE TABLE `xxx`\G
-- Output shows:
/*T![ttl] TTL=`start_time` + INTERVAL 6 MONTH */ -- ✅ Configuration updated correctly
/*T![ttl] TTL_ENABLE='ON' */
/*T![ttl] TTL_JOB_INTERVAL='1h' */
```
Step 4: Wait for TTL job to run
-- Check TTL job history
```sql
SELECT job_id, partition_name, create_time, expired_rows, deleted_rows
FROM mysql.tidb_ttl_job_history
WHERE partition_name = 'p20250512'
ORDER BY create_time DESC LIMIT 5;
-- Result: All jobs report 0 expired rows
```
Step 5: Check actual TTL query execution
```sql
-- From information_schema.SLOW_QUERY, observe TTL is still using old column:
SELECT LOW_PRIORITY SQL_NO_CACHE `key_id`, `start_time`
FROM `xxx` PARTITION(`p20250512`)
WHERE `key_id` < 3422893381
AND `stop_time` < FROM_UNIXTIME(1747023531) -- ❌ Still using stop_time!
ORDER BY `key_id`, `start_time` ASC
LIMIT 500;
```
### 2. What did you expect to see? (Required)
Expected Behavior:
After executing ALTER TABLE ... TTL = start_time + INTERVAL 6 MONTH:
1. TTL metadata should be updated in system tables
2. TTL internal queries should use the new column (start_time):
```sql
WHERE `key_id` < xxx
AND `start_time` < FROM_UNIXTIME(xxx) -- ✅ Should use start_time
```
3. TTL jobs should find and delete expired rows based on start_time
4. Data with start_time < NOW() - 6 MONTH should be deleted
### 3. What did you see instead (Required)
Actual Behavior:
1. TTL configuration shows updated correctly in SHOW CREATE TABLE:
/*T![ttl] TTL=`start_time` + INTERVAL 6 MONTH */ -- ✅ Shows start_time
2. BUT TTL internal execution still uses old column (stop_time):
WHERE `key_id` < xxx
AND `stop_time` < FROM_UNIXTIME(xxx) -- ❌ Still using stop_time
3. TTL jobs report zero rows because the wrong column is being checked:
```json
{
"total_rows": 0,
"success_rows": 0,
"error_rows": 0,
"expired_rows": 0,
"deleted_rows": 0
}
```
4. Millions of expired rows remain:
```sql
mysql> SELECT COUNT(*) FROM xxx
WHERE start_time < DATE_SUB(NOW(), INTERVAL 6 MONTH);
+----------+
| COUNT(*) |
+----------+
| 27610766 | -- ❌ Should have been deleted
+----------+
```
5. Manual verification shows the discrepancy:
```sql
-- Using start_time (expected by configuration) - returns rows:
SELECT COUNT(*) FROM xxx PARTITION(p20250512)
WHERE start_time < FROM_UNIXTIME(1747023531);
-- Result: 21728446 rows
-- Using stop_time (what TTL is actually using) - returns zero:
SELECT COUNT(*) FROM xxx PARTITION(p20250512)
WHERE stop_time < FROM_UNIXTIME(1747023531);
-- Result: 0 rows
```
Attempted fixes (all failed):
✗ ALTER TABLE ... TTL_ENABLE='OFF' then TTL_ENABLE='ON'
✗ Running ANALYZE TABLE to refresh statistics
✗ Waiting for multiple TTL job cycles (24+ hours)
✗ Restarting TiDB server
### 4. What is your TiDB version? (Required)
Release Version: v8.5.3
Edition: Community
Git Commit Hash: dc2548aac79a712265e831cff2a3a896bc0a5a38
Git Branch: HEAD
UTC Build Time: 2025-07-31 13:54:44
GoVersion: go1.23.8
Race Enabled: false
Check Table Before Drop: false
Store: tikv
Contributor guide
Assessment
This issue has not been assessed yet.