pingcap / pingcap/tidb

sysbench failed with schema out of date when we try to push the time of PD-server forward 5 minutes

Open
#38,248 1 comment 0 reactions 0 assignees View on GitHub
sig/sql-infra type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement
Here is my cluster info :
version v5.2.1
3 pd+ 3 tikv+1 tidb

And my test step:

1. start sysbench
```
sysbench --config-file=sysbench_config oltp_update_index --tables=1 --table-size=1000000 run  
```
2. move pd-leader's time forward 5 minutes during sysbench running
```
date && date -s `date -d "+5 minutes" "+%H:%M:%S"` 
Thu Sep  8 10:54:05 CST 2022
Thu Sep  8 10:59:05 CST 2022
```

sysbench meet the following error
```
[ 31s ] thds: 10 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 32s ] thds: 10 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 33s ] thds: 10 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
FATAL: mysql_stmt_execute() returned error 8027 (Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV) for query 'UPDATE sbtest1 SET k=k+1 WHERE id=?'
(last message repeated 1 times)
FATAL: `thread_run' function failed: /usr/local/share/sysbench/oltp_common.lua:458: SQL error, errno = 8027, state = 'HY000': Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV
(last message repeated 1 times)
FATAL: mysql_stmt_execute() returned error 8027 (Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV) for query 'UPDATE sbtest1 SET k=k+1 WHERE id=?'
(last message repeated 1 times)
FATAL: `thread_run' function failed: /usr/local/share/sysbench/oltp_common.lua:458: SQL error, errno = 8027, state = 'HY000': Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV
(last message repeated 1 times)
FATAL: mysql_stmt_execute() returned error 8027 (Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV) for query 'UPDATE sbtest1 SET k=k+1 WHERE id=?'
FATAL: `thread_run' function failed: /usr/local/share/sysbench/oltp_common.lua:458: SQL error, errno = 8027, state = 'HY000': Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV
FATAL: mysql_stmt_execute() returned error 8027 (Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV) for query 'UPDATE sbtest1 SET k=k+1 WHERE id=?'
FATAL: `thread_run' function failed: /usr/local/share/sysbench/oltp_common.lua:458: SQL error, errno = 8027, state = 'HY000': Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV
FATAL: mysql_stmt_execute() returned error 8027 (Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV) for query 'UPDATE sbtest1 SET k=k+1 WHERE id=?'
FATAL: `thread_run' function failed: /usr/local/share/sysbench/oltp_common.lua:458: SQL error, errno = 8027, state = 'HY000': Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV
FATAL: mysql_stmt_execute() returned error 8027 (Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV) for query 'UPDATE sbtest1 SET k=k+1 WHERE id=?'
FATAL: `thread_run' function failed: /usr/local/share/sysbench/oltp_common.lua:458: SQL error, errno = 8027, state = 'HY000': Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV
FATAL: mysql_stmt_execute() returned error 8027 (Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV) for query 'UPDATE sbtest1 SET k=k+1 WHERE id=?'
FATAL: `thread_run' function failed: /usr/local/share/sysbench/oltp_common.lua:458: SQL error, errno = 8027, state = 'HY000': Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV
FATAL: mysql_stmt_execute() returned error 8027 (Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV) for query 'UPDATE sbtest1 SET k=k+1 WHERE id=?'
FATAL: `thread_run' function failed: /usr/local/share/sysbench/oltp_common.lua:458: SQL error, errno = 8027, state = 'HY000': Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV
Error in my_thread_global_end(): 10 threads didn't exit
```

It make sense that `sysbench` output error that schema is out of date,
![image](https://user-images.githubusercontent.com/1085697/192987898-2757425b-7bf5-476f-8072-18ec87495aeb.png)
However, It will be more friendly if we update the schema when it is out of date, and if the schema not change, the transaction can still be committed, could we?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.