after "alter database replica 3" , the transaction keeps in the redoAction stage
Open
Nobody has claimed this yet.
bug
- Dominant language
- C
- Stars
- 25.1k
- Forks
- 5k
- Avg merge
- 4d 59m
- Merged PRs (30d)
- 7
Description
事务详情如下所示
taos> show transactions\G;
*************************** 1.row ***************************
id: 93177
create_time: 2023-12-22 20:08:55.710
stage: redoAction
oper: alter-db
db: program_analyze
stable:
failed_times: 29299
last_exec_time: 2023-12-23 14:17:46.859
last_action_info: action:20 code:0x520(Vnode not exist) msgType:alter-replica numOfEps:1 inUse:0 ep:0-h2.taosdata.com:6030
Query OK, 1 row(s) in set (0.001840s)
并且某个子节点一直报同步失败的错误,并出现core,无法正常启动加入集群
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/bin/taosd'.
Program terminated with signal 6, Aborted.
#0 0x00007f96690a0337 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-292.el7.x86_64 libgcc-4.8.5-39.el7.x86_64 libstdc++-4.8.5-39.el7.x86_64
(gdb) bt
#0 0x00007f96690a0337 in raise () from /lib64/libc.so.6
#1 0x00007f96690a1b68 in abort () from /lib64/libc.so.6
#2 0x00007f9669099156 in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007f9669099202 in __assert_fail () from /lib64/libc.so.6
#4 0x000000000124b1d0 in taosAssertDebug (condition=false, file=0x19bf078 "/data/release/3220/community/source/libs/sync/src/syncRaftLog.c", line=107, format=0x19bf05c "assert info not provided") at /data/release/3220/community/source/util/src/tlog.c:845
#5 0x0000000000eb7680 in raftLogRestoreFromSnapshot (pLogStore=0x7f964817ab50, snapshotIndex=-1) at /data/release/3220/community/source/libs/sync/src/syncRaftLog.c:107
#6 0x0000000000ea26e2 in syncNodeLogStoreRestoreOnNeed (pNode=0x7f96481ab240) at /data/release/3220/community/source/libs/sync/src/syncMain.c:794
#7 0x0000000000ea390c in syncNodeOpen (pSyncInfo=0x7f9658ab6040, vnodeVersion=5) at /data/release/3220/community/source/libs/sync/src/syncMain.c:1030
#8 0x0000000000e9f520 in syncOpen (pSyncInfo=0x7f9658ab6040, vnodeVersion=5) at /data/release/3220/community/source/libs/sync/src/syncMain.c:63
#9 0x00000000007c10a6 in vnodeSyncOpen (pVnode=0x7f96480008c0, path=0x7f9658ab6b60 "/data/lib/taos/vnode/vnode18", vnodeVersion=5) at /data/release/3220/community/source/dnode/vnode/src/vnd/vnodeSync.c:698
#10 0x00000000007ab744 in vnodeOpen (path=0x7f9658ab77e0 "vnode/vnode18", diskPrimary=0, pTfs=0x5e06930, msgCb=..., force=false) at /data/release/3220/community/source/dnode/vnode/src/vnd/vnodeOpen.c:474
#11 0x00000000007986e7 in vmOpenVnodeInThread (param=0x5e738c0) at /data/release/3220/community/source/dnode/mgmt/mgmt_vnode/src/vmInt.c:310
#12 0x00007f966a06ae65 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f966916888d in clone () from /lib64/libc.so.6
(gdb)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the failing transaction details and the restore path in libs/sync/src/syncRaftLog.c:107, then trace the callers in syncMain.c, vnodeSync.c, and vnodeOpen.c shown in the backtrace. Reproduce or inspect the alter-replica failure and snapshot restore behavior. Done means the transaction leaves redoAction and the affected node can rejoin and start without the assertion crash.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, sql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100