apache / apache/iotdb

关于0.13版本的TsFile 同步方式bug

Aperta
#8,618 6 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Java
Stelle
6.4k
Fork
1.2k
Merge medio
1g 23h
PR unite (30g)
115

Descrizione

0.13版本中,通过 tools\start-sync-client 的方式进行TsFile同步,这种方式未来是否还会继续提供??1.0中的Pipe、PipeSink方式是未来的支持方式吗??

0.13版本中,TsFile 同步发现bug,操作方式如下:

1、根据文档描述,配置了 发送端和接收端;
2、执行tools\start-sync-client ,发送端将数据(root.XXX存储组)同步到接收端;
3、接收端,通过delete STORAGE group root.XXX 方式,删除存储组;
4、发送端,再次执行 tools\start-sync-client,无法再次同步 数据到接收端。

发送端日志:

2022-12-27 10:47:43,118 [main] INFO o.a.i.d.s.c.SyncSenderDescriptor:82 - Start to read sync config file D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-sync-client.properties
2022-12-27 10:47:43,154 [main] INFO o.a.i.d.c.IoTDBDescriptor:128 - Start to read config file file:D:/apache-iotdb-0.13.0-all-bin/conf/iotdb-engine.properties
2022-12-27 10:47:43,165 [main] INFO o.a.i.d.c.IoTDBDescriptor:1296 - allocateMemoryForRead = 1125226905
2022-12-27 10:47:43,165 [main] INFO o.a.i.d.c.IoTDBDescriptor:1297 - allocateMemoryForWrite = 1500302540
2022-12-27 10:47:43,166 [main] INFO o.a.i.d.c.IoTDBDescriptor:1298 - allocateMemoryForSchema = 375075635
2022-12-27 10:47:43,168 [main] INFO o.a.i.t.c.c.TSFileDescriptor:123 - try loading iotdb-engine.properties from D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-engine.properties
2022-12-27 10:47:43,173 [main] INFO o.a.i.d.c.IoTDBDescriptor:260 - IoTDB enable memory control: true
2022-12-27 10:47:43,201 [Sync-Client] INFO o.a.i.d.c.IoTDBThreadPoolFactory:174 - new single scheduled thread pool: timedForceMLogThread
2022-12-27 10:47:43,223 [Sync-Client] INFO o.a.i.d.w.i.SingleFileLogReader:144 - open WAL file: mlog.bin size is 10086971
2022-12-27 10:47:43,850 [Sync-Client] INFO o.a.i.m.c.MetricConfigDescriptor:75 - Start to read config file D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-metric.yml
2022-12-27 10:47:43,873 [Sync-Client] INFO o.a.i.d.c.IoTDBThreadPoolFactory:195 - new scheduled thread pool: sync-client-timer
2022-12-27 10:47:43,955 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:481 - Receiver has received schema successfully.
2022-12-27 10:47:43,956 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:265 - There are 1 data dirs to be synced.
2022-12-27 10:47:43,956 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:268 - Start to sync data in data dir D:\apache-iotdb-0.13.0-all-bin\data\data, the process is 1/1
2022-12-27 10:47:43,957 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.r.SyncSenderLogAnalyzer:89 - log file D:\apache-iotdb-0.13.0-all-bin\data\data\sync-sender\192.168.235.134_9555\sync.log doesn't exist.
2022-12-27 10:47:43,963 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.m.SyncFileManager:94 - Start to get current local files in data folder D:\apache-iotdb-0.13.0-all-bin\data\data
2022-12-27 10:47:43,965 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.m.SyncFileManager:175 - Start to get last local files from last local file info D:\apache-iotdb-0.13.0-all-bin\data\data\sync-sender\192.168.235.134_9555\last_local_files.txt
2022-12-27 10:47:43,976 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:543 - Sync process starts to transfer data of storage group root.rczz, it has 1 virtual storage groups.
2022-12-27 10:47:43,976 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:597 - There has no deleted files to be synced in storage group root.rczz
2022-12-27 10:47:43,977 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:626 - There has no new tsfiles to be synced in storage group root.rczz
2022-12-27 10:47:43,978 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:580 - Sync process finished the task to sync data of storage group root.rczz.
2022-12-27 10:47:43,980 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:287 - Finish to sync data in data dir D:\apache-iotdb-0.13.0-all-bin\data\data, the process is 1/1
2022-12-27 10:47:43,988 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:299 - Sync process has finished.

接收端日志:
2022-12-27 10:47:43,886 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:98 - Start to sync with sender 192.168.235.1_8506306abea44db18da7f89dabf66265
2022-12-27 10:47:43,944 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:292 - Start to load metadata in sync process.
2022-12-27 10:47:43,944 [Sync] INFO o.a.i.d.w.i.SingleFileLogReader:144 - open WAL file: mlog.bin size is 0
2022-12-27 10:47:43,965 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:155 - Sync process started to receive data of storage group root.rczz
2022-12-27 10:47:43,977 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:336 - Sync process with sender 192.168.235.1_8506306abea44db18da7f89dabf66265 finished.
2022-12-27 10:47:44,003 [pool-25-IoTDB-Load-TsFile-1] INFO o.a.i.d.s.r.l.FileLoader:177 - Sync loading process for 192.168.235.1_8506306abea44db18da7f89dabf66265 has finished.
2022-12-27 10:48:21,873 [pool-3-IoTDB-timedQuerySqlCount-1] INFO o.a.i.d.s.b.QueryFrequencyRecorder:42 - Query count in current 1 minute 2

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia riproducendo il workflow 0.13 indicato con tools\start-sync-client: sync root.XXX, eliminalo sul ricevitore ed esegui nuovamente la sincronizzazione. Leggi i log del mittente e del ricevitore insieme a iotdb-sync-client.properties e iotdb-engine.properties per tracciare il motivo per cui non viene selezionato alcun file. Il lavoro è completo quando viene confermato il comportamento previsto della sincronizzazione ripetuta oppure il problema viene isolato con un test di regressione o una riproduzione chiara.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java
Ambito
databases, distributed-systems
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
42/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.