关于0.13版本的TsFile 同步方式bug
- Vorherrschende Sprache
- Java
- Sterne
- 6.4k
- Forks
- 1.2k
- Ø Merge
- 1 T. 23 Std.
- Gemergte PRs (30 T.)
- 115
Beschreibung
0.13版本中,通过 tools\start-sync-client 的方式进行TsFile同步,这种方式未来是否还会继续提供??1.0中的Pipe、PipeSink方式是未来的支持方式吗??
0.13版本中,TsFile 同步发现bug,操作方式如下:
1、根据文档描述,配置了 发送端和接收端;
2、执行tools\start-sync-client ,发送端将数据(root.XXX存储组)同步到接收端;
3、接收端,通过delete STORAGE group root.XXX 方式,删除存储组;
4、发送端,再次执行 tools\start-sync-client,无法再次同步 数据到接收端。
发送端日志:
2022-12-27 10:47:43,118 [main] INFO o.a.i.d.s.c.SyncSenderDescriptor:82 - Start to read sync config file D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-sync-client.properties
2022-12-27 10:47:43,154 [main] INFO o.a.i.d.c.IoTDBDescriptor:128 - Start to read config file file:D:/apache-iotdb-0.13.0-all-bin/conf/iotdb-engine.properties
2022-12-27 10:47:43,165 [main] INFO o.a.i.d.c.IoTDBDescriptor:1296 - allocateMemoryForRead = 1125226905
2022-12-27 10:47:43,165 [main] INFO o.a.i.d.c.IoTDBDescriptor:1297 - allocateMemoryForWrite = 1500302540
2022-12-27 10:47:43,166 [main] INFO o.a.i.d.c.IoTDBDescriptor:1298 - allocateMemoryForSchema = 375075635
2022-12-27 10:47:43,168 [main] INFO o.a.i.t.c.c.TSFileDescriptor:123 - try loading iotdb-engine.properties from D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-engine.properties
2022-12-27 10:47:43,173 [main] INFO o.a.i.d.c.IoTDBDescriptor:260 - IoTDB enable memory control: true
2022-12-27 10:47:43,201 [Sync-Client] INFO o.a.i.d.c.IoTDBThreadPoolFactory:174 - new single scheduled thread pool: timedForceMLogThread
2022-12-27 10:47:43,223 [Sync-Client] INFO o.a.i.d.w.i.SingleFileLogReader:144 - open WAL file: mlog.bin size is 10086971
2022-12-27 10:47:43,850 [Sync-Client] INFO o.a.i.m.c.MetricConfigDescriptor:75 - Start to read config file D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-metric.yml
2022-12-27 10:47:43,873 [Sync-Client] INFO o.a.i.d.c.IoTDBThreadPoolFactory:195 - new scheduled thread pool: sync-client-timer
2022-12-27 10:47:43,955 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:481 - Receiver has received schema successfully.
2022-12-27 10:47:43,956 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:265 - There are 1 data dirs to be synced.
2022-12-27 10:47:43,956 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:268 - Start to sync data in data dir D:\apache-iotdb-0.13.0-all-bin\data\data, the process is 1/1
2022-12-27 10:47:43,957 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.r.SyncSenderLogAnalyzer:89 - log file D:\apache-iotdb-0.13.0-all-bin\data\data\sync-sender\192.168.235.134_9555\sync.log doesn't exist.
2022-12-27 10:47:43,963 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.m.SyncFileManager:94 - Start to get current local files in data folder D:\apache-iotdb-0.13.0-all-bin\data\data
2022-12-27 10:47:43,965 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.m.SyncFileManager:175 - Start to get last local files from last local file info D:\apache-iotdb-0.13.0-all-bin\data\data\sync-sender\192.168.235.134_9555\last_local_files.txt
2022-12-27 10:47:43,976 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:543 - Sync process starts to transfer data of storage group root.rczz, it has 1 virtual storage groups.
2022-12-27 10:47:43,976 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:597 - There has no deleted files to be synced in storage group root.rczz
2022-12-27 10:47:43,977 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:626 - There has no new tsfiles to be synced in storage group root.rczz
2022-12-27 10:47:43,978 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:580 - Sync process finished the task to sync data of storage group root.rczz.
2022-12-27 10:47:43,980 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:287 - Finish to sync data in data dir D:\apache-iotdb-0.13.0-all-bin\data\data, the process is 1/1
2022-12-27 10:47:43,988 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:299 - Sync process has finished.
接收端日志:
2022-12-27 10:47:43,886 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:98 - Start to sync with sender 192.168.235.1_8506306abea44db18da7f89dabf66265
2022-12-27 10:47:43,944 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:292 - Start to load metadata in sync process.
2022-12-27 10:47:43,944 [Sync] INFO o.a.i.d.w.i.SingleFileLogReader:144 - open WAL file: mlog.bin size is 0
2022-12-27 10:47:43,965 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:155 - Sync process started to receive data of storage group root.rczz
2022-12-27 10:47:43,977 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:336 - Sync process with sender 192.168.235.1_8506306abea44db18da7f89dabf66265 finished.
2022-12-27 10:47:44,003 [pool-25-IoTDB-Load-TsFile-1] INFO o.a.i.d.s.r.l.FileLoader:177 - Sync loading process for 192.168.235.1_8506306abea44db18da7f89dabf66265 has finished.
2022-12-27 10:48:21,873 [pool-3-IoTDB-timedQuerySqlCount-1] INFO o.a.i.d.s.b.QueryFrequencyRecorder:42 - Query count in current 1 minute 2
Beitragsleitfaden
Rechercherichtung
Beginne damit, den aufgeführten 0.13-Workflow mit tools\start-sync-client zu reproduzieren: sync root.XXX, lösche es auf dem Empfänger und führe die Synchronisierung erneut aus. Lies die Sender- und Empfängerprotokolle zusammen mit iotdb-sync-client.properties und iotdb-engine.properties, um nachzuverfolgen, warum keine Dateien ausgewählt werden. Als abgeschlossen gilt die Bestätigung des erwarteten Verhaltens bei der erneuten Synchronisierung oder die Eingrenzung des Fehlers mit einem Regressionstest oder einer eindeutigen Reproduktion.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- java
- Bereich
- databases, distributed-systems
- Issue-Typ
- Bug
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 42/100