关于0.13版本的TsFile 同步方式bug
- Lenguaje dominante
- Java
- Estrellas
- 6.4k
- Forks
- 1.2k
- Merge medio
- 1 d 23 h
- PR fusionados (30 d)
- 115
Descripción
0.13版本中,通过 tools\start-sync-client 的方式进行TsFile同步,这种方式未来是否还会继续提供??1.0中的Pipe、PipeSink方式是未来的支持方式吗??
0.13版本中,TsFile 同步发现bug,操作方式如下:
1、根据文档描述,配置了 发送端和接收端;
2、执行tools\start-sync-client ,发送端将数据(root.XXX存储组)同步到接收端;
3、接收端,通过delete STORAGE group root.XXX 方式,删除存储组;
4、发送端,再次执行 tools\start-sync-client,无法再次同步 数据到接收端。
发送端日志:
2022-12-27 10:47:43,118 [main] INFO o.a.i.d.s.c.SyncSenderDescriptor:82 - Start to read sync config file D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-sync-client.properties
2022-12-27 10:47:43,154 [main] INFO o.a.i.d.c.IoTDBDescriptor:128 - Start to read config file file:D:/apache-iotdb-0.13.0-all-bin/conf/iotdb-engine.properties
2022-12-27 10:47:43,165 [main] INFO o.a.i.d.c.IoTDBDescriptor:1296 - allocateMemoryForRead = 1125226905
2022-12-27 10:47:43,165 [main] INFO o.a.i.d.c.IoTDBDescriptor:1297 - allocateMemoryForWrite = 1500302540
2022-12-27 10:47:43,166 [main] INFO o.a.i.d.c.IoTDBDescriptor:1298 - allocateMemoryForSchema = 375075635
2022-12-27 10:47:43,168 [main] INFO o.a.i.t.c.c.TSFileDescriptor:123 - try loading iotdb-engine.properties from D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-engine.properties
2022-12-27 10:47:43,173 [main] INFO o.a.i.d.c.IoTDBDescriptor:260 - IoTDB enable memory control: true
2022-12-27 10:47:43,201 [Sync-Client] INFO o.a.i.d.c.IoTDBThreadPoolFactory:174 - new single scheduled thread pool: timedForceMLogThread
2022-12-27 10:47:43,223 [Sync-Client] INFO o.a.i.d.w.i.SingleFileLogReader:144 - open WAL file: mlog.bin size is 10086971
2022-12-27 10:47:43,850 [Sync-Client] INFO o.a.i.m.c.MetricConfigDescriptor:75 - Start to read config file D:\apache-iotdb-0.13.0-all-bin\conf\iotdb-metric.yml
2022-12-27 10:47:43,873 [Sync-Client] INFO o.a.i.d.c.IoTDBThreadPoolFactory:195 - new scheduled thread pool: sync-client-timer
2022-12-27 10:47:43,955 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:481 - Receiver has received schema successfully.
2022-12-27 10:47:43,956 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:265 - There are 1 data dirs to be synced.
2022-12-27 10:47:43,956 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:268 - Start to sync data in data dir D:\apache-iotdb-0.13.0-all-bin\data\data, the process is 1/1
2022-12-27 10:47:43,957 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.r.SyncSenderLogAnalyzer:89 - log file D:\apache-iotdb-0.13.0-all-bin\data\data\sync-sender\192.168.235.134_9555\sync.log doesn't exist.
2022-12-27 10:47:43,963 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.m.SyncFileManager:94 - Start to get current local files in data folder D:\apache-iotdb-0.13.0-all-bin\data\data
2022-12-27 10:47:43,965 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.m.SyncFileManager:175 - Start to get last local files from last local file info D:\apache-iotdb-0.13.0-all-bin\data\data\sync-sender\192.168.235.134_9555\last_local_files.txt
2022-12-27 10:47:43,976 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:543 - Sync process starts to transfer data of storage group root.rczz, it has 1 virtual storage groups.
2022-12-27 10:47:43,976 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:597 - There has no deleted files to be synced in storage group root.rczz
2022-12-27 10:47:43,977 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:626 - There has no new tsfiles to be synced in storage group root.rczz
2022-12-27 10:47:43,978 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:580 - Sync process finished the task to sync data of storage group root.rczz.
2022-12-27 10:47:43,980 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:287 - Finish to sync data in data dir D:\apache-iotdb-0.13.0-all-bin\data\data, the process is 1/1
2022-12-27 10:47:43,988 [pool-2-IoTDB-sync-client-timer-1] INFO o.a.i.d.s.s.t.SyncClient:299 - Sync process has finished.
接收端日志:
2022-12-27 10:47:43,886 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:98 - Start to sync with sender 192.168.235.1_8506306abea44db18da7f89dabf66265
2022-12-27 10:47:43,944 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:292 - Start to load metadata in sync process.
2022-12-27 10:47:43,944 [Sync] INFO o.a.i.d.w.i.SingleFileLogReader:144 - open WAL file: mlog.bin size is 0
2022-12-27 10:47:43,965 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:155 - Sync process started to receive data of storage group root.rczz
2022-12-27 10:47:43,977 [Sync] INFO o.a.i.d.s.r.t.SyncServiceImpl:336 - Sync process with sender 192.168.235.1_8506306abea44db18da7f89dabf66265 finished.
2022-12-27 10:47:44,003 [pool-25-IoTDB-Load-TsFile-1] INFO o.a.i.d.s.r.l.FileLoader:177 - Sync loading process for 192.168.235.1_8506306abea44db18da7f89dabf66265 has finished.
2022-12-27 10:48:21,873 [pool-3-IoTDB-timedQuerySqlCount-1] INFO o.a.i.d.s.b.QueryFrequencyRecorder:42 - Query count in current 1 minute 2
Guía de contribución
Línea de trabajo
Empieza reproduciendo el flujo de trabajo 0.13 indicado con tools\start-sync-client: sync root.XXX, elimínalo en el receptor y vuelve a ejecutar la sincronización. Lee los registros del emisor y del receptor junto con iotdb-sync-client.properties y iotdb-engine.properties para rastrear por qué no se seleccionan archivos. Se considera terminado cuando se confirme el comportamiento esperado de la sincronización repetida o se aísle el fallo con una prueba de regresión o una reproducción clara.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- java
- Área
- databases, distributed-systems
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 42/100