tsdbwriter不能正确写入
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
2023-08-01 09:59:00.175 [job-0] INFO JobContainer - jobContainer starts to do split ...
2023-08-01 09:59:00.175 [job-0] INFO JobContainer - Job set Channel-Number to 2 channels.
2023-08-01 09:59:00.176 [job-0] INFO JobContainer - DataX Reader.Job [postgresqlreader] splits to [1] tasks.
2023-08-01 09:59:00.177 [job-0] INFO JobContainer - DataX Writer.Job [tsdbwriter] splits to [1] tasks.
2023-08-01 09:59:00.199 [job-0] INFO JobContainer - jobContainer starts to do schedule ...
2023-08-01 09:59:00.201 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2023-08-01 09:59:00.203 [job-0] INFO JobContainer - Running by standalone Mode.
2023-08-01 09:59:00.206 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks.
2023-08-01 09:59:00.210 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2023-08-01 09:59:00.210 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2023-08-01 09:59:00.217 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started
2023-08-01 09:59:00.218 [0-0-0-writer] INFO TSDBConverter - columnName: ["device_id","time","data"], columnType: ["tag","timestamp","field_string"]
2023-08-01 09:59:00.219 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [ select device_id,time,data from test;
] jdbcUrl:[jdbc:postgresql://xxxxx/fall_db].
2023-08-01 09:59:00.238 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [ select device_id,time,data from test;
] jdbcUrl:[jdbc:postgresql://xxxxx/fall_db].
2023-08-01 09:59:00.486 [0-0-0-writer] INFO TSDBWriter$Task - Task finished, write size: 1
2023-08-01 09:59:00.517 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[301]ms
2023-08-01 09:59:00.518 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks.
2023-08-01 09:59:10.215 [job-0] INFO StandAloneJobContainerCommunicator - Total 1 records, 9 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-08-01 09:59:10.215 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2023-08-01 09:59:10.216 [job-0] INFO JobContainer - DataX Writer.Job [tsdbwriter] do post work.
2023-08-01 09:59:10.216 [job-0] INFO JobContainer - DataX Reader.Job [postgresqlreader] do post work.
2023-08-01 09:59:10.216 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
2023-08-01 09:59:10.217 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /root/dataX/datax/hook
2023-08-01 09:59:10.517 [pool-1-batch-put-thread-1] INFO BatchPutRunnable - The thread pool-1-batch-put-thread-1 is interrupted
2023-08-01 09:59:10.517 [pool-2-multi-field-batch-put-thread-1] INFO MultiFieldBatchPutRunnable - The thread pool-2-multi-field-batch-put-thread-1 is inter rupted
2023-08-01 09:59:10.518 [pool-3-points-batch-put-thread-1] INFO PointsCollectionPutRunnable - The thread pool-3-points-batch-put-thread-1 is interrupted. c ause null
2023-08-01 09:59:10.519 [job-0] INFO TSDBClient - The tsdb client has closed.
2023-08-01 09:59:10.520 [job-0] INFO JobContainer -
[total cpu info] =>
averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00%
[total gc info] =>
NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCT ime
PS MarkSweep | 1 | 1 | 1 | 0.028s | 0.028s | 0.028s
PS Scavenge | 1 | 1 | 1 | 0.013s | 0.013s | 0.013s
2023-08-01 09:59:10.520 [job-0] INFO JobContainer - PerfTrace not enable!
2023-08-01 09:59:10.521 [job-0] INFO StandAloneJobContainerCommunicator - Total 1 records, 9 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2023-08-01 09:59:10.522 [job-0] INFO JobContainer -
任务启动时刻 : 2023-08-01 09:58:59
任务结束时刻 : 2023-08-01 09:59:10
任务总计耗时 : 10s
任务平均流量 : 0B/s
记录写入速度 : 0rec/s
读出记录总数 : 1
读写失败总数 : 0
【2023-08-01 09:59:10.517 [pool-1-batch-put-thread-1] INFO BatchPutRunnable - The thread pool-1-batch-put-thread-1 is interrupted
2023-08-01 09:59:10.517 [pool-2-multi-field-batch-put-thread-1] INFO MultiFieldBatchPutRunnable - The thread pool-2-multi-field-batch-put-thread-1 is inter rupted
2023-08-01 09:59:10.518 [pool-3-points-batch-put-thread-1] INFO PointsCollectionPutRunnable - The thread pool-3-points-batch-put-thread-1 is interrupted. c ause null
2023-08-01 09:59:10.519 [job-0] INFO TSDBClient - The tsdb client has closed.】
postgres数据库导入lindorm时序引擎指定数据库和指定的表中,感觉batch线程中断了,不知道是什么原因?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the tsdbwriter entry point named in the logs, then trace TSDBWriter$Task, TSDBClient, and the BatchPutRunnable, MultiFieldBatchPutRunnable, and PointsCollectionPutRunnable threads. Reproduce the PostgreSQL-to-Lindorm write using the reported configuration and determine why the job reports success while the batch threads are interrupted; done means the destination table contains the source record and the interruption has an explained or corrected outcome.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, postgresql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100