alibaba / alibaba/DataX

OpenTSDBReader任务配置多metric仅读取到第一个metric的原因及解决办法

Open
#455 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

故障现象:OpenTSDBReader任务配置文件多个metric仅有第一个metric信息可以被读取到
故障原因:OpenTSDBReader类的split方法,在进行task切分的时候首先按照metric进行切分(for循环),在同一个metric下按照小时进行切分(while循环)的时候,会将配置文件中读取到的beginDateTime进行递增赋值,但是在重新进入for循环时,未将beginDateTime进行复原,导致while循环条件一直为false
解决办法:将DateTime startDateTime = new DateTime(TimeUtils.getTimeInHour(startTime));该行代码放入columns的for循环内的第一行,如下所示:
`//DateTime startDateTime = new DateTime(TimeUtils.getTimeInHour(startTime));//直接注掉
DateTime endDateTime = new DateTime(TimeUtils.getTimeInHour(endTime));

// split by metric
for (String column : columns) {
DateTime startDateTime = new DateTime(TimeUtils.getTimeInHour(startTime));//修改位置
// split by time in hour
while (startDateTime.isBefore(endDateTime)) {
Configuration clone = this.originalConfig.clone();
clone.set(Key.COLUMN, Collections.singletonList(column));

clone.set(Key.BEGIN_DATE_TIME, startDateTime.getMillis());
startDateTime = startDateTime.plusHours(1);
// Make sure the time interval is [start, end).
// Because net.opentsdb.core.Query.setEndTime means less than or equal to the end time.
clone.set(Key.END_DATE_TIME, startDateTime.getMillis() - 1);
configurations.add(clone);

LOG.info("Configuration: {}", JSON.toJSONString(clone));
}
}`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in the OpenTSDBReader class's split method and inspect how startTime is reused across the metric loop and hourly loop. Verify a configuration with multiple metrics produces hourly configurations for every metric, with each metric starting from the original time range.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.