alibaba / alibaba/DataX

记因一行异常代码导致程序性能下降的case

Open
#537 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

为了实现某些特性的业务需求,我们经常会在插件的配置文件中增加一些自定义的参数,然后在程序运行时通过Configuration的getString(String path)之类的方法获取读或写插件的配置内容。但是,如果配置文件中没有设置相对应的参数名称,就可能造成程序性能大幅下降。

原因在于getString方法的底层findObjectInMap方法在获取配置时,如果获取不到,会抛出一个异常,再由上层的get方法捕获,并返回null。如果程序代码中每处理一条消息,都需要获取配置进行相应业务处理的话,下面代码频繁的抛出和捕获异常会导致程序性能大幅下降。在我的实际测试场景下,调用的越频繁,对性能的影响越大。

> 问题相关类:com.alibaba.datax.common.util.Configuration

```
public String getString(final String path) {
Object string = this.get(path);
if (null == string) {
return null;
}
return String.valueOf(string);
}
```

```
public Object get(final String path) {
this.checkPath(path);
try {
return this.findObject(path);
} catch (Exception e) {
return null;
}
}
```

```
private Object findObject(final String path) {
boolean isRootQuery = StringUtils.isBlank(path);
if (isRootQuery) {
return this.root;
}

Object target = this.root;

for (final String each : split2List(path)) {
if (isPathMap(each)) {
target = findObjectInMap(target, each);
continue;
} else {
target = findObjectInList(target, each);
continue;
}
}

return target;
}
```

```
private Object findObjectInMap(final Object target, final String index) {
boolean isMap = (target instanceof Map);
if (!isMap) {
throw new IllegalArgumentException(String.format(
"您提供的配置文件有误. 路径[%s]需要配置Json格式的Map对象,但该节点发现实际类型是[%s]. 请检查您的配置并作出修改.",
index, target.getClass().toString()));
}
Object result = ((Map) target).get(index);
if (null == result) {
// 在频繁调用场景下,这一步抛出异常会导致程序性能下降
throw new IllegalArgumentException(String.format(
"您提供的配置文件有误. 路径[%s]值为null,datax无法识别该配置. 请检查您的配置并作出修改.", index));
}
return result;
}
```

解决方法:
1、对于所有要使用到的配置参数,都需要在配置文件中设置
2、(建议)去掉抛异常的代码,因为该代码不会对上层方法的语义造成影响,反而会在频繁调用的场景下影响性能

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with com.alibaba.datax.common.util.Configuration, especially getString, get, findObject, and findObjectInMap. Trace the missing-path behavior and verify that frequent lookups of absent configuration values preserve the getString null result without repeatedly throwing and catching exceptions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.