DTStack / DTStack/jlogstash

outputs为es的时候,丢数据问题

Open
#42 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
269
Forks
121
PR merge metrics
No merged PRs in 30d

Description

* 前提是配置consistency参数为true
* es监听中afterBulk(long arg0, BulkRequest arg1, Throwable arg2)方法:
```java
public void afterBulk(long arg0, BulkRequest arg1, Throwable arg2) {
logger.error("bulk got exception:", arg2);

for(ActionRequest request : arg1.requests()){
addFailedMsg(request);
}

}
```
该方法中的`addFailedMsg(request)`传入的参数为`ActionRequest`,而 addFailedMsg(Object msg):
```java
public void addFailedMsg(Object msg) {
if (msg instanceof Map) {
super.addFailedMsg(msg);
return;
}

throw new IllegalArgumentException("addFailedMsg only accept Map instance");
}
```
中实际处理时候做了`msg`是否为 `Map` 的判断,所以从这里进去的数据永远不会被添加到 `failedMsgQueue`,从而导致数据丢失

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.