algolia / algolia/algoliasearch-client-java

Adding a configuration for max retries to handle system wide outages

未关闭
#789 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Java
星标
52
派生
33
PR 合并指标
30 天内没有已合并 PR

描述

- Algolia Client Version: 3.16.6
- Language Version: Java

### Description
Correct me if I am mistaken but [HttpTransport.executeWithRetry](https://github.com/algolia/algoliasearch-client-java/blob/master/algoliasearch-core/src/main/java/com/algolia/search/HttpTransport.java#L150-L192) will before sending a request, increase the timeout for the request based on the number of retries. After the response is received, it uses *RetryStrategy* to determine what to do with the results.

The function [RetryStrategy.decide](https://github.com/algolia/algoliasearch-client-java/blob/master/algoliasearch-core/src/main/java/com/algolia/search/RetryStrategy.java#L52-L72C4) will in the case of a request time out, will *always* retry.

During an Indexer service outage in July 13th, it appears that was happening for the entire length of the outage causing the Promises to hang and never fail. This becomes a pretty big issue when using `BatchIndexingResponse.waitTask()` like in the API examples.

### Proposed change
1. Adding a new configurable property to ConfigBase called something along the lines of `maxRetriesPerHost` and update `RetryStrategy` to save that value in its constructor.
2. Update `RetryStrategy.decide` to something along the lines of
```
} else if (response.isTimedOut()) {
boolean shouldKeepUp = maxRetriesPerHost == null || tryableHost.getRetryCount() <= maxRetriesPerHost;
tryableHost.setUp(shouldKeepUp);
tryableHost.setLastUse(AlgoliaUtils.nowUTC());
tryableHost.incrementRetryCount();
return RetryOutcome.RETRY;
}
```

### Why make the change
During the time of the indexing outage, it was not clear immediately why our requests were stuck. Looking at the code it seems like if the servers actually responded with server errors, the `StatefulHost`s would have been turned off one by one which would be closer to what I would expect from an outage. The current logic has no way of determining this type of issue, causing ever expanding timeouts without any way of intervening.

贡献指南

这个仓库没有索引到贡献指南

调研方向

从 algoliasearch-core/src/main/java/com/algolia/search/HttpTransport.java 开始,尤其关注 executeWithRetry,然后阅读 RetryStrategy.java 和 ConfigBase,以追踪 timeout 和重试次数的处理。确认请求超时时的行为,并将完成标准定义为:提供可按主机配置的重试上限,阻止重试次数无限增长,同时在未设置该上限时保留现有的重试行为。

由索引模型根据 Issue 内容生成。

评估

技术栈
java
领域
api
Issue 类型
功能
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
描述清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。