apache / apache/servicecomb-java-chassis

在SafeMode下如果pullInstances成功后无法在退出SafeMode后老化旧的实例

Open
#2,820 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.9k
Forks
814
Avg merge
8d 23h
Merged PRs (30d)
1

Description

**Describe the bug**
背景:servicecomb-java-chassis 2.1.5在使用过程中,我们的rpc调用服务端会校验客户端在请求头中携带的目标实例ID,发现某些rpc请求的实例ID对应已经老化的实例ID。
现象:服务缓存中存在已经老化的旧实例。
原因分析:RefreshableMicroserviceCache缓存会在pullInstance后更新记录的revisionId,如果在safemode期间pullInstance成功了会导致此处rev被更新的和sc中的一致,在退出安全模式后携带此rev再次pullInstance时如果此时实例在sc中未有变更会直接响应304,不会触发缓存刷新,导致在安全模式中不老化的实例无法正常下线。
解决建议:在安全模式下不更新缓存中记录的revisionId。
**To Reproduce**
复现方法:参考以下测试用例。
_RefreshableMicroserviceCacheTest.java_
```
@Test
public void refresh_exit_safe_mode_down_old_instance() {
ArrayList instances = new ArrayList<>();
findServiceInstancesOprHolder.value = params -> {
Assert.assertEquals("consumerId", params[0]);
Assert.assertEquals("app", params[1]);
Assert.assertEquals("svc", params[2]);
Assert.assertEquals("0.0.0.0+", params[3]);
MicroserviceInstances microserviceInstances = new MicroserviceInstances();
microserviceInstances.setNeedRefresh(!String.valueOf(instances.hashCode()).equals(params[4]));
microserviceInstances.setRevision(String.valueOf(instances.hashCode()));
microserviceInstances.setMicroserviceNotExist(false);

FindInstancesResponse instancesResponse = new FindInstancesResponse();
instancesResponse.setInstances(instances);

microserviceInstances.setInstancesResponse(instancesResponse);
return microserviceInstances;
};

// at the beginning, no instances in cache
List cachedInstances = microserviceCache.getInstances();
Assert.assertEquals(0, cachedInstances.size());
Assert.assertNull(microserviceCache.getRevisionId());

// find 1 instance from sc
MicroserviceInstance microserviceInstance = new MicroserviceInstance();
instances.add(microserviceInstance);
microserviceInstance.setInstanceId("instanceId00");

microserviceCache.refresh();
Assert.assertEquals(MicroserviceCacheStatus.REFRESHED, microserviceCache.getStatus());

cachedInstances = microserviceCache.getInstances();
Assert.assertEquals(1, cachedInstances.size());
MicroserviceInstance instance = cachedInstances.iterator().next();
Assert.assertEquals("instanceId00", instance.getInstanceId());
Assert.assertEquals("-738005329", microserviceCache.getRevisionId());

// 2nd time, clear pull instances
MicroserviceInstance microserviceInstance1 = new MicroserviceInstance();
instances.clear();
instances.add(microserviceInstance1);
microserviceInstance1.setInstanceId("instanceId01");

microserviceCache.refresh();
Assert.assertEquals(MicroserviceCacheStatus.REFRESHED, microserviceCache.getStatus());
cachedInstances = microserviceCache.getInstances();
Assert.assertEquals(1, cachedInstances.size());
Assert.assertEquals("instanceId01", cachedInstances.get(0).getInstanceId());

// 3nd time, 进入安全模式,第三次拉取不下线实例
microserviceCache.onSafeModeChanged(new SafeModeChangeEvent(true));

MicroserviceInstance microserviceInstance2 = new MicroserviceInstance();
instances.clear();
instances.add(microserviceInstance2);
microserviceInstance2.setInstanceId("instanceId02");

microserviceCache.refresh();
Assert.assertEquals(MicroserviceCacheStatus.REFRESHED, microserviceCache.getStatus());

cachedInstances = microserviceCache.getInstances();
Assert.assertEquals(2, cachedInstances.size());
Assert.assertEquals("instanceId02", cachedInstances.get(0).getInstanceId());
Assert.assertEquals("instanceId01", cachedInstances.get(1).getInstanceId());

// 4nd time, 退出安全模式,第四次拉取需要下线实例
microserviceCache.onSafeModeChanged(new SafeModeChangeEvent(false));

microserviceCache.refresh();
Assert.assertEquals(MicroserviceCacheStatus.REFRESHED, microserviceCache.getStatus());

cachedInstances = microserviceCache.getInstances();
Assert.assertEquals(1, cachedInstances.size());
Assert.assertEquals("instanceId02", cachedInstances.get(0).getInstanceId());
}
```

**Expected behavior**
在退出安全模式后可以及时下线老实例。

**Platform And Runtime (please complete the following information):**
环境无差别,稳定复现

**Additional context**

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.