Cached data accessor may be inconsistent even in the same controller
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
In https://github.com/apache/incubator-pinot/pull/3633, we leveraged `ZkCacheBaseDataAccessor` to cache instance configs in order to reduce the zk access.
While it has its drawback that it's only used to read instance config; writing/updating instance config is done in Helix code and it isn't controlled by Pinot. Thus, it's possible that the cached instance configs may be stale.
Thus, the current workaround is only use cached accessor for segment operations. For tenant/table level operations, we use direct data accessor. It's because segment operations are more frequent than the ones of tenant/table.
In the future, the following actions can be done for this issue:
* Contact Helix team to make `updateCache` method public in cached data accessor.
* Maintain a zkClient in PinotHelixResourceManager and override the methods like addInstance.
* Implement our own guava cache to cache instance configs.
Contributor guide
Assessment
This issue has not been assessed yet.