spring-projects / spring-projects/spring-data-redis

Filter keyspace expiration events in a shared Redis cluster

Open
#2,111 3 comments 0 reactions 1 assignee View on GitHub

@mp911de is already working on this.

Since Jul 19, 2021.

status: waiting-for-triage
Dominant language
Java
Stars
1.9k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Working on a legacy application which uses a single Redis cluster, I found a mixed scenario where Redis repositories are enabled and also the same cluster is used for a RedisCacheManager.

Due to the use of repositories, this routine is executed on MappingExpirationListener upon the expiration of each element in the cluster.
In order to determine if the key should be processed, a validity check is performed. Valid keys must contain at least one delimiter : as we can see here.

We use the same cluster for caching, and some keys contain the delimiter value :, so the listener is trying to execute the routine with them. Unfortunately, due to the nature of the cached values we end up having keys with the format FIXED_PART:VARIABLE_PART, so the listener is misinterpreting the key, and tries to perform a SREM(FIXED_PART, VARIABLE_PART) as we can see here. This leads to a high throughput in one node of the cluster (the one where FIXED_PART is assigned).

I was thinking about adding a prefix to the cached keys and then a filtering layer in the listener to discard them if they start with the prefix, but MappingExpirationListener is instantiated inside RedisKeyValueAdapter and there is no possibility to replace it.

Is there any known way to handle this type of situation?

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.