micronaut-projects / micronaut-projects/micronaut-cache
What's the best way to proceed without caching if an exception is hit in a Cache Annotation
- Dominant language
- PLSQL
- Stars
- 32
- Forks
- 36
- Avg merge
- 2h 2m
- Merged PRs (30d)
- 5
Description
### Cache Interceptor Usage for proceeding when exception occurs
Unanswered in Discord https://discord.com/channels/1121511613250412714/1174821270785900666/1174821270785900666
We hit a Production issue during an AWS elasticache cluster maintenance window where micronaut redis annotated operations would start crashing the requests when we really just wanted it to continue on and hit the database again. The only way we found to support our needs was to subclass the CacheInterceptor and toss a try/catch saying context.proceed(). This seems not ideal since it's global? but maybe there's some way to limit it's use?
Now ideally, maintainence window finishes at a time we are ok with (this is true today) but for some reason some app
instances never fully recovered and don't know why since we cannot recreate the issue of not reconnecting after the maintainence is done.
We CAN re-create the issue during maintenance of the cluster tho consistently and the app instance just constantly fails requests due to the cache exception.
We can leave the Redis part out of this discussion, but included for background.
I'm not expert on Redis clusters and redis-lettuce but it did seem like if we only have 1 instance of our app and we do a failover the app is completely broken until resolved (failover complete). With multiple instances we see some subset stuck constantly throwing these errors and understand that cluster mode is maybe queueing those things up until the topology is correct?
## Workaround
```kotlin
class CustomCacheInterceptor(
cacheManager: CacheManager, // Adjust the type parameter as needed
errorHandler: CacheErrorHandler,
asyncCacheErrorHandler: AsyncCacheErrorHandler,
@Named(TaskExecutors.IO) ioExecutor: ExecutorService,
beanContext: BeanContext
) : CacheInterceptor(cacheManager, errorHandler, asyncCacheErrorHandler, ioExecutor, beanContext) {
/**
* Allows Redis errors to continue
*/
override fun intercept(context: MethodInvocationContext): Any? {
return try {
super.intercept(context)
} catch (e: Exception) {
// Maybe add a metric
// Handle the exception, e.g., log it and proceed without caching
context.proceed()
}
}
}
```
## Example Errors
Errors we hit in production
```
i.m.m.h.indicator.HealthResult Health indicator [redis(Primary)] reported exception: io.lettuce.core.RedisConnectionException: Unable to connect
```
```
io.lettuce.core.RedisException: java.io.IOException: Connection reset by peer
at i.l.core.internal.Exceptions.bubble(Exceptions.java:83)
at io.lettuce.core.internal.Futures.awaitOrCancel(Futures.java:250)
```
```
i.l.c.RedisConnectionException: Unable to connect
at i.l.c.RedisConnectionException.create(RedisConnectionException.java:94)
at i.l.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:372)
at i.l.c.cluster.RedisClusterClient.connect(RedisClusterClient.java:403)
at i.l.c.cluster.RedisClusterClient.connect(RedisClusterClient.java:378)
```
## Existing Discussion
Only PR I could find related to someone wanted to subclass cacheInterceptor
https://github.com/micronaut-projects/micronaut-cache/pull/279
Contributor guide
Research direction
Start with CacheInterceptor and the related pull request #279, then reproduce the failure during Redis or ElastiCache maintenance using the reported workaround as context. Clarify the desired scoped behavior for cache exceptions and define how success should be verified without relying on a global subclass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kotlin, redis
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100