spring-cloud / spring-cloud/spring-cloud-gateway

Introduce Logging when reactiveScriptExecutor.eval returns flux.error

Open
#886 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement help wanted
Dominant language
Java
Stars
4.9k
Forks
3.5k
Avg merge
20h 57m
Merged PRs (30d)
8

Description

In order to determine that the target Redis services does not have lua scripts enabled, the flux.error should returned by the reactiveScriptExecutor.eval [1], should be shown to the developer maybe through logging when flux.onErrorResume is invoked in the RedisRateLimiter [2].

Spring Cloud version: Greenwich.RELEASE
Spring Boot version: 2.1.3.RELEASE

We found this in an Pivotal Application Services environment where the Redis data services 2.0.1 is deployed. Through debugging it would found that the plan being used had not enabled Lua Scripting.

image

The solution was to enable Lua Scripting for the plan and apply the change to all redis service instances using that plan.

This was difficult to troubleshoot because the only way we found to determine why the rate limiting rules were not being enforced was to set a debug break point in thereactiveScriptExecutor [1] which caught the exception with the following message:

org.springframework.data.redis.RedisSystemException: Error in execution; nested exception is io.lettuce.core.RedisCommandExecutionException: ERR unknown command `EVALSHA`, with args beginning with: `eec77786d43c65f7fd568900b5d2b63b692318dc`, `2`, `request_rate_limiter.{user}.tokens`, `request_rate_limiter.{user}.timestamp`, 

What would really help us out is if their was logging when the exception was encounter or if when spring cloud gateway starts up we could have some sort of warning that the scripting capability is not enabled therefore rate limiting will not be enforced? We are open to any other ideas or suggestions on how to troubleshoot this to remember for next time since this was our first pass with the spring cloud gateway.

[1] https://github.com/spring-projects/spring-data-redis/blob/2.1.x/src/main/java/org/springframework/data/redis/core/script/DefaultReactiveScriptExecutor.java#L121-L122
[2] https://github.com/spring-cloud/spring-cloud-gateway/blob/2.1.x/spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/filter/ratelimit/RedisRateLimiter.java#L228

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at spring-cloud-gateway-core/src/main/java/org/springframework/cloud/gateway/filter/ratelimit/RedisRateLimiter.java around line 228, where flux.onErrorResume handles the reactiveScriptExecutor.eval failure. Use the reported EVALSHA error as the diagnostic case and make the failure visible to developers through logging or an equivalent startup warning; done means the missing Lua-scripting capability is no longer silent.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, redis, spring, spring-boot
Domain
api, backend, observability
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.