ClusterLabs / ClusterLabs/resource-agents

redis RA waits until timeout even when we know the redis-server failed to start

Open
#1,484 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
519
Forks
608
Avg merge
6d 1h
Merged PRs (30d)
7

Description

```bash
output="$(su "$REDIS_USER" -s /bin/sh -c "cd '$REDIS_RUNDIR'; exec '$REDIS_SERVER' '$REDIS_CONFIG' --daemonize yes --unixsocket '$REDIS_SOCKET' --pidfile '$REDIS_PIDFILE'" 2>&1)"

while true; do
# wait for redis to start
typeset -A info
while read line; do
[[ "$line" == "#"* ]] && continue
[[ "$line" != *":"* ]] && continue
IFS=':' read -r key value <<< "$line"
info[$key]="$value"
done < <(redis_client info)

if (( info[loading] == 0 )); then
break
...
done

while ! [ -s "$REDIS_PIDFILE" ]; do
ocf_log debug "start: Waiting for pid file '$REDIS_PIDFILE' to appear"
sleep 1
done
```

If there is a problem that causes redis-server unable to start (e.g. redis-server has been run by root before which makes the log file can't be opened by user redis), instead of returning an error immediately this resource agent wait until timeout.

The resource agent execution steps are:
1. The redis-server fails to start. The unix socket file is not created.
2. redis-cli can't connect to the unix socket. ```redis_client info``` outputs nothing.
3. The associative array ```info``` is empty.
4. ```(( info[loading] == 0 ))``` evaluates to true. The first while loop breaks.
5. The resource agent stuck on the second while loop until timeout.

Maybe the RA could check ```output``` to see whether there is ```FATAL CONFIG FILE ERROR``` and returns error immediately if true so that we don't need to wait a long time until timeout. Even better if it feeds ```output``` to ```ocf_exit_reason``` before the return.

I'm also wondering whether it is intended for the first while loop to break immediately when the redis-server fails to start and redis.sock does not exist.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Redis resource agent's start path containing the shown shell loops, and trace how redis-server output and redis_client failures are handled. Reproduce a startup failure such as the described log-file permission error; done means reporting the failure promptly, including the captured output where appropriate, while preserving successful startup behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
redis, shell
Domain
databases, devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.