MistContoller fails to start other components
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 24
- Forks
- 16
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 1
Description
There is some race condition, which makes the execution of the `livepeer/catalyst` container flaky. In result, sometimes, when catalyst is started, there are no logs at all and the only running process is `MistController`.
### Steps to reproduce
#### Start multiple livepeer catalyst and find the ones with no logs
```
for i in $(seq 0 100); do CID=$(docker run -d --rm livepeer/catalyst); sleep 5; LOGS=$(docker logs $CID); [[ -z "$LOGS" ]] || docker kill $CID;done
```
This command will start 100 container one by one and kill all expect the ones that produce no logs. Then check the running containers and if there are none, then retry the command above. In my case, there was always at least one running container left.
```
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
79e3ec38e0c6 livepeer/catalyst "/usr/bin/MistContro…" 8 minutes ago Up 8 minutes 1935/tcp, 4242/tcp, 8080/tcp, 8889/udp sleepy_poincare
```
#### 2. Exec into the container and check the running processes
You should find that the only running process is `MistController`.
```
$ docker exec -it 79e3ec38e0c6 sh
# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 16080 4340 ? Ssl 13:47 0:00 /usr/bin/MistController -c /etc/livepeer/catalyst.json
root 14 0.1 0.0 2060 496 pts/0 Ss 13:57 0:00 sh
root 20 0.0 0.0 5480 2384 pts/0 R+ 13:57 0:00 ps aux
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure with the provided docker run loop, then inspect the container entry point /usr/bin/MistController and configuration at /etc/livepeer/catalyst.json. Check which component startup step can leave only MistController running. Done means repeated container starts produce logs and all expected components remain running.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100