ansible/openwhisk.yml fails for waiting kafka server started up
- Dominant language
- Scala
- Stars
- 6.8k
- Forks
- 1.2k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 2
Description
## Environment details:
* local deployment
* Ubuntu 16.04
* Docker version 19.03.13, build 4484c46d9d
## Steps to reproduce the issue:
1. cd tools/ubuntu-setup && ./all.sh
2. ansible-playbook setup.yml ; ansible-playbook prereq.yml (with envrionment variable setup for couchDB)
3. ./gradlew distDocker
4. ansible-playbook initdb.yml ; ansible-playbook wipe.yml
5. **ansible-playbook openwhisk.yml**
## Provide the actual results and outputs:
```
TASK [kafka : wait until the kafka server started up] ***********************************************************************************************************
Tuesday 01 December 2020 14:03:16 -0600 (0:00:27.886) 0:00:49.298 ******
FAILED - RETRYING: wait until the kafka server started up (10 retries left).
FAILED - RETRYING: wait until the kafka server started up (9 retries left).
FAILED - RETRYING: wait until the kafka server started up (8 retries left).
FAILED - RETRYING: wait until the kafka server started up (7 retries left).
FAILED - RETRYING: wait until the kafka server started up (6 retries left).
FAILED - RETRYING: wait until the kafka server started up (5 retries left).
FAILED - RETRYING: wait until the kafka server started up (4 retries left).
FAILED - RETRYING: wait until the kafka server started up (3 retries left).
FAILED - RETRYING: wait until the kafka server started up (2 retries left).
FAILED - RETRYING: wait until the kafka server started up (1 retries left).
fatal: [kafka0]: FAILED! => {"attempts": 10, "changed": true, "cmd": "(echo dump; sleep 1) | nc 172.17.0.1 2181 | grep /brokers/ids/0", "delta": "0:00:01.005511", "end": "2020-12-01 14:04:20.335370", "msg": "non-zero return code", "rc": 1, "start": "2020-12-01 14:04:19.329859", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
[FAILED]
> (echo dump; sleep 1) | nc 172.17.0.1 2181 | grep /brokers/ids/0
non-zero return code
PLAY RECAP ******************************************************************************************************************************************************
kafka0 : ok=9 changed=3 unreachable=0 failed=1
```
## Additional information you deem important:
* docker ps (not sure why kafka keeps restarting??)
```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3980b79c4ad0 wurstmeister/kafka:2.12-2.3.1 "start-kafka.sh" 9 minutes ago Restarting (1) Less than a second ago kafka0
471187e2ba20 zookeeper:3.4 "/docker-entrypoint.…" 10 minutes ago Up 10 minutes 0.0.0.0:2181->2181/tcp, 0.0.0.0:2888->2888/tcp, 0.0.0.0:3888->3888/tcp zookeeper0
```
* Tried this on a fresh Ubuntu 18.04 with same setup steps, no problem found.
Contributor guide
Research direction
Start in tools/ubuntu-setup and the Kafka role used by ansible-playbook openwhisk.yml, focusing on the task that runs nc against 172.17.0.1:2181 and checks /brokers/ids/0. Reproduce the Ubuntu 16.04 setup and inspect docker ps while Kafka restarts. Done means the playbook waits successfully and the kafka0 container remains running.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ansible, docker, kafka, ubuntu
- Domain
- devops, distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100