sameersbn / sameersbn/docker-gitlab

redis ok, postgresql ok, but gitlab aways Readiness probe failed

Open
#2,245 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
8.1k
Forks
2.1k
Avg merge
17h 47m
Merged PRs (30d)
7

Description

kubectl version

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:41:02Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0", GitCommit:"e19964183377d0ec2052d1f1fa930c4d7575bd50", GitTreeState:"clean", BuildDate:"2020-08-26T14:23:04Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}

cat /etc/centos-release

CentOS Linux release 7.8.2003 (Core)

[root@k8s-master01 3]# ll
total 4
drwxr-xr-x 11 root root 4096 Oct 18 09:11 docker-gitlab
[root@k8s-master01 3]# cd docker-gitlab/kubernetes/
[root@k8s-master01 kubernetes]# ll
total 32
-rwxr-xr-x 1 root root 462 Oct 18 09:11 deploy.sh
-rw-r--r-- 1 root root 3015 Oct 18 09:11 gitlab-rc.yml
-rw-r--r-- 1 root root 254 Oct 18 09:11 gitlab-svc.yml
-rw-r--r-- 1 root root 1203 Oct 18 09:11 postgresql-rc.yml
-rw-r--r-- 1 root root 202 Oct 18 09:11 postgresql-svc.yml
-rw-r--r-- 1 root root 799 Oct 18 09:11 redis-rc.yml
-rw-r--r-- 1 root root 181 Oct 18 09:11 redis-svc.yml
-rwxr-xr-x 1 root root 355 Oct 18 09:11 teardown.sh
[root@k8s-master01 kubernetes]# sh deploy.sh

[root@k8s-master01 kubernetes]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
dashboard-kubernetes-dashboard-869c98cb6c-87zc2 1/1 Running 106 12d 10.244.1.108 k8s-node01
gitlab-879z2 0/1 Running 1 7m17s 10.244.1.143 k8s-node01
postgresql-5cvmv 1/1 Running 0 7m17s 10.244.2.29 k8s-node02
redis-v8wc6 1/1 Running 0 7m17s 10.244.1.144 k8s-node01

log

kubectl logs gitlab-879z2

Loading /etc/docker-gitlab/runtime/env-defaults
Initializing logdir...
Initializing datadir...
Generating OpenSSH host keys... RSA DSA ECDSA ED25519
Installing configuration templates...
Configuring gitlab...

kubectl describe pod gitlab-879z2

Name: gitlab-879z2
Namespace: default
Priority: 0
Node: k8s-node01/192.168.3.118
Start Time: Sun, 18 Oct 2020 09:12:08 +0800
Labels: name=gitlab
Annotations:
Status: Running
IP: 10.244.1.143
IPs:
IP: 10.244.1.143
Controlled By: ReplicationController/gitlab
Containers:
gitlab:
Container ID: docker://89c5593b6cb907a386828b84673475001785ce6154f30985d2c9532d04e2be33
Image: sameersbn/gitlab:13.4.3
Image ID: docker-pullable://sameersbn/gitlab@sha256:31f629cb079e08505e923e4794988a9886114c08c4c1947053c409bee4607f68
Ports: 80/TCP, 22/TCP
Host Ports: 0/TCP, 0/TCP
State: Running
Started: Sun, 18 Oct 2020 09:12:09 +0800
Ready: False
Restart Count: 0
Liveness: http-get http://:80/ delay=180s timeout=5s period=10s #success=1 #failure=3
Readiness: http-get http://:80/ delay=5s timeout=1s period=10s #success=1 #failure=3
Environment:
TZ: Asia/Kolkata
GITLAB_TIMEZONE: Kolkata
GITLAB_SECRETS_DB_KEY_BASE: long-and-random-alpha-numeric-string
GITLAB_SECRETS_SECRET_KEY_BASE: long-and-random-alpha-numeric-string
GITLAB_SECRETS_OTP_KEY_BASE: long-and-random-alpha-numeric-string
GITLAB_ROOT_PASSWORD:
GITLAB_ROOT_EMAIL:
GITLAB_HOST: git.default.cluster.local
GITLAB_PORT: 80
GITLAB_SSH_PORT: 22
GITLAB_NOTIFY_ON_BROKEN_BUILDS: true
GITLAB_NOTIFY_PUSHER: false
GITLAB_BACKUP_SCHEDULE: daily
GITLAB_BACKUP_TIME: 01:00
DB_TYPE: postgres
DB_HOST: postgresql
DB_PORT: 5432
DB_USER: gitlab
DB_PASS: passw0rd
DB_NAME: gitlab_production
REDIS_HOST: redis
REDIS_PORT: 6379
SMTP_ENABLED: false
SMTP_DOMAIN: www.example.com
SMTP_HOST: smtp.gmail.com
SMTP_PORT: 587
SMTP_USER: mailer@example.com
SMTP_PASS: password
SMTP_STARTTLS: true
SMTP_AUTHENTICATION: login
IMAP_ENABLED: false
IMAP_HOST: imap.gmail.com
IMAP_PORT: 993
IMAP_USER: mailer@example.com
IMAP_PASS: password
IMAP_SSL: true
IMAP_STARTTLS: false
Mounts:
/home/git/data from data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-q2k54 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
default-token-q2k54:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-q2k54
Optional: false
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message


Normal Scheduled 77s default-scheduler Successfully assigned default/gitlab-879z2 to k8s-node01
Normal Pulled 77s kubelet Container image "sameersbn/gitlab:13.4.3" already present on machine
Normal Created 77s kubelet Created container gitlab
Normal Started 76s kubelet Started container gitlab
Warning Unhealthy 9s (x7 over 69s) kubelet Readiness probe failed: Get "http://10.244.1.143:80/": dial tcp 10.244.1.143:80: connect: connection refused

Gitlab aways Readiness probe failed

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with kubernetes/deploy.sh and the GitLab, PostgreSQL, and Redis manifests listed in the issue, then inspect the GitLab pod with kubectl logs and kubectl describe. Compare the readiness probe configuration with the startup behavior shown in the output; done means the deployed GitLab pod becomes Ready and the probe no longer reports connection refused.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, gitlab, kubernetes, postgresql, redis, shell
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.