sameersbn / sameersbn/docker-gitlab

problem deleting image or tag 16.6.1

Open
#2,867 13 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
8.1k
Forks
2.1k
Avg merge
17h 47m
Merged PRs (30d)
7

Description

Whenever I try to delete an image or a tag, I'm getting this error but the logs in the registry aren't showing anything, nor are those in Gitlab.
This installation was via a Linux package, I made the backup and left it in a new configuration with Docker Swarm, using Traefik as a reverse proxy

image

LOGS GITLAB:

gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,401 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,402 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,402 INFO Included extra file "/etc/supervisor/conf.d/gitlab-pages.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,402 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,402 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,402 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,403 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,403 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,403 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,403 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,403 INFO Set uid to user 0 succeeded
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,413 INFO RPC interface 'supervisor' initialized
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:07,413 INFO supervisord started with pid 1
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:08,418 INFO spawned: 'gitaly' with pid 897
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:08,423 INFO spawned: 'puma' with pid 898
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:08,428 INFO spawned: 'gitlab-workhorse' with pid 899
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:08,446 INFO spawned: 'sidekiq' with pid 900
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:08,451 INFO spawned: 'gitlab-pages' with pid 901
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:08,476 INFO spawned: 'sshd' with pid 907
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:08,504 INFO spawned: 'nginx' with pid 908
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:08,517 INFO spawned: 'cron' with pid 909
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:09,595 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:09,596 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:09,596 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:09,597 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:09,598 INFO success: gitlab-pages entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:09,598 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:09,599 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:49:09,599 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 09:55:20,380 INFO reaped unknown pid 1052
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 10:05:54,121 INFO reaped unknown pid 1084
gitlab_gitlab.1.6ovng6s6zitt@SRVDOCKER02    | 2023-12-14 10:09:14,909 INFO reaped unknown pid 1190

LOGS REGISTRY:

{"auth.user.name":"","err.code":"digest invalid","err.message":"provided digest did not match uploaded content","go.version":"go1.20.8","http.request.host":"registry_registry:5000","http.request.id":"09c7cdce-048b-4c9d-89e0-c368efc1b13f","http.request.method":"DELETE","http.request.remoteaddr":"10.0.1.232:48710","http.request.uri":"/v2/compmon/application/panorama_tunnel_ssh/nginx/manifests/alpine","http.request.useragent":"GitLab/16.6.1","http.response.contenttype":"application/json; charset=utf-8","http.response.duration":"7.641504ms","http.response.status":400,"http.response.written":98,"level":"error","msg":"response completed with error","time":"2023-12-15T12:18:36.491871799-03:00","vars.name":"compmon/application/panorama_tunnel_ssh/nginx","vars.reference":"alpine"}
registry_registry.1.og01zgrjx0v9@SRVDOCKER02    | 10.0.1.232 - - [15/Dec/2023:12:18:36 -0300] "DELETE /v2/compmon/application/panorama_tunnel_ssh/nginx/manifests/alpine HTTP/1.1" 400 98 "" "GitLab/16.6.1"

application_json.log

{"severity":"INFO","time":"2023-12-15T15:17:23.240Z","meta.caller_id":"ContainerRegistry::DeleteContainerRepositoryWorker","correlation_id":"5d8fd23d3c5c78765a575cda99fdafd7","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","container_repository_id":85,"container_repository_path":"compmon/application/panorama_tunnel_ssh/nginx","project_id":189,"third_party_cleanup_tags_service":true}
{"severity":"ERROR","time":"2023-12-15T15:17:23.298Z","meta.caller_id":"ContainerRegistry::DeleteContainerRepositoryWorker","correlation_id":"5d8fd23d3c5c78765a575cda99fdafd7","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","service_class":"Projects::ContainerRepository::DeleteTagsService","container_repository_id":85,"project_id":189,"message":"could not delete tags: alpine"}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing deletion of the image or tag in the Docker Swarm and Traefik setup, then compare the GitLab and registry logs shown in the issue. The registry request currently returns a 400 digest invalid error; done means the image or tag is deleted successfully and the related cleanup logs no longer report failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.