moby / moby/swarmkit

Service Scale rapidly changes many times and lots of container were in Created status.

Open
#2,214 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
3.7k
Forks
676
Avg merge
4d 9h
Merged PRs (30d)
6

Description

A 2 nodes cluster.
The docker version is

17.03.1-ce

The service is

services:
  log-stress:
    image: nginx:latest
    ports:
      - 80:80
    deploy:
      replicas: 200

then change the scale of the service rapidly, the scale number is random between 100 and 200.
Use

docker ps -a 

and thousands of containers ware:

4ef0a5a21323        nginx@sha256:41ad9967ea448d7c2b203c699b429abe1ed5af331cd92533900c6d77490e0268                                        "nginx -g 'daemon ..."   2 days ago          Created                                     sl1_log-stress.39.w2f9uim22879l6y4sb4dthdnw
d332478742f8        nginx@sha256:41ad9967ea448d7c2b203c699b429abe1ed5af331cd92533900c6d77490e0268                                        "nginx -g 'daemon ..."   2 days ago          Created                                     sl1_log-stress.38.ojw8pt1lpy6ksx8pyzhcvqt31
b3e5a60432b8        nginx@sha256:41ad9967ea448d7c2b203c699b429abe1ed5af331cd92533900c6d77490e0268                                        "nginx -g 'daemon ..."   2 days ago          Created                                     sl1_log-stress.7.wdqfor75sw6xwqoimib5cxs6e
47dd83f24743        nginx@sha256:41ad9967ea448d7c2b203c699b429abe1ed5af331cd92533900c6d77490e0268                                        "nginx -g 'daemon ..."   2 days ago          Created                                     sl1_log-stress.65.q2h2iedprlgyv4yp1jllycq3m
85d5f427930d        nginx@sha256:41ad9967ea448d7c2b203c699b429abe1ed5af331cd92533900c6d77490e0268                                        "nginx -g 'daemon ..."   2 days ago          Created                                     sl1_log-stress.29.xz1gd0osjk5ae9md0byjud8i8

The slot number is in 1~199, and there are many container in every slot.

sl1_log-stress.24 : 55
sl1_log-stress.25 : 14
sl1_log-stress.26 : 76
sl1_log-stress.27 : 48
sl1_log-stress.28 : 62
sl1_log-stress.29 : 45
sl1_log-stress.30 : 60

and I'v checked the containers in the same slot and their service ID are the same .

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the two-node setup with Docker 17.03.1-ce, the provided service definition, and rapid random scaling between 100 and 200 replicas. Start with docker ps -a and compare the repeated slot containers and service IDs. Done means rapid scaling no longer leaves large numbers of containers in Created status.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.