docker / docker/docker-py

Weird behavior with follow=True blocking execution when running container exits

Open
#3,084 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.2k
Forks
1.7k
Avg merge
13d 8h
Merged PRs (30d)
2

Description

Environment:

OS: Ubuntu 22.04.1 LTS
Docker Engine: 20.10.21
Python version: 3.10.6
docker-py: 6.0.1

Context

The Python script below runs wrapped by a GitLab runner process that is a CLI application written with Typer.
The logging driver of Docker is journald

GitLab Runner: 15.6.1
Typer: 0.7.0

Question

I am trying to print logs from a running container. (it's execution is very fast)

import docker

cli = docker.from_env()
container = cli.containers.run(image=image, detach=True, remove=False)

for cl in container.logs(stream=True, follow=True):
    typer.secho(cl.strip(), fg=typer.colors.BLUE)

container.remove()

For a reason, container.logs(stream=True, follow=True) is acting weirdly whenever the detached container exits. Sometimes the execution of my program resumes and finishes properly and very often, the underlying container exits and my program get stuck in the for loop, doing nothing.

This has for consequence that jobs running in GitLab are timeouting due to no exit code of my cli.

It seems most likely system related since I can run the cli from my laptop without trouble. Any ideas from where the problem could come?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the container.logs(stream=True, follow=True) call in the provided script and reproduce the behavior with Docker Engine 20.10.21, journald logging, and a short-lived detached container. Compare the stream behavior after the container exits and confirm that the CLI loop terminates and container.remove() is reached.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.