docker / docker/docker-py

Is there a reason why `client.images.build` doesn't stream output logs ?

Open
#2,182 8 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.2k
Forks
1.7k
Avg merge
13d 8h
Merged PRs (30d)
2

Description

Seems to be a recurrent question. I looked at the code and it seems to me there is no reason why the higher-level API generator couldn't return the output logs as they come. This could be achieved by using a sort of fake future for the image object returned by the build function. Something like that (very rough sketch) :

    def build(self, **kwargs):
        resp = self.client.api.build(**kwargs)
        if isinstance(resp, six.string_types):
            return self.get(resp)
        last_event = None
        image_id = None
        result_stream, internal_stream = itertools.tee(json_stream(resp))
        
        result = {}

        def gen():
            for chunk in internal_stream:
                if 'error' in chunk:
                    raise BuildError(chunk['error'], result_stream)
                if 'stream' in chunk:
                    match = re.search(
                        r'(^Successfully built |sha256:)([0-9a-f]+)$',
                        chunk['stream']
                    )
                    if match:
                        image_id = match.group(2)
                    else:
                        yield chunk
                last_event = chunk
            
            if image_id:
                result['image'] = self.get(image_id)
            else:
                raise BuildError(last_event or 'Unknown', result_stream)

        return result, gen()

Maybe this could be switched-on by adding an option to build for example stream_output ?

And this can be used like this :

    image, output_iter = docker_client.images.build(fileobj=fd, tag=args.tag)

    for line in output_iter:
        print(line)
    
    print(image)

Of course the nicer solution for that would be to use asyncio but that's then becoming more complicated :P

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the higher-level client.images.build path and the client.api.build response handling described in the issue. Determine how output can be yielded while retaining the final image result and BuildError details; done means build logs are available incrementally without breaking existing image retrieval behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.