Hangs while scanning dependencies only in Docker container

Open
#4,138 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
docker, python

Research direction

Start by reproducing the timing difference with the shown Dockerfile and the docker build/docker run ... pio run -v commands, then inspect the project's platformio.ini dependency configuration. Compare the dependency-scanning path and timings between native Windows and the Python 3.8.7-slim container; done means the container scan no longer has the reported extreme delay and the result is validated on the reproduction setup.

Written by the indexing model from the issue text.

Description

help wanted integration LDF
  • PlatformIO Core.
    If you’ve found a bug, please provide an information below.
Configuration

Operating system:
Host machine: Windows 10
Docker container:

FROM python:3.8.7-slim

ENV DEBIAN_FRONTEND=noninteractive
ENV PROJ_DIR /firmware

RUN apt-get update -q \
  && apt-get install --no-install-recommends -yq \
  lcov valgrind git \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

RUN python3 -m pip install --upgrade pip setuptools
RUN python3 -m pip install platformio

# this is a really crappy way to get all the required environment installed in the container
COPY . $PROJ_DIR
RUN pio run --project-dir ${PROJ_DIR} || true
RUN pio system prune -f
RUN rm -r $PROJ_DIR

PlatformIO Version (platformio --version):
Host: PlatformIO Core, version 5.2.3
Docker: PlatformIO Core, version 5.2.3

Description of problem

Compiling firmware on Docker takes a much longer time to scan the dependencies than a native Windows run. I know that the hypervisor can cause extra time but the running it with timestamps shows a 2.61s time on Windows vs 77.52 on Linux (~3000% difference).

Steps to Reproduce

If necessary I can try to create a minimal reproducible but you can use the above docker container and add it to a Platformio project and from the project directory:

docker build -t platformio-builder:latest .
docker run -v [pwd]:/firmware platformio run
Actual Results

Windows:

> filter timestamp {"$(Get-Date -Format o): $_"}; pio run -v | timestamp
...
2021-12-14T10:22:03.72: LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
2021-12-14T10:22:03.72: LDF Modes: Finder ~ chain, Compatibility ~ soft
2021-12-14T10:22:03.98: Found 87 compatible libraries
2021-12-14T10:22:03.98: Scanning dependencies...
2021-12-14T10:22:06.64: Dependency Graph
2021-12-14T10:22:06.64: |-- 
> docker run -v [pwd]:/firmware pio run -v | timestamp
...
10:24:25.59: LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
10:24:25.59: LDF Modes: Finder ~ chain, Compatibility ~ soft
10:24:33.58: Found 87 compatible libraries
10:24:33.58: Scanning dependencies...
10:25:51.10: Dependency Graph
10:25:51.11: |-- ...
Expected Results

The times are closer

If problems with PlatformIO Build System:

The content of platformio.ini:
A little minimal since there's a lot in it

[platformio]
description = Firmware

[env] ;Common is done for all environments
monitor_speed = 115200
monitor_flags= --raw

lib_extra_dirs =
  ./shared-config

; ...

;==========ESP32===========
[env:esp32]
platform = espressif32
board = esp32dev
framework = arduino
build_type = debug
debug_tool = esp-prog
debug_init_break = tbreak setup

Source file to reproduce issue:

Insert here...
Dominant language
Python
Stars
9.5k
Forks
905
Avg merge
2d 13h
Merged PRs (30d)
2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from platformio/platformio-core

All issues in platformio/platformio-core

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.