Hangs while scanning dependencies only in Docker container
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- docker, python
- Domain
- build-system, devops, performance
Research direction
Start by reproducing the timing difference with the shown Dockerfile and the docker build/docker run ... pio run -v commands, then inspect the project's platformio.ini dependency configuration. Compare the dependency-scanning path and timings between native Windows and the Python 3.8.7-slim container; done means the container scan no longer has the reported extreme delay and the result is validated on the reproduction setup.
Written by the indexing model from the issue text.
Description
- PlatformIO Core.
If you’ve found a bug, please provide an information below.
Configuration
Operating system:
Host machine: Windows 10
Docker container:
FROM python:3.8.7-slim
ENV DEBIAN_FRONTEND=noninteractive
ENV PROJ_DIR /firmware
RUN apt-get update -q \
&& apt-get install --no-install-recommends -yq \
lcov valgrind git \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN python3 -m pip install --upgrade pip setuptools
RUN python3 -m pip install platformio
# this is a really crappy way to get all the required environment installed in the container
COPY . $PROJ_DIR
RUN pio run --project-dir ${PROJ_DIR} || true
RUN pio system prune -f
RUN rm -r $PROJ_DIR
PlatformIO Version (platformio --version):
Host: PlatformIO Core, version 5.2.3
Docker: PlatformIO Core, version 5.2.3
Description of problem
Compiling firmware on Docker takes a much longer time to scan the dependencies than a native Windows run. I know that the hypervisor can cause extra time but the running it with timestamps shows a 2.61s time on Windows vs 77.52 on Linux (~3000% difference).
Steps to Reproduce
If necessary I can try to create a minimal reproducible but you can use the above docker container and add it to a Platformio project and from the project directory:
docker build -t platformio-builder:latest .
docker run -v [pwd]:/firmware platformio run
Actual Results
Windows:
> filter timestamp {"$(Get-Date -Format o): $_"}; pio run -v | timestamp
...
2021-12-14T10:22:03.72: LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
2021-12-14T10:22:03.72: LDF Modes: Finder ~ chain, Compatibility ~ soft
2021-12-14T10:22:03.98: Found 87 compatible libraries
2021-12-14T10:22:03.98: Scanning dependencies...
2021-12-14T10:22:06.64: Dependency Graph
2021-12-14T10:22:06.64: |--
> docker run -v [pwd]:/firmware pio run -v | timestamp
...
10:24:25.59: LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
10:24:25.59: LDF Modes: Finder ~ chain, Compatibility ~ soft
10:24:33.58: Found 87 compatible libraries
10:24:33.58: Scanning dependencies...
10:25:51.10: Dependency Graph
10:25:51.11: |-- ...
Expected Results
The times are closer
If problems with PlatformIO Build System:
The content of platformio.ini:
A little minimal since there's a lot in it
[platformio]
description = Firmware
[env] ;Common is done for all environments
monitor_speed = 115200
monitor_flags= --raw
lib_extra_dirs =
./shared-config
; ...
;==========ESP32===========
[env:esp32]
platform = espressif32
board = esp32dev
framework = arduino
build_type = debug
debug_tool = esp-prog
debug_init_break = tbreak setup
Source file to reproduce issue:
Insert here...
- Dominant language
- Python
- Stars
- 9.5k
- Forks
- 905
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from platformio/platformio-core
-
feature
Difficulty 4/5 3-5 days Newbie friendliness 48/100
platformio/platformio-core#5502 · 3 comments ·
-
feature platform
Difficulty 4/5 3-5 days Newbie friendliness 35/100
platformio/platformio-core#5497 ·
-
help wanted registry
Difficulty 5/5 Over a week Newbie friendliness 25/100
platformio/platformio-core#5484 · 3 comments ·
-
board feature framework platform
Difficulty 5/5 Over a week Newbie friendliness 25/100
platformio/platformio-core#5476 ·
-
enhancement package management
Difficulty 4/5 3-5 days Newbie friendliness 48/100
platformio/platformio-core#5463 ·
All issues in platformio/platformio-core
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
bancolombia/sentinel#23 ·
-
test md OpenCI
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
bug client
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100