astral-sh / astral-sh/python-build-standalone

Docker image builds hang on `apt-get update`

Open
#238 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.4k
Forks
314
Avg merge
1d 9h
Merged PRs (30d)
27

Description

I'm seeing extremely slow container builds on Linux that hang at 100% CPU during `apt-get update`. I see very little network IO, disk IO, and memory consumption. I've let it run for >30 minutes to no avail.

Specifically, it looks like this is occurring for the `image-gcc`, `image-xcb`, and `image-build` targets.

I can't figure out how to get verbose output from `apt` here, any recommendations would be helpful I've been spinning my wheels on it for a while.

Is it possible I'm misinterpreting the output?

e.g.

```
❯ ./build-linux.py
...
image-gcc> Step 9/10 : RUN apt-get update
image-xcb> ---> Running in 64f6de138eb1
image-build> ---> Running in 3eb38e067d05
image-gcc> ---> Running in 2e0cd76e45bc
```

```
❯ docker stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
64f6de138eb1 gifted_khayyam 99.94% 1.145MiB / 31.25GiB 0.00% 18.6kB / 0B 0B / 0B 3
2e0cd76e45bc infallible_carver 100.15% 1.188MiB / 31.25GiB 0.00% 18.6kB / 0B 0B / 0B 3
3eb38e067d05 flamboyant_hellman 100.14% 1.121MiB / 31.25GiB 0.00% 18.6kB / 0B 0B / 0B 3
```

```
❯ docker inspect 2e0cd76e45bc
[
{
"Id": "2e0cd76e45bce897a34c096651987a0e373e06ede8e7922fac2cbe0d31d6af31",
"Created": "2024-03-20T17:52:31.490588566Z",
"Path": "/bin/sh",
"Args": [
"-c",
"apt-get update"
],
...
```

```
❯ docker exec -it 2e0cd76e45bc top

top - 17:57:24 up 1:55, 0 users, load average: 3.21, 3.33, 3.15
Tasks: 4 total, 2 running, 2 sleeping, 0 stopped, 0 zombie
%Cpu(s): 7.5 us, 12.3 sy, 0.0 ni, 80.0 id, 0.0 wa, 0.1 hi, 0.1 si, 0.0 st
KiB Mem: 32771652 total, 16323288 used, 16448364 free, 1327952 buffers
KiB Swap: 0 total, 0 used, 0 free. 6118300 cached Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8 root 20 0 29100 656 384 R 100.0 0.0 4:52.27 apt-get
1 root 20 0 4344 1280 1280 S 0.0 0.0 0:00.01 sh
7 root 20 0 29104 4352 4096 S 0.0 0.0 0:00.00 apt-get
15 root 20 0 21980 2432 2048 R 0.0 0.0 0:00.01 top

```

I can `docker build` a derived Dockerfile like

```Dockerfile
# Debian Jessie.
FROM debian@sha256:32ad5050caffb2c7e969dac873bce2c370015c2256ff984b70c1c08b3a2816a0
MAINTAINER Gregory Szorc

RUN groupadd -g 1000 build && \
useradd -u 1000 -g 1000 -d /build -s /bin/bash -m build && \
mkdir /tools && \
chown -R build:build /build /tools

ENV HOME=/build \
SHELL=/bin/bash \
USER=build \
LOGNAME=build \
HOSTNAME=builder \
DEBIAN_FRONTEND=noninteractive

CMD ["/bin/bash", "--login"]
WORKDIR '/build'

# Jessie's signing keys expired in late 2022. So need to add [trusted=yes] to force trust.
# Jessie stopped publishing snapshots in March 2023.
RUN for s in debian_jessie debian_jessie-updates debian-security_jessie/updates; do \
echo "deb [trusted=yes] http://snapshot.debian.org/archive/${s%_*}/20230322T152120Z/ ${s#*_} main"; \
done > /etc/apt/sources.list && \
( \
echo 'APT::Get::Assume-Yes "true";'; \
echo 'APT::Install-Recommends "false";'; \
echo 'Acquire::Check-Valid-Until "false";'; \
echo 'Acquire::Retries "5";'; \
) > /etc/apt/apt.conf.d/99cpython-portable

RUN ( echo 'amd64'; \
echo 'i386'; \
) > /var/lib/dpkg/arch

RUN apt-get update

RUN apt-get install \
autoconf \
automake \
bison \
build-essential \
gawk \
gcc \
gcc-multilib \
libtool \
make \
tar \
texinfo \
xz-utils \
unzip
```

with no issues.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.