compiler-explorer / compiler-explorer/infra
Builder pins conan 1.59, forcing a pyenv 3.10 dependency
- Dominant language
- Python
- Stars
- 434
- Forks
- 429
- Avg merge
- 4d 21h
- Merged PRs (30d)
- 38
Description
`init/start-builder.sh` installs `conan==1.59`, which transitively pins
`PyYAML<=6.0`. That has no wheel for the Python 3.12 shipped by Ubuntu 24.04
and fails to build from source against modern Cython (`'build_ext' has no
attribute 'cython_sources'`). The same constraint is already documented in
`setup-conan.sh`, where the conan *server* was moved to 1.66.0 for exactly this
reason.
Consequence: the Linux library builder cannot use the system Python and depends
on a pyenv-installed 3.10.16 baked into the ce-ci `lin-builder` AMI. That
dependency was silently broken until now -- pyenv lived under `/root`, which the
`ubuntu`-run build job cannot traverse, so `python` fell through to the system
one. On jammy that happened to work because the system Python was also 3.10. On
noble it fails.
The immediate fix (in flight) moves pyenv to `/opt/pyenv` and adds a guard so a
fallthrough fails loudly instead of silently. That restores the status quo but
keeps the 3.10 pin.
Proposal: move the builder client to `conan==1.66.0` to match the server.
- 1.66 relaxes the PyYAML pin and installs cleanly on 3.12
- v1 wire protocol is stable across the 1.x series, so client/server stay
compatible (v2 would be a hard break)
- removes the pyenv dependency from the builder image entirely, along with the
`chmod -R a+rX` and PATH gymnastics
Worth doing separately from the 24.04 upgrade so a conan version change is not
tangled with an OS change. Needs a real library build matrix to validate, not
just one library.
Refs:
- `init/start-builder.sh` (conan install and PYENV_ROOT)
- `setup-conan.sh` (the 1.66 rationale for the server)
- ce-ci `packer/install-builder-runner.sh` (where pyenv is baked)
Contributor guide
Research direction
Read init/start-builder.sh to understand the Conan and PYENV_ROOT setup, then compare its client constraint with the 1.66 rationale in setup-conan.sh. Check ce-ci packer/install-builder-runner.sh for the baked pyenv dependency. Done means the builder no longer requires pyenv for this constraint and a real library build matrix succeeds on the supported builder.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100