Missing Ubuntu 24.04 (noble) packages for Tentacle 20.2.3 and 20.2.4
- Dominant language
- Shell
- Stars
- 44
- Forks
- 105
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 36
Description
Hello,
Tracker: https://tracker.ceph.com/issues/79633
**Description**
Ubuntu 24.04 LTS (noble) Debian packages were published for Tentacle 20.2.1 and 20.2.2, but are absent from 20.2.3 and 20.2.4. The rolling `debian-tentacle` repository has also lost its `noble` suite.
Because 20.2.4 is the hotfix for CVE-2025-30156, CVE-2026-39944, CVE-2026-50152 and CVE-2026-54330, operators running cephadm on Ubuntu 24.04 LTS currently have no upstream path to install the patched host tooling (`cephadm`, `ceph-common`, `ceph-volume`) at a version matching the 20.2.4 container image.
**Affected / working versions (`download.ceph.com`)**
| Release | jammy | noble | bookworm |
|---|---|---|---|
| 20.2.0 | present | missing (published, then pulled 20 Nov 2025) | present |
| 20.2.1 | present | present | present |
| 20.2.2 | present | present | present |
| 20.2.3 | present | missing | present |
| 20.2.4 | present | missing | present |
| debian-tentacle (rolling) | present | missing | present |
Squid (19.2.2–19.2.6) has never published a noble suite. Noble coverage in Tentacle therefore appears only in the 20.2.1–20.2.2 window and stops at 20.2.3.
20.2.0 is the exception that proves a suite can be dropped after publish: on 20 Nov 2025 noble packages for 20.2.0 were removed because ceph-mgr crashed on Python 3.12 (tracker #73930). Remaining `debian-20.2.0/dists/` is jammy + bookworm, dated that same day. The announcement said Ubuntu 24.04 would return in a later point release. 20.2.1 (2 Apr 2026) and 20.2.2 (15 Jun 2026) did so. 20.2.3 and 20.2.4 did not.
**Steps to reproduce**
```
# noble present for 20.2.1 and 20.2.2
curl -sI https://download.ceph.com/debian-20.2.1/dists/noble/Release | head -1
# HTTP/1.1 200 OK
curl -sI https://download.ceph.com/debian-20.2.2/dists/noble/Release | head -1
# HTTP/1.1 200 OK
# noble absent for 20.2.0 (after the Nov 2025 pull), 20.2.3 and 20.2.4
curl -sI https://download.ceph.com/debian-20.2.0/dists/noble/Release | head -1
curl -sI https://download.ceph.com/debian-20.2.3/dists/noble/Release | head -1
curl -sI https://download.ceph.com/debian-20.2.4/dists/noble/Release | head -1
# HTTP/1.1 404 Not Found
# jammy and bookworm are present for the same releases
curl -s https://download.ceph.com/debian-20.2.4/dists/
# bookworm/ jammy/
# rolling tree also lost noble
curl -s https://download.ceph.com/debian-tentacle/dists/
# bookworm/ jammy/
```
Evidence that noble was built and published previously:
```
curl -s https://download.ceph.com/debian-20.2.2/dists/noble/main/binary-amd64/Packages \
| grep -A3 -E '^Package: (cephadm|ceph-common|ceph-volume)$' \
| grep -E '^(Package|Version)'
# Package: ceph-common / Version: 20.2.2-1noble
# Package: ceph-volume / Version: 20.2.2-1noble
# Package: cephadm / Version: 20.2.2-1noble
curl -s https://download.ceph.com/debian-20.2.2/dists/noble/Release | head -4
# Origin: ceph.com
# Suite: stable
# Codename: noble
# Date: Mon, 15 Jun 2026 18:05:57 UTC
```
**Expected behaviour**
`20.2.4-1noble` packages published under `debian-20.2.4/dists/noble/` and in the rolling `debian-tentacle/dists/noble/` suite, matching the coverage shipped for 20.2.1 and 20.2.2.
**Impact**
Ubuntu 24.04 LTS is the current Ubuntu LTS baseline for new bare-metal Ceph deployments. The Ubuntu primary archive only carries Squid (`19.2.3-0ubuntu0.24.04.3`) for noble, so `download.ceph.com` is the only upstream source of Tentacle packages for this distro.
Sites that deployed 20.2.2 on noble from the official `20.2.2-1noble` packages are now unable to apply a security release without either pinning host tooling to a version older than the daemon container image, or rebuilding packages themselves.
Installing jammy `20.2.4-1jammy` debs on noble is not a workaround: `ceph-common` / `ceph-volume` are ABI-tied to 24.04.
**Shaman evidence — this looks like a release-matrix / publish gap, not “noble cannot build”**
CI does build noble. The official tag job for v20.2.4 did not include noble.
- v20.2.4 commit: `7f793731f1b39eb4f465e960113d2363c311b964`
- Jenkins: `ceph-dev-pipeline/8318` (18 Aug 2026)
- Builds: https://shaman.ceph.com/builds/ceph/tentacle/7f793731f1b39eb4f465e960113d2363c311b964/
| Distro on that SHA | Package version | Published to download.ceph.com? |
|---|---|---|
| Ubuntu jammy | 20.2.4-1jammy | yes (`debian-20.2.4/dists/jammy/`) |
| Debian bookworm | 20.2.4-1bookworm | yes |
| Rocky 10 / CentOS 9 | 20.2.4-0 | yes (RPM trees) |
| Ubuntu noble | not in that job | no |
So the missing suite is not that CI cannot produce `20.2.4-1noble`. The tag pipeline built jammy + bookworm + EL and skipped noble, and nothing later copied a noble tree to `download.ceph.com`.
Contrast with v20.2.3 commit `06c2f9c35b67055a8a6fb99d1be236b3c4832ace`: jammy/bookworm were built 4 Aug; a noble rebuild of the same tag SHA landed on Shaman on 27 Aug as `20.2.3-1noble`. That rebuild was never copied to `debian-20.2.3/dists/noble/`. `tentacle/latest/ubuntu/noble` currently points at this 20.2.3 CI repo, not 20.2.4:
```
deb [trusted=yes] https://3.chacra.ceph.com/r/ceph/tentacle/06c2f9c35b67055a8a6fb99d1be236b3c4832ace/ubuntu/noble/flavors/default/ noble main
```
Closest noble snapshot after v20.2.4 (8 commits later, not the tag): `20.2.4-8-ga1e131a2-1noble`. CI-only, unsigned, not a release pin.
```
deb [trusted=yes] https://2.chacra.ceph.com/r/ceph/tentacle/a1e131a20a3c32ea8e9b15bfbabc2fa76c1b286e/ubuntu/noble/flavors/default/ noble main
```
`https://shaman.ceph.com/api/repos/ceph/v20.2.4/latest/ubuntu/noble/repo/` returns “no repository is ready”.
Related: #66914 (Resolved — CI builds noble), #73930 (Resolved — reason 20.2.0 noble was pulled), #68584, #66825, cephadm-ansible #327.
OS recommendations still list Ubuntu 24.04 native Tentacle packages as Upcoming, which is out of date relative to 20.2.1/20.2.2 actually shipping noble.
**Questions**
1. Was noble coverage dropped after 20.2.2, or is this a break in the release/publishing pipeline? Shaman shows the v20.2.4 tag job never scheduled noble, while a v20.2.3 noble rebuild exists in CI and was not published.
2. If intentional, can this be documented and announced, given 20.2.1/20.2.2 established noble as a working target?
3. Can noble builds be produced for 20.2.4 out-of-band, given the CVE severity and the “upgrade as soon as possible” guidance in the release announcement?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the ceph-build release and publishing pipeline, the v20.2.4 tag job, and the Shaman build records linked in the issue. Compare the v20.2.3 noble rebuild and v20.2.4 package matrix, then verify that noble repositories are published for both debian-20.2.4 and debian-tentacle and that the expected packages are available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell, ubuntu
- Domain
- build-system, ci-cd, devops, release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100