Deb packages cannot be purged via apt
- Dominant language
- Go
- Stars
- 1.1k
- Forks
- 378
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 74
Description
My configuration management system detected that the way your buildkite-agent packages are built for Debian-based systems is not to spec, and prevents the full purging of packages, as per the following demonstration:
```
$ sudo apt install buildkite-agent
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
buildkite-agent
0 to upgrade, 1 to newly install, 0 to remove and 0 not to upgrade.
Need to get 0 B/12.3 MB of archives.
After this operation, 34.9 MB of additional disk space will be used.
Selecting previously unselected package buildkite-agent.
(Reading database ... 82096 files and directories currently installed.)
Preparing to unpack .../buildkite-agent_3.18.0-3219_amd64.deb ...
Unpacking buildkite-agent (3.18.0-3219) ...
Setting up buildkite-agent (3.18.0-3219) ...
_ _ _ _ _ _ _ _
| | (_) | | | | (_) | | |
| |__ _ _ _| | __| | | ___| |_ ___ __ _ __ _ ___ _ __ | |_
| '_ \| | | | | |/ _` | |/ / | __/ _ \ / _` |/ _` |/ _ \ '_ \| __|
| |_) | |_| | | | (_| | <| | || __/ | (_| | (_| | __/ | | | |_
|_.__/ \__,_|_|_|\__,_|_|\_\_|\__\___| \__,_|\__, |\___|_| |_|\__|
__/ |
|___/
You now need to add your agent token to "/etc/buildkite-agent/buildkite-agent.cfg"
and then you can start your agent by running "sudo systemctl enable buildkite-agent && sudo systemctl start buildkite-agent"
$ sudo apt purge buildkite-agent
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
buildkite-agent*
0 to upgrade, 0 to newly install, 1 to remove and 0 not to upgrade.
After this operation, 34.9 MB disk space will be freed.
Do you want to continue? [Y/n]
(Reading database ... 82119 files and directories currently installed.)
Removing buildkite-agent (3.18.0-3219) ...
Stopping buildkite-agent systemd service
Purging configuration files for buildkite-agent (3.18.0-3219) ...
$ ls -la /etc/buildkite-agent/
total 16
drwxr-xr-x 3 buildkite-agent buildkite-agent 4096 Jan 29 17:19 .
drwxr-xr-x 102 root root 4096 Jan 29 17:19 ..
-rw------- 1 buildkite-agent buildkite-agent 1734 Jan 29 17:19 buildkite-agent.cfg
drwxr-xr-x 2 buildkite-agent buildkite-agent 4096 Jan 29 17:19 hooks
$ dpkg -S /etc/buildkite-agent/buildkite-agent.cfg
dpkg-query: no path found matching pattern /etc/buildkite-agent/buildkite-agent.cfg
$
```
As you can see, `/etc/buildkite-agent` and its contents remain. It didn't take long to find the cause:
```
$ dpkg -e buildkite-agent_3.18.0-3219_amd64.deb
$ ls -l
total 12064
-rw-r--r-- 1 abolte abolte 12345570 Jan 29 17:26 buildkite-agent_3.18.0-3219_amd64.deb
drwxr-xr-x 2 abolte abolte 4096 Jan 21 16:59 DEBIAN
$ find DEBIAN/
DEBIAN/
DEBIAN/postrm
DEBIAN/prerm
DEBIAN/md5sums
DEBIAN/postinst
DEBIAN/preinst
DEBIAN/control
$
```
There is no `conffiles` file, and the `DEBIAN/postinst` script is copying other files that are installed:
https://github.com/buildkite/agent/blob/912a7b544d1ab79e23f1c8e1714fac75b5c561ab/packaging/linux/scripts/after-install-and-upgrade.sh#L36-L37
It goes without saying, that isn't working as intended. `dh_installdeb` is never able to learn about anything deployed under `/etc` on the target system. It's also deploying the same file to a system two times unnecessarily.
For an example of how it should be done, see the lighttpd package shipped with Debian or Ubuntu.
```
$ dpkg -e lighttpd_1.4.35-4ubuntu2.1_amd64.deb
$ dpkg -x lighttpd_1.4.35-4ubuntu2.1_amd64.deb .
$ find etc -type f
etc/lighttpd/lighttpd.conf
etc/lighttpd/conf-available/10-status.conf
etc/lighttpd/conf-available/10-cgi.conf
etc/lighttpd/conf-available/10-evasive.conf
etc/lighttpd/conf-available/10-rrdtool.conf
etc/lighttpd/conf-available/10-fastcgi.conf
etc/lighttpd/conf-available/10-ssl.conf
etc/lighttpd/conf-available/10-proxy.conf
etc/lighttpd/conf-available/05-auth.conf
etc/lighttpd/conf-available/10-dir-listing.conf
etc/lighttpd/conf-available/10-usertrack.conf
etc/lighttpd/conf-available/10-ssi.conf
etc/lighttpd/conf-available/15-fastcgi-php.conf
etc/lighttpd/conf-available/11-extforward.conf
etc/lighttpd/conf-available/10-no-www.conf
etc/lighttpd/conf-available/10-accesslog.conf
etc/lighttpd/conf-available/README
etc/lighttpd/conf-available/90-debian-doc.conf
etc/lighttpd/conf-available/10-userdir.conf
etc/lighttpd/conf-available/10-flv-streaming.conf
etc/lighttpd/conf-available/10-expire.conf
etc/lighttpd/conf-available/10-simple-vhost.conf
etc/lighttpd/conf-available/10-evhost.conf
etc/cron.daily/lighttpd
etc/ufw/applications.d/lighttpd
etc/logrotate.d/lighttpd
etc/init.d/lighttpd
$ cat DEBIAN/conffiles
/etc/cron.daily/lighttpd
/etc/init.d/lighttpd
/etc/lighttpd/conf-available/05-auth.conf
/etc/lighttpd/conf-available/10-accesslog.conf
/etc/lighttpd/conf-available/10-cgi.conf
/etc/lighttpd/conf-available/10-dir-listing.conf
/etc/lighttpd/conf-available/10-evasive.conf
/etc/lighttpd/conf-available/10-evhost.conf
/etc/lighttpd/conf-available/10-expire.conf
/etc/lighttpd/conf-available/10-fastcgi.conf
/etc/lighttpd/conf-available/10-flv-streaming.conf
/etc/lighttpd/conf-available/10-no-www.conf
/etc/lighttpd/conf-available/10-proxy.conf
/etc/lighttpd/conf-available/10-rrdtool.conf
/etc/lighttpd/conf-available/10-simple-vhost.conf
/etc/lighttpd/conf-available/10-ssi.conf
/etc/lighttpd/conf-available/10-ssl.conf
/etc/lighttpd/conf-available/10-status.conf
/etc/lighttpd/conf-available/10-userdir.conf
/etc/lighttpd/conf-available/10-usertrack.conf
/etc/lighttpd/conf-available/11-extforward.conf
/etc/lighttpd/conf-available/15-fastcgi-php.conf
/etc/lighttpd/conf-available/90-debian-doc.conf
/etc/lighttpd/conf-available/README
/etc/lighttpd/lighttpd.conf
/etc/logrotate.d/lighttpd
/etc/ufw/applications.d/lighttpd
$ # Yes they're the same:
$ diff <(find etc -type f | sort) <(sort DEBIAN/conffiles | cut -d '/' -f 2-)
```
The `conffiles` is just there in the above example for backwards compatibility with older systems, since newer systems will know that files under `/etc` are config files automatically - but it's probably a good idea for you to follow suit given the older distribution releases supported.
I think you should include your configuration files along with all the other files (removing lines in the shell scripts to copy them), and then specify those in `conffiles`. At this point apt should start behaving correctly (for config files at least). See https://www.debian.org/doc/manuals/maint-guide/dother.en.html for details.
I note in https://github.com/buildkite/agent/blob/master/scripts/build-debian-package.sh you're using the `fpm` tool to generate packages. I'm personally not a fan of packaging without standard Debian tools (surely the linter would have flagged such issues?) but a quick search suggests there is a `--config-files` argument available which may help.
https://github.com/jordansissel/fpm/pull/506/files
One thing that should be in the `postrm` script that isn't, are commands like:
```
deluser buildkite-agent || true
delgroup buildkite-agent || true
```
As it stands, the `buildkite-agent` user and group is also still left around on the system after purging.
If using fpm, you might consider employing the use of a free software tool like [AIDE](https://aide.github.io/) to ensure the package install/upgrade/remove/purge functions are behaving correctly.
Contributor guide
Research direction
Start with scripts/build-debian-package.sh and packaging/linux/scripts/after-install-and-upgrade.sh, then inspect the generated DEBIAN metadata, including conffiles and postrm. Reproduce installation, removal, and purge with apt; done means configuration files, the package directory, and the buildkite-agent user and group are handled correctly after purge.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- debian, shell
- Domain
- build-system, devops, release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100