allow fetching obsolete artifacts from snapshot.debian.org

Author: evgeni (evgeni)

Redmine Issue: 4866, https://pulp.plan.io/issues/4866

Debian removes old versions of packages that have been updated from their mirrors. Because of that using lazy sync against official Debian mirrors might end up with incomplete repos when Pulp syncs metadata on day 1, on day 2 a package gets updated/replaced and on day 3 pulp tries to actually get that package.

Debian offers a snapshot service at https://snapshot.debian.org/, where you can get almost every file ever seen in the Debian archive by accessing https://snapshot.debian.org/file/HASH (where HASH is currently the SHA1 hash of the file). The API docs of snapshot can be found at https://salsa.debian.org/snapshot-team/snapshot/raw/master/API

The idea would be to allow a secondary source in pulp_deb that is tried when the primary mirror replied with a 404.

This is probably also useful for other backends, not just Debian.

Snapshot currently only supports sha1 hashes, but the authors promised to accept patches for sha256 if anyone would write one (the source of the service is in the same repo as the api doc linked above).

Start by tracing the pulp_deb lazy-sync path and review the snapshot.debian.org API documentation linked in the issue. Define how a secondary source should be configured and used after a primary mirror returns 404, then verify that obsolete Debian artifacts can be fetched without breaking normal mirror retrieval.