Enable containerd mirror overrides per registry host in Peerd to support custom image pull routing logic

### Is your feature request related to a problem? Yes

# Problem

## Necessity of P2P Scenarios

A well-known challenge in the Kubernetes ecosystem is that every node in a cluster must pull the same container image, which leads to excessive registry load, throttling risk, and slower pod startup times. Peerd already addresses this by modifying each node’s containerd configuration to prioritize peer-to-peer (P2P) transfers before falling back to the upstream registry.

## Necessity of Custom Mirroring Scenarios

However, a broader and increasingly common set of challenges—particularly in multi-cloud and large-scale environments—involves **managing per-registry-host image pull routing logic**. These challenges often expose friction between _Application Developer_ and _Infrastructure Engineer_ personas. Below are four scenarios that commonly arise:

1. Single workload, multi-cloud
- **Pain today**: _Application Developers_ must edit image references (e.g., `my1.azurecr.io`, `xxx.dkr.ecr...`) or parameterize them in Helm charts and Kubernetes YAMLs for each target cloud.
- **Desired outcome**: _Infrastructure Engineers_ can remap registry hosts at the infrastructure level using a single P2P-aware tool, with zero changes required to application manifests.
2. Onboarding existing workloads to a new cloud
- **Pain today**: _Application Developers_ must re-author every chart or YAML to reflect cloud-specific registries, or they must refactor and parameterize existing files to make them cloud-agnostic.
- **Desired outcome**: _Infrastructure Engineers_ use a single P2P-aware tool to remap registry hosts and customize image pull routing logic, requiring zero changes to application deployment manifests.
3. Multi-registry fail-over
- **Pain today**: _Application Developers_ must modify deployment manifests during an outage or upstream registry unavailability.
- **Desired outcome**: _Infrastructure Engineers_ can use a single P2P-aware tool to map registry hosts to an ordered list of additional registry hosts to pull images. _Application Developers_ are not involved in any infrastructure or registry-related outages.
4. Load-balancing pulls in large clusters
- **Pain today**: _Application Developers_ must fragment workloads into separate deployments with different image sources to avoid throttling a single registry.
- **Desired outcome**: _Infrastructure Engineers_ use a single P2P-aware tool to define node-level mirroring logic (e.g., by node pool or nodeSelector), distributing image pulls across different registry hosts. Application teams deploy once; Peerd handles the rest.

## Lack of Unified Tooling for Containerd Mirror Configs

Currently, there is no elegant or unified solution to these problems without directly editing containerd configuration on each node. At best, _Infrastructure Engineers_ must manage multiple tools—one for P2P (e.g., Peerd), and another tool for remapping and failover logic—leading to tool conflicts. At worst, _Application Developers_ are pulled into infrastructure-level concerns, eroding the clean separation of responsibilities and slowing development velocity.

### What solution do you propose?

# Proposal

## Unify P2P and Custom Mirroring in a Single Tool

This proposal introduces **first-class support for containerd mirror overrides in Peerd**, enabling **Helm-driven**, **per-registry-host image pull routing**, customizable **at the node or node pool level** in a Kubernetes cluster. It unifies modifying containerd for two purposes—peer-to-peer distribution and registry mirror routing—into a single, extensible, and cloud-agnostic solution that is deployable on any Kubernetes cluster, whether in the cloud or on-premises.

## Auth-aware Experience

Crucially, the proposal also extends Peerd’s functionality to support containerd mirror configurations without introducing additional burden on users to manage authentication or secrets. Peerd should be able to "just work" on clusters that already have valid authentication configured—whether that’s via `ImagePullSecrets`, `dockerconfigjson`-backed secrets, or node-level identities such as:
* AKS: kubelet identity with role assignment to Azure Container Registry
* EKS: node IAM roles granting access to Amazon ECR
* GKE: node service account permissions for Artifact Registry

Peerd should be able to detect and reuse these existing credentials—without requiring application or infrastructure teams to redefine or duplicate them—ensuring seamless image pull authentication across all mirror endpoints.

## Proposed YAML Schema

To enable this functionality, Peerd should support the following `values.yaml` structure, allowing both P2P and custom mirror routing logic to be applied when deploying Peerd using a standard Helm command like `helm install peerd -f values.yaml`. The YAML syntax and definitions follow the YAML block below in the next section.

```yaml
peerd:
mirrorOverrides:
- host: docker.io
orderedRoutingLogic: [ "P2P", "originalHost", "mirrorEndpoints" ]
mirrorEndpoints:
mirrorSequence: sequential
mirrorHosts:
- host: my.azurecr.io
capabilities: ["pull", "resolve", "push"]
- host: aws_account_id.dkr.ecr.region.amazonaws.com
capabilities: ["pull", "resolve", "push"]
- host: onprem.company.com
orderedRoutingLogic: ...
mirrorEndpoints: ...
```

## YAML Schema Definition

All custom mirror overrides per registry host must be defined under `mirrorOverrides`. Under `mirrorOverrides` is a list that is used to override the image pull routing for each registry host.

`host`
- This is required. It specifies the registry host being overridden with custom image pull routing logic. The value must be just the host (such as `docker.io`, `ghcr.io`, or `my.azurecr.io`). Do not include repository paths like `docker.io/library/nginx`.

`orderedRoutingLogic`
- This is optional. It defines the exact order in which Peerd should apply its routing logic when pulling an image from this host. The list can include any combination of the following values: `"P2P"`, `"originalHost"`, and `"mirrorEndpoints"`. The default (if not specified) is to try all of them in the order: P2P first, then the original registry host, then mirror hosts. Each entry is attempted in the listed order, and fallback to the next only happens if the current one fails.

`mirrorEndpoints`
- This is optional. It defines a set of alternate registry mirrors to use if the original registry fails or is bypassed, depending on your routing logic. This section contains two subfields: `mirrorSequence` and `mirrorHosts`.

- `mirrorSequence`
- This is required if the parent `mirrorEndpoints` field is defined. It defines how Peerd will walk through the mirror hosts. For now, only `"sequential"` is supported, which means Peerd will try the mirror hosts one by one in the order they are listed.

- `mirrorHosts`
- This field is under `mirrorEndpoints`. It contains a list of `hosts` underneath. It defines the actual mirror registry hosts to try pulling from. Each mirror host can include several fields:

- `host`
- This is required. It is the hostname of the mirror registry (like `my.company.com`, `my.azurecr.io`, or `aws_account_id.dkr.ecr.region.amazonaws.com`).

- `imagePullSecrets`
- This is optional. You can specify one or more Kubernetes secrets of `kind: Secret` to authenticate with the mirror registry. Each secret should have a `name` field.

- `name`
- This is required if the parent `imagePullSecrets` is defined for this mirror. This is the name of the `kind: Secret` resource that contains the registry credentials for authenticating with the mirror registry.

- `useNodeIdentity`
- This is optional. If set to `true`, Peerd will assume that the node can authenticate to the mirror registry using its cloud-native identity. For example, this might be an Azure-managed identity in AKS, an IAM role in EKS, or a service account in GKE.
- If both `imagePullSecrets` and `useNodeIdentity` are set, Peerd will **not** accept the definition.

- `capabilities`
- This is required. It defines what the mirror is trusted to do. Valid options include:
- `"pull"`: The mirror can serve content for pull operations.
- `"resolve"`: The mirror is trusted to convert tags to digests (this should only be enabled for trusted mirrors, since resolving tags can affect image integrity).
- `"push"`: The mirror can accept pushed content (only relevant if pushing from nodes through Peerd becomes supported).


### What alternatives have you considered?

# Alternatives Considered

## Alternatives using Containerd `hosts.toml` Overrides

Currently, there is no Kubernetes-native way to apply custom image pull routing logic without directly modifying containerd configuration files on each node. While containerd supports per-host `hosts.toml` mirror overrides, managing these configurations manually—or via DaemonSets or init containers—is operationally fragile, hard to standardize, and poorly integrated with Helm-based or GitOps workflows.

Some organizations work around this by baking static `hosts.toml` files into custom node images, which creates cloud vendor lock-in and slows down iteration, as every registry change requires node pool rebuilds. Others patch containerd at runtime using systemd units or bootstrap scripts, but these lack portability, observability, and centralized management.

Moreover, none of these approaches support peer-to-peer (P2P) distribution. They are limited to static mirroring logic and cannot leverage cluster-local optimizations.

## Alternatives using Peerd Today

While Peerd already integrates with containerd’s mirroring capabilities, it does not currently support custom mirror routing logic per registry host. Organizations needing both P2P and custom mirroring are forced to layer Peerd on top of additional tooling—such as DaemonSets or node image modifications to configure containerd manually. This creates risk of misconfiguration or conflicts between Peerd and other low-level tooling that touches containerd. It also makes cluster-wide image routing harder to reason about and automate with multiple conflicting tools and workflows.

### Any additional context?

_No response_

### Are you willing to submit PRs to contribute to this feature?

- [x] Yes, I am willing to implement it.