goharbor / goharbor/harbor-cli
design: declarative Harbor configuration contract and reconciliation semantics
- Dominant language
- Go
- Stars
- 163
- Forks
- 211
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
## Summary
Define a small, versioned desired-state contract for exporting and reconciling Harbor API-managed configuration. This turns the export requested in #1034 into a reusable GitOps input without treating it as a database, artifact, or disaster-recovery backup.
```text
YAML/JSON files -> strict decode -> deterministic merge -> validate -> snapshot Harbor -> plan -> apply
```
The first implementation uses `apiVersion: goharbor.io/v1alpha1` and `kind: HarborConfiguration`.
## Goals
- Export portable configuration without generated IDs, observed state, or secrets.
- Apply one file or a directory of reusable base/default/environment overlays.
- Produce a readable create/update/no-op plan before mutation.
- Converge managed fields while leaving omitted fields and extra live resources alone.
- Keep the contract usable by a future Harbor controller/operator.
## Non-goals for v1alpha1
- Database, artifact, repository, or installation backup.
- Pruning/deleting resources omitted from configuration.
- User or robot-secret migration.
- Transactional rollback across Harbor APIs.
- A generic representation of raw Harbor API responses.
## Contract example
Base configuration:
```yaml
apiVersion: goharbor.io/v1alpha1
kind: HarborConfiguration
spec:
registries:
- name: docker-hub
type: docker-hub
url: https://hub.docker.com
projects:
- name: application
public: false
quota:
storage: 100000000000
replicationPolicies:
- name: mirror
mode: push
registry: docker-hub
trigger:
type: manual
```
Production overlay:
```yaml
apiVersion: goharbor.io/v1alpha1
kind: HarborConfiguration
spec:
projects:
- name: application
metadata:
autoScan: true
quota:
storage: 500000000000
replicationPolicies:
- name: mirror
enabled: true
trigger:
type: scheduled
cron: "0 0 * * * *"
```
```sh
harbor export -f harbor.yaml
harbor apply -f production/ --dry-run
harbor apply -f production/
```
## Composition and ownership rules
A directory is read recursively. Hidden files/directories are ignored; only `.yaml`, `.yml`, and `.json` files participate. Relative paths are sorted lexically before merging.
- Every file is one strict `HarborConfiguration` document; unknown fields and multiple YAML documents fail.
- Maps merge by key.
- Registries, projects, webhooks, and replication policies merge by portable name.
- Later explicitly supplied scalar/object fields win.
- Webhook events/targets, replication filters, and replication trigger are atomic: a later value replaces the earlier value.
- Duplicate names inside one file fail. The same name across files means overlay.
- Omitted fields are unmanaged. Explicit `false`, `0`, `""`, or `[]` are desired values where the Harbor API permits them.
- There are no null tombstones or implicit deletion in v1alpha1.
This supports the reusable-package model: base equipment, recommended defaults, required settings, then environment/team-specific upgrades.
## Resource boundary
| Resource | Identity/reference | Export | Create/update | Delete |
|---|---|---:|---:|---:|
| System configuration | Harbor API field name | yes | update | no |
| External registry | name | yes | yes; registry type is immutable | no |
| Project | name; registry references use name | yes | yes | no |
| Project quota | project name | yes | update | no |
| Webhook policy | project/name | yes | yes; all targets preserved | no |
| Replication policy | name; registry references use name | yes | yes | no |
| Users/robots/artifacts | — | no | no | no |
Names, rather than Harbor-generated IDs, keep documents portable between installations.
## Reconciliation architecture
1. Strictly decode and merge all input before contacting Harbor.
2. Validate schema identity, unique names, required fields, secret references, and local references.
3. Snapshot supported live resources and build name-to-model indexes.
4. Compare only fields present in desired state.
5. Order actions by dependency:
- registries
- projects
- quotas and webhooks
- replication policies
- system configuration last
6. Print the plan; require confirmation unless `--yes`, or stop after planning with `--dry-run`.
7. Execute mutating actions in order and stop on the first error.
Apply is idempotent but not transactional. A failure may leave earlier actions applied; fix the cause and rerun the same file. There is no automatic rollback because the Harbor APIs do not provide one transaction across these resource types.
## Human intervention and drift
- If a human changes a managed field, the next apply restores the configured value.
- If a human changes a field omitted from the document, apply leaves it alone.
- If a human creates an extra resource, apply leaves it alone.
- If live state changes after planning but before mutation, Harbor may accept, reject, or supersede the update. The CLI stops on an API error; optimistic locking can be considered later if Harbor exposes useful revisions/ETags.
- Applying a complete export intentionally takes ownership of every exported field. Small hand-written files can manage only selected fields.
## Impossible or context-dependent desired state
Syntactically valid configuration is not necessarily executable by a particular Harbor instance. The policy is:
1. Reject locally detectable problems before any mutation.
2. Evaluate live capability/precondition checks after snapshot and before apply.
3. Return a resource-scoped actionable error; never silently skip or invent a fallback.
4. Stop on runtime API rejection and preserve the original Harbor error context.
Examples:
- Updating an existing registry's immutable `type`: reject; do not delete/recreate it.
- Referencing a missing registry: reject before creating the dependent project/policy.
- Creating a webhook without a target or with a missing secret environment variable: reject.
- Supplying a system key unsupported by the linked Harbor API model: reject.
- Asking for a quota or transition Harbor itself disallows: report the API rejection and stop.
### Authentication-owned users
Users are deliberately outside v1alpha1. In an OIDC or LDAP environment, identities are owned by the external provider and Harbor may reject local user creation even if a future schema could express it.
If users are added later, they require an explicit capability matrix based on both current and desired authentication mode:
```text
desired users + final auth mode OIDC/LDAP -> preflight error, no mutation
desired users + supported local DB auth -> plan user actions
auth-mode transition + desired users -> validate the final state and ordering first
```
The config model must not imply that expressible means universally executable. Authentication-dependent resources should not be added until their ownership, transition ordering, and error behavior are agreed. The current choice is safer: strict decoding rejects unsupported `users` rather than accepting and ignoring them.
## Secrets
Exports never contain registry credentials, webhook auth headers, robot secrets, or secret system fields. Apply may resolve write-only values from environment references:
```yaml
credential:
type: basic
accessKeyFrom:
env: HARBOR_REGISTRY_USERNAME
accessSecretFrom:
env: HARBOR_REGISTRY_PASSWORD
```
Omitting a credential leaves the live credential unchanged. Secret references force an update because Harbor cannot return a comparable plaintext value.
## Decisions to confirm
- Keep top-level `harbor export` and `harbor apply` for whole-instance desired state; retain `harbor config` for system-only workflows.
- Keep apply additive in v1alpha1; design prune/delete separately with explicit ownership and safety controls.
- Keep users and robots out until authentication/ownership semantics are defined.
- Treat API-dependent impossibility as a preflight capability error where detectable, otherwise as a scoped runtime failure with safe rerun.
- Evolve the contract through new `apiVersion` values rather than silently changing field meaning.
## Acceptance criteria
- YAML and JSON round-trip through the versioned domain model.
- Export is stable, portable, and secret-free.
- File and directory apply produce deterministic plans.
- A second apply after convergence is no-op except for explicitly supplied write-only secrets.
- No omitted resource is deleted.
- Invalid overlays, missing references/secrets, immutable transitions, and unsupported fields fail clearly.
- Documentation states field ownership, list replacement, partial-apply behavior, and unsupported-resource boundaries.
Contributor guide
Research direction
Start with the existing `harbor export` and `harbor apply` command entry points, then trace how configuration is decoded and applied. Resolve the open contract decisions before implementation, including ownership, additive apply behavior, and capability failures. Done means versioned YAML/JSON round-tripping, deterministic plans, clear validation errors, secret-free export, idempotent convergence, and no deletion of omitted resources.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend-api-design, cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100