GoogleCloudPlatform / GoogleCloudPlatform/scion
feat(config): allow local host-path projection into file secrets without a Hub
- Dominant language
- Go
- Stars
- 1.7k
- Forks
- 263
- Avg merge
- 7h 6m
- Merged PRs (30d)
- 447
Description
## Context
Scion's file-secrets pipeline already handles host-file → container-path projection end-to-end:
- `writeFileSecrets` at `pkg/runtime/common.go:607-682` stages files on the host, creates parent directories, and emits `host:container:ro` mount specs
- Plumbed through all three runtimes: `pkg/runtime/docker.go:53`, `pkg/runtime/podman.go:131`, `pkg/runtime/apple_container.go:53`
- `scion hub secret set --type file --target @` at `cmd/hub_secret.go:105` exercises this path today
The only limitation: `api.RequiredSecret` (`pkg/api/types.go:472-482`) declares `{Key, Type, Target}` but not a local-host origin. Resolution of a `Key` into a `ResolvedSecret` goes through a Hub.
Users running Scion locally (no Hub) have no way to ask "inject `~/.ssh/id_ed25519` at `/home/scion/.ssh/id_ed25519`" through a config-driven path. They fall back to either:
- Raw `volumes:` mounts (works but sits at a different abstraction level than other secrets)
- Hand-placing files in template `home/` directories (leaks into the grove template tree and potentially into git)
- Standing up a Hub (overkill for single-user local development)
This matters for any Scion consumer whose agents need SSH keys, gh CLI tokens, kubeconfig, cloud credentials, or database URLs to do real work.
Adjacent history: #94 (closed) fixed a decode bug on the same `writeFileSecrets` path. This ask adds a local-origin resolution step *before* the Hub step, reusing the same downstream pipeline.
## Proposal
Add a `HostPath` field to a local `SecretMount` type (or extend `RequiredSecret`) and resolve it to a `ResolvedSecret` on the host side before the existing Hub-resolution step runs. All downstream machinery — `writeFileSecrets`, runtime mount injection — reused unchanged.
**Shape in `settings.yaml`:**
```yaml
harness_configs:
claude:
secrets:
- key: ssh_key
type: file
host_path: ~/.ssh/id_ed25519
target: /home/scion/.ssh/id_ed25519
- key: gh_hosts
type: file
host_path: ~/.config/gh/hosts.yml
target: /home/scion/.config/gh/hosts.yml
- key: kubeconfig
type: file
host_path: ~/.kube/config
target: /home/scion/.kube/config
```
**Implementation sketch:**
1. Extend the struct in `pkg/config/settings_v1.go` with an optional `HostPath string` on the secrets-declaration type (either `RequiredSecret` directly or a new local-only `SecretMount` sibling to keep the Hub-facing surface clean).
2. In `pkg/agent/run.go` around line 815 (before the `m.Runtime.Run` call), walk the config's declared secrets and for each one with `HostPath` set:
- Tilde-expand the path; resolve against the user's home
- Read the file; base64-encode
- Construct `api.ResolvedSecret{Type: file, Target, Value: base64(contents)}`
- Prepend/append to `runCfg.ResolvedSecrets`
3. If both `HostPath` and Hub resolution apply to the same key, precedence is explicit-loud failure (config error) rather than silent last-wins.
4. Read-only by default (already true in `writeFileSecrets`).
**Behavior for missing host file:** fail loudly at agent start with a clear error message (`secret "ssh_key": host_path ~/.ssh/id_ed25519 not found`), not silent skip. Consistent with how auth propagation fails today when a required credential file is absent.
## Why this shape
- **Additive, not disruptive.** No existing `settings.yaml` files break; `host_path` is optional.
- **Reuses end-to-end plumbing.** `writeFileSecrets`, mount-spec generation, runtime layer — all untouched.
- **Single mechanism for all file secrets.** Covers ssh, gh, kubeconfig, aws, stripe, db URLs, the lot. No ssh-specific flags, no per-credential-type shims.
- **No Hub required.** Local-dev ergonomics improved without weakening Hub-based deployments.
- **Precedent in the codebase.** `OverlayFileSecrets` at `pkg/harness/auth.go:113-136` already does host-file-to-ResolvedSecret conversion for auth-specific paths; this generalizes that pattern.
## Acceptance
- `HostPath` field exists on the secrets-declaration type in `settings_v1.go`, documented in the schema comment
- Local host files resolve to `ResolvedSecret` entries before runtime Run, verified by a unit test in `pkg/agent/run_test.go` (or equivalent) that exercises the resolution with a tempdir-based host file
- Integration test confirms `scion start` with a `host_path`-declared secret lands the file at the target path inside a live container, readable by the container user
- Clear error on missing host file; exit code non-zero, message identifies the offending key
- Existing Hub-based secret resolution unaffected — test matrix includes both paths
- `settings.yaml` schema doc + at least one example showing the SSH-key use case
## Out of scope
- Hot-reload of secrets into running agents (orthogonal — belongs with a pre-start-hook primitive)
- Templated container targets (e.g., `target: /home/{{ user }}/.ssh/id_rsa`) — ship with literal paths first
- Write-mode secrets (stays read-only like `writeFileSecrets` already enforces)
- Replacement for Hub-based flows — both coexist
## Open design questions
1. New `SecretMount` type vs. `HostPath` field on existing `RequiredSecret`? Leaning toward the former — keeps the Hub-required declaration surface clean and makes the local-origin intent explicit in the config.
2. Glob support (`host_path: ~/.ssh/id_*`)? Probably not for v1 — explicit > implicit, and expansion semantics would need their own spec.
3. Environment-variable expansion in `host_path`? Tilde-only is minimal and safe for v1.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with pkg/config/settings_v1.go and pkg/agent/run.go around the pre-runtime resolution path; compare the existing OverlayFileSecrets pattern in pkg/harness/auth.go. Review pkg/agent/run_test.go and the existing integration coverage before choosing how local declarations coexist with Hub secrets. Done means host_path files resolve before runtime execution, missing files fail clearly, Hub behavior remains intact, and the schema and SSH example are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100