aws-samples / aws-samples/appmod-blueprints

Port AMP platform secret (${prefix}/platform/amp) from legacy Terraform to Taskfile/Crossplane

Open
#733 0 comments 0 reactions 0 assignees View on GitHub
agentic-platform bug enhancement
Dominant language
Shell
Stars
105
Forks
62
Avg merge
11h 17m
Merged PRs (30d)
76

Description

## Summary

The Secrets Manager secret `${resource_prefix}/platform/amp` (e.g. `peeks/platform/amp`) is **not provisioned** in the new bootstrap model (Taskfile + EKS Capabilities + Crossplane). As a result, the metrics-driven progressive delivery analysis (Module 30.5) fails because the `appmodservice` KRO RGD / KubeVela components create an `ExternalSecret` that references this secret and it cannot sync.

This secret is currently only created by the **legacy Terraform** module `platform/infra/terraform/common/secrets.tf`, which is no longer executed end-to-end in the CloudFront / agent-platform bootstrap flow.

## Impact

- **Module 30.5 (Metrics-driven decisions)** is blocked on a fresh environment.
- `ExternalSecret` `amp-workspace-secrets-` stays in `SecretSyncedError` / `could not get secret data from provider`.
- The metrics `AnalysisTemplate` (Prometheus/AMP query) cannot authenticate against the AMP workspace.

## Root cause

The `appmodservice` RGD (and the KubeVela `metrics` path) expect a Secrets Manager secret with this exact shape:

```
Secret name : /platform/amp # e.g. peeks/platform/amp
Properties :
amp-workspace :
amp-region :
```

The only thing that creates it today is Terraform:

```hcl
# platform/infra/terraform/common/secrets.tf:156
resource "aws_secretsmanager_secret" "argorollouts_secret" {
name = "${local.context_prefix}/platform/amp"
...
}
resource "aws_secretsmanager_secret_version" "argorollouts_secret_version" {
secret_string = jsonencode({
amp-region = local.hub_cluster.region
amp-workspace = module.managed_service_prometheus.workspace_prometheus_endpoint
})
}
```

Note the in-code comment already flags it: `# Only for backward compatibility` / `# TODO: Move this to cluster config secret`.

In the new model this Terraform is not run, so the secret is missing even though the AMP workspace itself exists (created/managed elsewhere, e.g. `peeks-observability-amp`, and its endpoint is also present in `hub/secrets` as `amp_endpoint_url` / `amp_region`).

## Workaround applied during validation

Created the secret manually from the existing AMP workspace values:

```bash
HUB_SECRETS=$(aws secretsmanager get-secret-value --secret-id hub/secrets --region us-west-2 --query SecretString --output text)
AMP_URL=$(echo "$HUB_SECRETS" | jq -r '.amp_endpoint_url')
AMP_REGION=$(echo "$HUB_SECRETS" | jq -r '.amp_region')

aws secretsmanager create-secret --name peeks/platform/amp --region us-west-2 \
--secret-string "$(jq -n --arg url "$AMP_URL" --arg region "$AMP_REGION" \
'{"amp-workspace": $url, "amp-region": $region}')"
```

After this, the `ExternalSecret` reconciled to `SecretSynced` and Module 30.5 infra is functional.

## Proposed fix — port the Terraform logic into the new bootstrap

We need to recreate `${resource_prefix}/platform/amp` in the GitOps-first model. Two options:

### Option A — Taskfile (seed step)
Add the secret creation to the `hub:seed*` flow in
`cluster-providers/kind-crossplane/Taskfile.yaml`, alongside the existing
Secrets Manager seeding (similar to `hub:seed-secret`). Pull the AMP endpoint
from the AMP workspace (or reuse `amp_endpoint_url` already present in
`/secrets`) and write `{amp-workspace, amp-region}` to
`${RESOURCE_PREFIX}/platform/amp`.

### Option B — Crossplane (preferred, GitOps-native)
Manage the secret declaratively with the AWS provider that is already
installed (`provider-aws-*`). The AMP workspace CRDs are present
(`workspaces.amp.aws.upbound.io`), so we can:
- reference/observe the AMP `Workspace`, and
- create the Secrets Manager secret + version via Crossplane (or via an
ExternalSecret `PushSecret`), keeping the `{amp-workspace, amp-region}`
property shape the RGD expects.

Option B is more consistent with the rest of the platform (everything else is
moving to Crossplane / GitOps) and removes the last dependency on the legacy
Terraform `common/secrets.tf`.

## Acceptance criteria

- [ ] `${resource_prefix}/platform/amp` is created automatically on a fresh bootstrap (no manual step).
- [ ] Properties are exactly `amp-workspace` (AMP query endpoint URL) and `amp-region`.
- [ ] `ExternalSecret amp-workspace-secrets-` reaches `SecretSynced` without intervention.
- [ ] Module 30.5 metrics-driven analysis runs end-to-end on a clean environment.
- [ ] Legacy Terraform block in `platform/infra/terraform/common/secrets.tf` is removed or documented as deprecated once ported.

## Context

- Found during end-to-end validation of the KubeVela path on `feature/cloudfront-on-agent-platform`.
- Related cleanup: this is the last functional blocker for the KubeVela progressive-delivery path after the webhook/CloudFront routing fix.

Contributor guide

Open the contributing guide

Research direction

Start with platform/infra/terraform/common/secrets.tf:156 and the existing hub:seed-secret flow in cluster-providers/kind-crossplane/Taskfile.yaml; inspect how hub/secrets exposes amp_endpoint_url and amp_region, plus the installed provider-aws resources. Choose and document the bootstrap approach, then verify a fresh environment creates the exact platform/amp properties, reaches SecretSynced, and completes Module 30.5 without manual intervention.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes, prometheus, terraform
Domain
cloud, devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.