aws-samples / aws-samples/appmod-blueprints

Move EKS Capability log delivery into provisioning (RGD/composition Job, then declarative CRs)

Open
#750 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
105
Forks
62
Avg merge
11h 17m
Merged PRs (30d)
76

Description

## Summary

EKS Capability controller log delivery (KRO, and optionally ACK/Argo CD) is currently activated **imperatively** via the `enable-capability-logs` Taskfile task, invoked from `task install`. This works and is idempotent, but it is not declarative/GitOps-native for the KRO- and Crossplane-provisioned clusters (dev/prod and hub).

We should move log-delivery activation into the cluster **provisioning path** so every cluster gets it automatically as part of its desired state.

## Current state (option 1 — implemented)

- `Taskfile.yaml` → `enable-capability-logs` task sets up CloudWatch Vended Logs delivery per capability:
- account-level resource policy for `delivery.logs.amazonaws.com`
- `PutDeliverySource` (capability ARN, `EKS_CAPABILITY_KRO_LOGS` / `EKS_CAPABILITY_ACK_LOGS`)
- `PutDeliveryDestination` → `/aws/eks//capability/` log group
- `CreateDelivery`
- Invoked from the root `install` task; re-run after enabling more spokes.

## Why not declarative today (option 3 blocked)

Neither ACK nor Crossplane/upbound currently expose CRDs for the CloudWatch Logs **delivery** resources:
- ACK `cloudwatchlogs` only has `LogGroup` (no `DeliverySource`/`DeliveryDestination`/`Delivery`).
- No `*delivery*` CRDs from ACK or the upbound AWS provider.

So pure declarative CRs in the RGD/composition are not possible yet.

## Target options

### Option 2 — aws-cli Job embedded in provisioning (preferred interim)
Mirror the existing `eks-capabilities-job` pattern (Crossplane composition `gitops/abstractions/crossplane/platform-cluster/templates/composition.yaml`) which already runs `aws eks create-capability`. Add an equivalent step/Job that runs the `aws logs put-delivery-source/put-delivery-destination/create-delivery` setup after the capability is ACTIVE:
- **Hub (Crossplane):** extend the `eks-capabilities-job` to also configure log delivery.
- **Dev/Prod (KRO RGD):** add a Job/step to `gitops/addons/charts/kro/resource-groups/manifests/eks/rg-eks.yaml` (capabilities are created there via `eks.services.k8s.aws/Capability`). Manifest lives in Git; runs per-cluster automatically.

### Option 3 — declarative CRs (preferred long-term)
Once ACK ships `DeliverySource`/`DeliveryDestination`/`Delivery` CRDs (or the upbound provider adds them), replace the Job with declarative CRs in the RGD/composition and retire the Taskfile task.

### Option 4 -- wait for native support in ACK & Crossplane

## Acceptance criteria
- [ ] dev/prod (KRO) and hub (Crossplane) clusters get KRO (and optionally ACK) capability log delivery created automatically at provisioning time, GitOps-tracked.
- [ ] No reliance on the imperative `enable-capability-logs` task for newly provisioned clusters.
- [ ] Idempotent; safe on re-provision/upgrade.
- [ ] Revisit option 3 when ACK/Crossplane `Delivery*` CRDs become available.

## References
- Docs: Access EKS Capabilities controller logs (CloudWatch Vended Logs)
- Implemented in branch `feature/cloudfront-on-agent-platform` (`enable-capability-logs` task + `install` wiring)

Contributor guide

Open the contributing guide

Research direction

Start with the existing enable-capability-logs task in Taskfile.yaml and the eks-capabilities-job in gitops/abstractions/crossplane/platform-cluster/templates/composition.yaml. Then inspect gitops/addons/charts/kro/resource-groups/manifests/eks/rg-eks.yaml to understand the KRO capability path. Done means dev/prod and hub provisioning create capability log delivery idempotently without relying on the imperative task.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes
Domain
cloud, devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.