microsoft / microsoft/aspire

Add aks.WithClusterDefaults(...) for one-line, pit-of-success AKS + AGC + TLS setup

Open
#17,160 0 comments 0 reactions 0 assignees View on GitHub
area-app-model area-deployment kubernetes
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

## Summary

Add a one-line "do the safe thing" entry point on `AzureKubernetesEnvironmentResource` so the canonical TLS-terminating, AGC-fronted AKS recipe is a single call instead of ~10 explicit ones:

```csharp
var acmeEmail = builder.AddParameter("acme-email");

var aks = builder.AddAzureKubernetesEnvironment("aks")
.WithClusterDefaults(acmeEmail);

var api = builder.AddProject("api")
.WithExternalHttpEndpoints();
// → vnet, subnets, AGC LB, cert-manager, Let's Encrypt prod issuer,
// public gateway with HTTPS + 301 redirect + HSTS, and a /api route.
```

The current shape (`AddAzureVirtualNetwork` → `AddSubnet` × N → `AddAzureKubernetesEnvironment` → `WithSubnet` → `WithSystemNodePool` → `AddLoadBalancer` × N → `AddCertManager` → `AddIssuer().WithLetsEncryptProduction().WithHttp01Solver()` → `AddGateway().WithLoadBalancer().WithRoute().WithTls()`) is the right meccano set for power users, but it is verbose and has real footguns. By comparison `AddAzureContainerAppEnvironment("env")` is one line. AKS should have an equivalent "pit of success" entry point that keeps every existing extension reachable for customization.

## Why now

This is the natural follow-on to #17158. That issue makes `WithTls(issuer)` own the entire TLS posture (HTTPS listener + 301 redirect on :80 + HSTS `max-age=1y`). With those defaults in place, an auto-created gateway inherits a safe TLS story for free — so a `WithClusterDefaults` API can wire everything up without inventing any new TLS knobs.

**Sequencing:** ship #17158 first, then `WithClusterDefaults` on top. If we ship `WithClusterDefaults` before #17158, the redirect/HSTS HTTPRoutes have to be emitted from `WithClusterDefaults` itself as a stop-gap, and then deleted when #17158 lands. Strongly prefer ordering: #17158 first.

## Proposed API

```csharp
namespace Aspire.Hosting;

public static class AzureKubernetesClusterDefaultsExtensions
{
[Experimental("ASPIREAZURE0xx")]
public static IResourceBuilder WithClusterDefaults(
this IResourceBuilder builder,
IResourceBuilder acmeEmail,
Action? configure = null);
}

public sealed class ClusterDefaultsOptions
{
// Networking
public string AddressSpace { get; set; } = "10.100.0.0/16";
public string AksSubnetCidr { get; set; } = "10.100.0.0/22";
public string LoadBalancerSubnetCidr { get; set; } = "10.100.4.0/24";

// Compute
public string SystemNodePoolVmSize { get; set; } = "Standard_D2as_v5";
public int SystemNodePoolMinCount { get; set; } = 1;
public int SystemNodePoolMaxCount { get; set; } = 3;

// Names of auto-created child resources
public string LoadBalancerName { get; set; } = "public";
public string GatewayName { get; set; } = "public-gw";
public string CertManagerName { get; set; } = "cert-manager";
public string IssuerName { get; set; } = "letsencrypt";

// TLS / ACME
public LetsEncryptEnvironment AcmeEnvironment { get; set; } = LetsEncryptEnvironment.Production;
public bool EnableTls { get; set; } = true;

/// Forwarded to WithTls(issuer, ...) on the auto-created gateway.
/// Base defaults from #17158 (HTTPS + 301 + HSTS 1y) already produce
/// the pit-of-success posture; this hook only matters for users who
/// want to disable the redirect or tighten/loosen HSTS.
public Action? ConfigureTls { get; set; }

// Auto-routing
public bool AutoRouteExternalEndpoints { get; set; } = true;
/// Path template applied per resource. `{name}` is the resource name.
/// Default routes everything at `/{name}`; set to "/" to expose a
/// single service at the root.
public string RoutePathTemplate { get; set; } = "/{name}";
}

public enum LetsEncryptEnvironment { Staging, Production }
```

`acmeEmail` is **positional and required** because:

* TLS is on by default and Let's Encrypt requires it.
* A `ParameterResource` keeps the value out of source and per-environment.
* If we hid it inside `ClusterDefaultsOptions`, it would silently fail at publish time when missing — positional makes the requirement compile-time.

If a future user genuinely needs no TLS we'll add a no-arg overload that requires `EnableTls = false` in the options callback (otherwise it throws).

## What it provisions

| Default | Value / behavior |
| --- | --- |
| VNet | `{name}-vnet`, address space `10.100.0.0/16` |
| AKS subnet | `aks-nodes`, `10.100.0.0/22` (1024 IPs) |
| ALB subnet | `alb-public`, `10.100.4.0/24`, delegated to ServiceNetworking |
| AKS env | `WithSubnet(aks-nodes)`, system pool `Standard_D2as_v5` |
| Load balancer | `AddLoadBalancer("public", alb-public)` |
| cert-manager | `AddCertManager("cert-manager")` (pinned chart version) |
| Cluster issuer | `AddIssuer("letsencrypt").WithLetsEncryptProduction(acmeEmail).WithHttp01Solver()` |
| Gateway | `AddGateway("public-gw").WithLoadBalancer(public).WithTls(letsencrypt)` — TLS posture per #17158 (HTTPS listener + 301 redirect on :80 + HSTS `max-age=1y`) |
| Auto-routing | For each resource added with external HTTP/HTTPS endpoints, `gateway.WithRoute("/{resourceName}", endpoint)` is wired automatically |

All resources stay in the model and reachable, so an advanced user can keep mutating them after the call:

```csharp
aks.WithClusterDefaults(acmeEmail)
.WithSubnet(myExistingSubnet); // replaces the auto AKS subnet
```

`WithSubnet` / `WithSystemNodePool` / `WithContainerRegistry` already behave as replace semantics, so no new conflict-resolution code is needed.

## Auto-routing mechanism

The implementation hangs a `BeforeStartEvent` (run mode) **and** a publish-time pipeline step on the AKS env that:

1. Enumerates resources in the model.
2. Skips infrastructure (itself, ACR, vnet, subnets, dashboard, cert-manager, issuer, gateway, LB, etc.).
3. For each resource with at least one `EndpointAnnotation` whose `IsExternal == true`, adds `gateway.WithRoute(path, endpoint)` unless an existing route already covers that endpoint.
4. Idempotent — re-running produces no duplicate routes. User-authored routes always win; conflicts on path are resolved by skipping with a warning.

## Footguns this removes

* **Wrong CIDRs** — AKS default service CIDR is `10.0.0.0/16`; defaults pick `10.100.0.0/16` to avoid collision.
* **Missing AGC subnet delegation** — `AddLoadBalancer` already handles it but doing it for the user means they never have to think about it.
* **Issuer ↔ gateway wiring drift** — names are set in one place, can't drift.
* **Forgetting `WithLoadBalancer` on the gateway** — gateway is created already bound to the auto LB.
* **ACME staging vs prod confusion** — explicit enum, default documented.
* **Plain-HTTP 404 surprise** (the symptom that motivated #17158) — the auto gateway ships the 301 redirect on :80 and HSTS on :443 by default, so `curl http://...` returns a sensible response from the very first deploy.

## Example: full magic vs full control

Today (verbose, from `playground/CertManagerDemo/AppHost.cs`):

```csharp
var vnet = builder.AddAzureVirtualNetwork("vnet", "10.100.0.0/16");
var aksSubnet = vnet.AddSubnet("aks-nodes", "10.100.0.0/22");
var publicSubnet = vnet.AddSubnet("alb-public", "10.100.4.0/24");

var aks = builder.AddAzureKubernetesEnvironment("aks")
.WithSubnet(aksSubnet)
.WithSystemNodePool("Standard_D2as_v5");

var publicLb = aks.AddLoadBalancer("public", publicSubnet);
var acmeEmail = builder.AddParameter("acme-email");
var certManager = aks.AddCertManager("cert-manager");
var letsEncrypt = certManager.AddIssuer("letsencrypt-prod")
.WithLetsEncryptProduction(acmeEmail)
.WithHttp01Solver();

var api = builder.AddProject("api").WithExternalHttpEndpoints();

aks.AddGateway("public-gw")
.WithLoadBalancer(publicLb)
.WithRoute("/api", api.GetEndpoint("http"))
.WithTls(letsEncrypt);
```

After:

```csharp
var acmeEmail = builder.AddParameter("acme-email");

var aks = builder.AddAzureKubernetesEnvironment("aks")
.WithClusterDefaults(acmeEmail);

builder.AddProject("api").WithExternalHttpEndpoints();
```

Power users with a hybrid setup get the escape hatch:

```csharp
aks.WithClusterDefaults(acmeEmail, o =>
{
o.AddressSpace = "10.50.0.0/16";
o.AcmeEnvironment = LetsEncryptEnvironment.Staging;
o.AutoRouteExternalEndpoints = false; // keep infra, write routes by hand
o.ConfigureTls = tls =>
{
tls.Hsts.IncludeSubDomains = true;
tls.Hsts.Preload = true;
};
});
```

## File layout

* `src/Aspire.Hosting.Azure.Kubernetes/AzureKubernetesClusterDefaultsExtensions.cs` — new, ~250 lines
* `src/Aspire.Hosting.Azure.Kubernetes/ClusterDefaultsOptions.cs` — new, ~80 lines
* `src/Aspire.Hosting.Azure.Kubernetes/LetsEncryptEnvironment.cs` — new, enum
* `src/Aspire.Hosting.Azure.Kubernetes/AzureKubernetesEnvironmentResource.cs` — small additions to expose the auto-created LB/gateway via `internal` properties consumed by the auto-route step
* New `ASPIREAZURE0xx` experimental diagnostic id
* `playground/AksDefaultsDemo/` — minimal sample showing the 3-line variant
* `tests/Aspire.Hosting.Azure.Kubernetes.Tests/ClusterDefaultsTests.cs` — covers: bare call, override paths, auto-route gap-fill, idempotence, user-route-wins, `AutoRouteExternalEndpoints = false`, TLS opt-outs.

## Open questions

1. **Naming**: `WithClusterDefaults` vs `WithDefaults` vs `WithIngressDefaults`. `WithClusterDefaults` matches scope (cluster-wide, not just ingress).
2. **Multi-endpoint resources**: a resource exposing 2+ external endpoints — keep `/{name}` and let the user disambiguate, or auto-bucket to `/{name}-{endpoint}`?
3. **Non-Azure mirror**: should `KubernetesEnvironmentResource` get an analogous `WithClusterDefaults`? Probably yes, but a separate PR — Azure has Bicep-backed vnet/LB primitives that bare k8s does not.

## Related

* #17158 — `WithTls should redirect HTTP→HTTPS by default (port 80 is only for ACME)`. Hard prerequisite; the auto-gateway in `WithClusterDefaults` calls `WithTls(issuer)` and inherits its options bag.

cc @mitchdenny

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.