Cache anonymous public traffic safely at CloudFront

Outcome

Enable positive-TTL CloudFront caching only for explicitly classified anonymous public GET/HEAD responses, while keeping every authenticated, credential-bearing, personalized, mutation, preview, management, learner, registration, health, and operational request fail-closed and uncacheable.

The same edge boundary forwards a trustworthy optional country suggestion to #108, blocks broad abuse before Django, and gives the owner a predictable cost choice backed by current traffic and exact feature eligibility.

Product, architecture, and deployed-source authority

• 01 — Platform request/content activation flows
• 02 — URL, query, canonical, and preview behavior
• 03 — Atomic content activation
• 06 — Private Studio/admin API responses
• 07 — Web security, rate limits, privacy, observability, and targets
• 08 — CloudFront, protected ALB origin, noindex, Terraform portability, and cost
• 10 — Infrastructure/deployment/security verification
• Current accepted aws-infra module/root from #67–#70.

Current trusted source facts:

• the distribution has one default behavior with min/default/max TTL 0;
• it forwards all viewer headers, cookies, and query strings to Django;
• it has an optional web_acl_id input but the website module does not create the required ACL;
• CloudFront reaches the ALB over HTTPS/TLS 1.2;
• ALB ingress is limited to the inventoried AWS CloudFront origin-facing IPv4/IPv6 prefix lists;
• the ALB forwards only when both the public Host and generated X-Origin-Verify value match;
• development X-Robots-Tag noindex, nofollow is forced at the edge and remains required;
• application private/no-store responses and the #36 zero-TTL source contract are already tested.

This issue replaces the temporary all-dynamic cache posture; it must update #36 tests rather than weakening noindex or private-response guarantees.

Route classification is a code-owned contract

Create one versioned route-cache registry consumed by Django tests, cache-header middleware, deployed smoke, and Terraform/policy assertions. Every route is private/disabled unless explicitly classified. Adding a route without a classification fails CI.

Route class	Initial examples	Methods	Edge TTL / stale	Browser policy	Cache key
Fingerprinted static	/static/ versioned filenames	GET, HEAD	365 days; no stale error object	public, max-age=31536000, immutable	normalized path + gzip/brotli only; no cookies/query/viewer headers
Stable release asset	code-owned active-release asset paths that are not fingerprinted	GET, HEAD	24 hours; invalidated on activation	public, max-age=3600	path + encoding; no cookies/query/viewer headers
Editorial detail	approved article, podcast, person, book, docs, FAQ, wiki detail routes	GET, HEAD	600 seconds; stale-if-error up to 24 hours	max-age=0, must-revalidate; ETag/Last-Modified	canonical path + encoding; no query
Public hub/feed/sitemap	approved hubs, feeds, sitemap and explicit public JSON feeds	GET, HEAD	300 seconds; stale-if-error up to 1 hour	max-age=0, must-revalidate	canonical path; only exact page when registry allows pagination
Public course/event catalog/detail	anonymous-stable catalog/detail pages only	GET, HEAD	60 seconds; stale-if-error at most 5 minutes	max-age=0, must-revalidate	canonical path; exact allowlisted pagination only
Code-owned permanent redirect	explicit public alias/redirect manifest only	GET, HEAD	24 hours	public, max-age=300	normalized source path; query handled by redirect contract
Public 404	clean, credential-free, query-free unknown GET/HEAD	GET, HEAD	30 seconds; no stale-if-error	max-age=0	normalized path
Search/arbitrary query	search, filters not listed above	GET, HEAD	disabled in MVP	private/no-store	none
Private/dynamic	/accounts/, /studio/, /admin/, /cadmin/, learner/dashboard/enrollment/submission routes, onboarding/Slack, course/event registration forms or management links, preview, exports, authenticated/private API	all	disabled, including error caching	private, no-store	none
Operational	/health/, readiness, metrics, webhook/callback, job/provider endpoints	all	disabled	private, no-store or explicit operational equivalent	none
Unsafe/error	POST/PUT/PATCH/DELETE, 400/401/403/405/409/429/5xx, any response with Set-Cookie, private, no-store, Vary: *, CSRF, identity, PII, or capability state	applicable	disabled	no-store	none

Exact route names and path patterns are generated/reviewed from the actual URL registry; the examples above do not authorize an unlisted route. Public compatibility APIs remain disabled unless their owning contract explicitly proves anonymous, stable, bounded JSON.

All cache policies use min_ttl = 0 so an origin/edge no-store decision wins. Error-cache TTL is zero except the exact public 404 class. CloudFront never caches an origin error merely because an older successful representation exists. Stale-if-error is limited to already cached, anonymous public content and never applies to time-sensitive registration state or any private route.

Anonymous cache gate and fail-closed segmentation

A response is eligible for shared caching only when every condition is true:

• request method is GET or HEAD;
• the route registry class is public-cacheable;
• the viewer classifier proves the request has no Authorization, session/auth/preview/management credential, signed URL/cookie, CSRF-bearing cookie, or unknown/malformed credential-like state;
• Django renders an anonymous-stable representation and explicitly returns public plus the class s-maxage;
• response has no Set-Cookie, private, no-store, Vary: *, CSRF token, account-sensitive navigation/data, PII, or staff/learner state;
• status is allowed by the matrix.

Use a deterministic, versioned viewer-request edge function without a key-value store to remove any viewer-supplied internal classification header and set an internal anonymous-v1 or private marker. Any malformed cookie/header/query encoding, unknown credential-shaped cookie, Authorization syntax, preview/management token, or classifier failure becomes private.

The marker participates in the public-HTML cache key and is forwarded to Django. Explicit private path behaviors have the zero-TTL policy regardless of origin headers. On a mixed public path, credential-bearing requests go to origin and an edge origin-response guard forces private, no-store before cache storage, even if application code regresses. If this cannot be proved with the selected CloudFront function/Lambda@Edge and policy ordering, that route remains zero-TTL; security is not weakened to create a HIT.

Django remains authoritative too: every authenticated request and explicit private route returns private, no-store. Public templates must not place account-sensitive navigation/data in the anonymous cacheable representation. If a route cannot render one anonymous-stable representation without setting a cookie, it stays disabled in MVP.

Query, header, cookie, and poisoning policy

• Fingerprinted/static/detail/feed/sitemap keys contain no query parameter.
• A registry entry may allow one canonical positive-integer page parameter for a named hub. Duplicate, empty, overlong, out-of-range, invalidly encoded, or unexpected parameters make the response no-store or a safe 400; they never create cache variants.
• Known tracking keys may be stripped by a single safe canonical GET/HEAD redirect. They are never reflected into HTML or forwarded while omitted from the key.
• Search text and arbitrary filters are not cached in MVP.
• Host, User-Agent, Referer, Accept-Language, CloudFront country, X-Forwarded-*, viewer-supplied internal headers, and arbitrary cookies are not cache-key inputs.
• Accept-Encoding uses CloudFront gzip/brotli normalization, not raw attacker-controlled variants.
• Origin requests receive only the per-behavior header/cookie/query allowlist. The all-viewer policy is removed from cacheable behaviors and retained only where a private route truly needs that request context.
• Duplicate headers/query keys, alternate Host, path normalization, encoded separators, and poisoning canaries have negative tests.

Trusted country suggestion

Only onboarding/profile routes that need it receive CloudFront-Viewer-Country; those routes remain zero-TTL.

• CloudFront removes a viewer-supplied lookalike and adds its own geolocation header through the origin request policy.
• The existing CloudFront-prefix-list plus host plus generated origin-verification boundary remains intact. No implementation or evidence reads, prints, reconstructs, or returns the generated origin secret.
• Django enables trust only in the configured CloudFront deployment, accepts exactly a known uppercase ISO alpha-2 code, treats CloudFront special/unknown codes as missing, and exposes a request-scoped suggestion rather than a confirmed value.
• Country is not in any public cache key. Raw IP/header values are not retained for #108.
• Local/direct-origin tests default to no suggestion. A trusted test seam uses a boolean fixture, never a copied origin secret.

Invalidation and freshness

Network side effects occur after commit through a durable, idempotent invalidation job.

Content activation:

1. Candidate validation computes the changed public paths plus dependent hubs, redirects, search pages, sitemap, feeds, and stable asset paths before activation.
2. The activation transaction stores that route manifest and a unique invalidation intent keyed by distribution plus content release.
3. After commit, a worker submits/coalesces invalidations, records the provider ID/state, retries bounded transient failures, and alerts on terminal failure.
4. The first release may use exactly one coalesced /* invalidation for each activated release. Route/tag optimization is follow-up; it cannot reduce correctness.
5. A failed invalidation cannot roll back the already atomic content pointer. Bounded TTL keeps the old safe representation for no longer than its class TTL; failure is visible and never leaves indefinite mixed state.

Application deployment:

• fingerprinted assets require no invalidation;
• after the new web revision is ready and before release finalization, submit at most one idempotent /* invalidation keyed by exact application SHA and wait for Completed within the documented bound;
• failure fails finalization/alerts and keeps or restores a known-good application revision; rerun reuses the same logical intent;
• rollback invalidates again under the rollback release identity so templates/routes cannot remain mixed.

Invalidation paths contain public paths only, never query secrets, email, profile data, management links, or preview tokens. Operations expose safe counts/state, not raw sensitive URLs.

WAF and application abuse controls

Attach one Terraform-managed CloudFront-scope web ACL. No console-only rule, emergency edit, pricing subscription, or advanced paid product is allowed.

Initial rules:

• AWS managed common web-threat, known-bad-input, and IP-reputation protections selected within the chosen plan/rule quota;
• malformed/oversize request and disallowed-method/path protections;
• rate-based rules with these count-mode starting thresholds per source IP over five minutes:
  • ordinary cacheable public GET/HEAD: 2,000;
  • search, unknown-query, or other origin-bound anonymous reads: 300;
  • /api/ reads: 300;
  • signup/login/profile/Slack/course/event registration paths: 60;
• a separately bounded emergency block/rate rule controlled through reviewed Terraform input.

Run managed and rate rules in count mode for at least seven representative days in development, record aggregate matches/false positives, then enable blocking only for reviewed rules. Known exploit/IP-reputation matches may move sooner only with deterministic fixtures and no legitimate-user regression. Blocked/rate-limited requests never reach ALB/Django/ECS/database/email and return a safe non-cacheable response.

Application services retain stricter business limits: login/verification/resend/profile/registration by normalized identity plus safe IP class, and admin API by principal/capability. Edge IP limits are not account authorization and do not solve distributed botnets alone.

Do not trust User-Agent as crawler identity and do not create an unconditional verified-bot bypass. Production SEO crawlers may operate under ordinary public limits. robots.txt expresses crawl preference but is not enforcement. Use plan-provided common/self-identifying bot analytics or controls only when the selected tier supports them; targeted/advanced bot, fraud, account-takeover, CAPTCHA, or challenge products are not enabled in MVP. High-rate/cache-busting/known-reputation traffic is blocked through the baseline rules; distributed anomalies trigger alarms and the reviewed emergency rule.

Cost-plan decision and alarms

Use the cheapest option that satisfies the exact accepted distribution, cache, WAF, logging, Terraform, and bot-control contract. Do not reduce security, observability, or cache correctness to fit a pricing tier.

Before any plan subscription or apply, record a redacted read-only comparison using current AWS documentation and the latest 30 days of workload-only metrics (or the available shorter window plus explicit projection):

• request count, transfer, cache hit/miss, WAF evaluated/blocked, log ingestion, invalidation, edge-function/Lambda@Edge, ALB origin transfer/request, ECS, RDS, and residual service cost;
• normal, 10x viral, cache-busting agent, and distributed-bot scenarios;
• exact distribution/account eligibility, number of behaviors/WAF rules, function associations, standard versus real-time logging, common versus targeted bot features, and any unsupported association;
• pay-as-you-go total and the Free, Pro, Business, and Premium flat-rate features/allowances then current.

Safe selection rule:

1. Free is eligible only if all required logging, WAF rules, cache behaviors/policies, and usage fit; no required evidence may be dropped.
2. Prefer Pro when it is the cheapest flat tier whose real subscription eligibility check accepts the exact candidate and whose allowances cover the forecast with headroom.
3. If Pro rejects a required configuration, compare pay-as-you-go with Business using measured/projection totals; choose the cheaper sufficient option. Business is not selected merely to obtain advanced bots that are out of MVP.
4. Premium/advanced products require a new owner-approved cost issue.
5. If flat-plan lifecycle cannot be managed reproducibly through accepted infrastructure automation, retain pay-as-you-go until it can; do not make a console-only subscription.

At grooming time, current AWS documentation lists Free at $0/month, Pro at $15/month with 10M requests/50TB and 25 WAF rules, Business at $200/month, and no overage charges for flat plans; standard logging begins at Pro and targeted bot controls begin at Business. Recheck at implementation because pricing/features can change. AWS says requests blocked by WAF/DDoS do not count against flat-plan allowances; residual ALB/ECS/RDS, non-included edge compute, and unrelated services still require alarms.

Set workload budget/anomaly and traffic alarms with named owner/runbook for:

• 50%, 80%, and 100% of the selected allowance or forecast;
• cache hit ratio below 70% for cacheable route classes after warm-up;
• origin-request rate above twice the reviewed normal peak for 15 minutes;
• WAF block/rate anomaly, 4xx/5xx, invalidation failure/age, and edge-function error;
• ALB/ECS/RDS cost or load rising despite WAF/cache controls.

The emergency action is a reviewed Terraform rate/block toggle or cache disable/TTL-zero rollback. No alarm action broadens caching or exposes the origin.

Current official evidence to recheck:

• CloudFront flat-rate plan features, allowances, covered costs, and unsupported features
• CloudFront pricing comparison

Observability and privacy

Expose aggregate route class, viewer class, cache status, age bucket, invalidation state/latency, WAF rule label/action, origin-request rate, bytes, status, and function errors. Use bounded labels.

Do not log raw Cookie, Authorization, session/CSRF value, complete query, IP, country suggestion, origin-verification value, preview/management token, Slack link, profile field, or response body. Standard CloudFront/WAF logs use encrypted storage, bounded retention, least privilege, and redaction/field omission. Real-time logs are not required.

Development remains noindex, nofollow on HIT, MISS, redirect, error, asset, and WAF denial. Positive caching never changes production canonicals or makes a preview indexable.

Non-goals

• No caching of authenticated/private/personalized/registration/management/search responses, unsafe methods, or arbitrary query variants.
• No public origin, removal/readback of the generated origin guard, VPC/topology redesign, Route 53 zone/delegation change, production account mutation, or unrelated infrastructure import.
• No advanced/targeted Bot Control, fraud/account-takeover product, CAPTCHA, challenge, real-time logs, Origin Shield, multi-origin failover, or guaranteed botnet elimination in MVP.
• No SEO/indexing change, analytics/tracking addition, public-member country personalization, application microservice, or Redis cache.
• No content activation feature beyond the invalidation contract owned with its source issue.

Acceptance criteria

• One generated route-cache registry classifies every current route and CI fails on an unclassified route or a mismatch between Django, Terraform behavior, and deployed smoke expectations.
• The exact matrix above is implemented with min_ttl 0; representative anonymous public GET/HEAD reaches MISS then HIT/Age, while search/private/unsafe/error classes never store.
• Viewer classification removes spoofed internal headers, treats malformed/unknown credential state as private, and prevents authenticated/private responses from entering or being served from shared cache even after a warmed anonymous response.
• Application and edge both enforce private/no-store for all explicit private routes and credentialed mixed-route requests; Set-Cookie/CSRF/PII/identity/Vary:* canaries are never cached.
• Cache/origin request keys contain only the exact allowlists; Host/header/cookie/query duplication, encoding, normalization, tracking, unknown parameter, and cache-poisoning variants fail safely without unbounded misses.
• CloudFront country reaches only the zero-TTL consumer routes, cannot be viewer-forged or direct-origin supplied, validates to an optional suggestion, and stores/logs no raw IP/header.
• Content and deployment invalidation intents are durable, idempotent, coalesced, observable, secret-free, bounded, and follow the exact activation/deploy/rollback failure semantics.
• Terraform creates/attaches the reviewed WAF ACL and rules; count-mode evidence precedes block mode; blocked traffic is proven not to reach ALB/application/database/email work.
• WAF plus application limits cover ordinary content, search/query abuse, login/signup/Slack/registration, APIs, cache-busting agents, known-bad/IP reputation, and an explicit distributed-bot emergency path without User-Agent-only trust.
• A current, redacted cost/eligibility report applies the cheapest-sufficient selection rule to pay-as-you-go and every eligible flat tier; selected plan/configuration is reproducible, budgeted, and contains no silent advanced product.
• Cache/WAF/invalidation/allowance/origin alarms and runbooks use bounded non-PII metrics and a tested TTL-zero/emergency-rule rollback.
• Development noindex/no-follow, production canonical behavior, TLS, exact hosted-zone ownership, CloudFront-only origin lock, and infrastructure portability remain unchanged.
• Website and aws-infra Terraform/policy/unit/integration/security/migration/OpenAPI/full-Django/core-Playwright gates pass through repository-standard commands; no state/plan/credential/secret is committed.
• [HUMAN] After authorized deployment, redacted live evidence proves the full anonymous MISS->HIT/private bypass matrix, country suggestion, invalidation, WAF block-before-origin, selected plan/allowance alarms, noindex, and TTL-zero rollback without obtaining or exposing protected values.

Automated and fault scenarios

1. For every route class: GET/HEAD/unsafe method, empty/allowed/unknown/duplicate/malformed query, gzip/brotli, 200/redirect/404/4xx/429/5xx, Set-Cookie, ETag/304, expiry, stale origin, and invalid origin Cache-Control.
2. Warm anonymous cache, then request as two sessions, Authorization principals, CSRF-only cookie, preview/management token, malformed cookie/header, and anonymous again. Assert body/header/cache key/age never crosses a private boundary.
3. Poisoning matrix for Host, forwarded/internal/country headers, case/duplicate headers, encoded path separators, cache-buster queries, tracking redirect, Vary, conflicting cache directives, and compression variants.
4. Content activation changes detail/hub/feed/sitemap/redirect/stable asset; duplicate and concurrent activation; invalidation throttle/timeout/terminal failure; deploy, finalization failure, rollback, and job replay.
5. Country present/missing/unknown/lowercase/forged, direct origin, local/test mode, and #108 visible confirmation.
6. WAF fixtures for common exploit, bad input, known reputation, ordinary burst, cache-buster, dynamic-route rate, distributed sources, legitimate NAT/shared IP, accessibility client, and emergency toggle. Prove count/block results and no origin work.
7. Cost model for actual baseline, normal, 10x, cache-busting, WAF-blocked attack, and distributed bot; exercise flat eligibility failure and deterministic fallback without changing configuration/security.
8. Terraform negative fixtures reject zero/positive TTL drift, private behavior using public policy, all-viewer forwarding on cacheable routes, absent WAF/origin lock, excessive rule/behavior quota, unreviewed paid feature, console-only state, missing alarms, and production/sandbox coupling.
9. Kill/timeout edge function, invalidation worker, Django, and origin; prove classifier fails private, stale is bounded by class, alarms fire, and TTL-zero rollback restores safe dynamic service.

Browser and deployed acceptance

At desktop about 1440x900 and mobile about 390x844 using only synthetic accounts:

1. Load approved public editorial and catalog pages twice; inspect X-Cache/Age, body/canonical/noindex, assets, responsive layout, keyboard behavior, and non-error screenshots.
2. Sign in after warming those URLs; revisit as two accounts and via back/refresh/logout. No member state, navigation, CSRF, profile, enrollment, or Studio capability crosses cache.
3. Visit login, profile/Slack, registration form, learner dashboard, Studio, Django admin, admin API denial, health, search/query, preview fixture, deliberate 404, and 5xx fixture; each matches the matrix.
4. Exercise an editable country suggestion without placing it or account data in a cache key, URL, screenshot, or another account response.
5. Activate synthetic content and deploy/roll back a synthetic revision in the authorized environment; verify invalidation and bounded freshness.
6. Trigger only safe synthetic WAF count/block fixtures; verify denial is readable/non-cacheable and origin metrics show no corresponding work.
7. Capture no auth/token/secret/PII values. Store screenshots only under the project-local .tmp/screenshots/issue-109/ and inspect every image.

Required specification amendments

Implementation must update these exact authority files:

• _docs/specs/01-platform-architecture.md — anonymous cache gate and durable invalidation flow;
• _docs/specs/02-url-link-seo-compatibility.md — route/query/redirect/error cache and canonical interaction;
• _docs/specs/03-github-content-and-people.md — activation route manifest, invalidation, and bounded stale behavior;
• _docs/specs/04-courses-and-cohorts.md — public catalog/detail versus learner/registration cache classes;
• _docs/specs/05-events-registration-email.md — public event versus registration/management/private cache and rate limits;
• _docs/specs/06-studio-and-admin-api.md — explicit edge zero-TTL/private behavior for Studio/admin API and credentialed mixed routes;
• _docs/specs/07-security-privacy-operations.md — classifier trust, poisoning, WAF/bot/cost/observability/rollback rules;
• _docs/specs/08-aws-sandbox-terraform.md — cache/origin-request policies, edge function, country forwarding, WAF, plan selection, alarms, and portability;
• _docs/specs/09-migration-rollout-roadmap.md — deployment/content invalidation ordering and rollback;
• _docs/specs/10-verification-strategy.md — local/Terraform/deployed cache, WAF, cost, and failure matrix;
• _docs/specs/open-decisions.md — record the cheapest-sufficient pricing rule and deferred advanced bot products.

Update #36 source/deployed cache assertions from blanket zero TTL to the generated public/private matrix while preserving every noindex/private/canonical/analytics guard.

Dependencies and delivery

• Accepted #67–#70 infrastructure and #36 application/edge safety are the baseline.
• #78 is a hard prerequisite for any repeated sandbox Terraform plan/apply; #94 state/OIDC migration must be coordinated so #109 targets the accepted current root/state identity. Source work and local policy tests may be prepared without AWS mutation.
• #39/content activation consumes the invalidation service. The initial bounded wildcard remains correct if the optimized route manifest lands later.
• #66 owns the broader observability platform; this issue owns the minimum cache/WAF/invalidation/cost alarms required to operate its controls.
• #108 consumes the country suggestion but neither issue blocks the others functional path.
• Production pricing/account/domain remain separate; the development selection is not silently promoted.

This is a cross-repository website plus aws-infra change. Each repository follows the independent engineer/tester/PM lifecycle. No pull request. Live cache/WAF/pricing verification is HUMAN, so commits use Refs #109, retain/add human, and the issue remains open until the deployed criterion passes.

Start with the linked architecture specs, especially 08 — AWS sandbox Terraform and 10 — infrastructure/deployment/security verification, then inspect the current aws-infra module/root from #67–#70. Review the #36 zero-TTL tests and the actual URL registry before defining the route-cache contract. Done means the registry, CloudFront/WAF Terraform, invalidation flow, Django safeguards, and verification evidence satisfy the listed fail-closed requirements without weakening private or noindex behavior.