aws-samples / aws-samples/sample-autonomous-cloud-coding-agents
CRITICAL: escape the CloudFormation 500-resource ceiling β best-practice strategies (8 resources of headroom on main)
- Dominant language
- TypeScript
- Stars
- 143
- Forks
- 46
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 20
Description
> **Severity: CRITICAL / VITAL.** This is an active, deterministic build-and-deploy blocker, not a warning. ~~`main` has **8 of 500** CloudFormation resources of headroom on `compute_type=ecs`. The next feature that adds an authenticated route fails CI *and* cannot be deployed.~~ **`main` cannot synthesize at all** on `compute_type=lambda-microvm -c enableToolGateway=true` (**504/500**, throws). #681 already has.
> **π Edited 2026-09-03 to correct published errors.** Corrections are marked with ~~strikethrough~~ on the original claim followed by a bold **Correction:** note, so the record of what was wrong stays visible rather than being quietly rewritten. Several were my own arithmetic on unexamined premises (Β§3a, Β§3c, Β§5); the variant analysis was corrected by @theagenticguy in the comments and I re-verified the decisive cell myself. A consolidated list is in Β§9.
This issue is the **strategy and remediation surface** for the 500-resource-per-stack ceiling. #851 is the incident record (measurement, attribution, the failing run). This issue supersedes #851's provisional "Proposed remediation" section and is where options get iterated.
Everything below is **measured from a real synth** of `main` @ `521bf647` (CDK CLI 2.1129.0, `aws-cdk-lib` 2.261.0), not estimated. Reproduce commands are at the bottom. Re-verification noted where it happened at `81e71880`.
---
## 1. The constraint
CloudFormation allows **500 resources per stack**, ~~enforced at changeset creation~~. **Correction:** enforced *earlier* than that β CDK **throws at synth**, before CloudFormation is consulted (`MAX_RESOURCES = 500` in `aws-cdk-lib/core/lib/stack.js`: `if (numberOfResources > this.maxResources) throw ValidationError('TooManyResourcesInStack')`, with an `addInfo` band at 400β500). It is not a test-only limit: a real deploy of an over-budget branch is rejected before any resource is created.
~~| Context | Resources | Headroom |~~
~~|---|---|---|~~
~~| default (`agentcore`) | 480 | 20 |~~
~~| **`compute_type=ecs`** | **492** | **8** |~~
~~| `compute_type=ecs` + #681 | 501 | **over** |~~
**Correction β ECS is not the worst variant, and the worst cell already fails on `main` today.** `compute_type` is not the only additive deploy gate; `enableToolGateway` (ADR-019 P1, `agent.ts:313`) is a second orthogonal one and the two compose, so the gate space is a **product**. Table from @theagenticguy ([comment](https://github.com/aws-samples/sample-autonomous-cloud-coding-agents/issues/852#issuecomment-5521712485)), whose decisive cell I re-verified independently at `81e71880`:
| Context | Resources | Headroom |
|---|---:|---:|
| `agentcore` (default) | 480 | 20 |
| `agentcore` + `enableToolGateway=true` | 487 | 13 |
| `compute_type=ecs` | 492 | 8 β *this census's original "worst case"* |
| `compute_type=lambda-microvm` | 497 | 3 |
| `lambda-microvm` + image configured | 498 | 2 |
| `ecs` + `enableToolGateway=true` | 499 | 1 |
| **`lambda-microvm` + `enableToolGateway=true`** | **504** | **throws** β
re-verified |
| **`lambda-microvm` + `enableToolGateway=true` + image** | **505** | **throws** |
`enableToolGateway` costs a flat **+7** on all three substrates (480β487, 492β499, 497β504). The 492 this issue was built on is the **third-best of eight cells**; four are worse and two cannot synthesize.
Two properties make this expensive:
- **The signal is `INFO`.** CDK emits `Number of resources: 492 is approaching allowed maximum of 500` and fails nothing. The first hard failure lands on an *unrelated* PR, and the error names the **stack**, not the change that consumed the headroom β so authors reasonably conclude their own diff is broken.
- ~~**Variants differ by 12.** The default context is 480; ECS is 492. A guard that only synthesizes the default context would not have caught this.~~ **Correction: variants differ by 25** (480 β 505), across a **product of two gates**. A guard sweeping only `compute_type` reports 480/492/497 β all green β on a tree where `cdk synth` throws.
~~Whether the 500 limit is adjustable via Service Quotas is **unverified** β I could not query it (`servicequotas:ListServiceQuotas` denied for the available role). Historically it has been a hard, non-adjustable limit. Someone with quota-read access should confirm and close this question, because a "just raise it" answer would change the whole plan. **I am assuming it is hard.**~~
**Correction β resolved, it is hard.** AWS *Understand CloudFormation quotas* lists **Resources β 500** whose remedy column offers only a *workaround* ("separate your template into multiple templates by using, for example, nested stacks"), not a quota-increase path β unlike the adjustable quotas in the same table. Raised to 500 (and template body to 1 MB) on **2020-10-22**, globally, not per-account. Independently decisive: because CDK throws at **synth**, no account-level quota could rescue it. **This open question is closed.**
## 2. Where the 492 resources actually are
By top-level construct (`compute_type=ecs`):
| Construct | Resources | Share |
|---|---:|---:|
| **`TaskApi`** | **239** | **49%** |
| `AgentVpc` | 43 | 9% |
| `SlackIntegration` | 27 | 5% |
| `GitHubScreenshotIntegration` | 17 | 3% |
| `LinearIntegration` | 15 | 3% |
| `JiraIntegration` | 15 | 3% |
| `DnsFirewall` | 10 | 2% |
| everything else (~30 constructs) | 126 | 26% |
`TaskApi` alone is half the stack. Inside it:
| Type | Count |
|---|---:|
| `AWS::ApiGateway::Method` | 64 |
| `AWS::Lambda::Permission` | **62** |
| `AWS::ApiGateway::Resource` | 33 |
| `AWS::IAM::Role` | 23 |
| `AWS::Lambda::Function` | 22 |
| `AWS::IAM::Policy` | 22 |
| `AWS::ApiGateway::Authorizer` | 3 |
| singletons (`RestApi`, `Stage`, `Deployment`, `WebACL`, Cognito, β¦) | ~9 |
Across the whole parent stack: 49 Lambda functions, 68 Lambda permissions, and **112 IAM resources (57 roles + 55 policies) = 22% of the stack**.
~~Note the shape: **64 methods but only 22 functions**, and **62 permissions for those 22 functions**. The permissions β not the functions β are the anomaly, and that is where the cheap headroom is.~~
**Correction:** the permissions *are* the cheap headroom, but not for the stated reason. The 62 do not represent many methods sharing few functions β see Β§3a. They are **31 method/function pairs each carrying two permissions**, one real-stage and one test-invoke.
## 3. Tier 1 β mechanical, measured, no architecture change
`LambdaIntegration` in `aws-cdk-lib` 2.261.0 has two props that directly control permission fan-out. Both are verified against the installed source (`aws-apigateway/lib/integrations/lambda.js`), and **neither is set anywhere in `cdk/src/`** β so both are at their defaults across all 34 `LambdaIntegration` call sites.
### 3a. `scopePermissionToMethod: false` β ~~est. **β40** resources~~ measured **β30**, and **rejected**
Default `true`. With `true`, CDK calls `handler.addPermission()` **per method**, with `sourceArn: method.methodArn`. With `false`, it emits **one api-scoped permission per (function, API) pair**, keyed `ApiPermission.ApiScoped..` and deduplicated via a `node.findAll()` scan, using `method.api.arnForExecuteApi()`.
~~22 functions serving 64 methods β **62 permissions collapse to ~22**. Stack β **452**, headroom β **48** (6Γ current).~~
**Correction β the collapse premise was false.** The 32 real-stage permissions map to **32 distinct functions**. There is **zero** function sharing, so nothing collapses. Synthesized result is **462 / β30**, *identical* to 3b, because the mechanism is different than assumed: the `else` branch removes all 60 permissions and adds back 30 wildcards. Confirmed by synth, not inferred.
**Trade-off, stated plainly:** the source ARN widens from one method ARN to the whole API (all stages, methods, paths):
```
default: .../execute-api:...:{TaskApi}/{StageV1}/POST/tasks
3a: .../execute-api:...:{TaskApi}/*/*/*
```
~~The principal stays `apigateway.amazonaws.com` and the API is still pinned, so exploiting the difference requires API Gateway **control-plane** access to add or repoint a route β a compromise that already dwarfs this grant. This prop exists in `aws-cdk-lib` precisely as the sanctioned escape hatch for large APIs. It is a real reduction in least-privilege granularity and should be a **conscious, documented** decision, not a silent one.~~
**Correction β this is now a straight rejection, not a trade.** Since the measured saving is *identical* to 3b's (β30 parent / β34 app-wide), there is nothing to trade *for*. Same benefit, strictly worse posture. **Reject 3a.**
Additionally: **4346 of 4346 tests pass under 3a.** There is no assertion anywhere in `cdk/test/` guarding API Gateway Lambda permission scoping, so this wildcard widening would ship green. That gap should be closed regardless of which option is taken.
**Caveat (unchanged and confirmed in source):** when `scopePermissionToMethod: false`, `allowTestInvoke` is **ignored** and CDK emits `@aws-cdk/aws-apigateway:allowTestInvoke` as a warning. The two options are **alternatives, not additive** β setting both still yields β30.
### 3b. `allowTestInvoke: false` β measured **β30** parent, **β34** app-wide
Default `true`, which adds a **second** permission per method scoped to `method.testMethodArn` (the pseudo `test-invoke-stage`) purely so the API Gateway console "Test" button works.
Measured in the current template: of the parent stack's 68 permissions, **38 are real-stage and 30 are `test-invoke-stage`**. The nested `RegistryApi` stack is 4 of 8. (@theagenticguy reproduced the 38/30 split exactly on both the `ecs` and `lambda-microvm` templates.)
Stack **492 β 462**, headroom **38**. **Correction/addition:** the app-wide saving is **β34**, not β30 β the nested `RegistryApi` drops 4 as well (552 β 518 across all three templates), one per call site, 1:1. Removed set is **exactly** 30 `AWS::Lambda::Permission` matching `ApiPermissionTest` and nothing else. **Test breakage: none β 204/204 suites, 4346/4346 tests, 1 snapshot.**
The only capability lost is console test-invoke. For a production API this is *also* a least-privilege improvement, so it is defensible on its own merits independent of the resource count β which makes it the lowest-risk item on this list.
**Recommendation:** take **3b** unconditionally (it is a security improvement that happens to pay 30 resources). ~~Treat **3a** as the larger, deliberate follow-on if 38 headroom is judged insufficient β but note 3a *replaces* 3b's saving rather than adding to it.~~ **Correction: reject 3a outright** (Β§3a) β it cannot be a "larger" follow-on because it is not larger.
### 3c. Fail loudly, per variant β 0 resources, prevents recurrence
~~Add a synth-time budget assertion that **fails** above a threshold (e.g. 460) and runs for **every `compute_type` variant**, naming the variant and the count.~~ The current `INFO` annotation is why 492/500 went unnoticed until it broke someone else's PR. #735 proposes the same idea for template **bytes**; one guard should cover both axes.
**Correction β three errors here:**
1. **No custom code is needed.** `this.maxResources` is fed by the context key **`@aws-cdk/core:stackResourceLimit`**, which is **not set anywhere** in `cdk/cdk.json` or `cdk/src/`. Setting it to a budget makes synth **fail** with `TooManyResourcesInStack` naming the count. Caveats: it is one global number applied to all stacks including nested; it *replaces* the threshold rather than adding a warning tier; and it must be set only after the count is below the chosen budget.
2. **"Every `compute_type` variant" is the wrong dimension.** It must enumerate the **product** of every context flag that adds resources β today `compute_type` Γ `enableToolGateway`. As written the guard would report 480/492/497, all under 500, and pass a tree in which `cdk synth` throws.
3. **The example budget 460 is unusable.** It is below ECS's own post-fix 462 and well below the worst cell's post-fix 475, so Tier 1 alone would not reach it and Tier 2 would become a *prerequisite* for the guard rather than a follow-on.
Note also that `cdk.json` currently has **no `context` block at all**, and `App.loadContext()` only merges `props.context` plus CLI-supplied sources β so a `cdk.json`-only fix would never reach the Jest tests, which call `new App()` directly (`test/stacks/agent.test.ts:37`, `:365`, `:967`).
### 3d. `suppressTemplateIndentation: true` β **β311 KB, 0 resources** *(added 2026-09-03; missing from the original)*
`StackProps.suppressTemplateIndentation?: boolean` (default `false`), or context key `@aws-cdk/core:suppressTemplateIndentation`, makes CDK emit compact JSON (`stack.js`: `const indent = this._suppressTemplateIndentation ? void 0 : 1`). **Neither is set in this repo.**
Measured: parent template on disk **996,129 bytes**; the same content compact is **685,146 bytes**. **~311 KB (31.2%) of the template is pretty-print indentation.** CDK's own warning string names this prop as the remedy. Costs nothing but template readability, and costs **zero** resources. Implementation site: `cdk/src/main.ts:88`.
### 3e. Drop `defaultCorsPreflightOptions` β **β34 resources, β39 KB** *(added 2026-09-03; missing from the original)*
`cdk/src/constructs/task-api.ts:328` sets `allowOrigins: Cors.ALL_ORIGINS, allowMethods: Cors.ALL_METHODS`. This generates **34 MOCK `OPTIONS` methods β 53% of the stack's 64 `ApiGateway::Method` resources** β each advertising **7 verbs** (`DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT`) on paths that implement one or two, from any origin.
β34 resources and β39,176 compact bytes. Unlike 3b/3d this **changes the API contract**, so it is a product decision: no browser client exists today (`integrations/jira-forge-app` only calls `requestJira`, Jira's own API), and preflight can be re-added per-resource rather than globally. *(This row is arithmetic on measured components, not yet a standalone synth.)*
### Tier 1 combined, measured
| Step | Resources (ECS) | Template bytes | Headroom |
|---|---:|---:|---:|
| today | 492/500 (98.4%) | 996,129 (**99.6%**) | 8 |
| + 3d `suppressTemplateIndentation` | 492 (98.4%) | 685,146 (68.5%) | 8 |
| + 3b `allowTestInvoke: false` | **462** (92.4%) | 662,700 (66.3%) | 38 |
| + 3e drop CORS preflight | **428** (85.6%) | 623,524 (62.4%) | **72** |
At the measured ~8 resources per authenticated endpoint, runway goes from **1 endpoint to 9**. For the **worst cell**, 3b alone takes 505 β **475** (headroom 25).
## 4. Tier 2 β the structural fix, chosen on lifecycle
Tier 1 buys 30β40 resources. That is one or two more features, not a fix. The structural question is *where the stack boundary belongs*, and the answer should come from CDK's own guidance, not from what this repo already happens to do.
The relevant principles:
- **"Model with constructs, deploy with stacks."** Constructs are for logical grouping; a stack is a **unit of deployment and rollback**. Splitting to dodge a limit is the wrong framing β split where the *deployment lifecycle* differs, and the limit stops binding as a side effect.
- **Separate stateful from stateless.** This repo has **21 DynamoDB tables, 7 Secrets Manager secrets, several buckets, and a Cognito user pool** in the same stack as 49 Lambdas and a REST API.
- **Align with blast radius and ownership.** `AgentVpc` (43) + `DnsFirewall` (10) is networking with a stable interface and a lifecycle wholly unlike the API's.
### Measured PoC results *(added 2026-09-03)*
Three seams were cut and synthesized. **Split overhead is near-free on the count axis; the danger is elsewhere.**
| Seam | Parent | App-wide total | Cross-stack coupling | Verdict |
|---|---:|---:|---|---|
| `AgentVpc` + `DnsFirewall` β top-level `NetworkStack` | 492 β **437** (β55) | 552 β **555** (+3) | 4 exports, 6 `Fn::ImportValue` | **VIABLE** |
| `JiraIntegration` β own stack (shared `RestApi`) | 492 β 465 (β27) | 552 β 550 (β2, *really* +1) | **18 exports, 39 `Fn::ImportValue`** | **REJECT** |
The `NetworkStack` **+3** is exactly a duplicated `AwsCustomResource` provider Lambda + its role (2) and a second `CDKMetadata` (1). `Custom::VpcRestrictDefaultSG` **moved rather than duplicated**. Dependency is one-way and verified acyclic; a deliberately probed reverse reference fails hard with `DependencyCycle` and writes zero templates. Honest costs: `AgentStackProps` gains **required** `vpc` + `runtimeSecurityGroup` (killing the `props = {}` default), which breaks **125 of 148 tests** in 4 of 5 stack suites; a **new cdk-nag ERROR** appears because *path-scoped suppressions do not follow resources into a new stack*; and `DnsFirewall`'s allowlist, aggregated from `Blueprint` constructs inside `AgentStack`, must move out.
### β οΈ Anything attached to the shared `RestApi` is NOT extractable
This is the most important finding of the PoC. The `JiraIntegration` split **failed closed twice** with `DependencyCycle` β the good outcome. The variant that *succeeds* is the trap. It synthesizes green, passes **121 of 126 tests**, reports a resource count **2 lower than baseline**, and ships broken:
| Silent regression | Evidence |
|---|---|
| **Jira routes never reach the deployed stage** | `Deployment.DependsOn` 98 β 90 entries; Jira entries **8 β 0**. Jira stack has **zero** `Deployment`/`Stage`. `manifest.json` deploys the parent **first**. |
| 3 CORS preflight methods silently vanish | An imported `IRestApi` does not carry `defaultCorsPreflightOptions` |
| Solution UA (#319) lost | **0 of 3** Jira Lambdas carry `AWS_SDK_UA_APP_ID`; `SolutionUaAspect` is applied to `stack` only |
| All cost/provenance tags lost | **0** occurrences of `compute_type` / `github:sha` in the Jira template (parent has 209) |
The count "improved" only because 3 resources disappeared; corrected, the split is **+1**. The cycle is a genuine **2-cycle** β `api.root.addResource()` makes `Resource`/`Method` children of the **RestApi construct**, so they land in the API's stack pointing at the other stack's Lambda, while the `Lambda::Permission` sits in the Lambda's stack pointing back at the API's execute-api ARN β so moving the `RestApi` to a third stack does not help. `api.latestDeployment.addToLogicalId()` is unavailable on an imported `IRestApi`, and CloudFormation has **no cross-stack `DependsOn`**, so the only cycle-free fix is a hand-computed route fingerprint passed as a string prop: a silent-failure trap by construction.
**Conclusion: keep the `RestApi` and all 64 of its methods in one stack.** Jira, Linear, Slack and GitHub-screenshot are **not** independently extractable.
### Mechanism: separate stacks vs nested stacks
| | Separate top-level stacks | Nested stacks |
|---|---|---|
| Independently deployable / rollback-able | **Yes** | No β deploy serializes through the parent |
| Counts against parent's 500 | No | Yes, but as **1** `AWS::CloudFormation::Stack` |
| Cross-stack wiring | CDK generates `Export`/`Fn::ImportValue` from direct construct refs | Parameters/outputs through the parent |
| Overhead (measured) | **+1** (`AWS::CDK::Metadata`) | **+2** (`CDKMetadata` + `AWS::CloudFormation::Stack`) |
| Main friction | A consumed export **cannot be changed** without a two-phase deploy | Templates staged to S3; parent/child rollback coupled |
| Best fit | Components with **independent lifecycles** | A child with **no** independent lifecycle |
Cross-stack references themselves cost **0** resources (they become `Outputs` / `Fn::ImportValue`). Per-stack singletons *do* duplicate: `Custom::S3AutoDeleteObjects` +2, `Custom::LogRetention` +3, `Custom::VpcRestrictDefaultSG` +2, `CDKMetadata` +1, for each stack needing them.
**This repo already uses nested stacks** β `AgentRegistryStack` and `RegistryApi` (#246), holding 20 and 40 resources. **That is precedent, not justification.**
### Hard prerequisite: moving resources without destroying them
**A naive move of a stateful resource to another stack deletes it.** With 21 tables, several buckets, and a Cognito user pool in scope, this is the highest-risk part of any split. **All 45 stateful resources currently sit at `DeletionPolicy: Delete`** β a standalone P0 that must be fixed before any move, independent of mechanism.
~~CDK CLI 2.1129.0 (already the pinned version) ships **`cdk refactor [STACKS..]` β "Moves resources between stacks or within the same stack"**, backed by CloudFormation stack refactoring, which relocates resources while **preserving physical identity**. This is the capability that makes a stateful split safe, and it did not exist when this stack was designed.~~
**Correction β this oversold `cdk refactor`.** It exists in CLI 2.1129.0 but is **UNSTABLE and opt-in**: `cdk refactor --dry-run` reports *"Unstable feature use: 'refactor' is unstable. It must be opted in via '--unstable', e.g. `cdk refactor --unstable=refactor`"*. Eligibility is **not** a static list: it is per-resource-type `provisioningType == FULLY_MUTABLE`, checkable via `describe-type`, validated at refactor creation, with ineligible types reported by `describe-stack-refactor`. **I could not verify eligibility for this stack's 24 resource types** (`cloudformation:DescribeType` AccessDenied for the available role). Useful flags: `--dry-run`, `--override-file`, `--revert` (only valid if a mapping file was provided), `--force`, `--additional-stack-name`.
Before committing to any Tier 2 plan, someone must **validate `cdk refactor` end-to-end against a throwaway deployment** with the 21 tables populated, and confirm zero replacement. If it does not hold up, the fallback is `RemovalPolicy.RETAIN` + resource import. **Do not begin a split on the assumption that this works.**
## 5. Rejected β do not do these
- **Consolidating IAM roles/policies to reclaim count.** 112 resources (22%) is tempting. **Reject.** Per-function least privilege is the best practice being traded away, the saving is one-time, and this repo has already taken a review finding for an over-broad task role (#596). Narrow exception: functions with *provably identical* access profiles, never merging read-only with write.
- **Collapsing routes behind `{proxy+}` (a Lambda-lith).** Relocates authorization from API Gateway authorizers into application code, weakening the Cedar/HITL boundary. **Reject** as a resource-count strategy.
- **`scopePermissionToMethod: false`** β see Β§3a. **Reject:** identical saving, wildcard `SourceArn`.
- **Extracting any integration that attaches routes to the shared `RestApi`** β see Β§4. **Reject:** silent Deployment/UA/tag loss.
- **Waiting for #735's split.** #735 (1 MB template bytes) is a *different* limit with a *different* trigger. The two should share a solution but must not share a schedule assumption. This reasoning holds β measured, **count** is driven by route fan-out (permissions + methods + apigw resources = **165 of 492, 34%**) while **bytes** are driven by IAM (Policy + Role = **230,546 of 674,295 resource-bytes, 34%**): same share, entirely different resources.
~~#735 (1 MB template bytes, 98.4%)~~ **Correction, two errors:**
1. **My percentage used the wrong denominator.** CDK checks against `TEMPLATE_BODY_MAXIMUM_SIZE = **1e6**` (`stack.js`), not CloudFormation's 1,048,576. So ECS is **99.6%** of CDK's threshold, and `lambda-microvm` + image at **1,019,962 bytes is 102.0% β already over**.
2. **#735 already had the better variant answer.** It measured 983,796 bytes on 2026-08-06 and its body names the configuration ("Lambda MicroVMs backend + image configured") β the actual worst cell, identified a month before this census used ECS. Per-variant today: `agentcore` 961,216 / `ecs` 996,141 / `lambda-microvm` 1,014,062 / `lambda-microvm` + image 1,019,962.
**Critical asymmetry worth recording:** the resource check **throws**, but the byte check only calls `Annotations.addWarningV2('@aws-cdk/core:Stack.templateSize', β¦)`. **Bytes fail *open*.** The worst cell has been shipping over CDK's own byte threshold with nothing red. #735 also currently carries **no labels and no priority**.
## 6. Proposed sequencing
1. **Now, unblock:** `allowTestInvoke: false` across the 34 call sites. ~~β 492 β **462** (headroom 38). Unblocks #681/#306 and the next several features.~~ **Correction β that arithmetic is ECS-specific.** ECS 492 β 462 (headroom 38); **worst cell 505 β 475 (headroom 25)**, a third of the advertised margin. App-wide 552 β 518 (β34). Still unblocks #681/#306. Low risk, security-positive, zero test breakage.
2. **Now, free bytes:** `suppressTemplateIndentation: true` (Β§3d) β β311 KB, 0 resources. Takes the byte axis from 99.6% to 68.5% and resolves #735.
3. **Now, stop recurrence:** set `@aws-cdk/core:stackResourceLimit` (Β§3c β built in, no custom code) and sweep the **gate product**, not one gate. Add the missing test asserting permissions stay method-scoped.
4. **Decide:** `defaultCorsPreflightOptions` (Β§3e) β β34. Product decision, not a pure win.
5. **Fix `DeletionPolicy` on all 45 stateful resources β `RETAIN`.** Do this regardless; it is a prerequisite for anything below and a standalone P0.
6. **Validate `cdk refactor`** against a throwaway deployment with populated tables, using `--unstable=refactor`. Gate step 7 on the result.
7. **Split on lifecycle, if still needed:** `NetworkStack` first (Β§4 β measured +3 total / β55 parent). **Never** the shared-`RestApi` seam.
8. ~~**Only if needed:** `scopePermissionToMethod: false` as a documented least-privilege trade (Β§3a).~~ **Removed β rejected outright** (Β§3a).
## 7. Acceptance criteria
- [ ] **Every cell of the gate product** (`compute_type` Γ `enableToolGateway`) synthesizes, with documented headroom, and the numbers are asserted by tests β including the `lambda-microvm` + `enableToolGateway` pairing that has no coverage today
- [ ] A synth-time guard **fails** the build when any cell exceeds budget, naming cell and count
- [ ] #681 / #306 unblocked without weakening its authorization model
- [ ] `cdk refactor` behaviour on stateful resources validated (with `--unstable=refactor`) and written up before any split lands
- [ ] All 45 stateful resources at `DeletionPolicy: RETAIN`
- [x] ~~Whether the 500 limit is Service-Quotas-adjustable: answered~~ **Answered: it is hard and non-adjustable (Β§1).**
- [ ] #735 either resolved by the same split or explicitly re-scoped to bytes only β noting its byte figure is measured against `1e6`, and that the byte check only warns
- [ ] A test asserts API Gateway Lambda permissions remain **method-scoped** (no guard exists today)
- [ ] Stack-boundary rationale recorded as an ADR
## 8. Open questions for iteration
1. ~~Is 500 adjustable? (blocked on quota-read access)~~ **ANSWERED (Β§1): no. Hard, non-adjustable, and CDK throws at synth before CloudFormation is consulted.**
2. Does `cdk refactor` preserve physical IDs for `DynamoDB::Table` / `S3::Bucket` / `Cognito::UserPool` in practice, at this scale? **Still open** β and now additionally blocked on confirming per-type `FULLY_MUTABLE` eligibility.
3. ~~Is `scopePermissionToMethod: false` acceptable given the Cedar authorization model, or is per-method scoping load-bearing for it?~~ **ANSWERED: Cedar is *not* load-bearing here.** Cedar gates **agent tool calls**, not HTTP method authorization, so the two are independent. Per-method scoping is defensible on IAM least-privilege grounds alone β which is enough to reject 3a without invoking Cedar at all.
4. Stack boundary: stateful/stateless, or by domain (task-api / integrations / platform)? **Partially answered:** by-domain is **not available** β every integration attaches to the shared `RestApi` (Β§4). Stateful/stateless moves only ~11% of resources and leaves **100% of the growth** in the app stack, so it is a *rollback-safety* win, not a capacity fix. `NetworkStack` is the viable seam.
5. ~~Does the ECS variant's extra 12 resources justify its own budget, or should the guard use one budget for the worst variant?~~ **ANSWERED by @theagenticguy:** one budget for the worst **cell**, because the cells are the same stack with optional substrates switched on, not independently deployable products. But the guard must *evaluate* every cell to know which is worst, and must fail on the throwing cell rather than reporting the default's 480.
## 9. Corrections log (2026-09-03)
| Β§ | Original claim | Correction |
|---|---|---|
| header, Β§1 | ECS 492/8 is the worst case | **`lambda-microvm` + `enableToolGateway` = 504, throws today.** 8-cell gate product; ECS is 3rd best (@theagenticguy; decisive cell re-verified) |
| Β§1 | 500-adjustability "unverified" | **Resolved: hard, non-adjustable**; docs + synth-time throw |
| Β§1 | "Variants differ by 12" | Differ by **25**, across a *product* of two gates |
| Β§1 | "enforced at changeset creation" | Enforced *earlier* β CDK throws at synth |
| Β§2 | 62 permissions = many methods sharing 22 functions | **31 pairs Γ 2 permissions**; zero function sharing |
| Β§3a | `scopePermissionToMethod: false` est. **β40** | Measured **β30**, identical to 3b β **rejected outright** |
| Β§3b | β30 | **β34 app-wide** (nested `RegistryApi` drops 4) |
| Β§3c | Write a custom budget guard; e.g. 460; per `compute_type` | **Built-in `@aws-cdk/core:stackResourceLimit`**; 460 unusable; must sweep the **gate product** |
| Β§3 | (missing) | **New Β§3d** `suppressTemplateIndentation` β311 KB / 0 resources; **new Β§3e** CORS preflight β34 |
| Β§4 | `cdk refactor` "ships" | **Unstable, `--unstable=refactor`**; eligibility per-type `FULLY_MUTABLE`, unverified here |
| Β§4 | (missing) | **PoC results**: `NetworkStack` +3/β55 VIABLE; shared-`RestApi` seam **REJECT** (silent Deployment/UA/tag loss) |
| Β§5 | Bytes at 98.4% of 1 MB | CDK checks **`1e6`** β ECS **99.6%**, worst cell **102.0%**; byte check only **warns** (fails open); #735 already named the worst variant |
| Β§6 | step 1 β 462 / headroom 38 | ECS-specific; **worst cell 505 β 475 / headroom 25** |
| Β§8 | Q1, Q3, Q5 open | **All three answered** |
## Reproduce
```bash
cd cdk
npx cdk synth -q -c compute_type=ecs # writes cdk.out/backgroundagent-dev.template.json
# the throwing cell β no template is written, but the error carries a type census
npx cdk synth -q -c compute_type=lambda-microvm -c enableToolGateway=true
# total
jq '.Resources | keys | length' cdk.out/backgroundagent-dev.template.json
# by top-level construct
jq -r '.Resources | to_entries
| map(.value.Metadata["aws:cdk:path"] // "NO-PATH")
| map(split("/") | if length > 1 then .[1] else .[0] end)
| group_by(.) | map({c:.[0], n:length}) | sort_by(-.n) | .[][]' \
cdk.out/backgroundagent-dev.template.json | paste - -
# test-invoke-stage permissions (the 30)
jq -r '.Resources | to_entries | map(select(.value.Type=="AWS::Lambda::Permission"))
| map(if ((.value.Properties.SourceArn|tostring)|test("test-invoke-stage"))
then "test-invoke-stage" else "real-stage" end)
| group_by(.) | map({k:.[0],n:length}) | .[][]' \
cdk.out/backgroundagent-dev.template.json | paste - -
# the 34 CORS OPTIONS methods (Β§3e)
jq -r '[.Resources | to_entries[] | select(.value.Type=="AWS::ApiGateway::Method")
| select(.value.Properties.HttpMethod=="OPTIONS")] | length' \
cdk.out/backgroundagent-dev.template.json
# CDK's own ceilings, from the installed library
grep -ow 'MAX_RESOURCES=[0-9]*\|TEMPLATE_BODY_MAXIMUM_SIZE=[0-9e.]*' \
node_modules/aws-cdk-lib/core/lib/stack.js
```
**Caveats for anyone reproducing:** every synth in this environment (including an unmodified baseline) exits 1 on `ec2:DescribeAvailabilityZones` AccessDenied β pre-existing credential gap, templates are still written, and the resource annotation still prints. Two baseline synths also disagree on the `β¦CurrentVersion*` / `β¦GuardrailVersion*` logical-ID hashes: pre-existing synth non-determinism, net 0 resources. Establish both as controls before attributing anything to a change. CLI synth counts exactly **1 more** than `Template.fromStack` (the CLI emits `AWS::CDK::Metadata`, Jest does not).
Related: #851 (incident record / measurement), #735 (1 MB template bytes β and the better variant answer), #830 (closed, same byte wall), #306 / #681 (blocked by this), #246 (introduced the existing nested stacks).
Contributor guide
Assessment
This issue has not been assessed yet.