aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap
[AWS::SecurityAgent::Pentest] - [BUG] - readOnlyProperties path with a wildcard array index cannot be retrieved by Fn::GetAtt, and the error message misstates the cause
- Dominant language
- No language data
- Stars
- 1.1k
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
### Name of the resource
Other
### Resource Name
AWS::SecurityAgent::Pentest
### Issue Description
Category: (e) other coverage-related issue with the resource/attribute/option.
A resource type schema may declare a read-only property nested inside an array using a wildcard for the array index, for example:
"readOnlyProperties": [
"/properties/Assets/Actors/*/MfaForwardingAddress"
]
No `Fn::GetAtt` form can retrieve such a value:
- A concrete index (`!GetAtt Pentest.Assets.Actors.0.MfaForwardingAddress`) resolves as a schema location but is rejected as not read-only, because the declared path carries a wildcard where the request carries a literal index.
- A wildcard (`!GetAtt Pentest.Assets.Actors.*.MfaForwardingAddress`) is rejected as not existing in the schema.
Two separate validation stages are involved, which is why there is no template-side workaround: the path must first resolve against the schema, and must then match a `readOnlyProperties` entry. A concrete index passes the first and fails the second; a wildcard fails the first. Since the only declaration of the value uses a wildcard, no index value can ever satisfy the read-only comparison.
This is not specific to one service. In the published resource schema set (https://schema.cloudformation.us-east-1.amazonaws.com/CloudformationSchema.zip), 36 of 1,721 `AWS::` resource types declare read-only properties nested inside arrays with this notation, spanning AWS Elemental MediaPackage, Direct Connect, Amazon EFS, AWS IoT SiteWise, Amazon FraudDetector, AWS Glue, AWS WAFv2, Amazon SageMaker and others. None of them publish those nested paths in the Return values section of their reference page, so the declared values are unreachable from a template across the board. Long-standing examples:
- `AWS::MediaPackage::Channel` declares `/properties/HlsIngest/ingestEndpoints/*/{Id,Username,Password,Url}` and publishes only `Arn`.
- `AWS::DirectConnect::PrivateVirtualInterface` declares `/properties/BgpPeers/*/BgpPeerId` and publishes only `VirtualInterfaceArn` and `VirtualInterfaceId`.
Reproducible count:
curl -sO https://schema.cloudformation.us-east-1.amazonaws.com/CloudformationSchema.zip
unzip -q CloudformationSchema.zip -d schemas && cd schemas
python3 -c "
import json,glob
for f in glob.glob('aws-*.json'):
d=json.load(open(f))
ro=[p for p in d.get('readOnlyProperties',[]) if '/*/' in p]
if ro: print(d['typeName'], ro)
" | sort
There are two distinct problems here: the resolution behaviour itself, and an error message that points at the wrong cause.
### Expected Behavior
Either of the following would be acceptable:
1. A path declared in `readOnlyProperties` with a wildcard array index is retrievable by supplying a concrete index, so that `!GetAtt Pentest.Assets.Actors.0.MfaForwardingAddress` resolves to the value for that element, with the read-only check treating the wildcard segment as matching any index.
2. If the behaviour is intentional and such paths are not meant to be retrievable, then:
- the error message names the real cause, for example "the read-only property `/properties/Assets/Actors/*/MfaForwardingAddress` is declared with a wildcard array index and cannot be retrieved by `Fn::GetAtt`, which does not support addressing array elements"; and
- the `Fn::GetAtt` documentation states the caveat, since the page currently documents nested object traversal (`SourceSecurityGroup.OwnerAlias`) without noting that array elements cannot be addressed; and
- schema validation or submission tooling rejects or warns on a wildcard entry in `readOnlyProperties`, so resource types stop shipping declarations that can never be consumed.
### Observed Behavior
Deployment fails at execution time. The referenced resource itself reaches CREATE_COMPLETE, then the consumer of the attribute fails and the stack rolls back.
- `!GetAtt Pentest.Assets.Actors.0.MfaForwardingAddress` -> error stating the attribute "must be a readonly property in schema", despite `/properties/Assets/Actors/*/MfaForwardingAddress` being present in `readOnlyProperties` in the published schema (https://schema.cloudformation.eu-west-1.amazonaws.com/aws-securityagent-pentest.json).
- `!GetAtt Pentest.Assets.Actors.*.MfaForwardingAddress` -> error stating the attribute "does not exist in schema".
The first message is the misleading one: it asserts the property is not read-only when the schema declares that it is, which sends users to verify the schema, where they find the declaration present and conclude the reference should work.
The failure surfaces differently depending on where the reference sits, but is not avoidable in either place:
Reference in a resource property: that resource CREATE_FAILED, then rollback
Reference only in an Output: all resources CREATE_COMPLETE, then stack rollback
Neither `aws cloudformation validate-template` nor `aws cloudformation create-change-set` reports any problem. The change set reaches CREATE_COMPLETE and reports no issue, so pre-deployment review does not catch this class of error and it only appears on execution.
Compounding the confusion, the property reference pages render these server-generated read-only nested values inside the input syntax block with `Required: No`, which reads as settable:
- `MfaForwardingAddress` on the Pentest Actor page (https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-securityagent-pentest-actor.html)
- `BgpPeerId` on the Direct Connect BgpPeer page (https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-directconnect-privatevirtualinterface-bgppeer.html)
- `Username` and `Password` on the MediaPackage IngestEndpoint page, both described as "system-generated" while appearing as settable inputs (https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-mediapackage-channel-ingestendpoint.html)
### Test Cases
1. Retrieval by concrete index: given a resource type whose schema declares `/properties/A/B/*/C` in `readOnlyProperties`, a template using `!GetAtt R.A.B.0.C` should either resolve to the value of element 0 or fail with a message naming the wildcard-index cause. It should not report that the property is not read-only.
2. Same in an Output: the same reference used only in `Outputs[*].Value` should behave identically to the resource-property case, and should not allow every resource to reach CREATE_COMPLETE before failing the stack if the reference is statically known to be unresolvable.
3. Out-of-range index: `!GetAtt R.A.B.9.C` where the array has fewer than 10 elements should fail with a distinct out-of-range message, not with the read-only message.
4. Wildcard in a template: `!GetAtt R.A.B.*.C` should continue to be rejected, ideally with a message stating that wildcards are a schema-declaration notation and are not valid in a template reference.
5. Control, must not regress: `!GetAtt R.TopLevelReadOnly` on the same resource type should continue to resolve. Confirmed working today, so this is a regression guard.
6. Pre-deployment detection: `validate-template` and `create-change-set` on a template containing an unresolvable reference of this shape should surface the problem, rather than reporting CREATE_COMPLETE with no issue.
7. Drift detection: whatever the resolution decision, confirm the value still round-trips on a subsequent read/list call and that drift detection on the resource is unaffected, since these paths exist precisely so the service can return them to CloudFormation.
8. Cross-service coverage: run cases 1-4 against `AWS::MediaPackage::Channel` (`HlsIngest/ingestEndpoints/*/Url`) and `AWS::DirectConnect::PrivateVirtualInterface` (`BgpPeers/*/BgpPeerId`) so the fix is verified against the general pattern rather than a single resource type.
### Other Details
Related links:
- `Fn::GetAtt` reference, which documents nested object traversal but no array-element notation: https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/intrinsic-function-reference-getatt.html
- Definition of `readOnlyProperties` in the resource type schema: https://docs.aws.amazon.com/cloudformation-cli/latest/userguide/resource-type-schema.html
- `AWS::SecurityAgent::Pentest` reference page, whose Return values section lists only `CreatedAt`, `PentestId` and `UpdatedAt`: https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-securityagent-pentest.html
- Published schema for the resource type: https://schema.cloudformation.eu-west-1.amazonaws.com/aws-securityagent-pentest.json
- Full published schema set used for the 36-of-1,721 count: https://schema.cloudformation.us-east-1.amazonaws.com/CloudformationSchema.zip
Current workaround, for anyone hitting this: a Lambda-backed custom resource that calls the service read API directly and returns the value as response data, then reference the custom resource instead of the resource type (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html). For `AWS::SecurityAgent::Pentest` the relevant call is `securityagent:BatchGetPentests`, which is the same permission the resource type's own read handler declares in its published schema. This works but requires additional IAM permissions, a Lambda function and its lifecycle, for a value the service already returns to CloudFormation.
Separate documentation request filed for the `Fn::GetAtt` caveat and the property-page presentation of read-only nested values as settable inputs.
Contributor guide
Research direction
Start with the published AWS::SecurityAgent::Pentest schema and the Fn::GetAtt resolution path, reproducing the concrete-index and wildcard cases described in the issue. Compare the behavior with the MediaPackage and Direct Connect schemas, then verify the chosen resolution or error behavior, pre-deployment detection, regression cases, and drift behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100