aws-cloudformation / aws-cloudformation/cloudformation-coverage-roadmap

[AWS::SecurityAgent::Pentest] - [BUG] - readOnlyProperties path with a wildcard array index cannot be retrieved by Fn::GetAtt, and the error message misstates the cause

Open
#2,555 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
No language data
Stars
1.1k
Forks
62
PR merge metrics
No merged PRs in 30d

Description

### Name of the resource

Other

### Resource Name

AWS::SecurityAgent::Pentest

### Issue Description

Category: (e) other coverage-related issue with the resource/attribute/option.

A resource type schema may declare a read-only property nested inside an array using a wildcard for the array index, for example:

"readOnlyProperties": [
"/properties/Assets/Actors/*/MfaForwardingAddress"
]

No `Fn::GetAtt` form can retrieve such a value:

- A concrete index (`!GetAtt Pentest.Assets.Actors.0.MfaForwardingAddress`) resolves as a schema location but is rejected as not read-only, because the declared path carries a wildcard where the request carries a literal index.
- A wildcard (`!GetAtt Pentest.Assets.Actors.*.MfaForwardingAddress`) is rejected as not existing in the schema.

Two separate validation stages are involved, which is why there is no template-side workaround: the path must first resolve against the schema, and must then match a `readOnlyProperties` entry. A concrete index passes the first and fails the second; a wildcard fails the first. Since the only declaration of the value uses a wildcard, no index value can ever satisfy the read-only comparison.

This is not specific to one service. In the published resource schema set (https://schema.cloudformation.us-east-1.amazonaws.com/CloudformationSchema.zip), 36 of 1,721 `AWS::` resource types declare read-only properties nested inside arrays with this notation, spanning AWS Elemental MediaPackage, Direct Connect, Amazon EFS, AWS IoT SiteWise, Amazon FraudDetector, AWS Glue, AWS WAFv2, Amazon SageMaker and others. None of them publish those nested paths in the Return values section of their reference page, so the declared values are unreachable from a template across the board. Long-standing examples:

- `AWS::MediaPackage::Channel` declares `/properties/HlsIngest/ingestEndpoints/*/{Id,Username,Password,Url}` and publishes only `Arn`.
- `AWS::DirectConnect::PrivateVirtualInterface` declares `/properties/BgpPeers/*/BgpPeerId` and publishes only `VirtualInterfaceArn` and `VirtualInterfaceId`.

Reproducible count:

curl -sO https://schema.cloudformation.us-east-1.amazonaws.com/CloudformationSchema.zip
unzip -q CloudformationSchema.zip -d schemas && cd schemas
python3 -c "
import json,glob
for f in glob.glob('aws-*.json'):
d=json.load(open(f))
ro=[p for p in d.get('readOnlyProperties',[]) if '/*/' in p]
if ro: print(d['typeName'], ro)
" | sort

There are two distinct problems here: the resolution behaviour itself, and an error message that points at the wrong cause.

### Expected Behavior

Either of the following would be acceptable:

1. A path declared in `readOnlyProperties` with a wildcard array index is retrievable by supplying a concrete index, so that `!GetAtt Pentest.Assets.Actors.0.MfaForwardingAddress` resolves to the value for that element, with the read-only check treating the wildcard segment as matching any index.

2. If the behaviour is intentional and such paths are not meant to be retrievable, then:
- the error message names the real cause, for example "the read-only property `/properties/Assets/Actors/*/MfaForwardingAddress` is declared with a wildcard array index and cannot be retrieved by `Fn::GetAtt`, which does not support addressing array elements"; and
- the `Fn::GetAtt` documentation states the caveat, since the page currently documents nested object traversal (`SourceSecurityGroup.OwnerAlias`) without noting that array elements cannot be addressed; and
- schema validation or submission tooling rejects or warns on a wildcard entry in `readOnlyProperties`, so resource types stop shipping declarations that can never be consumed.

### Observed Behavior

Deployment fails at execution time. The referenced resource itself reaches CREATE_COMPLETE, then the consumer of the attribute fails and the stack rolls back.

- `!GetAtt Pentest.Assets.Actors.0.MfaForwardingAddress` -> error stating the attribute "must be a readonly property in schema", despite `/properties/Assets/Actors/*/MfaForwardingAddress` being present in `readOnlyProperties` in the published schema (https://schema.cloudformation.eu-west-1.amazonaws.com/aws-securityagent-pentest.json).
- `!GetAtt Pentest.Assets.Actors.*.MfaForwardingAddress` -> error stating the attribute "does not exist in schema".

The first message is the misleading one: it asserts the property is not read-only when the schema declares that it is, which sends users to verify the schema, where they find the declaration present and conclude the reference should work.

The failure surfaces differently depending on where the reference sits, but is not avoidable in either place:

Reference in a resource property: that resource CREATE_FAILED, then rollback
Reference only in an Output: all resources CREATE_COMPLETE, then stack rollback

Neither `aws cloudformation validate-template` nor `aws cloudformation create-change-set` reports any problem. The change set reaches CREATE_COMPLETE and reports no issue, so pre-deployment review does not catch this class of error and it only appears on execution.

Compounding the confusion, the property reference pages render these server-generated read-only nested values inside the input syntax block with `Required: No`, which reads as settable:
- `MfaForwardingAddress` on the Pentest Actor page (https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-securityagent-pentest-actor.html)
- `BgpPeerId` on the Direct Connect BgpPeer page (https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-directconnect-privatevirtualinterface-bgppeer.html)
- `Username` and `Password` on the MediaPackage IngestEndpoint page, both described as "system-generated" while appearing as settable inputs (https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-mediapackage-channel-ingestendpoint.html)

### Test Cases

1. Retrieval by concrete index: given a resource type whose schema declares `/properties/A/B/*/C` in `readOnlyProperties`, a template using `!GetAtt R.A.B.0.C` should either resolve to the value of element 0 or fail with a message naming the wildcard-index cause. It should not report that the property is not read-only.

2. Same in an Output: the same reference used only in `Outputs[*].Value` should behave identically to the resource-property case, and should not allow every resource to reach CREATE_COMPLETE before failing the stack if the reference is statically known to be unresolvable.

3. Out-of-range index: `!GetAtt R.A.B.9.C` where the array has fewer than 10 elements should fail with a distinct out-of-range message, not with the read-only message.

4. Wildcard in a template: `!GetAtt R.A.B.*.C` should continue to be rejected, ideally with a message stating that wildcards are a schema-declaration notation and are not valid in a template reference.

5. Control, must not regress: `!GetAtt R.TopLevelReadOnly` on the same resource type should continue to resolve. Confirmed working today, so this is a regression guard.

6. Pre-deployment detection: `validate-template` and `create-change-set` on a template containing an unresolvable reference of this shape should surface the problem, rather than reporting CREATE_COMPLETE with no issue.

7. Drift detection: whatever the resolution decision, confirm the value still round-trips on a subsequent read/list call and that drift detection on the resource is unaffected, since these paths exist precisely so the service can return them to CloudFormation.

8. Cross-service coverage: run cases 1-4 against `AWS::MediaPackage::Channel` (`HlsIngest/ingestEndpoints/*/Url`) and `AWS::DirectConnect::PrivateVirtualInterface` (`BgpPeers/*/BgpPeerId`) so the fix is verified against the general pattern rather than a single resource type.

### Other Details

Related links:

- `Fn::GetAtt` reference, which documents nested object traversal but no array-element notation: https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/intrinsic-function-reference-getatt.html
- Definition of `readOnlyProperties` in the resource type schema: https://docs.aws.amazon.com/cloudformation-cli/latest/userguide/resource-type-schema.html
- `AWS::SecurityAgent::Pentest` reference page, whose Return values section lists only `CreatedAt`, `PentestId` and `UpdatedAt`: https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-securityagent-pentest.html
- Published schema for the resource type: https://schema.cloudformation.eu-west-1.amazonaws.com/aws-securityagent-pentest.json
- Full published schema set used for the 36-of-1,721 count: https://schema.cloudformation.us-east-1.amazonaws.com/CloudformationSchema.zip

Current workaround, for anyone hitting this: a Lambda-backed custom resource that calls the service read API directly and returns the value as response data, then reference the custom resource instead of the resource type (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html). For `AWS::SecurityAgent::Pentest` the relevant call is `securityagent:BatchGetPentests`, which is the same permission the resource type's own read handler declares in its published schema. This works but requires additional IAM permissions, a Lambda function and its lifecycle, for a value the service already returns to CloudFormation.

Separate documentation request filed for the `Fn::GetAtt` caveat and the property-page presentation of read-only nested values as settable inputs.

Contributor guide

Open the contributing guide

Research direction

Start with the published AWS::SecurityAgent::Pentest schema and the Fn::GetAtt resolution path, reproducing the concrete-index and wildcard cases described in the issue. Compare the behavior with the MediaPackage and Direct Connect schemas, then verify the chosen resolution or error behavior, pre-deployment detection, regression cases, and drift behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.