cdk-cli: allow --no-previous-parameters to be configured in `cdk.json` alongside other config
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 74
Description
### Describe the feature
When using CDK aspects like `StringParameter.fromStringParameterAttributes` / `StringParameter.valueForStringParameter`; by default, they use CloudFormation Stack Parameters; which are fixed at the point of first deploy, unless we deploy with `cdk deploy --no-previous-parameters`.
This is often confusing and unexpected, as outlined in this issue:
- https://github.com/aws/aws-cdk/issues/7722
### Use Case
I want to enforce `--no-previous-parameters` to always be used in my `cdk.json`, so that I can commit it into my project repo.
### Proposed Solution
Allow `--no-previous-parameters` to be configured in `cdk.json` alongside existing config values.
### Other Information
You can see some of my deepdive comments on the following issue:
- https://github.com/aws/aws-cdk/issues/7722#issuecomment-1637516435
- https://github.com/aws/aws-cdk/issues/7722#issuecomment-1639188041
Notably in that 2nd comment, it appears that some very recent changes (~2 weeks ago) made in CDK `2.87.0` would sort of allow the root issue to be worked around in a different way if I made use of `forceDynamicReference`:
> > **Edit 3:** Looking closer at the CDK docs for `StringParameter.fromStringParameterAttributes(scope, id, attrs)`, and particularly `StringParameterAttributes`, there appears to be a `forceDynamicReference` option:
> >
> > * [docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ssm.StringParameter.html#static-fromwbrstringwbrparameterwbrattributesscope-id-attrs](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ssm.StringParameter.html#static-fromwbrstringwbrparameterwbrattributesscope-id-attrs)
> > * `StringParameterAttributes`: [docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ssm.StringParameterAttributes.html#forcedynamicreference](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ssm.StringParameterAttributes.html#forcedynamicreference)
> >
> > * > Use a dynamic reference as the representation in CloudFormation template level.
> > > By default, CDK tries to deduce an appropriate representation based on the parameter value (a CfnParameter or a dynamic reference). Use this flag to override the representation when it does not work.
>
> Ha.. so apparently `forceDynamicReference` and the underlying functionality related to it is brand new as of CDK `2.87.0` (released ~2 weeks ago):
>
> - https://github.com/aws/aws-cdk/releases/tag/v2.87.0
> - > ssm: cannot import a ssm parameter with a name containing unresolved token ([1f1b642](https://github.com/aws/aws-cdk/commit/1f1b6426f00031dc0f67b9690d33f718f394733c))
> > - https://github.com/aws/aws-cdk/issues/25749
> > - closes https://github.com/aws/aws-cdk/issues/17094
>
> > Previously, when we import a SSM parameter by `ssm.StringParameter.fromStringParameterAttributes`, we use `CfnParameter` to get the value.
> >
> > ```json
> > "Parameters": {
> > "importsqsstringparamParameter": {
> > "Type": "AWS::SSM::Parameter::Value",
> > "Default": {
> > "Fn::ImportValue": "some-exported-value-holding-the-param-name"
> > }
> > },
> > ```
> >
> > However, `Parameters..Default` only allows a concrete string value. If it contains e.g. intrinsic functions, we get an error like this from CFn: `Template format error: Every Default member must be a string.`
> >
> > This PR changes the behavior of `fromStringParameterAttributes` method. Now it uses `CfnDynamicReference` instead of `CfnParameter` if a parameter name contains unresolved tokens.
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749_
>
> > Another thing we can say about ssm parameters is that it doesn't differ much between [CfnParameters](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html#aws-ssm-parameter-types) and [dynamic references](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html). Reading through the document, it seems that most of the characteristics are the same, such as when it's resolved and updated, where it can be used in a template, etc. There are of course some differences e.g. max num of references (200 vs 60), but they seems trivial.
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749#discussion_r1213201913_
>
> > I'm now wondering whether switching a [parameter](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html#aws-ssm-parameter-types) to a [dynamic reference](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html) should really be considered as a breaking change. As far as I read the docs, there seems to be no remarkable difference between them. Given it's also very rare to use a lazy token for parameter names, we can tolerate the change, maybe under a feature flag.
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749#discussion_r1229498664_
>
> > If only we could stop using `CfnParameter` and use `CfnDynamicReference` instead for all cases... I see no point to use `CfnParameter` here. (Actually, isn't that the purpose of feature flags?)
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749#discussion_r1229662611_
>
> > There are some limitations https://github.com/aws/aws-cdk/pull/22239#issuecomment-1262069499
> >
> > _Originally posted by @corymhall in https://github.com/aws/aws-cdk/pull/25749#discussion_r1229669719_
>
> > So how about letting users choose which they use, parameter or dynamic reference? We'll add a property like `forceDynamicReference?: boolean` (default to false) to `CommonStringParameterAttributes`. This is kind of a leaky abstraction, but it should at least solve all the problem above. Plus we can easily ensure there is no breaking change, without adding any feature flag.
> >
> > _Originally posted by @tmokmss in https://github.com/aws/aws-cdk/pull/25749#discussion_r1229717891_
>
> _Originally posted by @0xdevalias in https://github.com/aws/aws-cdk/issues/7722#issuecomment-1639188041_
---
Then, on an internal issue, I noted the following while researching whether it was possible to configure this in `cdk.json`:
> ## Avoiding this issue coming up again in future
>
> Since I don't believe we really make use of CDK / CloudFormation parameters directly ourselves (and it's apparently not really a recommended thing to do either), I wonder if there would be any harm in us forcing CDK to always update them so that we avoid this issue in future?
>
> Looking in AWS web console -> CloudFormation, for each of our stacks:
>
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter + SSM parameters
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter + SSM parameters
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter
> - `REDACTED`: ✅ Seems to only use the `BootstrapVersion` parameter
>
> ✅ Based on the above, it looks like it should be safe to always deploy our stacks with `--no-previous-parameters`!
>
> ---
>
> Looking at the `cdk deploy --help`, here are the relevant parts that mention parameters:
>
> ```shell
> ⇒ yarn cdk-dev deploy --help
> yarn run v1.22.19
> $ yarn -s awsvault-dev:nosession cdk deploy --help
> cdk deploy [STACKS..]
>
> Deploys the stack(s) named STACKS into your AWS account
>
> Options:
> ..snip..
>
> -c, --context Add contextual string parameter (KEY=VALUE) [array]
>
> ..snip..
>
> --parameters Additional parameters passed to CloudFormation at
> deploy time (STACK:KEY=VALUE) [array] [default: {}]
> ..snip..
>
> --previous-parameters Use previous values for existing parameters (you
> must specify all parameters on every deployment if
> this is disabled) [boolean] [default: true]
> ..snip..
>
> ✨ Done in 6.61s.
> ```
>
> `--previous-parameters` defaults to `true`, which is why we have to use `--no-previous-parameters` to disable it.
>
> It seems that at least some of the CDK CLI flags can be configured in the project's `cdk.json` file:
>
> - https://docs.aws.amazon.com/cdk/v2/guide/cli.html#cli-config
>
> Unfortunately this doesn't seem to include the `previous-parameters` / `no-previous-parameters` option.
>
> We can confirm that by looking at CDK's `Configuration` class:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/settings.ts#L68-L158
>
> and `Settings` class:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/settings.ts#L232-L445
>
> Neither of which seem to contain entries for `previous-parameters` / `previousParameters`, which only seem to be meaningfully used by:
>
> The `cdk bootstrap` command:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/cli.ts#L510
>
> The `cdk deploy` command:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/cli.ts#L569
>
> And a commented out reference in the `cdk watch` command:
>
> https://github.com/aws/aws-cdk/blob/6c75581ae2b9537fa9d1d724b837fe81ae22d345/packages/aws-cdk/lib/cli.ts#L603
>
> ---
>
> Based on the above, it seems like our simplest/best option right now might be to just hardcode `--no-previous-parameters` into our deploy helper script in `package.json` or similar (⚠️ TODO: and probably also raise an upstream issue on the AWS CDK repository to explore whether this is something that can be added to the CDK)
>
> _Originally posted by @0xdevalias in https://github.com/PsychNEXUS/REMDR/issues/278#issuecomment-1637500082_
### Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
### CDK version used
2.84.0 (build f7c792f)
### Environment details (OS name and version, etc.)
macOS / N/A
Contributor guide
Research direction
Start in packages/aws-cdk/lib/settings.ts, reviewing Configuration and Settings, then trace how the deploy command uses the option in packages/aws-cdk/lib/cli.ts. Confirm the relevant setting can be configured in cdk.json alongside existing configuration and that cdk deploy honors it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100