Python: CdkFunction.environment is dereferenced prematurely instead of returning IResolvable
- Dominant language
- TypeScript
- Stars
- 2.9k
- Forks
- 267
- Avg merge
- 1d 25m
- Merged PRs (30d)
- 14
Description
### Describe the bug
[CdkFunction.environment](https://github.com/aws/aws-cdk/blob/784cd0e3303a/packages/aws-cdk-lib/aws-lambda/lib/function.ts#L807) is a lazily evaluated value which can be resolved to `CfnFunction.EnvironmentProperty`.
In TypeScript and Go, `environment` returns `LazyAny` and `typeregistry.anonymousProxy` respectively, both of which can then be resolved with `stack.resolve(environment)`.
In Python, `environment` returns an actual `EnvironmentProperty` instead of a reference, with `variables=None`. This resolves to an empty dict.
### Expected Behavior
I expected `CfnFunction.environment` to be an `IResolvable` value which could then be resolved with `stack.resolve(environment)`.
### Current Behavior
`CfnFunction.environment` is an empty `EnvironmentProperty` value with no reference, so it's not possible to get the environment variables from this value.
JSII_DEBUG output from Go:
```
> {"api":"get","property":"environment","objref":{"$jsii.byref":"aws-cdk-lib.aws_lambda.CfnFunction@10046"}}
[@jsii/kernel] get { '$jsii.byref': 'aws-cdk-lib.aws_lambda.CfnFunction@10046' } environment
[@jsii/kernel] value: LazyAny {
producer: { produce: [Function: produce] },
cache: false,
creationStack: [ 'Execute again with CDK_DEBUG=true to capture stack traces' ],
options: {}
}
[@jsii/kernel] serialize LazyAny {
producer: { produce: [Function: produce] },
cache: false,
creationStack: [ 'Execute again with CDK_DEBUG=true to capture stack traces' ],
options: {}
} {
serializationClass: 'Struct',
typeRef: {
type: { fqn: 'aws-cdk-lib.aws_lambda.CfnFunction.EnvironmentProperty' },
optional: true
}
} {
serializationClass: 'RefType',
typeRef: { type: { fqn: 'aws-cdk-lib.IResolvable' }, optional: true }
}
[@jsii/kernel] Returning value type by reference
[@jsii/kernel] ret: {
'$jsii.byref': 'Object@10047',
'$jsii.interfaces': [ 'aws-cdk-lib.aws_lambda.CfnFunction.EnvironmentProperty' ]
}
< {"ok":{"value":{"$jsii.byref":"Object@10047","$jsii.interfaces":["aws-cdk-lib.aws_lambda.CfnFunction.EnvironmentProperty"]}}}
```
From Python:
```
> {"objref":{"$jsii.byref":"aws-cdk-lib.aws_lambda.CfnFunction@10046"},"property":"environment","api":"get"}
[@jsii/kernel] get { '$jsii.byref': 'aws-cdk-lib.aws_lambda.CfnFunction@10046' } environment
[@jsii/kernel] value: LazyAny {
producer: { produce: [Function: produce] },
cache: false,
creationStack: [ 'Execute again with CDK_DEBUG=true to capture stack traces' ],
options: {}
}
[@jsii/kernel] serialize LazyAny {
producer: { produce: [Function: produce] },
cache: false,
creationStack: [ 'Execute again with CDK_DEBUG=true to capture stack traces' ],
options: {}
} {
serializationClass: 'Struct',
typeRef: {
type: { fqn: 'aws-cdk-lib.aws_lambda.CfnFunction.EnvironmentProperty' },
optional: true
}
} {
serializationClass: 'RefType',
typeRef: { type: { fqn: 'aws-cdk-lib.IResolvable' }, optional: true }
}
[@jsii/kernel] Returning value type by reference
[@jsii/kernel] ret: {
'$jsii.byref': 'Object@10047',
'$jsii.interfaces': [ 'aws-cdk-lib.aws_lambda.CfnFunction.EnvironmentProperty' ]
}
< {"ok":{"value":{"$jsii.byref":"Object@10047","$jsii.interfaces":["aws-cdk-lib.aws_lambda.CfnFunction.EnvironmentProperty"]}}}
> {"objref":{"$jsii.byref":"Object@10047"},"property":"variables","api":"get"}
[@jsii/kernel] get { '$jsii.byref': 'Object@10047' } variables
[@jsii/kernel] value: undefined
[@jsii/kernel] serialize undefined {
serializationClass: 'Map',
typeRef: { type: { collection: [Object] }, optional: true }
} {
serializationClass: 'RefType',
typeRef: { type: { fqn: 'aws-cdk-lib.IResolvable' }, optional: true }
}
[@jsii/kernel] ret: undefined
< {"ok":{}}
```
Note the extra request at the end of the Python logs. This is from the exact same line of code, `cfnFunction.Environment()` and `cfn_function.environment` respectively.
### Reproduction Steps
Paste each block into the stack file in the project created by `cdk init` for that language, then run `cdk ls`.
**Python:**
```py
from aws_cdk import Stack, aws_lambda as lambda_
from constructs import Construct
class CdkPyStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
func = lambda_.Function(
self,
"Function",
code=lambda_.Code.from_inline("_"),
runtime=lambda_.Runtime.PYTHON_3_10,
handler="_",
environment={"KEY": "value"},
)
cfn_function = func.node.default_child
assert isinstance(cfn_function, lambda_.CfnFunction)
environment = cfn_function.environment
print(environment)
print(self.resolve(environment))
```
Output:
```
EnvironmentProperty()
{}
CdkPyStack
```
**TypeScript (working, for comparison):**
```ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
export class CdkTsStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
let func = new lambda.Function(this, "Function", {
code: lambda.Code.fromInline("_"),
runtime: lambda.Runtime.PYTHON_3_10,
handler: "_",
environment: { KEY: "value" },
});
let cfnFunction = func.node.defaultChild as lambda.CfnFunction;
let environment = cfnFunction.environment;
console.log(environment);
console.log(this.resolve(environment));
}
}
```
Output:
```
LazyAny {
producer: { produce: [Function: produce] },
cache: false,
creationStack: [ 'Execute again with CDK_DEBUG=true to capture stack traces' ],
options: {}
}
{ variables: { KEY: 'value' } }
CdkTsStack
```
### Possible Solution
Here's what I've been able to figure out by stepping through with a debugger. Hopefully this helps.
`CfnFunction.environment` is a property which calls `jsii.get()`, aka `kernel.get()`.
https://github.com/aws/jsii/blob/14b5ed22fe87baf27be0f0ff61f6e423654baf39/packages/%40jsii/python-runtime/src/jsii/_kernel/__init__.py#L358-L366
The value returned from `kernel.get()` is an `ObjRef`, which is passed to ` _reference_map.resolve_reference()` via the `@_dereferenced` decorator and `_recursize_dereference()`.
https://github.com/aws/jsii/blob/14b5ed22fe87baf27be0f0ff61f6e423654baf39/packages/%40jsii/python-runtime/src/jsii/_kernel/__init__.py#L133-L151
The ref is something like `Object@10047`, so `class_fqn` is `Object`. The condition on line 104-106 below evaluates to True and it ends up creating and returning an `EnvironmentProperty` value with no data (because it hasn't been resolved with CDK yet).
https://github.com/aws/jsii/blob/14b5ed22fe87baf27be0f0ff61f6e423654baf39/packages/%40jsii/python-runtime/src/jsii/_reference_map.py#L102-L128
### Additional Information/Context
My use case is to create a [custom aspect](https://docs.aws.amazon.com/cdk/v2/guide/aspects.html) which adds default environment variables to all Lambdas in a stack. The aspect shouldn't overwrite variables set directly on a function, so it needs to know which ones already exist, if any.
### SDK version used
JSII 1.84.0, CDK lib and cli 2.84.0, Constructs 10.2.55
### Environment details (OS name and version, etc.)
Windows 10.0.14393 Build 14393, Python 3.11.3
Contributor guide
Research direction
Reproduce the Python and TypeScript behavior in a CDK-initialized project with cdk ls, then inspect the jsii runtime entry points in packages/@jsii/python-runtime/src/jsii/_kernel/__init__.py and _reference_map.py. Compare how the ObjRef is dereferenced for CfnFunction.environment. Done means Python preserves the lazy IResolvable behavior so resolving the property exposes the configured environment variables rather than an empty EnvironmentProperty.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, typescript
- Domain
- developer-experience, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100