aws / aws/aws-cdk

(python): install-order collision on aws_cdk/cloud_assembly_schema/__init__.py breaks artifact.messages

Open
#38,482 1 comment 0 reactions 0 assignees View on GitHub
bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### What happened

Two independently-installable Python distributions ship files into the **same package directory**, `aws_cdk/cloud_assembly_schema/`:

* `aws-cdk-lib` ships `aws_cdk/cloud_assembly_schema/__init__.py`
* `aws-cdk.cloud-assembly-schema` ships `aws_cdk/cloud_assembly_schema/__init__.py` **and** `aws_cdk/cloud_assembly_schema/_jsii/`

`aws-cdk-lib` depends on `aws-cdk.cloud-assembly-schema`, so both are always installed together. Because both wheels write the same path, **which build of `__init__.py` ends up on disk depends on the order the installer happens to unpack them** — wheels are not supposed to make install order semantically significant.

The two builds are not interchangeable. They declare **different jsii type names for the same classes**:

| distribution | `MetadataEntry.__jsii_type__` |
|---|---|
| `aws-cdk-lib` | `aws-cdk-lib.cloud_assembly_schema.MetadataEntry` |
| `aws-cdk.cloud-assembly-schema` | `@aws-cdk/cloud-assembly-schema.MetadataEntry` |

### Minimal reproducer (~30 seconds)

```bash
uv init repro && cd repro
uv add aws-cdk-lib
```

```python
# probe.py
import collections, importlib.metadata as md
TARGET = "aws_cdk/cloud_assembly_schema/__init__.py"
owners = collections.defaultdict(list)
for dist in md.distributions():
for f in (dist.files or []):
if str(f).replace("\\", "/").endswith(TARGET):
owners[TARGET].append((dist.metadata["Name"], dist.version))
print(owners)
import aws_cdk.cloud_assembly_schema as cas
print(cas.MetadataEntry.__jsii_type__)
```

Output:

```
{'aws_cdk/cloud_assembly_schema/__init__.py': [
('aws-cdk.cloud-assembly-schema', '54.16.0'),
('aws-cdk-lib', '2.263.0')]}
aws-cdk-lib.cloud_assembly_schema.MetadataEntry
```

Two distributions claiming one path is reproducible from a bare `uv add aws-cdk-lib` on current versions. (In this particular run `aws-cdk-lib`'s build won, which is the working outcome.)

### Observable consequence when the other build wins

The jsii **kernel** always returns the `aws-cdk-lib.`-prefixed fqn. Verified with `JSII_DEBUG=1`, byte-identical kernel payloads in a working and a failing environment:

```
< {"ok":{"value":{"$jsii.byref":"Object@10535",
"$jsii.interfaces":["aws-cdk-lib.cloud_assembly_schema.MetadataEntry"]}}}
```

When the standalone build is the one on disk, the Python class registers `@aws-cdk/cloud-assembly-schema.MetadataEntry`, which can never match what the kernel returned. Any stack carrying a synthesis message then raises while reading `CloudFormationStackArtifact.messages`:

```
KeyError: 'aws-cdk-lib.cloud_assembly_schema.MetadataEntry'
jsii/_reference_map.py, in build_interface_proxies_for_ref
```

In our app this hit exactly the three stacks that emit a SnapStart warning; stacks with zero messages are unaffected, so it presents as an intermittent failure tied to whether any warning exists.

### Why "just exclude the standalone package" is not a workaround

Excluding `aws-cdk.cloud-assembly-schema` resolves cleanly and then breaks CDK: `aws_cdk.cloud_assembly_schema._jsii` — the jsii assembly tarball — ships **only** in the standalone distribution. The directory is genuinely shared: `__init__.py` from either, `_jsii/` from the standalone alone.

Deleting the directory to force a clean reinstall also does not recover: the files survive `pip`/`uv` uninstall of *both* claiming distributions, and installs skip a path that already exists.

### Environment

**All observations are from a single Windows machine. I have not tested Linux or macOS**, and the install-order behaviour may well differ there.

* Reproducer: `aws-cdk-lib` 2.263.0, `aws-cdk.cloud-assembly-schema` 54.16.0, `jsii` 1.139.0, `constructs` 10.8.0
* Failure observed on: `aws-cdk-lib` 2.254.0, `aws-cdk.cloud-assembly-schema` 53.24.0, `jsii` 1.130.0
* Python 3.11.14 · uv 0.9.9 · Node v25.2.1 · Windows-10-10.0.26200-SP0

I observed both winners across repeated environment creations on this one machine, then later creations stopped reproducing the failing winner. **I'm deliberately not quoting a rate** — the variation is unattributed and one machine is not a sample.

### Suggested direction

The packaging defect stands independently of any installer: two separately-installable wheels should not write the same file path. Either the standalone distribution should not ship `aws_cdk/cloud_assembly_schema/__init__.py`, or `aws-cdk-lib` should not, so that install order cannot change program semantics.

Contributor guide

Open the contributing guide

Research direction

Reproduce the collision with the documented `uv init`, `uv add aws-cdk-lib`, and probe script, then inspect the wheel contents for `aws_cdk/cloud_assembly_schema/__init__.py` and `_jsii/`. Done means the two distributions no longer overwrite a shared file path and `CloudFormationStackArtifact.messages` can read synthesis messages regardless of install order.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.