aws / aws/aws-cdk

framework-integ: we aren't testing what we are shipping

Open
#36,343 0 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/integ-tests aws-cdk-lib bug effort/medium p1
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

When we turned on `"stripInternals": true`, which broke our library for consumers, that should have been caught by the integration tests. But it wasn't. Why not?

The reason is that we aren't compiling against the PUBLIC types of `aws-cdk-lib` in `@aws-cdk-testing/framework-integ`. Instead, we are compiling against the PRIVATE TYPES.

That means that directives like `"stripInternals": true` and `--strip-deprecated` (which remove elements from the `.d.ts` files) haven't been applied yet. It effectively means we are testing a different API surface than the one we are shipping to customers.

### The fix

The primary fix is easy. We can add the following to the config:

```ts
// @aws-cdk-testing/framework-integ/tsconfig.json
{
// ...
"references": [
{ "path": "../../aws-cdk-lib" }
]
}
```

Whenever we import a TypeScript file from a library, the compiler will see both the `.ts` and the `.d.ts` file. The `.ts` file has the source, and the `.d.ts` file has the public types, potentially stripped.

Without this project reference, the compiler will prefer the `.ts` file; with the project reference, it will prefer the `.d.ts` file.

So with the project reference, our integ tests will see the same API surface that our customers will.

### Blockers

We can't trivially do this, because we currently have a number of integ tests for long-deprecated APIs, and they will fail to compile as soon as we consume the public API description where those types have been removed.

Needs this first:

* https://github.com/aws/aws-cdk/issues/36342

Contributor guide

Open the contributing guide

Research direction

Start with @aws-cdk-testing/framework-integ/tsconfig.json and the prerequisite issue #36342. Add the aws-cdk-lib project reference only after deprecated-API integ tests are addressed, then compile the integration tests against the public .d.ts surface. Done means the tests compile while reflecting the API shipped to customers.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
build-system, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.