aws / aws/aws-cdk

core: BundlingDockerImage fails unit tests in Amazon Linux 2023

Open
#27,712 4 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/core bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

When I run unit tests for `aws-cdk-lib` in Amazon Linux 2023, `core/test/staging.test.ts` fails with invalid docker args.

`z,` in docker args is inserted in `bundling.ts > BundlingDockerImage > run > dockerArgs` ([link](https://github.com/aws/aws-cdk/blob/f5391f4938dbd983eaf10f058f2fdb8a2ed15712/packages/aws-cdk-lib/core/lib/bundling.ts#L287)).

`isSelinux()` returns true in Amazon Linux 2023 and unit tests don't consider the case.

### Expected Behavior

The module passes all tests.

### Current Behavior

The unit tests fails with errors:
```
Summary of all failing tests
FAIL core/test/staging.test.ts
● staging › with bundling

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"

310 | const assembly = app.synth();
311 | expect(
> 312 | readDockerStubInput()).toEqual(
| ^
313 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS`,
314 | );
315 | expect(fs.readdirSync(assembly.directory)).toEqual([

at Object. (core/test/staging.test.ts:312:30)

● staging › bundler reuses its output when it can

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"

389 | // We're testing that docker was run exactly once even though there are two bundling assets.
390 | expect(
> 391 | readDockerStubInputConcat()).toEqual(
| ^
392 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS`,
393 | );
394 |

at Object. (core/test/staging.test.ts:391:36)

● staging › uses asset hash cache with AssetHashType.OUTPUT

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"

434 | // and that the hash is based on the output
435 | expect(
> 436 | readDockerStubInputConcat()).toEqual(
| ^
437 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS`,
438 | );
439 |

at Object. (core/test/staging.test.ts:436:36)

● staging › bundler considers its options when reusing bundle output

expect(received).toEqual(expected) // deep equality

- Expected - 2
+ Received + 2

- run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS
+ run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS
- run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated --env UNIQUE_ENV_VAR=SOMEVALUE -w /asset-input alpine DOCKER_STUB_SUCCESS
+ run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated --env UNIQUE_ENV_VAR=SOMEVALUE -w /asset-input alpine DOCKER_STUB_SUCCESS

482 | // operating on the same source asset.
483 | expect(
> 484 | readDockerStubInputConcat()).toEqual(
| ^
485 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS\n` +
486 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated --env UNIQUE_ENV_VAR=SOMEVALUE -w /asset-input alpine DOCKER_STUB_SUCCESS`,
487 | );

at Object. (core/test/staging.test.ts:484:36)

● staging › bundler re-uses assets from previous synths

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"

596 |
597 | expect(
> 598 | readDockerStubInputConcat()).toEqual(
| ^
599 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS`,
600 | );
601 |

at Object. (core/test/staging.test.ts:598:36)

● staging › bundling throws when /asset-output is empty

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS_NO_OUTPUT"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS_NO_OUTPUT"

626 |
627 | expect(
> 628 | readDockerStubInput()).toEqual(
| ^
629 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS_NO_OUTPUT`,
630 | );
631 | });

at Object. (core/test/staging.test.ts:628:30)

● staging › bundling with BUNDLE asset hash type

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"

649 | // THEN
650 | expect(
> 651 | readDockerStubInput()).toEqual(
| ^
652 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS`,
653 | );
654 | expect(asset.assetHash).toEqual('33cbf2cae5432438e0f046bc45ba8c3cef7b6afcf47b59d1c183775c1918fb1f');

at core/test/staging.test.ts:651:30
at Object.fn (../../tools/@aws-cdk/cdk-build-tools/lib/deprecated-symbols.ts:35:5)

● staging › bundling with docker security option

expect(received).toEqual(expected) // deep equality

Expected: "run --rm --security-opt no-new-privileges -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"
Received: "run --rm --security-opt no-new-privileges -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"

674 | // THEN
675 | expect(
> 676 | readDockerStubInput()).toEqual(
| ^
677 | `run --rm --security-opt no-new-privileges ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS`,
678 | );
679 | expect(asset.assetHash).toEqual('33cbf2cae5432438e0f046bc45ba8c3cef7b6afcf47b59d1c183775c1918fb1f');

at Object. (core/test/staging.test.ts:676:30)

● staging › bundling with docker entrypoint

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input --entrypoint DOCKER_STUB_SUCCESS alpine DOCKER_STUB_SUCCESS"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input --entrypoint DOCKER_STUB_SUCCESS alpine DOCKER_STUB_SUCCESS"

699 | // THEN
700 | expect(
> 701 | readDockerStubInput()).toEqual(
| ^
702 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input --entrypoint DOCKER_STUB_SUCCESS alpine DOCKER_STUB_SUCCESS`,
703 | );
704 | expect(asset.assetHash).toEqual('33cbf2cae5432438e0f046bc45ba8c3cef7b6afcf47b59d1c183775c1918fb1f');

at Object. (core/test/staging.test.ts:701:30)

● staging › throws when bundling fails

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input this-is-an-invalid-docker-image DOCKER_STUB_FAIL"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input this-is-an-invalid-docker-image DOCKER_STUB_FAIL"

817 | })).toThrow(/Failed to bundle asset stack\/Asset/);
818 | expect(
> 819 | readDockerStubInput()).toEqual(
| ^
820 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input this-is-an-invalid-docker-image DOCKER_STUB_FAIL`,
821 | );
822 | });

at Object. (core/test/staging.test.ts:819:30)

● staging › bundling still occurs with partial wildcard

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"

1108 |
1109 | expect(
> 1110 | readDockerStubInput()).toEqual(
| ^
1111 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS`,
1112 | );
1113 | expect(asset.assetHash).toEqual('33cbf2cae5432438e0f046bc45ba8c3cef7b6afcf47b59d1c183775c1918fb1f'); // hash of MyStack/Asset

at Object. (core/test/staging.test.ts:1110:30)

● staging › bundling still occurs with a single wildcard

expect(received).toEqual(expected) // deep equality

Expected: "run --rm -u 1000:1000 -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"
Received: "run --rm -u 1000:1000 -v /input:/asset-input:z,delegated -v /output:/asset-output:z,delegated -w /asset-input alpine DOCKER_STUB_SUCCESS"

1132 |
1133 | expect(
> 1134 | readDockerStubInput()).toEqual(
| ^
1135 | `run --rm ${USER_ARG} -v /input:/asset-input:delegated -v /output:/asset-output:delegated -w /asset-input alpine DOCKER_STUB_SUCCESS`,
1136 | );
1137 | expect(asset.assetHash).toEqual('33cbf2cae5432438e0f046bc45ba8c3cef7b6afcf47b59d1c183775c1918fb1f'); // hash of MyStack/Asset

at Object. (core/test/staging.test.ts:1134:30)

Test Suites: 1 failed, 681 passed, 682 total
Tests: 12 failed, 3 skipped, 11166 passed, 11181 total
Snapshots: 20 passed, 20 total
Time: 157.228 s, estimated 158 s
Ran all test suites.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
```

### Reproduction Steps

```
cd packages/aws-cdk-lib
cd yarn build
cd yarn test
```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.103.0

### Framework Version

_No response_

### Node.js Version

v20.9.0

### OS

Amazon Linux 2023

### Language

TypeScript

### Language Version

_No response_

### Other information

docker version:
```
$ docker version
Client:
Version: 24.0.5
API version: 1.43
Go version: go1.20.10
Git commit: ced0996
Built: Fri Oct 13 00:00:00 2023
OS/Arch: linux/amd64
Context: default

Server:
Engine:
Version: 24.0.5
API version: 1.43 (minimum version 1.12)
Go version: go1.20.10
Git commit: a61e2b4
Built: Fri Oct 13 00:00:00 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.2
GitCommit: 0cae528dd6cb557f7201036e9f43420650207b58
runc:
Version: 1.1.7
GitCommit: f19387a6bec4944c770f7668ab51c4348d9c2f38
docker-init:
Version: 0.19.0
GitCommit: de40ad0
```

Contributor guide

Open the contributing guide

Research direction

Read packages/aws-cdk-lib/core/lib/bundling.ts at BundlingDockerImage.run and inspect the related expectations in packages/aws-cdk-lib/core/test/staging.test.ts. Run the staging unit tests in Amazon Linux 2023, focusing on the dockerArgs produced by isSelinux(). Done means the tests pass and the generated Docker arguments match the expected output.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, typescript
Domain
build-system, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.