asset staging: do not recurse into the output directory
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
## :bug: Bug Report
### What is the problem?
I have hit a corner case with the code related to copy directories. It's going in recursively until Node can no longer handle such long file names.
https://github.com/aws/aws-cdk/blob/16eb65867b79798779bf98a0316b1a05915720f1/packages/%40aws-cdk/assets/lib/fs/copy.ts#L7-L18
The offending snippet is when I had a docker asset declared as follows:
```javascript
new DockerImageAsset(this, 'app-image-asset', {
directory: '.',
repositoryName: 'my-repo/app',
});
```
Running `cdk deploy` showed me this error:
```
Error: ENAMETOOLONG: name too long, mkdir 'cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.03918990104d8973acd2ad0abe022844192fcce618cf74f7734a7077f4111d1f'
at Object.mkdirSync (fs.js:750:3)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:39:16)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
```
_(Declaring the directory as its full path did not work neither.)_
This is quite a serious issue for me, personally.
A possible solution is to exclude `cdk.out`. I think it's reasonable to assume that `cdk.out` name is reserved for this project.
As a temporary workaround for me, I've edited the code so that it looks like the following:
```javascript
const exclude = options.exclude || ['cdk.out'];
```
Another possible solution is to avoid recursion if we've already seen the directory in question.
Apparently, it's an issue where the docker build context is the same path as the cdk project.
That said, I'm not sure if the above is enough to reproduce the issue. When I comment the code to create the docker asset, the issue disappears (probably more of a symptom rather than the cause). I've checked if symlinks are the issue (apparently, it's not)
### Environment
- **CDK CLI Version:** 1.6.1
- **OS:** all
- **Language:** all
Contributor guide
Research direction
Start with packages/@aws-cdk/assets/lib/fs/copy.ts, especially the copyDirectory logic linked in the report. Reproduce with a DockerImageAsset using directory '.' and cdk deploy, then verify that staging does not recurse into cdk.out and avoids the reported ENAMETOOLONG failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, typescript
- Domain
- build-system, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100