core: Over-aggressive cache fingerprinting on synthesize
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
It appears that `cdk synth` triggers a call to `core/lib/fs/fingerprint.js`, specifically the function contentFingerprintMiss(file). This includes files that would normally be ignored, such as those in `.git` or `.idea` (or whatever IDE project folder you use).
This call can cause problems when some of these files are locked. For example, `.idea` now contains a lock file if the CoPilot plugin is enabled, meaning a `cdk synth` run on any project open in a current version of any Windows install of IntelliJ IDE (e.g. Webstorm) with the CoPilot extension may now fail, with a resulting error message starting with:
`Error: EBUSY: resource busy or locked, read`
and ending with
```
{
errno: -4082,
code: 'EBUSY',
syscall: 'read'
}
```
I don't know if it is relevant but this particular project I discovered this behavior on was also a mono-repo using pnpm.
### Expected Behavior
The `cdk synth` command should not fail when a file path that is not relevant to the cdk stack contains a locked file.
### Current Behavior
A file path containing a locked file in the same project as the cdk may cause the a `cdk synth` command to fail.
### Reproduction Steps
Unfortunately I don't have a code snippet to reproduce this and my project is private so I can't share it. Building a simple cdk project does not appear sufficient to trigger the issue but my complex pnpm monorepo project reproduces it consistently.
### Possible Solution
I would suggest a configuration (in cdk.json or in a .cdkignore file) that controls the scope of the cdk cache/fingerprint better. It should not be looking in `.git` or `.idea` (and probably other places) anyway. Similar to a `.gitignore` file, there should be a way to tell `cdk synth` which paths should be excluded entirely.
### Additional Information/Context
When troubleshooting this I added a log message to core/lib/fs/fingerprint.js as follows:
```
function contentFingerprintMiss(file) {
console.log(`Fingerprinting file: ${file}`);
```
Then when I do a `cdk synth` I can see a list of all the files getting called with a fingerprint miss. That list includes many files under `.git` and `.idea` (I use WebStorm), that `cdk synth` should not even care about. See the output snippet below where it hits the `xd.lck` file that is part of the copilot extension:
```
...
Fingerprinting file: C:\Users\JasonGillam\Projects\my-project\.idea\copilot\chatSessions\00000000000.xd
Fingerprinting file: C:\Users\JasonGillam\Projects\my-project\.idea\copilot\chatSessions\blobs\version
Fingerprinting file: C:\Users\JasonGillam\Projects\my-project\.idea\copilot\chatSessions\xd.lck
Error: EBUSY: resource busy or locked, read
at Object.readSync (node:fs:752:18)
at contentFingerprintMiss (C:\Users\JasonGillam\Projects\my-project\node_modules\.pnpm\aws-cdk-lib@2.121.1_constructs@10.3.0\node_modules\aws-cdk-lib\core\lib\fs\fingerprint.js:78:25)
...
```
I tested disabling the copilot extension, and this also resolves the issue, since it removes the lock file.
### CDK CLI Version
2.131.0
### Framework Version
_No response_
### Node.js Version
20.11.1
### OS
Windows 11
### Language
TypeScript
### Language Version
"typescript": "~5.3.3"
### Other information
_No response_
Contributor guide
Research direction
Start by reading core/lib/fs/fingerprint.js, especially contentFingerprintMiss(file), and trace how cdk synth selects files for fingerprinting. Compare the behavior of .git and .idea paths with the proposed cdk.json or .cdkignore configuration. Done means cdk synth no longer reads irrelevant locked files and still fingerprints relevant project files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100