bazel-contrib / bazel-contrib/bazel-lib

[Bug]: copy_directory copies .DS_Store leading to remote cache misses

Open
#887 8 comments 0 reactions 1 assignee Claimed by @gregmagolan View on GitHub
bug
Dominant language
Starlark
Stars
182
Forks
134
Avg merge
1d 46m
Merged PRs (30d)
1

Description

### What happened?

`.DS_Store` is a file sometimes added to a directory on MacOS by the operating system (it has to do with file system indexing I think). Sometimes these `.DS_Store` sneak themselves into npm package repositories (generated by `rules_js`) then copied with this action here:

https://github.com/aspect-build/rules_js/blob/d0ff155c73e3c7fee5d72485e00775bca1fde10a/npm/private/npm_package_store.bzl#L222-L232

I’m some debugging remote cache misses on MacOS and when I look at the execution log, I’m seeing `.DS_Store` appearing in the diff. Example from one of the execution log diffs I generated following “[Debugging Remote Cache Hits for Remote Execution](https://bazel.build/remote/cache-remote)”:

```diff
...

@@ -2229270,14 +2229002,6 @@
hash_function_name: "SHA-256"
}
is_tool: true
-}
-inputs {
- path: "external/npm__phosphor-react__1.4.1__-440667795/package/.DS_Store"
- digest {
- hash: "14dcaaf35f98cb101936646924d50e812d07e260ef2ccea177c1d268e1561200"
- size_bytes: 6148
- hash_function_name: "SHA-256"
- }
}
inputs {
path: "external/npm__phosphor-react__1.4.1__-440667795/package/LICENSE"
@@ -2262980,14 +2262704,6 @@
cacheable: true
mnemonic: "CopyDirectory"
actual_outputs {
- path: "bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/phosphor-react@1.4.1_-440667795/node_modules/phosphor-react/.DS_Store"
- digest {
- hash: "14dcaaf35f98cb101936646924d50e812d07e260ef2ccea177c1d268e1561200"
- size_bytes: 6148
- hash_function_name: "SHA-256"
- }
-}
-actual_outputs {
path: "bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/phosphor-react@1.4.1_-440667795/node_modules/phosphor-react/LICENSE"
digest {
hash: "bd618db104d07526fe78a9c28dbc2cc4c4286d756aa8b428e51b82de0f8d6aeb"

...
```

I know this is `copy_directory` related since when I go to the log file this is from and look for the command I see this snippet:

```
...

---------------------------------------------------------

command_args: "external/copy_directory_darwin_arm64/copy_directory"
command_args: "external/npm__phosphor-react__1.4.1__-440667795/package"
command_args: "bazel-out/darwin_arm64-fastbuild/bin/node_modules/.aspect_rules_js/phosphor-react@1.4.1_-440667795/node_modules/phosphor-react"
platform {
}
inputs {
path: "external/copy_directory_darwin_arm64/copy_directory"
digest {
hash: "eddc1d4e6a5142106850e1466e4cd384a343ca8be04579dc0add52e5d4f63ac7"
size_bytes: 1507746
hash_function_name: "SHA-256"
}
is_tool: true
}
inputs {
path: "external/npm__phosphor-react__1.4.1__-440667795/package/.DS_Store"
digest {
hash: "14dcaaf35f98cb101936646924d50e812d07e260ef2ccea177c1d268e1561200"
size_bytes: 6148
hash_function_name: "SHA-256"
}
}
inputs {
path: "external/npm__phosphor-react__1.4.1__-440667795/package/LICENSE"
digest {
hash: "bd618db104d07526fe78a9c28dbc2cc4c4286d756aa8b428e51b82de0f8d6aeb"
size_bytes: 1092
hash_function_name: "SHA-256"
}
}

...
```

How could I configure `copy_directory()` to ignore `.DS_Store` files if they’re present to prevent remote caching misses? Ideally I think `copy_directory()` should probably ignore `.DS_Store` files automatically since it’s very unlikely they’re contributing to the build.

I haven’t proven this is the incompatibility between the two MacOS machines which is causing the remote cache miss but figured it may be worth addressing whether or not it’s the incompatibility causing my remote caching misses.

### Version

Development (host) and target OS/architectures: MacOS arm64

Output of `bazel --version`:

```
Bazelisk version: v1.20.0
Build label: 7.2.0
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Mon Jun 10 13:04:55 2024 (1718024695)
Build timestamp: 1718024695
Build timestamp as int: 1718024695
```

Version of the Aspect rules, or other relevant rules from your
`WORKSPACE` or `MODULE.bazel` file:

- `aspect_bazel_lib` v1.42.3
- `rules_nodejs` v5.8.4
- `aspect_rules_js` v1.42.3

Language(s) and/or frameworks involved: Node.js, JavaScript, pnpm

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.