`pattern is too long` minimatch error during `electron/universal` auto-detect unpack logic
- Dominant language
- TypeScript
- Stars
- 148
- Forks
- 47
- Avg merge
- 3h 57m
- Merged PRs (30d)
- 5
Description
Hello all, I've encountered this bug repeatedly on universal builds
The issue seems to stem from the `electron/universal` unpack detection.
```
⨯ pattern is too long failedTask=build stackTrace=TypeError: pattern is too long
at assertValidPattern (/Users/mikemaietta/Development/electron-builder-test/node_modules/minimatch/minimatch.js:279:11)
at minimatch (/Users/mikemaietta/Development/electron-builder-test/node_modules/minimatch/minimatch.js:116:3)
at shouldUnpackPath (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:152:8)
at handleFile (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:171:22)
at next (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:199:5)
at createPackageFromFiles (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:203:2)
at Object.createPackageWithOptions (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:57:10)
at mergeASARs (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/universal/src/asar-utils.ts:215:5)
at makeUniversalApp (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/universal/src/index.ts:245:7)
```
### Steps to repro:
- Install `realm` to electron-quick-start project
- Assemble project with electron-builder (which internally uses electron/asar) using `universal` target
### Internal logic:
1. electron-builder is executing `electron/asar` with unpack minimatch (logged from within `electron/asar`'s `shouldUnpackPath` internal function for debugging purposes')
```
{node_modules/realm,/private/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/t-wh5lzR/asar-app-2/node_modules/realm/prebuilds/node/realm.node}
```
4. 2 asars are successfully generated with electron/asar
5. execute `electron/universal` on the two `.apps`.What seems to happen is that `electron/universal` is autodetecting what to unpack, and passes in a minimatch **with over 2.8k minimatch entries**
https://github.com/electron/universal/blob/caa0567b76bc06d0014e6fc5afb6d70d47651971/src/asar-utils.ts#L96-L109
Excerpt:
```
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/PrivacyInfo.xcprivacy
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/RealmJS.podspec
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/bindgen/src/realm_js_jsi_helpers.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/bindgen/src/realm_js_node_helpers.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/bindgen/vendor/realm-core/bindgen/src/realm_helpers.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/CMakeLists.txt
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/build.gradle
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/settings.gradle
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/AndroidManifest.xml
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/cpp/io_realm_react_RealmReactModule.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/cpp/jni_utils.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/cpp/jni_utils.hpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/cpp/platform.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/java/io/realm/react/RealmReactModule.java
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/java/io/realm/react/RealmReactPackage.java
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/java/io/realm/react/util/SSLHelper.java
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/apple/RealmReactModule.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/apple/RealmReactModule.mm
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/apple/platform.mm
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/jsi_externs.hpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/jsi_init.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/jsi_init.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/react_scheduler.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/react_scheduler.h
...waaaay more entries like this
```
### Proposed resolution options:
1. Only pass in the common root folder of each File Set (in this case `/node_modules/realm`) to minimatch (this requires changes to electron/universal detection logic)
2. Bypass minimatch with more explicit control over packaging options, similar as to asar ordering logic. (requires changes in electron/asar) Options:
a. Implement property that allows us to pass `unpackPaths: string[]` which allows explicit control
b. Leverage `options.ordering` configuration of `electron/asar` to pass in JSON config for each file to be unpacked
Personally, *I'd highly advocate for `2b`*, as it's relatively light of a change that is not breaking. AND it also makes sure this minimatch `pattern is too long` issue won't be ever be a roadblock for advanced users. `electron-builder` has a very complex copy-unpack-detection logic flow and we've hit the `pattern is too long` a few times previously, which we solved by implementing the approach of `#1` before passing the unpack minimatch into `electron/asar`, but I can imagine this issue will be hit again if I'm just receiving it on `electron/universal` for an electron-quick-start project installing only `realm`.
### Implementation Proposal:
Right now, the "ordering" file is simply of format:
```
:
```
Right now, electron/asar only pulls the `filename` value before the `:`
https://github.com/electron/asar/blob/121efebd2d02e374713cb513eb56a0e3db9534e9/src/asar.ts#L92-L95
The proposal is to leverage `` to be `JSON.stringify` of Type:
```
{
unpack: boolean // the required property for resolve this issue
// allows for future properties to be added for advanced usage without a breaking change
}
```
Taking this approach allows future configuration options to be allowed on a per-file basis should `electron/asar` need/desire to support more advanced usage
--------------------
[UPDATE] PR Fix https://github.com/electron/asar/pull/350
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.