boltpkg / boltpkg/bolt

Improve performance of symlinking

Open
#201 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.4k
Forks
97
PR merge metrics
No merged PRs in 30d

Description

Right now symlinking includes a _lot_ of intensive file operations:

https://github.com/boltpkg/bolt/blob/d2ca298ed057b4b9c2f05fe4e6d45f645ddc04e2/src/utils/fs.js#L48-L109

Basically every single symlink() we create we do the following operations:

```js
await mkdirp(dest)
await lstat(dest)
await rimraf(dest)
await symlink(dest, target)
```

This is _terrible_ for performance, with 500 Bolt packages needing just 10 symlinks each we have 5,000 symlinks which turns into 20,000 file operations not including the overhead of Node/libuv and internal operations of mkdirp/rimraf/etc

As part of running the install process we aggregate all of the symlinks we need to create, so I think we could make this much more optimized by manually writing out only the operations we need.

We might be able to optimize it even more by doing `readdir()`'s with `withFileTypes` and either assuming the symlinks are good or using `readlink()` to verify them cheaply

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.