aspect-build / aspect-build/rules_js
[FR]: Full support for the pnpm file protocol including the ability to link packages outside the main Bazel repository
- Dominant language
- Starlark
- Stars
- 378
- Forks
- 183
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 32
Description
### What is the current behavior?
To my knowledge, `npm_translate_lock` currently has limited support for the `file:` protocol and requires that paths be within the main repository and thus immediately visible to Bazel.
### Describe the feature
I'd like to propose extending support to allow external local packages to be linked in the pnpm lockfile. This way devs can replace registry fetched npm dependencies with local versions on disk which is very useful for testing patches prior to publishing new versions when relying on 2p/3p dependencies which are not vendored inside their monorepo.
This could be achieved under the hood by backing local npm package references with a [`new_local_repository`](https://bazel.build/rules/lib/repo/local)rule and some default BUILD file content that exposes a `pkg` target. That `pkg` target can be an `npm_package` rule which by default would include all sources discoverable via the `npm pack` command.
In order for developers to make use of this feature they would simply replace the version of their 2p/3p dependency with a path to the local repository and then update their pnpm lockfile.
### Internal monorepo package JSON
```json
{
"name": "my-monorepo-app",
"dependencies": {
"lodash": "file:/Users/johndoe/repos/lodash"
}
...
}
```
### pnpm-lock.yaml
```yaml
importers:
packages/my-monorepo-app:
devDependencies:
'lodash':
specifier: file:/Users/johndoe/repos/lodash
version: file:/Users/johndoe/repos/lodash
```
If the `npm_translate_lock` repository rule detects a local path outside the main repository, it would first check to see if a Bazel repository already exists at that path in which case it would back the npm dependency with a `local_repository` rule and assume a `pkg` target has been defined. If no such repository exists, it would back the local link with a `new_local_repository` repo rule and run `npm pack --dry-run` to determine which files would be included when the package is published in order to mimic how the local repository would behave if it were being fetched from a registry.
This workflow is purely for local development purposes.
## Is this already possible with `replace_package` in `npm_translate_lock`?
Yes and no, by extending support for the file protocol to pick up local npm packages outside the main Bazel repo we allow local packages to participate in dependency resolution. `replace_package` overwrites packages post resolution and thus if developers are testing changes in 2p/3p that includes dependency version changes or the introduction of new dependencies they are not currently able to do so without manually updating their pnpm lockfile by some other means.
Currently, in my monorepo we developed a utility script which adds `replace_package` attributes to the `npm_translate_lock` tag class for generated repositories using `new_local_repository`. These attributes are generated from a custom configuration JSON file. The actual `package.json` files that declare these dependencies remain untouched. In order to take care of the limitation where these packages are not able to participate in depdendency resolution we overrwrite package.json contents for locally linked dependencies [using a readPackage hook](https://pnpm.io/pnpmfile) in our `.pnpmfile.cjs`. This is a bit hacky and it has the downside that developers might commit lockfiles that describe dependencies which are inconsistent with the registry state of truth.
Contributor guide
Research direction
Start at npm_translate_lock's handling of the pnpm file: protocol and compare it with replace_package behavior. Review the proposed new_local_repository and local_repository paths, including npm pack --dry-run and the generated pkg target. Done means external local packages can participate in lockfile dependency resolution without the workaround described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100