posit-dev / posit-dev/positron
Builds: Compile built-in extensions only once (improve build speed & upstream alignment)
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 4.3k
- Forks
- 184
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 206
Description
Currently, the Positron build scripts create a universal macOS binary by doing a full build of arm64 and a full build of x86, and then stitching them together.
This sort of works, but it requires some hacks. For example, some of the build artifacts aren't compatible with this approach because the process that produces them is not stable. In order for us to stitch together the two independent builds using create-universal-app, every file must match, or support being fused. This is the reason for this code, which, while not outright smelly, does at least have a very unappetizing aroma.
This isn't the way that VS Code works; its core TypeScript compilation step happens only once. The compilation results, which aren't architecture-specific, are cached and reused for the arm64 build, the x86 build, and the Universal build. Here is where the compilation artifact is downloaded:
We cannot do this in Positron, however, because the extensions that we have added contain binaries that are architecture dependent. For example the jupyter-adapter extension uses an architecture-specific build of ZeroMQ, and the positron-r extension contains an architecture-specific build of the ark kernel.
In VS Code 1.80, a change was made (which I have thus far been unable to pinpoint) which causes further divergence of the arm64 and x86 files, such that even the minified Javascript is different between the builds.
Error: Expected all non-binary files to have identical SHAs when creating a universal build but "Contents/Resources/app/extensions/emmet/dist/node/emmetNodeMain.js" did not
at exports.makeUniversalApp (/Users/user229818/actions-runner/_work/positron/positron/build/node_modules/vscode-universal-bundler/dist/cjs/index.js:73:23)
at runMicrotasks (<anonymous>)
In order to update to 1.80 and beyond, we need to sort out our build system so that it is more like VS Code's, while still accommodating our use case. Here are some ideas:
- Put Positron's extensions into a separate build task and let that build task run per architecture. Keep VS Code's built-in extensions in their own build task that only runs once, just like it does in the upstream build system.
- Let all the extensions share a build task as they do today. For the Universal build, however, don't try to fuse the output of two different compilations. Instead start with the output of the amd64 build task, but overlay the native code build for x64 right before producing a Universal app.
- Adopt VS Code's build-once approach, producing a single
compilation.tar.gzfile. Hold ourselves to the same constraints VS Code does: do not include any native code in built-in extensions. Native binaries, likearkandzeromq, must be separately built and loaded into e.g.Resources. - Run away from home and live in the woods.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Compare the macOS workflow section in .github/workflows/build-release-macos.yml with the upstream compilation artifact handling referenced in build/azure-pipelines/darwin/product-build-darwin.yml. First determine which built-in extensions and native binaries differ by architecture; done means built-in extensions compile once while architecture-specific binaries remain correctly included in the arm64, x86, and Universal builds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, macos, typescript
- Domain
- build-system, devops
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100