mapbox / mapbox/mason

immutable dependencies

Open
#67 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
258
Forks
68
PR merge metrics
No merged PRs in 30d

Description

We had several days of build instability in mapbox-gl-native and mapbox-gl-js due to unexpected changes in upstream mason dependencies. In some cases the problem was that a mason branch was deleted while projects still used it, in other cases it was that the build script changed in ways that broke certain platforms. What's common to both cases is that they revealed an architectural weakness in how we manage dependencies with mason: there's no way to test or stage changes to packages without potentially breaking downstream projects.

This is closely related to an architecture invariant that I think every package system comes to adopt (or regret not adopting) at some point: once published, a particular version of a package should _never_ change, period. Being able to overwrite binaries or install scripts for a particular version is just asking for trouble -- namely, uncontrollable breakage of downstream projects.

Let's consider:
- Uploading build / install scripts to S3 along with binaries, so there's no direct dependency on particular git repositories or branches being available.
- Suffixing the package version number with a monotonically increasing "build" number, e.g. mesa-10.4.3-1, mesa-10.4.3-2.
- Requiring that dependencies be specified by package version and build number.
- Never overwriting a package-build version on S3.

This would ensure that we can safely iterate on packages in mason without risking uncontrollable downstream breakage, and downstream packages get reproducible builds and the assurance that dependencies won't change out from underneath them.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file, test, or entry point is named. Start by locating Mason's package publishing and dependency-resolution paths, then compare how S3 artifacts, package versions, build numbers, and branch-based dependencies are currently handled; done means a decided design and implementation plan for immutable package builds and reproducible downstream installs.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, git, python
Domain
build-system, devops, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.