axodotdev / axodotdev/cargo-dist
Make npm package reproducible?
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 149
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 32
Description
While working on #1321 / #1465, I ran into test failures which made me notice the npm-package.tar.gz has a different checksum from one run to the next.
I was tasked by @mistydemeo to make the failure go away to land #1465, and by @ashleygwilliams do investigate why the darn thing wasn't stable to begin with. I'm happy to report the content is stable but things that vary between different runs at different dates / on different platforms are:
* File timestamps
* GID/UID (group/user ID)
* Order of entries in the archive
```diff
diff --git a/linux_contents.txt b/mac_contents.txt
index 2a217a5..b9d8c3e 100644
--- a/linux_contents.txt
+++ b/mac_contents.txt
@@ -1,12 +1,12 @@
-drwxr-xr-x 0 1000 1000 0 Oct 23 15:48 package/
--rw-r--r-- 0 1000 1000 538 Oct 23 15:48 package/README.md
--rw-r--r-- 0 1000 1000 25504 Oct 23 15:48 package/npm-shrinkwrap.json
--rw-r--r-- 0 1000 1000 78 Oct 23 15:48 package/install.js
--rw-r--r-- 0 1000 1000 15 Oct 23 15:48 package/.gitignore
--rw-r--r-- 0 1000 1000 794 Oct 23 15:48 package/CHANGELOG.md
--rw-r--r-- 0 1000 1000 1066 Oct 23 15:48 package/LICENSE-MIT
--rw-r--r-- 0 1000 1000 10845 Oct 23 15:48 package/LICENSE-APACHE
--rw-r--r-- 0 1000 1000 3411 Oct 23 15:48 package/binary.js
--rw-r--r-- 0 1000 1000 2216 Oct 23 15:48 package/package.json
--rw-r--r-- 0 1000 1000 77 Oct 23 15:48 package/run-axolotlsay.js
--rw-r--r-- 0 1000 1000 5727 Oct 23 15:48 package/binary-install.js
+drwxr-xr-x 0 501 20 0 Oct 23 15:55 package/
+-rw-r--r-- 0 501 20 1066 Oct 22 23:37 package/LICENSE-MIT
+-rw-r--r-- 0 501 20 77 Oct 23 15:55 package/run-axolotlsay.js
+-rw-r--r-- 0 501 20 78 Oct 23 15:55 package/install.js
+-rw-r--r-- 0 501 20 2216 Oct 23 15:55 package/package.json
+-rw-r--r-- 0 501 20 15 Oct 23 15:55 package/.gitignore
+-rw-r--r-- 0 501 20 5727 Oct 23 15:55 package/binary-install.js
+-rw-r--r-- 0 501 20 538 Oct 22 23:37 package/README.md
+-rw-r--r-- 0 501 20 25504 Oct 23 15:55 package/npm-shrinkwrap.json
+-rw-r--r-- 0 501 20 3411 Oct 23 15:55 package/binary.js
+-rw-r--r-- 0 501 20 794 Oct 22 23:37 package/CHANGELOG.md
+-rw-r--r-- 0 501 20 10845 Oct 22 23:37 package/LICENSE-APACHE
```
This is all fixable imho, and would take limited effort. I would standardize on 1000 for GID/UID (I'm not sure npm cares either way, you have to go out of your way to get tar to preserve those). Entries can be sorted before being added to the archive.
I genuinely don't understand what's happening with timestamps, are we.. amending .tar files when building them? So the `Oct 22` timestamps are from when I ran a different test yesterday? I fear that might be the case, and I'd like to address that separately.
Contributor guide
Research direction
Start by reproducing the differing npm-package.tar.gz checksums and compare the linux_contents.txt and mac_contents.txt listings. The work is done when archive timestamps, UID/GID metadata, and entry ordering are stable across runs and platforms, with the existing test failures no longer occurring.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, rust
- Domain
- build-system, release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100