bazel-contrib / bazel-contrib/rules_go

Package's ".x" is not reproducible between dependency upgrade

Open
#4,111 11 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1.5k
Forks
760
Avg merge
1d 11h
Merged PRs (30d)
12

Description

This is a tracking issue for https://github.com/golang/go/issues/69547

I was upgrading our external dependencies recently and these 2 GoCompilePkg actions ran on the same package between 2 consecutive commits.

- https://app.buildbuddy.io/invocation/a21c41a4-d2d3-49b5-9fa2-928cb7043d1e?actionDigest=4006f4946e2d116787a3e26a3e2a677611942994ce2992e24cca9741de1f5e5f#action
- https://app.buildbuddy.io/invocation/0bd82434-6213-466b-aa76-e3c68250c92a?actionDigest=bfe3f607ffd5015d1eb076045805c1d074ef71c1454a84c583df589ab1e05f31#action

Specifically, the package is `server/util/db` with a single source file `db.go`. The upgraded dependency was `gorm.io/driver/{mysql,clickhouse,sqlite}`. There was no code change in db.go.

The expectation is for `db.x` output to stay the same between the 2 compilations as no exported data was change. However, we observed that there were some bytes differences between the 2 `.x` files, which caused downstream compile actions to all rebuild.

```bash
~> diffoscope db{1,2}.x
--- db1.x
+++ db2.x
│┄ Command `'nm -s {}'` failed with exit code 1. Standard output:
│┄ /Library/Developer/CommandLineTools/usr/bin/nm: error: for the -s option: bad number of arguments (must be two arguments)
│┄ /Library/Developer/CommandLineTools/usr/bin/nm: error: a.out: No such file or directory
@@ -34357,15 +34357,15 @@
00086340: 0104 0203 0405 1000 8108 0082 0803 7b03 ..............{.
00086350: 4303 0103 0203 4103 8001 0324 034a 031b C.....A....$.J..
00086360: 03d4 0303 d503 03dc 0303 d803 0081 080f ................
00086370: 010d 0203 0405 0607 0809 0a0b 0c0d 0e0b ................
00086380: 0083 0800 8408 0343 034a 03d4 0303 d503 .......C.J......
00086390: 03d8 0303 da03 0341 0365 0083 080a 0108 .......A.e......
000863a0: 0203 0405 0607 0809 1200 8508 0086 0803 ................
-000863b0: bd01 03d4 0303 d804 037b 0341 0380 0103 .........{.A....
+000863b0: bd01 03d8 0403 d403 037b 0341 0380 0103 .........{.A....
000863c0: 4a03 ea04 03d5 0303 d504 03d8 0303 2403 J.............$.
000863d0: ef04 03d6 0403 dc03 0085 0811 010f 0203 ................
000863e0: 0405 0607 0809 0a0b 0c0d 0e0f 1026 0087 .............&..
000863f0: 0800 8808 0328 036f 0355 0358 035a 0302 .....(.o.U.X.Z..
00086400: 0341 0324 03d9 0403 dc04 03dd 0403 0103 .A.$............
00086410: 2303 5703 5903 2903 2503 5f03 6a03 4403 #.W.Y.).%._.j.D.
00086420: 6d03 4a03 1b03 e604 03e7 0403 d402 0380 m.J.............
@@ -113551,9 +113551,9 @@
001bb8e0: 0806 0103 0004 0000 0117 0004 0d00 0017 ................
001bb8f0: 0102 0100 0101 00d0 080f 0002 1700 0f01 ................
001bb900: 0100 1701 040b 0b01 00d0 0815 1701 0001 ................
001bb910: 0103 0000 0c3a 1700 0401 0105 0100 d008 .....:..........
001bb920: 2014 0100 d008 2300 0400 0100 d008 270a .....#.......'.
001bb930: 0100 d108 0400 0117 0004 0101 0500 0100 ................
001bb940: d208 0300 0a01 00d3 0803 0001 1401 00d3 ................
-001bb950: 080a 0004 0001 00d4 0802 907f c3c9 9764 ...............d
-001bb960: 77e3 0a24 240a w..$$.
+001bb950: 080a 0004 0001 00d4 0802 8162 2d2d 5d3d ...........b--]=
+001bb960: fb45 0a24 240a .E.$$.
exit 1
```

I tried using various gcexports tools to decode the 2 files and diff them but the contents are pretty much identical. Consulting upstream Go team for more information on the 7-bytes diff.

Contributor guide

Open the contributing guide

Research direction

Start with upstream Go issue 69547, then reproduce the two GoCompilePkg actions for server/util/db, whose only source file is db.go, across the dependency upgrade. Use diffoscope and the mentioned gcexports tools to isolate the differing bytes and determine whether the .x output is expected to remain stable. Done means identifying the cause and documenting or implementing a confirmed path to reproducible output.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.