Make all generated archives reproducible
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 522
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 137
Description
To be able to validate a release artifact (jar, tarball, zip, etc) against the state of the source tree, the artifacts need to be generated in a reproducible way.
There are a few, totally build related differences as of today:
1. [x] jar files: `META-INF/MANIFEST.MF` have different attributes (**fixed**)
* `Apache-Polaris-Is-Release` (depends on the `-Prelease` vs just `-PjarWithGitinfo` property)
* `Apache-Polaris-Build-Git-Describe` (same as above)
* `Apache-Polaris-Build-Timestamp` - could be changed to the Git commit timestamp using UTC
* `Apache-Polaris-Build-System` - remove this one
* `Apache-Polaris-Build-Java-Version` - remove this one
2. [x] jar files: `META-INF/jandex.idx` are reported to be different (**fixed**)
3. [x] jar files: mtime of `META-INF/MANIFEST.MF` and `META-INF/jandex.idx` vary (**fixed**)
4. [ ] `..../quarkus/generated-bytecode.jar` class files differ. Since Quarkus 3.28.2, the generated bytecode is often the same, but there is still no guarantee.
5. [x] varying order of entries in Quarkus re-assembled `.../app/polaris-*.jar` archives (https://github.com/quarkusio/quarkus/issues/50578)
6. [x] "group write" POSIX permission varies between platforms for zip/tar archive entries (fixed via #2819 + #3146)
With all mentioned issues addressed, it should be possible to build distribution artifacts that match binary, leading to the same sha512.
7. [x] source tarballs have entries with "current mtime". The source tarballs should use a fixed mtime (fixed via #2823)
8. [x] "Top level POM" is not reproducible, because the POM `` and `` elements are dynamically populated at build time (fixed via #2826).
9. [ ] Helm chart package tarball (generated via `helm package`) is not reproducible, `helm package` has no options to "fix" entries' mtime and POSIX attributes. See: Releasy: make Helm package reproducible (#3086)
10. [x] Source-tarball - eliminate git-gzip risk (via #3075)
11. [ ] Build: Ensure reproducible .properties files (via #3089)
12. [X] `git archive --mtime` requires a timezone (via #3143)
13. [x] Non deterministic behavior with `withXml()` for `pom.xml` customization (via #3145)
Contributor guide
Research direction
Start by reviewing the three remaining items: Quarkus generated-bytecode.jar, Helm package tarballs, and generated .properties files; read the linked #3086 and #3089 issues for the Helm and properties work. Done means the remaining generated artifacts are reproducible so distribution builds match binary and produce the same sha512.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, helm, java
- Domain
- build-system, release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100