JakeWharton / JakeWharton/diffuse

Handle zip entry mismatch for APKs and JARs

Open
#560 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Kotlin
Stars
2.2k
Forks
118
Avg merge
2h 18m
Merged PRs (30d)
19

Description

While investigating https://issuetracker.google.com/issues/553673285 I found that diffuse gets very confused when the contents of the APK is the same, but the zip entry order is different. It makes it looks like the files are changing, but they are not.

```
$ ./diffuse diff --apk /usr/local/google/home/aurimas/Downloads/androidTest2/datastore-datastore-core-androidDeviceTest.apk /usr/local/google/home/aurimas/Downloads/androidTest1/datastore-datastore-core-androidDeviceTest.apk
OLD: datastore-datastore-core-androidDeviceTest.apk (signature: V2)
NEW: datastore-datastore-core-androidDeviceTest.apk (signature: V2)

│ compressed │ uncompressed
├────────────┬───────────┬────────────┼────────────┬────────────┬──────
APK │ old │ new │ diff │ old │ new │ diff
──────────┼────────────┼───────────┼────────────┼────────────┼────────────┼──────
dex │ 4.85 MiB │ 4.85 MiB │ 0 B │ 12.87 MiB │ 12.87 MiB │ 0 B
arsc │ 1.51 KiB │ 1.51 KiB │ 0 B │ 1.4 KiB │ 1.4 KiB │ 0 B
manifest │ 2.03 KiB │ 2.03 KiB │ 0 B │ 7.89 KiB │ 7.89 KiB │ 0 B
res │ 0 B │ 0 B │ 0 B │ 0 B │ 0 B │ 0 B
native │ 101.13 KiB │ 96.95 KiB │ -4.18 KiB │ 29.54 KiB │ 29.54 KiB │ 0 B
asset │ 0 B │ 0 B │ 0 B │ 0 B │ 0 B │ 0 B
other │ 110.24 KiB │ 76.53 KiB │ -33.71 KiB │ 219.75 KiB │ 219.75 KiB │ 0 B
──────────┼────────────┼───────────┼────────────┼────────────┼────────────┼──────
total │ 5.06 MiB │ 5.03 MiB │ -37.89 KiB │ 13.12 MiB │ 13.12 MiB │ 0 B

│ raw │ unique
├───────┬───────┬──────┼───────┬───────┬───────────
DEX │ old │ new │ diff │ old │ new │ diff
─────────┼───────┼───────┼──────┼───────┼───────┼───────────
files │ 13 │ 13 │ 0 │ │ │
strings │ 73942 │ 73942 │ 0 │ 68202 │ 68202 │ 0 (+0 -0)
types │ 10832 │ 10832 │ 0 │ 9543 │ 9543 │ 0 (+0 -0)
classes │ 8632 │ 8632 │ 0 │ 8632 │ 8632 │ 0 (+0 -0)
methods │ 72154 │ 72154 │ 0 │ 70568 │ 70568 │ 0 (+0 -0)
fields │ 17584 │ 17584 │ 0 │ 17428 │ 17428 │ 0 (+0 -0)

ARSC │ old │ new │ diff
─────────┼─────┼─────┼──────
configs │ 4 │ 4 │ 0
entries │ 4 │ 4 │ 0

=================
==== APK ====
=================

compressed │ uncompressed │
───────────┬────────────┼──────────┬──────┤
size │ diff │ size │ diff │ path
───────────┼────────────┼──────────┼──────┼─────────────────────────────────────────────
│ -33.71 KiB │ │ 0 B │ -
26.72 KiB │ -4.18 KiB │ 7.24 KiB │ 0 B │ ∆ lib/x86_64/libdatastore_shared_counter.so
───────────┼────────────┼──────────┼──────┼─────────────────────────────────────────────
26.72 KiB │ -37.89 KiB │ 7.24 KiB │ 0 B │ (total)
```

APKs can be found in https://issuetracker.google.com/issues/553673285 if you want to repro locally.

This is important because sadly both APKs and JARs can sometimes be produced in a non-determistic file order. It would be awesome if diffused understood that and stated "File entry order has changed" or something along those lines.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the reported output with the APKs linked in the issue, then trace the APK/JAR archive comparison entry point to see how differing zip order is interpreted. Done means identical contents no longer appear as changed solely because entry order differs, and the output explicitly reports that file entry order changed.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
cli, devtools
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.