Unity-Technologies / Unity-Technologies/UnityDataTools

Binary test data is not fully covered by .gitattributes and can be mangled by EOL conversion

Offen Anfängerfreundlich
#146 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

Vorherrschende Sprache
C#
Sterne
821
Forks
71
Ø Merge
3 Std. 13 Min.
Gemergte PRs (30 T.)
9

Beschreibung

Binary test data under TestCommon/Data is only partly covered by the binary rules in
.gitattributes, so some of it is stored as text and is exposed to end-of-line conversion.

.gitattributes sets * text=auto eol=lf and then names specific binary paths:

assetbundle binary
scenes binary
level* binary
*.dll binary
*.dylib binary
*.so binary
...

Unity data files whose names do not match those patterns fall through to text=auto, which leaves
the decision to git's heuristic — and that only looks for a NUL byte in the first 8000 bytes.

sharedassets0.assets.resS is the clearest case. It is 512 KB with just 8 NUL bytes, none of them
early, so git classifies it as text:

$ git check-attr -a TestCommon/Data/PlayerWithTypeTrees/sharedassets0.assets.resS
... text: auto
... eol: lf

$ git diff --numstat <commit-that-added-it>
1   0   TestCommon/Data/PlayerWithTypeTrees/sharedassets0.assets.resS

(A file git considered binary shows - - there, as the .assets and level* files in the same
folder do.)

Nothing is corrupted today. Both checked-in .resS files happen to contain zero CR bytes, so
eol=lf normalization is a no-op and they round-trip byte for byte — verified by comparing the
SHA-256 of the blob in git against the source file. This is a latent hazard, not a live bug.

The risk is the next binary fixture whose bytes happen to include 0d 0a. On checkout it would
have those bytes rewritten to 0a, producing a corrupt file that still looks plausible, and the
resulting test failure would point at the parser rather than at git. Anything without an extension
already covered by a binary rule is affected — .resS, .resource, .assets, .bundle,
.buildreport, .cf, and the extensionless CAB-* files.

Suggested fix: mark the data folder's binary formats explicitly, e.g.

TestCommon/Data/** -text

or per-extension binary rules for *.resS, *.resource, *.assets, *.bundle, *.buildreport,
*.cf alongside the existing ones. Worth checking afterwards that no already-committed file changes
content (they should not — the ones that would have been mangled are the ones that do not exist
yet), and that the README.md files inside TestCommon/Data are not caught by a blanket rule.

Found while adding Unity 6.7 test data in #145, where the new .resS reproduced the same
classification.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Beginne mit .gitattributes und prüfe die bestehenden Binärregeln zusammen mit TestCommon/Data. Verwende git check-attr für die aufgeführten Unity-Datendateien und verifiziere, dass README.md-Dateien Text bleiben. Bestätige mit git diff --numstat und SHA-256-Vergleichen, dass die gewählte Regel die EOL-Konvertierung verhindert, ohne bestehende committete Inhalte zu ändern.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
git
Bereich
tooling
Issue-Typ
Bug
Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Aktivitätsstatus
Aktiv
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
78/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.