Unity-Technologies / Unity-Technologies/UnityDataTools
CRC of objects with references is not comparable across separate databases
还没有人认领这个 Issue。
- 主要语言
- C#
- 星标
- 821
- 派生
- 71
- 平均合并
- 3 小时 13 分钟
- 30 天内合并 PR
- 9
描述
Summary
objects.crc32 is meant to be a content fingerprint, and the comparing-builds workflow expects you can analyze two builds into two separate databases and diff CRCs to find which objects changed. That works for leaf assets (Texture2D/Mesh/AudioClip — no references), but it is broken for any object that contains references (Materials, prefabs/GameObjects, MonoBehaviours, etc.): identical content produces different CRCs in two separate analyze runs.
Cause
When PPtrAndCrcProcessor.ExtractPPtr folds a reference into the CRC, it uses the resolved analyzer/database object id returned by the callback, not the PPtr's own identity:
var refId = m_Callback(m_ObjectId, fileId, pathId, ...); // analyzer db id
m_Crc32 = Crc32Algorithm.Append(m_Crc32, <refId bytes>);
That id comes from ObjectIdProvider.GetId((m_LocalToDbFileId[fileId], pathId)), and both the serialized-file id and the object id are assigned sequentially per analyze run. So the same logical object gets different ids in db1 vs db2 → different CRC for identical content → cross-database comparison reports spurious differences for every object that has references.
Why we can't just hash the raw PPtr (the tradeoff)
The obvious fix is to hash the raw on-disk PPtr (fileId + pathId) instead of the resolved id. But the resolved id is currently what makes within-database duplicate detection (view_potential_duplicates) work across bundles: two copies of the same object in different bundles reference the same target, and resolving through m_LocalToDbFileId (keyed by filename) + pathId normalizes them to the same id → same CRC → detected as duplicates.
fileId is a local index into a serialized file's external-reference list, so two copies of an object in different bundles can have different fileId values for the same target. Hashing the raw PPtr would therefore weaken duplicate detection. Deduplication is an important feature and is probably not well covered by tests yet, so we don't want to risk regressing it.
Options to evaluate
- Raw PPtr (
fileId+pathId) — simplest; fixes cross-db comparison in the common case; risks weakeningview_potential_duplicates(localfileIddiffers between bundles). - Stable target identity +
pathId— resolvefileIdto a stable identifier for the target file and hash that +pathId, so it is independent of the local index. This fixes cross-db comparison AND preserves cross-bundle duplicate detection, but the "stable identifier" differs by source:- Build output external references carry a path (e.g.
archive:/CAB-...), not a GUID. - Editor / Library references carry a GUID (the source asset's GUID).
So the CRC needs to mix in whichever ofExternalReference.Path/ExternalReference.Guidis populated (and a fixed marker for local refs,fileId == 0). Relies on those fields being present and stable.
More code: thread the external-reference info fromsf.ExternalReferencesinto the CRC.
- Build output external references carry a path (e.g.
- Status quo — cross-db comparison stays broken for referenced objects.
Prerequisite
Add test coverage for view_potential_duplicates / cross-bundle deduplication before changing the CRC, so a fix can be validated to not regress it.
Context
Discovered while reviewing #73 / #70. Note that this is independent of the CRC changes made there (the ManagedReferenceData size fix, the ComputeCRC chunking fix, and the cah:/ stream hashing) — those also change CRC values vs. older tool versions, so CRCs are not comparable across tool versions regardless.
Related: #44 (refs table).
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
先阅读 PPtrAndCrcProcessor.ExtractPPtr 和 ObjectIdProvider.GetId,跟踪已解析引用如何进入 CRC,然后检查 sf.ExternalReferences 和 view_potential_duplicates 工作流。建立跨数据库 CRC 相等性和跨 bundle 重复项检测的测试;完成的标准是被引用对象能够一致地进行比较,同时不削弱去重功能。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- csharp, unity
- 领域
- databases, devtools, testing-qa
- Issue 类型
- 缺陷
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 活跃
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100