Unity-Technologies / Unity-Technologies/UnityDataTools

Support calculating TypeTree overhead

Open
#127 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
821
Forks
71
Avg merge
3h 13m
Merged PRs (30d)
9

Description

Some AssetBundle or content directory builds can have a high overhead from the TypeTree data in the SerializedFile header.
This is particularly true if there is a very high granularity of files, and the project contains large MonoBehaivours (large in term of serialization description, the data size is not important)

This task is to update UnityDataTool to make it easier to "see" this overhead and pinpoint details about it (such as the existence of large MonoBehaviours, or widely repeated types). We do expose the raw information about TypeTrees through the serializd-file command, but it is very low level so far.

Option 1: update the Analysis database and capture the data during analysis so that TypeTrees have dedicated table in the database.

From the raw table data it should be made clear how to extract the overall size and duplication statistics (either based on queries in the docs or a custom view)

A POC already exists in this branch:
https://github.com/Unity-Technologies/UnityDataTools/compare/main...typetreeanalysis

That branch is out of data with the recent changed in UnityDataTool and probably needs some design adjustments, but can be a great starting point.

Option 2: update the serialized-file command to print TypeTree stats information in a convenient summary format, and to publish example in the documentation for how to do analysis based on that raw data. E.g. example script.

Option 3: Really low level would be to make use of the data already exposed by serialized-file and have a script that show how to populate that into a dedicated database.

Because the data could get pretty large for large builds i think the first is probably best.

Question: how should we handle a build that has external typetrees? In that case the serialized file have no typetrees so the new tables would be empty. There is probably minimal value to calculating a lot of stats for this case, because by design they are already deduplicated.
Detail: For builds with external typetrees then we can potentially skip creating the typetree tables and views (to avoid extra noise in the analyze database for builds where those stats don't apply)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing typetreeanalysis branch and the raw TypeTree data exposed by the serialized-file command. Compare the analysis-database, summary-output, and scripting options, including how external typetrees should be handled; done means a chosen approach makes TypeTree overhead and duplication details easier to analyze.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
database, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.