Unity-Technologies / Unity-Technologies/UnityDataTools
Support calculating TypeTree overhead
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 821
- Forks
- 71
- Avg merge
- 3h 13m
- Merged PRs (30d)
- 9
Description
Some AssetBundle or content directory builds can have a high overhead from the TypeTree data in the SerializedFile header.
This is particularly true if there is a very high granularity of files, and the project contains large MonoBehaivours (large in term of serialization description, the data size is not important)
This task is to update UnityDataTool to make it easier to "see" this overhead and pinpoint details about it (such as the existence of large MonoBehaviours, or widely repeated types). We do expose the raw information about TypeTrees through the serializd-file command, but it is very low level so far.
Option 1: update the Analysis database and capture the data during analysis so that TypeTrees have dedicated table in the database.
From the raw table data it should be made clear how to extract the overall size and duplication statistics (either based on queries in the docs or a custom view)
A POC already exists in this branch:
https://github.com/Unity-Technologies/UnityDataTools/compare/main...typetreeanalysis
That branch is out of data with the recent changed in UnityDataTool and probably needs some design adjustments, but can be a great starting point.
Option 2: update the serialized-file command to print TypeTree stats information in a convenient summary format, and to publish example in the documentation for how to do analysis based on that raw data. E.g. example script.
Option 3: Really low level would be to make use of the data already exposed by serialized-file and have a script that show how to populate that into a dedicated database.
Because the data could get pretty large for large builds i think the first is probably best.
Question: how should we handle a build that has external typetrees? In that case the serialized file have no typetrees so the new tables would be empty. There is probably minimal value to calculating a lot of stats for this case, because by design they are already deduplicated.
Detail: For builds with external typetrees then we can potentially skip creating the typetree tables and views (to avoid extra noise in the analyze database for builds where those stats don't apply)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing typetreeanalysis branch and the raw TypeTree data exposed by the serialized-file command. Compare the analysis-database, summary-output, and scripting options, including how external typetrees should be handled; done means a chosen approach makes TypeTree overhead and duplication details easier to analyze.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- database, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100