Unity-Technologies / Unity-Technologies/UnityDataTools
Support repacking Archives (Lz4 compression configuration)
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 821
- Forks
- 71
- Avg merge
- 3h 13m
- Merged PRs (30d)
- 9
Description
With #62 the archive command is exposing the blocks and directory section of the archive metadata.
This task is to add an option for "repacking" an existing Archive file (e.g. an AssetBundle) into a freshly compressed Archive that has the same content.
Unity already has AssetBundle.RecompressAssetBundleAsync which can handle some transformation, for example compressed to uncompressed.
However currently there is no way to influence how files are split into individual blocks for the block based compression (LZ4 and LZ4HC). Unity always uses 128KB block size and will span over file boundaries.
This task is to add a command for repacking an archive. It could take the following configuration parameters:
- more variability for the block sizes (max block size uncompressed (input) data, max block size compressed data)
- how file boundaries are handled (span or always split)
- padding between files
- minimum compression factor (at what ratio to fallback to uncompressed) - avoiding on a block by block based an extra layer of compression when the data is already compressed.
This would allow some optimization, including picking configurations specifically fine tuned to the target platform's characteristics.
In my understanding so long as the Block and Directory data structures are properly populated then the Unity Runtime will be able to load the archive data properly, e.g that it already could support reading a wider range of blocking.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the archive command and the block and directory metadata exposed by #62. Define how an existing Archive is read and repacked while preserving its content, then specify configuration for block sizes, file boundaries, padding, and compression fallback; done means Unity can load the resulting archive.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, unity
- Domain
- cli, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100