Size of the typst/packages repository

The typst/packages repository takes 1.9Gio on my machine with a current clone. As Typst gets more popular, its size will increase at a higher-than-linear speed, and there is a risk that it becomes painful in practice to operate with the package repository: at some point, people with low bandwidth will have trouble cloning the repository to contribute their own package.

The size of the repository is currently roughly:

• 500MiB of git metadata
• 1.4GiB of package data

(in particular, doing a shallow clone will not help much)

On my current checkout of the repository, there are

• 585 packages in total
• 343 packages which take less than 1Mio of disk space, they consume 117Mio in total
• 219 that take between 1Mio and 10Mio, they consume 724Mio in total
• 23 which take more than 10Mio, they consume 567Mio in total

In the short term, the following could work:

• in packaging guidelines, encourage people to stick to small packages below 1Mio (maybe templates need different recommendations)
• replace identical asset files by symbolic links, to avoid duplication of assets across different versions

Replacing identical asset files by symbolic links can be done by package authors if they are told how to do it, or by repository maintainers after the fact. (git already deduplicates its internal data, so it is not strictly necessary to do it at package-submission time.) A quick experiment suggests that doing this with the current repository should shrink its size from 1.4Gio to 947Mio, which is a sizeable win.

In the long term, I think that repository maintainers should maybe consider git-lfs or other options. The end goal would be that package authors do not need to download all other packages to submit theirs.

Start by measuring the repository's git metadata and package data separately, using the size breakdown in the issue as a baseline. Review the packaging guidelines and compare the proposed symlink approach with longer-term git-lfs or similar options; done means an agreed strategy that reduces repository growth and avoids requiring package authors to download all other packages.