mount=type=cache more in-depth explanation?

Hi, I'm trying to use cache mounts to speed up things like apt install, pip install or ccache when rebuilding my containers.

For each of these, I prefix each RUN command using apt/pip/ccache... with things like:

RUN --mount=type=cache,target=/var/cache/apt,rw --mount=type=cache,target=/var/lib/apt,rw  \

That seems to work sometimes, but I found out I can't understand the cache invalidation rules. The cache just from time to time disappears and I have to download all the cached packages again. I made explicit effort to remove all apt clean commands from the dockerfile, so the cache is not deleted programatically. And it also happens to CCache, which is not autodeleted.

Could somebody please explain how does it work and when exactly can the cache be reused, when (if at all) is it discarded, where is it saved...? What is the exact effect of the id parameter?

I also found out that if I want to create a cache directory for a non-root user, I can create it with cache mount option uid=1000. But if I use uid=1000,gid=1000, the folder gets ownership root:root, which is really weird. Can it be connected with me not using id and setting some caches with uid=0 and some with uid=1000? Does this create some kind of conflict?

And is there actually a good reason for setting the mode to anything else than 777? If it's only used during build, I don't get why anybody would care about permissions...

Thank you for helping.

Maybe some of the answers could be added to https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md ?

Start with frontend/dockerfile/docs/experimental.md and review the existing issue discussion about cache mounts. Document the cache reuse and invalidation behavior, storage location, id parameter, uid/gid interactions, and mode guidance, with the completed documentation added to the referenced file.