Sharing tools in different git worktrees

This might be a corner use case, but because of the way my editor works (emacs w/ projectile) I work on separate features in different worktrees.
Would it be possible to keep tools in a bazel cache place, indexed by the hash of the lock file (a bit too coarse, but acceptable) so that one doesn't have to rebuild in each worktree? Or maybe there's already a way I have missed.