Shorten the End-to-End test suite execution without reducing coverage

**Describe the problem/challenge you have**
It's taking upwards of 20+ minutes for a single E2E run (see [test-gh workflow](https://github.com/vmware-tanzu/carvel-kbld/actions/workflows/test-gh.yml?query=is%3Asuccess)).

This stretches the feedback loop for local development, for CI... it's waste from delays.

**Describe the solution you'd like**
The faster the better.

This might involve digging in to see where the long lead times are coming from and finding ways to optimize those operations.
Whatever we do, we cannot reduce the confidence we get from running the suite.

---
Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.