Duplicate Secret Error when Fetching Multiple Private Git Repos with Same Secret

**What steps did you take:**
I have an App that fetches two private git repos. They both use the same secret for auth.
```
...
fetch:
- git:
url: https://dev.azure.com/example/tap-lab/_git/gitops-platform
ref: origin/1.4
path: clusters/full/app-contents/desired-namespaces
secretRef:
name: git-creds-carvel
- git:
url: https://dev.azure.com/example/tap-lab/_git/gitops-platform
ref: origin/1.4
path: clusters/full/app-contents/desired-namespaces2
secretRef:
name: git-creds-carvel
....
```

**What happened:**
I get a reconcilation error
Useful Error Message: vendir: Error: Syncing directory '0': │ Syncing directory '.' with git contents: │ Fetching git repository: │ Expected to find one secret 'git-creds-carvel', but found multiple

**Environment:**

- kapp Controller version (execute `kubectl get deployment -n kapp-controller kapp-controller -o yaml` and the annotation is `kbld.k14s.io/images`): v0.38.4
- Kubernetes version (use `kubectl version`): v1.23