Fetch Failed: Repository Not Found

**Describe the bug**
I have a feeling that this might have something to do with #2096 , but i still wanted to report it anyways in case someone knows a solution. I am able to successfully "fetch", but unable to "pull/push" to ONLY a particular organization's repo. I have 2 GH accounts, one personal and one business. The personal one uses my default `~/.ssh/config` file. However, since the addition of my business account, I had to create an additional config `~/.ssh/business_config` (due to the business having an enterprise account still using github.com). Using terminal hooks, I am able to switch which config file to use based on the directory that I am in. For example, If I am in a directory that has the name of the business as a parent directory, then the `~/.ssh/business_config` file becomes `~/.ssh/config`, while the default/original config gets named something else, and then switches back if the previous condition is not met.

The above bug, is NOT an issue for all other repos, i.e when I am NOT in a business related directory/repo everything works.

**To Reproduce**
Steps to reproduce the behavior:
1. gitui (when in a directory that makes a different config file the `~/.ssh/config`)
2. Pull or Push
3. See error

**Expected behavior**
Ability to pull and push just like I can when not inside the business directory.

**Screenshots**

**Context (please complete the following information):**
- macOS Sonoma 14.4.1
- gitui nightly 2024-04-17 ()
- rustc 1.77.0 (aedd173a2 2024-03-17)

**Additional context**
Log output of the steps above:

```
09:47:53 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:154] open repo at: RefCell { value: Path(".") }
09:47:53 [TRACE] (6) mio::poll: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.11/src/poll.rs:551] registering event source with poller: token=Token(0), interests=READABLE
09:47:53 [TRACE] (6) mio::poll: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mio-0.8.11/src/poll.rs:551] registering event source with poller: token=Token(1), interests=READABLE
09:47:53 [TRACE] (1) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:102] request: [hash: 5495339667515457528] (type: WorkingDir)
09:47:53 [TRACE] (1) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:102] request: [hash: 7187341517787738567] (type: Stage)
09:47:53 [TRACE] (1) gitui: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/main.rs:214] app start: 16 ms
09:47:53 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:375] update
09:47:53 [TRACE] (1) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:96] request blocked, still pending
09:47:53 [TRACE] (1) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:96] request blocked, still pending
09:47:53 [TRACE] (3) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:160] status fetched: 7187341517787738567 (type: Stage)
09:47:53 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Status)
09:47:53 [TRACE] (2) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:160] status fetched: 5495339667515457528 (type: WorkingDir)
09:47:53 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Status)
09:47:53 [TRACE] (1) asyncgit::diff: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/diff.rs:101] request DiffParams { path: ".gitignore", diff_type: WorkDir, options: DiffOptions { ignore_whitespace: false, context: 3, interhunk_lines: 0 } }
09:47:53 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Diff)
09:47:53 [TRACE] (1) asyncgit::diff: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/diff.rs:101] request DiffParams { path: ".gitignore", diff_type: WorkDir, options: DiffOptions { ignore_whitespace: false, context: 3, interhunk_lines: 0 } }
09:47:55 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:278] event: Input(Key(KeyEvent { code: Char('f'), modifiers: KeyModifiers(0x0), kind: Press, state: KeyEventState(0x0) }))
09:47:55 [TRACE] (1) asyncgit::pull: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/pull.rs:75] request
09:47:55 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:375] update
09:47:55 [TRACE] (1) asyncgit::diff: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/diff.rs:101] request DiffParams { path: ".gitignore", diff_type: WorkDir, options: DiffOptions { ignore_whitespace: false, context: 3, interhunk_lines: 0 } }
09:47:55 [TRACE] (1) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:102] request: [hash: 1599612743743222641] (type: WorkingDir)
09:47:55 [TRACE] (1) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:102] request: [hash: 12243454712170726963] (type: Stage)
09:47:55 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Diff)
09:47:55 [TRACE] (4) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:160] status fetched: 12243454712170726963 (type: Stage)
09:47:55 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Status)
09:47:55 [TRACE] (3) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:160] status fetched: 1599612743743222641 (type: WorkingDir)
09:47:55 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Status)
09:47:55 [DEBUG] (7) asyncgit::sync::remotes::callbacks: creds: 'git@github.com:Business/some-repo-in-business.git' Some("git") (CredentialType(SSH_KEY | SSH_MEMORY | SSH_CUSTOM))
09:47:56 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Pull)
09:47:56 [ERROR] fetch error: git error:ERROR: Repository not found.
; class=Ssh (23); code=Eof (-20)
09:47:56 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Pull)
09:47:56 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:375] update
09:47:56 [TRACE] (1) asyncgit::diff: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/diff.rs:101] request DiffParams { path: ".gitignore", diff_type: WorkDir, options: DiffOptions { ignore_whitespace: false, context: 3, interhunk_lines: 0 } }
09:47:56 [TRACE] (1) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:102] request: [hash: 7892252436148838368] (type: WorkingDir)
09:47:56 [TRACE] (1) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:102] request: [hash: 4036534904951496123] (type: Stage)
09:47:56 [TRACE] (4) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:160] status fetched: 4036534904951496123 (type: Stage)
09:47:56 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Diff)
09:47:56 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Status)
09:47:56 [TRACE] (3) asyncgit::status: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/asyncgit-0.26.0/src/status.rs:160] status fetched: 7892252436148838368 (type: WorkingDir)
09:47:56 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:395] update_async: Git(Status)
09:47:57 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:278] event: Input(Key(KeyEvent { code: Enter, modifiers: KeyModifiers(0x0), kind: Press, state: KeyEventState(0x0) }))
09:47:57 [TRACE] (1) gitui::app: [/Users/gclark/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gitui-0.26.1/src/app.rs:278] event: Input(Key(KeyEvent { code: Char('c'), modifiers: KeyModifiers(CONTROL), kind: Press, state: KeyEventState(0x0) }))
```

asyncgit-0.26.0/src/pull.rs に示されている pull のエントリポイントと、asyncgit-0.26.0/src/sync/remotes/callbacks の SSH 認証情報処理から始めます。gitui-0.26.1/src/app.rs に操作がどこで起動されるかが示されています。代替 SSH 設定を使って業務用リポジトリから pull を再現し、その後 fetch が「Repository not found.」と報告する理由を追跡します。完了条件は、他の remotes を壊すことなく、そのリポジトリで pull と push が動作することです。

索引モデルが issue の本文から書いたものです。