error "SSH Port was not properly retrieved from SSHConfig." (fixed after running "vagrant up")

### Overview of the Issue

first run of `packer build .` fails, but succeeds if run after `vagrant up`

### Reproduction Steps

setup

0. test on `macos m1` w/ [homebrew](https://brew.sh/) installed
1. install [qemu](https://www.qemu.org/) `brew install qemu`
2. install [Vagrant](https://www.vagrantup.com/) `brew install --cask vagrant`
3. install [vagrant-qemu](https://github.com/ppggff/vagrant-qemu) plugin `vagrant plugin install vagrant-qemu`
4. install [packer](https://www.packer.io/) `brew install packer`

verify version

```console
➜ packer --version
1.8.4
➜ vagrant plugin list
vagrant-qemu (0.3.3, global)
➜ vagrant --version
Vagrant 2.3.2
➜ brew install qemu

Warning: qemu 7.1.0 is already installed and up-to-date.
To reinstall 7.1.0, run:
brew reinstall qemu
```

first run

```console
➜ dart_vagrant git:(setup_packer) ✗ packer build .
vagrant.this: output will be in this color.

==> vagrant.this: Creating a Vagrantfile in the build directory...
==> vagrant.this: Adding box using vagrant box add ...
vagrant.this: (this can take some time if we need to download the box)
==> vagrant.this: Calling Vagrant Up (this can take some time)...
==> vagrant.this: destroying Vagrant box...
==> vagrant.this: Deleting output directory...
Build 'vagrant.this' errored after 5 minutes 31 seconds: error: SSH Port was not properly retrieved from SSHConfig.

==> Wait completed after 5 minutes 31 seconds

==> Some builds didn't complete successfully and had errors:
--> vagrant.this: error: SSH Port was not properly retrieved from SSHConfig.

==> Builds finished but no artifacts were created.
```

run vagrant locally

```console
➜ vagrant init -m perk/ubuntu-2204-arm64

A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
➜ vagrant up

The provider 'qemu' could not be found, but was requested to
back the machine 'default'. Please use a provider that exists.

Vagrant knows about the following providers: docker, hyperv, virtualbox
➜ vagrant plugin install vagrant-qemu

Installing the 'vagrant-qemu' plugin. This can take a few minutes...
Fetching vagrant-qemu-0.3.3.gem
Installed the plugin 'vagrant-qemu (0.3.3)'!
➜ vagrant up

Bringing machine 'default' up with 'qemu' provider...
==> default: Box 'perk/ubuntu-2204-arm64' could not be found. Attempting to find and install...
default: Box Provider: libvirt
default: Box Version: >= 0
==> default: Loading metadata for box 'perk/ubuntu-2204-arm64'
default: URL: https://vagrantcloud.com/perk/ubuntu-2204-arm64
==> default: Adding box 'perk/ubuntu-2204-arm64' (v20221101.1) for provider: libvirt
default: Downloading: https://vagrantcloud.com/perk/boxes/ubuntu-2204-arm64/versions/20221101.1/providers/libvirt.box
default: Calculating and comparing box checksum...
==> default: Successfully added box 'perk/ubuntu-2204-arm64' (v20221101.1) for 'libvirt'!
==> default: Checking if box 'perk/ubuntu-2204-arm64' version '20221101.1' is up to date...
==> default: Importing a QEMU instance
default: Creating and registering the VM...
default: Successfully imported VM
==> default: Warning! The QEMU provider doesn't support any of the Vagrant
==> default: high-level network configurations (`config.vm.network`). They
==> default: will be silently ignored.
==> default: Starting the instance...
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:50022
default: SSH username: vagrant
default: SSH auth method: private key
default: Warning: Connection reset. Retrying...
default: Warning: Remote connection disconnect. Retrying...
default:
default: Vagrant insecure key detected. Vagrant will automatically replace
default: this with a newly generated keypair for better security.
default:
default: Inserting generated public key within guest...
default: Removing insecure key from the guest if it's present...
default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
➜ vagrant ssh

Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-52-generic aarch64)

* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage

System information as of Fri Nov 11 12:09:54 UTC 2022

System load: 0.0
Usage of /: 2.4% of 61.84GB
Memory usage: 4%
Swap usage: 0%
Processes: 95
Users logged in: 0
IPv4 address for eth0: 10.0.2.15
IPv6 address for eth0: fec0::5054:ff:fe12:3456

0 updates can be applied immediately.

The list of available updates is more than a week old.
To check for new updates run: sudo apt update

vagrant@ubuntu:~$ exit
logout
➜ vagrant destroy -f

==> default: Stopping the instance...
==> default: Destroying the instance...
```

second run

```console
➜ packer build .
vagrant.this: output will be in this color.

==> vagrant.this: Creating a Vagrantfile in the build directory...
==> vagrant.this: Adding box using vagrant box add ...
vagrant.this: (this can take some time if we need to download the box)
==> vagrant.this: Calling Vagrant Up (this can take some time)...
==> vagrant.this: Using SSH communicator to connect: 127.0.0.1
==> vagrant.this: Waiting for SSH to become available...
==> vagrant.this: Connected to SSH!
==> vagrant.this: Packaging box...
==> vagrant.this: destroying Vagrant box...
Build 'vagrant.this' finished after 7 minutes 22 seconds.

==> Wait completed after 7 minutes 22 seconds

==> Builds finished. The artifacts of successful builds are:
--> vagrant.this: Vagrant box 'package.box' for 'qemu' provider
```

### Plugin and Packer version

```console
➜ packer --version
1.8.4
```
```hcl
vagrant = {
version = ">= 1.0.2"
source = "github.com/hashicorp/vagrant"
}
```

### Simplified Packer Buildfile

Buildfile

```hcl
packer{
required_plugins {
# see https://github.com/hashicorp/packer-plugin-vagrant
vagrant = {
version = ">= 1.0.2"
source = "github.com/hashicorp/vagrant"
}
}
}

source "vagrant" "this" {
communicator = "ssh"
# see https://app.vagrantup.com/perk/boxes/ubuntu-2204-arm64
source_path = "perk/ubuntu-2204-arm64"
provider = "qemu"
add_force = true
}

build {
sources = [
"source.vagrant.this"
]
}
```

### Operating system and Environment details

neofetch

```console
'c. yakforward@yakforward.local
,xNMM. ---------------------------
.OMMMMo OS: macOS 13.0.1 22A400 arm64
OMMM0, Host: MacBookAir10,1
.;loddo:' loolloddol;. Kernel: 22.1.0
cKMMMMMMMMMMNWMMMMMMMMMM0: Uptime: 1 hour, 47 mins
.KMMMMMMMMMMMMMMMMMMMMMMMWd. Packages: 89 (brew)
XMMMMMMMMMMMMMMMMMMMMMMMX. Shell: zsh 5.8.1
;MMMMMMMMMMMMMMMMMMMMMMMM: Resolution: 1440x900
:MMMMMMMMMMMMMMMMMMMMMMMM: DE: Aqua
.MMMMMMMMMMMMMMMMMMMMMMMMX. WM: Quartz Compositor
kMMMMMMMMMMMMMMMMMMMMMMMMWd. WM Theme: Blue (Dark)
.XMMMMMMMMMMMMMMMMMMMMMMMMMMk Terminal: iTerm2
.XMMMMMMMMMMMMMMMMMMMMMMMMK. Terminal Font: Monaco 12
kMMMMMMMMMMMMMMMMMMMMMMd CPU: Apple M1
;KMMMMMMMWXXWMMMMMMMk. GPU: Apple M1
.cooc,. .,coo:. Memory: 1187MiB / 8192MiB

```

### Logs

[packerlog.txt](https://github.com/hashicorp/packer-plugin-vagrant/files/9989994/packerlog.txt)

#### related issues

- https://github.com/hashicorp/packer-plugin-ansible/issues/130
- https://github.com/ppggff/vagrant-qemu/issues/28