vagrant connector: `--limit` not used early enough and fail in `get_vagrant_config` of unavailble VMs

Describe the bug

With only 1 Vagrant VM existing and up, and an inventory referring to multiple Vagrant VMs by name, --limit is not applied early enough and the Vagrant config of non-existing VMs tries to be fetched but fails

I think the error starts from this line where the Vagrant connector tries to fetch a VM config before applying the limits

https://github.com/pyinfra-dev/pyinfra/blob/7540c41fa0284b8dd80dcbea24d3616414484d60/pyinfra/api/inventory.py#L101

My use case: well it's quite an edge case. I was trying something so replaced all our SSH targets with variations of @vagrant/XXX in order to be sure to not touch the real ones. I was trying to learn how --limit worked. Using mock @ssh/foo works fine as a workaround for me.

To Reproduce

Start one Vagrant VM. Have an inventory referring to two VMs where one doesn't exist yet, then apply a limit.

# inventory.py
prod_a = [
    "@vagrant/default",
]

prod_b = [
    "@vagrant/foo",
]

calling pyinfra with pyinfra inventory.py <some_operation> --limit prod_a will fail due to pyinfra trying to get the config info on prod_b anyways

Expected behavior

I would expect for pyinfra to handle the --limit argument and not try to make a connection to VMs not mentioned in --limit

Meta

• Include output of pyinfra --support.

> pyinfra --support
--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Linux
      Platform: Linux-6.5.9-arch2-1-x86_64-with-glibc2.38
      Release: 6.5.9-arch2-1
      Machine: x86_64
    pyinfra: v2.7
    Executable: /home/paul/git/<REDACTED>/.venv/bin/pyinfra
    Python: 3.11.2 (CPython, GCC 13.1.1 20230429)

• How was pyinfra installed (source/pip)?
  Pip

• Include pyinfra-debug.log (if one was created)
  empty

• Consider including output with -vv and --debug.