Error with kubectl when running with --network-address

### Description

I start colima like this: `colima start --with-kubernetes --memory 8 --network-address`

and then run kubectl and get this error:
```
$ kubectl get pods
E0626 12:45:47.674178 28658 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://192.168.64.2:50599/api?timeout=32s\": dial tcp 192.168.64.2:50599: connect: no route to host"
E0626 12:45:47.675852 28658 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://192.168.64.2:50599/api?timeout=32s\": dial tcp 192.168.64.2:50599: connect: no route to host"
E0626 12:45:47.677138 28658 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://192.168.64.2:50599/api?timeout=32s\": dial tcp 192.168.64.2:50599: connect: no route to host"
E0626 12:45:47.678439 28658 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://192.168.64.2:50599/api?timeout=32s\": dial tcp 192.168.64.2:50599: connect: no route to host"
E0626 12:45:47.679678 28658 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"https://192.168.64.2:50599/api?timeout=32s\": dial tcp 192.168.64.2:50599: connect: no route to host"
Unable to connect to the server: dial tcp 192.168.64.2:50599: connect: no route to host
```

In ~/.kube/config i changed the server address from routable ip to local and then it seems like everything is working ok: `server: https://192.168.64.2:50599` -> `server https://127.0.0.1:50599`

### Version

```
$ colima version
colima version 0.8.1
git commit: 96598cc5b64e5e9e1e64891642b91edc8ac49d16

runtime: docker
arch: aarch64
client: v28.1.1
server: v27.4.0
```
### Operating System

- [ ] macOS Intel <= 13 (Ventura)
- [ ] macOS Intel >= 14 (Sonoma)
- [ ] Apple Silicon <= 13 (Ventura)
- [x] Apple Silicon >= 14 (Sonoma)
- [ ] Linux

### Output of `colima status`

```
$ colima status
INFO[0002] colima is running using macOS Virtualization.Framework
INFO[0002] arch: aarch64
INFO[0002] runtime: docker
INFO[0002] mountType: sshfs
INFO[0002] address: 192.168.64.2
INFO[0002] socket: unix:///Users/faisalmemon/.colima/default/docker.sock
INFO[0002] kubernetes: enabled
```
### Reproduction Steps

1. colima start --with-kubernetes --memory 8 --network-address
2. kubectl get pods

### Expected behaviour

See pods

### Additional context

Error from kubectl