Issues connecting to a matrix homeserver through IP (`.well-known` HTTPS request fails)

### Steps to reproduce

Heya!
I'm trying to connect to my own Matrix server running on a VPS but without a domain assigned, so through an IP address.
I'm using "sign in manually", entering the server IP (it shows up in the list) and submitting the form.

### Outcome

#### What did you expect?

I receive an error that sliding sync is not configured for the server (i don't have that yet).

#### What happened instead?
It fails, showing that the homeserver is not reachable, and the actual error message under the hood is:

```
org.matrix.rustcomponents.sdk.ClientBuildException$ServerUnreachable: error sending request for url (https://ip.of.the.server/.well-known/matrix/client)
```
I have a self-signed CA certificate set up on the VPS, my localhost and on the emulator (shows up in Trusted Identites), verified the correctness of the cert via `curl` from localhost to the server, also it works via element web.

I checked in `DefaultUserCertificatesProvider` and there is a user cert that I assume corresponds to the one I installed on the emulator.

when i disable SSL in the matrix client sdk
```
private fun getBaseClientBuilder() = rustMatrixClientFactory
.getBaseClientBuilder(
sessionPath = sessionPath,
passphrase = pendingPassphrase,
slidingSyncProxy = AuthenticationConfig.SLIDING_SYNC_PROXY_URL,
)
.disableSslVerification() <---- HERE
.requiresSlidingSync()
```

it also correctly goes through to the "no sliding sync" error message.

the nginx config (no federation):
```
server {
listen 443 ssl;
listen [::]:443 ssl;

# For the federation port
listen 8448 ssl default_server;
listen [::]:8448 ssl default_server;

# server_name ip.of.the.server;

ssl_certificate /etc/nginx/certs/cert.pem;
ssl_certificate_key /etc/nginx/certs/key.pem;

location /.well-known/matrix/client {
return 200 '{ "m.homeserver": { "base_url": "https://ip.of.the.server" } }';
add_header Content-Type application/json;
add_header Access-Control-Allow-Origin *;
}

location ~ ^(/_matrix|/_synapse/client) {
# note: do not add a path (even a single /) after the port in `proxy_pass`,
# otherwise nginx will canonicalise the URI and cause signature verification
# errors.
proxy_pass http://localhost:8080;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;

# Nginx by default only allows file uploads up to 1M in size
# Increase client_max_body_size to match max_upload_size defined in homeserver.yaml
client_max_body_size 50M;

# Synapse responses may be chunked, which is an HTTP/1.1 feature.
proxy_http_version 1.1;
}
}
```

what am I missing here?

### Your phone model

Pixel 8 Pro (Android Studio - emulator)

### Operating system version

API 35

### Application version and app store

_No response_

### Homeserver

non-public homeserver, synapse via https://hub.docker.com/r/matrixdotorg/synapse/

### Will you send logs?

No

### Are you willing to provide a PR?

No