Federation errors on Dendrite Homserver

*This issue was originally created by [**@sytallax**](https://github.com/sytallax) at .*

### Background information

- **Dendrite version or git SHA**: Release v0.13.6
- **SQLite3 or Postgres?**: Postgres
- **Running in Docker?**: No
- **`go version`**: 1.22.2
- **Client used (if applicable)**: Element and Cinny

### Description

- **What** is the problem: Homeserver fails federation tester and its users cannot direct message matrix.org users
- **Who** is affected: Homeserver admin (me)
- **How** is this bug manifesting: Federation tester errors on "Version" record. Homeserver users cannot resolve to matrix.org users and vice versa
- **When** did this first appear: Homeserver has never federated. The homeserver went live one hour before posting

### Steps to reproduce

- Followed the installation guide on the Matrix github.io [site](https://matrix-org.github.io/dendrite/installation).
- I use `mydomain.net` in place of my actual domain going forward
- My homeserver is `mydomain.net` instead of the usual `matrix.mydomain.net`
- Dendrite listens on port 8008 (firewall allows ports 80, 443, 8448, 8008)
- Used nginx for my reverse proxy
- Used `python3-certbot-nginx` Debian package to create ssl certificate for `mydomain.net`
- Performed a manual installation of Dendrite
- Config files and relevent logs found below

### Federation test raw JSON output
```
{
"WellKnownResult": {
"m.server": "mydomain.net:443",
"CacheExpiresAt": 0
},
"DNSResult": {
"SRVSkipped": true,
"SRVCName": "",
"SRVRecords": null,
"SRVError": null,
"Hosts": {},
"Addrs": null
},
"ConnectionReports": {},
"ConnectionErrors": {},
"Version": {
"error": "Get \"matrix://mydomain.net/_matrix/federation/v1/version\": dial tcp: lookup mydomain.net on 8.8.8.8:53: no such host"
},
"FederationOK": false
}
```

### Relevent logs

#### Attempted to direct message my matrix.org account
```
Apr 05 22:15:37 REDACTED dendrite[53766]: time="2024-04-05T22:15:37.065414664Z" level=error msg="getProfile failed" error="contents=[123 34 101 114 114 99 111 100 101 34 58 34 77 95 85 78 65 85 84 72 79 82 73 90 69 68 34 44 34 101 114 114 111 114 34 58 34 70 97 105 108 101 100 32 116 111 32 102 105 110 100 32 97 110 121 32 107 101 121 32 116 111 32 115 97 116 105 115 102 121 58 32 95 70 101 116 99 104 75 101 121 82 101 113 117 101 115 116 40 115 101 114 118 101 114 95 110 97 109 101 61 39 105 98 114 97 104 105 109 109 117 102 116 101 101 46 110 101 116 39 44 32 109 105 110 105 109 117 109 95 118 97 108 105 100 95 117 110 116 105 108 95 116 115 61 49 55 49 50 51 53 53 51 51 57 54 57 51 44 32 107 101 121 95 105 100 115 61 91 39 101 100 50 53 53 49 57 58 112 121 75 67 117 106 39 93 41 34 125] msg=Failed to GET JSON (hostname \"matrix.org\" path \"/_matrix/federation/v1/query/profile\") code=401 wrapped=M_UNAUTHORIZED: Failed to find any key to satisfy: _FetchKeyRequest(server_name='mydomain.net', minimum_valid_until_ts=1712355339693, key_ids=['ed25519:pyKCuj'])" req.id=Q3DwDWHaf9vT req.method=GET req.path="/_matrix/client/v3/profile/@sytallax:matrix.org"
```
```
Apr 05 22:15:37 REDACTED dendrite[53766]: time="2024-04-05T22:15:37.317436431Z" level=error msg="Failed to manually update device lists for user" error="DeviceKeysForUser @sytallax:matrix.org returned no keys but wanted all keys, falling back to remote" server=matrix.org user_id="@sytallax:matrix.org"
```
```
Apr 05 22:15:51 REDACTED dendrite[53766]: time="2024-04-05T22:15:51.364564887Z" level=info msg="Sending invite" destination=matrix.org event_id="$gcd4br-I9D9EJN43ReadlSBI1ROVC-8EPS2l7GQlisc" room_id="!ooT0edQIcx1yyOQD:mydomain.net" room_version=10 user_id="@sytallax:matrix.org"
```
```
Apr 05 22:15:51 REDACTED dendrite[53766]: time="2024-04-05T22:15:51.773934479Z" level=error msg="fedClient.SendInvite failed" error="r.federation.SendInviteV2: failed to send invite: contents=[123 34 101 114 114 99 111 100 101 34 58 34 77 95 85 78 65 85 84 72 79 82 73 90 69 68 34 44 34 101 114 114 111 114 34 58 34 70 97 105 108 101 100 32 116 111 32 102 105 110 100 32 97 110 121 32 107 101 121 32 116 111 32 115 97 116 105 115 102 121 58 32 95 70 101 116 99 104 75 101 121 82 101 113 117 101 115 116 40 115 101 114 118 101 114 95 110 97 109 101 61 39 105 98 114 97 104 105 109 109 117 102 116 101 101 46 110 101 116 39 44 32 109 105 110 105 109 117 109 95 118 97 108 105 100 95 117 110 116 105 108 95 116 115 61 49 55 49 50 51 53 53 51 53 52 49 50 56 44 32 107 101 121 95 105 100 115 61 91 39 101 100 50 53 53 49 57 58 112 121 75 67 117 106 39 93 41 34 125] msg=Failed to PUT JSON (hostname \"matrix.org\" path \"/_matrix/federation/v2/invite/!ooT0edQIcx1yyOQD:mydomain.net/$gcd4br-I9D9EJN43ReadlSBI1ROVC-8EPS2l7GQlisc\") code=401 wrapped=M_UNAUTHORIZED: Failed to find any key to satisfy: _FetchKeyRequest(server_name='mydomain.net', minimum_valid_until_ts=1712355354128, key_ids=['ed25519:pyKCuj'])" event_id="$gcd4br-I9D9EJN43ReadlSBI1ROVC-8EPS2l7GQlisc" invitee="@sytallax:matrix.org" inviter="@REDACTED:mydomain.net" req.id=U7sV0JoWZZYC req.method=POST req.path=/_matrix/client/v3/createRoom room_id="!ooT0edQIcx1yyOQD:mydomain.net" user_id="@REDACTED:mydomain.net"
```
### NGINX Config

```
server {

server_name mydomain.net www.mydomain.net;

root /var/www/mydomain.net;
index index.html;

location / {
try_files $uri $uri/ =404;
}

listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/mydomain.net/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/mydomain.net/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

listen 8448 ssl;
listen [::]:8448 ssl;

location ~* ^(\/_matrix|\/_synapse|\/_client) {
proxy_pass http://localhost:8008;
proxy_set_header X-Forwarded-For $remote_addr;
client_max_body_size 50M;
}

# These sections are required for client and federation discovery
# (AKA: Client Well-Known URI)
location /.well-known/matrix/client {
return 200 '{"m.homeserver": {"base_url": "https://mydomain.net:443"}}';
default_type application/json;
add_header Access-Control-Allow-Origin *;
}

location /.well-known/matrix/server {
return 200 '{"m.server": "ibrahimuftee.net:443"}';
default_type application/json;
add_header Access-Control-Allow-Origin *;
}

}
server {
if ($host = www.mydomain.net) {
return 301 https://$host$request_uri;
} # managed by Certbot

if ($host = mydomain.net) {
return 301 https://$host$request_uri;
} # managed by Certbot

listen 80;
listen [::]:80;

server_name mydomain.net www.mydomain.net;
return 404; # managed by Certbot

}
```
#### NOTE: I modeled my nginx config after this sample config:
```
server {
server_name example.org;

listen 80;
listen [::]:80;

listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;

listen 8448 ssl http2 default_server;
listen [::]:8448 ssl http2 default_server;

location ~* ^(\/_matrix|\/_synapse|\/_client) {
proxy_pass http://localhost:8008;
proxy_set_header X-Forwarded-For $remote_addr;
client_max_body_size 50M;
}

# These sections are required for client and federation discovery
# (AKA: Client Well-Known URI)
location /.well-known/matrix/client {
return 200 '{"m.homeserver": {"base_url": "https://example.org"}}';
default_type application/json;
add_header Access-Control-Allow-Origin *;
}

location /.well-known/matrix/server {
return 200 '{"m.server": "example.org:443"}';
default_type application/json;
add_header Access-Control-Allow-Origin *;
}
}
```
### Dendrite YAML file (stripped of comments for legibility)
```
version: 2

global:
server_name: mydomain.net

private_key: matrix_key.pem

old_private_keys:

key_validity_period: 168h0m0s

database:
connection_string: postgresql://dendrite:REDACTED_SECRET@localhost/dendrite?sslmode=disable
max_open_conns: 90
max_idle_conns: 5
conn_max_lifetime: -1

cache:
max_size_estimated: 1gb

max_age: 1h

well_known_server_name: ""

well_known_client_name: ""

well_known_sliding_sync_proxy: ""

trusted_third_party_id_servers:
- matrix.org
- vector.im

disable_federation: false

presence:
enable_inbound: true
enable_outbound: true

report_stats:
enabled: false
endpoint: https://panopticon.matrix.org/push

server_notices:
enabled: false
local_part: "_server"
display_name: "Server Alerts"
avatar_url: ""
room_name: "Server Alerts"

jetstream:
addresses:

disable_tls_validation: false

storage_path: ./

topic_prefix: Dendrite

metrics:
enabled: false
basic_auth:
username: metrics
password: metrics

dns_cache:
enabled: false
cache_size: 256
cache_lifetime: "5m" # 5 minutes; https://pkg.go.dev/time@master#ParseDuration

app_service_api:
disable_tls_validation: false

config_files:

client_api:
registration_disabled: true

guests_disabled: true

registration_shared_secret: "REDACTED_SECRET"

enable_registration_captcha: false

recaptcha_public_key: ""
recaptcha_private_key: ""
recaptcha_bypass_secret: ""

turn:
turn_user_lifetime: "5m"
turn_uris:
turn_shared_secret: ""

rate_limiting:
enabled: true
threshold: 20
cooloff_ms: 500
exempt_user_ids:

federation_api:
send_max_retries: 16

disable_tls_validation: false

disable_http_keepalives: false

key_perspectives:
- server_name: matrix.org
keys:
- key_id: ed25519:auto
public_key: Noi6WqcDj0QmPxCNQqgezwTlBKrfqehY1u2FyWP9uYw
- key_id: ed25519:a_RXGa
public_key: l8Hft5qXKn1vfHrg3p4+W8gELQVo8N13JkluMfmn2sQ

prefer_direct_fetch: false

media_api:
base_path: ./media_store

max_file_size_bytes: 10485760

dynamic_thumbnails: false

max_thumbnail_generators: 10

thumbnail_sizes:
- width: 32
height: 32
method: crop
- width: 96
height: 96
method: crop
- width: 640
height: 480
method: scale

mscs:
mscs:

sync_api:

search:
enabled: true

index_path: "./searchindex"

language: "en"

user_api:
bcrypt_cost: 10

auto_join_rooms:

tracing:
enabled: false
jaeger:
serviceName: ""
disabled: false
rpc_metrics: false
tags: []
sampler: null
reporter: null
headers: null
baggage_restrictions: null
throttler: null

logging:
- type: std
level: info
- type: file
level: info
params:
path: ./logs
```