hasura / hasura/graphql-engine

Error fetching JWK: ConnectionFailure Network.Socket.connect: <socket: 57> only at first container start

Open
#9,659 4 comments 0 reactions 0 assignees View on GitHub
k/bug
Dominant language
TypeScript
Stars
32.1k
Forks
3k
PR merge metrics
PR metrics pending

Description

### Version Information

Server Version: v2.22.0

### Environment

Docker

### What is the current behaviour?

The first time I start my docker-compose, the docker container of Hasura gets an error and automatically restarts. The error is:

`Error fetching JWK: ConnectionFailure Network.Socket.connect: : does not exist (Connection refused)`

Seems like Hasura cannot download the JWK configuration from Keycloak. The strange thing is that, after the hasura container restarts automatically, everything works normally. The error appears only at the first startup.
Moreover, the keycloak container is healthy and the JWT url is indeed reachable before Hasura starts up.

The problem is that the first error block the docker compose startup procedure so every container that depends from Hasura will not start, even if Hasura restarts automatically and works correctly.

### What is the expected behaviour?

Hasura container should start normally at first startup

### How to reproduce the issue?

Try to run this docker compose

### Screenshots or Screencast

### Please provide any traces or logs that could help here.

```bash
Executing task: docker logs --tail 1000 -f 026d1e76a3a81cac0b018d55037f94f8a074792e3e69ab94a68bbdf3cc7710de

{"detail":{"info":{"admin_secret_set":true,"auth_hook":null,"auth_hook_mode":null,"console_assets_dir":null,"console_sentry_dsn":null,"cors_config":{"allowed_origins":"*","disabled":false,"ws_read_cookie":null},"enable_allowlist":false,"enable_console":true,"enable_maintenance_mode":false,"enable_metadata_query_logging":false,"enable_telemetry":true,"enabled_apis":["config","pgdump","metadata","graphql"],"enabled_log_types":["webhook-log","http-log","startup","query-log","websocket-log"],"events_fetch_batch_size":100,"experimental_features":[],"graceful_shutdown_timeout":60,"infer_function_permissions":true,"jwt_secret":[{"audience":null,"claims_format":"json","claims_namespace":"https://hasura.io/jwt/claims","header":null,"issuer":null,"jwk_url":"http://keycloak:8080/realms/vc/protocol/openid-connect/certs"}],"live_query_options":{"batch_size":100,"refetch_delay":1},"log_level":"info","port":8080,"remote_schema_permissions":false,"server_host":"HostAny","stringify_numeric_types":false,"transaction_isolation":"ISOLATION LEVEL READ COMMITTED","unauth_role":"guest","use_prepared_statements":true,"v1-boolean-null-collapse":false,"websocket_compression_options":"NoCompression","websocket_connection_init_timeout":"Refined (Seconds {seconds = 3s})","websocket_keep_alive":"KeepAliveDelay {unKeepAliveDelay = Refined (Seconds {seconds = 5s})}"},"kind":"server_configuration"},"level":"info","timestamp":"2023-05-22T12:06:13.105+0000","type":"startup"}
{"detail":{"info":{"database_url":"postgres://postgres:...@postgres:5432/postgres","retries":5},"kind":"postgres_connection"},"level":"info","timestamp":"2023-05-22T12:06:13.105+0000","type":"startup"}
{"detail":{"info":{"message":"source \"default\" is already at the latest catalog version (3).","source":"default"},"kind":"source_catalog_migrate"},"level":"info","timestamp":"2023-05-22T12:06:13.105+0000","type":"startup"}
{"detail":{"info":"Already at the latest catalog version (47); nothing to do.","kind":"catalog_migrate"},"level":"info","timestamp":"2023-05-22T12:06:13.105+0000","type":"startup"}
Error fetching JWK: ConnectionFailure Network.Socket.connect: : does not exist (Connection refused)
* Terminal will be reused by tasks, press any key to close it.
```

### Any possible solutions/workarounds you're aware of?

### Keywords

jwk fetch, socket, error, docker, startup, restart

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the first-start failure with the Docker Compose setup described in the issue, then trace the startup path that fetches the JWK URL from Keycloak. Done means Hasura starts successfully on the initial launch and containers depending on it can proceed without relying on an automatic restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose
Domain
authentication, backend, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.