hasura / hasura/graphql-engine

upgrade from 1.0.3 to 2.0.6 fails on metadata insert

Open
#7,411 2 comments 0 reactions 0 assignees View on GitHub
support/needs-more-info
Dominant language
TypeScript
Stars
32.1k
Forks
3k
PR merge metrics
PR metrics pending

Description

hasura fails to start on changing docker image from 1.0.3 -> 2.0.6

using cli 2.0.7
after upgrading metadata to v3

error seen in hasura logs

```
Attaching to botany_hasura
hasura_1 | {"type":"startup","timestamp":"2021-08-18T03:04:57.258+0000","level":"info","detail":{"kind":"server_configuration","info":{"live_query_options":{"batch_size":100,"refetch_delay":1},"transaction_isolation":"ISOLATION LEVEL READ COMMITTED","enable_maintenance_mode":false,"enabled_log_types":["http-log","websocket-log","startup","webhook-log","query-log"],"server_host":"HostAny","enable_allowlist":false,"remote_schema_permissions":false,"log_level":"info","auth_hook_mode":null,"use_prepared_statements":true,"unauth_role":null,"stringify_numeric_types":false,"v1-boolean-null-collapse":false,"graceful_shutdown_timeout":60,"enabled_apis":["metadata","graphql","config","pgdump"],"enable_telemetry":false,"enable_console":false,"auth_hook":null,"infer_function_permissions":true,"experimental_features":[],"events_fetch_batch_size":100,"jwt_secret":{"audience":null,"claims_format":"json","claims_namespace":"https://hasura.io/jwt/claims","key":"","header":null,"type":"","issuer":null},"cors_config":{"allowed_origins":"*","disabled":false,"ws_read_cookie":null},"websocket_compression_options":"NoCompression","console_assets_dir":null,"admin_secret_set":true,"port":8080,"websocket_keep_alive":"KeepAliveDelay {unKeepAliveDelay = Seconds {seconds = 5s}}"}}}
hasura_1 | {"type":"startup","timestamp":"2021-08-18T03:04:57.258+0000","level":"info","detail":{"kind":"postgres_connection","info":{"retries":1,"database_url":"postgre
s://postgres:...@postgres:5432/botany"}}}

====== ERROR HERE =======
hasura_1 | {"type":"startup","timestamp":"2021-08-18T03:04:57.258+0000","level":"error","detail":{"kind":"catalog_migrate","info":{"internal":{"statement":"\n INSERT INTO hdb_catalog.hdb_metadata(id, metadata)\n VALUES (1, $1::json)\n ","prepared":true,"error":{"exec_status":"FatalError","hint":null,"message":"duplicate key value violates unique constraint \"hdb_metadata_pkey\"","status_code":"23505","description":"Key (id)=(1) already exists."},"arguments":["(Oid 114,Just (\"{\\\"sources\\\":[{\\\"kind\\\":\\\"postgres\\\",\\\"name\\\":\\\"default\\\",\\\"tables\\\":[{\\\"select_permissions\\\":[{\\\"role\\\":\\\"user\\\",\\\"permission\\\":{\\\"columns\\\":[\\\"id\\\",\\\"actorId\\\",\\\"type\\\",\\\"subject\\\",\\\"source\\\",\\\"data\\\",\\\"created_at\\\",\\\"timestamp\\\",\\\"aboutUserId\\\",\\\"activityHash\\\",\\\"updated_at\\\",\\\"wasBackfilled\\\",\\\"projectName\\\"],\\\"filter\\\":{\\\"_or\\\":[{\\\"mappedProjects\\\":{\\\"organizationIntegration\\\":{\\\"organizationId\\\":{\\\"_eq\\\":\\\"x-hasura-org-id\\\"}}}},{\\\"actorId\\\":{\\\"_eq\\\":\\\"X-Hasura-User-Id\\\"}},{\\\"aboutUserId\\\":{\\\"_eq\\\":\\\"X-Hasura-User-Id\\\"}}]}}}],\\\"object_relationships\\\":[{\\\"using\\\":{\\\"foreign_key_constraint_on\\\":\\\"aboutUserId\\\"},\\\"name\\\":\\\"aboutUser\\\"},{\\\"using\\\":{\\\"manual_configuration\\\":{\\\"remote_table\\\":{\\\"schema\\\":\\\"public\\\",\\\"name\\\":\\\"user\\\"},\\\"insertion_order\\\":null,\\\"column_mapping\\\":{\\\"actorId\\\":\\\"id\\\"}}},\\\"name\\\":\\\"actor\\\"}],\\\"tabl:

```

repro steps:

starting with a remote postgres backup from a hasura 1.0.3 server in production...

1. start local hasura server 1.0.3 + cli 2.0.7
2. upgrade metadata 1->2, then 2->3
3. kill hasura
4. change docker image to 2.0.6
5. start hasura

expect: starts in new server version
actual: does not start. see error above.

when i truncate hdb_metadata table, the hasura startup can proceed and metadata gets re-populated from the metadata files.

what would have been the correct way to do this migration locally and apply to the remote production db?

Contributor guide

Open the contributing guide

Research direction

Start with the listed reproduction steps and the catalog_migrate startup error, focusing on the hdb_catalog.hdb_metadata insert and the metadata version transitions from 1 to 2 to 3. Compare the database state with the metadata files before and after the upgrade; done means the 2.0.6 server starts without the duplicate hdb_metadata primary-key error and preserves the migrated metadata.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgres
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.