hasura / hasura/graphql-engine

Sever: `pg_track_table` fails if previously tracked table has been deleted

Open
#8,805 0 comments 0 reactions 0 assignees View on GitHub
c/server k/bug t/native-dbs
Dominant language
TypeScript
Stars
32.1k
Forks
3k
PR merge metrics
PR metrics pending

Description

### Version Information

Server Version: `hasura/graphql-engine:v2.6.0`
CLI Version (for CLI related issue): NA

### Environment

docker container `hasura/graphql-engine:v2.6.0`

### What is the current behaviour?

The engine fails to track a postgres table if a previously tracked table has been deleted but not untracked.

### What is the expected behaviour?

I expect hasura to track tables regardless of whether previously tracked tables have been deleted without being untracked.

### How to reproduce the issue?

1. create a postgres database and point hasura to it
2. using `psql` create a schema, sgd1 with a table in it
3. open hasura console and track the table in schema `sgd1`
4. using `psql` create a new schema `sgd2` with a new table in it
5. using `psql` delete schema `sgd1`
6. using an http client, issue a `pg_track_table` request to track the table made in `sgd2`

Step 6 should fail and say that the metadata is in an inconsistent state.

### Screenshots or Screencast

NA

### Please provide any traces or logs that could help here.

Here's a log line from my api demonstrating the issue above - note that `sgd1.block` does not exist and that I'm trying to track `sgd2.block`:

```
[api-server] [16:02:55 UTC] ERROR: hasura api responded with error
[api-server] data: {
[api-server] "internal": [
[api-server] {
[api-server] "definition": {
[api-server] "schema": "sgd1",
[api-server] "name": "block"
[api-server] },
[api-server] "reason": "Inconsistent object: no such table/view exists in source: \"sgd1.block\"",
[api-server] "name": "table sgd1.block in source default",
[api-server] "type": "table"
[api-server] }
[api-server] ],
[api-server] "path": "$.args",
[api-server] "error": "cannot continue due to new inconsistent metadata",
[api-server] "code": "unexpected"
[api-server] }
[api-server] status: 400
[api-server] headers: {
[api-server] "transfer-encoding": "chunked",
[api-server] "date": "Tue, 16 Aug 2022 16:02:55 GMT",
[api-server] "server": "Warp/3.3.19",
[api-server] "content-type": "application/json; charset=utf-8"
[api-server] }
[api-server] reqId: "cl6wdivuq000aisp28dm51iip"
[api-server] config: {
[api-server] "transitional": {
[api-server] "silentJSONParsing": true,
[api-server] "forcedJSONParsing": true,
[api-server] "clarifyTimeoutError": false
[api-server] },
[api-server] "transformRequest": [
[api-server] null
[api-server] ],
[api-server] "transformResponse": [
[api-server] null
[api-server] ],
[api-server] "timeout": 0,
[api-server] "xsrfCookieName": "XSRF-TOKEN",
[api-server] "xsrfHeaderName": "X-XSRF-TOKEN",
[api-server] "maxContentLength": -1,
[api-server] "maxBodyLength": -1,
[api-server] "headers": {
[api-server] "Accept": "application/json, text/plain, */*",
[api-server] "Content-Type": "application/json",
[api-server] "X-Hasura-Role": "admin",
[api-server] "X-Hasura-Admin-Secret": "",
[api-server] "User-Agent": "axios/0.24.0",
[api-server] "Content-Length": 94,
[api-server] },
[api-server] "baseURL": "http://localhost:8080",
[api-server] "method": "post",
[api-server] "data": "{\"type\":\"pg_track_table\",\"args\":{\"source\":\"default\",\"table\":{\"schema\":\"sgd2\",\"name\":\"block\"}}}",
[api-server] "url": "/v1/metadata"
[api-server] }
```

Note that this is even detected during startup but no action is taken:

```
dev-env-hasura-1 | {"type":"startup","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":{"kind":"catalog_migrate","info":"Already at the latest catalog version (47); nothing to do."}}
dev-env-hasura-1 | {"type":"startup","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":{"kind":"schema-sync","info":"Schema sync enabled. Polling at Milliseconds {milliseconds = 1s}"}}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread asyncActionsProcessor (re)started"}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread runCronEventsGenerator (re)started"}
dev-env-hasura-1 | {"type":"startup","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":{"kind":"schema-sync","info":{"thread_id":"ThreadId 40","instance_id":"ac58d6ec-0ae4-41ae-94de-d94ecb52b576","message":"listener thread started"}}}
dev-env-hasura-1 | {"type":"metadata","timestamp":"2022-08-16T16:21:20.411+0000","level":"warn","detail":{"message":"Inconsistent Metadata!","info":{"objects":[{"definition":{"schema":"sgd1","name":"block"},"reason":"Inconsistent object: no such table/view exists in source: \"sgd1.block\"","name":"table sgd1.block in source default","type":"table"}]}}}
dev-env-hasura-1 | {"type":"startup","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":{"kind":"schema-sync","info":{"thread_id":"ThreadId 58","instance_id":"ac58d6ec-0ae4-41ae-94de-d94ecb52b576","message":"processor thread started"}}}
dev-env-hasura-1 | {"type":"startup","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":{"kind":"event_triggers","info":"starting workers"}}
dev-env-hasura-1 | {"type":"startup","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":{"kind":"scheduled_triggers","info":"preparing data"}}
dev-env-hasura-1 | {"type":"startup","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":{"kind":"telemetry","info":"Help us improve Hasura! The graphql-engine server collects anonymized usage stats which allows us to keep improving Hasura at warp speed. To read more or opt-out, visit https://hasura.io/docs/latest/graphql/core/guides/telemetry.html"}}
dev-env-hasura-1 | {"type":"startup","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":{"kind":"server","info":{"time_taken":5.201546335,"message":"starting API server"}}}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread SchemeUpdate.listener (re)started"}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread ourIdleGC (re)started"}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread processEventQueue (re)started"}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread processScheduledTriggers (re)started"}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread runTelemetry (re)started"}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread SchemeUpdate.processor (re)started"}
dev-env-hasura-1 | {"type":"metadata","timestamp":"2022-08-16T16:21:20.411+0000","level":"warn","detail":{"message":"Inconsistent Metadata!","info":{"objects":[{"definition":{"schema":"sgd1","name":"block"},"reason":"Inconsistent object: no such table/view exists in source: \"sgd1.block\"","name":"table sgd1.block in source default","type":"table"}]}}}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread asyncActionSubscriptionsProcessor (re)started"}
dev-env-hasura-1 | {"type":"unstructured","timestamp":"2022-08-16T16:21:20.411+0000","level":"info","detail":"Thread checkForUpdates (re)started"}
```

### Any possible solutions?

Users can make sure to untrack tables that no longer exist. I think this should be an configuration option for the engine itself so that this is automatically taken care of.

### Can you identify the location in the source code where the problem exists?

No

### If the bug is confirmed, would you be willing to submit a PR?

Probably not

### Keywords

hasura automatically delete table deleted

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with psql and the pg_track_table metadata request against schemas sgd1 and sgd2, using the steps and logs provided. Trace the metadata consistency handling for deleted tracked tables; done means tracking sgd2.block succeeds without the deleted sgd1.block entry preventing the request.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.