hasura / hasura/graphql-engine
Untracking tables in parallel results in serialization failure at postgres level
- Dominant language
- TypeScript
- Stars
- 32.1k
- Forks
- 3k
- PR merge metrics
- PR metrics pending
Description
### Version Information
Server Version: v2.6.1
### Environment
OSS
### What is the current behaviour?
When using the rest api to untrack tables, I get the following error: `serialization failure due to concurrent update`. It seems this happens if I try to untrack tables in parallel.
### What is the expected behaviour?
Untracking in parallel should succeed.
### How to reproduce the issue?
Write a script that that calls the hasura rest api to untrack tables, but do so in parallel without waiting for each individual request to complete.
### Please provide any traces or logs that could help here.
Here's the full log from hasura showing this issue as reported by datadog
```
{
"id": "AgAAAYtoO3sTKL9qHAAAAAAAAAAYAAAAAEFZdG9PNEdjQUFDYU5TQmw0cVNsYWdBQwAAACQAAAAAMDE4YjY4NTMtODdkOC00ZGQzLTg2OWItODNiMzJlNDFhNTlk",
"content": {
"timestamp": "2023-10-25T19:06:50.515Z",
"service": "graphql-engine",
"attributes": {
"level": "error",
"detail": {
"http_info": {
"method": "POST",
"ip": "34.222.178.92",
"http_version": "HTTP/1.1",
"url": "//v1/metadata",
"status": 400
},
"operation": {
"request_mode": "error",
"error": {
"path": "$",
"code": "concurrent-update",
"error": "serialization failure due to concurrent update"
},
"user_vars": {
"x-hasura-role": "admin"
},
"request_id": "cf86a4f6-cf06-487a-bd40-3a91642f3245",
"response_size": 96
},
"request_id": "cf86a4f6-cf06-487a-bd40-3a91642f3245"
},
"type": "http-log",
"timestamp": "2023-10-25T19:06:50.515+0000"
}
}
}
```
### Any possible solutions/workarounds you're aware of?
Make sure to untrack tables in serial.
It seems that this is related to how hasura connects to the underlying postres store since this seems to be a [postgres error](https://pganalyze.com/docs/log-insights/app-errors/U138).
### Keywords
hasura, postgres, track, error
Contributor guide
Research direction
Start by reproducing concurrent table-untrack requests through the v1/metadata REST endpoint, using the parallel script described in the issue and observing the PostgreSQL serialization error. Trace the metadata untracking request path and its database transaction handling; done means parallel untracking succeeds without a concurrent-update failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgres
- Domain
- api, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100