PostgREST / PostgREST/postgrest
[Question] Postgrest overhead w.r.t direct DB query
Nobody has claimed this yet.
- Dominant language
- Haskell
- Stars
- 27.7k
- Forks
- 1.2k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 54
Description
Environment
- PostgreSQL version: AWS RDS 13.3 (2 vCPU, 8GB Mem)
- PostgREST version: Docker (postgrest/postgrest:v8.0.0)
# 4 Pods
resources:
limits:
cpu: 1500m
ephemeral-storage: 1Gi
memory: 256Mi
requests:
cpu: 750m
ephemeral-storage: 250Mi
memory: 256Mi
- Operating system: AWS EKS (1.15)
Description of issue
We ran some load tests comparing Postgrest against direct DB queries, and saw that the overhead is significant. The setup is described below.
create table if not exists perf_test (
id bigserial,
name text,
value jsonb,
created_at timestamp with time zone default now() not null
);
create index if not exists idx_id
on perf_test (id);
Table has ~ 2M rows.
Tests were executed from K8s using Locust.
Benchmark DB query
def execute_query(self):
id = random.randint(1, 2000000)
id2 = id + 20
self.client.execute_query("select * from perf_test where id between {0} and {1}".format(id, id2))
Benchmark Postgrest request
def get_perf_test(self):
id = random.randint(1, 2000000)
id2 = id + 20
headers = {'Accept-Encoding': 'gzip', 'Connection': 'Keep-Alive', 'Keep-Alive': 'timeout=5, max=2000'}
self.client.get(f"/perf_test?id=geq.{id}&id=le.{id2}", name="/perf_test", headers=headers)
The DB queries are able to scale up to 2K TPS with a latency (P95) of ~ 2ms. The corresponding Postgrest requests are topping off at ~ 10ms (P95) with a TPS of ~50.
Is this amount of overhead expected?
Are there any recommended configurations / best practices to optimize throughput and latency of Postgrest?
Are there any benchmarks that you have done with this kind of comparison?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the supplied PostgreSQL schema, direct-query benchmark, and PostgREST request run through Locust on the stated Kubernetes setup. Review the reported latency and throughput difference, then identify the expected overhead, relevant configuration or best practices, and comparable benchmarks. The issue does not define a code change or concrete acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, haskell, kubernetes, postgresql
- Domain
- api, databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100