loopbackio / loopbackio/loopback-next

Loopback not utilizing connection pool fully

Open
#7,603 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
5.1k
Forks
1.1k
Avg merge
2d 21h
Merged PRs (30d)
27

Description

Using loopback datasources for postgres, we provided max connection pool of 20. As soon as app starts it makes two connection. When doing load test using jmeter, it is seen that only those two are being used. But after some time new connection were formed, to match the max number. But in any given second active connection were never more than 3. Also it was like utilizing a stack of connections because it is noticed that the last released connection was used. So, system was never able to use near 10-15 connections although they were there in idle state. This resulted in a tps of 160, for a process which comprised :

http in -> two db insert -> http out
The model had 200 fields, there are minimum validations, with no hooks, beforeSave, afterSave etc.

Attaching connection logs:

Start: \ => represents http in
Stop: \ => represents http out

DB-Write Start: \\ => represents beginning of write process, handing it off to loopback
DB-Write Stop: \\ => represents end of write process,

\#\#\# 2021-07-01T13:26:39.589Z Acquired \ => represents acquiring process by pg, got using pg.pool.on('aquire',...)
releasing connection\#\#\# \ => releasing of said connection back to pool
\#\#\# 2021-07-01T13:26:40.148Z Connect \ => Making new connections

Thanks.
[connect_findings.txt](https://github.com/strongloop/loopback-next/files/6755393/connect_findings.txt)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source files or tests are named. Start with the PostgreSQL datasource pool configuration and the attached connection logs, then reproduce the reported JMeter load pattern. Done means establishing why idle pool connections are not used concurrently and documenting or correcting the behavior so utilization matches the configured pool.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, typescript
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.