jackc / jackc/pgx

Memory Leak on ChunkReader.newBuf

Open
#845 29 comments 8 reactions 0 assignees View on GitHub
v4
Dominant language
Go
Stars
14.3k
Forks
1.1k
Avg merge
6d 9h
Merged PRs (30d)
11

Description

We have been seeing a memory leak that shows up over time, that we can't seem to get to the bottom of, specifically when in a transaction, and we do a SELECT during that query. This process is being run in parallel ~100 at a time, and will develop a significant leak after 5-10 minutes of running, that never gets cleared up.

The code that primarily shows the memory being retained (when reallocating the chunk reader buffer location). https://github.com/jackc/chunkreader/blob/3c4a99247c6656284dd2be8d74438ca9bbb4f342/chunkreader.go#L72-L76

pgx portion of the trace:
discover-api_inuse_space

Any insights that could help us narrow down what may be retaining the reference?

Contributor guide

Open the contributing guide

Research direction

Start with chunkreader.go lines 72-76 and the linked heap trace, then reproduce the reported workload: roughly 100 parallel processes performing SELECTs inside transactions for 5–10 minutes. Trace the retained buffer reference and document the cause or a focused regression test demonstrating the leak.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.