jackc / jackc/pgx

Memory leak in scanPlanString

Open
#2,039 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
14.3k
Forks
1.1k
Avg merge
6d 9h
Merged PRs (30d)
11

Description

**Describe the bug**
There is a memory leak while using pgx Select Scans. There is a consistent increase of memory usage by the package. Here is a pprof dump -
```
reflect.New:3256 github.com/jackc/pgx/v5/pgtype.scanPlanString - 4.65 GB
github.com/jackc/pgx/v5/pgtype.scanPlanString - 2.64 GB
reflect.New:3256 github.com/georgysavva/scany - 834.60 MB
*github.com/georgysavva/scany/v2/dbscan.(RowS) - 821.96 MB
reflect.Value.extendSlice:2848 reflect.Append - 816.12 MB
reflect.New:3256 github.com/georgysavva/scany - 794.36 MB
github.com.jackc/pgx/v5/pgtype.TextCodec.Deco - 568.66 MB
github.com.jackc/pgx/v5/pgtype.scanPlanString - 360.29 MB
github.com.jackc/pgx/v5/pgtype.scanPlanString - 320.38 MB
```

These are the top memory users in my application. The query I make is quite simple and is just made once every 30 minutes. The package does seem to be leaking the memory on every query though.

**To Reproduce**
Steps to reproduce the behavior:
You can probably just make a large query again and again and pprof the process to see the memory usage.

**Expected behavior**
Memory should only be used ephemerally by the package.

**Actual behavior**
Gigs of memory leak
**Version**
- Go: go 1.21
- PostgreSQL: pg 14

- pgx: github.com/jackc/pgx/v5 v5.5.5

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Research direction

Start with pgtype.scanPlanString and the pgtype.TextCodec decode path named in the pprof output. Reproduce repeated large queries with Go 1.21 and pgx v5.5.5, then compare memory profiles to determine whether allocations remain retained after each query. Done means the reported growth is explained and a verified change prevents the gigabytes of retained memory.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.