jackc / jackc/pgx

Map.memoizedEncodePlans concurrent map writes (v5.7.2)

Open
#2,294 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
14.3k
Forks
1.1k
Avg merge
6d 9h
Merged PRs (30d)
11

Description

Describe the bug
When using a simple protocol and concurrent queries, a panic occurs with the error concurrent map writes in the memoizedEncodePlans function in the pgtype.Map module.

fatal error: concurrent map writes
...
github.com/jackc/pgx/v5/pgtype.(*Map).planEncodeDepth(0xc01173e5b0, 0x0, 0x0, {0x3aeda0, 0x648cfc0}, 0x0)
        /-S/vendor/github.com/jackc/pgx/v5/pgtype/pgtype.go:1254 +0x206 fp=0xc00cd08b08 sp=0xc00cd08a58 pc=0x3ef3bc6
github.com/jackc/pgx/v5/pgtype.(*Map).PlanEncode(...)
        /-S/vendor/github.com/jackc/pgx/v5/pgtype/pgtype.go:1234
github.com/jackc/pgx/v5/pgtype.(*Map).Encode(0xc01173e5b0, 0x0, 0x0, {0x3aeda0, 0x648cfc0}, {0x6488500, 0x0, 0x0})
        /-S/vendor/github.com/jackc/pgx/v5/pgtype/pgtype.go:2001 +0x12d fp=0xc00cd08b58 sp=0xc00cd08b08 pc=0x3ef99ed
github.com/jackc/pgx/v5.convertSimpleArgument(0x4c8380?, {0x3aeda0?, 0x648cfc0?})
        /-S/vendor/github.com/jackc/pgx/v5/values.go:17 +0x34 fp=0xc00cd08ba0 sp=0xc00cd08b58 pc=0x3fe8e14
github.com/jackc/pgx/v5.(*Conn).sanitizeForSimpleQuery(0xc014d0e900, {0xc021f80340, 0x185}, {0xc01aa19190, 0x1, 0xc01aa19190?})
        /-S/vendor/github.com/jackc/pgx/v5/conn.go:1244 +0x166 fp=0xc00cd08bf8 sp=0xc00cd08ba0 pc=0x3fdfba6
github.com/jackc/pgx/v5.(*Conn).Query(0xc014d0e900, {0x121c170?, 0xc044094230?}, {0xc021f80340, 0x185}, {0xc01aa19180, 0x2, 0x2})
        /-S/vendor/github.com/jackc/pgx/v5/conn.go:838 +0xbab fp=0xc00cd08de0 sp=0xc00cd08bf8 pc=0x3fdc06b
github.com/jackc/pgx/v5/stdlib.(*Conn).QueryContext(0xc01bb55e40, {0x121c170, 0xc044094230}, {0xc021f80340, 0x185}, {0xc03d1b9f80, 0x1, 0x5f32600?})
        /-S/vendor/github.com/jackc/pgx/v5/stdlib/sql.go:493 +0x32d fp=0xc00cd08ed0 sp=0xc00cd08de0 pc=0x40223ed
...

Expected behavior

No concurrent map writes panic should occur.

Actual behavior

A panic with the concurrent map writes error occurs when executing concurrent queries, indicating an attempt to write to the map simultaneously.

Version

Go: 1.22.5
pgx: v5.7.2

Additional context

Part of the core dump:

(dlv) bt
 0  0x0000000002fd1ca1 in runtime.raise
    at /-S/contrib/go/_std_1.22/src/runtime/sys_linux_amd64.s:154
 1  0x0000000002fb1805 in runtime.dieFromSignal
    at /-S/contrib/go/_std_1.22/src/runtime/signal_unix.go:923
 2  0x0000000002fb1f06 in runtime.sigfwdgo
    at /-S/contrib/go/_std_1.22/src/runtime/signal_unix.go:1128
 3  0x0000000002fb0425 in runtime.sigtrampgo
    at /-S/contrib/go/_std_1.22/src/runtime/signal_unix.go:432
 4  0x0000000002fd1ca1 in runtime.raise
    at /-S/contrib/go/_std_1.22/src/runtime/sys_linux_amd64.s:153
 5  0x0000000002fb1805 in runtime.dieFromSignal
    at /-S/contrib/go/_std_1.22/src/runtime/signal_unix.go:923
 6  0x0000000002f99c3a in runtime.crash
    at /-S/contrib/go/_std_1.22/src/runtime/signal_unix.go:1005
 7  0x0000000002f99c3a in runtime.fatalthrow.func1
    at /-S/contrib/go/_std_1.22/src/runtime/panic.go:1203
 8  0x0000000002fce48a in runtime.systemstack
    at /-S/contrib/go/_std_1.22/src/runtime/asm_amd64.s:509
 9  0x0000000002fce428 in runtime.systemstack_switch
    at /-S/contrib/go/_std_1.22/src/runtime/asm_amd64.s:474
10  0x0000000002f99b85 in runtime.fatalthrow
    at /-S/contrib/go/_std_1.22/src/runtime/panic.go:1192
11  0x0000000002f9981c in runtime.fatal
    at /-S/contrib/go/_std_1.22/src/runtime/panic.go:1042
12  0x0000000002f6f54e in runtime.mapassign
    at /-S/contrib/go/_std_1.22/src/runtime/map.go:596
13  0x0000000003ef3bc6 in github.com/jackc/pgx/v5/pgtype.(*Map).planEncodeDepth
    at /-S/vendor/github.com/jackc/pgx/v5/pgtype/pgtype.go:1254
14  0x0000000003ef99ed in github.com/jackc/pgx/v5/pgtype.(*Map).PlanEncode
    at /-S/vendor/github.com/jackc/pgx/v5/pgtype/pgtype.go:1234
15  0x0000000003ef99ed in github.com/jackc/pgx/v5/pgtype.(*Map).Encode
    at /-S/vendor/github.com/jackc/pgx/v5/pgtype/pgtype.go:2001
16  0x0000000003fe8e14 in github.com/jackc/pgx/v5.convertSimpleArgument
    at /-S/vendor/github.com/jackc/pgx/v5/values.go:17
17  0x0000000003fdfba6 in github.com/jackc/pgx/v5.(*Conn).sanitizeForSimpleQuery
    at /-S/vendor/github.com/jackc/pgx/v5/conn.go:1244
18  0x0000000003fdc06b in github.com/jackc/pgx/v5.(*Conn).Query
    at /-S/vendor/github.com/jackc/pgx/v5/conn.go:838
19  0x00000000040223ed in github.com/jackc/pgx/v5/stdlib.(*Conn).QueryContext
    at /-S/vendor/github.com/jackc/pgx/v5/stdlib/sql.go:493

...

oid = 0
format = 0

Possible solution

Protect map modification https://github.com/jackc/pgx/blob/04bcc0219dc3acf67f27e68decd6dffe97334779/pgtype/pgtype.go#L1235 with mutex.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pgtype/pgtype.go around planEncodeDepth and the memoizedEncodePlans map modification identified in the issue. Reproduce concurrent simple-protocol queries, then verify the change under concurrent execution or the race detector. Done means concurrent queries no longer produce a concurrent map writes panic.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.