cockroachdb / cockroachdb/cockroach
sql: data race on eval.Context.ParseHelper
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
We just hit a crash in our internal testing on 25.1.0-beta:
```
I250203 04:39:19.362486 1 util/log/flags.go:226 [-] 1 stderr capture started
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x581a10]
goroutine 5318132 gp=0xc012f5f500 m=9 mp=0xc000b00008 [running]:
panic({0x6475d00?, 0xca32750?})
GOROOT/src/runtime/panic.go:779 +0x158 fp=0xc012a41be8 sp=0xc012a41b38 pc=0x49c658
github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1()
pkg/sql/colexecerror/error.go:137 +0x554 fp=0xc012a41d70 sp=0xc012a41be8 pc=0x11af514
panic({0x6475d00?, 0xca32750?})
GOROOT/src/runtime/panic.go:770 +0x132 fp=0xc012a41e20 sp=0xc012a41d70 pc=0x49c632
runtime.panicmem(...)
GOROOT/src/runtime/panic.go:261
runtime.sigpanic()
GOROOT/src/runtime/signal_unix.go:881 +0x378 fp=0xc012a41e80 sp=0xc012a41e20 pc=0x4b5ff8
strings.ToLower({0x0?, 0xb?})
GOROOT/src/strings/strings.go:639 +0x30 fp=0xc012a41f30 sp=0xc012a41e80 pc=0x581a10
github.com/cockroachdb/cockroach/pkg/util/timeutil/pgdate.(*fieldExtract).Extract(0xc010bd3138, {0xc003540324, 0xb})
pkg/util/timeutil/pgdate/field_extract.go:147 +0x197 fp=0xc012a42020 sp=0xc012a41f30 pc=0xcf93b7
github.com/cockroachdb/cockroach/pkg/util/timeutil/pgdate.ParseTimeWithoutTimezone({0x0?, 0x0?, 0xc01304aaf0?}, {0x0?, 0x0?}, {0xc003540324, 0xb}, 0x0?)
pkg/util/timeutil/pgdate/parsing.go:204 +0x165 fp=0xc012a426c0 sp=0xc012a42020 pc=0xcfcf85
github.com/cockroachdb/cockroach/pkg/sql/sem/tree.ParseDTime({0x8875cc0, 0xc010bd2e08}, {0xc003540324, 0xb}, 0x3e8)
pkg/sql/sem/tree/datum.go:2312 +0x1df fp=0xc012a42778 sp=0xc012a426c0 pc=0x1553edf
github.com/cockroachdb/cockroach/pkg/sql/sem/eval.performCastWithoutPrecisionTruncation({0x8851be8, 0xc0285c86e0}, 0xc010bd2e08, {0x88cc1c0?, 0xc010e9c200?}, 0xcb3c900, 0x1)
pkg/sql/sem/eval/cast.go:764 +0x3f5e fp=0xc012a43170 sp=0xc012a42778 pc=0x1ac415e
github.com/cockroachdb/cockroach/pkg/sql/sem/eval.performCast({0x8851be8?, 0xc0285c86e0?}, 0x0?, {0x88cc1c0?, 0xc010e9c200?}, 0xcb3c900, 0x28?)
pkg/sql/sem/eval/cast.go:99 +0x2c fp=0xc012a431b8 sp=0xc012a43170 pc=0x1ac016c
github.com/cockroachdb/cockroach/pkg/sql/sem/eval.PerformCast(...)
pkg/sql/sem/eval/cast.go:64
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.(*castNativeToDatumOp).Next.func1()
bazel-out/k8-opt/bin/pkg/sql/colexec/colexecbase/cast.eg.go:1438 +0x2ed fp=0xc012a43340 sp=0xc012a431b8 pc=0x2d239ad
github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).PerformOperation(0xc02a199710, {0xc012a43428, 0x1, 0x8?}, 0xc012a433d8)
pkg/sql/colmem/allocator.go:443 +0x71 fp=0xc012a43380 sp=0xc012a43340 pc=0x273acd1
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.(*castNativeToDatumOp).Next(0xc010e22a88)
bazel-out/k8-opt/bin/pkg/sql/colexec/colexecbase/cast.eg.go:1347 +0x1d4 fp=0xc012a43440 sp=0xc012a43380 pc=0x2d23674
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*vectorTypeEnforcer).Next(0xc02c9ff450)
pkg/sql/colexec/colexecutils/operator.go:152 +0x24 fp=0xc012a43488 sp=0xc012a43440 pc=0x2c446e4
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecproj.projLEDatumDatumOp.Next({{{{{...}}, {{...}}}, 0xc02a199710, 0x1, 0x3, 0x4, 0x0}})
bazel-out/k8-opt/bin/pkg/sql/colexec/colexecproj/proj_non_const_ops.eg.go:42166 +0x78 fp=0xc012a43550 sp=0xc012a43488 pc=0x313abf8
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecproj.(*projLEDatumDatumOp).Next(0xc016f97140?)
:1 +0x96 fp=0xc012a435f0 sp=0xc012a43550 pc=0x31cd316
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*selBoolOp).Next(0xc010e0c3c0)
pkg/sql/colexec/colexecutils/bool_vec_to_sel.go:137 +0x27 fp=0xc012a43650 sp=0xc012a435f0 pc=0x2c43507
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*BoolVecToSelOp).Next(0xc02c9ff4f0)
pkg/sql/colexec/colexecutils/bool_vec_to_sel.go:57 +0x2c fp=0xc012a436a0 sp=0xc012a43650 pc=0x2c4326c
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*vectorTypeEnforcer).Next(0xc02c9ff540)
pkg/sql/colexec/colexecutils/operator.go:152 +0x24 fp=0xc012a436e8 sp=0xc012a436a0 pc=0x2c446e4
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.constBoolOp.Next({{{{0x8858ce8, 0xc02c9ff540}}, {{0x8851be8, 0xc0285c86e0}}}, 0xc02a199710, 0x5, 0x1})
bazel-out/k8-opt/bin/pkg/sql/colexec/colexecbase/const.eg.go:169 +0x68 fp=0xc012a437b0 sp=0xc012a436e8 pc=0x2d63968
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.(*constBoolOp).Next(0x6a32280?)
:1 +0x68 fp=0xc012a43830 sp=0xc012a437b0 pc=0x2d851e8
github.com/cockroachdb/cockroach/pkg/sql/colexec.(*caseOp).Next.func1()
pkg/sql/colexec/case.go:227 +0xb2 fp=0xc012a438d8 sp=0xc012a43830 pc=0x2d95832
github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).PerformOperation(0xc02a199710, {0xc012a43a48, 0x1, 0xc0285c8600?}, 0xc012a43a20)
pkg/sql/colmem/allocator.go:443 +0x71 fp=0xc012a43918 sp=0xc012a438d8 pc=0x273acd1
github.com/cockroachdb/cockroach/pkg/sql/colexec.(*caseOp).Next(0xc029f56d80)
pkg/sql/colexec/case.go:222 +0x490 fp=0xc012a43a60 sp=0xc012a43918 pc=0x2d95270
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*selBoolOp).Next(0xc010e0c500)
pkg/sql/colexec/colexecutils/bool_vec_to_sel.go:137 +0x27 fp=0xc012a43ac0 sp=0xc012a43a60 pc=0x2c43507
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*BoolVecToSelOp).Next(0xc02c9ff770)
pkg/sql/colexec/colexecutils/bool_vec_to_sel.go:57 +0x2c fp=0xc012a43b10 sp=0xc012a43ac0 pc=0x2c4326c
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.(*simpleProjectOp).Next(0xc0224102a0)
pkg/sql/colexec/colexecbase/simple_project.go:113 +0x28 fp=0xc012a43b68 sp=0xc012a43b10 pc=0x2d1e6a8
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*deselectorOp).Next(0xc022410300)
pkg/sql/colexec/colexecutils/deselector.go:48 +0x32 fp=0xc012a43c28 sp=0xc012a43b68 pc=0x2c43972
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc.(*Outbox).sendBatches.func1()
pkg/sql/colflow/colrpc/outbox.go:268 +0xc2 fp=0xc012a43cf8 sp=0xc012a43c28 pc=0x2c8daa2
github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError(0x50?)
pkg/sql/colexecerror/error.go:162 +0x57 fp=0xc012a43d38 sp=0xc012a43cf8 pc=0x11aef77
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc.(*Outbox).sendBatches(0xc010847220?, {0x88520b8?, 0xc010847220?}, {0x7c902015bbf8?, 0xc028e50400?}, 0x88520b8?, 0xc010847220?)
pkg/sql/colflow/colrpc/outbox.go:259 +0xc6 fp=0xc012a43da0 sp=0xc012a43d38 pc=0x2c8d966
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc.(*Outbox).runWithStream(0xc018412270, {0x88520b8, 0xc010847220}, {0x7c902015bbf8, 0xc028e50400}, 0xc01f82b750, 0xc028e50380)
pkg/sql/colflow/colrpc/outbox.go:397 +0x185 fp=0xc012a43e38 sp=0xc012a43da0 pc=0x2c8e625
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc.(*Outbox).Run(0xc018412270, {0x8851c20, 0xc02a199950}, {0x8800020, 0xc0044c9860}, 0x1, 0xa, 0xc01f82b750, 0x2540be400)
pkg/sql/colflow/colrpc/outbox.go:220 +0x43a fp=0xc012a43f40 sp=0xc012a43e38 pc=0x2c8d2ba
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreator).setupRemoteOutputStream.func1({0x8851c20?, 0xc02a199950?}, 0x0?)
pkg/sql/colflow/vectorized_flow.go:755 +0x86 fp=0xc012a43f98 sp=0xc012a43f40 pc=0x3643ba6
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreator).setupRemoteOutputStream.(*vectorizedFlowCreator).accumulateAsyncComponent.func2.1()
pkg/sql/colflow/vectorized_flow.go:717 +0x5c fp=0xc012a43fe0 sp=0xc012a43f98 pc=0x3643a7c
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc012a43fe8 sp=0xc012a43fe0 pc=0x4d8c61
created by github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreator).setupRemoteOutputStream.(*vectorizedFlowCreator).accumulateAsyncComponent.func2 in goroutine 5318048
pkg/sql/colflow/vectorized_flow.go:715 +0xb7
```
I think this is because we introduced a data race on `eval.Context.ParseHelper` in 048f8333c4f6f5ca1b5089bb51108c0d7c578541. Namely, we reuse the same `eval.Context` among most operators (those that don't explicitly modify it), and if we have a plan that performs date parsing in two operators _concurrently_, the data race will occur.
Jira issue: CRDB-47187
Contributor guide
Assessment
This issue has not been assessed yet.