hashicorp / hashicorp/terraform-plugin-sdk
go data race condition in acceptance tests when `resource.ParallelTest` is used
- Dominant language
- Go
- Stars
- 485
- Forks
- 244
- Avg merge
- 19h 57m
- Merged PRs (30d)
- 4
Description
### SDK version
```
{
"Path": "github.com/hashicorp/terraform-plugin-sdk",
"Version": "v1.6.0"
}
```
Occurs on 1.5.0 as well.
### Relevant provider source code
```go
resource.ParallelTest(t, resource.TestCase{
```
https://github.com/Didainius/terraform-provider-fake/blob/test-race-condition/fake/resource_fake_one_test.go#L10
### Terraform Configuration Files
This is not specific to any configuration. The _main_ point is to use `resource.ParallelTest` instead of `resource.Test`
```hcl
...
```
### Debug Output
```
# make test
Sometimes it must be run multiple times as it doesn't always hit, but I would say 3 out of 5 runs do
cd fake && TF_ACC=1 go test -race -v .
=== RUN TestAccFakeOne
=== PAUSE TestAccFakeOne
=== RUN TestAccFakeOne2
=== PAUSE TestAccFakeOne2
=== RUN TestAccFakeOne3
=== PAUSE TestAccFakeOne3
=== RUN TestAccFakeOne4
=== PAUSE TestAccFakeOne4
=== RUN TestAccFakeOne5
=== PAUSE TestAccFakeOne5
=== CONT TestAccFakeOne3
=== CONT TestAccFakeOne4
=== CONT TestAccFakeOne5
=== CONT TestAccFakeOne2
=== CONT TestAccFakeOne
==================
WARNING: DATA RACE
Write at 0x00c0003164e8 by goroutine 125:
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).Configure()
/Users/user/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.6.0/internal/helper/plugin/grpc_provider.go:478 +0x34e
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_Configure_Handler()
/Users/user/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.6.0/internal/tfplugin5/tfplugin5.pb.go:3135 +0x2fc
google.golang.org/grpc.(*Server).processUnaryRPC()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:995 +0x994
google.golang.org/grpc.(*Server).handleStream()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:1275 +0x1343
google.golang.org/grpc.(*Server).serveStreams.func1.1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:710 +0xc8
Previous write at 0x00c0003164e8 by goroutine 177:
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).Configure()
/Users/user/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.6.0/internal/helper/plugin/grpc_provider.go:478 +0x34e
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_Configure_Handler()
/Users/user/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.6.0/internal/tfplugin5/tfplugin5.pb.go:3135 +0x2fc
google.golang.org/grpc.(*Server).processUnaryRPC()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:995 +0x994
google.golang.org/grpc.(*Server).handleStream()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:1275 +0x1343
google.golang.org/grpc.(*Server).serveStreams.func1.1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:710 +0xc8
Goroutine 125 (running) created at:
google.golang.org/grpc.(*Server).serveStreams.func1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:708 +0xb8
google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/internal/transport/http2_server.go:429 +0x1679
google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/internal/transport/http2_server.go:470 +0x3d7
google.golang.org/grpc.(*Server).serveStreams()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:706 +0x19a
google.golang.org/grpc.(*Server).handleRawConn.func1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:668 +0x4c
Goroutine 177 (running) created at:
google.golang.org/grpc.(*Server).serveStreams.func1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:708 +0xb8
google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/internal/transport/http2_server.go:429 +0x1679
google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/internal/transport/http2_server.go:470 +0x3d7
google.golang.org/grpc.(*Server).serveStreams()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:706 +0x19a
google.golang.org/grpc.(*Server).handleRawConn.func1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:668 +0x4c
==================
--- FAIL: TestAccFakeOne5 (0.36s)
testing.go:853: race detected during execution of test
--- FAIL: TestAccFakeOne2 (0.36s)
testing.go:853: race detected during execution of test
--- FAIL: TestAccFakeOne4 (0.36s)
testing.go:853: race detected during execution of test
--- FAIL: TestAccFakeOne3 (0.37s)
testing.go:853: race detected during execution of test
--- FAIL: TestAccFakeOne (0.37s)
testing.go:853: race detected during execution of test
FAIL
FAIL github.com/Didainius/terraform-provider-fake/fake 0.580s
FAIL
make: *** [test] Error 1
```
### Expected Behavior
I would not expect a data race here
### Actual Behavior
Data race occured in go acceptance test
### Steps to Reproduce
We have hit this problem in terraform-provider-vcd but as part of debugging I created a repo (https://github.com/Didainius/terraform-provider-fake/tree/test-race-condition) with fake provider to isolate the problem and ensure our codebase does not misbehave
1. `git clone --single-branch --branch test-race-condition https://github.com/Didainius/terraform-provider-fake.git`
2. Run `make test` which runs `cd fake && TF_ACC=1 go test -race -v .`
**Note** Sometimes it doesn't happen on the first run, but it happens at least 3 times out of 5 for me (usually even more).
**Note2.** Adding --parallel=1 to the test suite `cd fake && TF_ACC=1 go test --parallel=1 -race -v .` causes no data races to occur (quite obvious, but still noting)
### References
I haven't found related issues in this repo so far.
Contributor guide
Research direction
Start by running `cd fake && TF_ACC=1 go test -race -v .` from the linked test repository and review `fake/resource_fake_one_test.go`, where `resource.ParallelTest` reproduces the issue. Then inspect `internal/helper/plugin/grpc_provider.go` around line 478, identified in the race report. Done means repeated parallel acceptance-test runs complete without race warnings, while `--parallel=1` remains unnecessary.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, grpc
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100