hashicorp / hashicorp/terraform-plugin-sdk

go data race condition in acceptance tests when `resource.ParallelTest` is used

Open
#319 2 comments 0 reactions 0 assignees View on GitHub
bug subsystem/tests
Dominant language
Go
Stars
485
Forks
244
Avg merge
19h 57m
Merged PRs (30d)
4

Description

### SDK version

```
{
"Path": "github.com/hashicorp/terraform-plugin-sdk",
"Version": "v1.6.0"
}
```

Occurs on 1.5.0 as well.

### Relevant provider source code

```go
resource.ParallelTest(t, resource.TestCase{
```
https://github.com/Didainius/terraform-provider-fake/blob/test-race-condition/fake/resource_fake_one_test.go#L10
### Terraform Configuration Files

This is not specific to any configuration. The _main_ point is to use `resource.ParallelTest` instead of `resource.Test`
```hcl
...
```

### Debug Output

```
# make test
Sometimes it must be run multiple times as it doesn't always hit, but I would say 3 out of 5 runs do
cd fake && TF_ACC=1 go test -race -v .
=== RUN TestAccFakeOne
=== PAUSE TestAccFakeOne
=== RUN TestAccFakeOne2
=== PAUSE TestAccFakeOne2
=== RUN TestAccFakeOne3
=== PAUSE TestAccFakeOne3
=== RUN TestAccFakeOne4
=== PAUSE TestAccFakeOne4
=== RUN TestAccFakeOne5
=== PAUSE TestAccFakeOne5
=== CONT TestAccFakeOne3
=== CONT TestAccFakeOne4
=== CONT TestAccFakeOne5
=== CONT TestAccFakeOne2
=== CONT TestAccFakeOne
==================
WARNING: DATA RACE
Write at 0x00c0003164e8 by goroutine 125:
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).Configure()
/Users/user/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.6.0/internal/helper/plugin/grpc_provider.go:478 +0x34e
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_Configure_Handler()
/Users/user/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.6.0/internal/tfplugin5/tfplugin5.pb.go:3135 +0x2fc
google.golang.org/grpc.(*Server).processUnaryRPC()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:995 +0x994
google.golang.org/grpc.(*Server).handleStream()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:1275 +0x1343
google.golang.org/grpc.(*Server).serveStreams.func1.1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:710 +0xc8

Previous write at 0x00c0003164e8 by goroutine 177:
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).Configure()
/Users/user/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.6.0/internal/helper/plugin/grpc_provider.go:478 +0x34e
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_Configure_Handler()
/Users/user/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.6.0/internal/tfplugin5/tfplugin5.pb.go:3135 +0x2fc
google.golang.org/grpc.(*Server).processUnaryRPC()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:995 +0x994
google.golang.org/grpc.(*Server).handleStream()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:1275 +0x1343
google.golang.org/grpc.(*Server).serveStreams.func1.1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:710 +0xc8

Goroutine 125 (running) created at:
google.golang.org/grpc.(*Server).serveStreams.func1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:708 +0xb8
google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/internal/transport/http2_server.go:429 +0x1679
google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/internal/transport/http2_server.go:470 +0x3d7
google.golang.org/grpc.(*Server).serveStreams()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:706 +0x19a
google.golang.org/grpc.(*Server).handleRawConn.func1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:668 +0x4c

Goroutine 177 (running) created at:
google.golang.org/grpc.(*Server).serveStreams.func1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:708 +0xb8
google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/internal/transport/http2_server.go:429 +0x1679
google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/internal/transport/http2_server.go:470 +0x3d7
google.golang.org/grpc.(*Server).serveStreams()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:706 +0x19a
google.golang.org/grpc.(*Server).handleRawConn.func1()
/Users/user/go/pkg/mod/google.golang.org/grpc@v1.23.0/server.go:668 +0x4c
==================
--- FAIL: TestAccFakeOne5 (0.36s)
testing.go:853: race detected during execution of test
--- FAIL: TestAccFakeOne2 (0.36s)
testing.go:853: race detected during execution of test
--- FAIL: TestAccFakeOne4 (0.36s)
testing.go:853: race detected during execution of test
--- FAIL: TestAccFakeOne3 (0.37s)
testing.go:853: race detected during execution of test
--- FAIL: TestAccFakeOne (0.37s)
testing.go:853: race detected during execution of test
FAIL
FAIL github.com/Didainius/terraform-provider-fake/fake 0.580s
FAIL
make: *** [test] Error 1

```

### Expected Behavior

I would not expect a data race here
### Actual Behavior

Data race occured in go acceptance test
### Steps to Reproduce

We have hit this problem in terraform-provider-vcd but as part of debugging I created a repo (https://github.com/Didainius/terraform-provider-fake/tree/test-race-condition) with fake provider to isolate the problem and ensure our codebase does not misbehave

1. `git clone --single-branch --branch test-race-condition https://github.com/Didainius/terraform-provider-fake.git`
2. Run `make test` which runs `cd fake && TF_ACC=1 go test -race -v .`

**Note** Sometimes it doesn't happen on the first run, but it happens at least 3 times out of 5 for me (usually even more).

**Note2.** Adding --parallel=1 to the test suite `cd fake && TF_ACC=1 go test --parallel=1 -race -v .` causes no data races to occur (quite obvious, but still noting)

### References

I haven't found related issues in this repo so far.

Contributor guide

Open the contributing guide

Research direction

Start by running `cd fake && TF_ACC=1 go test -race -v .` from the linked test repository and review `fake/resource_fake_one_test.go`, where `resource.ParallelTest` reproduces the issue. Then inspect `internal/helper/plugin/grpc_provider.go` around line 478, identified in the race report. Done means repeated parallel acceptance-test runs complete without race warnings, while `--parallel=1` remains unnecessary.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, grpc
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.