Failpoints are getting executed while running tests using `go test -v`
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 895
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
Bug Report
- What did you do? If possible, provide a recipe for reproducing the error.
I have the following setup (all in the same directory):
go.mod
module example.com/modv1
go 1.20
require (
github.com/pingcap/failpoint v0.0.0-20220801062533-2eaa32854a6c
github.com/stretchr/testify v1.8.4
)
require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/pingcap/errors v0.11.4 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
samplestruct.go
package main
import (
"log"
"github.com/pingcap/failpoint"
)
type SampleStruct struct {
cnt int
}
func (s *SampleStruct) increment() {
if _, _err_ := failpoint.Eval(_curpkg_("testPanic")); _err_ == nil {
log.Printf("[SampleStruct] SUCCESS!!! Executed testPanic FP: The count for s is %v", s.cnt)
}
s.cnt++
}
func (s *SampleStruct) decrement() {
s.cnt--
}
func (s *SampleStruct) incrementFP() string {
log.Printf("[SampleStruct] Executing incrementFP with old cnt %v", s.cnt)
s.cnt++
log.Printf("[SampleStruct] Executing incrementFP with new cnt %v", s.cnt)
if _, _err_ := failpoint.Eval(_curpkg_("doubleIncrement")); _err_ == nil {
s.cnt++
log.Printf("[SampleStruct] SUCCESS!!! Executed doubleIncrement FP. The count for s is %v", s.cnt)
}
return "single"
}
main.go
package main
import (
"fmt"
"log"
"os"
)
func main() {
log.Printf(fmt.Sprintf("[main] This is main function being executed in an executable at path %v", os.Args[0]))
//os.Setenv("Go_FAILPOINTS", "main/doubleIncrement=return(true);main/testPanic=return(true)")
log.Printf(fmt.Sprintf("[main] After FP code in main function"))
sampleStruct := SampleStruct{cnt: 1}
log.Printf("[main] sampleStruct cnt before increment is %v", sampleStruct.cnt)
sampleStruct.increment()
log.Printf("[main] sampleStruct cnt after increment is %v", sampleStruct.cnt)
log.Printf("[main] sampleStruct cnt before double increment is %v", sampleStruct.cnt)
sampleStruct.incrementFP()
log.Printf("[main] sampleStruct cnt after double increment is %v", sampleStruct.cnt)
}
sample_test.go
package main
import (
"log"
"testing"
"os/exec"
"os"
"fmt"
"bytes"
"github.com/stretchr/testify/assert"
)
func TestFailpoint(t *testing.T) {
log.Printf("[TestFailpoint] Executing TestFailpoint")
mySampleStruct := SampleStruct{cnt: 2}
assert.Equal(t, mySampleStruct.cnt, 2)
// fpsToEnable := map[string]struct{}{
// "doubleIncrement": {},
// }
// cmd := exec.Command("export", "GO_FAILPOINTS='main/doubleIncrement=return(true);main/testPanic=return(true)'")
// if err := cmd.Run(); err != nil {
// log.Fatal(fmt.Sprintf("Encountered error while setting up the GO_FAILPOINTS environment variable. Error: %v", err))
// } else {
// log.Printf("Successfully set up the GO_FAILPOINTS environment variable")
// }
os.Setenv("GO_FAILPOINTS","'main/doubleIncrement=return(true);main/testPanic=return(true)'")
log.Printf("[TestFailpoint] Successfully set up the GO_FAILPOINTS environment variable")
cmd := exec.Command("./modv1")
var out bytes.Buffer
var stderr bytes.Buffer
cmd.Stdout = &out
cmd.Stderr = &stderr
if err := cmd.Start(); err != nil {
log.Printf(fmt.Sprintf("[TestFailpoint] Error while starting modv1 executable. Error: %v", err))
log.Printf("[TestFailpoint] Command error is: " + stderr.String())
} else {
log.Printf("[TestFailpoint] Successfully started modv1 executable")
log.Printf("[TestFailpoint] Result: " + out.String() + ",Error: " + stderr.String())
}
if err := cmd.Wait(); err != nil {
log.Printf(fmt.Sprintf("[TestFailpoint] Error while waiting for modv1 executable. Error: %v", err))
log.Printf("[TestFailpoint] Command error is: " + stderr.String())
} else {
log.Printf("[TestFailpoint] Successfully waited for modv1 executable")
log.Printf("[TestFailpoint] Result: " + out.String() + ",Error: " + stderr.String())
}
//failpoint.Enable("doubleIncrement", "return(1)")
log.Printf("[TestFailpoint] Calling increment")
mySampleStruct.increment()
log.Printf("[TestFailpoint] Called increment")
log.Printf("[TestFailpoint] Calling incrementFP")
result := mySampleStruct.incrementFP()
log.Printf("[TestFailpoint] Called incrementFP")
assert.Equal(t, mySampleStruct.cnt, 4)
assert.Equal(t, result, "single")
os.Unsetenv("GO_FAILPOINTS")
log.Printf("[TestFailpoint] Successfully unset the GO_FAILPOINTS environment variable")
}
-
What did you expect to see?
When I rungo buildand set the environment variableexport GO_FAILPOINTS=main/doubleIncrement=return(true);main/testPanic=return(true)and then execute the./modv1executable I see that the failpoints insamplestruct.goare hit and the failpoint closure is executed. I expected same behavior when I rungo test -vi.e. when theTestFailpointtest is executed. -
What did you see instead?
I did not see the failpoints insamplestruct.gogetting executed while running the testTestFalipoint. Is this expected behavior? Is the failpoint code executed only in the compiled binarymodv1and not during test execution even though the environment variableGO_FAILPOINTSis kept around (and additionally also set/unset during) test execution? -
Versions of the failpoint
$ ./failpoint-ctl -V
ReleaseVersion ed9079f
BuildTS 2023-09-01 02:53:58
GitHash ed9079f47f3761cbbeadaded49d48e3264c23427
GitBranch fp-testing
GoVersion go version go1.21.0 darwin/arm64
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the report with samplestruct.go, main.go, and sample_test.go by comparing go test -v with the built ./modv1 executable. Inspect how GO_FAILPOINTS is set and consumed during TestFailpoint and verify whether failpoints run in the test binary. Done means the behavior is explained and the expected test-time behavior is covered by a reproducible test or clear documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100