pingcap / pingcap/failpoint

Failpoints are getting executed while running tests using `go test -v`

Open
#79 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
895
Forks
67
PR merge metrics
No merged PRs in 30d

Description

Bug Report

  1. What did you do? If possible, provide a recipe for reproducing the error.

I have the following setup (all in the same directory):
go.mod

module example.com/modv1

go 1.20

require (
	github.com/pingcap/failpoint v0.0.0-20220801062533-2eaa32854a6c
	github.com/stretchr/testify v1.8.4
)

require (
	github.com/davecgh/go-spew v1.1.1 // indirect
	github.com/pingcap/errors v0.11.4 // indirect
	github.com/pmezard/go-difflib v1.0.0 // indirect
	gopkg.in/yaml.v3 v3.0.1 // indirect
)

samplestruct.go

package main

import (
	"log"

	"github.com/pingcap/failpoint"
)

type SampleStruct struct {
	cnt int
}

func (s *SampleStruct) increment() {
	if _, _err_ := failpoint.Eval(_curpkg_("testPanic")); _err_ == nil {
		log.Printf("[SampleStruct] SUCCESS!!! Executed testPanic FP: The count for s is %v", s.cnt)
	}
	s.cnt++
}

func (s *SampleStruct) decrement() {
	s.cnt--
}

func (s *SampleStruct) incrementFP() string {
	log.Printf("[SampleStruct] Executing incrementFP with old cnt %v", s.cnt)
	s.cnt++
	log.Printf("[SampleStruct] Executing incrementFP with new cnt %v", s.cnt)
	if _, _err_ := failpoint.Eval(_curpkg_("doubleIncrement")); _err_ == nil {
		s.cnt++
		log.Printf("[SampleStruct] SUCCESS!!! Executed doubleIncrement FP. The count for s is %v", s.cnt)
	}
	return "single"
}

main.go

package main

import (
	"fmt"
	"log"
	"os"
)

func main() {
	log.Printf(fmt.Sprintf("[main] This is main function being executed in an executable at path %v", os.Args[0]))
	//os.Setenv("Go_FAILPOINTS", "main/doubleIncrement=return(true);main/testPanic=return(true)")
	log.Printf(fmt.Sprintf("[main] After FP code in main function"))

	sampleStruct := SampleStruct{cnt: 1}
	log.Printf("[main] sampleStruct cnt before increment is %v", sampleStruct.cnt)
	sampleStruct.increment()
	log.Printf("[main] sampleStruct cnt after increment is %v", sampleStruct.cnt)
	log.Printf("[main] sampleStruct cnt before double increment is %v", sampleStruct.cnt)
	sampleStruct.incrementFP()
	log.Printf("[main] sampleStruct cnt after double increment is %v", sampleStruct.cnt)
}

sample_test.go

package main

import (
	"log"
	"testing"
	"os/exec"
	"os"
	"fmt"
	"bytes"
	"github.com/stretchr/testify/assert"
)

func TestFailpoint(t *testing.T) {
	log.Printf("[TestFailpoint] Executing TestFailpoint")
	mySampleStruct := SampleStruct{cnt: 2}
	assert.Equal(t, mySampleStruct.cnt, 2)

	// fpsToEnable := map[string]struct{}{
	// 	"doubleIncrement": {},
	// }

	// cmd := exec.Command("export", "GO_FAILPOINTS='main/doubleIncrement=return(true);main/testPanic=return(true)'")
	// if err := cmd.Run(); err != nil {
	// 	log.Fatal(fmt.Sprintf("Encountered error while setting up the GO_FAILPOINTS environment variable. Error: %v", err))
	// } else {
	// 	log.Printf("Successfully set up the GO_FAILPOINTS environment variable")
	// }

	os.Setenv("GO_FAILPOINTS","'main/doubleIncrement=return(true);main/testPanic=return(true)'")
	log.Printf("[TestFailpoint] Successfully set up the GO_FAILPOINTS environment variable")

	cmd := exec.Command("./modv1")
	var out bytes.Buffer
	var stderr bytes.Buffer
	cmd.Stdout = &out
	cmd.Stderr = &stderr
	if err := cmd.Start(); err != nil {
		log.Printf(fmt.Sprintf("[TestFailpoint] Error while starting modv1 executable. Error: %v", err))
		log.Printf("[TestFailpoint] Command error is: " + stderr.String())
	} else {
		log.Printf("[TestFailpoint] Successfully started modv1 executable")
		log.Printf("[TestFailpoint] Result: " + out.String() + ",Error: " + stderr.String())
	}

	if err := cmd.Wait(); err != nil {
		log.Printf(fmt.Sprintf("[TestFailpoint] Error while waiting for modv1 executable. Error: %v", err))
		log.Printf("[TestFailpoint] Command error is: " + stderr.String())
	} else {
		log.Printf("[TestFailpoint] Successfully waited for modv1 executable")
		log.Printf("[TestFailpoint] Result: " + out.String() + ",Error: " + stderr.String())
	}

	//failpoint.Enable("doubleIncrement", "return(1)")
	log.Printf("[TestFailpoint] Calling increment")
	mySampleStruct.increment()
	log.Printf("[TestFailpoint] Called increment")
	log.Printf("[TestFailpoint] Calling incrementFP")
	result := mySampleStruct.incrementFP()
	log.Printf("[TestFailpoint] Called incrementFP")
	assert.Equal(t, mySampleStruct.cnt, 4)
	assert.Equal(t, result, "single")

	os.Unsetenv("GO_FAILPOINTS")
	log.Printf("[TestFailpoint] Successfully unset the GO_FAILPOINTS environment variable")
}
  1. What did you expect to see?
    When I run go build and set the environment variable export GO_FAILPOINTS=main/doubleIncrement=return(true);main/testPanic=return(true) and then execute the ./modv1 executable I see that the failpoints in samplestruct.go are hit and the failpoint closure is executed. I expected same behavior when I run go test -v i.e. when the TestFailpoint test is executed.

  2. What did you see instead?
    I did not see the failpoints in samplestruct.go getting executed while running the test TestFalipoint. Is this expected behavior? Is the failpoint code executed only in the compiled binary modv1 and not during test execution even though the environment variable GO_FAILPOINTS is kept around (and additionally also set/unset during) test execution?

  3. Versions of the failpoint

$ ./failpoint-ctl -V
ReleaseVersion ed9079f
BuildTS 2023-09-01 02:53:58
GitHash ed9079f47f3761cbbeadaded49d48e3264c23427
GitBranch fp-testing
GoVersion go version go1.21.0 darwin/arm64

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the report with samplestruct.go, main.go, and sample_test.go by comparing go test -v with the built ./modv1 executable. Inspect how GO_FAILPOINTS is set and consumed during TestFailpoint and verify whether failpoints run in the test binary. Done means the behavior is explained and the expected test-time behavior is covered by a reproducible test or clear documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.