Bug: Unexpected method calls are "forgotten"

Open
#608 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
go
Domain
testing-qa

Research direction

Start with the reproducer in the issue and inspect mock/mock.go, especially Mock.MethodCalled and AssertExpectations. Reproduce the suppressed panic, then trace how unexpected calls are recorded or reported. Done means the unexpected call still panics and is also surfaced by AssertExpectations without breaking normal mock behavior.

Written by the indexing model from the issue text.

Description

bug pkg-mock

In a nutshell: this test passes, despite the expectations being incorrect:

https://go.dev/play/p/2r54p3TxXqr

package durable

import (
	"testing"
	"github.com/stretchr/testify/mock"
)

type mymock struct {
	mock.Mock
}

func (m *mymock) Run() error {
	args := m.Called()
	return args.Error(0)
}

func TestPanics(t *testing.T) {
	m := new(mymock)
	run := func() {
		defer func() { recover() } () // intentionally suppress panic

		m.On("Run").Return(nil).Once()

		m.Run()
		m.Run() // panics
		t.Fatal("Unreachable")
	}

	run()

	m.AssertExpectations(t) // passes
}

Granted, this isn't great code. But panic-capturing code is somewhat common in frameworks, and this can lead to tests passing despite clearly having incorrect mocks. The fact that AssertExpectations doesn't match behavior seems like a bug in testify.


So far as I can tell, this is caused by this code below, in mock/mock.go.
Because the panic occurs before the call.totalCalls++ op (and possibly others!), no record of a panic-ing call exists except for the panic.

func (m *Mock) MethodCalled(methodName string, arguments ...interface{}) Arguments {
	m.mutex.Lock()
	found, call := m.findExpectedCall(methodName, arguments...)

	if found < 0 {
		// we have to fail here - because we don't know what to do
		// as the return arguments.  This is because:
		//
		//   a) this is a totally unexpected call to this method,
		//   b) the arguments are not what was expected, or
		//   c) the developer has forgotten to add an accompanying On...Return pair.

		closestFound, closestCall := m.findClosestCall(methodName, arguments...)
		m.mutex.Unlock()

		if closestFound {
			panic(fmt.Sprintf("\n\nmock: Unexpected Method Call\n-----------------------------\n\n%s\n\nThe closest call I have is: \n\n%s\n\n%s\n", callString(methodName, arguments, true), callString(methodName, closestCall.Arguments, true), diffArguments(closestCall.Arguments, arguments)))
		} else {
			panic(fmt.Sprintf("\nassert: mock: I don't know what to return because the method call was unexpected.\n\tEither do Mock.On(\"%s\").Return(...) first, or remove the %s() call.\n\tThis method was unexpected:\n\t\t%s\n\tat: %s", methodName, methodName, callString(methodName, arguments, true), assert.CallerInfo()))
		}
	}

	if call.Repeatability == 1 {
		call.Repeatability = -1
	} else if call.Repeatability > 1 {
		call.Repeatability--
	}
	call.totalCalls++
	// ...
}

I don't know what all would be involved in correcting this though. Ideally this should both panic (which generally aborts the test prematurely) and record the error for reporting in AssertExpectations in case it's suppressed.

Dominant language
Go
Stars
26.2k
Forks
1.9k
Avg merge
2d 4h
Merged PRs (30d)
2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stretchr/testify

All issues in stretchr/testify

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.