Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make mocks independent at subtest level #34

Open
rahmatrhd opened this issue May 15, 2023 · 1 comment
Open

Make mocks independent at subtest level #34

rahmatrhd opened this issue May 15, 2023 · 1 comment
Assignees
Labels

Comments

@rahmatrhd
Copy link
Member

rahmatrhd commented May 15, 2023

Summary
Some of our tests use shared mock instances across subtests within a test suite. This could cause expectations set from subtest A still leave open in subtest B if it's not fulfilled in subtest A. This is due to a limitation from testify that before/after test hook is not available for subtest. Currently, we avoid that issue by ensuring the number of expectations matches the actual executions using .Once(). But a similar issue still could happen in parallel tests.

The second issue is in a complex function that makes many calls from dependencies, we had to set the expectations for all executed calls in each subtest resulting in the code being not DRY and making the test harder to read/understand.

Proposed solution
In testify v1.8.2, before and after test hooks are now supported for subtests. We can use this to ensure every subtest starts with empty expectations.

To also fix the second issue as well as for the parallel testing, mocks need to be decoupled from the suite struct and can be initialized at anytime when needed.

Proposing following approach:

package appeal_test

type mock struct {
	mockRepository *appeal.Repository
	mockXService   *appeal.XService

	service *appeal.Service
}

func newMock() mock {
	m := mock{
		mockRepository: new(mockRepository),
		mockXService:   new(mockXService),
	}
	m.service = appeal.NewService(m.mockRepository, m.mockXService)
	return m
}

func (m *mock) setSuccessExpectations() {
	// always expect mock.Anything in params and return success values
	m.mockRepository.EXPECT().Create(mock.Anything, mock.Anything).Return(nil)
	m.mockXService.EXPECT().Create(mock.Anything, mock.Anything).Return(nil)
}

type AppealTestSuite struct {
	suite.Suite
	mock // shared mock
}

func (s *AppealTestSuite) SetupSubTest() {
	s.mock = newMock() // reinitialize mock for each sub test
}

// usage example
func (s *AppealTestSuite) TestCreate() {
	s.Run("repository returns an error", func() {
		s.mockRepository.EXPECT().Create(mock.Anything, mock.Anything).Return(errors.New("error"))
		_, err := s.service.Create(context.Background(), appeal.CreateRequest{})
		s.Error(err)
	})

	s.Run("test specific case", func() {
		s.setSuccessExpectations() // set common expectations
		
		// only specify expectations related to the tested case:
		s.mockXService.EXPECT().Create(mock.Anything, mock.Anything).Return(errors.New("error"))

		_, err := s.service.Create(context.Background(), appeal.CreateRequest{})
		s.Error(err)
	})

	s.Run("parallel tests", func() {
		testCases := []struct {}{
			// ...
		}

		for _, tc := range testCases {
			tc := tc
			s.Run(tc.name, func() {
				s.T().Parallel()

				independentMock := newMock()
				// set expectations for independentMock
				// ...
				//_, err := independentMock.service.Create(context.Background(), appeal.CreateRequest{})
			})
		}
	}
}
  1. decoupled mocks from suite struct into an independent struct
  2. restart mocks in every subtests s.mock = newMock()
  3. for parallel tests, use an independent mock instead of a shared one
  4. introduced setSuccessExpectations() to avoid writing common expectations multiple times across subtests

Note: currently trying out this approach for appeal service as a POC, will raise the PR sson

@rahmatrhd rahmatrhd added the test label May 15, 2023
@bsushmith
Copy link
Collaborator

bsushmith commented Jun 1, 2023

func (m *mock) setSuccessExpectations() {
	// always expect mock.Anything in params and return success values
	m.mockRepository.EXPECT().Create(mock.Anything, mock.Anything).Return(nil)
	m.mockXService.EXPECT().Create(mock.Anything, mock.Anything).Return(nil)
}

I don't think we should have something like this which always expects only mock.Anything. We can try to abstract common expectations, but that should not come at a cost of proper expectation checks.

I understand that we do use mock.Anything pervasively in our code, but that is something that should change rather than be embraced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

2 participants