Excess expectations in one test affect expectations in later tests in the test group

[JackNWhite]

CMock 2.6.0, SPARC-V8 GCC 4.2.1

In my unit tests, I have a test group for each function under test. Let's call my function under test result_t FUT(structural_t *p_input).

Originally, FUT() assumed reported a problem if the pointer was NULL:

result_t
FUT(structural_t *p_input)
{
  result_t result = OK;
  uint32_t thing  = 0u;

  if (NULL == p_input)
    result = NULL_PTR_ERR;

  if (true == p_input->some_var)
  {
    result = otherStuff(p_input); // May also cause call to generateFailureReport()
    thing = 1;
  }

 if (result != OK)
 {
    generateFailureReport(result, thing);
 }

  return result;
}

Because the failure report came to rely on the pointer, FUT() was changed to check the pointer and exclude the failure report if it's null:

result_t
FUT(structural_t *p_input)
{
  result_t result = OK;

  if (NULL == p_input)
  {
    result = NULL_PTR_ERR;
  }

  else
  {
    if (true == p_input->some_var)
    {
      result = otherStuff(p_input); // May also cause call to generateFailureReport()
    }

    if (result != OK)
    {
      generateFailureReport(result, p_input->stuff);
    }
  }

  return result;
}

The original unit tests looked a bit like this:

TEST_GROUP(FUT);

TEST(FUT, NullInput)
{
  generateFailureReport_Expect(NULL_PTR_ERR, 0);
  
  result_t result = FUT(NULL);

  TEST_ASSERT_EQUAL(FAIL, result);
}

TEST(FUT, OtherStuffHappensAndCausesFailureReport)
{
   generateFailureReport_Expect(OTHER_ERR, SOME_STUFF);
  
  result_t result = FUT(&actual_input);

  TEST_ASSERT_EQUAL(OTHER_ERR, result);
}

After making the change in the production code, there is no longer an expectation of the generateFailureReport() mock being called in the NullInput test, however, that test does not fail. Instead, the expectation generateFailureReport_Expect(NULL_PTR_ERR) is saved for test OtherStuffHappensAndCausesFailureReport.

The upshot is that TEST(FUT, NullInput) succeeds, even though the CMock expectation is incorrect (there shouldn't be one) and TEST(FUT, OtherStuffHappensAndCausesFailureReport) fails, even though its CMock expectation is correct.

This become difficult when you have a long train of tests in one group and you need to remove a few of these ignored expectations.

The example test should look like this:

TEST_GROUP(FUT);

TEST(FUT, NullInput)
{
  result_t result = FUT(NULL);

  TEST_ASSERT_EQUAL(FAIL, result);
}

TEST(FUT, OtherStuffHappensAndCausesFailureReport)
{
   generateFailureReport_Expect(OTHER_ERR, SOME_STUFF);
  
  result_t result = FUT(&actual_input);

  TEST_ASSERT_EQUAL(OTHER_ERR, result);
}