Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(xUnitXmlPublisher): Include assertion message in test output #208

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

stefanbesler
Copy link
Contributor

@stefanbesler stefanbesler commented Mar 18, 2023

This PR write the actual assertion failure to the output of xUnitXmlPublisher. Previously only the message from test was written into message.

@dfreiberger fyi

@stefanbesler stefanbesler force-pushed the feature/xUnitXmlPublisher_AssertionFailure branch from 55a1946 to c4f6242 Compare November 18, 2023 16:37
@stefanbesler stefanbesler force-pushed the feature/xUnitXmlPublisher_AssertionFailure branch from c4f6242 to c7c8563 Compare November 18, 2023 16:40
@stefanbesler
Copy link
Contributor Author

stefanbesler commented Nov 18, 2023

Maybe some explanation on how to verify this:

  • Enable xUnit publishing by setting the following TcUnit parameters in TcUnitVerifier.plcproj
    • XUNITENABLEPUBLISH = TRUE

image

If any test fails, the C:\tcunit_xunit_testresults.xml will now correctly show the EXP and ACT values in the assertion message

<testsuites disabled="" failures="116" tests="144" time="6.1428994">
   <testsuite id="0" name="PRG_TEST.PrimitiveTypes" tests="46" failures="23" time="1.4893e-3">
      <testcase name="Test_ANY_Equals" classname="PRG_TEST.PrimitiveTypes" time="2.35e-5" status="PASS" />
      <testcase name="Test_ANY_Differ_DataType" classname="PRG_TEST.PrimitiveTypes" time="1.47e-5" status="FAIL">
         <failure message="Values differ; EXP: (Type class = INT), ACT: (Type class = UINT)" type="Array2D_LREAL" />
      </testcase>
      <testcase name="Test_BOOL_Equals" classname="PRG_TEST.PrimitiveTypes" time="3.8e-6" status="PASS" />
      <testcase name="Test_BOOL_Differ" classname="PRG_TEST.PrimitiveTypes" time="1.18e-5" status="FAIL">
         <failure message="Values differ; EXP: TRUE, ACT: FALSE" type="Array2D_LREAL" />
      </testcase>
      <testcase name="Test_BYTE_Equals" classname="PRG_TEST.PrimitiveTypes" time="4.4e-6" status="PASS" />
      <testcase name="Test_BYTE_Differ" classname="PRG_TEST.PrimitiveTypes" time="1.14e-5" status="FAIL">
         <failure message="Values differ; EXP: 0xAB, ACT: 0xCD" type="Array2D_LREAL" />
      </testcase>

Without the PR, failure > message only shows "Values differ"

@sagatowski
Copy link
Member

sagatowski commented Nov 23, 2023

I'll start to look into this as soon as your previous PR is fixed/completed.

See: #210 & #226

@sagatowski
Copy link
Member

@stefanbesler Have you noticed the problems described in #226?

@dfreiberger
Copy link
Contributor

@stefanbesler thanks for this, it will be good to have these more granular results.

@TimSalzmann
Copy link

@stefanbesler, Hi, I really want to have this feature. If I can help, please let me know

@sagatowski
Copy link
Member

I agree, this is a great feature.
I do however have to get the stuff fixed that will be included in version 1.3 first.
For this milestone (https://github.com/tcunit/TcUnit/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+1.3.0.0%22) there is only one ticket left to be solved, and this one has to be solved first before I look into integrating more PRs.

@dfreiberger & @TimSalzmann, you are of course welcome to help me figure out the problem!

@sagatowski sagatowski added this to the Release 1.4.0.0 milestone Nov 30, 2023
@tcunit tcunit deleted a comment from iadonkey Dec 2, 2023
@tcunit tcunit deleted a comment from iadonkey Dec 2, 2023
@tcunit tcunit deleted a comment from iadonkey Dec 2, 2023
@tcunit tcunit deleted a comment from iadonkey Dec 2, 2023
@sagatowski
Copy link
Member

Realized that the discussion in this PR went very OffTopic, cleared it up. Sorry for that!

@stefanbesler I'm working on the last pieces to release 1.3. After that, I will look into this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants