Merge remote-tracking branch 'upstream/master' into feature/xUnitXmlP…

…ublisher_AssertionFailure

.gitignore

@@ -369,4 +369,11 @@ healthchecksdb
 MigrationBackup/
 
 # Ionide (cross platform F# VS Code tools) working folder
-.ionide/
+.ionide/
+
+# Jekyll
+_site
+.sass-cache
+.jekyll-cache
+.jekyll-metadata
+vendor

CONTRIBUTING.md

@@ -10,8 +10,8 @@ We love your input! We want to make contributing to this project as easy and tra
 ## We Develop with Github
 We use github to host code, to track issues and feature requests, as well as accept pull requests.
 
-## We Use [Github Flow](https://guides.github.com/introduction/flow/index.html), So All Code Changes Happen Through Pull Requests
-Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://guides.github.com/introduction/flow/index.html)). We actively welcome your pull requests:
+## We Use [Github Flow](https://docs.github.com/en/get-started/quickstart/github-flow), So All Code Changes Happen Through Pull Requests
+Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://docs.github.com/en/get-started/quickstart/github-flow)). We actively welcome your pull requests:
 
 1. Fork the repo and create your branch from `master`.
 2. If you've added new functionality, it's necessary to add tests. For this there is a separate test project called [TcUnit-Verifier](https://github.com/tcunit/TcUnit/tree/master/TcUnit-Verifier). In that project there are several test suites defined to test the different functionality of TcUnit. Please read the [README.MD](https://github.com/tcunit/TcUnit/blob/master/TcUnit-Verifier/README.md) in that project for further instructions. **No new functionality will be accepted without any proper tests**.
@@ -38,14 +38,13 @@ We use GitHub issues to track public bugs. Report a bug by [opening a new issue]
 - Notes (possibly including why you think this might be happening, or stuff you tried that didn't work)
 
 ## Use a Consistent Coding Style
-* Make sure to edit the project with the same version of Visual Studio as the master branch. All software (TcUnit and TcUnit-Verifier) has been developed using Visual Studio 2013. Note that VS2013 shell (provided with TwinCAT 3.1.4022.x) can't open the TcUnit-Verifier_DotNet (as it's a .NET/C# project). Instead it's recommended to use [VS2013 community edition](https://visualstudio.microsoft.com/vs/older-downloads/) which can be used both for TwinCAT and .NET/C# projects
-* Make sure to edit the project with the same build of TwinCAT XAE as the master branch. Which build of TwinCAT that is currently used can be deduced from the [TcUnit.tsproj](https://github.com/tcunit/TcUnit/blob/master/TcUnit/TcUnit.tsproj)-file. The "TcVersion"-attribute will give you the version of TwinCAT that was lastly used to edit the project. The build is the third number from the left (for example 4020, 4022 or 4024)
-* Make sure that your TwinCAT development environment uses spaces instead of tabs. The default behaviour of the TwinCAT development environment is to use tabs so it needs to be changed, which can be done according to [this guide](https://alltwincat.com/2017/04/14/replace-tabs-with-whitespaces/)
+* Make sure to edit the project with the same version of Visual Studio as the master branch. All software (TcUnit and TcUnit-Verifier) has been developed using Visual Studio 2019. Note that the TwinCAT XAE Shell (provided with TwinCAT XAE installer) can't open the TcUnit-Verifier_DotNet (as it's a .NET/C# project). Instead it's recommended to use [VS2019 community edition](https://visualstudio.microsoft.com/vs/older-downloads/) which can be used both for TwinCAT and .NET/C# projects
+* Make sure to use TwinCAT XAE version 3.1.4024.44 or later
 * The prefixes of naming of function blocks/variables/etc from the [Beckhoff TwinCAT3 identifier/name conventions](https://infosys.beckhoff.com/english.php?content=../content/1033/tc3_plc_intro/18014401873267083.html&id=) are ignored as a modern integrated development environment (as Visual Studio) gives all the hints/information of data types etc
 * Make sure to set your TwinCAT development environment to use Separate LineIDs. This is available in the TwinCAT XAE under **Tools→Options→TwinCAT→PLC Environment→Write options→Separate LineIDs** (set this to TRUE, more information is available [here](https://infosys.beckhoff.com/english.php?content=../content/1033/tc3_userinterface/18014403202147467.html&id=))
 
 ## License
-By contributing, you agree that your contributions will be licensed under its MIT License.
+By contributing, you agree that your contributions will be licensed under the MIT License.
 
 ## References
 This document was adapted from briandk's excellent [contribution guidelines template](https://gist.github.com/briandk/3d2e8b3ec8daf5a27a62).

README.md

@@ -1,9 +1,11 @@
-# TcUnit - TwinCAT unit testing framework
+![TcUnit logo](https://github.com/tcunit/TcUnit/blob/master/img/tcunit-logo.png)
+[![GitHub license](https://img.shields.io/github/license/Naereen/StrapDown.js.svg)](https://github.com/tcunit/TcUnit/blob/master/LICENSE)
+[![Open Source? Yes!](https://badgen.net/badge/Open%20Source%20%3F/Yes%21/blue?icon=github)](https://github.com/TcUnit/TcUnit)
 
 
-Welcome to the documentation TcUnit - an xUnit testing framework for [Beckhoff's TwinCAT3](https://www.beckhoff.com/twincat3/).
+# TcUnit - TwinCAT unit testing framework
 
-![TwinCAT logo](https://github.com/tcunit/TcUnit/blob/master/img/TcUnit-logo.jpg)
+Welcome to the documentation TcUnit - an xUnit testing framework for [Beckhoff's TwinCAT3](https://www.beckhoff.com/twincat3/).
 
 **Main documentation site is available on:**  
 **[www.tcunit.org](https://www.tcunit.org)**
@@ -27,6 +29,7 @@ Check out the [TcUnit-Runner project](https://github.com/tcunit/TcUnit-Runner).
 That's fantastic! But please read the [CONTRIBUTING](CONTRIBUTING.md) first.
 
 **Have any questions? Found a bug or want to discuss an idea?**  
-Check the [F.A.Q](https://tcunit.org/frequently-asked-questions/). Check the [open](https://github.com/tcunit/TcUnit/issues) and [closed](https://github.com/tcunit/TcUnit/issues?q=is%3Aissue+is%3Aclosed) issues.
+Check the [F.A.Q](https://tcunit.org/frequently-asked-questions/).
+Check the [open](https://github.com/tcunit/TcUnit/issues) and [closed](https://github.com/tcunit/TcUnit/issues?q=is%3Aissue+is%3Aclosed) issues.
 If your issue does not already exist, create a [new](https://github.com/tcunit/TcUnit/issues/new).
 For general ideas/discussions, use the [discussions](https://github.com/tcunit/TcUnit/discussions).

TcUnit-Verifier/README.md

@@ -1,42 +1,31 @@
 # TcUnit-Verifier
-This project is can be used to verify the different functions of
-[TcUnit](https://www.github.com/tcunit/TcUnit). It has various test suites to
-test different functions in TcUnit. It consists of two separate programs:
+This project is can be used to verify the different functions of [TcUnit](https://www.github.com/tcunit/TcUnit).
+It has various test suites to test different functions in TcUnit.
+It consists of two separate programs:
 - TcUnit-Verifier_TwinCAT
 - TcUnit-Verifier_DotNet
 
-When verifying that TcUnit works as expected a developer needs to add tests to
-both of these programs.
+When verifying that TcUnit works as expected a developer needs to add tests to both of these programs.
 
 ![TcUnit-Verifier concept](img/TcUnit-Verifier_Concept_1280.png)
 
 ## TcUnit-Verifier_TwinCAT
 TcUnit-Verifier_TwinCAT (TcVT) is as the name implies a TwinCAT project.
-It holds a reference to and instantiates the TcUnit framework. It is defined by
-several test suites (function blocks) that make use of TcUnit and its different
-functionalitites. An example of a test is to verify that TcUnit prints a message
-if an assertion of an INT has failed (expected value not equals to
-asserted value). This means per definition that the verifier will have failed
-tests as results, because that is what we want to test. Everytime a test suite
-is created in TcVT that tests the functionality of TcUnit it is expected
-that either:
-- Some specific output is created by TcUnit (error-log), for example asserting a
-  value that differs from the expected value
-- Some specific output is **not** created by TcUnit (error-log), for example
-  asserting a value that is equal to the expected value
+It holds a reference to and instantiates the TcUnit framework.
+It is defined by several test suites (function blocks) that make use of TcUnit and its different functionalitites.
+An example of a test is to verify that TcUnit prints a message if an assertion of an INT has failed (expected value not equals to asserted value).
+This means per definition that the verifier will have failed tests as results, because that is what we want to test.
+Everytime a test suite is created in TcVT that tests the functionality of TcUnit it is expected that either:
+- Some specific output is created by TcUnit (error-log), for example asserting a value that differs from the expected value
+- Some specific output is **not** created by TcUnit (error-log), for example asserting a value that is equal to the expected value
 
-Creating a test suite for TcUnit thus requires that the developer needs to know
-what TcUnit should print (or not print). This means that running this TwinCAT
-project is not enough to know whether TcUnit behaves as expected, as it would
-require to manually check all the output from TcVT. Because we don't manually
-want to check that the output provided by TcUnit-Verifier_TwinCAT everytime it's
-running, another program is necessary that verifies that the output of the TcVT
-is as expected.
+Creating a test suite for TcUnit thus requires that the developer needs to know what TcUnit should print (or not print).
+This means that running this TwinCAT project is not enough to know whether TcUnit behaves as expected, as it would require to manually check all the output from TcVT.
+Because we don't manually want to check that the output provided by TcUnit-Verifier_TwinCAT everytime it's running, another program is necessary that verifies that the output of the TcVT is as expected.
 
 ## TcUnit-Verifier_DotNet
-The TcUnit-Verifier_DotNet (TcVD) is a (Visual Studio 2013) C# program that
-opens and runs the TcUnit-Verifier_TwinCAT project by the usage of the TwinCAT
-automation interface. It basically does the following:
+The TcUnit-Verifier_DotNet (TcVD) is a C# program that opens and runs the TcUnit-Verifier_TwinCAT project by the usage of the TwinCAT automation interface.
+It basically does the following:
 - Starts Visual Studio (using the same version that was used developing TcVT)
 - Opens TcVT
 - Does a clean/build of TcVT
@@ -47,16 +36,11 @@ automation interface. It basically does the following:
 - Collects all results from the error list in Visual Studio
 - Checks that the output is as defined/expected
 
-The different tests in TcVD follow the same naming-schema as for the TcVT tests
-so for example you can find a structured text (ST) function block (FB)
-`FB_PrimitiveTypes.TcPOU` in TcVT and the accompanying C#-class
-`FB_PrimitiveTypes.cs`. The C# classes verify and test that the output from the
-ST function blocks is correct. Thus a complete test of a specific function in
-TcUnit needs to be developed in pair, both a FB of tests needs to be added to
-TcVT as well a class of tests needs to be added to TcVD.
+The different tests in TcVD follow the same naming-schema as for the TcVT tests, so for example you can find a structured text (ST) function block (FB) `FB_PrimitiveTypes.TcPOU` in TcVT and the accompanying C#-class `FB_PrimitiveTypes.cs`.
+The C# classes verify and test that the output from the ST function blocks is correct.
+Thus a complete test of a specific function in TcUnit needs to be developed in pair, both a FB of tests needs to be added to TcVT as well a class of tests needs to be added to TcVD.
 
-All test classes are instantiated in the class `Program.cs` starting from the
-lines:
+All test classes are instantiated in the class `Program.cs` starting from the lines:
 ```
 /* Insert the test classes here */
 new FB_PrimitiveTypes(errors);
@@ -67,15 +51,10 @@ new FB_AssertEveryFailedTestTwice(errors);
 ...
 ```
 
-To create a new test class and make sure that it will be running all that is
-necessary is to make sure to instantiate it with the argument `errors`
-(just as above). If you have added a test in TcVT that is supposed to fail, and
-thus adding an additional failed test to the output, you need to increment the
-variable `expectedNumberOfFailedTests` in TcVD by one for every failed test
-that you have added. 
+To create a new test class and make sure that it will be running all that is necessary is to make sure to instantiate it with the argument `errors` (just as above).
+If you have added a test in TcVT that is supposed to fail, and thus adding an additional failed test to the output, you need to increment the variable `expectedNumberOfFailedTests` in TcVD by one for every failed test that you have added. 
 
-For example, if we in the PRG_TEST-program of TcVT have a function block
-instantiated in this way:
+For example, if we in the PRG_TEST-program of TcVT have a function block instantiated in this way:
 ```
 PROGRAM PRG_TEST
 VAR
@@ -91,31 +70,23 @@ This is an example of how it can look running the TcUnit-Verifier_DotNet:
 
 ```
 C:\Code\GitHub_TcUnit\TcUnit\TcUnit-Verifier\TcUnit-Verifier_DotNet\TcUnit-Verifier\bin\Debug>TcUnit-Verifier.exe -v "C:\Code\GitHub_TcUnit\TcUnit\TcUnit-Verifier\TcUnit-Verifier_TwinCAT\TcUnit-Verifier_TwinCAT.sln"
-2022-01-07 18:58:05 [INFO ] - Starting TcUnit-Verifier...
-2022-01-07 18:58:05 [INFO ] - In Visual Studio solution file, found visual studio version 12.0
-2022-01-07 18:58:05 [INFO ] - Loading the Visual Studio Development Tools Environment (DTE)...
-2022-01-07 18:58:11 [INFO ] - Cleaning and building TcUnit-Verifier_TwinCAT solution...
-2022-01-07 18:58:11 [INFO ] - Generating TcUnit-Verifier_TwinCAT boot project...
-2022-01-07 18:58:25 [INFO ] - Activating TcUnit-Verifier_TwinCAT configuration...
-2022-01-07 18:58:28 [INFO ] - Restarting TwinCAT...
-2022-01-07 18:58:28 [INFO ] - Waiting for TcUnit-Verifier_TwinCAT to finish running tests...
-2022-01-07 18:58:38 [INFO ] - ... got 409 report lines so far.
-2022-01-07 18:58:48 [INFO ] - ... got 934 report lines so far.
-2022-01-07 18:58:59 [INFO ] - ... got 1357 report lines so far.
-2022-01-07 18:59:01 [INFO ] - Asserting results...
-2022-01-07 18:59:01 [INFO ] - Done.
-2022-01-07 18:59:01 [INFO ] - Closing the Visual Studio Development Tools Environment (DTE), please wait...
-2022-01-07 18:59:23 [INFO ] - Exiting application...
+2023-04-29 22:15:29 [INFO ] - Starting TcUnit-Verifier...
+2023-04-29 22:15:29 [INFO ] - In Visual Studio solution file, found visual studio version 16.0
+2023-04-29 22:15:29 [INFO ] - Loading the Visual Studio Development Tools Environment (DTE)...
+2023-04-29 22:15:41 [INFO ] - Cleaning and building TcUnit-Verifier_TwinCAT solution...
+2023-04-29 22:15:41 [INFO ] - Generating TcUnit-Verifier_TwinCAT boot project...
+2023-04-29 22:15:54 [INFO ] - Activating TcUnit-Verifier_TwinCAT configuration...
+2023-04-29 22:15:57 [INFO ] - Restarting TwinCAT...
+2023-04-29 22:15:57 [INFO ] - Waiting for TcUnit-Verifier_TwinCAT to finish running tests...
+2023-04-29 22:16:07 [INFO ] - ... got 347 report lines so far.
+2023-04-29 22:16:17 [INFO ] - ... got 872 report lines so far.
+2023-04-29 22:16:28 [INFO ] - ... got 1398 report lines so far.
+2023-04-29 22:16:39 [INFO ] - ... got 1618 report lines so far.
+2023-04-29 22:16:41 [INFO ] - Asserting results...
+2023-04-29 22:16:42 [INFO ] - Done.
+2023-04-29 22:16:42 [INFO ] - Closing the Visual Studio Development Tools Environment (DTE), please wait...
+2023-04-29 22:17:02 [INFO ] - Exiting application...
 ```
-If there was an error in the TcUnit framework this would be shown between the
-lines `Asserting results...` and `Done.`. If nothing is shown between these
-lines (like in the example) it means that TcUnit behaves according to the tests
-defined in TcUnit-Verifier. As TcVD starts the TwinCAT runtime it is required 
-that a (trial) runtime license for TwinCAT is activated on the PC of where TcVD
-and TcVT will be running.
-
-To compile the project you will also need to install
-[log4net](https://logging.apache.org/log4net/).
-Open the project with VS2013. Go to
-`TOOLS->NuGet Package Manager->Package Manager Console`.
-In the console enter `Install-Package log4net -Version 2.0.14`.
+If there was an error in the TcUnit framework this would be shown between the lines `Asserting results...` and `Done.`.
+If nothing is shown between these lines (like in the example) it means that TcUnit behaves according to the tests defined in TcUnit-Verifier.
+As TcVD starts the TwinCAT runtime it is required  that a (trial) runtime license for TwinCAT is activated on the PC of where TcVD and TcVT will be running.

TcUnit-Verifier/TcUnit-Verifier_DotNet/TcUnit-Verifier.sln

@@ -1,7 +1,7 @@
 
 Microsoft Visual Studio Solution File, Format Version 12.00
-# Visual Studio 2013
-VisualStudioVersion = 12.0.40629.0
+# Visual Studio Version 16
+VisualStudioVersion = 16.0.33529.622
 MinimumVisualStudioVersion = 10.0.40219.1
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TcUnit-Verifier", "TcUnit-Verifier\TcUnit-Verifier.csproj", "{13F197A8-6EF6-4799-A221-37B97BA42100}"
 EndProject

TcUnit-Verifier/TcUnit-Verifier_DotNet/TcUnit-Verifier/App.config

@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="utf-8"?>
 <configuration>
     <startup> 
-        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.1"/>
+        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2"/>
     </startup>
 </configuration>

TcUnit-Verifier/TcUnit-Verifier_DotNet/TcUnit-Verifier/FB_ExtendedTestInformation.cs

@@ -9,6 +9,7 @@ public FB_ExtendedTestInformation(IEnumerable<ErrorList.Error> errors, string te
         {
             Check_TestSuite_Statistics();
             AssertTestClassName();
+            AssertTestDuration();
             Test_BOOL_AssertFailed();
             Test_BOOL_AssertSuccess();
             Test_BYTE_TwoFailedAsserts();
@@ -36,6 +37,11 @@ private void AssertTestClassName()
             AssertMessageCount(className, 5, EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
         }
 
+        private void AssertTestDuration()
+        {
+            string duration = "Test duration=%f";
+            AssertMessageCount(duration, 260, EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
+        }
 
         private void Test_BOOL_AssertFailed()
         {

TcUnit-Verifier/TcUnit-Verifier_DotNet/TcUnit-Verifier/FB_TestNumberOfAssertionsCalculation.cs

@@ -16,27 +16,27 @@ public FB_TestNumberOfAssertionsCalculation(IEnumerable<ErrorList.Error> errors,
 
         private void TestMixed33SuccessulAnd9FailedAssertions()
         {
-            AssertContainsMessage("| Test status=FAIL, number of asserts=42", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
+            AssertContainsMessage("| Test status=FAIL, number of asserts=42, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
         }
 
         private void TestWith43SuccessfulAssertions()
         {
-            AssertContainsMessage("| Test status=PASS, number of asserts=43", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
+            AssertContainsMessage("| Test status=PASS, number of asserts=43, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
         }
 
         private void TestWith44FailedAssertions()
         {
-            AssertContainsMessage("| Test status=FAIL, number of asserts=44", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
+            AssertContainsMessage("| Test status=FAIL, number of asserts=44, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
         }
 
         private void TestWith45SuccessfulArrayAssertions()
         {
-            AssertContainsMessage("| Test status=PASS, number of asserts=45", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
+            AssertContainsMessage("| Test status=PASS, number of asserts=45, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
         }
 
         private void TestWith46FailedArrayAssertions()
         {
-            AssertContainsMessage("| Test status=FAIL, number of asserts=46", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
+            AssertContainsMessage("| Test status=FAIL, number of asserts=46, duration=%f", EnvDTE80.vsBuildErrorLevel.vsBuildErrorLevelLow);
         }
 
     }