StandAlone unit-Testing framework for software written in C
STAT is designed to promote an instant assimilation of Test-Driven Development in conditions of large-scale codebases that are constrained with heavy load of legacy code. The framework was conceived with special regard to limitations of embedded environments and to implementation-concepts widely accepted in embedded programming. It is based on a really great open-source project named Unity (which can be found also on GitHub).
- 1. Background
- 2. Installation
- 3. Usage
- 4. Integration
- 5. Maintainer
- 6. Contribution
- 9. License
- 8. Definitions
We tried several unit-test frameworks, and there are quite a few really great products. So, we stated the following principles as our requirements for the framework we were looking for:
- Simplicity – test-setup shall be simple, fast and intuitive
- Speed – execution shall be fast and focused to support TDD short cycles
- Lightweight – portability to embedded platforms shall be simple (planned for the future)
- Comprehensive feedback – better logging ⇒ lesser step-by-step debugging ⇒ better efficiency
- Reproducibility– tests shall be reproducible and deterministic
- Test-code sharing – reduce the inevitable code-duplication (e.g. test-doubles)
- Automation – test automation shall be very simple to achieve
- CUT-Isolation – decouple from noise-impact of other FW-components, HW or OS
The last one we found most critical for our needs due to constraints described in Conceptual Model. The CUT-Isolation principle was the one that we couldn't achieve with existing alternative frameworks.
In addition, we had certain limitations that hardened integration of other solutions in our environment. Our lab machines were limitted to run only Windows OS, Python 2.7, MS Visual Studio and our target build tool-chain. No lua, Ruby, CMake or any other things of that kind.
Eventually, it was decided to build our own framework that will meet all these requirements. But we didn't do everythign from scratch. We derived our solution from Unity Harness.
Please refer the Getting Started user-guide. It describes how to install and to setup the framework, and how to compose unit-tests over it.
- Command-line - full description of all possible command-line arguments to control behavior of STAT
- Unity Assertions Reference - the original 'Unity' user-guide with usage-description of assertion macros
- STAT-Mock Library - user-guide describing how to use built-in Mock library of STAT framework
- Conceptual Model - conceptual model underlying the design of the framework, a valuable knowledge to become Pro with STAT
- Unity Assertions Cheat Sheet - a cook-book with macros worth printing and framing
- Unity Configuration Guide - the default configuration of Unity defined within STAT can be overwritten
STAT-framework is built to run both on Windows and Linux.
STAT-framework has an integrated support for build-tools, which are determined based on OS, localy instlled tools and user-configuration:
-
On Windows: Microsoft Visual Studio including all versions from 2005 up to 2019 (any edition, i.e. Community, Professional and Enterprise).
-
On Linux: GNU Compiler, i.e. GCC
STAT is designed to minimize its requirements from the system. Therefore, it uses a precompiled GNU Make Tool (included in the repo).
This decision was dictated by restrictions of our lab, which didn't allow installing any additional software.
MSVS Versioning
If not explicitly specified in file ".statconfig", the latest MSVS version on the machine gets identified and gets set as active build-tools. Otherwise, the specified version is selected instead. The latter is useful to fixate on common single version for all developer and lab machines.
Framework provides services to generate project files for certain IDEs, by the means of command-line. The following IDEs are currently supported:
- Microsoft Visual Studio of version corresponding to the selected build-tools
- The generated project file for this IDE supports step-by-step debugging
- Source Insight 4.0 - it is a unique IDE, which also is a code-analyzer, actually one of a kind
- Visual-Studio Code - the generated workspace for this framework supports step-by-step debugging on Linux
Arseniy Aharonov - the maintainer and the main contributor
An open source project thrives on contribution of different people, and it may come in many forms. Some develope the very code, others provide constructive feedback, and some share wisdom through guidance and consultation. And all of these are very valuable contributions made by very valuable contributors:
- Eitan Talianker
- Dr. Eitan Farchi
- Udi Arie
- Oran DeBotton
- Luna Benarroch
- Leah Adler
STAT wasn't build from scratch. Several valuable components underlay the framework:
- Unity harness source-files - a key component for the framework
- Visual Studio Locator (a.k.a.
vswhere) - a utility to locate Visual Studio in newer versions of MSVS. - GNU Make tool - controls generation of executables from sources based on makefiles
- License: GPLv3+ license
- Source: on GNU.org
Though we couldn't find something that will answer all our requirements, we found the one that was very close.
Unity harness is that great "almost"-match with the following clear advantages:
- Minimalistic in size and dependencies
- Can be compiled almost on any platform
- Provides rich and strong assertion mechanism
- Prints very comprehensive log and results
- Tolerant to test-failures
- Failing test (if properly built) doesn't crash the subsequent tests
An additional advantage worthy of a separate mentioning is the very fact that Unity is written in C, which is the same language we use for our production code.
Writing in one language gives the same sense of developing, no matter whether it is the production code or unit-tests, without constant switching between different language paradigms.
Feel free to impact! Open an issue.
The base-code of the STAT framework is licensed under MIT license. In addition, this repo contains a binary of GNU-Make tool, which is distributed under GPLv3+ license.
These definitions are provided to clarify terms used above.
- STAT – stands for standalone testing framework
- CUT – means Code Under Test; usually it is a module or a sub-module that is tested
- DOC – means Depended-on Component; in the literature this term is used to describe external dependencies that our CUT depends on
- CUT-Isolation – is a method of decoupling the CUT from the other parts of FW-Code for unit-testing purposes, meaning how we deal with DOCs in our unit-tests
- FW - Firmware
- Test-Double – is a substitution of an operational version of DOC with a test-version that emulates a real interface, but serves testing goals
- Test-package – in terms of STAT-framework, it represents a group of unit-tests that run as a single executable; it usually covers a specific CUT
- TDD - Test-Driven Development, see also TDD for more details