-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[devel_transport] Async protocol support for utest (rebased 02-03-2016) #6
Conversation
Tested with
|
@niklas-arm I've already merged your devel_transport branch here. |
Ok, I've added new (breaking) features to utest.
I've updated all utest selftests, which was done with a simple find and replace. Since we have to modify existing unittest written in utest anyway to update the |
@@ -15,7 +15,7 @@ | |||
* limitations under the License. | |||
*/ | |||
#include "mbed-drivers/mbed.h" | |||
#include "mbed-drivers/test_env.h" | |||
#include "greentea-client/test_env.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎉 🎆 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎊 🍪 🍸 😍
@niklas-arm Cool! :) Nice new features for utest! |
Just as a clarification, the new features aren't on the master branch yet, as they are built on this PR. |
@niklas-arm I will merge it but not right away, got other things to break...But will remember about this! |
These changes will be a minor version ie. v1.10.0. Two things:
The changes from my branch won't be merged into here, since they are a breaking change. |
Also remember to close #4. |
Current utest test results: mbedgt: test suite report: +---------------+---------------+----------------------------------------------+--------+--------------------+-------------+ | target | platform_name | test suite | result | elapsed_time (sec) | copy_method | +---------------+---------------+----------------------------------------------+--------+--------------------+-------------+ | frdm-k64f-gcc | K64F | utest-test-case_async_validate | OK | 12.66 | shell | | frdm-k64f-gcc | K64F | utest-test-case_control_async | FAIL | 19.17 | shell | | frdm-k64f-gcc | K64F | utest-test-case_control_repeat | OK | 13.69 | shell | | frdm-k64f-gcc | K64F | utest-test-case_setup_failure | OK | 11.28 | shell | | frdm-k64f-gcc | K64F | utest-test-case_teardown_failure | FAIL | 11.57 | shell | | frdm-k64f-gcc | K64F | utest-test-control_type | OK | 11.9 | shell | | frdm-k64f-gcc | K64F | utest-test-readme | OK | 13.01 | shell | | frdm-k64f-gcc | K64F | utest-test-readme_lambdas | OK | 12.99 | shell | | frdm-k64f-gcc | K64F | utest-test-test_assertion_failure_test_setup | OK | 10.58 | shell | | frdm-k64f-gcc | K64F | utest-test-test_setup_failure | OK | 10.62 | shell | +---------------+---------------+----------------------------------------------+--------+--------------------+-------------+ mbedgt: test case report: +---------------+---------------+----------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+ | target | platform_name | test suite | test case | passed | failed | result | elapsed_time (sec) | +---------------+---------------+----------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+ | frdm-k64f-gcc | K64F | utest-test-case_control_repeat | Control: CaseNext | 1 | 0 | OK | 0.09 | | frdm-k64f-gcc | K64F | utest-test-case_control_repeat | Control: NoRepeat | 1 | 0 | OK | 0.09 | | frdm-k64f-gcc | K64F | utest-test-case_control_repeat | Control: RepeatAll | 10 | 0 | OK | 0.13 | | frdm-k64f-gcc | K64F | utest-test-case_control_repeat | Control: RepeatHandler | 10 | 0 | OK | 0.44 | | frdm-k64f-gcc | K64F | utest-test-control_type | Testing combinations of different group | 1 | 0 | OK | 0.13 | | frdm-k64f-gcc | K64F | utest-test-control_type | Testing combinations of same group | 1 | 0 | OK | 0.13 | | frdm-k64f-gcc | K64F | utest-test-control_type | Testing constants | 1 | 0 | OK | 0.09 | | frdm-k64f-gcc | K64F | utest-test-control_type | Testing constructors | 1 | 0 | OK | 0.1 | | frdm-k64f-gcc | K64F | utest-test-case_async_validate | Validate: Attributed Validation: Cancel Repeat | 1 | 0 | OK | 0.25 | | frdm-k64f-gcc | K64F | utest-test-case_async_validate | Validate: Attributed Validation: Enable Repeat Handler | 2 | 0 | OK | 0.26 | | frdm-k64f-gcc | K64F | utest-test-case_async_validate | Validate: Multiple Premature Validation | 1 | 0 | OK | 0.13 | | frdm-k64f-gcc | K64F | utest-test-case_async_validate | Validate: Multiple Validation | 1 | 0 | OK | 0.21 | | frdm-k64f-gcc | K64F | utest-test-case_async_validate | Validate: Premature Validation | 1 | 0 | OK | 0.12 | | frdm-k64f-gcc | K64F | utest-test-case_async_validate | Validate: Simple Validation | 1 | 0 | OK | 0.2 | | frdm-k64f-gcc | K64F | utest-test-case_setup_failure | Setup handler returns ABORT | 0 | 1 | FAIL | 0.0 | | frdm-k64f-gcc | K64F | utest-test-case_setup_failure | Setup handler returns CONTINUE | 1 | 0 | OK | 0.0 | | frdm-k64f-gcc | K64F | utest-test-case_setup_failure | Setup handler returns IGNORE | 0 | 1 | FAIL | 0.0 | | frdm-k64f-gcc | K64F | utest-test-readme_lambdas | Asynchronous Test (200ms timeout) | 1 | 0 | OK | 0.12 | | frdm-k64f-gcc | K64F | utest-test-readme_lambdas | Asynchronous Timeout Repeat | 1 | 0 | OK | 1.43 | | frdm-k64f-gcc | K64F | utest-test-readme_lambdas | Repeating Test | 2 | 0 | OK | 0.14 | | frdm-k64f-gcc | K64F | utest-test-readme_lambdas | Simple Test | 1 | 0 | OK | 0.1 | | frdm-k64f-gcc | K64F | utest-test-readme | Asynchronous Test (200ms timeout) | 1 | 0 | OK | 0.14 | | frdm-k64f-gcc | K64F | utest-test-readme | Asynchronous Timeout Repeat | 1 | 0 | OK | 1.44 | | frdm-k64f-gcc | K64F | utest-test-readme | Repeating Test | 2 | 0 | OK | 0.15 | | frdm-k64f-gcc | K64F | utest-test-readme | Simple Test | 1 | 0 | OK | 0.09 | | frdm-k64f-gcc | K64F | utest-test-case_teardown_failure | Teardown handler returns ABORT | 1 | 0 | OK | 0.12 | | frdm-k64f-gcc | K64F | utest-test-case_teardown_failure | Teardown handler returns ABORT but is IGNORED | 1 | 0 | OK | 0.15 | | frdm-k64f-gcc | K64F | utest-test-case_teardown_failure | Teardown handler returns CONTINUE | 1 | 0 | OK | 0.13 | | frdm-k64f-gcc | K64F | utest-test-case_control_async | Control: Await | 1 | 0 | OK | 1.45 | | frdm-k64f-gcc | K64F | utest-test-case_control_async | Control: CaseNext | 1 | 0 | OK | 0.09 | | frdm-k64f-gcc | K64F | utest-test-case_control_async | Control: NoTimeout | 1 | 0 | OK | 0.1 | | frdm-k64f-gcc | K64F | utest-test-case_control_async | Control: RepeatAllOnTimeout | 1 | 0 | OK | 0.15 | | frdm-k64f-gcc | K64F | utest-test-case_control_async | Control: RepeatHandlerOnTimeout | 1 | 0 | OK | 1.82 | | frdm-k64f-gcc | K64F | utest-test-case_control_async | Control: Timeout (Failure) | 0 | 1 | FAIL | 0.27 | | frdm-k64f-gcc | K64F | utest-test-case_control_async | Control: Timeout (Success) | 1 | 0 | OK | 0.11 | +---------------+---------------+----------------------------------+--------------------------------------------------------+--------+--------+--------+--------------------+
From @niklas-arm: * Report success to greentea for expected failures. * Use the greentea setup handlers for correct reporting. * Remove GREENTEA_START() from all tests. * Reorder greentea_send_kv to be more readable.
"C:/jslv3_1/ws/ybt@2/test/readme_lambdas/main.cpp", line 77: Error: #2958: the body of a value-returning lambda with no explicit return type must be a single return statement Specification specification([](const size_t number_of_cases) {
@niklas-arm I rebased this PR already, I need to remove commits and update change-log. |
@niklas-arm Niklas, please release this as |
@niklas-arm I'm ready to release with yotta registry |
Merged manually, since I needed to correct the changelog format ;-) Released as v1.10.0. |
Please do not merge !, work in progress.
Note: I've extracted
mbed-drivers/test_env.h
APIs outside of mbed-drivers to greentea-client!Description
This change ports to Greentea with Async feature inside. It is a breaking change.
Changes
greentea_handlers
to separate file.__testcase_count
__testcase_start
__testcase_finish
__testcase_summary
test/
to support Greentea test case reporting.