Platformio unit test runs the same test case twice

[lightway29]

Trying to run a sample test case for mbed framework. Got everything working but running into an issue where the same test case runs twice and one fails and one passes.

https://community.platformio.org/t/platformio-unit-test-runs-the-same-test-case-twice/36404/3

platformio.ini

[env:lpc1768]
platform = nxplpc
board = lpc1768
framework = mbed
test_speed = 9600
test_port = /dev/cu.usbmodem142302

test_sample.cpp

#include <mbed.h>
#include <unity.h>


#define WAIT_TIME_MS 2000

void test_numbers(void)
{
    TEST_ASSERT_EQUAL(13,13);
}



void setUp(void) {
    // set stuff up here
   
}

void tearDown(void) {
    // clean stuff up here
}


int main()
{
    thread_sleep_for(WAIT_TIME_MS);
    UNITY_BEGIN(); // IMPORTANT LINE!
    RUN_TEST(test_numbers);
    UNITY_END();

}

Folder structure

[valeros]

Hi @lightway29, there might be a conflict between the Unity distribution bundled with the mbed package and the one used by PlatformIO. Could you please try to add a special file .mbedignore to the root of your project (next to platformio.ini) with the following content:

mbed-os/features/frameworks/unity/*

Then clean and test again.

[lightway29]

> Hi @lightway29, there might be a conflict between the Unity distribution bundled with the mbed package and the one used by PlatformIO. Could you please try to add a special file .mbedignore to the root of your project (next to platformio.ini) with the following content:

mbed-os/features/frameworks/unity/*

Then clean and test again.

Hi @valeros,

Cleaned and checked including the `mbedignore' file to the root. Still get the same result.

Have attached the sample project if it helps
pio-testframework-mbed.zip

[valeros]

That's strange, what version of the nxplpc dev-platform are you using?

[lightway29]

That's strange, what version of the nxplpc dev-platform are you using?

@valeros @ivankravets
I have also experimented on the following

• By removing both versions of nxplpc 5.3.0 and 9.0.0 then by installing 10.0.0
• Tried on a windows freshly installed windows pc as well.

FYI - I tried with Arduino and that works perfectly

[valeros]

Please delete all previous platform instances, clean the project, run your tests in verbose mode via pio test -vvv > piotest.log.txt 2>&1 and attach here that log file.

[lightway29]

Please delete all previous platform instances, clean the project, run your tests in verbose mode via pio test -vvv > piotest.log.txt 2>&1 and attach here that log file.

@valeros

Attached piotest log file below.

piotest.log.txt

[valeros]

Strange, according to the log file the Unity sources from the framework are still compiled even though they should be excluded from the build. What's more, I also compiled your project example and it seems to work as expected on Linux and Windows. What happens if you manually remove the /Users/YOUR_USER/.platformio/packages/framework-mbed/features/frameworks/unity folder?

[lightway29]

Strange, according to the log file the Unity sources from the framework are still compiled even though they should be excluded from the build. What's more, I also compiled your project example and it seems to work as expected on Linux and Windows. What happens if you manually remove the /Users/YOUR_USER/.platformio/packages/framework-mbed/features/frameworks/unity folder?

@valeros
That did the trick. Removed all the frameworks related to Mbed and ran the code it worked fine. One thing I have noticed is that the board needs to be removed and plugged back again every time I run the test. Simply pressing the reset button won't run the test.

[valeros]

One thing I have noticed is that the board needs to be removed and plugged back again every time I run the test. Simply pressing the reset button won't run the test.

Not sure how the USB communication is implemented on this board, maybe it's a virtual COM port, so you can try adding a delay before UNITY_BEGIN just to make sure the MCU have enough time to reinitialize USB.

[IngmarPaetzold]

I have a similar issue: when running several testcases, it works perfectly in native environment, but on the target (Arduino Mega 2560), it sometimes runs all, sometimes only the last one, and sometimes the latter twice (-> see this issue here).
However, I do not have the mbed package, thus, cannot remove it. Maybe this is the same bug?
See the description in platformio forum here: https://community.platformio.org/t/unit-test-runs-only-one-testcase-on-target/42778