Skip to content
This repository has been archived by the owner on Jun 28, 2024. It is now read-only.

Issue#133 - Can't run Parameterized tests #184

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

rtorsten
Copy link

#133

Workaround to be able run parameterized tests.

Root cause:
Need to identify count of parameters and generate test name (with support custom name) before run them. It is not available from APK (we can't use reflection in APK to invoke method like junit does for regular unit tests).

Solution:
Pass whole class to android runner without method name and it will run all parameterized tests there.
Parser will correctly got tests result and display them in report. It also will correctly rerun specified parameterized tests if rerun enabled.

For proper work also exclude such tests from fatal crashed tests logic because such test never run with such name (by class name only). It will be executed with custom name based on index or custom name provided in Parameters annotation.

Also, in test suite loader we put such tests at begin of queue because they can consume a lot of time due to count of tests in class. So running them at first give us benefits in case of use multiple devices.

Also, adding support for Allure parameterized tests - same logic just different annotation for discover test classes.

Side-effect: Overall progress printer will display incorrect percent of finished tests because at start we have 1 parameterized test but can't detect how much exactly parameterized tests inside.

Limitation: Can't filter parameterized tests by methods because we invoke whole class and all tests inside, not methods.

Workaround to be able run parameterized tests.

Root cause:
Need to identify count of parameters and generate test name (with support custom name) before run them. It is not available from APK (we can't use reflection in APK to invoke method like junit does for regular unit tests).

Solution:
Pass whole class to android runner without method name and it will run all parameterized tests there.
Parser will correctly got tests result and display them in report. It also will correctly rerun specified parameterized tests if rerun enabled.

For proper work also exclude such tests from fatal crashed tests logic because such test never run with such name (by class name only). It will be executed with custom name based on index or custom name provided in Parameters annotation.

Also, in test suite loader we put such tests at begin of queue because they can consume a lot of time due to count of tests in class. So running them at first give us benefits in case of use multiple devices.

Also, adding support for Allure parameterized tests - same logic just different annotation for discover test classes.

Side-effect: Overall progress printer will display incorrect percent of finished tests because at start we have 1 parameterized test but can't detect how much exactly parameterized tests inside.

Limitation: Can't filter parameterized tests by methods because we invoke whole class and all tests inside, not methods.
@kronar
Copy link

kronar commented Jan 19, 2021

@davidalbers @QAutomatron any chances for this PR to be merged to mainline ?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants