Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail in before hook are considere as another test case #200

Closed
jeremie-val opened this issue Mar 29, 2023 · 12 comments
Closed

Fail in before hook are considere as another test case #200

jeremie-val opened this issue Mar 29, 2023 · 12 comments
Labels
bug Something isn't working

Comments

@jeremie-val
Copy link

Bug :
When executing a tests, if the test succeed, it create a test case on allure side.
If the test fail in the before hook, it consider it as another test case

To Reproduce

  1. Create a test case with a before hook.
  2. Execute the same test case twice : the first time it succeed. the second time it failed in the before hook

Expected behavior
The test case should be the same whenever it failed (in the before / before all / after / after all hook)

Environment (please complete the following information):

  • Cypress version: "^10.9.0"
  • cypress-allure-plugin version: "^2.37.0"

Capture d’écran du 2023-03-28 12-00-57
Capture d’écran du 2023-03-28 12-00-52

@jeremie-val jeremie-val added the bug Something isn't working label Mar 29, 2023
@Shelex Shelex added the help wanted Extra attention is needed label Mar 29, 2023
@Shelex
Copy link
Owner

Shelex commented Mar 29, 2023

Thank you for opening an issue.

That ui is quite different from allure report, so I have no idea how to address it or verify.
Moreover "another test case" is identified by allure with historyId, which is a hash of the test name, so if test name was not changed - it is exactly the same execution result in allure, and will be linked to previous results if history is saved.

@jeremie-val
Copy link
Author

The interface you see it the one from Allure TestOps :)
nearly the same as allure report.

And we do not change the test name, the only difference in the 2 tests are the one which fail in the "before" hook, and the other one which succeed.

For example, the test which fail in the step is consider as the same as the succeed one.

Maybe it could come from the "full name" which are different (and visibile on the screenshot).
We have :
"before all" hook for "should update an existing client" and should update an existing client

@jeremie-val
Copy link
Author

jeremie-val commented Mar 29, 2023

Here are the 2 allure results generated from our tests :

@Shelex Shelex removed the help wanted Extra attention is needed label Mar 29, 2023
@Sheth90
Copy link

Sheth90 commented Apr 3, 2023

Hello team,
I have configured the allure report successfully with my setup. I used version conf as below.
Cypress: 12.5.1
After executing my test case. I am getting results for 2nd method, not for the first method.
TestBody of the first method is overridden by the second method's tc. Please help to resolve this issue with the allure report.

Screenshot_6
Screenshot_7
Screenshot_8
image

@Shelex
Copy link
Owner

Shelex commented Apr 3, 2023

@Sheth90 your issue is not related to this one, as I do not see any failures of before hook, so I converted it to separate issue.
Further communication will be conducted in #201

@Shelex
Copy link
Owner

Shelex commented Apr 5, 2023

Hi @jeremie-val
Issue is resolved in v2.38.3

Now plugin takes original test name for calculating history id hash.

@jeremie-val
Copy link
Author

Thank you very much, I'll have a look.

@Sheth90
Copy link

Sheth90 commented Apr 6, 2023

@jeremie-val @Shelex how to install or update allure report version to v2.38.3?

@jeremie-val
Copy link
Author

Just have to change the version in the package.json file.
That's what you did here : #201 (comment)

Just let @Shelex have a look :)

@Sheth90
Copy link

Sheth90 commented Apr 6, 2023

But still same problem persists. What to do for it? @jeremie-val As like #201

@Shelex
Copy link
Owner

Shelex commented Apr 6, 2023

@Sheth90 you have a completely separate issue so I left a comment in #201

@jeremie-val
Copy link
Author

Validated today, a fail in a before hook is now integrated to the same test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants