Can we user ReportPortal?

[VPazzini]

Is it possible to use/send jgiven reports to ReportPortal? Is there a plugin or are there plans to develop one?

[janschaefer]

Not yet. I was not even aware of ReportPortal. However, as JGiven has no own test runner, the JUnit or TestNG integration can be used, I think. What else would you expect on top of that?

[VPazzini]

Yes, it works with testng, but when using with JGiven the report sent is really minimal, it is basically a pass/fail. I was hoping for something more like the plugin for cucumber where the Gherkin stories appear on report portal.

[llipavsky]

I just tried to run it with junit5 - (with reportportal junit5 integration). Unfortunatelly it does not work at all. Well, it does, but all tests are reported as passed. Since junit5 reportportal agent uses junit5 extensions api, it seems that some exceptions are catched by JGiven and not propagated correctly (when using reportportal junit5 agent with non-jgiven tests, it works well)

[llipavsky]

I have added fully working code that demostrates the problem, if someone can look at it/debug it/whatever (my java skills are not that good to do it myself). It uses public reportportal demo server, so no need to local installation

https://github.com/llipavsky/reportportal-jgiven-example-tests

[marektester]

Hi, any update on this? Looks like problem is still valid. I'm using:

com.tngtech.jgiven jgiven-junit5 1.0.0
com.epam.reportportal agent-java-junit5 5.0.5

[l-1squared]

Hi @marektester ,

no, we have not yet looked into the issue. I am currently resolving issue #102 :) I might have a look into this one after that.

[mwesox]

Any news? :)

[l-1squared]

@mwesox Hi,
unfortunately I totally forgot about the issue over summer. I asked @fudler to help out and he'll take a look at the issue tomorrow morning.

[l-1squared]

@mwesox Hi, I managed to carry out a small investigation this morning. The root issue is that JGiven captures all test errors to make sure its machinery runs correctly at all times. For JUnit5 the errors are rethrown in the AfterEach method of the JGivenExtension. ReportPortal however attempts to report the state of the test in the afterTestExecution which comes before AfterEach in the JUnit5 Lifecycle.

We are currently experimenting whether it would be feasible to also rethrow the error in the afterTestExecution section. This would at least allow a user to use both extensions together by making sure that the JGiven callbacks run before the ReportPortal callbacks.
Unfortunately I see a great potential that such a change breaks several test cases both in our an in our users' code. Therefore, even if this turns out to appear a feasible route, it would still require extensive testing before release.

A solution that is also easy to integrate for users appears to require its own ReportPortal integration as of now. I cannot make any promises on that.

[l-1squared]

good news: switching the afterEach method to afterTestExecution doesn't fall apart immediately.
Bad news: I don't think we run enough tests to be certain that nothing breaks

[l-1squared]

With PR #827 we changed the time when JGiven releases the collected errors back to JUnit5. Therefore, with the next update, it should be possible to integrate ReportPortal and JGiven like so: ReportPortal JGiven POC.

I will attempt to make this a proper example project. However, having the example project be self-contained and tested with ReportPortal might prove quite the challenge.