Unit testing with Jest 🃏

[D34THWINGS]

Would it be a good idea to implement unit testing with jest ? As it becomes more and more of a standard way of testing React apps. I will open a PR if you judges that it's worth it.

[xaviervia]

This sounds interesting, and for sure a goal of Sagui is to keep up with the practices in the React community. To be frank I haven’t used Jest in more than a year and a half, when Triceratops were still a concern for farmers, so I took a look and it seems pretty nice, but there are some mentions of it not being compatible with Karma. This would make sense being that it uses JSDom and that should conflict with running in a browser.

Sagui provides Karma support because it is a fairly common use case to run the tests in a Selenium grid—I personally rarely do that though, since running unit tests in a grid is not essential. What do you think about this?

[D34THWINGS]

My latest experience in React application testing is:

• Unit testing with Jest using no browser because the purpose is to test functionality only
• Feature testing using CucumberJS in a Selenium grid to ensure each target platform runs the UI correctly.

I thought we could turn this into a preset to allow people to change between Karma/Jasmine and Jest as they want, leaving Karma as the default to prevent breaking changes. And in further major release, enable jest as the default preset for testing to enforce good practices.

[xaviervia]

That sounds great to me.

[pirelenito]

Awesome suggestion!

However, before doing any changes, I think it is important to highlight some points.

I don't think we should support both solutions (Jest and Karma), it should be either one or the other. Although we could support both during a transition period, I would only start implement Jest if we agree it is the way going forward.

Regarding Jest vs Karma, I'm on the same boat as @xaviervia, I haven't used it in awhile, but I'm still a little concerned that I cannot run the tests in a real browser.

Regarding implementation in Sagui, it is also important to notice that in this transition period, both frameworks would need to be a dependency of Sagui, making it a little bit more bloated.

I'm currently on vacation, and I'll be back on proper codding in about three weeks time. By then I would like to do some trials on Jest before we could make any decisions.

What do you think? Does it make sense? It would be a "big change", so I guess we should do it very consciously.

Thanks again @D34THWINGS for opening this issue!

[xaviervia]

@pirelenito actually I am personally not on board with Karma. I believe that the speed and the overall approach is not ideal—and I think that if the plan for a test is to to integration testing (ie. running it in many runtimes) it should be done with the integration tests, not with unit tests. I don’t have a clear idea about Jest, but I would be totally on board with getting rid of Karma and switching to a plain runner. Don’t want to rant to much about it, but not maintaining PhantomJS support would be a blessing for me.

I agree this is a major change, like major major. But I think @D34THWINGS’s proposal takes that into consideration with the side-by-side until we decide later on. And while we should think it through before doing a switch like this, it’s true that the community is moving to Jest and we should probably have support for it.

[D34THWINGS]

I ask my colleague about their conception about Unit testing and I found out that it's only responsibility is to test that your code outputs the right thing. The browser compatibility and proper working should be done with integration testing.

Maybe we should provide a task for integration testing. Many people find it difficult to set up such tests but you just had to write step files and gherkin files it would be much better. But again it would bring some dependencies with.

Karma is actually bringing a lot of dependencies where Jest only brings itself. Enzyme for exemple should be the choice of the user not ours. So we could trade Unit test dependencies for integration testing dependencies.

I agree with @pirelenito on the fact that it would bring a lot of bloat to Sagui and it could be better to not have this cohabitation. Although the breaking change could be massive. If you use mocha or jasmine it could be pretty simple to convert your tests as the API is exactly the same but otherwise it could be painful. But as @xaviervia told, Karma is actually so painful to configure and very slow to run due to the usage of Phantom (which is a monstrosity). I used to work with Karma for an Angular stack and it was a shame. Some tests were failing in Phantom but not in Chrome and it was ridiculous. We were forced to adapt our code to Phantom.

I was starting a new project with Sagui (my 10th actually) and I found out that the Karma stack is just so painful compared to Jest in my latest project. I made a little trip on the internet and no-one is talking about Karma for React applications, it's like Browserify for React applications, it doesn't exist no more. So my intention opening this issue is to keep this project that I love working with to be up to date with actual React development process.

BTW enjoy your vacations @pirelenito 😛

[pirelenito]

A big point for Jest is that more and more we use tools such as Babel, CoreJS and React that abstract away the quirkiness of different browsers and make testing on the real thing less and less of a real concern.

So I went back on my previous decision and as @xaviervia said, I think it could probably be fine to have a PR and try this out in an RC.

[pirelenito]

@D34THWINGS, I'm very happy to hear you are using Sagui!

Thank you so much for helping us out on pushing this project forward!

Totally agree on your latest points, we could probably have a proper integration test solution with something like gherkin latter on.

[D34THWINGS]

You're welcome ! And thank you both for your work on this project !

I'll see if I can spend some few hours on this weekend. I would be glad to put a brick on this great building !

[xaviervia]

I used to work with Karma for an Angular stack and it was a shame. Some tests were failing in Phantom but not in Chrome and it was ridiculous. We were forced to adapt our code to Phantom.

It makes me so happy to read this :D and know that I'm not alone!

[Nevon]

I'm not a Sagui user, and I'm only an extremely occasional contributor, so take my opinion for what's it's worth.

I think that you should be mindful of the fatigue of your users. If you have 2000 unit tests, you are not going to be switching from Karma+Mocha to Jest. Instead, you are going to say: "Oh well, I guess I'm staying on Sagui 8 forever then".

Even if you were willing to invest the resources to rewrite all those tests, it's very possible that some of them can't be converted to Jest, because they're more integration tests than unit tests. So until there's a path available for integration testing, you will either lose out on that coverage completely, or have to build it outside of Sagui.

While Jest is doing quite well, it's by no means the only game in town. Mocha and Karma are still very widely used. I would caution a bit of patience and see if Jest actually becomes the defacto standard before switching over. It may very well be that the concept of decoupling your unit tests from browsers and running against jsdom becomes the standard, but Jest might not necessarily be the solution for it.

On the other hand, if you are going to make that switch, it's probably better to do it now rather than when you have thousands of users ;)

[joaomilho]

I seem to agree w/ @Nevon as well. I would prefer Sagui to be a little less omakase on this one. Using Karma has lots of advantages for other kinds of tests, like integration.

[joaomilho]

Well, if it's 2017 and you haven't watched https://www.youtube.com/watch?v=E99FnoYqoII

[xaviervia]

I think we can do this transition in a non-fatigue manner. Say that we introduce Jest support in v9 or v10 as a flag that allows to switch between the two for the test runner: for many codebases, either Karma + Jasmine or Jest will do just fine, since except for the expectation library they have pretty much the same API (and Jest has support for the browser APIs).

Something like:

// sagui.config.js
module.exports = {
  javaScript: {
    test: 'jest'
  }
}

Longer term, if we are happy with this change, we can change to have Jest as the default instead of Karma. By that time it will probably be fine to tell consumers to just add the test: 'karma' flag in their sagui.config.js file if they want to keep using it.

[Nevon]

That is indeed the way it should be introduced, if it is. The only downside is that it introduces an additional maintenance burden. Also, you know... prettier/prettier#40

[xaviervia]

Well, that principle guides Sagui as well: to a practical extent, it doesn’t have configuration (but we have to be pragmatic, otherwise it would become just a boilerplate tool).
The thing here is that we are trying to roll out a replacement of a feature. That’s why I think configuration here is useful.

[blackbing]

add jest support ++

[deepsweet]

Hey, I have some upvotes

I found out that the Karma stack is just so painful compared to Jest in my latest project

Absolutely same situation. Right now we are just re-testing React and polyfills implementations over and over across different browsers.

[kaumac]

Is this being implemented or at least considered @pirelenito?

I'm considering migrating a project from CRA to Sagui, specially because of the scape hatch that CRA doesn't offer out of the box. But the lack of Jest support is a big "no-no" for me.

[xaviervia]

@kaumac this is most likely coming, but it is quite a breaking change so some other updates were prioritized before.

[Kadrei]

any updates on this?

[pirelenito]

Sorry for the long silence.

For most of my use-cases Karma has proven as a good enough solution and I still believe it is the most flexible and "future proof" solution. However, I do understand that Jest has a much better UX and it scales much better on bigger projects, so I'm OK on moving Sagui to support Jest over Karma.

To make the process smooth, first we need to:

• Validate how the integration will be done;
• Confirm that current projects using Sagui will be able to migrate to Jest without compromises;
• Have a proper migration plan.

The plan is to get a beta release rolling in the next couple of weeks so that projects can try it out.

If everything goes well, we will have an intermediary release with both Jest and Karma enabled, until on a latter release Karma is faded out.

I've spoken with @deepsweet since he has tried out Jest + Sagui in the past. We've agreed that we will be working together on this.

Keep posted for new updates and thanks for using Sagui ❤️