Jest & Jasmine Augmentations, Limitations & Future

[dprevost-LMI]

As discovered while working on this PR, here is the issue to document the limitations around augmentation usage, mostly of Jasmine, and verify whether the future of it is to continue supporting them or drop them.

Jest:

1. As documented here and demonstrated in this project, we can utilize Jest with @jest/globals and @types/jest to use Jest's expect with wdio augmentation, but it is not a plug-and-play, out-of-the-box solution.
2. As shown here, to use Jest’s augmentation, we also need to register the wdio custom matcher manually.
3. Additionally, to utilize the soft assertion service, we need to register it, as shown here.
4. The typing @types/jest is not offered and supported by Jest’s maintainer, and there is a long-standing issue around that here to get rid of @types/jest
5. When using Jest’s augmentation of the wdio matchers toMatchSnapshot and toMatchInlineSnapshot, they conflict with one of Jest’s.
6. A workaround to avoid requiring the registration of the wdio matcher & soft assertion is to force the global.expect (as shown here) to be one of the wdio options, but this could also break other parts of a monorepo project.

Jasmine

1. As shown here, it is possible to use expectAsync to exploit Jasmine’s asynchronous matcher augmentation, even though supporting expectAsync explicitly might not have been a desired outcome.
2. In the above, the Jasmine default asymmetric matcher, like jasmine.stringContaining, is not working at runtime even though it “compiles”. We need to explicitly import wdioExpect and use it like wdioExpect.stringContaining
3. The Jasmine augmentation is missing the soft assertion service function and therefore not usable, and even though it would need registration, like in the case of Jest’s
4. When using wdio-jasmine-framework, we force the expect to be expectAsync (here). Although it has yet to be tested, I wonder if doing so somehow impairs the usage of Jasmine.expect.
   • Below, some testing was done to identify inconsistencies with Jasmine augmentation and expect vs expectAsync
5. Using wdio-jasmine-framework simplifies most of the above, but some details still elude my understanding since I have never used Jasmine.
6. Using soft assertions is type working, but generates the error TypeError: expect.soft is not a function at runtime
7. Using expect(Promise.resolve(true)).resolves.toEqual(true) is type working but generates TypeError: Cannot read properties of undefined (reading 'toEqual')

Conclusion

After enumerating the limitations above, it becomes clear that the complexity of using and maintaining augmentations is evident. Therefore, should we continue supporting them officially?

Possible Options

1. Simplest: We do not support augmentation and only provide support to explicit usage with import { expect } from 'expect-webdriverio’
   1. Global ambient support could be provided optionally, IF it does not interfere with existing Jest or Jasmine.
2. Continue to support Augmentations, but provide better information about them.
   1. Document if we support just a custom matcher or more like soft assertions
   2. Document better the type of project/configuration we support or do not support!
   3. In the case of Jest, provide a similar project as wdio-jasmine-framework?
   4. For Jasmine, enter a specific bug and attempt to resolve the limitation.

Note: Some details may be inaccurate, or some stated limitations may work correctly with a different configuration that I’m not aware of. Please point it out, and we will document it here at first. Additionally, this ticket could be split into two tickets, one for Jest and another for Jasmine, if required.

Update - Jasmine typing inconsistency

Project Config 1 - Wdio as Global

• expect is forced to expectAsync
• typing shows expect = ExpectWebDriverIO.Expect
• expectAsync shows as a typing error (obviously)

jasmine.Asymmetric matcher inconsistency

//Asymmetric matchers inconsistency
// Works and valid typing
expect('test').toEqual(jasmine.stringContaining('test'))
// Works and valid typing
expect('test').toEqual(expect.stringContaining('test'))

// Works and valid typing
await expect(browser).toHaveUrl(expect.stringContaining('webdriver'))
// Does not works but valid typing
await expect(browser).toHaveUrl(jasmine.stringContaining('webdriver'))

Forced expect as expectAsync impact

The following shows toEqual, which is usually synchronous but now async under Jamsine, and the typing is not clear since it returns a typing of void | Promise<void>

// “Works” and “valid” typing, but only with Jasmine, this needs to be awaited because of `expect` being `expectAsync`
const test = expect(true).toEqual(true)
console.log('test', test) // => test Promise { <pending> }

// expectAsync forced as expect inconsistency
// Works, but not a valid typing since the expect is expectAsync
await expect(Promise.resolve(false)).toBeResolvedTo(true)

Project Config 2 - Jasmine as Global

• Jasmine as Global ambiant (expect = jasmine)
• expect-webdriverio/jasmine augmentation in types

jasmine.Asymmetric matcher inconsistency

// Works and valid typing
await expectAsync(browser).toHaveUrl(wdioExpect.stringContaining('webdriver'))
// Does not work and is not valid typing
await expectAsync(browser).toHaveTitle(jasmine.stringContaining('WebdriverIO'))
// Does work and is valid typing too
await expectAsync(Promise.resolve('test1')).toBeResolvedTo(jasmine.stringContaining('test1'))