Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

toast-alerts popping up due to unrelated error can break e2e tests #98

Open
surchs opened this issue Mar 26, 2024 · 1 comment
Open
Labels
bug:test Defect causing our test suite to fail, although no functional error is occurring. _flag:stale [BOT ONLY] Flag issue that hasn't been updated in a while and needs to be triaged again type:bug Defects in shipped code and fixes for those defects

Comments

@surchs
Copy link
Contributor

surchs commented Mar 26, 2024

This is a bit of an odd one that points to a bigger problem with the tests:

If I run an e2e test like https://github.com/neurobagel/query-tool/blob/c890a4a5a2c221993b8240ff106b44847df4ba2c/cypress/e2e/Alert.cy.ts
and don't mock all the required remote data (as we currently do), and some of the required remote data fails (e.g. is unavailable or in the wrong form), then I will get the pretty error toasts like so:

image

If they happen to cover up an element that I expect to be able to interact with, then cypress will cleverly error out.

That's the fyi part. The "what now" part is: we should not rely on any running remote API being available to ensure our e2e tests pass. We don't yet have any real cross-app integration e2e tests. And changing the response model like we do in #60 is a good example where this reliance on a production API really comes to bite us.

So the short answer is: no more requests to prod APIs in e2e tests. Short term: mock it all. Longer term: have a dedicated test stack of all the services our app requires to talk to

@surchs surchs added bug:test Defect causing our test suite to fail, although no functional error is occurring. type:bug Defects in shipped code and fixes for those defects labels Mar 26, 2024
Copy link

We want to keep our issues up to date and active. This issue hasn't seen any activity in the last 75 days.
We have applied the _flag:stale label to indicate that this issue should be reviewed again.
When you review, please reread the spec and then apply one of these three options:

  • prioritize: apply the flag:schedule label to suggest moving this issue into the backlog now
  • close: if the issue is no longer relevant, explain why (give others a chance to reply) and then close.
  • archive: sometimes an issue has important information or ideas but we won't work on it soon. In this case
    apply the someday label to show that this won't be prioritized. The stalebot will ignore issues with this
    label in the future. Use sparingly!

@github-actions github-actions bot added the _flag:stale [BOT ONLY] Flag issue that hasn't been updated in a while and needs to be triaged again label Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug:test Defect causing our test suite to fail, although no functional error is occurring. _flag:stale [BOT ONLY] Flag issue that hasn't been updated in a while and needs to be triaged again type:bug Defects in shipped code and fixes for those defects
Projects
Status: No status
Development

No branches or pull requests

1 participant