Mix Task to compare chrome and ChromeDriver versions

[djgoku]

Problem

When running wallaby tests if chrome/chromium and chromedriver versions do not match major, minor and build version numbers [1] you can get errors that may not make sense. Below is an example from chrome version 99.0.4844.74 and chromedriver version 97.0.4692.71:

** (RuntimeError) invalid session id
code: |> Module.view(module.token.id)
stacktrace:
  (wallaby 0.29.1) lib/wallaby/httpclient.ex:136: Wallaby.HTTPClient.check_for_response_errors/1
  (wallaby 0.29.1) lib/wallaby/httpclient.ex:56: Wallaby.HTTPClient.make_request/5
  (wallaby 0.29.1) lib/wallaby/webdriver_client.ex:329: Wallaby.WebdriverClient.visit/2
  (wallaby 0.29.1) lib/wallaby/driver/log_checker.ex:6: Wallaby.Driver.LogChecker.check_logs!/2
  (wallaby 0.29.1) lib/wallaby/browser.ex:1235: Wallaby.Browser.visit/2

[1] https://sites.google.com/chromium.org/driver/downloads/version-selection

Possible Solution

Add a mix task that compares chrome version and chromedriver version. We have proof of concept here that works on linux, macOS x86 and macOS M1 here: main...BinaryNoggin:add-mix-task-for-checking-chromedriver

We do need to bullet proof the differences in linux, macOS and look into also supporting Windows.

@thejohncotton @401matthall @amclain @adkron

[mhanberg]

Thank you for opening a discussion on this topic!

This has been a long standing source of confusion for Wallaby users.

While a mix task is certainly a convenient way to check if your versions are in sync, I think discoverability is still a bit of a problem here. I don't think users will know to run this mix task.

I do think we can use the code in your patch to add a version check to the wallaby drivers to check this on startup.

Any version of this would also need to account for the user's configured Chrome binary and configured chromedriver binary, as the user is able to use other browsers like Vivaldi, Edge, Chromium, Brave (other chromium based browsers).

What do you think?

[thejohncotton]

@mhanberg we thought about this for a bit and thought we'd attempt a PR that builds the version compare into application start (near minimum version compatibility checks).

I'm thinking we probably don't quite have supporting other chromium based browsers quite figured out here, do you have any thoughts on how we could support those, or at least not disrupt them?

#684