Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The current version of the migrator is not compatible with the Spark version on EMR #211

Open
julienrf opened this issue Aug 30, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@julienrf
Copy link
Collaborator

Currently, the migrator uses Scala 2.13 whereas EMR provides out-of-the-box support for Spark on Scala 2.12. This makes it impossible to use the Migrator on EMR using the built-in Spark images.

There two possible solutions:

  1. Switch to Scala 2.12 in the migrator. This should be simple to do (maybe we have to add a dependency on scala-collection-compat).
  2. Document how to build custom EMR images, or provide our own EMR images correctly pre-configured. This option needs to be investigated.
@julienrf julienrf added the enhancement New feature or request label Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant