-
Just open the folder with VS Code and reopen it in a container. ! IMPORTANT: it can take a while, it is installing some libraries and dependencies.
-
in the terminal just run:
sbt run -
if you prefer, you can package the code and then run it with spark-submit.
-
Register udf_pandas provoke some error in teh DAG but the code works well.
-
Notifications
You must be signed in to change notification settings - Fork 0
macwinux/pyspark-in-scala
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
How to comunicate pyspark with spark scala through sql
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published