long-li-neville-scio-scala-on-google-cloud.md

#Scio - Scala on Google Cloud

Neville Li (sinisa_lyh)

http://www.lyh.me/

Spotify

Abstract

We'll talk about Scio, a Scala API for Google Cloud Dataflow and Apache Beam, and how Spotify uses Scala for data processing and machine learning on Google Cloud Platform. We'll cover topics including Algebird, macros, and shapeless.

Notes

Related links: https://github.com/spotify/scio https://github.com/spotify/ratatool https://github.com/nevillelyh/shapeless-datatype

Bio

Neville is a software engineer at Spotify who works mainly on data infrastructure and tools for machine learning and advanced analytics. In the past few years he has been driving the adoption of Scala and new data tools for music recommendation, including Scalding, Spark, Storm and Parquet. Before that he worked on search quality at Yahoo! and old school distributed systems like MPI.