README.md

spark_pyspark-scala

spark with python_jupyter

Semi-Project

1. Recommendation System(MovieLens)

• pre-processing(pyspark) : 전처리.pyspark
• Recommendation system(scala) : 추천시스템 내용.scala

2. Classification Model(Kaggle : Stumbleupon)

• 잡다한 분류모형.scala

1. RDD_Transformation_Action

• RDD이해 및 Transformation,Action

2. DataFrame

• DataFrame이해 및 Query

3. Data modeling

• Pre-Processing&EDA

4. MLlib

• 간단한 머신러닝 모형 적용

5. ML

• 파이프라인/하이퍼파라미터 최적화/ML/Topic_Mining

6. Streaming