Movie Recommender System by Duc Nguyen.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Movie Recommender System by Duc Nguyen**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *Introduction*\n",
    "\n",
    "Recommender systems are among one of the most popular applications of data science today. They are used to predict the preference of a user or what rating they would give to a particular item based on their data. We can safely assume that, almost every major company are using some sort of recommender systems. For example, Netflix could suggest recommended shows to watch, YouTube does the same but also suggests what video to play next for the autoplay, Amazon recommends products to customers based on their liking, Facebook showing you possible friends to add and so on.\n",
    "\n",
    "Nowadays, these recommender systems are used to personalize your experience on the Internet, suggesting you what to watch, what to buy or what to eat. Even though people’s preferences or taste differ from person to person, it still follows some sort of pattern. People usually like things that are pretty like other things that they like but also people tend to have similar preferences or taste as the people that they are close with. So, recommender systems will try to find and recognize these patterns to help predict and suggest to you that you might want to buy, watch and etc. For companies like Netflix, Amazon, YouTube, Facebook, some of their successes revolves around the capability of their recommender systems. Netflix even held an open competition called Netflix Prize in 2009 for the best algorithm to predict user ratings for films and the winner won $1,000,000.\n",
    "\n",
    "There are a lot of popular recommender systems for specific topics like restaurants, online dating, financial services but also it has been developed to explore research articles and experts. It is estimated that 35% of the consumer purchases for Amazon came from recommendations. The use of recommendation systems is all over us. The web experience that is individually personalized, suggesting us what to buy, listen, watch, or eat. This is a very powerful tool that allows you to build visibility of the products to recommend those relevant products or services to people. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *Recommender Systems*\n",
    "\n",
    "We are going to be focusing on the recommender system that is made for movies/tv shows. Basically, we are building a system where we predict or suggest certain movies/tv shows based on the user’s data like their preferences and ratings. Five different recommender system types will be used: \n",
    "\n",
    "1. Simple Recommender\n",
    "2. Correlation Recommender\n",
    "3. Content-Based Recommender\n",
    "4. Collaborative Filtering\n",
    "5. Hybrid Recommender"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *Datasets*\n",
    "\n",
    "The datasets that we are going to use are the following 3 datasets:\n",
    "1)\tNetflix Prize dataset\n",
    "-\tThe dataset that was used during the 2009 Netflix Prize competition\n",
    "-\t17770 movies/tv shows\n",
    "-\t480189 users\n",
    "-\t100 million ratings \n",
    "-\tDate\n",
    "-\tSource : https://www.kaggle.com/netflix-inc/netflix-prize-data\n",
    "\n",
    "2)\tMovieLens dataset\n",
    "-\t100,000 ratings\n",
    "-\t9,000 movies\n",
    "-\t600 users\n",
    "-\tSource: https://grouplens.org/datasets/movielens/\n",
    "\n",
    "3)\tTMDB dataset\n",
    "-\t4802 movies\n",
    "-\tKeywords\n",
    "-\tPopularity\n",
    "-\tSource: https://www.kaggle.com/tmdb/tmdb-movie-metadata/ or https://raw.githubusercontent.com/codeheroku/Introduction-to-Machine-Learning/master/Building%20a%20Movie%20Recommendation%20Engine/movie_dataset.csv\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *Analysis*\n",
    "\n",
    "Firstly, we should analyze our datasets for things like ranking distribution, user ratings or most popular movies/tv shows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import os\n",
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "from sklearn.model_selection import\ttrain_test_split\n",
    "from sklearn.decomposition import TruncatedSVD\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "\n",
    "### Dataset 1\n",
    "\n",
    "# netflix data to dataframes with their corresponding columns\n",
    "netflix_data = pd.read_csv(\"netflix_ratings.csv\", sep=\",\", names = [\"movieId\",\"userId\", \"rating\", \"date\"])\n",
    "netflix_data[\"date\"] = pd.to_datetime(netflix_data[\"date\"])\n",
    "\n",
    "netflix_titles = pd.read_csv('netflix/movie_titles.csv', engine='python')\n",
    "\n",
    "netflix = pd.merge(netflix_data, netflix_titles, on='movieId')\n",
    "\n",
    "netflix = netflix.drop(netflix.columns[4], axis = 1)\n",
    "\n",
    "# dividing the data into a smaller set\n",
    "netflix_train, netflix_test = train_test_split(netflix, test_size = 0.005)\n",
    "\n",
    "\n",
    "### Dataset 2\n",
    "\n",
    "ml_movies = pd.read_csv('movielens/movies.csv')\n",
    "\n",
    "ml_ratings = pd.read_csv('movielens/ratings.csv')\n",
    "\n",
    "movielens = pd.merge(ml_ratings, ml_movies, on='movieId')\n",
    "\n",
    "movielens['genres'] = movielens['genres'].str.replace(\"|\",\" \")\n",
    "\n",
    "\n",
    "### Dataset 3\n",
    "\n",
    "tmdb = pd.read_csv('tmdb_dataset.csv')\n",
    "tmdb = tmdb.rename(columns = {'vote_average' : 'rating', \n",
    "                                            'vote_count' : 'rating_counts'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *Netflix Prize dataset*\n",
    "\n",
    "After combining all the text files into one and creating a DataFrame with the their corresponding columns here is what it looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>userId</th>\n",
       "      <th>rating</th>\n",
       "      <th>date</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1488844</td>\n",
       "      <td>3</td>\n",
       "      <td>2005-09-06</td>\n",
       "      <td>Dinosaur Planet</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>822109</td>\n",
       "      <td>5</td>\n",
       "      <td>2005-05-13</td>\n",
       "      <td>Dinosaur Planet</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>885013</td>\n",
       "      <td>4</td>\n",
       "      <td>2005-10-19</td>\n",
       "      <td>Dinosaur Planet</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>30878</td>\n",
       "      <td>4</td>\n",
       "      <td>2005-12-26</td>\n",
       "      <td>Dinosaur Planet</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>823519</td>\n",
       "      <td>3</td>\n",
       "      <td>2004-05-03</td>\n",
       "      <td>Dinosaur Planet</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId   userId  rating       date            title\n",
       "0        1  1488844       3 2005-09-06  Dinosaur Planet\n",
       "1        1   822109       5 2005-05-13  Dinosaur Planet\n",
       "2        1   885013       4 2005-10-19  Dinosaur Planet\n",
       "3        1    30878       4 2005-12-26  Dinosaur Planet\n",
       "4        1   823519       3 2004-05-03  Dinosaur Planet"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "netflix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Ratings Distribution**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtEAAAHwCAYAAABg0TMJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkHklEQVR4nO3de7RkZ1kn/u9DJxmQJARDxJALAWRAhkuAJqBhuClMuAaRkTBcJKIZ1iKIyowGR0GMMygICgaMGYjADyUzXMSAEXRGCMNNknAJuQjEyCWGkASEEEAg5Pn9Ubvx0Hafrjfp6qrT/fmsVatq7/3WrqfOqtX9Pe956t3V3QEAAOZ3k2UXAAAAG40QDQAAg4RoAAAYJEQDAMAgIRoAAAYJ0QAAMEiIBthJqurUqvr1Zdextar61ap61U4837VVdfvp8Wuq6rd24rlX8mcIsLWyTjSwp6qqTye5dZLvJLk2yTuSnNjd187x3Kcl+dnuvv8ia5yjjncnuV+SbyfpJJ9K8sYkv9fd37wB53p9d88duKvqNUku6+5fG3mt6blPywr8DAFuCDPRwJ7u0d29b5Ijk9wzyXOXW84NcmJ375fk4CTPSXJckrOqqnbmi1TVXjvzfAAbmRANkKS7r0jyzszCdJKkqk6qqr+vqq9W1UVV9RPT/h9OcmqSH5laG7487f9ua0NVPaiqLquq51TVlVX1+ao6fs25D6yqt1XVNVV1TlX9VlW9dzpWVfV70/O+UlXnV9Vd53gPX+vudyd5TJIfSfLI6Xy/UVWvnx7ftKpeX1VfrKovT69966r670n+fZJTpvd0yjS+q+qZVfWpzGa5t+z7oTUvfauq+uvp53R2Vd12GnfENPa74buq3l1VPzvPz3Da/rmquqSqvlRVZ1bVbdYc66p6RlV9qqr+qapesbN/cQDYHiEaIElVHZrk4UkuWbP77zMLlrdI8oIkr6+qg7v74iTPSPKB7t63uw/Yzml/cHruIUmenuQVVXXL6dgrknxtGvPT022LhyV5QJJ/m+SAJE9I8sV530t3fzbJuVPtW/vpqabDkhw4vY9vdPd/S/L/MpvV3re7T1zznMcmuW+Su2znJZ+U5OQkt0ry0SR/MkeNO/wZVtVDkrwwyU9lNsv+mSRnbDXsUUnuk+Qe07j/sKPXBtgZNmSIrqrTpxmaC+YY+3tV9dHp9sktsx0Ak7dW1VeTfC7JlUmev+VAd7+xuy/v7uu7+39lNhN71MC5v53kN7v72919VmZ913eqqk1JfjLJ87v76919UZLXbvW8/ZLcObPvrlzc3Z8ffF+XJ/n+7dR0YJIf6u7vdPd53X3NDs71wu7+Und/YzvH/6K73zP1YP+3zGaXDxusd1uelOT07v7wdO7nTuc+Ys2Y3+7uL0+/OLwra/6SALBIGzJEJ3lNkmPmGdjdv9jdR3b3kUn+IMlbFlgXsPE8duonflBmofVWWw5U1VOnX8C/PP0Cfte1x+fwxe6+bs3215Psm+SgJHtlFty3+O7j7v6bJKdkNlv9hao6rar2H3pXs9nvL21j//+XWdvKGVV1eVW9qKr23sG5Pjfv8elLmV9KcpvtD5/bbTKbfV577i9m9t62uGLN4y0/X4CF25Ahurvfk63+c6iqO1TVO6rqvKr6f1V152089YlJ3rBLigQ2lO4+O7Nf0H83Saa+3v+Z5MQkB07tBhck2dJze2OWNroqyXVJDl2z73tmbrv75d197yT/LrO2jv8678mnWeB7Z9ae8T2mWfEXdPddkvxoZu0QT91yeDun3NF7/W7tVbVvZjPgl2fWrpIk37dm7A8OnPfyJLddc+6bZzaL/o87eB7Awm3IEL0dpyV51vSfzn9J8sq1B6f/EG+X5G+WUBuwMfx+kodW1ZFJbp5ZyLsqSaYvBa79ct8XkhxaVfuMvkh3fyezv4r9RlV93/RL/5Ygm6q6T1Xdd5oh/lqSf85sGb51Ted6YJI/T/KhJGdtY8yDq+puU0vJNZm1d2w59xeS3H70/SR5RFXdf/pZnJzkb7v7c919VWaB98lVtamqfibJHdY8b0c/wz9NcnxVHVlV/ybJ/5jO/ekbUCPATrVbhOhp5uNHk7yxqj6a5I8y+xLKWscledP0nxfAvzKFvtcl+fWpT/klST6QWdi7W5L3rRn+N0kuTHJFVV19A17uxMy+4HdFZi0Wb0iyZV3n/TObBf+nzNoZvphphnw7Tpn6ur+Q2S8Cb05yTHdfv42xP5jkTZkF6IuTnJ3k9dOxlyV5/LTSxcsH3sufZtZL/qXMZsCftObYz2U2i/7FzGbV37/m2Lo/w+7+v0l+fXo/n88sgB83UBfAwmzYi61MXyx5e3ffdeoV/ER3bx2c147/SJJndvf7tzcGYFmq6neS/GB3//QOBwOwdLvFTPT0zfJ/qKr/mHx3jdV7bDleVXdKcsvMZpQAlq6q7lxVd5/+vToqsyXw/mzZdQEwnw0ZoqvqDZkF4jvV7GIGT8/sz4dPr6qPZfbnwWPXPOWJSc7ojTrtDuyO9susL/prSf53Zq0jf77UigCY24Zt5wAAgGXZkDPRAACwTEI0AAAM2mvZBYy61a1u1UccccSyywAAYDd33nnnXd3dB23r2IYL0UcccUTOPffcZZcBAMBurqo+s71j2jkAAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwaGEhuqpOr6orq+qC7Ryvqnp5VV1SVedX1b0WVQsAAOxMi5yJfk2SY9Y5/vAkd5xuJyT5wwXWAgAAO83CQnR3vyfJl9YZcmyS1/XMB5McUFUHL6oeAADYWZbZE31Iks+t2b5s2vevVNUJVXVuVZ171VVX7ZLiAABge5YZomsb+3pbA7v7tO7e3N2bDzrooAWXBQAA69tria99WZLD1mwfmuTyJdUCANt19gMeuOwSmNMD33P2sktgD7HMmegzkzx1WqXjfkm+0t2fX2I9AAAwl4XNRFfVG5I8KMmtquqyJM9PsneSdPepSc5K8ogklyT5epLjF1ULAADsTAsL0d39xB0c7yTPXNTrAwDAorhiIQAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYJAQDQAAg4RoAAAYJEQDAMAgIRoAAAYJ0QAAMEiIBgCAQUI0AAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYJAQDQAAg4RoAAAYJEQDAMAgIRoAAAYJ0QAAMEiIBgCAQUI0AAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABi017ILAFglR//B0csugQHve9b7ll0CsIcyEw0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYJAQDQAAg4RoAAAYJEQDAMAgIRoAAAYJ0QAAMGihIbqqjqmqT1TVJVV10jaO36Kq3lZVH6uqC6vq+EXWAwAAO8PCQnRVbUryiiQPT3KXJE+sqrtsNeyZSS7q7nskeVCSl1TVPouqCQAAdoZFzkQfleSS7r60u7+V5Iwkx241ppPsV1WVZN8kX0py3QJrAgCAG22RIfqQJJ9bs33ZtG+tU5L8cJLLk3w8ybO7+/qtT1RVJ1TVuVV17lVXXbWoegEAYC6LDNG1jX291fZ/SPLRJLdJcmSSU6pq/3/1pO7Tuntzd28+6KCDdnadAAAwZJEh+rIkh63ZPjSzGee1jk/ylp65JMk/JLnzAmsCAIAbbZEh+pwkd6yq201fFjwuyZlbjflskh9Lkqq6dZI7Jbl0gTUBAMCNtteiTtzd11XViUnemWRTktO7+8KqesZ0/NQkJyd5TVV9PLP2j1/p7qsXVRMAAOwMCwvRSdLdZyU5a6t9p655fHmShy2yBgAA2NlcsRAAAAYJ0QAAMEiIBgCAQUI0AAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYJAQDQAAg4RoAAAYJEQDAMAgIRoAAAYJ0QAAMEiIBgCAQUI0AAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABi01zyDquqWSW6T5BtJPt3d1y+0KgAAWGHbDdFVdYskz0zyxCT7JLkqyU2T3LqqPpjkld39rl1SJQAArJD1ZqLflOR1Sf59d3957YGquneSp1TV7bv71QusDwAAVs52Q3R3P3SdY+clOW8hFQEAwIrb4RcLq+roqrr59PjJVfXSqrrt4ksDAIDVNM/qHH+Y5OtVdY8kv5zkM5m1eQAAwB5pnhB9XXd3kmOTvKy7X5Zkv8WWBQAAq2ueJe6+WlXPTfLkJA+oqk1J9l5sWQAAsLrmmYl+QpJvJnl6d1+R5JAkL15oVQAAsMJ2OBM9BeeXrtn+bPREAwCwB5tndY7HVdWnquorVXVNVX21qq7ZFcUBAMAqmqcn+kVJHt3dFy+6GACAjeKU57xt2SUwpxNf8uidfs55eqK/IEADAMC/mGcm+tyq+l9J3prZFwyTJN39lkUVBQAAq2yeEL1/kq8nediafZ1EiAYAYI80z+ocx++KQgAAYKOYZ3WOQ6vqz6rqyqr6QlW9uaoO3RXFAQDAKprni4V/nOTMJLfJ7EIrb5v2AQDAHmmeEH1Qd/9xd1833V6T5KAF1wUAACtrnhB9dVU9uao2TbcnJ/niogsDAIBVNU+I/pkkP5XkiiSfT/L4aR8AAOyR5lmd47NJHrMLagEAgA1huyG6qn65u19UVX+Q2brQ36O7f36hlQEAwIpabyZ6y6W+z90VhQAAwEax3RDd3W+b7l+768oBAIDVt147x9uyjTaOLbpbnzQAAHuk9do5fneXVQEAABvIeu0cZ+/KQgAAYKNYr53j41m/nePuC6kIAABW3HrtHI/aZVUAAMAGsl47x2d2ZSEAALBRrNfO8d7uvn9VfTXf29ZRSbq79194dQAAsILWm4m+/3S/364rBwAAVt96PdHfVVW3THLY2vHd/eFFFQUAAKtshyG6qk5O8rQklya5ftrdSR6yuLIAAGB1zTMT/VNJ7tDd3xo9eVUdk+RlSTYleVV3//Y2xjwoye8n2TvJ1d39wNHXAQCAXWmeEH1BkgOSXDly4qralOQVSR6a5LIk51TVmd190ZoxByR5ZZJjuvuzVfUDI68BAADLME+IfmGSj1TVBUm+uWVndz9mB887Kskl3X1pklTVGUmOTXLRmjH/Kclbuvuz0zmHgjoAACzDPCH6tUl+J8nH8y890fM4JMnn1mxfluS+W435t0n2rqp3J9kvycu6+3UDrwEAALvcPCH66u5++Q04d21j39aXEd8ryb2T/FiSmyX5QFV9sLs/+T0nqjohyQlJcvjhh9+AUgAAYOeZJ0SfV1UvTHJmvredY0dL3F2W2bJ4Wxya5PJtjLm6u7+W5GtV9Z4k90jyPSG6u09LclqSbN68eesgDgAAu9Q8Ifqe0/391uybZ4m7c5Lcsapul+QfkxyXWQ/0Wn+e5JSq2ivJPpm1e/zeHDUBAMDS7DBEd/eDb8iJu/u6qjoxyTszW+Lu9O6+sKqeMR0/tbsvrqp3JDk/s37rV3X3BTfk9QAAYFfZboiuqicn+dPu3uaXCavqDkkO7u73bu8c3X1WkrO22nfqVtsvTvLikaIBAGCZ1puJPjCzpe3OS3JekquS3DTJDyV5YJKrk5y08AoBAGDFbDdEd/fLquqUzHqfj05y9yTfSHJxkqdsWdsZAAD2NOv2RHf3d5L89XQDAACS3GTZBQAAwEYjRAMAwCAhGgAABu0wRFfVs6tq/5p5dVV9uKoetiuKAwCAVTTPTPTPdPc1SR6W5KAkxyf57YVWBQAAK2yeEF3T/SOS/HF3f2zNPgAA2OPME6LPq6q/yixEv7Oq9svsEt0AALBHWned6MnTkxyZ5NLu/npVHZhZSwcAAOyR5gnRR073t6/6bhfHV6pqr+6+biFVAQDACpsnRL8yyb2SnJ9ZL/Rdp8cHVtUzuvuvFlgfAACsnHl6oj+d5J7dvbm7753knkkuSPLjSV60wNoAAGAlzROi79zdF27Z6O6LMgvVly6uLAAAWF3ztHN8oqr+MMkZ0/YTknyyqv5Nkm8vrDIAAFhR88xEPy3JJUl+IckvJrl02vftJA9eUF0AALCydjgT3d3fSPKS6ba1a3d6RQAAsOJ2GKKr6ugkv5HktmvHd/ftF1cWAACsrnl6ol+dWRvHeUm+s9hyAABg9c0Tor/S3X+58EoAAGCDmCdEv6uqXpzkLUm+uWVnd394YVUBAMAKmydE33e637xmXyd5yM4vBwAAVt88q3NYxg4AANbYboiuqid39+ur6pe2dby7X7q4sgAAYHWtNxN98+l+v20c6wXUAgAAG8J2Q3R3/9H08P909/vWHpvWjgYAgD3SPJf9/oM59wEAwB5hvZ7oH0nyo0kO2qovev8kmxZdGAAArKr1eqL3SbLvNGZtX/Q1SR6/yKIAAGCVrdcTfXaSs6vqNd39mV1YEwAArLR5Lrby9emKhf8uyU237OxuF1sBAGCPNM8XC/8kyd8luV2SFyT5dJJzFlgTAACstHlC9IHd/eok3+7us7v7Z5Lcb8F1AQDAypqnnePb0/3nq+qRSS5PcujiSgIAgNU2T4j+raq6RZLnZLY+9P5JfmGRRQEAwCrbYYju7rdPD7+S5MGJKxYCALBnW+9iK5uS/FSSQ5K8o7svqKpHJfnVJDdLcs9dUyIAAKyW9WaiX53ksCQfSvLyqvpMkh9JclJ3v3UX1AYAACtpvRC9Ocndu/v6qrppkquT/FB3X7FrSgMAgNW03hJ33+ru65Oku/85yScFaAAAWH8m+s5Vdf70uJLcYdquJN3dd194dQAAsILWC9E/vMuqAACADWS7Ibq7P7MrCwEAgI1inst+AwAAawjRAAAwaLshuqr+73T/O7uuHAAAWH3rfbHw4Kp6YJLHVNUZma3K8V3d/eGFVgYAACtqvRD9vCQnJTk0yUu3OtZJHrKoogAAYJWttzrHm5K8qap+vbtP3oU1AQDASltvJjpJ0t0nV9Vjkjxg2vXu7n77YssCAIDVtcPVOarqhUmeneSi6fbsaR8AAOyRdjgTneSRSY7s7uuTpKpem+QjSZ67yMIAAGBVzbtO9AFrHt9iAXUAAMCGMc9M9AuTfKSq3pXZMncPiFloAAD2YPN8sfANVfXuJPfJLET/SndfsejCAABgVc0zE53u/nySMxdcCwAAbAjz9kQDAAATIRoAAAatG6Kr6iZVdcGuKgYAADaCdUP0tDb0x6rq8F1UDwAArLx5vlh4cJILq+pDSb62ZWd3P2ZhVQEAwAqbJ0S/YOFVAADABjLPOtFnV9Vtk9yxu/9PVX1fkk2LLw0AAFbTDlfnqKqfS/KmJH807TokyVsXWBMAAKy0eZa4e2aSo5NckyTd/akkP7DIogAAYJXNE6K/2d3f2rJRVXsl6cWVBAAAq22eEH12Vf1qkptV1UOTvDHJ2xZbFgAArK55QvRJSa5K8vEk/znJWUl+bZFFAQDAKptndY7rq+q1Sf42szaOT3S3dg4AAPZYOwzRVfXIJKcm+fskleR2VfWfu/svF10cAACsonkutvKSJA/u7kuSpKrukOQvkgjRAADskebpib5yS4CeXJrkygXVAwAAK2+7M9FV9bjp4YVVdVaS/51ZT/R/THLOLqgNAABW0noz0Y+ebjdN8oUkD0zyoMxW6rjlPCevqmOq6hNVdUlVnbTOuPtU1Xeq6vFzVw4AAEuy3Zno7j7+xpy4qjYleUWShya5LMk5VXVmd1+0jXG/k+SdN+b1AABgV5lndY7bJXlWkiPWju/ux+zgqUcluaS7L53Oc0aSY5NctNW4ZyV5c5L7zF01AAAs0Tyrc7w1yaszu0rh9QPnPiTJ59ZsX5bkvmsHVNUhSX4iyUMiRAMAsEHME6L/ubtffgPOXdvYt/VFWn4/ya9093eqtjV8OlHVCUlOSJLDDz/8BpQCAAA7zzwh+mVV9fwkf5Xkm1t2dveHd/C8y5Ictmb70CSXbzVmc5IzpgB9qySPqKrruvutawd192lJTkuSzZs3u1oiAABLNU+IvluSp2TWcrGlnaOn7fWck+SOU0/1PyY5Lsl/Wjugu2+35XFVvSbJ27cO0AAAsGrmCdE/keT23f2tkRN393VVdWJmq25sSnJ6d19YVc+Yjp86XC0AAKyAeUL0x5IckBtwlcLuPivJWVvt22Z47u6njZ4fAACWYZ4Qfeskf1dV5+R7e6J3tMQdAADsluYJ0c9feBUAALCB7DBEd/fZu6IQAADYKOa5YuFX8y/rO++TZO8kX+vu/RdZGAAArKp5ZqL3W7tdVY/N7JLeAACwR7rJ6BOmdZx3tEY0AADstuZp53jcms2bZHaVQVcNBABgjzXP6hyPXvP4uiSfTnLsQqoBAIANYJ6e6ON3RSEAALBRbDdEV9Xz1nled/fJC6gHAABW3noz0V/bxr6bJ3l6kgOTCNEAAOyRthuiu/slWx5X1X5Jnp3k+CRnJHnJ9p4HAAC7u3V7oqvq+5P8UpInJXltknt19z/tisIAAGBVrdcT/eIkj0tyWpK7dfe1u6wqAABYYetdbOU5SW6T5NeSXF5V10y3r1bVNbumPAAAWD3r9UQPX80QAAD2BIIyAAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYNBeyy4Alumzv3m3ZZfAnA5/3seXXQIAfJeZaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYJAQDQAAg4RoAAAYJEQDAMAgIRoAAAYJ0QAAMEiIBgCAQUI0AAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYNBCQ3RVHVNVn6iqS6rqpG0cf1JVnT/d3l9V91hkPQAAsDMsLERX1aYkr0jy8CR3SfLEqrrLVsP+IckDu/vuSU5Octqi6gEAgJ1lkTPRRyW5pLsv7e5vJTkjybFrB3T3+7v7n6bNDyY5dIH1AADATrHIEH1Iks+t2b5s2rc9T0/ylwusBwAAdoq9Fnju2sa+3ubAqgdnFqLvv53jJyQ5IUkOP/zwnVUfAADcIIucib4syWFrtg9NcvnWg6rq7kleleTY7v7itk7U3ad19+bu3nzQQQctpFgAAJjXIkP0OUnuWFW3q6p9khyX5My1A6rq8CRvSfKU7v7kAmsBAICdZmHtHN19XVWdmOSdSTYlOb27L6yqZ0zHT03yvCQHJnllVSXJdd29eVE1AQDAzrDInuh091lJztpq36lrHv9skp9dZA0AALCzuWIhAAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYJAQDQAAg4RoAAAYtNeyC9hV7v1fX7fsEpjTeS9+6rJLAABYl5loAAAYJEQDAMAgIRoAAAYJ0QAAMEiIBgCAQUI0AAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYJAQDQAAg4RoAAAYJEQDAMAgIRoAAAYJ0QAAMEiIBgCAQUI0AAAMEqIBAGCQEA0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEFCNAAADBKiAQBgkBANAACDhGgAABgkRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogEAYJAQDQAAg4RoAAAYJEQDAMAgIRoAAAYJ0QAAMEiIBgCAQQsN0VV1TFV9oqouqaqTtnG8qurl0/Hzq+pei6wHAAB2hoWF6KralOQVSR6e5C5JnlhVd9lq2MOT3HG6nZDkDxdVDwAA7CyLnIk+Kskl3X1pd38ryRlJjt1qzLFJXtczH0xyQFUdvMCaAADgRltkiD4kyefWbF827RsdAwAAK2WvBZ67trGvb8CYVNUJmbV7JMm1VfWJG1nb7uJWSa5edhE7W/3uTy+7hI1ut/xc5Pnb+ueCAbvl56J+3ufiRtr9PhflM7ET7Hafi2e99AY/9bbbO7DIEH1ZksPWbB+a5PIbMCbdfVqS03Z2gRtdVZ3b3ZuXXQerxeeCbfG5YFt8LtgWn4v5LLKd45wkd6yq21XVPkmOS3LmVmPOTPLUaZWO+yX5Snd/foE1AQDAjbawmejuvq6qTkzyziSbkpze3RdW1TOm46cmOSvJI5JckuTrSY5fVD0AALCzLLKdI919VmZBee2+U9c87iTPXGQNuzktLmyLzwXb4nPBtvhcsC0+F3OoWY4FAADm5bLfAAAwSIjegKrq9Kq6sqouWHYtrI6qOqyq3lVVF1fVhVX17GXXxPJV1U2r6kNV9bHpc/GCZdfEaqiqTVX1kap6+7JrYTVU1aer6uNV9dGqOnfZ9aw67RwbUFU9IMm1mV3t8a7LrofVMF3t8+Du/nBV7ZfkvCSP7e6LllwaS1RVleTm3X1tVe2d5L1Jnj1dJZY9WFX9UpLNSfbv7kctux6Wr6o+nWRzd+9Wa0QvipnoDai735PkS8uug9XS3Z/v7g9Pj7+a5OK4Auger2eunTb3nm5mT/ZwVXVokkcmedWya4GNSoiG3VBVHZHknkn+dsmlsAKmP9t/NMmVSf66u30u+P0kv5zk+iXXwWrpJH9VVedNV4tmHUI07Gaqat8kb07yC919zbLrYfm6+zvdfWRmV4U9qqq0ge3BqupRSa7s7vOWXQsr5+juvleShyd55tQ+ynYI0bAbmXpe35zkT7r7Lcuuh9XS3V9O8u4kxyy3Epbs6CSPmfpfz0jykKp6/XJLYhV09+XT/ZVJ/izJUcutaLUJ0bCbmL5A9uokF3f3S5ddD6uhqg6qqgOmxzdL8uNJ/m6pRbFU3f3c7j60u49IclySv+nuJy+5LJasqm4+fSk9VXXzJA9LYhWwdQjRG1BVvSHJB5Lcqaouq6qnL7smVsLRSZ6S2azSR6fbI5ZdFEt3cJJ3VdX5Sc7JrCfakmbA1m6d5L1V9bEkH0ryF939jiXXtNIscQcAAIPMRAMAwCAhGgAABgnRAAAwSIgGAIBBQjQAAAwSogFWWFV9Z1qu8IKqetuWNZ/XGX/k2qUNq+oxVXXSwgsF2MNY4g5ghVXVtd297/T4tUk+2d3/fZ3xT0uyubtP3EUlAuyRzEQDbBwfSHJIklTVUVX1/qr6yHR/p6raJ8lvJnnCNHv9hKp6WlWdMj3nNVX18mn8pVX1+Gn/TarqlVV1YVW9varOWnPst6vqoqo6v6p+d0nvG2Dl7LXsAgDYsaralOTHMru0ezK7dPcDuvu6qvrxJP+ju3+yqp6XNTPR08z0WgcnuX+SOyc5M8mbkjwuyRFJ7pbkB5JcnOT0qvr+JD+R5M7d3TtqJQHYkwjRAKvtZlX10cxC7nlJ/nraf4skr62qOybpJHvPeb63dvf1SS6qqltP++6f5I3T/iuq6l3T/muS/HOSV1XVXyRxuXCAiXYOgNX2je4+Msltk+yT5JnT/pOTvKu775rk0UluOuf5vrnmcW11/z26+7okRyV5c5LHJnnHSOEAuzMhGmAD6O6vJPn5JP+lqvbObCb6H6fDT1sz9KtJ9hs8/XuT/OTUG33rJA9KkqraN8ktuvusJL+Q5MgbWD7AbkeIBtgguvsjST6W5LgkL0rywqp6X5JNa4a9K8ldtnyxcM5TvznJZUkuSPJHSf42yVcyC+Nvr6rzk5yd5Bd3yhsB2A1Y4g6AVNW+3X1tVR2Y5ENJju7uK5ZdF8Cq8sVCAJLZjPMBmfVdnyxAA6zPTDQAAAzSEw0AAIOEaAAAGCREAwDAICEaAAAGCdEAADBIiAYAgEH/P6vFRbEdZfoDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plotting the distribution of ratings\n",
    "plt.figure(figsize = (12, 8))\n",
    "ax = sns.countplot(x = \"rating\", data = netflix)\n",
    "\n",
    "plt.title(\"Ratings Distribution\")\n",
    "plt.xlabel(\"Ratings\")\n",
    "plt.ylabel(\"Number of Ratings (millions)\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Users With Most Ratings**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "userId\n",
       "305344     5394\n",
       "387418     5342\n",
       "2439493    5051\n",
       "1664010    4856\n",
       "2118461    4551\n",
       "1639792    3023\n",
       "1314869    2976\n",
       "1461435    2971\n",
       "1932594    2728\n",
       "2606799    2711\n",
       "Name: rating, dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# users with most ratings\n",
    "netflix_user_ratings = netflix.groupby(\"userId\")[\"rating\"].count().sort_values(ascending = False)\n",
    "netflix_user_ratings.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Global Average Rating**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The global average rating for this dataset is 3.59\n"
     ]
    }
   ],
   "source": [
    "# gloval average rating\n",
    "netflix_global_rating = netflix['rating'].values.sum() / np.count_nonzero(netflix['rating'].values)\n",
    "print('The global average rating for this dataset is %.2f' % netflix_global_rating)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Most Popular Movies/TV Shows**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "      <th>rating_counts</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>The Simpsons: Season 5</th>\n",
       "      <td>4.542563</td>\n",
       "      <td>17292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lord of the Rings: The Fellowship of the Ring</th>\n",
       "      <td>4.434708</td>\n",
       "      <td>149866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Finding Nemo (Widescreen)</th>\n",
       "      <td>4.415523</td>\n",
       "      <td>140979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Simpsons: Season 3</th>\n",
       "      <td>4.404792</td>\n",
       "      <td>22498</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Godfather</th>\n",
       "      <td>4.404083</td>\n",
       "      <td>71614</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CSI: Season 1</th>\n",
       "      <td>4.396755</td>\n",
       "      <td>17013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Alias: Season 2</th>\n",
       "      <td>4.391627</td>\n",
       "      <td>14427</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Simpsons: Treehouse of Horror</th>\n",
       "      <td>4.386774</td>\n",
       "      <td>15469</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Alias: Season 1</th>\n",
       "      <td>4.344422</td>\n",
       "      <td>16683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pride and Prejudice</th>\n",
       "      <td>4.338748</td>\n",
       "      <td>19885</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 rating  rating_counts\n",
       "title                                                                 \n",
       "The Simpsons: Season 5                         4.542563          17292\n",
       "Lord of the Rings: The Fellowship of the Ring  4.434708         149866\n",
       "Finding Nemo (Widescreen)                      4.415523         140979\n",
       "The Simpsons: Season 3                         4.404792          22498\n",
       "The Godfather                                  4.404083          71614\n",
       "CSI: Season 1                                  4.396755          17013\n",
       "Alias: Season 2                                4.391627          14427\n",
       "The Simpsons: Treehouse of Horror              4.386774          15469\n",
       "Alias: Season 1                                4.344422          16683\n",
       "Pride and Prejudice                            4.338748          19885"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# popular movies/tv shows based on highest average rating\n",
    "netflix_popular_movies = pd.DataFrame(netflix.groupby(\"title\")[\"rating\"].mean())\n",
    "# counting how many users rated the title\n",
    "netflix_popular_movies['rating_counts'] = pd.DataFrame(netflix.groupby('title')['rating'].count())\n",
    "\n",
    "# We will use cutoff m as the 90th percentile, so for a movie to be recommended\n",
    "# it must have more votes than at least 90% of the movies on the list\n",
    "m = netflix_popular_movies['rating_counts'].quantile(0.9)\n",
    "\n",
    "netflix_popular_movies[netflix_popular_movies.rating_counts >= m].sort_values('rating', ascending = False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *MovieLens dataset*\n",
    "\n",
    "Now we will be doing the same analysis for the MovieLens dataset and here is what the data looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4.0</td>\n",
       "      <td>964982703</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>4.0</td>\n",
       "      <td>847434962</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1106635946</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1510577970</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>17</td>\n",
       "      <td>1</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1305696483</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure Animation Children Comedy Fantasy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userId  movieId  rating   timestamp             title  \\\n",
       "0       1        1     4.0   964982703  Toy Story (1995)   \n",
       "1       5        1     4.0   847434962  Toy Story (1995)   \n",
       "2       7        1     4.5  1106635946  Toy Story (1995)   \n",
       "3      15        1     2.5  1510577970  Toy Story (1995)   \n",
       "4      17        1     4.5  1305696483  Toy Story (1995)   \n",
       "\n",
       "                                        genres  \n",
       "0  Adventure Animation Children Comedy Fantasy  \n",
       "1  Adventure Animation Children Comedy Fantasy  \n",
       "2  Adventure Animation Children Comedy Fantasy  \n",
       "3  Adventure Animation Children Comedy Fantasy  \n",
       "4  Adventure Animation Children Comedy Fantasy  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movielens.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Ratings Distribution**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuAAAAHwCAYAAADn4NoPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAprklEQVR4nO3dedytZV0v/s9XcErAAZAQUBz4ZQ6FSUQHj2MpWQ4VFr1yzKL8QUfP8Zx+aIOacdRKPZFpWRiYFRJOaFh5HOjYURBwYMrcGSqBgBPghALf3x/rftyL3bOfvfZmr/t59lrv9+u1Xute1z0833V5Fx8urnXd1d0BAADGcZv1LgAAAJaJAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcYIOoqj+uqt9c7zq2VFUvrKo/24nX+2pV3WfYPqWqfmcnXntD9iHAtLIOOMCOqarLkuyX5KYkX03yd0mO7+6vznDuM5P8Ync/bJ41zlDHB5IckeTbSTrJp5L8TZJXd/cNO3CtN3X3zGG9qk5Jcnl3/8b2/K3h3GdmA/QhwPYyAg5w6zyhu/dIcmiShyR5wfqWs0OO7+49k+yf5PlJjklyVlXVzvwjVbX7zrwewK5KAAfYCbr780n+PpMgniSpqhOq6l+r6vqquqSqfnJo/94kf5zkh4fpGF8Z2r8zHaOqHllVl1fV86vq6qq6sqqeNXXtvavqnVV1XVV9pKp+p6o+OOyrqnr1cN61VfWJqnrQDN/ha939gSRPTPLDSX58uN6Lq+pNw/YdqupNVfXFqvrK8Lf3q6oTk/znJK8ZvtNrhuO7qo6rqk9lMrq+0na/qT+9T1W9Z+ins6vqXsNxBw/Hfie4V9UHquoXZ+nD4fMvVdWmqvpSVZ1ZVfeY2tdV9StV9amq+nJV/dHO/pcOgNUI4AA7QVUdmOTHkmyaav7XTELpnZO8JMmbqmr/7r40ya8k+VB379Hdd9nKZb97OPeAJM9O8kdVdddh3x8l+dpwzDOG14rHJnl4kv8nyV2S/GySL876Xbr7s0nOG2rf0jOGmg5KsvfwPb7R3b+e5P9kMpq+R3cfP3XOk5P8UJIHbOVP/nySlybZJ8nHkvzlDDVusw+r6tFJXpbkZzIZ3f9MktO2OOwnkvxgku8fjnvctv42wK0lgAPcOm+vquuTfC7J1UletLKju/+mu6/o7pu7+82ZjAAfvh3X/naS3+7ub3f3WZnMM/+eqtotyU8neVF3f727L0ly6hbn7Znk/pn81ufS7r5yO7/XFUnutpWa9k5yv+6+qbvP7+7rtnGtl3X3l7r7G1vZ/7fd/Y/DnPNfz2RU+6DtrHc1P5/kDd19wXDtFwzXPnjqmJd391eGf+l4f6b+CwbAvAjgALfOk4f504/MJPDus7Kjqp5eVR8bpmp8JcmDpvfP4IvdfePU568n2SPJvkl2zyT0r/jOdne/L8lrMhklv6qqXl9Ve23Xt5qMun9plfa/yGSqzWlVdUVV/W5V3XYb1/rcrPuHH7B+Kck9tn74zO6Ryaj39LW/mMl3W/H5qe2V/gWYKwEcYCfo7rOTnJLk95NkmMf8p0mOT7L3MEXioiQrc4xvzRJU1yS5McmBU223GDHu7pO6+6FJHpjJVJT/MevFh9Hnh2YypeQWhtH4l3T3A5L8p0ymcDx9ZfdWLrmt7/qd2qtqj0xG3q/IZIpNknzX1LHfvR3XvSLJvaaufadMRu//fRvnAcyVAA6w8/yvJD9aVYcmuVMmAfGaJBl+QDn9Q8irkhxYVbfb3j/S3TcleWuSF1fVd1XV/bM5BKeqfrCqfmgYmf5akm9mslTimoZrPSLJO5Kcm+SsVY55VFU9eJgGc10mU1JWrn1Vkvts7/dJ8viqetjQFy9Nck53f667r8kkLD+1qnarql9Ict+p87bVh3+V5FlVdWhV3T7J/xyufdkO1Aiw0wjgADvJEBjfmOQ3h3nZr0zyoUyC4oOT/NPU4e9LcnGSz1fVF3bgzx2fyY8hP5/JtJC/TrKybvdemYy+fzmTKRhfzDAyvxWvGeaxX5XJv0S8JclR3X3zKsd+d5IzMgnflyY5O8mbhn1/kOToYUWRk7bju/xVJnPnv5TJyPvPT+37pUxG77+YyWj+/53at2Yfdvd7k/zm8H2uzCS8H7MddQHMhQfxACyAqnpFku/u7mds82AA1pURcIBdUFXdv6q+b1jz+/BMlil823rXBcC2eSoZwK5pz0ymndwjk+UPX5nJ3G0ANjhTUAAAYESmoAAAwIgEcAAAGNHSzQHfZ599+uCDD17vMgAAWHDnn3/+F7p73y3bly6AH3zwwTnvvPPWuwwAABZcVX1mtXZTUAAAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEe2+3gUAAIvlxS9+8XqXMBeL+r0YnxFwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARjS3AF5VB1XV+6vq0qq6uKqeO7S/uKr+vao+NrweP3XOC6pqU1V9sqoeN9X+0Kq6cNh3UlXV0H77qnrz0H5OVR08r+8DAAA7wzxHwG9M8vzu/t4kRyQ5rqoeMOx7dXcfOrzOSpJh3zFJHpjkqCSvrardhuNfl+TYJIcMr6OG9mcn+XJ33y/Jq5O8Yo7fBwAAbrW5BfDuvrK7Lxi2r09yaZID1jjlSUlO6+4buvvfkmxKcnhV7Z9kr+7+UHd3kjcmefLUOacO22ckeczK6DgAAGxEo8wBH6aGPCTJOUPT8VX1iap6Q1XddWg7IMnnpk67fGg7YNjesv0W53T3jUmuTbL3PL4DAADsDHMP4FW1R5K3JHled1+XyXSS+yY5NMmVSV65cugqp/ca7Wuds2UNx1bVeVV13jXXXLN9XwAAAHaiuQbwqrptJuH7L7v7rUnS3Vd1903dfXOSP01y+HD45UkOmjr9wCRXDO0HrtJ+i3Oqavckd07ypS3r6O7Xd/dh3X3Yvvvuu7O+HgAAbLd5roJSSU5Ocml3v2qqff+pw34yyUXD9plJjhlWNrl3Jj+2PLe7r0xyfVUdMVzz6UneMXXOM4bto5O8b5gnDgAAG9Luc7z2kUmeluTCqvrY0PbCJD9XVYdmMlXksiS/nCTdfXFVnZ7kkkxWUDmuu28azntOklOS3DHJu4dXMgn4f1FVmzIZ+T5mjt8HAAButbkF8O7+YFafo33WGuecmOTEVdrPS/KgVdq/meQpt6JMAAAYlSdhAgDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEcwvgVXVQVb2/qi6tqour6rlD+92q6j1V9anh/a5T57ygqjZV1Ser6nFT7Q+tqguHfSdVVQ3tt6+qNw/t51TVwfP6PgAAsDPMcwT8xiTP7+7vTXJEkuOq6gFJTkjy3u4+JMl7h88Z9h2T5IFJjkry2qrabbjW65Icm+SQ4XXU0P7sJF/u7vsleXWSV8zx+wAAwK02twDe3Vd29wXD9vVJLk1yQJInJTl1OOzUJE8etp+U5LTuvqG7/y3JpiSHV9X+Sfbq7g91dyd54xbnrFzrjCSPWRkdBwCAjWj3Mf7IMDXkIUnOSbJfd1+ZTEJ6Vd19OOyAJB+eOu3yoe3bw/aW7SvnfG641o1VdW2SvZN8YT7fBIAtnfjUo9e7hJ3u1990xnqXACywuf8Is6r2SPKWJM/r7uvWOnSVtl6jfa1ztqzh2Ko6r6rOu+aaa7ZVMgAAzM1cA3hV3TaT8P2X3f3WofmqYVpJhverh/bLkxw0dfqBSa4Y2g9cpf0W51TV7knunORLW9bR3a/v7sO6+7B99913Z3w1AADYIfNcBaWSnJzk0u5+1dSuM5M8Y9h+RpJ3TLUfM6xscu9Mfmx57jBd5fqqOmK45tO3OGflWkcned8wTxwAADakec4BPzLJ05JcWFUfG9pemOTlSU6vqmcn+WySpyRJd19cVacnuSSTFVSO6+6bhvOek+SUJHdM8u7hlUwC/l9U1aZMRr6PmeP3AQCAW21uAby7P5jV52gnyWO2cs6JSU5cpf28JA9apf2bGQI8AADsCjwJEwAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMKLdZzmoqu6a5B5JvpHksu6+ea5VAQDAgtpqAK+qOyc5LsnPJbldkmuS3CHJflX14SSv7e73j1IlAAAsiLVGwM9I8sYk/7m7vzK9o6oemuRpVXWf7j55jvUBAMBC2WoA7+4fXWPf+UnOn0tFAACwwLb5I8yqOrKq7jRsP7WqXlVV95p/aQAAsHhmWQXldUm+XlXfn+TXknwmk6kpAADAdpolgN/Y3Z3kSUn+oLv/IMme8y0LAAAW0yzLEF5fVS9I8tQkD6+q3ZLcdr5lAQDAYpplBPxnk9yQ5Nnd/fkkByT5vblWBQAAC2qbI+BD6H7V1OfPxhxwAADYIbOsgvJTVfWpqrq2qq6rquur6roxigMAgEUzyxzw303yhO6+dN7FAADAoptlDvhVwjcAAOwcs4yAn1dVb07y9kx+jJkk6e63zqsoAABYVLME8L2SfD3JY6faOokADgAA22mWVVCeNUYhAACwDGZZBeXAqnpbVV1dVVdV1Vuq6sAxigMAgEUzy48w/zzJmUnukclDeN45tAEAANtplgC+b3f/eXffOLxOSbLvnOsCAICFNEsA/0JVPbWqdhteT03yxXkXBgAAi2iWAP4LSX4myeeTXJnk6KENAADYTrOsgvLZJE8coRYAAFh4Ww3gVfVr3f27VfWHmaz7fQvd/V/mWhkAACygtUbAVx4/f94YhQAAwDLYagDv7ncO76eOVw4AACy2taagvDOrTD1Z0d3mhQMAwHZaawrK749WBQAALIm1pqCcPWYhAACwDNaagnJh1p6C8n1zqQgAABbYWlNQfmK0KgAAYEmsNQXlM2MWAgAAy2CtKSgf7O6HVdX1ueVUlErS3b3X3KsDAIAFs9YI+MOG9z3HKwcAABbbWnPAv6Oq7prkoOnju/uCeRUFAACLapsBvKpemuSZST6d5OahuZM8en5lAQDAYpplBPxnkty3u78172IAAGDR3WaGYy5Kcpc51wEAAEthlhHwlyX5aFVdlOSGlcbufuLcqgIAgAU1SwA/NckrklyYzXPAAQCAHTBLAP9Cd58090oAAGAJzBLAz6+qlyU5M7ecgmIZQgAA2E6zBPCHDO9HTLVZhhAAAHbANgN4dz9qjEIAAGAZbHUZwqp6alWttf++VfWw+ZQFAACLaa0R8L0zWX7w/CTnJ7kmyR2S3C/JI5J8IckJc68QAAAWyFYDeHf/QVW9JpO53kcm+b4k30hyaZKndfdnxykRAAAWx5pzwLv7piTvGV4AAMCtNMuj6AEAgJ1EAAcAgBEJ4AAAMKJtBvCqem5V7VUTJ1fVBVX12DGKAwCARTPLCPgvdPd1SR6bZN8kz0ry8rlWBQAAC2qWAF7D++OT/Hl3f3yqbesnVb2hqq6uqoum2l5cVf9eVR8bXo+f2veCqtpUVZ+sqsdNtT+0qi4c9p1UVTW0376q3jy0n1NVB8/4nQEAYN3MEsDPr6p/yCSA/31V7Znk5hnOOyXJUau0v7q7Dx1eZyVJVT0gyTFJHjic89qq2m04/nVJjk1yyPBaueazk3y5u++X5NVJXjFDTQAAsK5mCeDPzuSJlz/Y3V9PcrtMpqGsqbv/McmXZqzjSUlO6+4buvvfkmxKcnhV7Z9kr+7+UHd3kjcmefLUOacO22ckeczK6DgAAGxUswTwQ4f3+1TVDyS5V5KvVdWaD/FZw/FV9Ylhispdh7YDknxu6pjLh7YDhu0t229xTnffmOTaJHvvYE0AADCKWQL4a5N8OMnrk/xpkg8lOS3Jv+zAaiivS3LfTEL9lUleObSvNnLda7Svdc5/UFXHVtV5VXXeNddcs10FAwDAzjRLAL8syUO6+7DufmiShyS5KMmPJPnd7flj3X1Vd9/U3TdnEuYPH3ZdnuSgqUMPTHLF0H7gKu23OGcYjb9ztjLlpbtfP9R/2L777rs9JQMAwE41SwC/f3dfvPKhuy/JJJB/env/2DCne8VPZhLkk+TMJMcMK5vcO5MfW57b3Vcmub6qjhjmdz89yTumznnGsH10kvcN88QBAGDDmmUe9yer6nWZTDtJkp/NZPrJ7ZN8e2snVdVfJ3lkkn2q6vIkL0ryyKo6NJOpIpcl+eUk6e6Lq+r0JJckuTHJcd1903Cp52Syosodk7x7eCXJyUn+oqo2ZTLyfcwM3wUAANbVLAH8mUn+3yTPy2Te9QeT/PdMwvejtnZSd//cKs0nr3H8iUlOXKX9vCQPWqX9m0mesmblAACwwWwzgHf3NzL5seQrV9n91Z1eEQAALLBtBvCqOjLJizNZfvA7x3f3feZXFgAALKZZpqCcnOS/Jjk/yU3bOBYAAFjDLAH82u5+97YPAwAAtmWWAP7+qvq9JG9NcsNKY3dfMLeqAABgQc0SwH9oeD9sqq2TPHrnlwMAAIttllVQtrrUIAAAsH22GsCr6qnd/aaq+m+r7e/uV82vLAAAWExrjYDfaXjfc5V9HvkOAAA7YKsBvLv/ZNj83939T9P7hrXBAQCA7XSbGY75wxnbAACAbVhrDvgPJ/lPSfbdYh74Xkl2m3dhAACwiNaaA367JHsMx0zPA78uydHzLAoAABbVWnPAz05ydlWd0t2fGbEmAABYWLM8iOfrw5MwH5jkDiuN3e1BPAAAsJ1m+RHmXyb55yT3TvKSJJcl+cgcawIAgIU1SwDfu7tPTvLt7j67u38hyRFzrgsAABbSLFNQvj28X1lVP57kiiQHzq8kAABYXLME8N+pqjsneX4m63/vleR58ywKAAAW1TYDeHe/a9i8NsmjEk/CBACAHbXWg3h2S/IzSQ5I8nfdfVFV/USSFya5Y5KHjFMiAAAsjrVGwE9OclCSc5OcVFWfSfLDSU7o7rePUBsAACyctQL4YUm+r7tvrqo7JPlCkvt19+fHKQ0AABbPWgH8W919c5J09zer6l+EbwAAdsT3n/H3613CXHz86Mdt9zlrBfD7V9Unhu1Kct/hcyXp7v6+7S8RABbXpSe+b71L2Om+99c9+Bp2trUC+PeOVgUAACyJrQbw7v7MmIUAAMAymOVR9AAAwE4igAMAwIi2GsCr6r3D+yvGKwcAABbbWj/C3L+qHpHkiVV1Wiarn3xHd18w18oAAGABrRXAfyvJCUkOTPKqLfZ1EusSAQDAdlprFZQzkpxRVb/Z3S8dsSYAAFhYa42AJ0m6+6VV9cQkDx+aPtDd75pvWQAAsJi2uQpKVb0syXOTXDK8nju0AQAA22mbI+BJfjzJod19c5JU1alJPprkBfMsDAAAFtGs64DfZWr7znOoAwAAlsIsI+AvS/LRqnp/JksRPjxGvwEAYIfM8iPMv66qDyT5wUwC+P/X3Z+fd2EAALCIZhkBT3dfmeTMOdcCAAALb9Y54AAAwE4ggAMAwIjWDOBVdZuqumisYgAAYNGtGcCHtb8/XlX3HKkeAABYaLP8CHP/JBdX1blJvrbS2N1PnFtVAACwoGYJ4C+ZexUAALAkZlkH/OyquleSQ7r7f1fVdyXZbf6lAQDA4tnmKihV9UtJzkjyJ0PTAUnePseaAABgYc2yDOFxSY5Mcl2SdPenktx9nkUBAMCimiWA39Dd31r5UFW7J+n5lQQAAItrlgB+dlW9MMkdq+pHk/xNknfOtywAAFhMswTwE5Jck+TCJL+c5KwkvzHPogAAYFHNsgrKzVV1apJzMpl68snuNgUFAAB2wDYDeFX9eJI/TvKvSSrJvavql7v73fMuDgAAFs0sD+J5ZZJHdfemJKmq+yb52yQCOAAAbKdZ5oBfvRK+B59OcvWc6gEAgIW21RHwqvqpYfPiqjoryemZzAF/SpKPjFAbAAAsnLWmoDxhavuqJI8Ytq9Jcte5VQQAAAtsqwG8u581ZiEAALAMZlkF5d5JfjXJwdPHd/cT51cWAAAspllWQXl7kpMzefrlzXOtBgAAFtwsAfyb3X3S3CsBAIAlMEsA/4OqelGSf0hyw0pjd18wt6oAAGBBzRLAH5zkaUkenc1TUHr4DAAAbIdZAvhPJrlPd39r3sUAAMCim+VJmB9Pcpc51wEAAEthlhHw/ZL8c1V9JLecA24ZQgAA2E6zBPAXzb0KAABYEtsM4N199hiFAADAMtjmHPCqur6qrhte36yqm6rquhnOe0NVXV1VF0213a2q3lNVnxre7zq17wVVtamqPllVj5tqf2hVXTjsO6mqami/fVW9eWg/p6oO3u5vDwAAI9tmAO/uPbt7r+F1hyQ/neQ1M1z7lCRHbdF2QpL3dvchSd47fE5VPSDJMUkeOJzz2qrabTjndUmOTXLI8Fq55rOTfLm775fk1UleMUNNAACwrmZZBeUWuvvtmWEN8O7+xyRf2qL5SUlOHbZPTfLkqfbTuvuG7v63JJuSHF5V+yfZq7s/1N2d5I1bnLNyrTOSPGZldBwAADaqbc4Br6qfmvp4mySHZfIgnh2xX3dfmSTdfWVV3X1oPyDJh6eOu3xo+/awvWX7yjmfG651Y1Vdm2TvJF9Y5Tscm8koeu55z3vuYOkAAHDrzbIKyhOmtm9Mclkmo88702oj171G+1rn/MfG7tcneX2SHHbYYTv6Lw8AAHCrzbIKyrN24t+7qqr2H0a/909y9dB+eZKDpo47MMkVQ/uBq7RPn3N5Ve2e5M75j1NeAABgQ9lqAK+q31rjvO7ul+7A3zszyTOSvHx4f8dU+19V1auS3COTH1ue2903DauwHJHknCRPT/KHW1zrQ0mOTvK+YZ44AABsWGuNgH9tlbY7ZbL6yN5J1gzgVfXXSR6ZZJ+qujyTB/q8PMnpVfXsJJ9N8pQk6e6Lq+r0JJdkMs3luO6+abjUczJZUeWOSd49vJLk5CR/UVWbMhn5PmategAAYCPYagDv7leubFfVnkmem+RZSU5L8sqtnTd1/s9tZddjtnL8iUlOXKX9vCQPWqX9mxkCPAAA7CrWnANeVXdL8t+S/HwmS/79QHd/eYzCAABgEa01B/z3kvxUJquHPLi7vzpaVQAAsKDWehDP8zP5QeRvJLli6nH018/yKHoAAOA/WmsO+HY/JRMAAFibkA0AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwot3XuwAAgEV1+t8cvt4lzMXPPOXc9S5hl2YEHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiARwAAAYkQAOAAAjEsABAGBEAjgAAIxo9/UuANh1nP3wR6x3CTvdI/7x7PUuAYAlYwQcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjGj39S4AYFf0mue/c71L2OmOf+UT1rsEgKVgBBwAAEYkgAMAwIgEcAAAGJEADgAAI1qXAF5Vl1XVhVX1sao6b2i7W1W9p6o+Nbzfder4F1TVpqr6ZFU9bqr9ocN1NlXVSVVV6/F9AABgVus5Av6o7j60uw8bPp+Q5L3dfUiS9w6fU1UPSHJMkgcmOSrJa6tqt+Gc1yU5Nskhw+uoEesHAIDttpGmoDwpyanD9qlJnjzVflp339Dd/5ZkU5LDq2r/JHt194e6u5O8ceocAADYkNYrgHeSf6iq86vq2KFtv+6+MkmG97sP7Qck+dzUuZcPbQcM21u2AwDAhrVeD+I5sruvqKq7J3lPVf3zGseuNq+712j/jxeYhPxjk+Se97zn9tYKAAA7zbqMgHf3FcP71UneluTwJFcN00oyvF89HH55koOmTj8wyRVD+4GrtK/2917f3Yd192H77rvvzvwqAACwXUYP4FV1p6rac2U7yWOTXJTkzCTPGA57RpJ3DNtnJjmmqm5fVffO5MeW5w7TVK6vqiOG1U+ePnUOAABsSOsxBWW/JG8bVgzcPclfdfffVdVHkpxeVc9O8tkkT0mS7r64qk5PckmSG5Mc1903Ddd6TpJTktwxybuHFwAAbFijB/Du/nSS71+l/YtJHrOVc05McuIq7ecledDOrhEAAOZlIy1DCAAAC08ABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGtPt6F8DG9NnffvB6l7DT3fO3LlzvEgAAjIADAMCYBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEAYEQCOAAAjEgABwCAEQngAAAwot3XuwDY6I78wyPXu4Sd7p9+9Z/WuwQAWFpGwAEAYEQCOAAAjEgABwCAEQngAAAwIgEcAABGJIADAMCILEM45aH/443rXcJOd/7vPX29SwAAYIoRcAAAGJEADgAAIxLAAQBgRAI4AACMSAAHAIARCeAAADAiARwAAEYkgAMAwIgEcAAAGJEADgAAIxLAAQBgRAI4AACMSAAHAIARCeAAADAiARwAAEYkgAMAwIh2+QBeVUdV1SeralNVnbDe9QAAwFp26QBeVbsl+aMkP5bkAUl+rqoesL5VAQDA1u3SATzJ4Uk2dfenu/tbSU5L8qR1rgkAALZqVw/gByT53NTny4c2AADYkKq717uGHVZVT0nyuO7+xeHz05Ic3t2/usVxxyY5dvj4PUk+OWqhq9snyRfWu4gNQl9M6IfN9MVm+mIzfbGZvthMX2ymLzbbKH1xr+7ed8vG3dejkp3o8iQHTX0+MMkVWx7U3a9P8vqxippFVZ3X3Yetdx0bgb6Y0A+b6YvN9MVm+mIzfbGZvthMX2y20ftiV5+C8pEkh1TVvavqdkmOSXLmOtcEAABbtUuPgHf3jVV1fJK/T7Jbkjd098XrXBYAAGzVLh3Ak6S7z0py1nrXsQM21JSYdaYvJvTDZvpiM32xmb7YTF9spi820xebbei+2KV/hAkAALuaXX0OOAAA7FIE8DmqqqOq6pNVtamqTlhl/yOr6tqq+tjw+q31qHMMVfWGqrq6qi7ayv6qqpOGvvpEVf3A2DWOZYa+WIr7oqoOqqr3V9WlVXVxVT13lWOW4r6YsS+W5b64Q1WdW1UfH/riJascsyz3xSx9sRT3xYqq2q2qPlpV71pl31LcFyu20RdLc19U1WVVdeHwPc9bZf+GvC92+TngG1VV7Zbkj5L8aCbLJX6kqs7s7ku2OPT/dPdPjF7g+E5J8pokb9zK/h9Lcsjw+qEkrxveF9EpWbsvkuW4L25M8vzuvqCq9kxyflW9Z4v/G1mW+2KWvkiW4764Icmju/urVXXbJB+sqnd394enjlmW+2KWvkiW475Y8dwklybZa5V9y3JfrFirL5Llui8e1d1bW/N7Q94XRsDn5/Akm7r70939rSSnJXnSOte0brr7H5N8aY1DnpTkjT3x4SR3qar9x6luXDP0xVLo7iu7+4Jh+/pM/kGy5ZNsl+K+mLEvlsLwv/VXh4+3HV5b/lhpWe6LWfpiaVTVgUl+PMmfbeWQpbgvkpn6gs025H0hgM/PAUk+N/X58qz+D9QfHv7z4rur6oHjlLYhzdpfy2Kp7ouqOjjJQ5Kcs8Wupbsv1uiLZEnui+E/rX8sydVJ3tPdS3tfzNAXyZLcF0n+V5JfS3LzVvYvzX2RbfdFsjz3RSf5h6o6vyZPPt/ShrwvBPD5qVXathy5uCCTR5R+f5I/TPL2eRe1gc3SX8tiqe6LqtojyVuSPK+7r9ty9yqnLOx9sY2+WJr7ortv6u5DM3m68eFV9aAtDlma+2KGvliK+6KqfiLJ1d19/lqHrdK2cPfFjH2xFPfF4Mju/oFMppocV1UP32L/hrwvBPD5uTzJQVOfD0xyxfQB3X3dyn9eHNYzv21V7TNeiRvKNvtrWSzTfTHMa31Lkr/s7reucsjS3Bfb6otlui9WdPdXknwgyVFb7Fqa+2LF1vpiie6LI5M8saouy2RK56Or6k1bHLMs98U2+2KJ7ot09xXD+9VJ3pbJFOBpG/K+EMDn5yNJDqmqe1fV7ZIck+TM6QOq6rurqobtwzP53+OLo1e6MZyZ5OnDr5WPSHJtd1+53kWth2W5L4bveHKSS7v7VVs5bCnui1n6Yonui32r6i7D9h2T/EiSf97isGW5L7bZF8tyX3T3C7r7wO4+OJN/nr6vu5+6xWFLcV/M0hfLcl9U1Z2GH66nqu6U5LFJtlxhbEPeF1ZBmZPuvrGqjk/y90l2S/KG7r64qn5l2P/HSY5O8pyqujHJN5Ic0wv6ZKSq+uskj0yyT1VdnuRFmfygaKUvzkry+CSbknw9ybPWp9L5m6EvluW+ODLJ05JcOMxxTZIXJrlnsnT3xSx9sSz3xf5JTq3JSlK3SXJ6d79ri//fuSz3xSx9sSz3xaqW9L5Y1ZLeF/sledvw7xq7J/mr7v67XeG+8CRMAAAYkSkoAAAwIgEcAABGJIADAMCIBHAAABiRAA4AACMSwAEWWFXdVFUfq6qLquqdK+tKr3H8oVX1+KnPT6yqE+ZeKMASsQwhwAKrqq929x7D9qlJ/qW7T1zj+GcmOay7jx+pRIClYwQcYHl8KMkByeTpeFX1f6vqo8P79wxP7f3tJD87jJr/bFU9s6peM5xzSlWdNBz/6ao6emi/TVW9tqourqp3VdVZU/teXlWXVNUnqur31+l7A2wonoQJsASGpyk+JpNH3ieTR5o/fHhq748k+Z/d/dNV9VuZGgEfRsSn7Z/kYUnun8kjns9I8lNJDk7y4CR3T3JpkjdU1d2S/GSS+3d3b2v6C8CyEMABFtsdh8fbH5zk/CTvGdrvnMljzg9J0kluO+P13t7dNye5pKr2G9oeluRvhvbPV9X7h/brknwzyZ9V1d8medet/TIAi8AUFIDF9o3uPjTJvZLcLslxQ/tLk7y/ux+U5AlJ7jDj9W6Y2q4t3m+hu29McniStyR5cpK/257CARaVAA6wBLr72iT/Jcl/r6rbZjIC/u/D7mdOHXp9kj238/IfTPLTw1zw/ZI8Mkmqao8kd+7us5I8L8mhO1g+wEIRwAGWRHd/NMnHkxyT5HeTvKyq/inJblOHvT/JA1Z+hDnjpd+S5PIkFyX5kyTnJLk2kyD/rqr6RJKzk/zXnfJFAHZxliEE4Farqj26+6tVtXeSc5Mc2d2fX++6ADYiP8IEYGd417DKye2SvFT4Btg6I+AAADAic8ABAGBEAjgAAIxIAAcAgBEJ4AAAMCIBHAAARiSAAwDAiP5/m/uIcsE5XZEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plotting the distribution of ratings\n",
    "plt.figure(figsize = (12, 8))\n",
    "ax = sns.countplot(x = \"rating\", data = movielens)\n",
    "\n",
    "plt.title(\"Ratings Distribution\")\n",
    "plt.xlabel(\"Ratings\")\n",
    "plt.ylabel(\"Number of Ratings (millions)\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Users With Most Ratings**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "userId\n",
       "414    2698\n",
       "599    2478\n",
       "474    2108\n",
       "448    1864\n",
       "274    1346\n",
       "610    1302\n",
       "68     1260\n",
       "380    1218\n",
       "606    1115\n",
       "288    1055\n",
       "Name: rating, dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# users with most ratings\n",
    "movielens_user_ratings = movielens.groupby(\"userId\")[\"rating\"].count().sort_values(ascending = False)\n",
    "movielens_user_ratings.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Global Average Rating**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The global average rating for this dataset is 3.50\n"
     ]
    }
   ],
   "source": [
    "# gloval average rating\n",
    "movielens_global_rating = movielens['rating'].values.sum() / np.count_nonzero(movielens['rating'].values)\n",
    "print('The global average rating for this dataset is %.2f' % movielens_global_rating)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Most Popular Movies/TV Shows**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "      <th>rating_counts</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Shawshank Redemption, The (1994)</th>\n",
       "      <td>4.429022</td>\n",
       "      <td>317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sunset Blvd. (a.k.a. Sunset Boulevard) (1950)</th>\n",
       "      <td>4.333333</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philadelphia Story, The (1940)</th>\n",
       "      <td>4.310345</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lawrence of Arabia (1962)</th>\n",
       "      <td>4.300000</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hoop Dreams (1994)</th>\n",
       "      <td>4.293103</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Godfather, The (1972)</th>\n",
       "      <td>4.289062</td>\n",
       "      <td>192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fight Club (1999)</th>\n",
       "      <td>4.272936</td>\n",
       "      <td>218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cool Hand Luke (1967)</th>\n",
       "      <td>4.271930</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb (1964)</th>\n",
       "      <td>4.268041</td>\n",
       "      <td>97</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Rear Window (1954)</th>\n",
       "      <td>4.261905</td>\n",
       "      <td>84</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                      rating  rating_counts\n",
       "title                                                                      \n",
       "Shawshank Redemption, The (1994)                    4.429022            317\n",
       "Sunset Blvd. (a.k.a. Sunset Boulevard) (1950)       4.333333             27\n",
       "Philadelphia Story, The (1940)                      4.310345             29\n",
       "Lawrence of Arabia (1962)                           4.300000             45\n",
       "Hoop Dreams (1994)                                  4.293103             29\n",
       "Godfather, The (1972)                               4.289062            192\n",
       "Fight Club (1999)                                   4.272936            218\n",
       "Cool Hand Luke (1967)                               4.271930             57\n",
       "Dr. Strangelove or: How I Learned to Stop Worry...  4.268041             97\n",
       "Rear Window (1954)                                  4.261905             84"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# popular movies/tv shows based on highest average rating\n",
    "movielens_popular_movies = pd.DataFrame(movielens.groupby(\"title\")[\"rating\"].mean())\n",
    "# counting how many users rated the title\n",
    "movielens_popular_movies['rating_counts'] = pd.DataFrame(movielens.groupby('title')['rating'].count())\n",
    "\n",
    "# We will use cutoff m as the 90th percentile, so for a movie to be recommended\n",
    "# it must have more votes than at least 90% of the movies on the list\n",
    "m = movielens_popular_movies['rating_counts'].quantile(0.9)\n",
    "\n",
    "movielens_popular_movies[movielens_popular_movies.rating_counts >= m].sort_values('rating', ascending = False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *TMDB dataset*\n",
    "\n",
    "Lastly the same analysis for the TMDB dataset will be done and here is what the data looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>budget</th>\n",
       "      <th>genres</th>\n",
       "      <th>homepage</th>\n",
       "      <th>id</th>\n",
       "      <th>keywords</th>\n",
       "      <th>original_language</th>\n",
       "      <th>original_title</th>\n",
       "      <th>overview</th>\n",
       "      <th>popularity</th>\n",
       "      <th>...</th>\n",
       "      <th>runtime</th>\n",
       "      <th>spoken_languages</th>\n",
       "      <th>status</th>\n",
       "      <th>tagline</th>\n",
       "      <th>title</th>\n",
       "      <th>rating</th>\n",
       "      <th>rating_counts</th>\n",
       "      <th>cast</th>\n",
       "      <th>crew</th>\n",
       "      <th>director</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>237000000</td>\n",
       "      <td>Action Adventure Fantasy Science Fiction</td>\n",
       "      <td>http://www.avatarmovie.com/</td>\n",
       "      <td>19995</td>\n",
       "      <td>culture clash future space war space colony so...</td>\n",
       "      <td>en</td>\n",
       "      <td>Avatar</td>\n",
       "      <td>In the 22nd century, a paraplegic Marine is di...</td>\n",
       "      <td>150.437577</td>\n",
       "      <td>...</td>\n",
       "      <td>162.0</td>\n",
       "      <td>[{\"iso_639_1\": \"en\", \"name\": \"English\"}, {\"iso...</td>\n",
       "      <td>Released</td>\n",
       "      <td>Enter the World of Pandora.</td>\n",
       "      <td>Avatar</td>\n",
       "      <td>7.2</td>\n",
       "      <td>11800</td>\n",
       "      <td>Sam Worthington Zoe Saldana Sigourney Weaver S...</td>\n",
       "      <td>[{'name': 'Stephen E. Rivkin', 'gender': 0, 'd...</td>\n",
       "      <td>James Cameron</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>300000000</td>\n",
       "      <td>Adventure Fantasy Action</td>\n",
       "      <td>http://disney.go.com/disneypictures/pirates/</td>\n",
       "      <td>285</td>\n",
       "      <td>ocean drug abuse exotic island east india trad...</td>\n",
       "      <td>en</td>\n",
       "      <td>Pirates of the Caribbean: At World's End</td>\n",
       "      <td>Captain Barbossa, long believed to be dead, ha...</td>\n",
       "      <td>139.082615</td>\n",
       "      <td>...</td>\n",
       "      <td>169.0</td>\n",
       "      <td>[{\"iso_639_1\": \"en\", \"name\": \"English\"}]</td>\n",
       "      <td>Released</td>\n",
       "      <td>At the end of the world, the adventure begins.</td>\n",
       "      <td>Pirates of the Caribbean: At World's End</td>\n",
       "      <td>6.9</td>\n",
       "      <td>4500</td>\n",
       "      <td>Johnny Depp Orlando Bloom Keira Knightley Stel...</td>\n",
       "      <td>[{'name': 'Dariusz Wolski', 'gender': 2, 'depa...</td>\n",
       "      <td>Gore Verbinski</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>245000000</td>\n",
       "      <td>Action Adventure Crime</td>\n",
       "      <td>http://www.sonypictures.com/movies/spectre/</td>\n",
       "      <td>206647</td>\n",
       "      <td>spy based on novel secret agent sequel mi6</td>\n",
       "      <td>en</td>\n",
       "      <td>Spectre</td>\n",
       "      <td>A cryptic message from Bond’s past sends him o...</td>\n",
       "      <td>107.376788</td>\n",
       "      <td>...</td>\n",
       "      <td>148.0</td>\n",
       "      <td>[{\"iso_639_1\": \"fr\", \"name\": \"Fran\\u00e7ais\"},...</td>\n",
       "      <td>Released</td>\n",
       "      <td>A Plan No One Escapes</td>\n",
       "      <td>Spectre</td>\n",
       "      <td>6.3</td>\n",
       "      <td>4466</td>\n",
       "      <td>Daniel Craig Christoph Waltz L\\u00e9a Seydoux ...</td>\n",
       "      <td>[{'name': 'Thomas Newman', 'gender': 2, 'depar...</td>\n",
       "      <td>Sam Mendes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>250000000</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>http://www.thedarkknightrises.com/</td>\n",
       "      <td>49026</td>\n",
       "      <td>dc comics crime fighter terrorist secret ident...</td>\n",
       "      <td>en</td>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Following the death of District Attorney Harve...</td>\n",
       "      <td>112.312950</td>\n",
       "      <td>...</td>\n",
       "      <td>165.0</td>\n",
       "      <td>[{\"iso_639_1\": \"en\", \"name\": \"English\"}]</td>\n",
       "      <td>Released</td>\n",
       "      <td>The Legend Ends</td>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>7.6</td>\n",
       "      <td>9106</td>\n",
       "      <td>Christian Bale Michael Caine Gary Oldman Anne ...</td>\n",
       "      <td>[{'name': 'Hans Zimmer', 'gender': 2, 'departm...</td>\n",
       "      <td>Christopher Nolan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>260000000</td>\n",
       "      <td>Action Adventure Science Fiction</td>\n",
       "      <td>http://movies.disney.com/john-carter</td>\n",
       "      <td>49529</td>\n",
       "      <td>based on novel mars medallion space travel pri...</td>\n",
       "      <td>en</td>\n",
       "      <td>John Carter</td>\n",
       "      <td>John Carter is a war-weary, former military ca...</td>\n",
       "      <td>43.926995</td>\n",
       "      <td>...</td>\n",
       "      <td>132.0</td>\n",
       "      <td>[{\"iso_639_1\": \"en\", \"name\": \"English\"}]</td>\n",
       "      <td>Released</td>\n",
       "      <td>Lost in our world, found in another.</td>\n",
       "      <td>John Carter</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2124</td>\n",
       "      <td>Taylor Kitsch Lynn Collins Samantha Morton Wil...</td>\n",
       "      <td>[{'name': 'Andrew Stanton', 'gender': 2, 'depa...</td>\n",
       "      <td>Andrew Stanton</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   index     budget                                    genres  \\\n",
       "0      0  237000000  Action Adventure Fantasy Science Fiction   \n",
       "1      1  300000000                  Adventure Fantasy Action   \n",
       "2      2  245000000                    Action Adventure Crime   \n",
       "3      3  250000000               Action Crime Drama Thriller   \n",
       "4      4  260000000          Action Adventure Science Fiction   \n",
       "\n",
       "                                       homepage      id  \\\n",
       "0                   http://www.avatarmovie.com/   19995   \n",
       "1  http://disney.go.com/disneypictures/pirates/     285   \n",
       "2   http://www.sonypictures.com/movies/spectre/  206647   \n",
       "3            http://www.thedarkknightrises.com/   49026   \n",
       "4          http://movies.disney.com/john-carter   49529   \n",
       "\n",
       "                                            keywords original_language  \\\n",
       "0  culture clash future space war space colony so...                en   \n",
       "1  ocean drug abuse exotic island east india trad...                en   \n",
       "2         spy based on novel secret agent sequel mi6                en   \n",
       "3  dc comics crime fighter terrorist secret ident...                en   \n",
       "4  based on novel mars medallion space travel pri...                en   \n",
       "\n",
       "                             original_title  \\\n",
       "0                                    Avatar   \n",
       "1  Pirates of the Caribbean: At World's End   \n",
       "2                                   Spectre   \n",
       "3                     The Dark Knight Rises   \n",
       "4                               John Carter   \n",
       "\n",
       "                                            overview  popularity  ... runtime  \\\n",
       "0  In the 22nd century, a paraplegic Marine is di...  150.437577  ...   162.0   \n",
       "1  Captain Barbossa, long believed to be dead, ha...  139.082615  ...   169.0   \n",
       "2  A cryptic message from Bond’s past sends him o...  107.376788  ...   148.0   \n",
       "3  Following the death of District Attorney Harve...  112.312950  ...   165.0   \n",
       "4  John Carter is a war-weary, former military ca...   43.926995  ...   132.0   \n",
       "\n",
       "                                    spoken_languages    status  \\\n",
       "0  [{\"iso_639_1\": \"en\", \"name\": \"English\"}, {\"iso...  Released   \n",
       "1           [{\"iso_639_1\": \"en\", \"name\": \"English\"}]  Released   \n",
       "2  [{\"iso_639_1\": \"fr\", \"name\": \"Fran\\u00e7ais\"},...  Released   \n",
       "3           [{\"iso_639_1\": \"en\", \"name\": \"English\"}]  Released   \n",
       "4           [{\"iso_639_1\": \"en\", \"name\": \"English\"}]  Released   \n",
       "\n",
       "                                          tagline  \\\n",
       "0                     Enter the World of Pandora.   \n",
       "1  At the end of the world, the adventure begins.   \n",
       "2                           A Plan No One Escapes   \n",
       "3                                 The Legend Ends   \n",
       "4            Lost in our world, found in another.   \n",
       "\n",
       "                                      title rating rating_counts  \\\n",
       "0                                    Avatar    7.2         11800   \n",
       "1  Pirates of the Caribbean: At World's End    6.9          4500   \n",
       "2                                   Spectre    6.3          4466   \n",
       "3                     The Dark Knight Rises    7.6          9106   \n",
       "4                               John Carter    6.1          2124   \n",
       "\n",
       "                                                cast  \\\n",
       "0  Sam Worthington Zoe Saldana Sigourney Weaver S...   \n",
       "1  Johnny Depp Orlando Bloom Keira Knightley Stel...   \n",
       "2  Daniel Craig Christoph Waltz L\\u00e9a Seydoux ...   \n",
       "3  Christian Bale Michael Caine Gary Oldman Anne ...   \n",
       "4  Taylor Kitsch Lynn Collins Samantha Morton Wil...   \n",
       "\n",
       "                                                crew           director  \n",
       "0  [{'name': 'Stephen E. Rivkin', 'gender': 0, 'd...      James Cameron  \n",
       "1  [{'name': 'Dariusz Wolski', 'gender': 2, 'depa...     Gore Verbinski  \n",
       "2  [{'name': 'Thomas Newman', 'gender': 2, 'depar...         Sam Mendes  \n",
       "3  [{'name': 'Hans Zimmer', 'gender': 2, 'departm...  Christopher Nolan  \n",
       "4  [{'name': 'Andrew Stanton', 'gender': 2, 'depa...     Andrew Stanton  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmdb.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Ratings Distribution**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAAHwCAYAAACYMcj+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjPklEQVR4nO3dfditZV0n/O8v8B3wBZAQUNCYDLEwd4jZY740yWQJVk6UJRoT1QETNj4v6FTaGKPTpE0+pkXigOMLQ2pKhqYR0lPjgBskedMkRdjxttMUtEJefs8f69q23N773mvDfe573zefz3Hcx7rWeZ3XtX7rXseG7z73uc6zujsAAMDK+pbVLgAAANYjQRsAAAYQtAEAYABBGwAABhC0AQBgAEEbAAAGELQBdrKq+r2q+tXVrmNrVfWKqnrLCt7vK1X12On4zKr6jRW89y75OwSYV9bRBlheVV2bZL8kdyX5SpIPJTm5u7+ywLUvTvLvuvv7Rta4QB0fTXJUkjuSdJLPJPnDJL/d3bffg3u9vbsXDuVVdWaSTd39KzvyWtO1L84u8DsE2FFGtAEW8yPdvUeSI5I8KcnLV7ece+Tk7t4zyf5JXpbkuCTnVVWt5ItU1e4reT+AtUrQBtgB3X1Tkj/NLHAnSarq1Kr626q6raquqqrnT+3fkeT3kjx1mkbxpan969MoquoZVbWpql5WVbdU1Y1V9ZK5e+9dVX9cVbdW1cer6jeq6i+nc1VVvz1d9+Wq+mRVHb7Ae/hqd380yfOSPDXJc6f7vaqq3j4dP7Cq3l5VX6iqL02vvV9VnZbk/0jyxuk9vXHq31V1UlV9JrPR8i1t3zb30vtU1Uem39OFVfWYqd/BU9+vB/Sq+mhV/btFfofT85+rqmuq6otVdW5VPWruXFfVL1TVZ6rqH6rqd1f6LxcASxG0AXZAVR2Y5N8kuWau+W8zC58PTfLrSd5eVft399VJfiHJx7p7j+5+2DZu+63TtQckOSHJ71bVw6dzv5vkq1Of46efLX4wydOT/KskD0vyE0m+sOh76e7rkmycat/a8VNNByXZe3of/9Td/zHJ/5fZ6Pge3X3y3DXHJnlKksO28ZIvTPLqJPskuSzJOxaocbu/w6p6VpLXJPm3mY3Wfz7J2Vt1++Ek35Pku6Z+z9neawPcW4I2wGLeV1W3Jbk+yS1JXrnlRHf/YXff0N13d/f/zGxE98gduPcdSf5Td9/R3edlNg/826tqtyQ/luSV3f2P3X1VkrO2um7PJI/P7Ds3V3f3jTv4vm5I8oht1LR3km/r7ru6+5LuvnU793pNd3+xu/9pG+f/pLv/YpoT/h8zG6U+aAfrXcoLk7y1uy+d7v3y6d4Hz/V5bXd/afrLxQWZ+xcJgFEEbYDFHDvNb35GZsF2ny0nqupFVXXZNMXiS0kOnz+/gC90951zz/8xyR5J9k2ye2bhfouvH3f3nyd5Y2aj3jdX1elVtdcOvavZKPoXl2j/H5lNkTm7qm6oqt+sqvtt517XL3p++iLpF5M8atvdF/aozEax5+/9hcze2xY3zR1v+f0CDCVoA+yA7r4wyZlJfitJpnnGf5Dk5CR7T1MbrkiyZQ7wvVnaaXOSO5McONf2DSPA3f2G7n5ykidkNoXk/1r05tNo8pMzmwryDabR9V/v7sOSfG9mUy9etOX0Nm65vff69dqrao/MRtJvyGxqTJI8eK7vt+7AfW9I8pi5ez8ks9H4v9vOdQBDCdoAO+6/JfnXVXVEkodkFgQ3J8n0Rcb5LyTenOTAqrr/jr5Id9+V5L1JXlVVD66qx+dfwm6q6nuq6inTSPNXk/xzZksQLmu61/cneX+Si5Oct0SfZ1bVE6fpK7dmNpVky71vTvLYHX0/SX6oqr5v+l28OslF3X19d2/OLBT/dFXtVlU/m+Rxc9dt73f4ziQvqaojquoBSf7zdO9r70GNACtG0AbYQVMwfFuSX53mTb8uyccyC4RPTPJXc93/PMmVSW6qqr+/By93cmZfSrwps+kc70qyZd3rvTIbTf+HzKZOfCHTSPs2vHGaZ35zZn9ZeE+So7v77iX6fmuSd2cWsq9OcmGSt0/nfifJj08reLxhB97LOzOb2/7FzEbSXzh37ucyG43/Qmaj8/9r7tyyv8PuPj/Jr07v58bMQvpxO1AXwBA2rAFYQ6rqvyT51u4+frudAVhVRrQBdmFV9fiq+s5pzewjM1v+749Wuy4Ats/uXQC7tj0zmy7yqMyWFXxdZnOrAdjFmToCAAADmDoCAAADCNoAADDAup2jvc8++/TBBx+82mUAALCOXXLJJX/f3fsudW7dBu2DDz44GzduXO0yAABYx6rq89s6Z+oIAAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMsPtqFwDAruGdF1232iWwhJ96yqNXuwTgHjKiDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwwL2lV1UFVdUFVXV9WVVXXK1P6qqvq7qrps+vmhuWteXlXXVNWnq+o5c+1PrqrLp3NvqKoaVTcAAKyE3Qfe+84kL+vuS6tqzySXVNVHpnO/3d2/Nd+5qg5LclySJyR5VJI/q6p/1d13JXlzkhOT/O8k5yU5OskHB9YOAAD3yrAR7e6+sbsvnY5vS3J1kgOWueSYJGd39+3d/bkk1yQ5sqr2T7JXd3+suzvJ25IcO6puAABYCTtljnZVHZzkSUkumppOrqpPVtVbq+rhU9sBSa6fu2zT1HbAdLx1OwAA7LKGB+2q2iPJe5K8tLtvzWwayOOSHJHkxiSv29J1ict7mfalXuvEqtpYVRs3b958b0sHAIB7bGjQrqr7ZRay39Hd702S7r65u+/q7ruT/EGSI6fum5IcNHf5gUlumNoPXKL9m3T36d29obs37Lvvviv7ZgAAYAeMXHWkkpyR5Orufv1c+/5z3Z6f5Irp+Nwkx1XVA6rqkCSHJrm4u29McltVHTXd80VJ3j+qbgAAWAkjVx15WpKfSXJ5VV02tb0iyU9W1RGZTf+4NsnPJ0l3X1lV5yS5KrMVS06aVhxJkl9McmaSB2W22ogVRwAA2KUNC9rd/ZdZen71ectcc1qS05Zo35jk8JWrDgAAxrIzJAAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADDAsaFfVQVV1QVVdXVVXVtUpU/sjquojVfWZ6fHhc9e8vKquqapPV9Vz5tqfXFWXT+feUFU1qm4AAFgJI0e070zysu7+jiRHJTmpqg5LcmqS87v70CTnT88znTsuyROSHJ3kTVW123SvNyc5Mcmh08/RA+sGAIB7bVjQ7u4bu/vS6fi2JFcnOSDJMUnOmrqdleTY6fiYJGd39+3d/bkk1yQ5sqr2T7JXd3+suzvJ2+auAQCAXdJOmaNdVQcneVKSi5Ls1903JrMwnuSRU7cDklw/d9mmqe2A6XjrdgAA2GUND9pVtUeS9yR5aXffulzXJdp6mfalXuvEqtpYVRs3b96848UCAMAKGRq0q+p+mYXsd3T3e6fmm6fpIJkeb5naNyU5aO7yA5PcMLUfuET7N+nu07t7Q3dv2HfffVfujQAAwA4auepIJTkjydXd/fq5U+cmOX46Pj7J++faj6uqB1TVIZl96fHiaXrJbVV11HTPF81dAwAAu6TdB977aUl+JsnlVXXZ1PaKJK9Nck5VnZDkuiQvSJLuvrKqzklyVWYrlpzU3XdN1/1ikjOTPCjJB6cfAADYZQ0L2t39l1l6fnWSPHsb15yW5LQl2jcmOXzlqgMAgLHsDAkAAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwgKANAAADCNoAADCAoA0AAAMI2gAAMICgDQAAAwjaAAAwwHaDdlWdUlV71cwZVXVpVf3gzigOAADWqkVGtH+2u29N8oNJ9k3ykiSvHVoVAACscYsE7ZoefyjJf+/uv55rAwAAlrBI0L6kqj6cWdD+06raM8ndY8sCAIC1bfcF+pyQ5Igkn+3uf6yqvTObPgIAAGzDIkH7iOnxsVVfnzHy5aravbvvHFIVAACscYsE7Tcl+e4kn8xsbvbh0/HeVfUL3f3hgfUBAMCatMgc7WuTPKm7N3T3k5M8KckVSX4gyW8OrA0AANasRYL247v7yi1PuvuqzIL3Z8eVBQAAa9siU0c+XVVvTnL29PwnkvxNVT0gyR3DKgMAgDVskRHtFye5JslLk/xyks9ObXckeeagugAAYE3b7oh2d/9TktdNP1v7yopXBAAA68B2g3ZVPS3Jq5I8Zr5/dz92XFkAALC2LTJH+4zMpoxckuSuseUAAMD6sEjQ/nJ3f3B4JQAAsI4sErQvqKr/muS9SW7f0tjdlw6rCgAA1rhFgvZTpscNc22d5FkrXw4AAKwPi6w6Ygk/AADYQdsM2lX109399qr6D0ud7+7XjysLAADWtuVGtB8yPe65xLkeUAsAAKwb2wza3f370+GfdfdfzZ+b1tYGAAC2YZEt2P/fBdsAAIDJcnO0n5rke5Psu9U87b2S7Da6MAAAWMuWm6N9/yR7TH3m52nfmuTHRxYFAABr3XJztC9McmFVndndn9+JNQEAwJq3yIY1/zjtDPmEJA/c0tjdNqwBAIBtWOTLkO9I8qkkhyT59STXJvn4wJoAAGDNWyRo793dZyS5o7sv7O6fTXLU4LoAAGBNW2TqyB3T441V9dwkNyQ5cFxJAACw9i0StH+jqh6a5GWZrZ+9V5KXjiwKAADWuu0G7e7+wHT45STPTOwMCQAA27PchjW7Jfm3SQ5I8qHuvqKqfjjJK5I8KMmTdk6JAACw9iw3on1GkoOSXJzkDVX1+SRPTXJqd79vJ9QGAABr1nJBe0OS7+zuu6vqgUn+Psm3dfdNi9y4qt6a5IeT3NLdh09tr0ryc0k2T91e0d3nTedenuSEJHcl+aXu/tOp/clJzsxsFP28JKd0d+/ImwSAteqdF1232iWwlZ96yqNXuwTWiOWW9/tad9+dJN39z0n+ZtGQPTkzydFLtP92dx8x/WwJ2YclOS6zTXGOTvKmaepKkrw5yYlJDp1+lronAADsUpYb0X58VX1yOq4kj5ueV5Lu7u9c7sbd/RdVdfCCdRyT5Ozuvj3J56rqmiRHVtW1Sfbq7o8lSVW9LcmxST644H0BAGBVLBe0v2PQa55cVS9KsjHJy7r7HzL7wuX/nuuzaWq7Yzreun1JVXViZqPfefSj/bMOAACrZ5tTR7r788v93MPXe3OSxyU5IsmNSV43tddSJSzTvq2aT+/uDd29Yd99972HJQIAwL23yBbsK6a7b+7uu6a533+Q5Mjp1KbMVjjZ4sDMdqDclG/chXJLOwAA7NJ2atCuqv3nnj4/yRXT8blJjquqB1TVIZl96fHi7r4xyW1VdVRVVZIXJXn/zqwZAADuiW0G7ao6f3r8L/fkxlX1riQfS/LtVbWpqk5I8ptVdfn0pcpnJvnlJOnuK5Ock+SqJB9KclJ33zXd6heTvCXJNUn+Nr4ICQDAGrDclyH3r6rvT/K8qjo7W82X7u5Ll7txd//kEs1nLNP/tCSnLdG+Mcnhy70WAADsapYL2r+W5NTM5kW/fqtzneRZo4oCAIC1bptBu7vfneTdVfWr3f3qnVgTAACsecuNaCdJuvvVVfW8JE+fmj7a3R8YWxYAAKxt2111pKpek+SUzL6oeFWSU6Y2AABgG7Y7op3kuUmOmNa+TlWdleQTSV4+sjAAAFjLFl1H+2Fzxw8dUAcAAKwri4xovybJJ6rqgsyW+Ht6jGYDAMCyFvky5Luq6qNJviezoP3/dPdNowsDAIC1bJER7UxboZ87uBYAAFg3Fp2jDQAA7ABBGwAABlg2aFfVt1TVFTurGAAAWC+WDdrT2tl/XVWP3kn1AADAurDIlyH3T3JlVV2c5KtbGrv7ecOqAgCANW6RoP3rw6sAAIB1ZpF1tC+sqsckObS7/6yqHpxkt/GlAQDA2rXdVUeq6ueSvDvJ709NByR538CaAABgzVtkeb+Tkjwtya1J0t2fSfLIkUUBAMBat0jQvr27v7blSVXtnqTHlQQAAGvfIkH7wqp6RZIHVdW/TvKHSf54bFkAALC2LRK0T02yOcnlSX4+yXlJfmVkUQAAsNYtsurI3VV1VpKLMpsy8unuNnUEAACWsd2gXVXPTfJ7Sf42SSU5pKp+vrs/OLo4AABYqxbZsOZ1SZ7Z3dckSVU9LsmfJBG0AQBgGxaZo33LlpA9+WySWwbVAwAA68I2R7Sr6kenwyur6rwk52Q2R/sFST6+E2oDAIA1a7mpIz8yd3xzku+fjjcnefiwigAAYB3YZtDu7pfszEIAAGA9WWTVkUOS/PskB8/37+7njSsLAADWtkVWHXlfkjMy2w3y7qHVAADAOrFI0P7n7n7D8EoAAGAdWSRo/05VvTLJh5PcvqWxuy8dVhUAAKxxiwTtJyb5mSTPyr9MHenpOQAAsIRFgvbzkzy2u782uhgAAFgvFtkZ8q+TPGxwHQAAsK4sMqK9X5JPVdXH841ztC3vBwAA27BI0H7l8CoAAGCd2W7Q7u4Ld0YhAACwniyyM+Rtma0ykiT3T3K/JF/t7r1GFgYAAGvZIiPae84/r6pjkxw5qiAAAFgPFll15Bt09/tiDW0AAFjWIlNHfnTu6bck2ZB/mUoCAAAsYZFVR35k7vjOJNcmOWZINQAAsE4sMkf7JTujEAAAWE+2GbSr6teWua67+9UD6gEAgHVhuRHtry7R9pAkJyTZO4mgDQAA27DNoN3dr9tyXFV7JjklyUuSnJ3kddu6DgAA2M4c7ap6RJL/kOSFSc5K8t3d/Q87ozAAAFjLlpuj/V+T/GiS05M8sbu/stOqAgCANW65DWteluRRSX4lyQ1Vdev0c1tV3bpzygMAgLVpuTnaO7xrJAAAMCNMAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAgjYAAAwgaAMAwACCNgAADCBoAwDAAII2AAAMIGgDAMAAw4J2Vb21qm6pqivm2h5RVR+pqs9Mjw+fO/fyqrqmqj5dVc+Za39yVV0+nXtDVdWomgEAYKWMHNE+M8nRW7WdmuT87j40yfnT81TVYUmOS/KE6Zo3VdVu0zVvTnJikkOnn63vCQAAu5xhQbu7/yLJF7dqPibJWdPxWUmOnWs/u7tv7+7PJbkmyZFVtX+Svbr7Y93dSd42dw0AAOyydvYc7f26+8YkmR4fObUfkOT6uX6bprYDpuOt2wEAYJe2q3wZcql5171M+9I3qTqxqjZW1cbNmzevWHEAALCjdnbQvnmaDpLp8ZapfVOSg+b6HZjkhqn9wCXal9Tdp3f3hu7esO+++65o4QAAsCN2dtA+N8nx0/HxSd4/135cVT2gqg7J7EuPF0/TS26rqqOm1UZeNHcNAADssnYfdeOqeleSZyTZp6o2JXllktcmOaeqTkhyXZIXJEl3X1lV5yS5KsmdSU7q7rumW/1iZiuYPCjJB6cfAADYpQ0L2t39k9s49ext9D8tyWlLtG9McvgKlgYAAMPtKl+GBACAdUXQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGGBVgnZVXVtVl1fVZVW1cWp7RFV9pKo+Mz0+fK7/y6vqmqr6dFU9ZzVqBgCAHbGaI9rP7O4junvD9PzUJOd396FJzp+ep6oOS3JckickOTrJm6pqt9UoGAAAFrUrTR05JslZ0/FZSY6daz+7u2/v7s8luSbJkTu/PAAAWNxqBe1O8uGquqSqTpza9uvuG5Nkenzk1H5Akuvnrt00tX2TqjqxqjZW1cbNmzcPKh0AALZv91V63ad19w1V9cgkH6mqTy3Tt5Zo66U6dvfpSU5Pkg0bNizZBwAAdoZVGdHu7humx1uS/FFmU0Furqr9k2R6vGXqvinJQXOXH5jkhp1XLQAA7LidHrSr6iFVteeW4yQ/mOSKJOcmOX7qdnyS90/H5yY5rqoeUFWHJDk0ycU7t2oAANgxqzF1ZL8kf1RVW17/nd39oar6eJJzquqEJNcleUGSdPeVVXVOkquS3JnkpO6+axXqBgCAhe30oN3dn03yXUu0fyHJs7dxzWlJThtcGgAArJhdaXk/AABYNwRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYYDV2hgTIOy+6brVLYCsXf+6LOfKQR6x2GQDrhhFtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABhg99UuYL1550XXrXYJLOGnnvLo1S4BALiPEbS5T/AXIABgZzN1BAAABhC0AQBgAEEbAAAGELQBAGCANRO0q+roqvp0VV1TVaeudj0AALCcNRG0q2q3JL+b5N8kOSzJT1bVYatbFQAAbNtaWd7vyCTXdPdnk6Sqzk5yTJKrVrUqAOA+x5Kxu6Zdcc+MNTGineSAJNfPPd80tQEAwC5prYxo1xJt/U2dqk5McuL09CtV9emhVS1tnyR/vwqvy87lc75v8Dmvfz7j+waf833AC1fvc37Mtk6slaC9KclBc88PTHLD1p26+/Qkp++sopZSVRu7e8Nq1sB4Puf7Bp/z+uczvm/wOd837Iqf81qZOvLxJIdW1SFVdf8kxyU5d5VrAgCAbVoTI9rdfWdVnZzkT5PsluSt3X3lKpcFAADbtCaCdpJ093lJzlvtOhawqlNX2Gl8zvcNPuf1z2d83+Bzvm/Y5T7n6v6m7xQCAAD30lqZow0AAGuKoL2CbBO/vlXVQVV1QVVdXVVXVtUpq10T41TVblX1iar6wGrXwhhV9bCqendVfWr6c/3U1a6JlVdVvzz9N/uKqnpXVT1wtWvi3quqt1bVLVV1xVzbI6rqI1X1menx4atZYyJorxjbxN8n3JnkZd39HUmOSnKSz3hdOyXJ1atdBEP9TpIPdffjk3xXfN7rTlUdkOSXkmzo7sMzW1DhuNWtihVyZpKjt2o7Ncn53X1okvOn56tK0F45X98mvru/lmTLNvGsE919Y3dfOh3fltn/lO1Qug5V1YFJnpvkLatdC2NU1V5Jnp7kjCTp7q9195dWtShG2T3Jg6pq9yQPzhL7cLD2dPdfJPniVs3HJDlrOj4rybE7s6alCNorxzbx9yFVdXCSJyW5aJVLYYz/luT/TnL3KtfBOI9NsjnJf5+mCL2lqh6y2kWxsrr775L8VpLrktyY5Mvd/eHVrYqB9uvuG5PZ4FiSR65yPYL2Clpom3jWvqraI8l7kry0u29d7XpYWVX1w0lu6e5LVrsWhto9yXcneXN3PynJV7ML/DMzK2uao3tMkkOSPCrJQ6rqp1e3Ku5LBO2Vs9A28axtVXW/zEL2O7r7vatdD0M8LcnzqurazKaAPauq3r66JTHApiSbunvLv0q9O7PgzfryA0k+192bu/uOJO9N8r2rXBPj3FxV+yfJ9HjLKtcjaK8g28Svc1VVmc3nvLq7X7/a9TBGd7+8uw/s7oMz+3P8591tBGyd6e6bklxfVd8+NT07yVWrWBJjXJfkqKp68PTf8GfHl17Xs3OTHD8dH5/k/atYS5I1tDPkrs428fcJT0vyM0kur6rLprZXTLuWAmvPv0/yjmlw5LNJXrLK9bDCuvuiqnp3kkszWznqE9kFdw9kx1XVu5I8I8k+VbUpySuTvDbJOVV1QmZ/yXrB6lU4Y2dIAAAYwNQRAAAYQNAGAIABBG0AABhA0AYAgAEEbQAAGEDQBlgHququqrqsqq6oqj+uqodtp/8RVfVDc8+fV1V2RgRYQZb3A1gHquor3b3HdHxWkr/p7tOW6f/iJBu6++SdVCLAfY4RbYD152NJDkiSqjqyqv5XVX1ievz2aYOW/5TkJ6ZR8J+oqhdX1Runa86sqjdM/T9bVT8+tX9LVb2pqq6sqg9U1Xlz515bVVdV1Ser6rdW6X0D7FLsDAmwjlTVbpltM33G1PSpJE+fdq/9gST/ubt/rKp+LXMj2tMI97z9k3xfksdntq3xu5P8aJKDkzwxySMz28r6rVX1iCTPT/L47u7tTVsBuK8QtAHWhwdV1WWZBeFLknxkan9okrOq6tAkneR+C97vfd19d5Krqmq/qe37kvzh1H5TVV0wtd+a5J+TvKWq/iTJB+7tmwFYD0wdAVgf/qm7j0jymCT3T3LS1P7qJBd09+FJfiTJAxe83+1zx7XV4zfo7juTHJnkPUmOTfKhHSkcYL0StAHWke7+cpJfSvJ/VtX9MhvR/rvp9Ivnut6WZM8dvP1fJvmxaa72fkmekSRVtUeSh3b3eUlemuSIe1g+wLoiaAOsM939iSR/neS4JL+Z5DVV9VdJdpvrdkGSw7Z8GXLBW78nyaYkVyT5/SQXJflyZoH9A1X1ySQXJvnlFXkjAGuc5f0AWFhV7dHdX6mqvZNcnORp3X3TatcFsCvyZUgAdsQHplVF7p/k1UI2wLYZ0QYAgAHM0QYAgAEEbQAAGEDQBgCAAQRtAAAYQNAGAIABBG0AABjg/wfelRtdrHu4nwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plotting the distribution of ratings\n",
    "plt.figure(figsize = (12, 8))\n",
    "ax = sns.distplot(tmdb[\"rating\"], bins = 6, kde = False)\n",
    "\n",
    "plt.title(\"Ratings Distribution\")\n",
    "plt.xlabel(\"Ratings\")\n",
    "plt.ylabel(\"Number of Ratings\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Movies With Most Ratings**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "title\n",
       "Inception                  13752\n",
       "The Dark Knight            12002\n",
       "Avatar                     11800\n",
       "The Avengers               11776\n",
       "Deadpool                   10995\n",
       "Interstellar               10867\n",
       "Django Unchained           10099\n",
       "Guardians of the Galaxy     9742\n",
       "The Hunger Games            9455\n",
       "Mad Max: Fury Road          9427\n",
       "Name: rating_counts, dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# users with most ratings\n",
    "tmdb_movie_ratings = tmdb.groupby(\"title\")[\"rating_counts\"].sum().sort_values(ascending = False)\n",
    "tmdb_movie_ratings.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Global Average Rating**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The global average rating for this dataset is 6.17\n"
     ]
    }
   ],
   "source": [
    "# gloval average rating\n",
    "tmdb_global_rating = tmdb['rating'].values.sum() / np.count_nonzero(tmdb['rating'].values)\n",
    "print('The global average rating for this dataset is %.2f' % tmdb_global_rating)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Most Popular Movies/TV Shows**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>popularity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>546</th>\n",
       "      <td>Minions</td>\n",
       "      <td>875.581305</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>Interstellar</td>\n",
       "      <td>724.247784</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>788</th>\n",
       "      <td>Deadpool</td>\n",
       "      <td>514.569956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>Guardians of the Galaxy</td>\n",
       "      <td>481.098624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>127</th>\n",
       "      <td>Mad Max: Fury Road</td>\n",
       "      <td>434.278564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Jurassic World</td>\n",
       "      <td>418.708552</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>Pirates of the Caribbean: The Curse of the Bla...</td>\n",
       "      <td>271.972889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>Dawn of the Planet of the Apes</td>\n",
       "      <td>243.791743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200</th>\n",
       "      <td>The Hunger Games: Mockingjay - Part 1</td>\n",
       "      <td>206.227151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>Big Hero 6</td>\n",
       "      <td>203.734590</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 title  popularity\n",
       "546                                            Minions  875.581305\n",
       "95                                        Interstellar  724.247784\n",
       "788                                           Deadpool  514.569956\n",
       "94                             Guardians of the Galaxy  481.098624\n",
       "127                                 Mad Max: Fury Road  434.278564\n",
       "28                                      Jurassic World  418.708552\n",
       "199  Pirates of the Caribbean: The Curse of the Bla...  271.972889\n",
       "82                      Dawn of the Planet of the Apes  243.791743\n",
       "200              The Hunger Games: Mockingjay - Part 1  206.227151\n",
       "88                                          Big Hero 6  203.734590"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# movies/tv shows with highest popularity score\n",
    "tmdb_popular_movies = tmdb[['title', 'popularity']].sort_values('popularity', ascending = False)\n",
    "tmdb_popular_movies.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *Recommending Movies/TV Shows*\n",
    "\n",
    "After analyzing and cleaning the data a little bit, we can proceed to recommending movies/tv shows to users using those 5 recommender systems. However, for recommender systems such as **Correlation Recommender, Content-Based Recommender, Collaborative Filtering and Hybrid Recommender**, we would have to reduce our data quite bit since the dataset is extremely huge. The reasoning is because it requires a lot of memory and it gives out error whenever we run on the full dataset. So, the Netflix dataset will be reduced to **0.5%** of its original for testing purposes.\n",
    "\n",
    "**Objective**\n",
    "\n",
    "It is simply to learn from the user's data and recommend the best movies/tv shows based on their preferences, taste, or pattern."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *Simple Recommender*\n",
    "\n",
    "This recommender offers a generalized recommendations to all the users, basically movies/tv shows that are extremely popular among users and will have a high probability of being liked by everyone. It will be using the IMDb's weighted rating formula to recommend.\n",
    "\n",
    "**weighted rating (WR)** = (v/(v+m))R+(m/(v+m))C\n",
    "\n",
    "**R** is the average rating of the movie/tv show\n",
    "\n",
    "**v** is the number of votes for the movie/tv show\n",
    "\n",
    "**m** is the minimum votes required to be listed in the suggestion\n",
    "\n",
    "**C** is the mean vote across the whole data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Simple Recommender ###\n",
    "\n",
    "def simple_rec(dataset):\n",
    "    '''\n",
    "    A function that offers a generalized recommendations to all the users,\n",
    "    basically movies/tv shows that are extremely popular among users and\n",
    "    will have a high probability of being liked by everyone\n",
    "    '''\n",
    "    \n",
    "    # fiding the mean of our rating by title\n",
    "    ratings_mean = pd.DataFrame(dataset.groupby('title')['rating'].mean())\n",
    "    \n",
    "    # counting how many users rated the title\n",
    "    ratings_mean['rating_counts'] = pd.DataFrame(dataset.groupby('title')['rating'].count())\n",
    "\n",
    "    # if dataset 3 is used then we use the existing rating_counts column\n",
    "    if dataset is tmdb:\n",
    "        ratings_mean['rating_counts'] = pd.DataFrame(tmdb.groupby('title')['rating_counts'].sum())\n",
    "\n",
    "    C = ratings_mean['rating'].mean()\n",
    "    \n",
    "    # We will use cutoff m as the 90th percentile, so for a movie to be recommended\n",
    "    # it must have more votes than at least 90% of the movies on the list\n",
    "    m = ratings_mean['rating_counts'].quantile(0.9)\n",
    "    \n",
    "    # df with the required minimum vote\n",
    "    weighted_ratings_mean = ratings_mean.copy().loc[ratings_mean['rating_counts'] >= m]\n",
    "    \n",
    "    \n",
    "    # R is the average rating of the movie/tv show\n",
    "    # v is the number of votes for the movie/tv show\n",
    "    # m is the minimum votes required to be listed in the suggestion\n",
    "    # C is the mean vote across the whole data\n",
    "    \n",
    "\n",
    "    # Funtion that computes the weighted rating (based on IMDb's function/formula)\n",
    "    def weighted_rating(x, m = m, C = C):\n",
    "        v = x['rating_counts']\n",
    "        R = x['rating']\n",
    "        \n",
    "        # returning the formula\n",
    "        return (v / (v + m) * R) + (m / (m + v) * C)\n",
    "    \n",
    "    # new column score for our weighted rating\n",
    "    weighted_ratings_mean['score'] = weighted_ratings_mean.apply(weighted_rating, axis = 1)\n",
    "    \n",
    "    # sorting our score by descending order\n",
    "    weighted_ratings_mean = weighted_ratings_mean.sort_values('score', ascending = False)\n",
    "    \n",
    "    # returning the weighted ratings df\n",
    "    return weighted_ratings_mean.head(30)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the generalized recommendations to the users based on the score:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Netflix**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "      <th>rating_counts</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Lord of the Rings: The Fellowship of the Ring</th>\n",
       "      <td>4.434708</td>\n",
       "      <td>149866</td>\n",
       "      <td>4.343890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Finding Nemo (Widescreen)</th>\n",
       "      <td>4.415523</td>\n",
       "      <td>140979</td>\n",
       "      <td>4.320952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Sixth Sense</th>\n",
       "      <td>4.325245</td>\n",
       "      <td>151292</td>\n",
       "      <td>4.243338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Godfather</th>\n",
       "      <td>4.404083</td>\n",
       "      <td>71614</td>\n",
       "      <td>4.232825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Silence of the Lambs</th>\n",
       "      <td>4.311687</td>\n",
       "      <td>128446</td>\n",
       "      <td>4.217637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Braveheart</th>\n",
       "      <td>4.294423</td>\n",
       "      <td>135601</td>\n",
       "      <td>4.206344</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pirates of the Caribbean: The Curse of the Black Pearl</th>\n",
       "      <td>4.153908</td>\n",
       "      <td>193941</td>\n",
       "      <td>4.099031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ray</th>\n",
       "      <td>4.186012</td>\n",
       "      <td>108606</td>\n",
       "      <td>4.089122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shrek 2</th>\n",
       "      <td>4.144543</td>\n",
       "      <td>153996</td>\n",
       "      <td>4.077153</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Batman Begins</th>\n",
       "      <td>4.236699</td>\n",
       "      <td>54922</td>\n",
       "      <td>4.053062</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                      rating  rating_counts  \\\n",
       "title                                                                         \n",
       "Lord of the Rings: The Fellowship of the Ring       4.434708         149866   \n",
       "Finding Nemo (Widescreen)                           4.415523         140979   \n",
       "The Sixth Sense                                     4.325245         151292   \n",
       "The Godfather                                       4.404083          71614   \n",
       "The Silence of the Lambs                            4.311687         128446   \n",
       "Braveheart                                          4.294423         135601   \n",
       "Pirates of the Caribbean: The Curse of the Blac...  4.153908         193941   \n",
       "Ray                                                 4.186012         108606   \n",
       "Shrek 2                                             4.144543         153996   \n",
       "Batman Begins                                       4.236699          54922   \n",
       "\n",
       "                                                       score  \n",
       "title                                                         \n",
       "Lord of the Rings: The Fellowship of the Ring       4.343890  \n",
       "Finding Nemo (Widescreen)                           4.320952  \n",
       "The Sixth Sense                                     4.243338  \n",
       "The Godfather                                       4.232825  \n",
       "The Silence of the Lambs                            4.217637  \n",
       "Braveheart                                          4.206344  \n",
       "Pirates of the Caribbean: The Curse of the Blac...  4.099031  \n",
       "Ray                                                 4.089122  \n",
       "Shrek 2                                             4.077153  \n",
       "Batman Begins                                       4.053062  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# simple recommenders for the datasets\n",
    "netflix_weighted = simple_rec(netflix)\n",
    "netflix_weighted.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**MovieLens**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "      <th>rating_counts</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Shawshank Redemption, The (1994)</th>\n",
       "      <td>4.429022</td>\n",
       "      <td>317</td>\n",
       "      <td>4.337455</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Godfather, The (1972)</th>\n",
       "      <td>4.289062</td>\n",
       "      <td>192</td>\n",
       "      <td>4.162486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fight Club (1999)</th>\n",
       "      <td>4.272936</td>\n",
       "      <td>218</td>\n",
       "      <td>4.161569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Star Wars: Episode IV - A New Hope (1977)</th>\n",
       "      <td>4.231076</td>\n",
       "      <td>251</td>\n",
       "      <td>4.136995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Usual Suspects, The (1995)</th>\n",
       "      <td>4.237745</td>\n",
       "      <td>204</td>\n",
       "      <td>4.123742</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pulp Fiction (1994)</th>\n",
       "      <td>4.197068</td>\n",
       "      <td>307</td>\n",
       "      <td>4.121510</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Schindler's List (1993)</th>\n",
       "      <td>4.225000</td>\n",
       "      <td>220</td>\n",
       "      <td>4.119775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Matrix, The (1999)</th>\n",
       "      <td>4.192446</td>\n",
       "      <td>278</td>\n",
       "      <td>4.110113</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Star Wars: Episode V - The Empire Strikes Back (1980)</th>\n",
       "      <td>4.215640</td>\n",
       "      <td>211</td>\n",
       "      <td>4.107498</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Forrest Gump (1994)</th>\n",
       "      <td>4.164134</td>\n",
       "      <td>329</td>\n",
       "      <td>4.095743</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                      rating  rating_counts  \\\n",
       "title                                                                         \n",
       "Shawshank Redemption, The (1994)                    4.429022            317   \n",
       "Godfather, The (1972)                               4.289062            192   \n",
       "Fight Club (1999)                                   4.272936            218   \n",
       "Star Wars: Episode IV - A New Hope (1977)           4.231076            251   \n",
       "Usual Suspects, The (1995)                          4.237745            204   \n",
       "Pulp Fiction (1994)                                 4.197068            307   \n",
       "Schindler's List (1993)                             4.225000            220   \n",
       "Matrix, The (1999)                                  4.192446            278   \n",
       "Star Wars: Episode V - The Empire Strikes Back ...  4.215640            211   \n",
       "Forrest Gump (1994)                                 4.164134            329   \n",
       "\n",
       "                                                       score  \n",
       "title                                                         \n",
       "Shawshank Redemption, The (1994)                    4.337455  \n",
       "Godfather, The (1972)                               4.162486  \n",
       "Fight Club (1999)                                   4.161569  \n",
       "Star Wars: Episode IV - A New Hope (1977)           4.136995  \n",
       "Usual Suspects, The (1995)                          4.123742  \n",
       "Pulp Fiction (1994)                                 4.121510  \n",
       "Schindler's List (1993)                             4.119775  \n",
       "Matrix, The (1999)                                  4.110113  \n",
       "Star Wars: Episode V - The Empire Strikes Back ...  4.107498  \n",
       "Forrest Gump (1994)                                 4.095743  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movielens_weighted = simple_rec(movielens)\n",
    "movielens_weighted.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TMDB**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "      <th>rating_counts</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>The Shawshank Redemption</th>\n",
       "      <td>8.5</td>\n",
       "      <td>8205</td>\n",
       "      <td>8.056605</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fight Club</th>\n",
       "      <td>8.3</td>\n",
       "      <td>9413</td>\n",
       "      <td>7.937031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Dark Knight</th>\n",
       "      <td>8.2</td>\n",
       "      <td>12002</td>\n",
       "      <td>7.918229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pulp Fiction</th>\n",
       "      <td>8.3</td>\n",
       "      <td>8428</td>\n",
       "      <td>7.902252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Inception</th>\n",
       "      <td>8.1</td>\n",
       "      <td>13752</td>\n",
       "      <td>7.861698</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Godfather</th>\n",
       "      <td>8.4</td>\n",
       "      <td>5893</td>\n",
       "      <td>7.848152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Interstellar</th>\n",
       "      <td>8.1</td>\n",
       "      <td>10867</td>\n",
       "      <td>7.807644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Forrest Gump</th>\n",
       "      <td>8.2</td>\n",
       "      <td>7927</td>\n",
       "      <td>7.800810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Lord of the Rings: The Return of the King</th>\n",
       "      <td>8.1</td>\n",
       "      <td>8064</td>\n",
       "      <td>7.725001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Empire Strikes Back</th>\n",
       "      <td>8.2</td>\n",
       "      <td>5879</td>\n",
       "      <td>7.695059</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               rating  rating_counts     score\n",
       "title                                                                         \n",
       "The Shawshank Redemption                          8.5           8205  8.056605\n",
       "Fight Club                                        8.3           9413  7.937031\n",
       "The Dark Knight                                   8.2          12002  7.918229\n",
       "Pulp Fiction                                      8.3           8428  7.902252\n",
       "Inception                                         8.1          13752  7.861698\n",
       "The Godfather                                     8.4           5893  7.848152\n",
       "Interstellar                                      8.1          10867  7.807644\n",
       "Forrest Gump                                      8.2           7927  7.800810\n",
       "The Lord of the Rings: The Return of the King     8.1           8064  7.725001\n",
       "The Empire Strikes Back                           8.2           5879  7.695059"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "tmdb_weighted = simple_rec(tmdb)\n",
    "tmdb_weighted.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *Correlation Recommender*\n",
    "\n",
    "This recommender offers recommendations to the users based on the correlation between the ratings of the movies/tv shows, in order to find the similarity between them. Since the TMDB doesn't have individual ratings of users and only average ratings, we will be testing this recommender on Netflix and MovieLens dataset only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Correlation Recommender ###\n",
    "\n",
    "\n",
    "def corr_rec(dataset, title):\n",
    "    '''\n",
    "    A function that offers recommendations to the users based on the\n",
    "    correlation between the ratings of the movies/tv shows\n",
    "    '''    \n",
    "    \n",
    "    # pivot table by userId with movies as columns and rating as values\n",
    "    movie_rating_pivot = dataset.pivot_table(index='userId', columns='title', \n",
    "                                              values='rating')\n",
    "    \n",
    "    # specific movie and the user's rating for it\n",
    "    user_rating = movie_rating_pivot[title]\n",
    "    \n",
    "    # correlation between other movies\n",
    "    similar_movies_corr = movie_rating_pivot.corrwith(user_rating)\n",
    "    \n",
    "    # creating a dataframe from the correlations and dropping NA values\n",
    "    similar_movies = pd.DataFrame(similar_movies_corr, columns=['correlation'])\n",
    "    similar_movies.dropna(inplace=True)\n",
    "    \n",
    "    # fiding the mean of our rating by title\n",
    "    ratings_mean = pd.DataFrame(dataset.groupby('title')['rating'].mean())\n",
    "    \n",
    "    # counting how many users rated the title\n",
    "    ratings_mean['rating_counts'] = pd.DataFrame(dataset.groupby('title')['rating'].count())\n",
    "    \n",
    "    # We will use cutoff m as the 90th percentile, so for a movie to be recommended\n",
    "    # it must have more votes than at least 90% of the movies on the list\n",
    "    m = ratings_mean['rating_counts'].quantile(0.9)\n",
    "    \n",
    "    # adding rating_counts column\n",
    "    similar_movies = similar_movies.join(ratings_mean['rating_counts'])\n",
    "    \n",
    "    # filtering the similar movies based on the min requirement of rating_counts\n",
    "    similar_movies = similar_movies[similar_movies['rating_counts'] \n",
    "                                    > m].sort_values('correlation', ascending=False)\n",
    "    \n",
    "    # returning the similar movies df\n",
    "    return similar_movies.head(30)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Netflix**\n",
    "\n",
    "We will be trying to suggest a user a list of movies/tv shows using the Correlation Recommender for **Lord of the Rings: The Fellowship of the Ring**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>correlation</th>\n",
       "      <th>rating_counts</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Lord of the Rings: The Fellowship of the Ring</th>\n",
       "      <td>1.0</td>\n",
       "      <td>782</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Lilo and Stitch</th>\n",
       "      <td>1.0</td>\n",
       "      <td>203</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cocoon</th>\n",
       "      <td>1.0</td>\n",
       "      <td>156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Finding Neverland</th>\n",
       "      <td>1.0</td>\n",
       "      <td>569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>From Hell</th>\n",
       "      <td>1.0</td>\n",
       "      <td>151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fried Green Tomatoes</th>\n",
       "      <td>1.0</td>\n",
       "      <td>416</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Love Actually</th>\n",
       "      <td>1.0</td>\n",
       "      <td>489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Rat Race</th>\n",
       "      <td>1.0</td>\n",
       "      <td>103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jaws</th>\n",
       "      <td>1.0</td>\n",
       "      <td>378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Garden State</th>\n",
       "      <td>1.0</td>\n",
       "      <td>512</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               correlation  rating_counts\n",
       "title                                                                    \n",
       "Lord of the Rings: The Fellowship of the Ring          1.0            782\n",
       "Lilo and Stitch                                        1.0            203\n",
       "Cocoon                                                 1.0            156\n",
       "Finding Neverland                                      1.0            569\n",
       "From Hell                                              1.0            151\n",
       "Fried Green Tomatoes                                   1.0            416\n",
       "Love Actually                                          1.0            489\n",
       "Rat Race                                               1.0            103\n",
       "Jaws                                                   1.0            378\n",
       "Garden State                                           1.0            512"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# correlation recommenders for datasets\n",
    "netflix_corr = corr_rec(netflix_test, 'Lord of the Rings: The Fellowship of the Ring')\n",
    "netflix_corr.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**MovieLens**\n",
    "\n",
    "We will be trying to suggest a user a list of movies/tv shows using the Correlation Recommender for **Forrest Gump**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>correlation</th>\n",
       "      <th>rating_counts</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Forrest Gump (1994)</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bio-Dome (1996)</th>\n",
       "      <td>0.723238</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fatal Attraction (1987)</th>\n",
       "      <td>0.701856</td>\n",
       "      <td>36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X-Men: Days of Future Past (2014)</th>\n",
       "      <td>0.682284</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mr. Holland's Opus (1995)</th>\n",
       "      <td>0.652144</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Waterboy, The (1998)</th>\n",
       "      <td>0.632631</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grumpy Old Men (1993)</th>\n",
       "      <td>0.630896</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>U-571 (2000)</th>\n",
       "      <td>0.622004</td>\n",
       "      <td>33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Powder (1995)</th>\n",
       "      <td>0.601555</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pretty in Pink (1986)</th>\n",
       "      <td>0.600267</td>\n",
       "      <td>33</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   correlation  rating_counts\n",
       "title                                                        \n",
       "Forrest Gump (1994)                   1.000000            329\n",
       "Bio-Dome (1996)                       0.723238             31\n",
       "Fatal Attraction (1987)               0.701856             36\n",
       "X-Men: Days of Future Past (2014)     0.682284             30\n",
       "Mr. Holland's Opus (1995)             0.652144             80\n",
       "Waterboy, The (1998)                  0.632631             28\n",
       "Grumpy Old Men (1993)                 0.630896             29\n",
       "U-571 (2000)                          0.622004             33\n",
       "Powder (1995)                         0.601555             28\n",
       "Pretty in Pink (1986)                 0.600267             33"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movielens_corr = corr_rec(movielens, 'Forrest Gump (1994)')\n",
    "movielens_corr.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *Content-Based Recommender*\n",
    "\n",
    "This recommender offers recommendations based on the content, in other words, it looks for similar features like genres, cast, keywords, director or all combined. In our case, we will be only focusing on one feature only, for MovieLens it will be **genres** and for TMDB it will be **keywords**. Since Netflix dataset doesn't have any features we cannot run this recommender."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Content-Based Recommender ###\n",
    "\n",
    "def content_rec(dataset, title, feature):\n",
    "    '''\n",
    "    A function that offers recommendations based on the content, in other words,\n",
    "    it looks for similar features like genres, cast, keywords or director (you select).\n",
    "    '''       \n",
    "\n",
    "    # removing duplicate titles from the dataset\n",
    "    dup_removal = dataset.drop_duplicates(subset=['title'])\n",
    "    # resetting the indices\n",
    "    dup_removal = dup_removal.reset_index()\n",
    "    \n",
    "    # filling NA values with blanks in our feature\n",
    "    dup_removal[feature] = dup_removal[feature].fillna('')\n",
    "    \n",
    "    # Panda Series for the titles and their indices\n",
    "    title_indices = pd.Series(dup_removal.index, index=dup_removal['title'])\n",
    "    \n",
    "    # importing CountVectorizer and creating the count matrix\n",
    "    cv = CountVectorizer()\n",
    "    # content based recommender based on 'genres'\n",
    "    count_matrix = cv.fit_transform(dup_removal[feature])\n",
    "    # cosine similarity for the distance between them\n",
    "    cosine_sim = cosine_similarity(count_matrix)\n",
    "    \n",
    "    # index of the movie/tv show that we select\n",
    "    index = title_indices[title]\n",
    "    \n",
    "    # similarity scores of all the movies/tv shows\n",
    "    sim_scores = list(enumerate(cosine_sim[index]))\n",
    "    \n",
    "    # removing the movie/tv show that we selected to avoid duplicates\n",
    "    sim_scores.pop(index)\n",
    "    \n",
    "    # sorting the scores by descending order\n",
    "    sim_scores = sorted(sim_scores, key=lambda x: x[1], reverse=True)\n",
    "    \n",
    "    \n",
    "    # list of indices of those movies/tv shows\n",
    "    movie_indices = [i[0] for i in sim_scores]\n",
    "    \n",
    "    # list of indices of those  movies/tv shows\n",
    "    sim_scores_sorted = [i[1] for i in sim_scores]\n",
    "    \n",
    "    # similar movies/tv shows based on cosine similarity score\n",
    "    similar_movies = pd.DataFrame(sim_scores_sorted, \n",
    "                               index=dup_removal['title'].iloc[movie_indices], \n",
    "                               columns = ['content_based'])\n",
    "    \n",
    "    # returning the df\n",
    "    return similar_movies\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**MovieLens**\n",
    "\n",
    "We will be trying to suggest a user a list of movies/tv shows using the Content-Based Recommender for **The Avengers** with the feature **genres**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>content_based</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Star Wars: Episode II - Attack of the Clones (2002)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Spider-Man 2 (2004)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Superman Returns (2006)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Avatar (2009)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Amazing Spider-Man, The (2012)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Oblivion (2013)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Captain America: The Winter Soldier (2014)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Star Trek (2009)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Star Trek Into Darkness (2013)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Hunger Games: Catching Fire (2013)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    content_based\n",
       "title                                                            \n",
       "Star Wars: Episode II - Attack of the Clones (2...            1.0\n",
       "Spider-Man 2 (2004)                                           1.0\n",
       "Superman Returns (2006)                                       1.0\n",
       "Avatar (2009)                                                 1.0\n",
       "Amazing Spider-Man, The (2012)                                1.0\n",
       "Oblivion (2013)                                               1.0\n",
       "Captain America: The Winter Soldier (2014)                    1.0\n",
       "Star Trek (2009)                                              1.0\n",
       "Star Trek Into Darkness (2013)                                1.0\n",
       "The Hunger Games: Catching Fire (2013)                        1.0"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# content based recommenders for datasets\n",
    "movielens_content = content_rec(movielens, 'Avengers, The (2012)', 'genres')\n",
    "movielens_content.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TMDB**\n",
    "\n",
    "We will be trying to suggest a user a list of movies/tv shows using the Content-Based Recommender for **Avatar** with the feature **keywords**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>content_based</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Space Dogs</th>\n",
       "      <td>0.526235</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Gravity</th>\n",
       "      <td>0.507093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jason X</th>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cargo</th>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Space Chimps</th>\n",
       "      <td>0.478091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Star Wars: Clone Wars: Volume 1</th>\n",
       "      <td>0.424264</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Guardians of the Galaxy</th>\n",
       "      <td>0.421637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Planet of the Apes</th>\n",
       "      <td>0.421637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Alien</th>\n",
       "      <td>0.421637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>In the Shadow of the Moon</th>\n",
       "      <td>0.421637</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 content_based\n",
       "title                                         \n",
       "Space Dogs                            0.526235\n",
       "Gravity                               0.507093\n",
       "Jason X                               0.500000\n",
       "Cargo                                 0.500000\n",
       "Space Chimps                          0.478091\n",
       "Star Wars: Clone Wars: Volume 1       0.424264\n",
       "Guardians of the Galaxy               0.421637\n",
       "Planet of the Apes                    0.421637\n",
       "Alien                                 0.421637\n",
       "In the Shadow of the Moon             0.421637"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmdb_content = content_rec(tmdb, 'Avatar', 'keywords')\n",
    "tmdb_content.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *Collaborative Filtering (Model-Based)*\n",
    "\n",
    "This recommender tries to find similarities between users or certain movies that are liked by similar users to recommend other similar movies. There are two types of collaborative techniques which are **Memory-Based** and **Model-Based**. We will be using **Model-Based** where it will try to fill out the matrix of preferences of users for the movies and try to predict what rating they will give to the missing items and suggest them.\n",
    "\n",
    "Since the TMDB doesn't have individual ratings of users and only average ratings, we will be testing this recommender on Netflix and MovieLens dataset only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Collaborative Filtering (Model-Based) ###\n",
    "\n",
    "def collab_fil_rec(dataset, title): \n",
    "    '''\n",
    "    A function that tries to to fill out the matrix of preferences of users for\n",
    "    the movies and try to predict what rating they  will give to the missing \n",
    "    items and suggest them (model based collaborative filtering using Truncated SVD)'\n",
    "    '''\n",
    "\n",
    "    # pivot table by movie titles with userId as columns and rating as values\n",
    "    ratings = dataset.pivot_table(index='title', columns='userId', \n",
    "                                              values='rating').fillna(0)\n",
    "    \n",
    "    # substantiating an SVD object\n",
    "    svd = TruncatedSVD(n_components = 200)\n",
    "    matrix = svd.fit_transform(ratings)\n",
    "    \n",
    "    # converting to dataframe\n",
    "    matrix_df = pd.DataFrame(matrix, index = dataset.title.unique().tolist())\n",
    "    \n",
    "    # finding a specific movie compared to others\n",
    "    movies = np.array(matrix_df.loc[title]).reshape(1, -1)\n",
    "    \n",
    "    # score for the similarities\n",
    "    score = cosine_similarity(matrix_df, movies).reshape(-1)\n",
    "    \n",
    "    # dataframe using the score for all the movies\n",
    "    similar_df = pd.DataFrame(score, index = matrix_df.index, columns = ['collaborative'])\n",
    "    \n",
    "    # sorting by the score\n",
    "    similar_df = similar_df.sort_values('collaborative', ascending=False)\n",
    "    \n",
    "    # returning the dataframe of similar movies\n",
    "    return similar_df\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Netflix**\n",
    "\n",
    "We will be trying to suggest a user a list of movies/tv shows using the Collaborative Filtering for **CSI: Season 1**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>collaborative</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>CSI: Season 1</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Rocky Horror Picture Show</th>\n",
       "      <td>0.624646</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Godzilla Against Mechagodzilla</th>\n",
       "      <td>0.624114</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beatles: From the Beginning to the End</th>\n",
       "      <td>0.623895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ethics and the World Crisis: A Dialogue with the Dalai Lama</th>\n",
       "      <td>0.621932</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Stand</th>\n",
       "      <td>0.597155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Trip</th>\n",
       "      <td>0.597108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Avenging Angelo</th>\n",
       "      <td>0.596112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mac</th>\n",
       "      <td>0.593824</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>The Brady Bunch Variety Hour</th>\n",
       "      <td>0.571164</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    collaborative\n",
       "CSI: Season 1                                            1.000000\n",
       "The Rocky Horror Picture Show                            0.624646\n",
       "Godzilla Against Mechagodzilla                           0.624114\n",
       "Beatles: From the Beginning to the End                   0.623895\n",
       "Ethics and the World Crisis: A Dialogue with th...       0.621932\n",
       "The Stand                                                0.597155\n",
       "The Trip                                                 0.597108\n",
       "Avenging Angelo                                          0.596112\n",
       "Mac                                                      0.593824\n",
       "The Brady Bunch Variety Hour                             0.571164"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# collaborative filtering recommenders for datasets\n",
    "netflix_cf = collab_fil_rec(netflix_test, 'CSI: Season 1')\n",
    "netflix_cf.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**MovieLens**\n",
    "\n",
    "We will be trying to suggest a user a list of movies/tv shows using the Collaborative Filtering for **Star Wars: Episode IV**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>collaborative</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Star Wars: Episode IV - A New Hope (1977)</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enigma (2001)</th>\n",
       "      <td>0.998929</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Naked Gun: From the Files of Police Squad!, The (1988)</th>\n",
       "      <td>0.977265</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Frozen Planet (2011)</th>\n",
       "      <td>0.971852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shining Through (1992)</th>\n",
       "      <td>0.971852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Made (2001)</th>\n",
       "      <td>0.971852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Last Dragon, The (1985)</th>\n",
       "      <td>0.971852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Metropolis (1927)</th>\n",
       "      <td>0.971852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Remember Me (Ricordati di me) (2003)</th>\n",
       "      <td>0.971852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bride &amp; Prejudice (2004)</th>\n",
       "      <td>0.971852</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    collaborative\n",
       "Star Wars: Episode IV - A New Hope (1977)                1.000000\n",
       "Enigma (2001)                                            0.998929\n",
       "Naked Gun: From the Files of Police Squad!, The...       0.977265\n",
       "Frozen Planet (2011)                                     0.971852\n",
       "Shining Through (1992)                                   0.971852\n",
       "Made (2001)                                              0.971852\n",
       "Last Dragon, The (1985)                                  0.971852\n",
       "Metropolis (1927)                                        0.971852\n",
       "Remember Me (Ricordati di me) (2003)                     0.971852\n",
       "Bride & Prejudice (2004)                                 0.971852"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movielens_cf = collab_fil_rec(movielens, 'Star Wars: Episode IV - A New Hope (1977)')\n",
    "movielens_cf.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *Hybrid Recommender*\n",
    "\n",
    "This recommender combines two of the most popular recommender systems which are **Content-Based Recommender** and **Collaborative Filtering** to combine the strengths of both recommender systems. We will be only using MovieLens dataset since it is the only dataset that can be used for both Content-Based Recommender and Collaborative Filtering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Hybrid Recommender ###\n",
    "\n",
    "def hybrid_rec(title):\n",
    "    '''\n",
    "    A function that combines both content-based and collaborative filtering to\n",
    "    combine the strengths of both recommender systems\n",
    "    '''\n",
    "    \n",
    "    # content based\n",
    "    hybrid = content_rec(movielens, title, 'genres')\n",
    "    # collaborative filtering\n",
    "    cf = collab_fil_rec(movielens, title)\n",
    "    \n",
    "    # joining the dataframes\n",
    "    hybrid = hybrid.join(cf, how = 'outer')\n",
    "    \n",
    "    # score for the hybrid recommender\n",
    "    hybrid['hybrid'] = (hybrid['content_based'] + hybrid['collaborative']) / 2\n",
    "    \n",
    "    # sorting the values by hybrid score\n",
    "    hybrid = hybrid.sort_values('hybrid', ascending=False)\n",
    "    \n",
    "    \n",
    "    # returning the df\n",
    "    return hybrid.head(30)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**MovieLens**\n",
    "\n",
    "We will be trying to suggest a user a list of movies/tv shows using the Hybrid Recommender for **Batman Begins**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>content_based</th>\n",
       "      <th>collaborative</th>\n",
       "      <th>hybrid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Close Range (2015)</th>\n",
       "      <td>0.816497</td>\n",
       "      <td>0.426797</td>\n",
       "      <td>0.621647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>From Paris with Love (2010)</th>\n",
       "      <td>0.816497</td>\n",
       "      <td>0.426797</td>\n",
       "      <td>0.621647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bad Company (2002)</th>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.554188</td>\n",
       "      <td>0.610427</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Gone in 60 Seconds (1974)</th>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.502655</td>\n",
       "      <td>0.584661</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Take Aim at the Police Van (Sono gosôsha wo nerae: 'Jûsangô taihisen' yori) (1960)</th>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.499277</td>\n",
       "      <td>0.582972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>State Property 2 (2005)</th>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.499277</td>\n",
       "      <td>0.582972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>All About the Benjamins (2002)</th>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.499277</td>\n",
       "      <td>0.582972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>V. I. Warshawski (1991)</th>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.499277</td>\n",
       "      <td>0.582972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Undisputed III: Redemption (2010)</th>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.499277</td>\n",
       "      <td>0.582972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Last Stand, The (2013)</th>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.499277</td>\n",
       "      <td>0.582972</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    content_based  \\\n",
       "Close Range (2015)                                       0.816497   \n",
       "From Paris with Love (2010)                              0.816497   \n",
       "Bad Company (2002)                                       0.666667   \n",
       "Gone in 60 Seconds (1974)                                0.666667   \n",
       "Take Aim at the Police Van (Sono gosôsha wo ner...       0.666667   \n",
       "State Property 2 (2005)                                  0.666667   \n",
       "All About the Benjamins (2002)                           0.666667   \n",
       "V. I. Warshawski (1991)                                  0.666667   \n",
       "Undisputed III: Redemption (2010)                        0.666667   \n",
       "Last Stand, The (2013)                                   0.666667   \n",
       "\n",
       "                                                    collaborative    hybrid  \n",
       "Close Range (2015)                                       0.426797  0.621647  \n",
       "From Paris with Love (2010)                              0.426797  0.621647  \n",
       "Bad Company (2002)                                       0.554188  0.610427  \n",
       "Gone in 60 Seconds (1974)                                0.502655  0.584661  \n",
       "Take Aim at the Police Van (Sono gosôsha wo ner...       0.499277  0.582972  \n",
       "State Property 2 (2005)                                  0.499277  0.582972  \n",
       "All About the Benjamins (2002)                           0.499277  0.582972  \n",
       "V. I. Warshawski (1991)                                  0.499277  0.582972  \n",
       "Undisputed III: Redemption (2010)                        0.499277  0.582972  \n",
       "Last Stand, The (2013)                                   0.499277  0.582972  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# hybrid recommender for dataset\n",
    "movielens_hybrid = hybrid_rec('Batman Begins (2005)')\n",
    "movielens_hybrid.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# *Conclusion*\n",
    "\n",
    "In this project, we have tried 5 different recommendation systems based on different algorithms and structures. We can clearly see that each of them has their own intricacies and how they differ a lot from each other. Even though, Content-Based Recommender and Collaborative Filtering are the most popular types, there are still a lot of other complex and different recommending engines. The more data there is to collect, the more robust the recommendation model will be and it becomes extremely appealing. There is reason why it is so common to find this application used in so many sites and it is one of the most popular application of data science today. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "oldHeight": 573,
   "position": {
    "height": "595px",
    "left": "1530px",
    "right": "20px",
    "top": "112px",
    "width": "350px"
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "varInspector_section_display": "block",
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}