Part 4b - Siamese Networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Finding Similar Songs on Spotify - Part 2: Siamese Networks\n",
    "\n",
    "In the first part of this tutorial I have introduced the traditional distance based approach to similarity estimations. The main idea is that features are extracted from the audio content. These features are numeric descriptions of semantically relevant information. An example for a high-level feature is the number of beats per minute which is a description for the tempo of a song. Music feature-sets are more abstract and describe the spectral or rhythmical distribution of energy. These are not single but vectors of numbers. Thus, a song is semantically described by this vector and if the set of extracted features spans over various music characteristics such as rhythm, timbre, harmonics, complexity, etc. then calculating the similarity of the vector's numbers is considered to be an approximation of music similarity. Thus, the lower the numerical distance between two vectors, the higher their acoustic similarity. For this reason these approaches are known as *Distance based* methods. They mainly depend on the selected sets of features and on the similarity metric chosen to compare their values.\n",
    "\n",
    "In the second part of this tutorial we are now focussing on an approach, where the feature representation, as well as the similarity function is learned from the underlying dataset.\n",
    "\n",
    "\n",
    "## Tutorial Overview\n",
    "\n",
    "1. Loading data\n",
    "2. Preprocess data\n",
    "3. Define Model\n",
    "4. Fit Model\n",
    "5. Evaluate Model\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requiremnts\n",
    "\n",
    "The requirements are the same as for the first part of the tutorials. Please follow the instructions of part one if you have trouble running this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/schindler/anaconda/python2/envs/py36/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"1\"\n",
    "\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-08-24T10:20:32.488000Z",
     "start_time": "2017-08-24T10:20:32.483000Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "# visualization\n",
    "%matplotlib inline\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "matplotlib.style.use('ggplot')\n",
    "from IPython.display import IFrame\n",
    "\n",
    "# numeric and scientific processing\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy.interpolate import RectBivariateSpline\n",
    "\n",
    "# misc\n",
    "import os\n",
    "import progressbar\n",
    "\n",
    "# spotify API\n",
    "import spotipy\n",
    "import spotipy.util as util\n",
    "\n",
    "# local caching\n",
    "from joblib import Memory\n",
    "\n",
    "# functions from Tutorial Part 1\n",
    "import tutorial_functions as tut_func"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading Data\n",
    "\n",
    "We will use the same data that we downloaded from Spotify in Part 1 of the Tutorial. Because we used the joblib library, we will not have to wait that long, because the data is already cached on our harddrive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Update the following two variables according the credentials you received from Spotify"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "SPOTIFY_USER = \"\"\n",
    "\n",
    "os.environ[\"SPOTIPY_CLIENT_ID\"]     = \"\"\n",
    "os.environ[\"SPOTIPY_CLIENT_SECRET\"] = \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same playlists as used in Part 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "playlists = [\n",
    "    \n",
    "     {\"name\": \"clubbeats\",    \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DXbX3zSzB4MO0\"},\n",
    "     {\"name\": \"softpop\",      \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWTwnEm1IYyoj\"},\n",
    "     {\"name\": \"electropop\",   \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX4uPi2roRUwU\"},\n",
    "     {\"name\": \"rockclassics\", \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWXRqgorJj26U\"},\n",
    "     {\"name\": \"rockhymns\",    \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX4vth7idTQch\"},\n",
    "     {\"name\": \"soft_rock\",    \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX6xOPeSOGone\"},\n",
    "     {\"name\": \"metalcore\",    \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWXIcbzpLauPS\"}, \n",
    "     {\"name\": \"metal\",        \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWWOaP4H0w5b0\"},\n",
    "     {\"name\": \"classic_metal\",\"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX2LTcinqsO68\"},\n",
    "     {\"name\": \"grunge\",       \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DX11ghcIxjcjE\"},\n",
    "     {\"name\": \"hiphop\",       \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DWVdgXTbYm2r0\"},\n",
    "     {\"name\": \"poppunk\",      \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DXa9wYJr1oMFq\"},\n",
    "     {\"name\": \"classic\",      \"uri\": \"spotify:user:spotify:playlist:37i9dQZF1DXcN1fAVSf7CR\"}\n",
    "    \n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Connect to the Spotify API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-08-24T10:20:33.260000Z",
     "start_time": "2017-08-24T10:20:33.256000Z"
    }
   },
   "outputs": [],
   "source": [
    "token = util.prompt_for_user_token(SPOTIFY_USER, \n",
    "                                   \"playlist-modify-public\", \n",
    "                                   redirect_uri=\"http://localhost/\")\n",
    "\n",
    "sp = spotipy.Spotify(auth=token)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the local chache directory. This should be the same as in Part 1 of the tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2017-08-24T10:20:32.875000Z",
     "start_time": "2017-08-24T10:20:32.870000Z"
    }
   },
   "outputs": [],
   "source": [
    "memory = Memory(cachedir='/home/schindler/tmp/spotify/', verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately I was not able to move this function to the tutorial_functions.py file, due to the @memory annotation. (If you know a way how to solve this, please create a Github-issue with your solution)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "@memory.cache\n",
    "def get_spotify_data(track_id):\n",
    "    \n",
    "    # meta-data\n",
    "    track_metadata      = sp.track(track_id)\n",
    "    album_metadata      = sp.album(track_metadata[\"album\"][\"id\"])\n",
    "    artist_metadata     = sp.artist(track_metadata[\"artists\"][0][\"id\"])\n",
    "    \n",
    "    # feature-data\n",
    "    sequential_features = sp.audio_analysis(track_id)\n",
    "    trackbased_features = sp.audio_features([track_id])\n",
    "    \n",
    "    return track_metadata, album_metadata, artist_metadata, sequential_features, trackbased_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start loading the Spotify Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 98% (1059 of 1078) |#################### | Elapsed Time: 0:02:10 ETA:  0:00:03"
     ]
    }
   ],
   "source": [
    "# Get Playlist meta-data\n",
    "playlists = tut_func.get_playlist_metadata(sp, playlists)\n",
    "\n",
    "# Get track-ids of all playlist entries\n",
    "playlists = tut_func.get_track_ids(sp, playlists)\n",
    "\n",
    "num_tracks_total = np.sum([playlist[\"num_tracks\"] for playlist in playlists])\n",
    "\n",
    "# Fetch data and features from Spotify\n",
    "pbar = progressbar.ProgressBar(max_value=num_tracks_total)\n",
    "pbar.start()\n",
    "\n",
    "raw_track_data      = []\n",
    "processed_track_ids = []\n",
    "\n",
    "for playlist in playlists:\n",
    "\n",
    "    for track_id in playlist[\"track_ids\"]:\n",
    "\n",
    "        try:\n",
    "            # avoid duplicates in the data-set\n",
    "            if track_id not in processed_track_ids:\n",
    "\n",
    "                # retrieve data from Spotify\n",
    "                spotify_data = get_spotify_data(track_id)\n",
    "\n",
    "                raw_track_data.append([playlist[\"name\"], spotify_data])\n",
    "                processed_track_ids.append(track_id)\n",
    "\n",
    "        except Exception as e:\n",
    "            print(e)\n",
    "\n",
    "        pbar.update(len(raw_track_data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aggregate feature-data\n",
    "\n",
    "Currently we only have a list of raw data-objects retrieved from the Spotify API. We need to transform this information to a more structured format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Aggregate Meta-data\n",
    "metadata = tut_func.aggregate_metadata(raw_track_data)\n",
    "\n",
    "# Aggregate Feature-data\n",
    "feature_data = tut_func.aggregate_featuredata(raw_track_data, metadata)\n",
    "\n",
    "# standardize sequential_features\n",
    "feature_data -= feature_data.mean(axis=0)\n",
    "feature_data /= feature_data.std(axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Siamese Networks\n",
    "\n",
    "A Siamese neural network is a neural network architecture where two inputs are fed into the same stack of network layers. This is where the name comes from. The shared layers are \"similar\" to Siamese Twins. By feeding two inputs to the shared layers, two representations are generated which can be used for comparison. To train the network according a certain task, it requires labelled data. To learn a simlarity function, these labels should indicate if the two input are similar or dissimilar.\n",
    "\n",
    "This is exactly the approach initially described by Hadsell-et-al.'06 (http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf). The authors create pairs of simlar and dissimilar images. These are fed into a Siamese NEtwork stack. Finally, the model calculates the eucledian distance between the two generated representations. A contrastive loss is used, to optimize the learned simlarity.\n",
    "\n",
    "To calculate the similarity between a seed image and the rest of the collection, the model is applied to predict the distance between this seed image and every other. The result is a list of distances which has to be sorted descendingly.\n",
    "\n",
    "The following code example follows this approach:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Keras**\n",
    "\n",
    "We use the high-level deep learning API Keras: https://keras.io/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models           import Model\n",
    "from keras.layers           import Input, Lambda, Dense, Bidirectional\n",
    "from keras.layers.recurrent import LSTM\n",
    "from keras.layers.merge     import concatenate\n",
    "from keras.optimizers       import Nadam\n",
    "from keras import backend as K"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we define a distance measure to compare the two representations. We will be using the well known Eucledian distance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def euclidean_distance(vects):\n",
    "    x, y = vects\n",
    "    return K.sqrt(K.maximum(K.sum(K.square(x - y), axis=1, keepdims=True), K.epsilon()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Siamese Network Architecture\n",
    "\n",
    "Now we define the Siamese Network Architecture. It consists of two fully connected layers. These layers are shared among the \"Siamese twins\". The network takes two inputs. One goes to the left twin, the other to the right one. The Eucledian distance of the output of each twin is calculated which is the final output of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_siamese_network(input_dim):\n",
    "\n",
    "    # input layers\n",
    "    input_left  = Input(shape=input_dim)\n",
    "    input_right = Input(shape=input_dim)\n",
    "\n",
    "    # shared fully connected layers\n",
    "    shared_fc_1 = Dense(100, activation=\"selu\")\n",
    "    shared_fc_2 = Dense(100, activation=\"selu\")    \n",
    "    \n",
    "    # siamese layers\n",
    "    left_twin  = shared_fc_1(shared_fc_2(input_left))\n",
    "    right_twin = shared_fc_1(shared_fc_2(input_right))\n",
    "\n",
    "    # calc difference\n",
    "    distance = Lambda(euclidean_distance,\n",
    "                      output_shape=lambda x: x[0])([left_twin, right_twin])\n",
    "\n",
    "    return Model([input_left, input_right], distance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Contrastive Loss**\n",
    "\n",
    "The contrastive loss is based on the Euclidean distance and measures the cost of data pairs. The objective of the contrastive loss is to minimize the distance between a similar pair and to separate any two dissimilar data with a distance margin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def contrastive_loss(y_true, y_pred):\n",
    "    margin = 1\n",
    "    return K.mean(y_true * K.square(y_pred) + (1 - y_true) * K.square(K.maximum(margin - y_pred, 0)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Data-Pairs\n",
    "\n",
    "Now we have to prepare and partition the input data. Because it is a pair-wise comparison approach, we have to create pairs of input instances. Especially, we will create sequences of genuine or similar pairs followed by impostor or dissimilar pairs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_pairs(feature_data, metadata, num_pairs_per_track):\n",
    "    \n",
    "    data_pairs = []\n",
    "    labels     = []\n",
    "    \n",
    "    for row_id, q_track in metadata.sample(frac=1).iterrows():\n",
    "        \n",
    "        for _ in range(num_pairs_per_track):\n",
    "            \n",
    "            # search similar and dissimilar examples\n",
    "            pos_example = metadata[metadata.playlist == q_track.playlist].sample(1)\n",
    "            neg_example = metadata[metadata.playlist != q_track.playlist].sample(1)\n",
    "\n",
    "            # create feature pairs\n",
    "            \n",
    "            # genuine pair\n",
    "            data_pairs.append([feature_data[[row_id]][0], feature_data[[pos_example.index]][0]])\n",
    "            labels.append(1)\n",
    "\n",
    "            # impostor pair\n",
    "            data_pairs.append([feature_data[[row_id]][0], feature_data[[neg_example.index]][0]])\n",
    "            labels.append(0)\n",
    "\n",
    "    return np.array(data_pairs), np.array(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exectue the function to prepare the input data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(21200, 2, 69)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create pairs\n",
    "data_pairs, labels = create_pairs(feature_data, metadata, 10)\n",
    "\n",
    "# check - how many instances have we created?\n",
    "data_pairs.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the Siamese Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the model\n",
    "model_orig = create_siamese_network(data_pairs[:,0].shape[1:])\n",
    "\n",
    "# define the optimizer\n",
    "opt = Nadam(lr=0.001)\n",
    "\n",
    "# compile the model\n",
    "model_orig.compile(loss      = contrastive_loss, \n",
    "                   optimizer = opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHCtJREFUeJzt3X2QJGWB5/Hvk5nVL1XdM7yUwziAvK+gxO14GstcYPgCooxgjOL5nBznocc5cqfrKJ6GqLEQiwR3EYssxuoZs66ihOI9e8IxvJ07ARKid44u7gHi6CoIMswM0DMyr/1SlfncH5nV0zPdPV1dM9WV2fn7RNRUVnZW1vN09fyerKcyn8d47xERkeIIel0AERGZHwW3iEjBKLhFRApGwS0iUjAKbhGRglFwi4gUjIJbRKRgFNwiIgUTtbORtfYZYA8QA03n3Bu6WSgREZldW8GdeatzbqTNbXU5pojI/Jl2NppPcM/L1q1bO3pevV5nZKTd9mFxKXPdodz1V93LWXc4UP8VK1a0/Zx2+7g98A/W2kettWs7Kp2IiBwVpp1Bpqy1K5xzW621y4CNwJ875350yDZrgbUAzrnXT0xMdFSgKIpoNpsdPbfoylx3KHf9Vfdy1h0O1L+vrw/a7CppK7instZeD+x1zv3VYTbz6iqZvzLXHcpdf9W9nHWHaV0lR6eP21pbAwLn3J5s+e3AXx5RSUVk0fHeMzY2RpIkGNNW/gDwwgsvMD4+3sWS5Yf3niAIGBgYmNfv6FDtfDl5AnCXtba1/Xedc/+741cUkUVpbGyMSqVCFM3vnIcoigjDsEulyp9ms8nY2BiDg4Md72PO37Bz7mngTzt+BREphSRJ5h3aZRRF0RF/wtCVkyJyVBzJR/+yOdLfVW6C23tPcu/3GP+nn/a6KCIiuZab4DbG4H9wFxO/UHCLSGfOOuusXhdhQeQmuAGoDpHs3dPrUoiI5Fr+gnufgltEjoz3nhtuuIELLriACy+8kLvvvhtITz287LLLuOiii7jgggvYtGkTcRzziU98YnLb9evX97j0c8vXV8C1IbyOuEUKL/ne3+Kf+3172xpDOxcCmpNPI3j/h9va5/3338+TTz7Jxo0b2blzJ+985ztZtWoVd911F29+85tZt24dcRwzOjrKk08+yfbt23nooYcA2LVrV1uv0Us5O+Kukezd3etSiEjB/exnP+Pd7343YRjyile8glWrVvHYY4+xcuVKnHPcfPPNbN68maGhIV71qlfxhz/8gS984Qv88Ic/ZHh4uNfFn1OujrhNdQi/73ftXfMpIrnV7pExdGesktmO4FetWsX3v/99HnzwQdatW8fVV1/N+973PjZu3MjDDz/Mbbfdxj333MOXvvSlo1qeoy1fR9w1fTkpIkdu1apVbNiwgTiO2bFjB5s2bWLlypVs2bKFer3OFVdcwfvf/36eeOIJdu7cSZIkXHLJJXz605/miSee6HXx55SrI26qQzAxjm80MJVKr0sjIgW1evVqHn30US666CKMMXz+859n2bJlOOf42te+RhRF1Go1br31VrZt28Y111xDkiQAXHvttT0u/dzmPTpgmzoaHTD54f34736N4K++hVl6bBeKlW8aJa289V8Mdd+/fz/VanXezyvjsK5Tf1edjA6Yr66Sai2937+3t+UQEcmxXAW3qQ2lC/sU3CIis8lVcFPNgltH3CIis8plcHsFt4jIrPIV3JNdJft6Ww4RkRzLV3AP6stJEZG55Cq4TRRhBqoKbhGRw8hVcAOYoWGdVSIiXXe4sbufe+45LrjgggUszfzkLriDoWF9OSkichj5uuQdMLVhdZWIFNzX//EFfv/Hsba2NW0O63rasQP8xzecMOvPb7zxRk488UQ++MEPAnDzzTdjjOGnP/0pu3btotls8pnPfIZ3vOMdbZWrZWxsjGuvvZbHH3+cMAy57rrrOP/88/nNb37DNddcw8TEBN571q9fz/Lly/nIRz7Ctm3bSJKEdevWsWbNmnm9XjtyF9zB0DBs2dnrYohIwaxZs4brrrtuMrjvuecevvOd7/DhD3+Y4eFhdu7cybve9S7e/va3z2uy3ttuuw2ABx98kN/97ndcfvnlPPLII9x+++1cddVVXHbZZUxMTBDHMQ899BDLly/n9ttvB2D37u4MU5274DY19XGLFN3hjowPdbTGKjn33HMZGRlh+/bt7Nixg6VLl7Js2TKuv/56Nm3ahDGG7du389JLL7Fs2bK29/vzn/+cD33oQwCceeaZnHTSSTz99NO8/vWv58tf/jLbtm1j9erVnH766Zx99tnccMMN3HjjjbztbW/jvPPOO+J6zSSXfdzqKhGRTlxyySXcd999bNiwgTVr1nDnnXeyY8cOHnjgATZu3Ei9Xmd8fHxe+5ytG+c973kP3/zmNxkYGOCKK67gxz/+MWeccQYPPPAAZ599NjfddBO33HLL0ajWNLkLbjM0nA7t2mz0uigiUjBr1qzh7rvv5r777uOSSy5hz5491Ot1KpUKP/nJT9iyZcu893neeedx1113AfDUU0/x/PPPc8YZZ/Dss89yyimncNVVV3HRRRexefNmtm/fzuDgIO9973u5+uqruza2d+66SoLaknRh/15YUr6hXUWkc69+9avZt28fy5cv54QTTuCyyy7jyiuvZPXq1bz2ta/lzDPPnPc+r7zySj772c9y4YUXEoYht9xyC/39/WzYsIE777yTKIpYtmwZn/zkJ3nsscf44he/iDGGSqXCTTfd1IVa5mw8boDar37B7luuJ/jLr2JeedJRLla+LYYxmY9Emeu/GOqu8bjbt7jG4waCWjZRp/q5RURmlLuuEjOk4BaRhbF582Y+/vGPH7Suv7+fe++9t0clak/ugjvIgtvv26vZ3kUKpEvdrl11zjnnsHHjxgV/3SP9XamrRESOiiAIStdX3Ylms0kQHFn05u6I2wxNOatERApjYGCAsbExxsfH53VlYn9//7zPrS4q7z1BEDAwMHBE+8lfcEcR9A9oMgWRgjHGMDg4OO/nLYYzahZa28FtrQ2BfwSed85d2r0ikU5hpiNuEZEZzaejZR2wuVsFOUi1pqFdRURm0VZwW2tPAi4Bvt7d4mRqOuIWEZlNu0fcfw18Bki6WJYDqkMaIVBEZBZz9nFbay8FXnTOPWqtfcthtlsLrAVwzlGv1zsrUBQxcNzxTDz3+473UVRRFJWuzlOVuf6qeznrDp3Vf86xSqy1NwEfAJrAALAEuNM59+8O87SOxyqp1+u8+JX/hn/kB4R/4zraR1GV/dv1MtdfdS9n3aGzsUrmPOJ2zl0LXAuQHXH/lzlC+8jVajA+hm8209MDRURkUu6unATSPm7QF5QiIjOY1+Gsc+5h4OGulGSqqcG95Jiuv5yISJHk8ojb1LLg1pklIiLT5DK4Dxxx67J3EZFD5Tq4dfWkiMh0+QzuWi29V3CLiEyTz+Cuqo9bRGQ2uQxuE1Wgr19H3CIiM8hlcAMa2lVEZBb5De7aEF6TKYiITJPf4K7WdMQtIjKDHAe3ukpERGaS2+A2Cm4RkRnlNripDWnCYBGRGeQ3uKtDMD6KbzZ7XRIRkVzJd3ADjOqoW0RkqvwGd+uyd109KSJykNwGt9FkCiIiM8ptcGsWHBGRmeU3uLPJFLy6SkREDpLf4NZkCiIiMypAcOuIW0RkqtwGt6lUoK9PwS0icojcBjeQHnWrj1tE5CC5D27NOykicrDcB7e+nBQROVi+g7umrhIRkUPlOriNJlMQEZkm18FNbVjBLSJyiHwHd3UIxkbxcdzrkoiI5Eb+gxv0BaWIyBT5Du7W0K7qLhERmZTr4NbQriIi0+U6uCe7SnRKoIjIpHwHd2toVx1xi4hMiubawFo7APwI6M+2/5/Oueu6XTBAIwSKiMygnSPuceAC59yfAiuBi621q7pbrIy6SkREppnziNs554FWclaym+9moVoODO2q0wFFRFrmDG4Aa20IPAqcCXzFObdphm3WAmsBnHPU6/XOChRFBz33paEl9CVNlna4vyI5tO5lU+b6q+7lrDt0Vn/jffsHz9baY4C7gD93zv3yMJv6rVu3zqsgLfV6nZGRkcnH8XUfg+UnEv6nazvaX5EcWveyKXP9Vfdy1h0O1H/FihUApp3nzOusEufcy8DDwMXzLl2nNJmCiMhB5gxua+0rsiNtrLWDwNuAX3e7YJNqQzqrRERkinb6uF8JfCvr5w4A55y7t7vFOsBUa/gtzyzUy4mI5F47Z5U8DrxuAcoys6qOuEVEpsr3lZOQBvfofnyioV1FRKAIwV3T0K4iIlPlP7h12buIyEFyH9yTQ7vu0xG3iAgUILg1mYKIyMHyH9xVDe0qIjJVYYJbV0+KiKTyH9w1fTkpIjJV7oPbVPqg0qfgFhHJ5D64gezqSZ1VIiIChQnuGl593CIiQFGCWyMEiohMKkZwa6ApEZFJhQhuo8kUREQmFSK4064SfTkpIgJFCe5qDUb3aWhXEREKE9zZRTij+3tbDhGRHChWcKufW0SkGMFtdNm7iMikQgS3JlMQETmgUMHtNZmCiEhBgluTKYiITCpGcKurRERkUiGC2/T1Q1TRWSUiIhQkuAENNCUikilOcFeHNO+kiAiFCu6axisREaFQwa0RAkVEoEDBbdTHLSICFCi4NZmCiEiqWME9uh+fJL0uiYhITxUnuGs18F5Du4pI6UVzbWCtPRn4NrAcSID1zrlbu12waaZePdkaLVBEpITaOeJuAp9yzp0DrAI+aq19TXeLNZ3RZe8iIkAbwe2c2+ac+0W2vAfYDJzY7YJNo8kURESAefZxW2tPBV4HbOpKaQ5HkymIiABt9HG3WGuHgO8Dn3DO7Z7h52uBtQDOOer1emcFiqIZnxsbzwhQC6Da4b7zbra6l0WZ66+6l7Pu0Fn9jfd+zo2stRXgXuAHzrkvtbFfv3Xr1nkVpKVerzMyMjJ9h+PjJB97H+ayKwlWv7ejfefdbHUvizLXX3UvZ93hQP1XrFgBYNp5zpxdJdZaA/wdsLnN0O6Ovj6IInWViEjptdNVcj7wAeAJa+3/y9Z9zjl3f/eKNZ0xRldPiojQRnA7535Mm4fvXaeBpkRECnTlJEBNY3KLiBQruKtDGpNbREqvUMFtqjX1cYtI6RUquNXHLSJStOCuDcHoPg3tKiKlVqzgrg6lQ7uOaWhXESmv4gU3qLtEREqtUMFtarV0QWeWiEiJFSq40ZjcIiIKbhGRoilkcHv1cYtIiRUruDWZgohIwYK7rx9CDe0qIuVWqOBOh3atwT6dVSIi5VWo4AbS7hIdcYtIiRUvuKsa2lVEyq2Qwa0rJ0WkzAoX3EbTl4lIyRUuuKnVdMm7iJRa8YI7mwVHQ7uKSFkVM7h9AmOjvS6JiEhPFC+4dfWkiJRc4YLbaKApESm5wgW3JlMQkbIrXnBrMgURKbniBXdraFd1lYhISRU2uNXHLSJlVbzg7h+AMFQft4iUVuGCOx3aVZe9i0h5FS64gcmrJ0VEyqigwV3TvJMiUlrFDG5NpiAiJVbI4NbQriJSZtFcG1hrvwFcCrzonDu3+0Vqg4JbREqsnSPu24CLu1yO+WkN7ep9r0siIrLg5gxu59yPgJ0LUJb21WqQaGhXESmnQvZx6+pJESmzOfu422WtXQusBXDOUa/XOytQFM353LHlK9gFHNMXUenwdfKonbovZmWuv+pezrpDZ/U/asHtnFsPrM8e+pGRkY72U6/Xmeu5Pk6nLXv5+S2Y4eM6ep08aqfui1mZ66+6l7PucKD+K1asaPs56ioRESmYOYPbWnsH8H+BV1trt1hrr+p+sebQGtpVV0+KSAnN2VXinLt8IQoyL5pMQURKrJhdJf2DEATqKhGRUipkcGtoVxEps0IGN5AGt/q4RaSEihvctSHNOykipVTc4K7W9OWkiJRSYYPbqKtEREqqsMGtyRREpKyKG9zZWSUa2lVEyqbYwZ0kMK6hXUWkXAoc3NnVk/v0BaWIlEthg9vUNNCUiJRTYYNbIwSKSFkVP7h1SqCIlExxgzvrKtHVkyJSNsUNbnWViEhJFTe4B7KhXdVVIiIlU9jgTod2remIW0RKp7DBDWhoVxEppcIHt76cFJGyyVVwjzWT+T2hOqShXUWkdHIT3I3Ys+6+3/MXD/yaLbvG23qOqamrRETKZ85Z3hdK4j1vPGUJ9/3zTn742xHectoS/s25dZYP983+JH05KSIllJsj7v4o4AMrX8Hff/ANvOvVx/LIM3v4z/c8zVc3bWdkf2PmJ2loVxEpodwccbccW+3jP7z+BNaccxx//8sdbHzqZR56ehcX/8kx/OvXHM8xg1OKXJsytOtAtXeFFhFZQLk54j7U8dUKV//Zcr76rtN506lLuO83f2Tt3U/x7X96kT3jcbrR5Hgl+oJSRMojt8HdcsJQHx//V6/kby49nfNOGubOX+1k7d1P8b0nRhjtz8Yr+clG/FO/xo+396WmiEiR5a6rZDYnLunjU29cwXtfexx3PDHCHY+PcG/leC79k0s57f88xtKHf8LSxn6WHr+UgZNPgVNOx7zqDDj5dMygulFEZPEoTHC3nHrsANe+6SR+u2OUOx4f4Y7Gm2DFmw7apj+eYOkze1n6260snfhnloYJS2r9LF06zDHLjmO4fjzV2gCD1UGq/RGDlZDBKKASmh7VSkSkfYUL7pazjh/kL956MiP7G+zc32TXWMyu8SYvj8XsHmvy8p4l7Nq1lx2jDZ5uwG76aI6G8Czw7B5gz7R9VpImg77JIE0GiRkMEqqBZzCASmiIAkMYGqIgIArTWxiG6XIUEEYRURRmt4hqf0i1L6LWX6HaH1EbqFAdqFAJc99DJSI5VtjgbqlXK9SrlTm3896z9+Xd7H7mWXaN7GBsvMHoRMzoRMz+RsJo07M/gdEYRn2Q3oh42URsDfpoBBFNExAHIbEJaJqIZhCQmHDKqyTZbZbTFzP98QTVeIxac4xqMkEtHqeajDNITIgnCAyRMYSBIQwgDNLGImjdTzYaacORNiYhYRQQhSFh1pBEYUgYhel9JSKKIsJKSKUSUYkiokpEX1+FqC8iiCoQRRBG6QBeIpJbhQ/udhljGD52KcPH/gtOnOdzfZJA3ITGBDQb0GhMLicTE8QTTZqNCeJGg2ajSaPRoDERM9b07GvCvjhtFPbHhn2xYX8SsC8x7PMh+32NvX6YF4gYNyHNxBADMQGxMdn9IUfoHmhmt3mZvWGJkiZREtOXNIh8TGXKLfIJEQkhnghPiCc0nsiQPjYQGbJ1aWMTGagEUJlyHwVQCYN0XWgmP8VUwpBKZNhaG2bP2Bg+jEhMQBKEJCYkCYL0sQlJwtZygDdpI5U2cAFBFBAGWYMWBARROLnc+mSUbmuIDFlDaAiNIQpQgyWFUZrgPhImCCDog8r0qzjD7HaY6zvbVq/XGRkZmbbee0/ioZl4Yu+JE4gTTyOOiZsxcaNB3GjSbDaz+5i4mT1uJunjOFuOYxqxp9FMaMQJjcSnj5PWjfTmW/cRTQ9NDzGGcW9oYoizW7ocTN6nn0YCmiYkIUgbmXg+v4XaHD9vNT5HX+ATIh8T+oRwyrLBAya9Nwbjs/tDd2CYXOcxJMbgyW4GEg48TrJ1rWUDpL+xrGHMbgGk60xrXfo4AALjp7yomSzDgeXp64Js2ZgDp5QZkzZaJtvKmLS2rV0f1KBN2ZdpLU99Tcga7rRRjAImG8dKSLYuazzDgCgwLBl6kdHR0bQMrVtgZnkcYAx4E9Dwnkac/p02vaeZQCM5cN9IPM3sb7uZeAIDfaGhEhgqYTBl2dAXpt9xVQKTrg/TbUIDgTHpfWAIDIQmvZ/22KTPP+haky5p6xWstRcDt5Jm1Nedc/+1q6WSg5jsDycMDo2KfLe7cfYfZrJhiD0TzZhG1sg0mjGNRkyj2aTRTFhSrbJvz26CJCYgIUgSAh+n90k85XGM8TEmjkkST5x44iSZspzdfHqfJJ44a/gSskbIQ9MbYp+2K7E3NCH9lONJG6JsHT5tf/Ce9CJdn16t6w/6Yfoz7/F4Au8J8ODTZUOCaS37bDlb12oI4qxxTDCTn7aSrIFMpnz6Sn8eTL589g9MXkE88yeHgxqPrNFoLZO9LiZtptL1prXnGczyGpA23kFE04TEQUjThDQn72f6mz26p/EGPkk/QfqYShJTSZqEJHgME0FEMwiZMBUaQYQ/yp+ylsajfPvfv+6o7nMmc/7Pt9aGwFeAi4AtwM+ttRucc7/qduGk2FrdEv0H/ZXN/n3EbJ84yuBo1T1tUJL0imLvs/vscZKkyeoTwEOSNiz4qdtOXUe6jimPZ7z3MzwvPmSfE5Ak+MQT+4RmnKRHynHCwGCVvXv3kmT7S3zWMCb+oHV4j0+yBjJJqJgk7c7DE5GGdKtLL3tiVvesbLTKmO7TZw14wxsmMDQw6bIPaHqYICBpNaZZA58AiU8fp41qdu/Tn1X6KkAOghv4M+B3zrmnAay13wPWAApukZwxxoAJIQjn3rgHDGkXTQUYzNb1ssFudXPO1UGXN+2cl3Yi8NyUx1uydSIi0gPtHHHP1Ak0revLWrsWWAvgnKNer3dWoCjq+LlFV+a6Q7nrr7qXs+7QWf3bCe4twMlTHp8EbD10I+fcemB99tB3+tFH/ZzlrDuUu/6qeznrDgfqv2LFiraf005w/xw4y1p7GvA88H7g33ZWRBEROVJz9nE755rAx4AfAJvTVe7JbhdMRERm1taJwM65+4H7u1wWERFpg0Y7EhEpGAW3iEjBmC5NtKvZe0VE5q+ta/C7dcRtOr1Zax89kucX+Vbmupe9/qp778uRk/q3RV0lIiIFo+AWESmYPAb3+rk3WbTKXHcod/1V9/Kad/279eWkiIh0SR6PuEVE5DByM4VK2WfZsdY+Qzr1fAw0nXNv6G2Jusda+w3gUuBF59y52brjgP8BnAo8A1jn3B97VcZumqX+1wMfBl7KNvtcdsXyomKtPRn4NrCcdO6B9c65W8vw/h+m7tczz/c+F0fcU2bZWQ28BrjcWvua3paqJ97qnFu5mEM7cxtw8SHrPgs86Jw7C3gwe7xY3cb0+gPckr3/KxdjaGeawKecc+cAq4CPZv/Xy/D+z1Z3mOd7n4vgZsosO865CaA1y44sQs65HwE7D1m9BvhWtvwt4N0LWqgFNEv9S8E5t80594tseQ/pwHUnUoL3/zB1n7e8BLdm2UmvNv0Ha+2j2aQUZXOCc24bpH/gwLIel6cXPmatfdxa+w1r7bG9Lky3WWtPJZ2gcRMle/8PqTvM873PS3DPdMVQ2U53Od859y9Ju4s+aq19U68LJAvqvwNnACuBbcDNvS1Od1lrh4DvA59wzu3udXkW0gx1n/d7n5fgbmuWncXMObc1u38RuIu0+6hMXrDWvhIgu3+xx+VZUM65F5xzsXMuAf6WRfz+W2srpMH1HefcndnqUrz/M9W9k/c+L8E9OcuOtbaPdJadDT0u04Kx1tastcOtZeDtwC97W6oFtwG4Mlu+Eri7h2VZcK3QyryHRfr+W2sN8HfAZufcl6b8aNG//7PVvZP3PjcX4Fhr3wn8NenpgN9wzt3Y4yItGGvt6aRH2ZCeovndxVx/a+0dwFuAOvACcB3wvwAHvAr4A/A+59yi/AJvlvq/hfSjsic9He4jrT7fxcRa+0bgEeAJ0lPiAD5H2te7qN//w9T9cub53ucmuEVEpD156SoREZE2KbhFRApGwS0iUjAKbhGRglFwi4gUjIJbRKRgFNwiIgWj4BYRKZj/D0vqp0nmJh+DAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f23c59f4358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_orig.fit([data_pairs[:, 0], data_pairs[:, 1]], \n",
    "                labels, \n",
    "                batch_size       = 24, \n",
    "                verbose          = 0, \n",
    "                epochs           = 25, \n",
    "                callbacks        = [tut_func.PlotLosses()], \n",
    "                validation_split = 0.1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluate\n",
    "\n",
    "Now that we have a trained model, we want to evaluate its performance. We will first play around with some examples, listen to the results and judge by our subjective interpretation before we persue a general evaluation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": true
   },
   "source": [
    "### Evaluate by Example\n",
    "\n",
    "The following function calculated the distances between a given query track and all other tracks of the collection. The result is a list of distances where the smallest distance coresponds with the most similar track. The list is sorted descendingly and the top-ten similar tracks are presented below the information of the query track. The Spotify playlist we created at the beginning will also be updated with the query results. Thus, you can listen to it in your Spotify client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def similar(model, query_idx):\n",
    "    \n",
    "    print(metadata.iloc[query_idx])\n",
    "    \n",
    "    # calclulate predicted distances between query track and all others\n",
    "    res = [model.predict([feature_data[[query_idx]], feature_data[[i]]]) for i in range(feature_data.shape[0])]\n",
    "\n",
    "    # reshape\n",
    "    res = np.array(res)\n",
    "    res = res.reshape(res.shape[0])\n",
    "\n",
    "    # get sorted indexes in ascending order (smallest distance to query track first)\n",
    "    si = np.argsort(res)\n",
    "    \n",
    "    # output filter\n",
    "    display_cols = [\"artist_name\", \"title\", \"album_name\", \"year\",\"playlist\"]\n",
    "    \n",
    "    return metadata.loc[si, display_cols][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check the results for some individual tracks by supplying the index to our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "track_id                                  637McrDyDQ9CffkFuZGRpq\n",
      "artist_name                                                KSHMR\n",
      "title                                                    Kolkata\n",
      "album_name                                               Kolkata\n",
      "label                                                     Dharma\n",
      "duration                                                  206250\n",
      "popularity                                                    54\n",
      "year                                                        2017\n",
      "genres         [big room, deep big room, edm, electro house, ...\n",
      "playlist                                               clubbeats\n",
      "Name: 33, dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_name</th>\n",
       "      <th>title</th>\n",
       "      <th>album_name</th>\n",
       "      <th>year</th>\n",
       "      <th>playlist</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>KSHMR</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Hardwell</td>\n",
       "      <td>Get Low</td>\n",
       "      <td>Get Low</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Monik</td>\n",
       "      <td>Palkia</td>\n",
       "      <td>Palkia</td>\n",
       "      <td>2016</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Noise Cans</td>\n",
       "      <td>No War (feat. Jesse Royal) - Yellow Claw Remix</td>\n",
       "      <td>No War (feat. Jesse Royal) [Yellow Claw Remix]</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>Calvin Logue</td>\n",
       "      <td>Warrior</td>\n",
       "      <td>Warrior EP</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>Stadiumx</td>\n",
       "      <td>Spacebird</td>\n",
       "      <td>Spacebird</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Timmy Trumpet</td>\n",
       "      <td>Punjabi</td>\n",
       "      <td>Punjabi</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>Tom &amp; Jame</td>\n",
       "      <td>In The Air</td>\n",
       "      <td>In The Air</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>Dropgun</td>\n",
       "      <td>Little Drop</td>\n",
       "      <td>Little Drop</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Don Diablo</td>\n",
       "      <td>Momentum</td>\n",
       "      <td>Momentum</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      artist_name                                           title  \\\n",
       "33          KSHMR                                         Kolkata   \n",
       "16       Hardwell                                         Get Low   \n",
       "9           Monik                                          Palkia   \n",
       "26     Noise Cans  No War (feat. Jesse Royal) - Yellow Claw Remix   \n",
       "49   Calvin Logue                                         Warrior   \n",
       "70       Stadiumx                                       Spacebird   \n",
       "8   Timmy Trumpet                                         Punjabi   \n",
       "41     Tom & Jame                                      In The Air   \n",
       "37        Dropgun                                     Little Drop   \n",
       "6      Don Diablo                                        Momentum   \n",
       "\n",
       "                                        album_name  year   playlist  \n",
       "33                                         Kolkata  2017  clubbeats  \n",
       "16                                         Get Low  2018  clubbeats  \n",
       "9                                           Palkia  2016  clubbeats  \n",
       "26  No War (feat. Jesse Royal) [Yellow Claw Remix]  2017  clubbeats  \n",
       "49                                      Warrior EP  2018  clubbeats  \n",
       "70                                       Spacebird  2017  clubbeats  \n",
       "8                                          Punjabi  2017  clubbeats  \n",
       "41                                      In The Air  2017  clubbeats  \n",
       "37                                     Little Drop  2017  clubbeats  \n",
       "6                                         Momentum  2017  clubbeats  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar(model_orig, 33)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### General Evaluation\n",
    "\n",
    "So far we have only tested our similarity retrieval algorithms on a few single examples. To evaluate, if the presented approaches perform differently, we perform a full evaluation. To estimate the performance we measure precision and recall of our algorithms. These are standard information retrieval measures of relevance. In the context of this tutorial **precision** measures how many tracks of a given resultlist belong to the same playlist as the query song (relative to the length of the resultlist). **Recall** measures how many tracks of the query song's playlist are contained in the resultlist (relative to the length of the playlist)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(similarity_function, cut_off):\n",
    "\n",
    "    all_precisions = []\n",
    "    all_recall     = []\n",
    "\n",
    "    for idx in metadata.index.values:\n",
    "\n",
    "        dist           = similarity_function(feature_data, feature_data[[idx]])\n",
    "        dist           = np.array(dist).reshape(len(dist))\n",
    "        similar_tracks = metadata.loc[np.argsort(dist)[:cut_off]]\n",
    "        same_label     = similar_tracks[\"playlist\"] == metadata.loc[idx, \"playlist\"]\n",
    "        precision      = same_label.sum() / float(cut_off)\n",
    "        all_precisions.append(precision)\n",
    "\n",
    "        recall = float(same_label.sum()) / metadata[metadata.playlist == metadata.loc[idx, \"playlist\"]].shape[0]\n",
    "        all_recall.append(recall)\n",
    "\n",
    "    all_precisions = np.array(all_precisions)\n",
    "    all_recall     = np.array(recall)\n",
    "\n",
    "    print(\"Precision\", all_precisions.mean())\n",
    "    print(\"Recall:\",   all_recall.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the evauation for the trained Siamese Network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluate(lambda x,y: [model_orig.predict([x[[i]], y]) for i in range(feature_data.shape[0])], 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Discussion of Results\n",
    "\n",
    "More than 90% precision is quite an exciting number, but several flaws of the experimental design have to be considered. The model is evaluated according its ability rank tracks higher which belong to the same playlist of the query track. This does not imply music similarity in general. The set of playlists used in this tutorial contains some very broad lists which span over several music genres. And even songs of the same genre may sound completely different.\n",
    "\n",
    "The single exploratory examples show, that, althoug from the same playlist, many ranked results do not really fit."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Improve Performance through prior Knowledge\n",
    "\n",
    "The original approach only uses genuine and impostor pairs and does not consider any further prior knowledge. In that sense if two tracks belong to the same playlist, they are considered similar, if not, than they are not. But, because we have chosen genre-playlists, there are genres that are more similar than others. This is of course highly subjective and depends on the listening behaviour and experience of a listener.\n",
    "\n",
    "The following list represents my own interpretation of genre similarities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "sim = [[[\"clubbeats\",     \"electropop\"],    0.8],\n",
    "       [[\"clubbeats\",     \"softpop\"],       0.4],\n",
    "       [[\"electropop\",    \"hiphop\"],        0.4],\n",
    "       [[\"softpop\",       \"soft_rock\"],     0.2],\n",
    "       [[\"softpop\",       \"electropop\"],    0.4],\n",
    "       [[\"softpop\",       \"hiphop\"],        0.1],\n",
    "       [[\"rockclassics\",  \"rockhymns\"],     0.7],\n",
    "       [[\"soft_rock\",     \"rockclassics\"],  0.3],\n",
    "       [[\"soft_rock\",     \"rockhymns\"],     0.3],\n",
    "       [[\"metalcore\",     \"metal\"],         0.7],\n",
    "       [[\"metalcore\",     \"classic_metal\"], 0.6],\n",
    "       [[\"metal\",         \"classic_metal\"], 0.8],\n",
    "       [[\"classic_metal\", \"grunge\"],        0.5],\n",
    "       [[\"metal\",         \"grunge\"],        0.5],\n",
    "       [[\"rockhymns\",     \"grunge\"],        0.2],\n",
    "       [[\"poppunk\",       \"metal\"],         0.6],\n",
    "       [[\"poppunk\",       \"classic_metal\"], 0.4],\n",
    "       [[\"poppunk\",       \"rockhymns\"],     0.5],\n",
    "       [[\"poppunk\",       \"rockclassics\"],  0.4]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code creates a symmetric lookup-table from the list above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>clubbeats</th>\n",
       "      <th>softpop</th>\n",
       "      <th>electropop</th>\n",
       "      <th>rockclassics</th>\n",
       "      <th>rockhymns</th>\n",
       "      <th>soft_rock</th>\n",
       "      <th>metalcore</th>\n",
       "      <th>metal</th>\n",
       "      <th>classic_metal</th>\n",
       "      <th>grunge</th>\n",
       "      <th>hiphop</th>\n",
       "      <th>poppunk</th>\n",
       "      <th>classic</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>clubbeats</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>softpop</th>\n",
       "      <td>0.4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>electropop</th>\n",
       "      <td>0.8</td>\n",
       "      <td>0.4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rockclassics</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rockhymns</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>soft_rock</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0.3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>metalcore</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>metal</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.7</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>classic_metal</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>grunge</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>hiphop</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>poppunk</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.6</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>classic</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               clubbeats  softpop  electropop  rockclassics  rockhymns  \\\n",
       "clubbeats            1.0      0.4         0.8           0.0        0.0   \n",
       "softpop              0.4      1.0         0.4           0.0        0.0   \n",
       "electropop           0.8      0.4         1.0           0.0        0.0   \n",
       "rockclassics         0.0      0.0         0.0           1.0        0.7   \n",
       "rockhymns            0.0      0.0         0.0           0.7        1.0   \n",
       "soft_rock            0.0      0.2         0.0           0.3        0.3   \n",
       "metalcore            0.0      0.0         0.0           0.0        0.0   \n",
       "metal                0.0      0.0         0.0           0.0        0.0   \n",
       "classic_metal        0.0      0.0         0.0           0.0        0.0   \n",
       "grunge               0.0      0.0         0.0           0.0        0.2   \n",
       "hiphop               0.0      0.1         0.4           0.0        0.0   \n",
       "poppunk              0.0      0.0         0.0           0.4        0.5   \n",
       "classic              0.0      0.0         0.0           0.0        0.0   \n",
       "\n",
       "               soft_rock  metalcore  metal  classic_metal  grunge  hiphop  \\\n",
       "clubbeats            0.0        0.0    0.0            0.0     0.0     0.0   \n",
       "softpop              0.2        0.0    0.0            0.0     0.0     0.1   \n",
       "electropop           0.0        0.0    0.0            0.0     0.0     0.4   \n",
       "rockclassics         0.3        0.0    0.0            0.0     0.0     0.0   \n",
       "rockhymns            0.3        0.0    0.0            0.0     0.2     0.0   \n",
       "soft_rock            1.0        0.0    0.0            0.0     0.0     0.0   \n",
       "metalcore            0.0        1.0    0.7            0.6     0.0     0.0   \n",
       "metal                0.0        0.7    1.0            0.8     0.5     0.0   \n",
       "classic_metal        0.0        0.6    0.8            1.0     0.5     0.0   \n",
       "grunge               0.0        0.0    0.5            0.5     1.0     0.0   \n",
       "hiphop               0.0        0.0    0.0            0.0     0.0     1.0   \n",
       "poppunk              0.0        0.0    0.6            0.4     0.0     0.0   \n",
       "classic              0.0        0.0    0.0            0.0     0.0     0.0   \n",
       "\n",
       "               poppunk  classic  \n",
       "clubbeats          0.0      0.0  \n",
       "softpop            0.0      0.0  \n",
       "electropop         0.0      0.0  \n",
       "rockclassics       0.4      0.0  \n",
       "rockhymns          0.5      0.0  \n",
       "soft_rock          0.0      0.0  \n",
       "metalcore          0.0      0.0  \n",
       "metal              0.6      0.0  \n",
       "classic_metal      0.4      0.0  \n",
       "grunge             0.0      0.0  \n",
       "hiphop             0.0      0.0  \n",
       "poppunk            1.0      0.0  \n",
       "classic            0.0      1.0  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get all playlist-names from our dataset\n",
    "playlist_names = [pl[\"name\"] for pl in playlists]\n",
    "\n",
    "# create the lookup-table\n",
    "playlist_similarities = pd.DataFrame(np.zeros((len(playlist_names),len(playlist_names))), \n",
    "                                     index   = playlist_names, \n",
    "                                     columns = playlist_names)\n",
    "\n",
    "# self-similarity\n",
    "for i in range(len(playlist_names)):\n",
    "    for j in range(len(playlist_names)):\n",
    "        if i == j:\n",
    "            playlist_similarities.iloc[i,j] = 1.0\n",
    "\n",
    "# genre-similarities\n",
    "for s in sim:\n",
    "    playlist_similarities.loc[s[0][0],s[0][1]] = s[1]\n",
    "    playlist_similarities.loc[s[0][1],s[0][0]] = s[1]\n",
    "\n",
    "# show results\n",
    "playlist_similarities"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train network with prior knowledge\n",
    "\n",
    "With this lookup-table we can create more accurate input pairs. Insted of similar/dissimilar we can now apply the supplied similarites:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_pairs_with_sims(feature_data, metadata, num_pairs_per_track, playlist_similarities):\n",
    "    \n",
    "    data_pairs = []\n",
    "    labels     = []\n",
    "    \n",
    "    for row_id, q_track in metadata.sample(frac=1).iterrows():\n",
    "        \n",
    "        for _ in range(num_pairs_per_track):\n",
    "            \n",
    "            # search similar and dissimilar examples\n",
    "            pos_example = metadata[metadata.playlist == q_track.playlist].sample(1)\n",
    "            neg_example = metadata[metadata.playlist != q_track.playlist].sample(1)\n",
    "\n",
    "            # create feature pairs\n",
    "            \n",
    "            # similar\n",
    "            data_pairs.append([feature_data[[row_id]][0], feature_data[[pos_example.index]][0]])\n",
    "            labels.append(playlist_similarities.loc[q_track.playlist, pos_example.playlist])\n",
    "\n",
    "            # dissimilar\n",
    "            data_pairs.append([feature_data[[row_id]][0], feature_data[[neg_example.index]][0]])\n",
    "            labels.append(playlist_similarities.loc[q_track.playlist, neg_example.playlist])\n",
    "\n",
    "    return np.array(data_pairs), np.array(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute the function to prepare the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(21200, 2, 69)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get pairs\n",
    "data_pairs, labels = create_pairs_with_sims(feature_data, metadata, 10, playlist_similarities)\n",
    "\n",
    "# check - how many instances have we created?\n",
    "data_pairs.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the Siamese Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the model\n",
    "model_sim = create_siamese_network(data_pairs[:,0].shape[1:])\n",
    "\n",
    "# define the optimizer\n",
    "opt = Nadam(lr=0.001)\n",
    "\n",
    "# compile the model\n",
    "model_sim.compile(loss      = contrastive_loss, \n",
    "                  optimizer = opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model on the adapted data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHGRJREFUeJzt3X2QJGWB5/Hvk5lV3V3VDTNSDEPzqgwnCHHi6h0TwcbpDg4yojsC4XOw3B56HCMR7oriaYgaB3FIcH8ssBi7d8asL7icq/vcATII6E6AhGjo4OIJyI6eiC8MzAz0DMO8dtdLPvdHZnX39HRPV9dMdWd2/j4RSWVVZWY9T9fwyyefynzSeO8REZH8CBa6ACIiMjcKbhGRnFFwi4jkjIJbRCRnFNwiIjmj4BYRyRkFt4hIzii4RURyRsEtIpIzUY+2q8sxRUTmznSyUK+Cm5dffrmr9Wq1GiMjI0e5NPlQ5LpDseuvuhez7jBR/+Hh4Y7XUVeJiEjOKLhFRHJGwS0ikjM96+MWkWLx3jM6OkocxxjT0W9sAGzfvp2xsbEeliw7vPcEQUB/f/+c/kZTKbhF5KgYHR2lVCoRRXOLlSiKCMOwR6XKnmazyejoKAMDA11vQ10lInJUxHE859AuoiiKiOP4iLah4BaRo+JIDv2L5kj/VpkJbu898Xe+xdj//clCF0VEJNMyE9zGGPz37qf+MwW3iHTnzDPPXOgizIvMBDcAlUHivXsWuhQiIpmWveDep+AWkSPjveeWW25h1apVXHjhhTzwwANAcurhZZddxurVq1m1ahWbNm2i1Wrx8Y9/fHzZ9evXL3DpZ5etn4Crg3i1uEVyL/7W3+Ff/G1nyxqD97OPS2dOeSPBFdd2tM2HH36Y5557jo0bN7Jz507e+973snLlSu6//37e+c53cv3119NqtThw4ADPPfcc27Zt47HHHgPg9ddf7+gzFlLGWtxV4r27F7oUIpJzTz75JB/4wAcIw5Djjz+elStX8vTTT3PeeefhnOP2229n8+bNDA4Ocuqpp/KHP/yBz3/+83z/+99naGhooYs/q0y1uE1lEL/v+c7GNRSRzOq0ZQzJec3NZvOofv5MLfiVK1dy77338uijj3L99ddz3XXX8cEPfpCNGzfy+OOPc/fdd/Pggw9yxx13HNXyHG3ZanFX9eOkiBy5lStXsmHDBlqtFjt27GDTpk2cd955bNmyhVqtxlVXXcUVV1zBs88+y86dO4njmEsuuYRPfepTPPvsswtd/FllqsVNZRDqY/hGA1MqLXRpRCSn1qxZw1NPPcXq1asxxvC5z32OZcuW4ZzjS1/6ElEUUa1Wueuuu9i6dSs33HDD+NWMN9544wKXfnamkx8FuuC7uZFC/P2H8f/wJYK/+jrm2KU9KFa2aUD54tZ/MdR9//79VCqVOa/Xi66SrJv8t5pyI4Wjdwcca+3vgD1AC2g6597RVWlnU6kmj/v3QgGDW0SkE3PpKvkT51xPmwSmOpjcrHLf3l5+jIhIrmXrx8nKYPK4X8EtIjKTToPbA/9krX3KWruuZ6VJg9sruEVEZtRpV8kFzrmXrbXLgI3W2l86534weYE00NcBOOeo1WpzLkxcjngVGDRQ6WL9vIuiqKu/22JR5Povhrpv37696/G4izaOd19f3/j33c13P+ezSqy1NwN7nXN/dZjFujqrxLdaxNddivnTPyN4/xVzXj/vFsOZBUeiyPVfDHXXWSWdO9KzSmbtKrHWVq21Q+154CLgF12X+DBMGGIGKurjFhE5jE76uE8AfmitfRp4EnjIOffdXhXIDA7prBIR6bnDjd394osvsmrVqnkszdzM2rHknHsBeOs8lAWAoHoMTbW4RURmlLlfBMzgkLpKRHLuy/+8nd++NtrRsqbDYV3fuLSf//yOE2Z8/9Zbb+Wkk07iQx/6EAC33347xhh+8pOf8Prrr9NsNvn0pz/Ne97zno7K1TY6OsqNN97IM888QxiG3HTTTVxwwQX86le/4oYbbqBer+O9Z/369SxfvpyPfOQjbN26lTiOuf7661m7du2cPq8TmQvuoDoEr+1Y6GKISM6sXbuWm266aTy4H3zwQb7xjW9w7bXXMjQ0xM6dO3n/+9/PRRddNKeb9d59990APProozz//PNceeWVPPHEE9xzzz1cc801XHbZZdTrdVqtFo899hjLly/nnnvuAWD37t4MU5254FYft0j+Ha5lPNXROqvk3HPPZWRkhG3btrFjxw6OPfZYli1bxs0338ymTZswxrBt2zZeffVVli1b1vF2f/rTn/LhD38YgBUrVnDyySfzwgsv8Pa3v50vfvGLbN26lTVr1vCmN72Js846i1tuuYVbb72Vd7/73Zx//vlHXK/pZOvKSSBQV4mIdOmSSy7hoYceYsOGDaxdu5b77ruPHTt28Mgjj7Bx40ZqtRpjY2Nz2uZM3TiXXnopX/va1+jv7+eqq67ihz/8IWeccQaPPPIIZ511Frfddht33nnn0ajWITIX3KY6lAzt2mwsdFFEJGfWrl3LAw88wEMPPcQll1zCnj17qNVqlEolfvSjH7Fly5Y5b/P888/n/vvvB+A3v/kNL730EmeccQa///3vOe2007jmmmtYvXo1mzdvZtu2bQwMDHD55Zdz3XXX9Wxs78x1lQSD6W2D9u+FYzRCoIh07s1vfjP79u1j+fLlnHDCCVx22WVcffXVrFmzhnPOOYcVK1bMeZtXX301n/nMZ7jwwgsJw5A777yTvr4+NmzYwH333UcURSxbtoxPfOITPP3003zhC1/AGEOpVOK2227rQS0zNh43QHXzz9h9x80E/+1/YE48+SgXK9sWw9VzR6LI9V8MddeVk53r+ZWT8y2oHpPMqJ9bRGRa2e4qERHpoc2bN/Oxj33soNf6+vr4zne+s0Al6kzmgttUk+D2+/bqbu8iOdKjbteeOvvss9m4ceO8f+6R/q2y11WiFrdILgVBULi+6m40m02C4MiiN7Mtbl2EI5Iv/f39jI6OMjY2NqcrE/v6+uZ8bnVeee8JgoD+/v4j2k72gjuKoG9ALW6RnDHGMDAwMOf1FsMZNfMtc10lAFSranGLiMwgm8FdGdR9J0VEZpDZ4FZXiYjI9DIc3PsWuhQiIpmUyeA26uMWEZlRJoNbXSUiIjPLbnBraFcRkWllM7irg8mjWt0iIofIZnBX0uDepx8oRUSmymRwm4pa3CIiM8lkcFOpJo8KbhGRQ2QzuNM+bq9TAkVEDpHN4FZXiYjIjBTcIiI5k8ngToZ27ddZJSIi0+h4PG5rbQj8M/CSc+59vStSSldPiohMay4t7uuBzb0qyCEqVQ3tKiIyjY6C21p7MnAJ8OXeFmeSqlrcIiLT6bTF/dfAp4G4h2U5WGVQIwSKiExj1j5ua+37gFecc09Za991mOXWAesAnHPUarXuChRF1Go1Xn/DcdRf/G3X28mjdt2Lqsj1V92LWXforv7Ge3/YBay1twF/DjSBfuAY4D7n3H84zGr+5ZdfnlNB2to3Do3/8Sv4J75H+Deuq+3kUdFvmlrk+qvuxaw7TNR/eHgYwHSyzqwtbufcjcCNAGmL+7/MEtpHR7UKY6P4ZjM5PVBERICMnscN6CIcEZEZzKkp65x7HHi8JyWZanJwH7NkXj5SRCQPMtviNu2bKejMEhGRg2Q2uCda3LrsXURksswHt66eFBE5WHaDu6qbKYiITCe7wV1RH7eIyHQyG9wmKkG5Ty1uEZEpMhvcAFSHFNwiIlNkO7grVbxupiAicpBsB7eGdhUROUS2g1t3wREROUSmg9souEVEDpHp4E5upqA+bhGRybId3NUqjB3AN5sLXRIRkczIdnC3L8I5oFa3iEhbPoJbV0+KiIzLdHCPD+2qHyhFRMZlOrh1FxwRkUPlIri9ukpERMZlO7jHh3bVj5MiIm3ZDm51lYiIHCLTwa2hXUVEDpXp4AbSqycV3CIibdkP7uqg7jspIjJJ9oO7UtWPkyIik+QguNVVIiIyWeaDOxnadc9CF0NEJDMyH9xUNbSriMhk2Q/uyqCGdhURmSQfwQ0a2lVEJBXNtoC1th/4AdCXLv9/nHM39bpg49qXve/bC0PHztvHiohkVSct7jFglXPurcB5wMXW2pW9LdYEo8veRUQOMmuL2znngXZqltLJ97JQB1Fwi4gcZNbgBrDWhsBTwArgb51zm3paqsmqE0O7mnn7UBGR7DLed954ttYuAe4H/tI594sp760D1gE4595er9e7KlAURTQnnUHSem0HI//p/Qyt+ySVNZd3tc28mFr3oily/VX3YtYdJupfLpeBztqnHbW425xzu6y1jwMXA7+Y8t56YH361I+MjMxl0+NqtRqT1/WNBgB7X9nO/i63mRdT6140Ra6/6l7MusNE/YeHhzteZ9YfJ621x6ctbay1A8C7gV92Xco5MqUSlMvq4xYRSXXS4j4R+Hrazx0Azjn3nd4WawqNVyIiMq6Ts0qeAd42D2WZWUVDu4qItGX/yklIWtwa2lVEBMhLcFfVVSIi0paL4DaVqn6cFBFJ5SK4k64SBbeICOQpuEcP4FuthS6JiMiCy0dwV9vjlegHShGRfAS3BpoSERmXi+DW0K4iIhNyEdwH3UxBRKTg8hHcaYtbV0+KiOQsuNVVIiKSl+Bun1WirhIRkXwEtymVoVTW6YAiIuQkuAFdPSkikspRcFf146SICHkK7uqQ+rhFRMhVcKurREQEchTcydCu+nFSRCQ3wa0fJ0VEEvkK7gP78bGGdhWRYstPcGtoVxERIE/BrcveRUSAHAX3+NCu+9TiFpFiy01wjw/tqha3iBRcfoJbQ7uKiAA5DG5dPSkiRZef4K7qx0kREchRcE8M7argFpFiy01wA+nVkzqrRESKLZptAWvtKcDfA8uBGFjvnLur1wWbVqWKVx+3iBRcJy3uJvBJ59zZwErgo9bat/S2WDPQCIEiIrMHt3Nuq3PuZ+n8HmAzcFKvCzYtDTQlIjJ7V8lk1trTgbcBm6Z5bx2wDsA5R61W665AUTTjuq+/4Tjq27Z0ve2sO1zdi6DI9Vfdi1l36K7+HQe3tXYQuBf4uHNu99T3nXPrgfXpUz8yMjKngrTVajVmWjcOS/g9u2d8P+8OV/ciKHL9Vfdi1h0m6j88PNzxOh2dVWKtLZGE9jecc/d1Wb4jV6nCgX0a2lVECm3W4LbWGuArwGbn3B29L9JhtK+ePLB/QYshIrKQOukquQD4c+BZa+3P09c+65x7uHfFmsHky96rQ/P+8SIiWTBrcDvnfgiYeSjLrEx1EA86s0RECi1/V06CgltECi2Xwa2rJ0WkyPIV3O2bKSi4RaTA8hXc6ioREclXcJtyH0QlBbeIFFqughtIB5rS0K4iUlz5C+7KoH6cFJFCy2FwV9VVIiKFlsPg1tCuIlJsuQtuUx3U6YAiUmi5C27dd1JEii6fwa2hXUWkwPIX3O2rJzW0q4gUVP6Ce/LQriIiBZS74Da67F1ECi53wa3xSkSk6PIX3NX20K46s0REiil/wa0Wt4gUnIJbRCRn8hfc5TJEkc4qEZHCyl1wG2M0XomIFFrughtIhnZVcItIQeUzuHUzBREpsHwGd0UjBIpIceUyuI1upiAiBZbL4KY6pOAWkcLKZ3BXBuHAfnwcL3RJRETmXT6Du1oF7zW0q4gUUjTbAtbarwLvA15xzp3b+yJ1YPLVk+nYJSIiRdFJi/tu4OIel2NONLSriBTZrMHtnPsBsHMeytI53UxBRAosp33canGLSHHN2sfdKWvtOmAdgHOOWq3WXYGiaNZ1W8YzAlQDqHT5OVnUSd0XsyLXX3UvZt2hu/ofteB2zq0H1qdP/cjISFfbqdVqzLauH6sDsHf7dvZ3+TlZ1EndF7Mi1191L2bdYaL+w8PDHa+Tz66S9tCu6ioRkQKaNbittd8Efgy82Vq7xVp7Te+LdXga2lVEimzWrhLn3JXzUZA500BTIlJQ+ewqAahqTG4RKab8BndFY3KLSDHlNrg1tKuIFFVug1t93CJSVPkN7uogHNinoV1FpHDyG9yVwWRo11EN7SoixZLv4AZ1l4hI4eQ2uE21mszozBIRKZjcBjcak1tECkrBLSKSM7kPbq8+bhEpmPwGt26mICIFld/gLvdBqKFdRaR4chvcydCuVdins0pEpFhyG9xA0l2iFreIFEy+g7syiN+3Z6FLISIyr3If3LoAR0SKJlPBvfnV/Yw1Wx0vb3T7MhEpoKN2l/cjta/e4ubHtjDwxFb+9M1LuPhfLaFSCg+/UrWqsUpEpHAy0+KulkP+67tOZsXxVb7+81e59tu/4ZvPvMqescO0wCsa2lVEiiczLW6Ac06o8M5zTuXHv3qR//2LHXzr2R18e/NrrDlzCWvPfgNLB6YUd/LQru1L4EVEFrlMBXfbmccN8Nl3nszvXhvl3ud28sAvd/LQ/3uN1Wccy6VvOY7jq6VkweqkoV0V3CJSEJkM7rbTl/bzyT8e5srdNe79lx1899e7+O6vd/EnbzqWy99yHCdWBvGA//b/wp/1rzGnrYDhUzFRpqslInJEcpFww8eU+cuVJ/Lvz61x/+YdbHz+dR574XUuOPEE1r51Fac9+yThkz/AA0QlOOWNmNPPhNNXYE47E048CRPM8kOniEhO5CK425YNlvjIv1mOPbfGA5t38sivd/HE0osx51/MUMmwxDRY0tjLkn07OfalbSx9/ucsqT/BkniUJbWlLB1eztBppxENnwIDA9A/AH0DUO5LLqEXEcmBXAV329KBiA/90TIuP+c4fvziHnbsb/DagRa7RpvsGh3kl5Xj2TV0BvWWP3jFFgS/ian8ahdRvIOSbxLFLSLfIiImwlMyEBlPZCAKDKXAEIWGUhBQCk0yRQGlMKAchZRLIaVSNDGVS5TKZUp9JcIwIowCgjAkjELCMCQ0hiAgeTQQGEMYGEID5dEmY82YUmgItCMRkRnkMrjbhvpCLlqxZNr3vPccaMbsOtDitdEmu0abvLavwa6RnezdU6fZNMnUCmi2QhoxNFsxTQ9jMezz0PSk7xsaJkymIKIeRMRmuq4XD9TTqRvPj89FvkXkY8rElNqT8ZMmKAVQbj8G6WMYUApIdiyhSXcwhlIYUo4CSunOphxFlMsh5VKUTH0R5VKZvv4S5VKkHYdIhuU6uA/HGEOlFFIphQwfU570znFdb9N7D60m1MdojY7SGBujMTpGY7ROY2yM+liTer1Oo96g0WgSt1q04phW7IlbMa04Jo49cRzTiiH2yXutlif2Hh+EjNabNGJo+GSqe0PTG+oYGj6gYQIaBDRMyAET0Qyi8Z1JI4hoBCH1oEQzmOmrjdOpcdi6luIG5bhJKW5S8jG0c9wYfPuJSf8zOeTT+RBPiWTHUzbp/KTHZN5TDiZ2ROW+PhrNJpgAApM8GgNBMGm+/XqyTGgMYRgQBIYwCJKjlzAgCALCMJ2C9lFPQGACgvT95DHEtJdJj4CC9GPa82aa10Jj0uKYKeul7wfa8UnvLNrg7gVjTPLjZ1QiqgwSAQNHcfu1Wo2RkZGOl/etFjSb0GrA+HwTmk3iZpNGo0Gj2aJeb9FoNmk0WtSbLerNmLH0sd6Mqbdi6i2fTLGn3oJ6nE4eGjEY78HHyXnz3kPcnk8eTfu9OHmthaGeHqXUTUTDhOxvzwdR8t74TqdEbILuD1QO/qsArXRaOMZ7QmJCPKFPH4kJvSdqv86kZYxJ/n4wsSMc3zEmj2bKazGGFuaQxxaG2Jv0rzDxXuyTnU/yedM8Gk9IskwEBGmXYcmQ7GADDjrSi9pHesZQCttHfSbduYM3ATHg08/3GLwx468l84b+/pcYGxudtCNMuhMNhiBInrd3nibdwZJuJ/aG2CR1a9cx+aykedLy7eWSP6sxJrnq0EBAsvNNNzd+lJlWATPp/cAcvGzymJYpnTdAOTKcf/JQL/9pAR0Gt7X2YuAuIAS+7Jz77z0tlXTEhCGEIdB3yHthOvXPd6G64L2n2YqpLV3KyCuvQNxKplYymUnz4+/FLXwzTo5eWi1azRZx3KLV9LRazeRIpxmnRzwxcSum2fL4OCb27SOf5OgnjpMjnjiO8e157yeW8T4JBO+J43YgePz464yHQ3tqmSQwmwTj4dk6aN7QNMH4a94EeB8n+x18+w8z/ugn/ljgwbd3CL5F4GMCH6fPk2nieWt8PvCelgnGp9iEk56n88HEfGwCGiZiNGjvZEs0gpBGUKIeRDRNRD0szfXbnvIIsHhG+FzSOsD5//FtPf+cWYPbWhsCfwusBrYAP7XWbnDO/UuvCyfFYIyhFIWU+sqUBvKwqzn65nq0NZVvH/3E/qAjoYlp6nMPpHuZZE+QLkP6ejqMRPu1KUdU7W36OKYZexqtZKq3kuUDYoz3BD55NKTzeILYJ7uu2BN4z+DgILt37yb2cXow54lJdpKe9s6TSTvK5LOD9HPaO64g/bzAxwR4grg18VqctPN97JNtpnWJ052gj5MdcVLV5PO99+mfKnk9ThZOujVpr5fstJPlYoL+ASADwQ38W+B559wLANbabwFrAQW3SEYYY8CE8z76kAHK6dStY2o16kew0yqiToL7JODFSc+3AOdPXchauw5YB+Cco1ardVegKOp63bwrct2h2PVX3YtZd+iu/p0E93Q/j/upLzjn1gPr2+93e9h3pIeMeVbkukOx66+6F7PuMFH/4eHhjtfp5MBqC3DKpOcnAy/PsWwiInKUdNLi/ilwprX2jcBLwBXAn/W0VCIiMqNZW9zOuSbwF8D3gM3JS+65XhdMRESm19F53M65h4GHe1wWERHpQGZuXSYiIp1RcIuI5Izx/pAz+46GnmxURGSR62h0sl61uE23k7X2qSNZP89Tkete9Pqr7gtfjozUvyPqKhERyRkFt4hIzmQxuNfPvsiiVeS6Q7Hrr7oX15zr36sfJ0VEpEey2OIWEZHDyMyty4p+lx1r7e9IbgXSAprOuXcsbIl6x1r7VeB9wCvOuXPT194A/CNwOvA7wDrnXluoMvbSDPW/GbgWeDVd7LPpFcuLirX2FODvgeUk9yBY75y7qwjf/2HqfjNz/O4z0eKedJedNcBbgCuttW9Z2FItiD9xzp23mEM7dTdw8ZTXPgM86pw7E3g0fb5Y3c2h9Qe4M/3+z1uMoZ1qAp90zp0NrAQ+mv6/XoTvf6a6wxy/+0wEN5PusuOcqwPtu+zIIuSc+wGwc8rLa4Gvp/NfBz4wr4WaRzPUvxCcc1udcz9L5/eQDFx3EgX4/g9T9znLSnBPd5edriqUYx74J2vtU+ndhIrmBOfcVkj+gQPLFrg8C+EvrLXPWGu/aq1dutCF6TVr7ekkN2jcRMG+/yl1hzl+91kJ7umuGCra6S4XOOf+iKS76KPW2n+30AWSefU/gTOA84CtwO0LW5zestYOAvcCH3fO7V7o8synaeo+5+8+K8Fd+LvsOOdeTh9fAe4n6T4qku3W2hMB0sdXFrg888o5t90513LOxcDfsYi/f2ttiSS4vuGcuy99uRDf/3R17+a7z0pwj99lx1pbJrnLzoYFLtO8sdZWrbVD7XngIuAXC1uqebcBuDqdvxp4YAHLMu/aoZW6lEX6/VtrDfAVYLNz7o5Jby3673+munfz3WfmAhxr7XuBvyY5HfCrzrlbF7hI88Za+yaSVjYkp2j+w2Kuv7X2m8C7gBqwHbgJ+DbggFOBPwAfdM4tyh/wZqj/u0gOlT3J6XAfaff5LibW2j8GngCeJTklDuCzJH29i/r7P0zdr2SO331mgltERDqTla4SERHpkIJbRCRnFNwiIjmj4BYRyRkFt4hIzii4RURyRsEtIpIzCm4RkZz5/4mY19hYu73FAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f23c44dbeb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_sim.fit([data_pairs[:, 0], data_pairs[:, 1]], \n",
    "               labels, \n",
    "               batch_size       = 24, \n",
    "               verbose          = 0, \n",
    "               epochs           = 25, \n",
    "               callbacks        = [tut_func.PlotLosses()], \n",
    "               validation_split = 0.1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check the results for some individual tracks by supplying the index to our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "track_id                                  637McrDyDQ9CffkFuZGRpq\n",
      "artist_name                                                KSHMR\n",
      "title                                                    Kolkata\n",
      "album_name                                               Kolkata\n",
      "label                                                     Dharma\n",
      "duration                                                  206250\n",
      "popularity                                                    54\n",
      "year                                                        2017\n",
      "genres         [big room, deep big room, edm, electro house, ...\n",
      "playlist                                               clubbeats\n",
      "Name: 33, dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_name</th>\n",
       "      <th>title</th>\n",
       "      <th>album_name</th>\n",
       "      <th>year</th>\n",
       "      <th>playlist</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>KSHMR</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>David Tort</td>\n",
       "      <td>Yo Te Prefiero (feat. Dennisse Jackson)</td>\n",
       "      <td>Yo Te Prefiero (feat. Dennisse Jackson)</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Noise Cans</td>\n",
       "      <td>No War (feat. Jesse Royal) - Yellow Claw Remix</td>\n",
       "      <td>No War (feat. Jesse Royal) [Yellow Claw Remix]</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Monik</td>\n",
       "      <td>Palkia</td>\n",
       "      <td>Palkia</td>\n",
       "      <td>2016</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>Hardwell</td>\n",
       "      <td>Safari</td>\n",
       "      <td>Safari</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>Dropgun</td>\n",
       "      <td>Little Drop</td>\n",
       "      <td>Little Drop</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Dylan Taylor</td>\n",
       "      <td>Frankenstein</td>\n",
       "      <td>Frankenstein</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>J Balvin</td>\n",
       "      <td>Mi Gente - Hardwell &amp; Quintino Remix</td>\n",
       "      <td>Mi Gente (Hardwell &amp; Quintino Remix)</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>304</th>\n",
       "      <td>The Him</td>\n",
       "      <td>Everybody Hurts</td>\n",
       "      <td>Everybody Hurts</td>\n",
       "      <td>2017</td>\n",
       "      <td>electropop</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>TV Noise</td>\n",
       "      <td>Milkshake</td>\n",
       "      <td>Milkshake</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      artist_name                                           title  \\\n",
       "33          KSHMR                                         Kolkata   \n",
       "58     David Tort         Yo Te Prefiero (feat. Dennisse Jackson)   \n",
       "26     Noise Cans  No War (feat. Jesse Royal) - Yellow Claw Remix   \n",
       "9           Monik                                          Palkia   \n",
       "36       Hardwell                                          Safari   \n",
       "37        Dropgun                                     Little Drop   \n",
       "32   Dylan Taylor                                    Frankenstein   \n",
       "3        J Balvin            Mi Gente - Hardwell & Quintino Remix   \n",
       "304       The Him                                 Everybody Hurts   \n",
       "48       TV Noise                                       Milkshake   \n",
       "\n",
       "                                         album_name  year    playlist  \n",
       "33                                          Kolkata  2017   clubbeats  \n",
       "58          Yo Te Prefiero (feat. Dennisse Jackson)  2018   clubbeats  \n",
       "26   No War (feat. Jesse Royal) [Yellow Claw Remix]  2017   clubbeats  \n",
       "9                                            Palkia  2016   clubbeats  \n",
       "36                                           Safari  2018   clubbeats  \n",
       "37                                      Little Drop  2017   clubbeats  \n",
       "32                                     Frankenstein  2018   clubbeats  \n",
       "3              Mi Gente (Hardwell & Quintino Remix)  2017   clubbeats  \n",
       "304                                 Everybody Hurts  2017  electropop  \n",
       "48                                        Milkshake  2018   clubbeats  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar(model_sim, 33)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "General model evaluation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluate(lambda x,y: [model_sim.predict([x[[i]], y]) for i in range(feature_data.shape[0])], 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Discussion of Results\n",
    "\n",
    "Although the performance of the evaluation has dropped significantly, the ranked result-lists seem to be more appropriate. From an acoustic similarity point of view, the improved model seems to fit better to track-based music similarity."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Improve Performance through Identity\n",
    "\n",
    "So far we have taught the network what is similar and what not, but we have not shown it, what is identical. All input pairs created so far missed to pass identical data. In the following step, we will include identical pairs into the training instances. To emphasis the identity, only identical pairs will be assigned a label of 1. All other similarity values of the lookup-table will be decreased by 0.1. Thus, tracks of the same playlist will have a similarity value 0f 0.9."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_pairs_with_sims_and_identity(feature_data, metadata, num_pairs_per_track, playlist_similarities):\n",
    "    \n",
    "    data_pairs = []\n",
    "    labels     = []\n",
    "    \n",
    "    for row_id, q_track in metadata.sample(frac=1).iterrows():\n",
    "        \n",
    "        # identical pair\n",
    "        data_pairs.append([feature_data[[row_id]][0], feature_data[[row_id]][0]])\n",
    "        labels.append(1)\n",
    "        \n",
    "        for _ in range(num_pairs_per_track):\n",
    "            \n",
    "            # search similar and dissimilar examples\n",
    "            pos_example = metadata[metadata.playlist == q_track.playlist].sample(1)\n",
    "            neg_example = metadata[metadata.playlist != q_track.playlist].sample(1)\n",
    "\n",
    "            # create feature pairs\n",
    "            \n",
    "            # similar pair\n",
    "            data_pairs.append([feature_data[[row_id]][0], feature_data[[pos_example.index]][0]])\n",
    "            sim_val = playlist_similarities.loc[q_track.playlist, pos_example.playlist] - 0.1\n",
    "            labels.append(np.max([0, sim_val]))\n",
    "\n",
    "            # dissimilar pair\n",
    "            data_pairs.append([feature_data[[row_id]][0], feature_data[[neg_example.index]][0]])\n",
    "            sim_val = playlist_similarities.loc[q_track.playlist, neg_example.playlist] - 0.1\n",
    "            labels.append(np.max([0, sim_val]))\n",
    "\n",
    "    return np.array(data_pairs), np.array(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute the function to prepare the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pairs, labels = create_pairs_with_sims_and_identity(feature_data, metadata, 10, playlist_similarities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the model\n",
    "model_sim_id = create_siamese_network(data_pairs[:,0].shape[1:])\n",
    "\n",
    "# define the optimizer\n",
    "opt = Nadam(lr=0.001)\n",
    "\n",
    "# compile the model\n",
    "model_sim_id.compile(loss      = contrastive_loss, \n",
    "                  optimizer = opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model on the adapted data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAG2pJREFUeJzt3X+MHOWd5/H3U109PdPdBpttjGMwYLAFBKR1NqvgE6tNDmKCMTknjvIIlpwIx2GQshcT9hLFSSRYEcTd6YCQ1e5y3vwgy2U39+iAw/zarMUPQdDhsOQOCOfLLcEJAWyDf+Df0zPd9dwfVT0zHs8wPW33dNXU5yWNqn9U9XyfbvtTzzxd9ZTx3iMiItkR9LoAERGZHgW3iEjGKLhFRDJGwS0ikjEKbhGRjFFwi4hkjIJbRCRjFNwiIhmj4BYRyZiwS6+r0zFFRKbPtLNSt4Kbd955p6PtarUaO3fuPM7VZEOe2w75br/ans+2w2j7Fy5c2PY2GioREckYBbeISMYouEVEMqZrY9wiki/eewYHB4miCGPa+o4NgB07dlCv17tYWXp47wmCgP7+/mm9R+MpuEXkuBgcHKRYLBKG04uVMAwpFApdqip9Go0Gg4ODDAwMdPwaGioRkeMiiqJph3YehWFIFEXH9BoKbhE5Lo7lT/+8Odb3KjXB7b0nevQn1P/XC70uRUQk1VIT3MYY/E8fYugXCm4R6czSpUt7XcKMSE1wA1CuEh3Y3+sqRERSLX3BfVDBLSLHxnvPbbfdxsUXX8wll1zCww8/DMSHHq5Zs4YVK1Zw8cUXs3nzZprNJjfddNPIuhs2bOhx9VNL11fAlSpePW6RzIt+8jf4321tb11j8H7qeenMosUEV17f1ms+/vjjvPbaa2zatIndu3dz+eWXs3z5ch566CE+/vGPs27dOprNJocPH+a1115j+/btPPXUUwDs3bu3rd/RSynrcVeIDuzrdRUiknE///nP+cxnPkOhUODkk09m+fLlvPzyyyxbtgznHHfeeSdbtmyhWq1y+umn8+abb/Ktb32Lp59+mjlz5vS6/CmlqsdtylX8wdfbm9dQRFKr3Z4xxMc1NxqN4/r7J+vBL1++nAceeIAnn3ySdevWceONN/L5z3+eTZs28cwzz3DffffxyCOPcNdddx3Xeo63dPW4K/pyUkSO3fLly9m4cSPNZpNdu3axefNmli1bxltvvUWtVuPqq6/myiuv5NVXX2X37t1EUcSqVav46le/yquvvtrr8qeUqh435SoM1fHDw5hisdfViEhGrVy5kpdeeokVK1ZgjOGb3/wm8+fPxznHvffeSxiGVCoV7rnnHrZt28bNN988cjbj+vXre1z91Ew7Xwp0wHdyIYXo6cfxf3cvwX/+EebEeV0oK900oXx+2z8b2n7o0CHK5fK0t+vGUEnajX2vxl1Ioa2R4nQNlZQr8fLQwd7WISKSYqkKblOpxjcOHehtISIiKZaq4Kas4BYRmUrbX05aawvAPwFvO+eu6Eo1SXD7gwd0SKCIyCSm0+NeB2zpViEAaKhERGRKbQW3tfY0YBXwva5WM9D6clLBLSIymXZ73N8BvgYc22UbpmDCENNfhoM6qkREZDJTjnFba68A3nXOvWSt/cQHrLcWWAvgnKNWq3VU0M7qHIrRMCd2uH2WhWHY8fs2G+S5/bOh7Tt27Oj40mW9uOTZ4sWL2bp14omw3nzzTb7whS/w7LPPduV3l0qlkc+7k8++nXfrIuBfWWsvB/qBE6y1/9U594WxKznnNgCt+RB9pycTmOocBnfvYjjjJyN0YjachHEs8tz+2dD2er3e0UV/e3kCzmS/t9lsfuDzx6per4983uNOwGnLlMHtnFsPrAdIetz/fnxoH0+mXNUYt0jGfe+fdrB1z2Bb65o2p3VdPK+ff/uHp0z6/O23386pp57KF7/4RQDuvPNOjDG88MIL7N27l0ajwde+9jU+9alPtVVXy+DgIOvXr+eVV16hUChwyy23cNFFF/GrX/2Km2++maGhIbz3bNiwgQULFnDDDTewbds2oihi3bp1rF69elq/rx3pmqsECKpz4O03e12GiGTM6tWrueWWW0aC+5FHHuHHP/4x119/PXPmzGH37t18+tOf5tJLL53WxXrvu+8+AJ588klef/11rrrqKp577jnuv/9+rrvuOtasWcPQ0BDNZpOnnnqKBQsWcP/99wOwb193pqmeVnA7554BnulKJQlTnaNT3kUy7oN6xuMdr6GSCy64gJ07d7J9+3Z27drFiSeeyPz587n11lvZvHkzxhi2b9/Oe++9x/z589t+3RdffJFrr70WgCVLlnDaaafxxhtv8NGPfpTvfve7bNu2jZUrV3LWWWdx7rnnctttt3H77bfzyU9+kgsvvPCY2zWRdJ05CQSVORoqEZGOrFq1iscee4yNGzeyevVqHnzwQXbt2sUTTzzBpk2bqNVq1Ov1ab3mZMM4n/3sZ/nhD39If38/V199NT/72c84++yzeeKJJzj33HO54447uPvuu49Hs46SuuA21ROgPojP2WxhInLsVq9ezcMPP8xjjz3GqlWr2L9/P7VajWKxyPPPP89bb7017de88MILeeihhwD49a9/zdtvv83ZZ5/Nb3/7W8444wyuu+46VqxYwZYtW9i+fTsDAwN87nOf48Ybb+za3N7pHOOGuNd9wtzeFiMimXLOOedw8OBBFixYwCmnnMKaNWu45pprWLlyJeeffz5LliyZ9mtec801fP3rX+eSSy6hUChw9913UyqV2LhxIw8++CBhGDJ//ny+8pWv8PLLL/Ptb38bYwzFYpE77rijC61M2XzcAJXXXmLfd/6c4La/wiw47TiXlW6z4ZCwY5Hn9s+Gtms+7vbNrvm4GdPjPqhxbhGRiaRuqMRUWkMlOrJERLpry5YtfPnLXz7isVKpxKOPPtqjitqTuuBu9bj9IU3tKpIlXRp27arzzjuPTZs2zfjvPdb3KnVDJaZ6QnxDhwSKZEoQBLkbq+5Eo9EgCI4tetPX465ojFski/r7+xkcHKRer0/rzMRSqTTtY6uzyntPEAT09/cf0+ukLrhNsQh9JfW4RTLGGMPAwMC0t5sNR9TMtNQNlQDxJcz05aSIyITSGdyVKl49bhGRCaUzuMsV9bhFRCaR0uCu6stJEZFJpDK4dTEFEZHJpTK4qSi4RUQmk87gLldh8DA+ue6biIiMSm9wg76gFBGZQDqDu1KJlxouERE5SiqD24z0uBXcIiLjpTK4KSc9bh0SKCJylJQGd9zj1tmTIiJHS3Vwa6hERORo6QzuShLcGioRETlKKoPbFPug2KfDAUVEJpDK4AaSqV3V4xYRGS/FwV3Rl5MiIhNIb3BXNEOgiMhE0hvcGioREZlQaoPb6PJlIiITSm1wa2pXEZGJpTe4yxU4fAgfaWpXEZGxUhzcmtpVRGQiCm4RkYxJbXCbiuYrERGZSGqDWxNNiYhMLPXB7Q9qqEREZKz0BrcuXyYiMqFwqhWstf3As0ApWf+/O+du6XZhGioREZlYOz3uOnCxc+73gWXAZdba5d0tC0xfCcKi5isRERlnyh63c84DrfQsJj++m0WN0NmTIiJHmTK4Aay1BeAlYAnwl865zV2tqqVc1dSuIiLjGO/b7zxba+cCDwH/zjn3y3HPrQXWAjjnPjo0NNRRQWEY0mg0ANi9/gZMX4l5f/7djl4ra8a2PY/y3H61PZ9th9H29/X1AZi2tpnOL3DOvW+tfQa4DPjluOc2ABuSu37nzp3TeekRtVqN1rbNYgne30Onr5U1Y9ueR3luv9qez7bDaPsXLlzY9jZTfjlprT056WljrR0APgn8346rnAZTrmiMW0RknHZ63B8CfpSMcweAc8492t2yErqYgojIUdo5quQV4CMzUMvRKtVkatcIE6T3XCERkZmU7jQsV8F7OHyo15WIiKRG+oMbNFwiIjJGqoPbaL4SEZGjpDq4R3rcOu1dRGRENoJbPW4RkRGZCG6d9i4iMirdwd26fJkupiAiMiLdwd1XgkKooRIRkTFSHdzGGChXdKV3EZExUh3cgObkFhEZJ/3BrTm5RUSOkIng1nHcIiKjUh/cRjMEiogcIfXBTUVfToqIjJX+4C5X4dBBfBT1uhIRkVTIRnD7CAYP97oSEZFUSH9wVzRfiYjIWKkPbqOJpkREjpD64KaczMmtQwJFRIBMBHerx60jS0REIEPBrbMnRURi6Q9uXb5MROQI6Q/u0gAEgca4RUQSqQ/ueGpXnfYuItKS+uAGRs6eFBGRrAR3pYrXUImICJCV4C5XNFQiIpLIRHBralcRkVGZCG5dvkxEZFQ2grs1tav3va5ERKTnshPckaZ2FRGBzAR36+xJHRIoIpKJ4Daak1tEZEQmghvNyS0iMiJbwa2TcEREMhLcFU3tKiLSko3g1lCJiMiIcKoVrLWLgL8FFgARsME5d0+3CztCf2tqVx1VIiLSTo+7AfyZc+48YDnwJWvth7tb1pHiqV01X4mICLQR3M65bc65XyS39wNbgFO7XdhRNF+JiAgwzTFua+2ZwEeAzV2p5oOUq/pyUkSENsa4W6y1VeAB4Cbn3L4Jnl8LrAVwzlGr1TorKAwn3HbPiXPxhw9xUoevmwWTtT0v8tx+tT2fbYfO2m/ambjJWlsEHgV+6py7q43X9e+88860Cmmp1Wrs3LnzqMej//Kf8L/bSuHbf93R62bBZG3Pizy3X23PZ9thtP0LFy4EMO1sM+VQibXWAN8HtrQZ2t2hMW4REaC9oZKLgH8NvGqt/d/JY99wzj3evbImUImPKvHex0eZiIjk1JTB7Zz7GW1237uqXIVmE+qD8XHdIiI5lY0zJ0FnT4qIJDIT3JraVUQklpngHp0hUKe9i0i+ZS+41eMWkZzLUHDHly/T2ZMiknfZCe6KLqYgIgJZCu7+MhijoRIRyb3MBLcJAhio6ErvIpJ7mQluIB4uUY9bRHIuW8FdruLV4xaRnMtYcOsqOCIimQpuU67qqBIRyb1MBbfGuEVEshbcyZzc7Vz8QURktspecDcaMDTU60pERHomW8FdiU9713CJiORZpoLbaKIpEZFsBffo1K4KbhHJr4wFt4ZKREQyFtxxj1tTu4pInmUruHX5MhGRjAX3QDle6vJlIpJjmQpuExSSqV3V4xaR/MpUcAOaaEpEci97wV2p4nU4oIjkWPaCu6yJpkQk3zIa3PpyUkTyK3PBbTS1q4jkXOaCm3JFp7yLSK5lMLir0BjGD9V7XYmISE9kM7hB49wiklvZC26d9i4iOZe54Nac3CKSd5kL7tE5uTVUIiL5lL3gTi5fpqldRSSvshfcGioRkZzLYHAnV8HRsdwiklOZC+54ateyetwiklvhVCtYa38AXAG865y7oPsltUETTYlIjrXT474PuKzLdUxPuYLXCTgiklNTBrdz7llg9wzU0r5yVWPcIpJbmRvjBnQVHBHJtSnHuNtlrV0LrAVwzlGr1TorKAyn3HbvSTWGfvt6x78jrdpp+2yW5/ar7flsO3TW/uMW3M65DcCG5K7fuXNnR69Tq9WYatuoEOIP7Jtyvaxpp+2zWZ7br7bns+0w2v6FCxe2vU1Gh0qqMDSEHx7udSUiIjNuyuC21v498D+Bc6y1b1lrr+t+WVPQ2ZMikmNTDpU4566aiUKmpXX25KEDcOK83tYiIjLDMjlUYlpzcuuQQBHJoUwGt4ZKRCTPMh3cmtpVRPIom8Fd0cUURCS/shncA2O+nBQRyZlMBrcJQygNKLhFJJcyGdxAfAkzzRAoIjmU3eAuV/XlpIjkUqaDW0MlIpJH2Q5unYAjIjmU2eA2GuMWkZzKbHBrqERE8irbwV0fxDcava5ERGRGZTe4K5qvRETyKbvBrYmmRCSnMhvcpqypXUUknzIb3KMXU9CRJSKSLxkObk3tKiL5lN3grmiGQBHJp+wGt8a4RSSnMhvcJixCX0k9bhHJncwGN6CzJ0Ukl7Id3JUqXpcvE5GcyXZwlyvqcYtI7mQ8uDVUIiL5k+ngNgpuEcmhsNcFjHXn8+/w+4vqfOyUkBNKhak3qFRBY9wikjOpCe5Dw03ePTDMXzy3lWJguOiMOVy2dC7n1gYwxky8UbkK9cP4RiO+8ruISA6kJu3KxQL/8VNn8L4f4CcvbuWZrft4Zus+zpxb4rKlc/n44hMoF8f1wlsn4Rw+BHNOmPmiRUR6IHVj3EtOrnDjxxbwwzVL+NKFCwgM3PviDq598HX+avN23tg9OLqyTnsXkRxKTY97vIFiwKVL5rLi7BN5ffcgT/y/93l6615++vr7LP29fi5bOpc/Kp9AEYju/BacfhZm0WLMosWw6CyonTL5EIuISIalNrhbjDEs/b0Blv6LAf7NH8zn6a17+Yd/fp+/eGE73y/O4Y9XrmfR3reZ+96bzH3u58yrb2Le0D5KpT447UzMorPi5elnwcLTMcW+XjdJROSYpD64x6qWCnz63JO44px5/J93D/MP//w+T/7OM9w3D069AE4dXbdMg3nDB5i7Zw/ztu1m3vO/Ye7wAeaVi5wwp0x/WKC/r0CpL6S/GNJXKtLf30exVCLo74e+fiiVoJTc7uuDsAjFYrwMi5ggdSNNIpIDmQruFmMM559S5vxTykTes6/eZM/hxpifJnsGG+w5PI89hz/Erw/U2TPYZNBPMHQynPwciu8GPqLUHKLUHKI/ep9Sc5hSc4i+qEHoGxSj0Z+QiCKe0EQUgaLxhAaKgadoIAyIl8YQFqBoDMWCIQwMYcFQDALCMCAMAophwIE5FYbqQwSFgEIYUigUKIQBQRgSFEKCMIRCCIUCtG63lsXi6P0whEJrBxNCEGjYSGQWyWRwjxUYw9z+kLn9IYvnffC6h4cj9hxusH+oyWAjYrARUW94BoebDA4OUR8apl4fZnCoyOBwP/XhZrJOxFAEg5Gn4WE4goY3DHvDMPFPwwc0TAc98NZF6ocY2XlMphA1KfiIwEcUfIOCHyL0TfqiYYpRg76oQV9zmGI0HN8e+zhNikT0EREYMBhMAAEGExgMEAQGYwKMARMYAmMwyU8QBBSMoRAYgsAQFIL4sSB+LmgtCwUKhfh2GMTrhwEUku3DQrxNWDAUgoBiIaBQCCgEAfV5c/H790MQEBdXAGPi+60fE0xw34yuH0z0/Lj7xkDSLpEsynxwT8dAMWCgi2Pc3nsakWc48gw349uNMbeHx90feawZL0ulAfbt20fUbNJsRjSbTaJmRDOK4mWzQDOKaEaeqBnRiKKRbevN+HWHIqh7OBDBkIehyDBMQN3Hy+HjfSBRlPwcB2E0RBiFFH2DYjREGDUp+gZh1CSMGoS+Gf+lkywLvgkYIgwY8Jj4xxg8jLkd38cY8J6Cj+IdIBFB637rNp7ARwT4ZJ1kM2Mgeb1W8EO8bD3mRx6LD9cK8PEOEU9gIPA+3g+NPB7vcwI8YaGAj5oj2wWm9dzoOoWR+4bA+KSGAB8YPEHc1iCuITKF+D0xY55LfiIM3sRVRGb0PRu9nTwHo+1lTLuPegxIdvitx+L6TdzeZFlodQIMSWcgfq5aqVA/fDje0Sd/kcY7/HgnXwgCwkKy0y+MPh53FMZ2HoKRToVp7fRbO25jRnbws2GHnavg7jaTDIUUC0Bx+tvXajV27iwd97rG8t7HoeYh8uDxR9yO4sSL/9P6+Lmmjx+PkuXI/ejo+81mg6gR73gazSjZAcU7mWbT02g2aUZ+5H4ziuKdWNNT6Kty8NAgw1HfmB0d8TL5S2fYQz1ZNj1JVIMZue2BOOjwyXJknbjtkYdmEkxNnywxI7cjzOjzjP4nN37kXYzvj7/tR+9HSTVRshOJkh1M1MlfZZ3wI6VkwCCMvM9jC292/IpB8pdp4H2yg46SV0/+xYzs6CHeIR+5428xeIz38c7Wj+6Ij3i8tSP2nhN9ne9ce1HHdbdLwZ0zxsTDIhiIT2dKT+8j3nHt7HUZXedHdoTxztAD8+adxHs7dx29cxy37tglJJ1dM/opmqR3m/SHk96/oZVFY3vC8c4uSgIIwMd/FYws4+fiLG0mRUTxXt9Ho7ejaPR+FMWdgQgiHxFFPm5v1Krb46MoaUeEj2CgXGHv/v00Ip/s1ONlM9lhtx4bTm43o9b703qvJn6fmuPuQ7zzZkwIQxS/D340jBmz9J441D1EydK3Oja0OjiM/IUyMEOJ2tavsdZeBtxD/H/9e865/9DVqkRmMWMMhXE7zkop5HA78/PMQnnZYR9PU/7dZq0tAH8JrAQ+DFxlrf1wtwsTEZGJtTPg9jHgdefcG865IeAnwOruliUiIpNpZ6jkVOB3Y+6/BVw4fiVr7VpgLYBzjlqt1llBYdjxtlmX57ZDvtuvtuez7dBZ+9sJ7om+vTrq+2rn3AZgQ+v5Tses8jzelee2Q77br7bns+0w2v6FCxe2vU07QyVvAYvG3D8NeGeatYmIyHHSTo/7RWCptXYx8DZwJfAnXa1KREQmNWWP2znXAP4U+CmwJX7IvdbtwkREZGJtHcftnHsceLzLtYiISBuM9105LzYzJ9uKiKRIW6cyd2viBNPpj7X2pWPZPss/eW573tuvtve+jpS0vy26EoCISMYouEVEMiaNwb1h6lVmrTy3HfLdfrU9v6bd/m59OSkiIl2Sxh63iIh8gNRcSCHvc35ba38D7Ce+7EfDOfeHva2oe6y1PwCuAN51zl2QPHYS8N+AM4HfANY5t6dXNXbTJO2/FbgeeC9Z7RvJ+ROzirV2EfC3wALi6xBscM7dk4fP/wPafivT/OxT0ePWnN8j/qVzbtlsDu3EfcBl4x77OvCkc24p8GRyf7a6j6PbD3B38vkvm42hnWgAf+acOw9YDnwp+b+eh89/srbDND/7VAQ3mvM7V5xzzwK7xz28GvhRcvtHwGdmtKgZNEn7c8E5t80594vk9n7iaTROJQef/we0fdrSEtwTzfndUYMyzAP/aK19KZnbPG9Occ5tg/gfODC/x/X0wp9aa1+x1v7AWjuv18V0m7X2TOAjwGZy9vmPaztM87NPS3BPdMZQ3g53ucg59wfEw0Vfstb+ca8Lkhn118DZwDJgG3Bnb8vpLmttFXgAuMk5t6/X9cykCdo+7c8+LcGd+zm/nXPvJMt3gYeIh4/yZIe19kMAyfLdHtczo5xzO5xzTedcBPwNs/jzt9YWiYPrx865B5OHc/H5T9T2Tj77tAT3yJzf1to+4jm/N/a4phljra1Ya+e0bgOXAr/sbVUzbiNwTXL7GuDhHtYy41qhlfgss/Tzt9Ya4PvAFufcXWOemvWf/2Rt7+SzT80JONbay4HvEB8O+APn3O09LmnGWGvPIu5lQ3yI5t/N5vZba/8e+ARQA3YAtwD/A3DA6cCbwOedc7PyC7xJ2v8J4j+VPfHhcDe0xnxnE2vtHwHPAa8SHxIH8A3isd5Z/fl/QNuvYpqffWqCW0RE2pOWoRIREWmTgltEJGMU3CIiGaPgFhHJGAW3iEjGKLhFRDJGwS0ikjEKbhGRjPn/oJBib7q6nlMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f23a259fdd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_sim_id.fit([data_pairs[:, 0], data_pairs[:, 1]], \n",
    "               labels, \n",
    "               batch_size       = 24, \n",
    "               verbose          = 0, \n",
    "               epochs           = 25, \n",
    "               callbacks        = [tut_func.PlotLosses()], \n",
    "               validation_split = 0.1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check the results for some individual tracks by supplying the index to our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "track_id                                  637McrDyDQ9CffkFuZGRpq\n",
      "artist_name                                                KSHMR\n",
      "title                                                    Kolkata\n",
      "album_name                                               Kolkata\n",
      "label                                                     Dharma\n",
      "duration                                                  206250\n",
      "popularity                                                    54\n",
      "year                                                        2017\n",
      "genres         [big room, deep big room, edm, electro house, ...\n",
      "playlist                                               clubbeats\n",
      "Name: 33, dtype: object\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_name</th>\n",
       "      <th>title</th>\n",
       "      <th>album_name</th>\n",
       "      <th>year</th>\n",
       "      <th>playlist</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>KSHMR</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>Kolkata</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>Sander van Doorn</td>\n",
       "      <td>Riff - SvD x David Tort Remix</td>\n",
       "      <td>Riff (SvD x David Tort Remix)</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>Mike Williams</td>\n",
       "      <td>Step Up</td>\n",
       "      <td>Step Up</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Hardwell</td>\n",
       "      <td>Get Low</td>\n",
       "      <td>Get Low</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Galantis</td>\n",
       "      <td>Rich Boy - Quintino Remix</td>\n",
       "      <td>Rich Boy (Remixes)</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>Garmiani</td>\n",
       "      <td>Fogo</td>\n",
       "      <td>Fogo</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Maeva Carter</td>\n",
       "      <td>Escape</td>\n",
       "      <td>Escape</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Armin van Buuren</td>\n",
       "      <td>My Symphony (The Best Of Armin Only Anthem)</td>\n",
       "      <td>My Symphony (The Best Of Armin Only Anthem)</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Dylan Taylor</td>\n",
       "      <td>Frankenstein</td>\n",
       "      <td>Frankenstein</td>\n",
       "      <td>2018</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>Alejandro Diego</td>\n",
       "      <td>Underground</td>\n",
       "      <td>Underground</td>\n",
       "      <td>2017</td>\n",
       "      <td>clubbeats</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         artist_name                                        title  \\\n",
       "33             KSHMR                                      Kolkata   \n",
       "79  Sander van Doorn                Riff - SvD x David Tort Remix   \n",
       "52     Mike Williams                                      Step Up   \n",
       "16          Hardwell                                      Get Low   \n",
       "25          Galantis                    Rich Boy - Quintino Remix   \n",
       "34          Garmiani                                         Fogo   \n",
       "22      Maeva Carter                                       Escape   \n",
       "14  Armin van Buuren  My Symphony (The Best Of Armin Only Anthem)   \n",
       "32      Dylan Taylor                                 Frankenstein   \n",
       "60   Alejandro Diego                                  Underground   \n",
       "\n",
       "                                     album_name  year   playlist  \n",
       "33                                      Kolkata  2017  clubbeats  \n",
       "79                Riff (SvD x David Tort Remix)  2017  clubbeats  \n",
       "52                                      Step Up  2017  clubbeats  \n",
       "16                                      Get Low  2018  clubbeats  \n",
       "25                           Rich Boy (Remixes)  2017  clubbeats  \n",
       "34                                         Fogo  2017  clubbeats  \n",
       "22                                       Escape  2018  clubbeats  \n",
       "14  My Symphony (The Best Of Armin Only Anthem)  2017  clubbeats  \n",
       "32                                 Frankenstein  2018  clubbeats  \n",
       "60                                  Underground  2017  clubbeats  "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar(model_sim_id, 33)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "General model evaluation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluate(lambda x,y: [model_sim_id.predict([x[[i]], y]) for i in range(feature_data.shape[0])], 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Discussion of Results\n",
    "\n",
    "Now the model can differentiate between highly and somehow similar."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Using LSTMs to train on Sequential Data\n",
    "\n",
    "So long we have by large ignored the temporal information provided by the Spotify features. As mentioned in the first part of the tutorial, these features are extracted from segmented audio data. For each of these segments a feature vector is provided. So far we have aggregated this information into a single vector by calculating statistical values to abstract from the temporal representation and create a general descriptor for the entire track.\n",
    "\n",
    "In the following part we will harness this temporal information. Recurrent Neural Networks are perfect for analysing sequential data. We will be using the Long Short-Term Memory (LSTM) to train our model on the sequential data.\n",
    "\n",
    "### Preparing input-data for the LSTM\n",
    "\n",
    "Now it gets a little more complicated. LSTMs do not take a single input vector as input, but sequences of vectors, also referred to as timesteps. In this tutorial a timestep corresponds with a segment provided by the Spotify API. Such a segment contains the same features used before, but on a smaller temporal scale. We will again caoncatenate those features into a single vector and use several consecutive segments as input for the LSTM.\n",
    "\n",
    "Unfortunately, du to the onset detection used by Spotify to segment the audio data, these segments do not have the same length. The following chart shows the distribution of the segment length's:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQgAAAFpCAYAAABgXrYAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEnlJREFUeJzt3X9sldd9x/G3sR0Ba7NIvX8s/KhGpTt1DFWqhBwNTyp1O5WgLom06Tg/uihbF5JI5G7abrROQFrRVAqZy5aregQWVdmqrHAWTXdscsc0daiTGXHSbf0jTduLqDoMhNSMRKoIi4G7P2yYMT74Mdzrx9f3/ZIs53nu4blf6cYfnXOe85zbUa/XkaSZLMm7AEkLlwEhKcmAkJRkQEhKMiAkJRkQkpIMCElJBoSkJANCUpIBISmpK8f3do23lJ+OLI3yDAhOnTqV59vrFhQKBcbGxvIuQzdhxYoVmds6xJCUZEBISjIgJCUZEJKSDAhJSQaEpCQDQlKSASEpyYCQlGRAaE6q1Sp9fX0sW7aMvr4+qtVq3iWpiXJdaq3WUq1W2bVrFwMDA2zevJmhoSHK5TIA9913X87VqRnsQSizSqXCwMAAvb29dHd309vby8DAAJVKJe/S1CQGhDKr1Wr09PRcc66np4darZZTRWo2A0KZFYtFRkZGrjk3MjJCsVjMqSI1mwGhzEqlEuVymeHhYcbHxxkeHqZcLlMqlfIuTU3SkeN3c9bdD6L1VKtVKpUKtVqNYrFIqVRygrLFTO4HkWnDGANCN8UNY1rXXALCIYakJANCUpIBISnJgJCUZEBISjIgJCUZEJKSDAhJSQaEpCQDQlKSASEpyYCQlGRASEoyICQlGRCSkmbd1TqE8HXgs8DbMcZ1M7z+EPDHk4c/A56IMX6voVVKykWWHsRLwKYbvP5j4BMxxo8BXwb2NaAuSQvArD2IGON3Qgi/eIPXj0w5PAqsakBdkhaARs9BfB74VoOvKSknDftmrRDCJ5kIiF+7QZstwBaAGCOFQqFRb6951tXV5efXBhoSECGEjwEvAnfHGM+m2sUY9/H/cxR1Nz1tXW5a27omN63N5JaHGCGEDwN/B/x2jPFHt3o9SQvHrNvehxC+CWwECsAZ4ItAN0CM8YUQwovAbwI/mfwnF2OM6zO8t9vetzB7EK3L78VQ0xkQrcvvxZDUEAaEpCQDQlKSASEpyYCQlGRASEoyICQlGRCSkgwISUkGhKQkA0JSkgEhKcmAkJRkQEhKMiAkJRkQkpIMCElJBoSkJANCUpIBISnJgJCUZEBISjIgJCUZEJKSDAhJSQaEpCQDQlKSASEpyYCQlGRASEoyICQlGRCSkgwISUkGhKQkA0JSkgEhKcmAkJRkQEhKMiAkJRkQkpIMCElJBoSkJANCUpIBISmpa7YGIYSvA58F3o4xrpvh9Q7geWAzcB54JMb4H40uVNL8y9KDeAnYdIPX7waKkz9bgD23XpakhWDWgIgxfgf4nxs0uRf46xhjPcZ4FLgjhHBnowqUlJ9GzEGsBE5MOR6dPCepxc06B5FBxwzn6jM1DCFsYWIYQoyRQqHQgLdXHrq6uvz82kAjAmIUWD3leBVwaqaGMcZ9wL7Jw/rY2FgD3l55KBQK+Pm1phUrVmRu24iAOAhsDSHsB+4C3o0xnm7AdSXlLMttzm8CG4FCCGEU+CLQDRBjfAEYYuIW5zEmbnP+TrOKlTS/Our1GacL5kP91KkZRyJqAQ4xWtfkEGOmucPruJJSUpIBISnJgJCUZEBISjIgJCUZEJKSDAhJSQaEpCQDQlKSASEpyYCQlGRASEoyICQlGRCSkgwISUkGhKQkA0JSkgEhKcmAkJRkQEhKMiAkJRkQkpIMCElJBoSkJANCUpIBISnJgJCUZEBISjIgJCUZEJKSDAhJSQaEpCQDQlKSASEpyYCQlGRASEoyICQlGRCSkgwISUkGhKQkA0JSkgEhKcmAkJTUlaVRCGET8DzQCbwYY3x22usfBv4KuGOyzRdijEMNrlXSPJu1BxFC6AQGgbuBtcADIYS105ptB2KM8ePA/cBfNLpQSfMvyxCjBzgWYzweY3wf2A/cO61NHbh98r9/HjjVuBIl5SXLEGMlcGLK8Shw17Q2XwL+OYTwJPBzwKcbUp2kXGUJiI4ZztWnHT8AvBRj/GoI4VeBb4QQ1sUYL09tFELYAmwBiDFSKBRupmYtAF1dXX5+bSBLQIwCq6ccr+L6IcTngU0AMcZ/DyEsBQrA21MbxRj3AfsmD+tjY2M3U7MWgEKhgJ9fa1qxYkXmtlkC4jWgGEJYA5xkYhLywWlt/hv4FPBSCOGXgaXATzNXIWlBmnWSMsZ4EdgKHALenDgV3wgh7Awh3DPZ7I+AR0MI3wO+CTwSY5w+DJHUYjrq9dz+juunTnmzo1U5xGhdk0OMmeYWr+NKSklJBoSkJANCUpIBISnJgNCcVKtV+vr6WLZsGX19fVSr1bxLUhNleppTgolw2LVrFwMDA2zevJmhoSHK5TIA9913X87VqRnsQSizSqXCwMAAvb29dHd309vby8DAAJVKJe/S1CQGhDKr1Wr09PRcc66np4darZZTRWo2hxjKrFgssnv3bg4dOkStVqNYLPKZz3yGYrGYd2lqEnsQymzDhg0MDg7S39/P2bNn6e/vZ3BwkA0bNuRdmprEgFBmR44cYevWrRw4cIAPfehDHDhwgK1bt3LkyJG8S1OT+CyGMlu9ejXHjx+nu7v76rMY4+PjfOQjH+HEiROzX0ALgs9iqCmKxSIjIyPXnBsZGXEOYhEzIJRZqVSiXC4zPDzM+Pg4w8PDlMtlSqVS3qWpSRxiaE6q1SqVSuXqXYxSqeQiqRYzlyGGAaGb4n4Qrcs5CEkNYUBISjIgJCUZEJKSDAhJSQaEpCQDQlKSASEpyYCQlGRASEoyICQlGRCSkgwISUkGhKQkA0JSkgEhKcmAkJRkQEhKMiAkJRkQkpIMCElJBoSkJANCUpIBISnJgJCUZEBoTqrVKn19fSxbtoy+vj6q1WreJamJuvIuQK2jWq2ya9cuBgYG2Lx5M0NDQ5TLZQC/n3ORsgehzCqVCgMDA/T29tLd3U1vby8DAwNUKpW8S1OTZOpBhBA2Ac8DncCLMcZnZ2gTgC8BdeB7McYHG1inFoBarUZPT88153p6eqjVajlVpGabtQcRQugEBoG7gbXAAyGEtdPaFIE/AXpjjL8C/EETalXOisUiu3fvvmYOYvfu3RSLxbxLU5NkGWL0AMdijMdjjO8D+4F7p7V5FBiMMZ4DiDG+3dgytRBs2LCBwcFB+vv7OXv2LP39/QwODrJhw4a8S1OTZBlirAROTDkeBe6a1uaXAEIIw0wMQ74UY/yn6RcKIWwBtgDEGCkUCjdTs3Ly6quv8tRTT/HKK6/wzDPP8NGPfpSnnnqKgwcP+lkuUlkComOGc/UZrlMENgKrgH8LIayLMb4ztVGMcR+w78o1xsbG5latcvWDH/yAoaEhnnzySQqFAmNjY4yPj/Pcc8/hZ9k6VqxYkbltliHGKLB6yvEq4NQMbf4+xjgeY/wx8EMmAkOLSLFYZGRk5JpzIyMjzkEsYlkC4jWgGEJYE0K4DbgfODitTRX4JEAIocDEkON4IwtV/kqlEuVymeHhYcbHxxkeHqZcLlMqlfIuTU3SUa9PHy1cL4SwGfhzJuYXvh5j/EoIYSfweozxYAihA/gqsAm4BHwlxrh/lsvWT52a3hHRQletVqlUKtRqNYrFIqVSyUVSLWZyiDHT1MF1MgVEkxgQLezKHIRaz1wCwpWUkpIMCElJBoTmxKc524tPcyozn+ZsP/YglJlPc7YfA0KZ+TRn+zEglJkrKduPAaHMXEnZflwopTlxJWXrcyWlms6VlK3LlZSSGsKAkJRkQEhKMiAkJRkQmhOfxWgvPouhzHwWo/3Yg1BmPovRfgwIZeazGO3HgFBmPovRfpyDUGalUoknnniC5cuXc/LkSVauXMn58+fZuXNn3qWpSexB6KbkuERf88iAUGaVSoU9e/Zw9OhRLly4wNGjR9mzZ4+TlIuYAaHMarUab7311jXrIN566y0nKRcxn+ZUZuvXr+fSpUt87Wtfu7oOYuvWrXR2dvL666/nXZ4y8mlOSQ1hQCizM2fOsG3bNnbs2MHtt9/Ojh072LZtG2fOnMm7NDWJtzmVWbFY5M477+Tb3/721Q1jhoeHXQexiNmDUGbuSdl+nKTUnLgnZetzT0o1nXtSti7vYkhqCANCc+KGMe3FuxjKzA1j2o89CGXmhjHtx4BQZm4Y034MCGXmhjHtxzkIZeaGMe3HHoRuihvGtAcDQpm5YUz7MSCUWa1W4/Tp09esgzh9+rSTlIuYS62V2fr163nnnXcYHx/n4sWLdHV10d3dzR133OGGMS1kLkutnaRUZu+++y7vvfceS5ZMdDwvX77Me++9R0dHpv/X1IIcYiiz8+fPAxPBMPX3lfNafDIFRAhhUwjhhyGEYyGEL9yg3W+FEOohhPWNK1ELzdNPP825c+d4+umn8y5FTTZrQIQQOoFB4G5gLfBACGHtDO0+CJSAVxtdpBaOpUuXsm7dOrq7u1m3bh1Lly7NuyQ1UZY5iB7gWIzxOEAIYT9wL/D9ae2+DDwHlBtaoebdypUrk69duHCBEEKmf3Py5MmG1qX5lyUgVgInphyPAndNbRBC+DiwOsb4jyEEA6LFpf6wV61aRb1eZ8mSJVy+fPnq746ODkZHR+e5Ss2HLAEx0xT11XujIYQlwJ8Bj8x2oRDCFmALQIyRQqGQrUotCI8//jgvvPDC1bsWHR0ddHR08Pjjj/tZLlJZAmIUWD3leBUwdQHDB4F1wOHJrucvAAdDCPfEGK+5OR5j3Afsmzysu2VZa9m+fTsXLlzg5Zdf5tKlS3R2dvLwww+zfft2t59rIZPrIDKZdaFUCKEL+BHwKeAk8BrwYIzxjUT7w0B5ejjMwIVSLezSo/fQ+ZcH8y5DN6Ghe1LGGC8CW4FDwJsTp+IbIYSdIYR7bqVQSQubS611U+xBtC53tZbUEAaEpCQDQlKSASEpyYCQlGRASEoyICQlGRCSkgwISUkGhKQkA0JSkgEhKcmAkJRkQEhKMiAkJRkQkpLcMKYNXfr9B+H8z/IuA5Z/gM7n/ybvKtqO382pGzv/s1veDapQKNzyRrWXHnXHwoXOIYakJANCUpIBISnJgJCUZEBISjIgJCUZEJKSXCjVhv7hwDt5l3DVb/TfkXcJbceFUrqhzf/y8MJZKNXv1/ctZA4xJCUZEJKSDAhJSQaEpCQDQlKSdzHa1K0+an2mEUUs/0AjrqImch2EbsqlR++55Vulysdc1kE4xJCUZEBISjIgJCUZEJKSDAhJSQaEpCQDQlKSASEpyYCQlJRpqXUIYRPwPNAJvBhjfHba638I/B5wEfgp8Lsxxp80uFZJ82zWHkQIoRMYBO4G1gIPhBDWTmv2n8D6GOPHgFeA5xpdqKT5l6UH0QMcizEeBwgh7AfuBb5/pUGM8V+ntD8KfK6RRUrKR5Y5iJXAiSnHo5PnUj4PfOtWipK0MGTpQcz01NeMj4CGED4HrAc+kXh9C7AFIMZIoVDIWKYWmjPg59cGsgTEKLB6yvEq4LrntEMInwa2AZ+IMf7vTBeKMe4D9k0e1m91V2Tly8+vNU0+7p1JloB4DSiGENYAJ4H7gQenNgghfBzYC2yKMb6dvVRJC9mscxAxxovAVuAQ8ObEqfhGCGFnCOHKtkR/CnwA+NsQwn+FENxJRFoE3FFKN8UdpVqXO0pJaggDQnOyfft21qxZw4eHvsuaNWvYvn173iWpiRxi6DorV95omUt2J0+ebMh11Fh+ea9uSeoPe9WqVdTrdTo6Oq77PTo6Os9Vaj44xFBmV3qbO3bs4Ny5c+zYseOa81p8DAjNycaNG3nsscdYvnw5jz32GBs3bsy7JDWRAaE5OXz4MHv37uX8+fPs3buXw4cP512SmshJSmV2ZQ5iyZIlXL58+epv5yBai+sg1BSPPPIIAJcvX77m95XzWny8i6HMnnnmGQBefvll3n//fW677TYeeuihq+e1+DjE0E0pFAo+zdmiHGJIaggDQlKSASEpyYCQlGRASEoyICQlGRCSkgwISUkGhKQkA0JSUq5LrfN6Y0kLf6l1hz+t+xNC+G7eNfhzSz+ZOMSQlGRASEoyIHSz9s3eRK0uz0lKSQucPQhJSW45pzkJIWwCngc6gRdjjM/mXJKayCGGMgshdAI/An4dGAVeAx6IMX4/18LUNA4xNBc9wLEY4/EY4/vAfuDenGtSExkQmouVwIkpx6OT57RIGRCai5lW4DlGXcQMCM3FKLB6yvEqwO8uWMS8i6G5eA0ohhDWACeB+4EH8y1JzWQPQpnFGC8CW4FDwJsTp+Ib+ValZvI2p6QkexCSkgwISUkGhKQkA0JSkgEhKcmAkJRkQEhKMiAkJf0fvzPDO3XKMmUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f23a25af160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "segment_lengths = []\n",
    "\n",
    "for _, spotify_data in raw_track_data:\n",
    "    \n",
    "    _, _, _, f_sequential, f_trackbased = spotify_data\n",
    "    segment_lengths.append((f_trackbased[0][\"duration_ms\"] / 1000.) / len(f_sequential[\"segments\"]))\n",
    "\n",
    "# plot the distribution\n",
    "pd.DataFrame(segment_lengths).boxplot(figsize=(4,6));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It can be seen, that those vary quite strong between 0.2 and 0.4 seconds. This is a problem when we use a fixed number of segments. For example, the analyzed time-span of 20 segments could range from 4 to 8 seconds. This has an effect on the represented tempo of a track and the model would learn a distorted representation of music similarity. Thus, we will rescale the featurespace using interpolation.\n",
    "\n",
    "Additionally to the sequential features, the track based features will be used too as an additional input to the neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def aggregate_features_sequential(seq_data, track_data, len_segment_frames, len_segment_sec, m_data, with_year=False, with_popularity=False):\n",
    "    \n",
    "    # sequential data\n",
    "    segments = seq_data[\"segments\"]\n",
    "    sl       = len(segments)\n",
    "    \n",
    "    mfcc              = np.array([s[\"timbre\"]            for s in segments])\n",
    "    chroma            = np.array([s[\"pitches\"]           for s in segments])\n",
    "    loudness_max      = np.array([s[\"loudness_max\"]      for s in segments]).reshape((sl,1))\n",
    "    loudness_start    = np.array([s[\"loudness_start\"]    for s in segments]).reshape((sl,1))\n",
    "    loudness_max_time = np.array([s[\"loudness_max_time\"] for s in segments]).reshape((sl,1))\n",
    "    duration          = np.array([s[\"duration\"]          for s in segments]).reshape((sl,1))\n",
    "    confidence        = np.array([s[\"confidence\"]        for s in segments]).reshape((sl,1))\n",
    "        \n",
    "    # concatenate sequential features\n",
    "    sequential_features = np.concatenate([mfcc, chroma, loudness_max, loudness_start, \n",
    "                                          loudness_max_time, duration, confidence], axis=1)\n",
    "\n",
    "    # calculate length of segment (in ms)\n",
    "    length_of_track        = track_data[0][\"duration_ms\"] / 1000.\n",
    "    length_of_segment      = length_of_track / len(segments)\n",
    "    num_segments_for_n_sek = int(np.round(len_segment_sec / length_of_segment))\n",
    "    \n",
    "    # select a random lstm-input-segment from the aggregated feature data\n",
    "    offset  = np.random.randint(0, sl - num_segments_for_n_sek)\n",
    "    segment = sequential_features[offset:(offset+num_segments_for_n_sek),:]\n",
    "    \n",
    "    # re-scale segment length to desired length (in seconds)\n",
    "    x  = np.arange(segment.shape[0])\n",
    "    y  = np.arange(segment.shape[1])\n",
    "    xx = np.linspace(x.min(),x.max(),len_segment_frames)\n",
    "\n",
    "    newKernel = RectBivariateSpline(x,y,segment, kx=2,ky=2)\n",
    "    segment   = newKernel(xx,y)\n",
    "        \n",
    "    # track-based data\n",
    "    track_features = [track_data[0][\"acousticness\"],     # acoustic or not?\n",
    "                      track_data[0][\"danceability\"],     # danceable?\n",
    "                      track_data[0][\"energy\"],           # energetic or calm?\n",
    "                      track_data[0][\"instrumentalness\"], # is somebody singing?\n",
    "                      track_data[0][\"liveness\"],         # live or studio?\n",
    "                      track_data[0][\"speechiness\"],      # rap or singing?\n",
    "                      track_data[0][\"tempo\"],            # slow or fast?\n",
    "                      track_data[0][\"time_signature\"],   # 3/4, 4/4, 6/8, etc.\n",
    "                      track_data[0][\"valence\"]]          # happy or sad?\n",
    "    \n",
    "    if with_year:\n",
    "        track_features.append(int(m_data[\"year\"]))\n",
    "        \n",
    "    if with_popularity:\n",
    "        track_features.append(int(m_data[\"popularity\"]))\n",
    "        \n",
    "    \n",
    "    return segment, track_features\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets prepare the input features. We will be using the following parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# number of input-vectors for the LSTM\n",
    "len_segment_frames = 24\n",
    "\n",
    "# number of seconds these vectors describe\n",
    "len_segment_sec    = 6."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "execute the function to prepare the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sequential_features.shape: (1060, 24, 29)\n",
      "trackbased_features.shape: (1060, 11)\n"
     ]
    }
   ],
   "source": [
    "sequential_features = []\n",
    "trackbased_features = []\n",
    "\n",
    "for i, (_, spotify_data) in enumerate(raw_track_data):\n",
    "    \n",
    "    _, _, _, f_sequential, f_trackbased = spotify_data\n",
    "    \n",
    "    seq_feat, track_feat = aggregate_features_sequential(f_sequential, \n",
    "                                                         f_trackbased, \n",
    "                                                         len_segment_frames,\n",
    "                                                         len_segment_sec,\n",
    "                                                         metadata.loc[i],\n",
    "                                                         with_year=True,\n",
    "                                                         with_popularity=True)\n",
    "    \n",
    "    sequential_features.append(seq_feat)\n",
    "    trackbased_features.append(track_feat)\n",
    "    \n",
    "sequential_features = np.asarray(sequential_features)\n",
    "trackbased_features = np.asarray(trackbased_features)\n",
    "\n",
    "print(\"sequential_features.shape:\", sequential_features.shape)\n",
    "print(\"trackbased_features.shape:\", trackbased_features.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalization\n",
    "\n",
    "As in the examples before, we need to normalize the data before using it with the neural networks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# standardize sequential_features\n",
    "rows, x, y = sequential_features.shape\n",
    "sequential_features  = sequential_features.reshape(rows, (x * y))\n",
    "sequential_features -= sequential_features.mean(axis=0)\n",
    "sequential_features /= sequential_features.std(axis=0)\n",
    "sequential_features  = sequential_features.reshape(rows, x, y)\n",
    "\n",
    "# standardize trackbased_features\n",
    "trackbased_features -= trackbased_features.mean(axis=0)\n",
    "trackbased_features /= trackbased_features.std(axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Data-Pairs\n",
    "\n",
    "Also like before, we have to prepare the paired training examples. We continue with the last approach presented in this part, which is based on the mutual similarities and includes identities. Each pair contains the sequential and track-based inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_pairs_with_sims_and_identity_segments(sequential_features, trackbased_features, metadata, num_pairs_per_track, playlist_similarities):\n",
    "    \n",
    "    data_pairs_seq   = []\n",
    "    data_pairs_track = []\n",
    "    labels           = []\n",
    "    \n",
    "    for row_id, q_track in metadata.sample(frac=1).iterrows():\n",
    "        \n",
    "        data_pairs_seq.append([sequential_features[[row_id]][0], sequential_features[[row_id]][0]])\n",
    "        data_pairs_track.append([trackbased_features[[row_id]][0], trackbased_features[[row_id]][0]])\n",
    "        labels.append(1)\n",
    "        \n",
    "        for _ in range(num_pairs_per_track):\n",
    "            \n",
    "            # search similar and dissimilar examples\n",
    "            pos_example = metadata[metadata.playlist == q_track.playlist].sample(1)\n",
    "            neg_example = metadata[metadata.playlist != q_track.playlist].sample(1)\n",
    "\n",
    "            # create feature pairs\n",
    "            data_pairs_seq.append([sequential_features[[row_id]][0],   \n",
    "                                   sequential_features[[pos_example.index]][0]])\n",
    "            data_pairs_track.append([trackbased_features[[row_id]][0], \n",
    "                                     trackbased_features[[pos_example.index]][0]])\n",
    "            \n",
    "            sim_val = playlist_similarities.loc[q_track.playlist, pos_example.playlist] - 0.1\n",
    "            labels.append(np.max([0, sim_val]))\n",
    "\n",
    "            data_pairs_seq.append([sequential_features[[row_id]][0],   \n",
    "                                   sequential_features[[neg_example.index]][0]])\n",
    "            data_pairs_track.append([trackbased_features[[row_id]][0], \n",
    "                                     trackbased_features[[neg_example.index]][0]])\n",
    "            \n",
    "            sim_val = playlist_similarities.loc[q_track.playlist, neg_example.playlist] - 0.1\n",
    "            labels.append(np.max([0, sim_val]))\n",
    "\n",
    "    return np.array(data_pairs_seq), np.array(data_pairs_track), np.asarray(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute the function to prepare the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_pairs_seq, data_pairs_track, labels = \\\n",
    "    create_pairs_with_sims_and_identity_segments(sequential_features,\n",
    "                                                 trackbased_features,\n",
    "                                                 metadata, \n",
    "                                                 10, \n",
    "                                                 playlist_similarities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### The Siamese Network Architecture using LSTMs\n",
    "\n",
    "The network architecture consists again of shared \"twin\" layers. But, now two layers are shared. First the fully connected layers which train on the track-based features. Second, a bi-directional LSTM which trains on the re-scaled sequential data provided by the Spotify API. The network thus now takes four inputs. The outputs of the fully connected layer and the LSTM are finally joined and the Eucledian distance of each twin is calculated which is the final output of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_siamese_network_with_lstm(data_pairs_seq, data_pairs_track):\n",
    "\n",
    "    # sequential input\n",
    "    input_seq_left  = Input(shape=data_pairs_seq[:, 0].shape[1:])\n",
    "    input_seq_right = Input(shape=data_pairs_seq[:, 0].shape[1:])\n",
    "\n",
    "    # track-based input\n",
    "    input_track_left  = Input(shape=data_pairs_track[:, 0].shape[1:])\n",
    "    input_track_right = Input(shape=data_pairs_track[:, 0].shape[1:])\n",
    "\n",
    "    # shared bi-directional LSTM layer for the sequential features\n",
    "    bdlstm = Bidirectional(LSTM(29, return_sequences=False, activation=\"selu\"))\n",
    "\n",
    "    # shared fully connected layers for the track-based features\n",
    "    shared_fc_1 = Dense(11, activation=\"selu\")\n",
    "    shared_fc_2 = Dense(11, activation=\"selu\")   \n",
    "\n",
    "    # left twin\n",
    "    seq_resp_left   = bdlstm(input_seq_left)\n",
    "    track_resp_left = shared_fc_1(shared_fc_2(input_track_left))\n",
    "    left_twin       = concatenate([seq_resp_left, track_resp_left], axis=1)\n",
    "\n",
    "    # right twin\n",
    "    seq_resp_right   = bdlstm(input_seq_right)\n",
    "    track_resp_right = shared_fc_1(shared_fc_2(input_track_right))\n",
    "    right_twin       = concatenate([seq_resp_right, track_resp_right], axis=1)\n",
    "\n",
    "    # calc difference\n",
    "    distance = Lambda(euclidean_distance,\n",
    "                      output_shape=lambda x: x[0])([left_twin, right_twin])\n",
    "\n",
    "    return Model([input_seq_left, input_seq_right, input_track_left, input_track_right], distance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the model\n",
    "model_rnn = create_siamese_network_with_lstm(data_pairs_seq, data_pairs_track)\n",
    "\n",
    "# define the optimizer\n",
    "opt = Nadam(lr=0.001)\n",
    "\n",
    "# compile the model\n",
    "model_rnn.compile(loss      = contrastive_loss, \n",
    "                  optimizer = opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHNdJREFUeJzt3X2QXGWh5/Hvc/rMa88kwduGvPBqyAqCtyJSJnvxXlkgakBvBNenoNgtsFgQC0qUe7VErcIVKe7WFrBYWrLRiwjF6n32Ei5BoHZToCVylwh4xRhSKPJiwiSEZCAk8959nv3jnO70TCaZns7MdM45v09VV/d56XOeftL5nWeec/o5xnuPiIhkS9DqAoiIyMxTuIuIZJDCXUQkgxTuIiIZpHAXEckghbuISAYp3EVEMkjhLiKSQQp3EZEMClu4b/00VkSkOWaqFVoZ7vT19TX1vlKpxO7du2e4NOmkuoipHmKqh1iW62HJkiUNraduGRGRDFK4i4hkkMJdRCSDWtrnLiL5471neHiYKIowZsrzgk154403GBkZmZVtzwXvPUEQ0NnZ2XQdKdxFZE4NDw/T1tZGGM5e/IRhSKFQmLXtz4Vyuczw8DBdXV1NvV/dMiIyp6IomtVgz4owDImiqOn3K9xFZE7NVldMFh1JXaUu3P32V9l////E73+n1UURETlqpS7c2dXHwD//GN7a0+qSiEhKLV++vNVFmHXpC/dib/w8sK+15RAROYqlL9y7e+Lnwf2tLYeIpJ73nptvvplzzz2X8847j4ceegiIL6W8+OKLWb16Neeeey6bNm2iUqnwxS9+sbbuunXrWlz6w0vfKetiHO5+YP/UI+eIyFEt+ukP8NtemdFtmuNPhv/0+YbWffTRR9myZQsbN26kv7+fCy64gFWrVvHggw/ykY98hOuvv55KpcLQ0BBbtmxh586dPPHEEwDs3bt3Rss909Lbcle3jIgcoV//+td86lOfolAo8O53v5tVq1bx/PPPs2LFCpxz3HbbbWzdupWenh5OOOEE/vznP/ONb3yDn//85/T29ra6+IeVvpZ7RyeEobplRDIguOSqlu7f+8lHHl+1ahUPPPAAjz/+ONdffz3XXHMNn/nMZ9i4cSO/+MUvuOeee3j44Ye5/fbb57jEjUtdy90YQ9AzDwYU7iJyZFatWsWGDRuoVCrs2bOHTZs2sWLFCrZv306pVOKyyy7jkksuYfPmzfT39xNFERdeeCFf/vKX2bx5c6uLf1jpa7kDptiLV7iLyBFas2YNzz33HKtXr8YYw9e//nUWLlyIc4677rqLMAwpFovceeed7NixgxtuuKH2q9Ebb7yxxaU/PHOoP0vmgG/2Zh3BbV9nzAQUbrh5houUPlm+KcF0qB5iaaiHwcFBuru7Z3UfYRhSLpdndR9zYbK6Sm7WMeX1JKnrloG45a4TqiIih5bKcA961ecuInI46Qz3nnm6WkZE5DBSGe6mpxeGBvGVSquLIiJyVEpluAc98+IXgwOtLYiIyFEqleFuNHiYiMhhpTLcg95qy1397iIik0lluB9ouSvcRWR2HW7s923btnHuuefOYWkal8pwr7bcvVruIiKTSuXwA7UTqupzF0m1Hz77Bq+8NTyj2zz5mE6uWbX0kMtvueUWli5dyhVXXAHAbbfdhjGGp59+mr1791Iul/nKV77Cxz72sWntd3h4mBtvvJHf/e53FAoFbrrpJs4++2xefPFFbrjhBkZHR/Hes27dOhYtWsTnPvc5duzYQRRFXH/99axdu/ZIPvZBUhnu6pYRkWatXbuWm266qRbuDz/8MPfffz9XXXUVvb299Pf388lPfpKPfvSj07pB9T333APA448/zksvvcSll17Kk08+yX333ceVV17JxRdfzOjoKJVKhSeeeIJFixZx3333AfDOOzN/T+h0hnsYQmeXTqiKpNx/OevYOd/nGWecwe7du9m5cyd79uxh/vz5LFy4kG9+85ts2rQJYww7d+7kzTffZOHChQ1v95lnnuGzn/0sAKeccgrHHXccL7/8Mh/84Af5zne+w44dO1izZg3vec97OPXUU7n55pu55ZZbOP/881m5cuWMf85U9rkD8U071C0jIk248MILeeSRR9iwYQNr165l/fr17Nmzh8cee4yNGzdSKpUYGRmZ1jYPNQjjRRddxI9+9CM6Ozu57LLL+NWvfsWyZct47LHHOPXUU7n11lu54447ZuJjjZPecC/24PUjJhFpwtq1a3nooYd45JFHuPDCC9m3bx+lUom2tjaeeuoptm/fPu1trly5kgcffBCAP/3pT7z++ussW7aM1157jRNPPJErr7yS1atXs3XrVnbu3ElXVxef/vSnueaaa2ZlbPhUdssAoJEhRaRJ733vexkYGGDRokUce+yxXHzxxVx++eWsWbOG008/nVNOOWXa27z88sv56le/ynnnnUehUOCOO+6go6ODDRs2sH79esIwZOHChXzpS1/i+eef59vf/jbGGNra2rj11ltn/DOmcjz3UqnEGzf/PezYRuFb35vhYqVLGsbvnguqh1ga6kHjuTcud+O5A5hij06oiogcQnq7ZZITqt77aV2uJCIyXVu3buULX/jCuHkdHR387Gc/a1GJpjZluFtrjwfuBRYBEbDOOXfnhHUMcCdwATAIXOGc+83MF7dOsRfKZRgdhY6OWd2ViMycFnYFN+20005j48aNc77fI6mrRrplysDfOedOA1YB11pr3zdhnTXA8uRxNfD9pkvUqGJP/KyTqiKpEgRBJvrDZ1u5XCYImu85n7Ll7pzbAexIXu+z1m4FlgIv1K22FrjXOeeBp621C6y1i5P3zgpT7MFD3O/+rtJs7UZEZlhnZyfDw8OMjIzMWpdqR0fHtK9TP5p47wmCgM7Ozqa3Ma0+d2vtScAHgE0TFi0FttVNb0/mjQt3a+3VxC17nHOUSs2FchiGzF+8lLeA+WGB9ia3kwVhGDZdj1mieoipHmJZuVrmSDQc7tbaHuAB4IvOuYkDIUx2+D2os8g5tw5YV13e7CVbpVKJveX4Fnt7+7ZjFh3f1HayIA2Xvs0F1UNM9RDLcj0kl0JOqaEOHWttG3Gw3++cWz/JKtuB+oQ9DmjuIvZGJYOHadhfEZGDNXK1jAH+EdjqnLv9EKttAK6z1v4UWAnsnc3+diC+FBI0MqSIyCQa6ZY5G/jPwGZr7W+TeV8DTgBwzt0FPEp8GeRLxJdCfnbmizpBZxcEgX7IJCIyiUaulvkVU/zUNblK5tqZKlQjjDEaX0ZE5BBSO/wAkPxKVS13EZGJ0h3uxR6dUBURmUS6w10tdxGRSaU63DUypIjI5FId7jqhKiIyuXSHe3cPDA7go0qrSyIiclRJd7hXR4YcGmxtOUREjjLpDvduDfsrIjKZVIe7ScaXYWCgtQURETnKpDrcdcMOEZHJZSLc9UMmEZHxMhHu+iGTiMh46Q53nVAVEZlUqsPdhG3Q0alfqYqITJDqcAc0voyIyCTSH+4aGVJE5CAZCHeNLyMiMlH6w727qG4ZEZEJUh/uptirE6oiIhOkPtx1QlVE5GDpD/diD4yN4kdHWl0SEZGjRgbCPRk8TF0zIiI16Q/3bg1BICIyUerD3Wh8GRGRg6Q+3GuDhw3qWncRkar0h3vSLeN1ww4RkZr0h3vtbkxquYuIVKU/3Du7wATqcxcRqZP6cDdBAMWiLoUUEamT+nAHkl+pqltGRKQqG+Fe7NWwvyIidbIR7hoZUkRknEyEu9GY7iIi42Qi3Cn2wKCucxcRqcpGuHf3wOB+fBS1uiQiIkeFcKoVrLV3A58Adjnnzphk+TnAQ8Aryaz1zrlvzWQhp1TsBe9hePDAQGIiIjk2ZbgD9wDfBe49zDpPOuc+MSMlakb9yJAKdxGRqbtlnHO/BPrnoCxNq40MqcshRUSAxlrujfj31trngT7g751zW2Zou43R+DIiIuPMRLj/BjjRObffWnsB8C/A8slWtNZeDVwN4JyjVCo1tcMwDMe9tzx4HHuA3sDQ2eQ202piXeSV6iGmeoipHsB476dcyVp7EvCzyU6oTrLuq8BZzrndU6zq+/r6GinjQUqlErt3H9i8f7uf6MtXYC77PME5a5raZlpNrIu8Uj3EVA+xLNfDkiVLAMxU6x3xpZDW2kXWWpO8/lCyzT1Hut1pqd2NSd0yIiLQ2KWQPwHOAUrW2u3ATUAbgHPuLuA/Ap+31paBIeAS59zUfw7MINPWDu3t+iGTiEhiynB3zl06xfLvEl8q2VrdGoJARKQqG79QBSj24DV4mIgIkLFw13XuIiKx7IS7btghIlKTmXA3GhlSRKQmM+GulruIyAHZCfdiL4yO4MfGWl0SEZGWy1C4a/AwEZGq7IR7t36lKiJSlZlwN9WRIdVyFxHJTriPu2GHiEjOZSfckz53r24ZEZEshbu6ZUREqrIT7l3dYIy6ZUREyFC4myCArqLCXUSEDIU7EPe7K9xFRDIW7t09+EGdUBURyVa4F3vVchcRIWPhbtQtIyICZCzcdcMOEZFYtsK9Ow537+f0/twiIkedbIV7sQeiCIaHWl0SEZGWyla4a2RIEREgY+FeGxlSJ1VFJOcyFe66YYeISCxb4a5uGRERIGvhnnTLeLXcRSTnshXuumGHiAiQtXBvb4ewTd0yIpJ7mQp3Y0zcNTM40OqiiIi0VKbCHYDuom61JyK5l71w18iQIiJZDHeNDCkikrlwN909oBt2iEjOZS7c424ZnVAVkXzLYLgXYWQIXy63uiQiIi2TwXBPBg/Tr1RFJMfCqVaw1t4NfALY5Zw7Y5LlBrgTuAAYBK5wzv1mpgvasPpfqc5b0LJiiIi0UiMt93uAjx9m+RpgefK4Gvj+kRereaaowcNERKYMd+fcL4H+w6yyFrjXOeedc08DC6y1i2eqgNPWrWF/RURmos99KbCtbnp7Mq81qiND6lp3EcmxKfvcG2AmmTfpHaqttVcTd93gnKNUKjW1wzAMD/neqL2NN4Ee4+lucvtpcri6yBPVQ0z1EFM9zEy4bweOr5s+DuibbEXn3DpgXTLpd+/e3dQOS6USh3qvjyoA7N+1k8Emt58mh6uLPFE9xFQPsSzXw5IlSxpabybCfQNwnbX2p8BKYK9zbscMbLcpJihAV1EjQ4pIrjVyKeRPgHOAkrV2O3AT0AbgnLsLeJT4MsiXiC+F/OxsFbZh3UVdLSMiuTZluDvnLp1iuQeunbESzYRir06oikiuZe8XqhCPDKlLIUUkxzIZ7qa7R90yIpJrmQx33bBDRPIuo+FehMH9eD/p5fYiIpmX0XDvhUoFRoZaXRIRkZbIZrjXRobUte4ikk+ZDHeNDCkieZfJcNcNO0Qk77IZ7vU37BARyaFshnvSLePVLSMiOZXNcNcNO0Qk57IZ7h2dUAjVLSMiuZXJcDfGaHwZEcm1TIY7AN096nMXkdzKbrgXe9QtIyK5ld1w71a3jIjkV2bD3WhkSBHJscyGu06oikieZTfcu3tgaBBfLre6JCIicy674V4dX2ZII0OKSP5kONyL8bP63UUkhzIb7qbacte17iKSQ5kN9wPjy6hbRkTyJ7vhrpEhRSTHMhzuumGHiORXdsO9SydURSS/MhvuJgyhs0snVEUklzIb7oDGlxGR3Mp2uBd78OqWEZEcyni496rlLiK5lO1w79aY7iKST5kOd1Ps0QlVEcmlTId79YSq977VJRERmVPZDvdiL5TLMDrS6pKIiMypjId7Mr6M+t1FJGcyHe6mGu6D6ncXkXwJG1nJWvtx4E6gAPzQOfcPE5ZfAfx34PVk1nedcz+cwXI2p1stdxHJpynD3VpbAL4HrAa2A89Yazc4516YsOo/Oeeum4UyNk/dMiKSU410y3wIeMk597JzbhT4KbB2dos1Q5KRITXsr4jkTSPdMkuBbXXT24GVk6z3aWvt3wB/AL7knNs2cQVr7dXA1QDOOUql0vRLDIRh2NB7o2IXbwJFA8Um93W0a7Qusk71EFM9xFQPjYW7mWTexAvHHwZ+4pwbsdZeA/wYOHfim5xz64B11W3s3r17OmWtKZVKNPJe7z0EAQO7djLU5L6Odo3WRdapHmKqh1iW62HJkiUNrddIuG8Hjq+bPg7oq1/BObenbvIHwH9raO+zzBgTd82oz11EcqaRPvdngOXW2pOtte3AJcCG+hWstYvrJv8W2DpzRTxCGvZXRHJoypa7c65srb0O+D/El0Le7ZzbYq39FvCsc24D8AVr7d8CZaAfuGIWyzw9xR6dUBWR3DEtHHfF9/X1Tb3WJKbTn1a587/Cvr0UvnF7U/s62mW5b3E6VA8x1UMsy/WQ9LlPdi50nEz/QhU0MqSI5FPmw1037BCRPMp+uHf3wOAAPqq0uiQiInMm++FeGzxsoLXlEBGZQ9kP9+rgYeqaEZEcyXy4m2R8Gf2QSUTyJPPhrpEhRSSPchPu+iGTiORJbsJdfe4ikifZD3fdjUlEcijz4W7CNujoVLiLSK5kPtyBuGtG3TIikiP5CPdujQwpIvmSj3DXDTtEJGfyEe7dRXXLiEiu5CLcjVruIpIzuQh33WpPRPImH+Fe7IGxUfzoSKtLIiIyJ3IS7ho8TETyJXXh/sb+Ub775Cv8cc8QDd//VcP+ikjOhK0uwHT9Yfcw//u3O/hJ5Dm2p42zT+jlr0+cx8nHdGDM5PeMNcUePOheqiKSG6kL978+aR7nnXECjz7/Kk+9to+Htvaz/oV+Fve2cfYJ8/jwib2ctGBC0GvwsFTwb+2Bl1+EE5dhSse2ujgiqZa6cAeY1xly/rIFnL9sAe+MVHh62z6eeu0d1r+wh3/esoel89o5+4RePnziPE5c0FHrlokeuBfz3L/CwiWwcDFm4RI4djGm2m0jc8p7Dzu24f/tafxvN8GrfzywcPHxmL88C/P+s2DZaZgwlV9VkZYxDfdbzzzf19fX1BtLpRK7d+8+aP7e4TL/b9s+nnptH7/fNUjk4fj57Zx9fC9/9Yefs/jPmynseh36J7y3Z964sOfdizHHLoGFSzDdxabKOFcOVRdHKx9V4OUX8f+2Cf/bp2HXjnjByf8Os2IlZvnp+Ff/iN/8LPxhC1TK0FXEvG8F/OVZmDPOxMw75qDtpq0eZovqIZbleliyZAnA5H3QdTIV7vXeHirzr0mLfsuuITwQGJjXUWBBR4H5QZkF0TALRvcxf6Cf+e/sYn5/Hwv6X2fB6H7mje0n9BGEIbR3QHtn/NzRkTzH06a988C86vy2Nii0xe8tFJLnNkz1dRhCIXmEdc9h2/jXybMJDn3eOw1fYj86Alt/h//t0/jnfw379saf89T3Y1aswqz4EGbBXxz8vuFBeOF5/OZn8Zufg7398YKTlmPen7TqT1yGCYJU1MNcUD3EslwPuQ/3ev1DZZ59fT9vDozx9nCZt4crvD2UPA+XGa1MXge9pkwXZUIfUfARYVQh9BUKUZkweRQqZcLKGIXKGGF5LJ7vK7RXxmiPxuiIRmmvlGmPqtNjtWXxo1ybDn2lto8wKsf79JU43KthXwiTg0cc/GFbG+XIx0cuE4AxECTPJjjwuv45KEChEA+HXKg/AIUHDkiFtvHzC4XkvUG83eq0CeKDVm1ZAQrxa//2W3GYb/kNjAxDZ1ccyCtWYs74IGMdXQyORgyMRQyOVRgYjRguR7QXDF1hQGdbQFcY0NUW0FEwtPe9Apufw//+ubhv3nuYtwBz+pl0n3ASg6Nj4w6K1ddmknm1A2kwsZ7qP4sZ/5mrrzHxfy0T1J4PdTJ/rmU51KYjy/WgcG+0EN4zVI7YmwR9Nfir08PliHLkKUdQjjyVyCfTnor3By0bq0SUK56Rimc08kQzUL0hEQU8IRGhrz5HFKhgMER1/4Y+eeDrpw31xTDexwcpX6FQO2DVTUfleLq6zEcEPiLAg/cEeIz3ca7hMT6KXyfLwBN4T2QCBrvmM/iuRQz2vouBti4GxzyDYxGDY3G9TkdgoKstoDMM6Aqga2yQzsF36HpnD0F5BOOpfdK4XBOnx78O6p6D6mf0cW3Vf2bjk+XEy6vLausk6xWS9wfGU/CeIPl3C2q1Xz0oGIxn/AECMNUXQVyzkTF4DJEJ8CZ+jkxQmxcZU5vviZcVAkOQNAzafESBiBBPgQohPvneRLXXYbV8xtTFhYmnIXmuW5ZM+6RsHvAm/oS1cibrRZjap4/qthcQX4Ntkt0E3sfPxtSKEeApACYI4r0Yk1RdvH9jPCapOwMQBPGzieuxq7OTsdHhZFvJNuv2UftcUPs3qTWIag2jifPMgYN7/bIoSh4V8FHddN386utkuTn9A5gz/2pa3/+qRsM992epjDF0txXobiuwuLd9xrdfjjyjlYjRsmekEjFa8fGjHMUHgLp55WjyRyU5eIzVH1wqnrL3dHR0MDoyciAYqvkBB313q/ER+WSb/sD2Kh4qyfYHa/ute3iP9x7vkwPGhNfVnI7q5hlj6O4IKbYV6G4POKYt4Lh58evutqA2v/51ZxgwWokYGotb8fGzZ2gsYqicPGrLOhieN59df7EUgoByuQx15TzwOpkmKXOt7HG5IyDycSj52jRExPOiqf8fydFmdJJ5ScsnbpgkB+LkwG6Sg7qprse4F4dlksZMfaMBqM0LONAQCrwHA6tfGeaiM4/4Ux5W7sN9toWBIQwKdLfNzvaz/OfndMx2PUTJAaz6XEn+KquMm66bFzFuut5kfyxPFiNxa9MkLc64IWKS1wGm1nisrmOM4ZhjjmHX7j2UI8b9lXlQY8HHDYRKcoBvplup2mgIklZ18gcHAeZAQ5cDrfHqLqr1WD2QVg+21YZBtd6qjYZq3VQPyNX68xPqsn5ZV3eR/QP7438H6vZX+3ccv/2K95P+u4zfQXJ0qP5pXG0kmOpfx0kDIfmLJkrWi/dVbTjE7znmuNm/Qk/hLtKAaoA28NdwS5V6OjDDM/8XaNqo0ZPC4QdERGRqCncRkQxSuIuIZJDCXUQkgxTuIiIZpHAXEckghbuISAYp3EVEMqilY8u0asciIik35a/pWtlyN80+rLXPHcn7s/RQXageVA+5rIcpqVtGRCSDFO4iIhmU1nBf1+oCHEVUFzHVQ0z1EMt9PbTyhKqIiMyStLbcRUTkMFI3nru19uPAnUAB+KFz7h9aXKSWsNa+CuwDKkDZOXdWa0s0d6y1dwOfAHY5585I5r0L+CfgJOBVwDrn3mpVGefCIerhm8BVwJvJal9zzj3amhLODWvt8cC9wCLie3Osc87dmcfvRL1UtdyttQXge8Aa4H3Apdba97W2VC31H5xzK/IU7Il7gI9PmPdV4HHn3HLg8WQ66+7h4HoAuCP5XqzIerAnysDfOedOA1YB1ya5kMfvRE2qwh34EPCSc+5l59wo8FNgbYvLJHPMOfdLoH/C7LXAj5PXPwY+NaeFaoFD1EPuOOd2OOd+k7zeB2wFlpLD70S9tIX7UmBb3fT2ZF4eeeD/Wmufs9Ze3erCHAWOdc7tgPg/O7CwxeVppeustb+z1t5trT2m1YWZS9bak4APAJvI+XcibeE+2S+z8nq5z9nOuTOJu6iutdb+TasLJEeF7wPLgBXADuC21hZn7lhre4AHgC86595pdXlaLW3hvh04vm76OKCvRWVpKedcX/K8C3iQuMsqz96w1i4GSJ53tbg8LeGce8M5V3HORcAPyMn3wlrbRhzs9zvn1iezc/2dSFu4PwMst9aebK1tBy4BNrS4THPOWlu01vZWXwMfBX7f2lK13Abg8uT15cBDLSxLy1TDLHEROfheWGsN8I/AVufc7XWLcv2dSN2PmKy1FwD/g/hSyLudc7e0uEhzzlr7HuLWOsSXs/6vPNWDtfYnwDlACXgDuAn4F8ABJwB/Bj7jnMv0ycZD1MM5xF0ynvjyv89V+52zylr7YeBJYDPxpZAAXyPud8/Vd6Je6sJdRESmlrZuGRERaYDCXUQkgxTuIiIZpHAXEckghbuISAYp3EVEMkjhLiKSQQp3EZEM+v8lWPGp1WM2UwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f23c503c198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_rnn.fit([data_pairs_seq[:, 0],  data_pairs_seq[:, 1], \n",
    "               data_pairs_track[:,0], data_pairs_track[:,1]], \n",
    "              labels, \n",
    "              batch_size       = 24, \n",
    "              verbose          = 0, \n",
    "              epochs           = 25,\n",
    "              callbacks        = [tut_func.PlotLosses()], \n",
    "              validation_split = 0.1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate by Example\n",
    "\n",
    "Due to the changed input, we also have to adapt the evaluation function to display example result-lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def similar_rnn(model, query_idx):\n",
    "    \n",
    "    print metadata.iloc[query_idx]\n",
    "    \n",
    "    # calclulate predicted distances between query track and all others\n",
    "    res = [model.predict([sequential_features[[query_idx]], sequential_features[[i]], \n",
    "                          trackbased_features[[query_idx]], trackbased_features[[i]]]) \\\n",
    "           for i in range(feature_data.shape[0])]\n",
    "\n",
    "    # reshape\n",
    "    res = np.array(res)\n",
    "    res = res.reshape(res.shape[0])\n",
    "\n",
    "    # get sorted indexes in ascending order (smallest distance to query track first)\n",
    "    si = np.argsort(res)\n",
    "    \n",
    "    # output filter\n",
    "    display_cols = [\"artist_name\", \"title\", \"album_name\", \"year\",\"playlist\"]\n",
    "    \n",
    "    return metadata.loc[si, display_cols][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check the results for some individual tracks by supplying the index to our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "similar_rnn(model_rnn, 330)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(similarity_function, cut_off):\n",
    "\n",
    "    all_precisions = []\n",
    "    \n",
    "    pbar = progressbar.ProgressBar()\n",
    "\n",
    "    for idx in pbar(metadata.index.values):\n",
    "\n",
    "        dist           = similarity_function(sequential_features, sequential_features[[idx]], \n",
    "                                             trackbased_features, trackbased_features[[idx]])\n",
    "        \n",
    "        dist           = np.array(dist).reshape(len(dist))\n",
    "        \n",
    "        similar_tracks = metadata.loc[np.argsort(dist)[::-1][:cut_off]]\n",
    "        same_label     = similar_tracks[\"playlist\"] == metadata.loc[idx, \"playlist\"]\n",
    "        precision      = same_label.sum() / float(cut_off)\n",
    "        all_precisions.append(precision)\n",
    "\n",
    "    all_precisions = np.array(all_precisions)\n",
    "\n",
    "    return all_precisions.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluate(lambda w,x,y,z: [model.predict([w[[i]],x,y[[i]],z]) for i in range(sequential_features.shape[0])], 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusions\n",
    "\n",
    "In this part of the tutorial I have demonstrated how to use neural networks to train a similarity function. First the model trained on the same aggregated features as used in part one of this tutorial. A huge improvement in performance could be noticed. Further, it was demonstrated how to harness temporal relationships by training LSTMs on the sequential data provided by Spotify.\n",
    "\n",
    "In the next part of this tutorial series I will demonstrate how to use the so far disregarded genre labels provided by Spotify. I will show how to apply tag-similarity metrics to automatically create the similarity lookup table for the ground-truth. This facilitates more precise input-pairs for the model to train on."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "notify_time": "5",
  "toc": {
   "toc_cell": false,
   "toc_number_sections": true,
   "toc_threshold": 6,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}