2020.09.12 Predicting the 2020-21 EPL season before matchday 1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting the 2020/21 EPL season!!!\n",
    "Credit to [Tuan Doan Nguyen](https://towardsdatascience.com/o-jogo-bonito-predicting-the-premier-league-with-a-random-model-1b02fa3a7e5a) for inspiration :)   \n",
    "Blue is the colour, football is the game...  \n",
    "KTBFFH"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from scipy.stats import poisson\n",
    "from glob import glob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data prep - create kaggle-like data set for all historical EPL match results (1993-)\n",
    "source csv -> http://www.football-data.co.uk/englandm.php  \n",
    "check against -> https://www.premierleague.com/results?co=1&se=2&cl=-1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 10425 entries, 0 to 10424\n",
      "Data columns (total 13 columns):\n",
      " #   Column      Non-Null Count  Dtype         \n",
      "---  ------      --------------  -----         \n",
      " 0   Date        10424 non-null  datetime64[ns]\n",
      " 1   HomeTeam    10424 non-null  object        \n",
      " 2   AwayTeam    10424 non-null  object        \n",
      " 3   FTHG        10424 non-null  float64       \n",
      " 4   FTAG        10424 non-null  float64       \n",
      " 5   FTR         10424 non-null  object        \n",
      " 6   HTHG        9500 non-null   float64       \n",
      " 7   HTAG        9500 non-null   float64       \n",
      " 8   HTR         9500 non-null   object        \n",
      " 9   Referee     7600 non-null   object        \n",
      " 10  HS          6460 non-null   float64       \n",
      " 11  Attendance  759 non-null    float64       \n",
      " 12  Time        380 non-null    object        \n",
      "dtypes: datetime64[ns](1), float64(6), object(6)\n",
      "memory usage: 1.0+ MB\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>HomeTeam</th>\n",
       "      <th>AwayTeam</th>\n",
       "      <th>FTHG</th>\n",
       "      <th>FTAG</th>\n",
       "      <th>FTR</th>\n",
       "      <th>HTHG</th>\n",
       "      <th>HTAG</th>\n",
       "      <th>HTR</th>\n",
       "      <th>Referee</th>\n",
       "      <th>HS</th>\n",
       "      <th>Attendance</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1993-08-14</td>\n",
       "      <td>Southampton</td>\n",
       "      <td>Everton</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>A</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1993-08-14</td>\n",
       "      <td>Arsenal</td>\n",
       "      <td>Coventry</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>A</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1993-08-14</td>\n",
       "      <td>Aston Villa</td>\n",
       "      <td>QPR</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>H</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1993-08-14</td>\n",
       "      <td>Chelsea</td>\n",
       "      <td>Blackburn</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>A</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1993-08-14</td>\n",
       "      <td>Liverpool</td>\n",
       "      <td>Sheffield Weds</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>H</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date     HomeTeam        AwayTeam  FTHG  FTAG FTR  HTHG  HTAG  HTR  \\\n",
       "0 1993-08-14  Southampton         Everton   0.0   2.0   A   NaN   NaN  NaN   \n",
       "1 1993-08-14      Arsenal        Coventry   0.0   3.0   A   NaN   NaN  NaN   \n",
       "2 1993-08-14  Aston Villa             QPR   4.0   1.0   H   NaN   NaN  NaN   \n",
       "3 1993-08-14      Chelsea       Blackburn   1.0   2.0   A   NaN   NaN  NaN   \n",
       "4 1993-08-14    Liverpool  Sheffield Weds   2.0   0.0   H   NaN   NaN  NaN   \n",
       "\n",
       "  Referee  HS  Attendance Time  \n",
       "0     NaN NaN         NaN  NaN  \n",
       "1     NaN NaN         NaN  NaN  \n",
       "2     NaN NaN         NaN  NaN  \n",
       "3     NaN NaN         NaN  NaN  \n",
       "4     NaN NaN         NaN  NaN  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# data ingestion\n",
    "files = glob('./E0*.csv')\n",
    "\n",
    "matches = pd.DataFrame()\n",
    "for file in files:\n",
    "    #print(file)\n",
    "    matches = pd.concat([matches, \n",
    "                         pd.read_csv(file, usecols=range(1,12), encoding = \"latin\", date_parser='pandas.to_datetime')]\n",
    "                       ).drop_duplicates()\n",
    "\n",
    "\n",
    "matches['Date'] = pd.to_datetime(matches['Date'], dayfirst=True)\n",
    "matches = matches.sort_values('Date').reset_index(drop=True)\n",
    "matches.dropna(axis=1, how='all', inplace=True)\n",
    "matches.to_csv('./matches.csv')\n",
    "matches.info()\n",
    "matches.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>FTHG</th>\n",
       "      <th>FTAG</th>\n",
       "      <th>HTHG</th>\n",
       "      <th>HTAG</th>\n",
       "      <th>HS</th>\n",
       "      <th>Attendance</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>HomeTeam</th>\n",
       "      <th>AwayTeam</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">Arsenal</th>\n",
       "      <th>Aston Villa</th>\n",
       "      <td>2.714286</td>\n",
       "      <td>1.142857</td>\n",
       "      <td>0.857143</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>18.833333</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Birmingham</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blackburn</th>\n",
       "      <td>3.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>19.500000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blackpool</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bolton</th>\n",
       "      <td>3.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">Wolves</th>\n",
       "      <th>Tottenham</th>\n",
       "      <td>1.500000</td>\n",
       "      <td>2.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>1.250000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Watford</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West Brom</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West Ham</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.500000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Wigan</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>10.500000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1036 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                          FTHG      FTAG      HTHG      HTAG         HS  \\\n",
       "HomeTeam AwayTeam                                                         \n",
       "Arsenal  Aston Villa  2.714286  1.142857  0.857143  0.571429  18.833333   \n",
       "         Birmingham   2.000000  1.000000  1.000000  1.000000  18.000000   \n",
       "         Blackburn    3.500000  0.500000  1.500000  0.500000  19.500000   \n",
       "         Blackpool    6.000000  0.000000  3.000000  0.000000  26.000000   \n",
       "         Bolton       3.500000  0.500000  0.500000  0.500000  23.000000   \n",
       "...                        ...       ...       ...       ...        ...   \n",
       "Wolves   Tottenham    1.500000  2.500000  0.500000  1.250000  13.000000   \n",
       "         Watford      1.000000  1.000000  0.500000  1.000000  10.000000   \n",
       "         West Brom    2.000000  3.000000  1.500000  0.500000  12.000000   \n",
       "         West Ham     2.000000  0.333333  0.666667  0.000000  14.500000   \n",
       "         Wigan        2.000000  1.500000  0.500000  1.500000  10.500000   \n",
       "\n",
       "                      Attendance  \n",
       "HomeTeam AwayTeam                 \n",
       "Arsenal  Aston Villa         NaN  \n",
       "         Birmingham          NaN  \n",
       "         Blackburn           NaN  \n",
       "         Blackpool           NaN  \n",
       "         Bolton              NaN  \n",
       "...                          ...  \n",
       "Wolves   Tottenham           NaN  \n",
       "         Watford             NaN  \n",
       "         West Brom           NaN  \n",
       "         West Ham            NaN  \n",
       "         Wigan               NaN  \n",
       "\n",
       "[1036 rows x 6 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# data processing\n",
    "is_2020 = matches['Date'] > '2019-07'\n",
    "last_10 = matches['Date'] > '2010-07'\n",
    "#last_5 = (matches['Date'] > '2013-07') & (matches['Date'] < '2018-07')\n",
    "last_5 = matches['Date'] > '2015-07'\n",
    "\n",
    "# aggregate by home away team and return avg home away goals\n",
    "matches[last_10].groupby(['HomeTeam', 'AwayTeam']).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Man United          190\n",
       "Arsenal             190\n",
       "Liverpool           190\n",
       "Chelsea             190\n",
       "Man City            190\n",
       "Everton             190\n",
       "Tottenham           190\n",
       "West Ham            171\n",
       "Newcastle           171\n",
       "Stoke               152\n",
       "Southampton         152\n",
       "West Brom           152\n",
       "Crystal Palace      133\n",
       "Sunderland          133\n",
       "Aston Villa         133\n",
       "Swansea             133\n",
       "Leicester           114\n",
       "Bournemouth          95\n",
       "Burnley              95\n",
       "Norwich              95\n",
       "Watford              95\n",
       "Fulham               95\n",
       "Wolves               76\n",
       "Wigan                57\n",
       "QPR                  57\n",
       "Brighton             57\n",
       "Hull                 57\n",
       "Bolton               38\n",
       "Huddersfield         38\n",
       "Cardiff              38\n",
       "Blackburn            38\n",
       "Birmingham           19\n",
       "Sheffield United     19\n",
       "Reading              19\n",
       "Middlesbrough        19\n",
       "Blackpool            19\n",
       "Name: HomeTeam, dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "matches[last_10]['HomeTeam'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>HomeTeam</th>\n",
       "      <th>AwayTeam</th>\n",
       "      <th>FTHG</th>\n",
       "      <th>FTAG</th>\n",
       "      <th>FTR</th>\n",
       "      <th>HTHG</th>\n",
       "      <th>HTAG</th>\n",
       "      <th>HTR</th>\n",
       "      <th>Referee</th>\n",
       "      <th>HS</th>\n",
       "      <th>Attendance</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8753</th>\n",
       "      <td>2016-01-24</td>\n",
       "      <td>Arsenal</td>\n",
       "      <td>Chelsea</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>A</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>A</td>\n",
       "      <td>M Clattenburg</td>\n",
       "      <td>9.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8959</th>\n",
       "      <td>2016-09-24</td>\n",
       "      <td>Arsenal</td>\n",
       "      <td>Chelsea</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>H</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>H</td>\n",
       "      <td>M Oliver</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9502</th>\n",
       "      <td>2018-01-03</td>\n",
       "      <td>Arsenal</td>\n",
       "      <td>Chelsea</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>D</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>D</td>\n",
       "      <td>A Taylor</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9890</th>\n",
       "      <td>2019-01-19</td>\n",
       "      <td>Arsenal</td>\n",
       "      <td>Chelsea</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>H</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>H</td>\n",
       "      <td>A Taylor</td>\n",
       "      <td>13.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10240</th>\n",
       "      <td>2019-12-29</td>\n",
       "      <td>Arsenal</td>\n",
       "      <td>Chelsea</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>A</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>H</td>\n",
       "      <td>C Pawson</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Date HomeTeam AwayTeam  FTHG  FTAG FTR  HTHG  HTAG HTR  \\\n",
       "8753  2016-01-24  Arsenal  Chelsea   0.0   1.0   A   0.0   1.0   A   \n",
       "8959  2016-09-24  Arsenal  Chelsea   3.0   0.0   H   3.0   0.0   H   \n",
       "9502  2018-01-03  Arsenal  Chelsea   2.0   2.0   D   0.0   0.0   D   \n",
       "9890  2019-01-19  Arsenal  Chelsea   2.0   0.0   H   2.0   0.0   H   \n",
       "10240 2019-12-29  Arsenal  Chelsea   1.0   2.0   A   1.0   0.0   H   \n",
       "\n",
       "             Referee    HS  Attendance   Time  \n",
       "8753   M Clattenburg   9.0         NaN    NaN  \n",
       "8959        M Oliver  14.0         NaN    NaN  \n",
       "9502        A Taylor  14.0         NaN    NaN  \n",
       "9890        A Taylor  13.0         NaN    NaN  \n",
       "10240       C Pawson   NaN         NaN  14:00  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# head to head between every team pairing in the last 10 seasons\n",
    "h2h = matches[last_5].groupby(['HomeTeam', 'AwayTeam']).mean()\n",
    "matches[last_5].groupby(['HomeTeam', 'AwayTeam']).get_group(('Arsenal', 'Chelsea'))\n",
    "#h2h.filter(like='Chelsea', axis=0)\n",
    "#h2h.loc[('Arsenal', 'Chelsea')]\n",
    "#h2h.loc[['Chelsea']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modeling - get score function using nothing but the Poisson distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0, 4)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# modeling - get score function\n",
    "home = 'Brighton'\n",
    "away = 'Chelsea'\n",
    "\n",
    "def get_score(home, away):\n",
    "    # head to head results in last 5 seasons\n",
    "    home_mean = h2h.loc[(home, away)][0]\n",
    "    away_mean = h2h.loc[(home, away)][1]\n",
    "    \n",
    "    # simulate score by random sampling from parametrized Poisson distribution\n",
    "    home_score = poisson.rvs(home_mean, size=1)[0]\n",
    "    away_score = poisson.rvs(away_mean, size=1)[0]\n",
    "\n",
    "    return (home_score, away_score)\n",
    "\n",
    "get_score(home, away)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(12, (0, 3), 0.12),\n",
       " (12, (0, 1), 0.12),\n",
       " (11, (0, 2), 0.11),\n",
       " (10, (1, 1), 0.1),\n",
       " (9, (1, 2), 0.09),\n",
       " (8, (1, 3), 0.08),\n",
       " (5, (0, 5), 0.05),\n",
       " (5, (0, 0), 0.05),\n",
       " (4, (2, 1), 0.04),\n",
       " (3, (2, 2), 0.03),\n",
       " (3, (2, 0), 0.03),\n",
       " (3, (1, 5), 0.03),\n",
       " (3, (1, 4), 0.03),\n",
       " (3, (0, 4), 0.03),\n",
       " (2, (2, 3), 0.02),\n",
       " (1, (3, 4), 0.01),\n",
       " (1, (3, 3), 0.01),\n",
       " (1, (3, 2), 0.01),\n",
       " (1, (1, 6), 0.01),\n",
       " (1, (1, 0), 0.01),\n",
       " (1, (0, 11), 0.01),\n",
       " (1, (0, 6), 0.01)]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# simulate score over 10000 trials and plot histogram of most probable result\n",
    "sims = {}\n",
    "trials = 100\n",
    "for i in range(trials):\n",
    "    score = get_score(home, away)\n",
    "    sims[score] = sims.get(score, 0) + 1\n",
    "\n",
    "hist = []\n",
    "for k, v in sims.items():\n",
    "    p = v / trials\n",
    "    hist.append((v, k, p))\n",
    "    \n",
    "hist.sort(reverse=True)\n",
    "hist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('0-2', 13.4)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# modeling - get score function returns most probable result between two sides\n",
    "home = 'Brighton'\n",
    "away = 'Chelsea'\n",
    "trials = 10000\n",
    "\n",
    "def get_score(home, away):\n",
    "    try:\n",
    "        # head to head results in last 5 seasons\n",
    "        home_mean = h2h.loc[(home, away)][0]\n",
    "        away_mean = h2h.loc[(home, away)][1]\n",
    "\n",
    "        # simulate score by random sampling from parametrized Poisson distribution\n",
    "        home_scores = poisson.rvs(home_mean, size=trials).astype(str)\n",
    "        away_scores = poisson.rvs(away_mean, size=trials).astype(str)\n",
    "\n",
    "        scores = pd.DataFrame(data={'home':home_scores, 'away':away_scores})\n",
    "        scores['result'] = scores['home'] + '-' + scores['away']\n",
    "        predictions = scores['result'].value_counts()\n",
    "        probability = round(predictions / trials * 100, 1)\n",
    "        \n",
    "        return predictions.index[0], probability[0]\n",
    "\n",
    "    except KeyError:\n",
    "        # return NA for teams with no head to head record in last 5 seasons        \n",
    "        return 'N/A', 'N/A'\n",
    "    \n",
    "get_score(home, away)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data analysis - predict scores using most probable result\n",
    "Simulate 20/21 season opening week - games starting 2020-09-12"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Home</th>\n",
       "      <th>Away</th>\n",
       "      <th>Predictions</th>\n",
       "      <th>Probability%</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Fulham</td>\n",
       "      <td>Arsenal</td>\n",
       "      <td>1-5</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Crystal Palace</td>\n",
       "      <td>Southampton</td>\n",
       "      <td>0-0</td>\n",
       "      <td>16.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Liverpool</td>\n",
       "      <td>Leeds</td>\n",
       "      <td>N/A</td>\n",
       "      <td>N/A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>West Ham</td>\n",
       "      <td>Newcastle</td>\n",
       "      <td>2-1</td>\n",
       "      <td>9.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>West Brom</td>\n",
       "      <td>Leicester</td>\n",
       "      <td>0-2</td>\n",
       "      <td>9.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Tottenham</td>\n",
       "      <td>Everton</td>\n",
       "      <td>2-0</td>\n",
       "      <td>11.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Sheffield United</td>\n",
       "      <td>Wolves</td>\n",
       "      <td>1-0</td>\n",
       "      <td>37.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Brighton</td>\n",
       "      <td>Chelsea</td>\n",
       "      <td>0-2</td>\n",
       "      <td>13.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Home         Away Predictions Probability%\n",
       "0            Fulham      Arsenal         1-5            7\n",
       "1    Crystal Palace  Southampton         0-0         16.9\n",
       "2         Liverpool        Leeds         N/A          N/A\n",
       "3          West Ham    Newcastle         2-1          9.3\n",
       "4         West Brom    Leicester         0-2          9.5\n",
       "5         Tottenham      Everton         2-0         11.8\n",
       "6  Sheffield United       Wolves         1-0         37.2\n",
       "7          Brighton      Chelsea         0-2         13.3"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# data analysis - simulate 20/21 season opening week (week 1 games starting 2020-09-12)\n",
    "home_teams = ['Fulham', 'Crystal Palace', 'Liverpool', 'West Ham', 'West Brom', 'Tottenham', 'Sheffield United', 'Brighton']\n",
    "away_teams = ['Arsenal', 'Southampton', 'Leeds', 'Newcastle', 'Leicester', 'Everton', 'Wolves', 'Chelsea']\n",
    "\n",
    "week1 = pd.DataFrame(data={'Home':home_teams, 'Away':away_teams})\n",
    "week1['Predictions'] = week1.apply(lambda x: get_score(x.Home, x.Away)[0], axis=1)\n",
    "week1['Probability%'] = week1.apply(lambda x: get_score(x.Home, x.Away)[1], axis=1)\n",
    "week1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Come on Chelsea!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}