02.Country-Node-creation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Storing the countries in Neo4j\n",
    "\n",
    "We have a master list of ISO countries that we can use to create our base country nodes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "iso_data = pd.read_json('./data/iso3166_country_codes.json')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>alt_name</th>\n",
       "      <th>iso3166_code</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>AF</td>\n",
       "      <td>Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Aland</td>\n",
       "      <td>AX</td>\n",
       "      <td>Aland Islands</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Albanie</td>\n",
       "      <td>AL</td>\n",
       "      <td>Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Algerie</td>\n",
       "      <td>DZ</td>\n",
       "      <td>Algeria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Samoa americaines</td>\n",
       "      <td>AS</td>\n",
       "      <td>American Samoa</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            alt_name iso3166_code            name\n",
       "0        Afghanistan           AF     Afghanistan\n",
       "1              Aland           AX   Aland Islands\n",
       "2            Albanie           AL         Albania\n",
       "3            Algerie           DZ         Algeria\n",
       "4  Samoa americaines           AS  American Samoa"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iso_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now we can loop over the data and insert each country with it's corresponding code and name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from neo4j.v1 import GraphDatabase\n",
    "driver = GraphDatabase.driver(\"bolt://10.0.0.1:7687\", auth=(\"myusername\", \"mypassword\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we create a list of all the new nodes we wish to create and their associated properties. Then we can connect to the neo4j database and run a Cypher command to insert and create new nodes; we use MERGE in case we have any duplicate entries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "countryList = [{'code': row.iso3166_code, 'name': row['name'].strip().upper()} for ind, row in iso_data.iterrows()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with driver.session() as session:\n",
    "    session.run((\"UNWIND {list} AS d \"\n",
    "                 \"MERGE (c:Country {code: d.code, name: d.name})\"),\n",
    "                {\"list\": countryList})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combining the ISO country codes with secrecy data\n",
    "We want to incorporate finanical secrecy data into our modelling of the network. We used an edited version of the financial secrecy classification of countries but a similar data set is available to the public as an Excel file here:\n",
    "- Financial Secrecy Index 2018 Results; https://www.financialsecrecyindex.com/introduction/fsi-2018-results\n",
    "\n",
    "Be warned the data will take a little bit of cleaning up afterwards it can be used in a similar fashion as indicated below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "secrecy_csv = './data/financialSecrecy.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "code2name = {row['code']: row['name'] for row in countryList}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Country & Nationality mapping\n",
    "\n",
    "In many instances we will need to be able to convert between country and nationality when connecting people and companies to countries.\n",
    "\n",
    "We have three different source files that we can use, natively stored as JSON we will convert them to pickles for reasy use within python. The 3 files are:\n",
    "\n",
    "- Country names to country code mappings\n",
    "- Country code to country name mappings\n",
    "- Nationality to country code mappings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def convert_score(datum):\n",
    "    components = datum.split('-')\n",
    "    if len(components) == 1:\n",
    "        return float(components[0])\n",
    "    else:\n",
    "        return (float(components[0][1:])+float(components[1][:-1]))/2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "secrecy_df = pd.read_csv(secrecy_csv)\n",
    "secrecy_df['final_secrecy_score'] = secrecy_df['Secrecy Score'].map(convert_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>RANK</th>\n",
       "      <th>Jurisdiction</th>\n",
       "      <th>(Formula) FSI Value</th>\n",
       "      <th>Secrecy Score</th>\n",
       "      <th>Global Scale Weight</th>\n",
       "      <th>Unnamed: 5</th>\n",
       "      <th>final_secrecy_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NA7</td>\n",
       "      <td>Maldives</td>\n",
       "      <td>-</td>\n",
       "      <td>(76-84)</td>\n",
       "      <td>0.000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NA7</td>\n",
       "      <td>Paraguay</td>\n",
       "      <td>-</td>\n",
       "      <td>(75-83)</td>\n",
       "      <td>0.001</td>\n",
       "      <td>NaN</td>\n",
       "      <td>79.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NA7</td>\n",
       "      <td>Gambia</td>\n",
       "      <td>-</td>\n",
       "      <td>(73-81)</td>\n",
       "      <td>0.000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NA7</td>\n",
       "      <td>Tanzania</td>\n",
       "      <td>-</td>\n",
       "      <td>(73-81)</td>\n",
       "      <td>0.006</td>\n",
       "      <td>NaN</td>\n",
       "      <td>77.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NA7</td>\n",
       "      <td>Bolivia</td>\n",
       "      <td>-</td>\n",
       "      <td>(72-80)</td>\n",
       "      <td>0.001</td>\n",
       "      <td>NaN</td>\n",
       "      <td>76.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  RANK Jurisdiction (Formula) FSI Value Secrecy Score  Global Scale Weight  \\\n",
       "0  NA7     Maldives                   -       (76-84)                0.000   \n",
       "1  NA7     Paraguay                   -       (75-83)                0.001   \n",
       "2  NA7       Gambia                   -       (73-81)                0.000   \n",
       "3  NA7     Tanzania                   -       (73-81)                0.006   \n",
       "4  NA7      Bolivia                   -       (72-80)                0.001   \n",
       "\n",
       "  Unnamed: 5  final_secrecy_score  \n",
       "0        NaN                 80.0  \n",
       "1        NaN                 79.0  \n",
       "2        NaN                 77.0  \n",
       "3        NaN                 77.0  \n",
       "4        NaN                 76.0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "secrecy_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "secrecy_df['COUNTRY_CODE'] = secrecy_df.Jurisdiction.map(lambda x: country_names_2_codes.get(x.strip().upper(), 'missing'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "missing_regions = secrecy_df[secrecy_df.COUNTRY_CODE == 'missing'].Jurisdiction.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Samoa', 'Marshall Islands', 'Malaysia (Labuan)', 'Monaco',\n",
       "       'Turks & Caicos Islands', 'US Virgin Islands', 'Anguilla',\n",
       "       'Curacao', 'Montserrat', 'Korea', 'Portugal (Madeira)'], dtype=object)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "missing_regions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from fuzzywuzzy import fuzz\n",
    "from fuzzywuzzy import process"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Samoa [('AMERICAN SAMOA', 90), ('CAMBODIA', 62), ('ZAMBIA', 55)]\n",
      "St Lucia [('ST. LUCIA', 95), ('ST. VINCENT & GRENADINES', 86), ('ST. KITTS & NEVIS', 86)]\n",
      "Brunei Darussalam [('BRUNEI', 90), ('RUSSIA', 75), ('BURUNDI', 64)]\n",
      "Marshall Islands [('CHANNEL ISLANDS', 71), ('COOK ISLANDS', 70), ('ÌÉLAND ISLANDS', 70)]\n",
      "St Vincent & the Grenadines [('ST. VINCENT & GRENADINES', 95), ('ST. KITTS & NEVIS', 86), ('ST. LUCIA', 86)]\n",
      "St Kitts & Nevis [('ST. KITTS & NEVIS', 97), ('ST. VINCENT & GRENADINES', 86), ('ST. LUCIA', 86)]\n",
      "United Arab Emirates (Dubai) [('UNITED ARAB EMIRATES', 95), ('UNITED KINGDOM', 86), ('UNITED STATES', 86)]\n",
      "Malaysia (Labuan) [('MALAYSIA', 90), ('MALTA', 72), ('ALBANIA', 64)]\n",
      "Monaco [('MOROCCO', 62), ('MACEDONIA', 60), ('NAN', 60)]\n",
      "Turks & Caicos Islands [('CAYMAN ISLANDS', 86), ('COOK ISLANDS', 86), ('ÌÉLAND ISLANDS', 86)]\n",
      "Macao [('MACAU', 80), ('JAMAICA', 67), ('MALTA', 60)]\n",
      "US Virgin Islands [('BRITISH VIRGIN ISLANDS', 86), ('COOK ISLANDS', 70), ('ÌÉLAND ISLANDS', 70)]\n",
      "Anguilla [('ANGOLA', 71), ('GUINEA', 57), ('BANGLADESH', 56)]\n",
      "Curacao [('CUBA', 68), ('USA', 60), ('CROATIA', 57)]\n",
      "Montserrat [('SIERRA LEONE', 61), ('SERBIA', 60), ('MONTENEGRO', 60)]\n",
      "Korea [('SOUTH KOREA', 90), ('NORTH KOREA', 90), ('KENYA', 60)]\n",
      "Portugal (Madeira) [('PORTUGAL', 90), ('IRAN', 68), ('IRAQ', 68)]\n",
      "Czech Republic [('CENTRAL AFRICAN REPUBLIC', 86), ('DOMINICAN REPUBLIC', 69), ('REPUBLIC OF GHANA', 69)]\n"
     ]
    }
   ],
   "source": [
    "for region in missing_regions:\n",
    "    print(region, process.extract(region, country_names_2_codes.keys(), limit=3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are certainly things we are missing and need to be added to our data\n",
    "\n",
    "- St Lucia == ST. LUCIA\n",
    "- Brunei Darussalam == BRUNEI\n",
    "- St Vincent & the Grenadines == ST. VINCENT & GRENADINES\n",
    "- St Kitts & Nevis == ST. KITTS & NEVIS\n",
    "- United Arab Emirates (Dubai) == UNITED ARAB EMIRATES\n",
    "- Macao == MACAU\n",
    "- Czech Republic == CZECHIA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "extra_names = ['St Lucia', 'Brunei Darussalam', 'St Vincent & the Grenadines', 'St Kitts & Nevis', 'United Arab Emirates (Dubai)', 'Macao', 'Czech Republic']\n",
    "extra_names = [name.upper() for name in extra_names]\n",
    "name4code = ['ST. LUCIA', 'BRUNEI', 'ST. VINCENT & GRENADINES', 'ST. KITTS & NEVIS', 'UNITED ARAB EMIRATES', 'MACAU', 'CZECHIA']\n",
    "\n",
    "extra_names2codes = {vals[0]: country_names_2_codes.get(vals[1]) for vals in zip(extra_names, name4code)}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "country_names_2_codes.update(extra_names2codes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.to_pickle(country_names_2_codes, \"data/combined_country_map.pkl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we then re-run the above to get the new secrecy numbers we get a new dataframe that we can use to enrich the country nodes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "filtered_df = secrecy_df[secrecy_df.COUNTRY_CODE != 'missing']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MV 80.0\n",
      "PY 79.0\n",
      "GM 77.0\n",
      "TZ 77.0\n",
      "BO 76.0\n",
      "TW 71.0\n",
      "DO 69.0\n",
      "VE 68.0\n",
      "ME 64.0\n",
      "VU 87.0\n",
      "LR 83.0\n",
      "LC 83.0\n",
      "BN 83.0\n",
      "AG 81.0\n",
      "LB 79.0\n",
      "BS 79.0\n",
      "BZ 79.0\n",
      "NR 78.91\n",
      "BB 78.0\n",
      "VC 78.0\n",
      "KN 78.0\n",
      "AE 77.0\n",
      "AD 77.0\n",
      "LI 76.0\n",
      "GT 76.0\n",
      "GD 76.0\n",
      "DM 76.0\n",
      "CK 76.0\n",
      "BH 74.0\n",
      "CH 73.0\n",
      "HK 72.0\n",
      "PA 72.0\n",
      "MU 72.0\n",
      "UY 71.0\n",
      "BW 71.0\n",
      "SC 71.0\n",
      "MO 70.0\n",
      "SM 70.0\n",
      "SG 69.0\n",
      "AW 68.0\n",
      "GH 67.0\n",
      "GI 67.0\n",
      "BM 66.0\n",
      "MK 66.0\n",
      "KY 65.0\n",
      "JE 65.0\n",
      "GG 64.0\n",
      "TR 64.0\n",
      "IM 64.0\n",
      "PH 63.0\n",
      "SA 61.0\n",
      "US 60.0\n",
      "VG 60.0\n",
      "JP 58.0\n",
      "DE 56.0\n",
      "LU 55.0\n",
      "CR 55.0\n",
      "CN 54.0\n",
      "AT 54.0\n",
      "RU 54.0\n",
      "CL 54.0\n",
      "IL 53.0\n",
      "BR 52.0\n",
      "MT 50.0\n",
      "CY 50.0\n",
      "SK 50.0\n",
      "NL 48.0\n",
      "CA 46.0\n",
      "NZ 46.0\n",
      "IS 46.0\n",
      "MX 45.0\n",
      "LV 45.0\n",
      "EE 44.0\n",
      "FR 43.0\n",
      "AU 43.0\n",
      "ZA 42.0\n",
      "GB 41.0\n",
      "BE 41.0\n",
      "IE 40.0\n",
      "IN 39.0\n",
      "NO 38.0\n",
      "SE 36.0\n",
      "PL 36.0\n",
      "HU 36.0\n",
      "GR 36.0\n",
      "IT 35.0\n",
      "CZ 35.0\n",
      "SI 34.0\n",
      "ES 33.0\n",
      "DK 31.0\n",
      "FI 31.0\n"
     ]
    }
   ],
   "source": [
    "with driver.session() as session:\n",
    "    for i, row in filtered_df.iterrows():\n",
    "        print(row['COUNTRY_CODE'], row['final_secrecy_score'])\n",
    "        result = session.run(\"MATCH (c:Country {code: {code}}) SET c.secrecy_score = {score}\", \n",
    "                             code=row['COUNTRY_CODE'], \n",
    "                             score=row['final_secrecy_score'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Country codes to names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "country_codes_2_names = json.load(open('data/clean_country_code_map.json', 'r'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From our data we created clean country codes, do they match with what ISO data we have?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DQ DQ\n",
      "XK KOSOVO\n"
     ]
    }
   ],
   "source": [
    "for key in country_codes_2_names.keys():\n",
    "    result = code2name.get(key, '**')\n",
    "    if result == '**':\n",
    "        print(key, country_codes_2_names.get(key))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like we are missing Kosovo which has code XK. DQ seems to be a mistake as it is Dominica and has real code DM. So let's add Kosovo and then save the output to a pkl."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "code2name['XK'] = 'KOSOVO'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pd.to_pickle(code2name, \"data/clean_country_code_map.pkl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also fix the DQ in the country_codes_2_names file ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DOMINICA\n"
     ]
    }
   ],
   "source": [
    "country_codes_2_names.pop('DQ', None)\n",
    "country_codes_2_names['DM'] = code2name.get('DM')\n",
    "print(country_codes_2_names['DM'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Country names to codes\n",
    "\n",
    "We also need a reverse lookup and since there are a variety of ways of doing the lookup let's integrate all the names we have to their respective codes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "country_names_2_codes = json.load(open(\"data/country_name_2_code_map.json\", \"r\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "country_names_2_codes = {str(k).upper(): str(v).upper()for k,v in country_names_2_codes.items()}\n",
    "country_names_2_codes.get('DQ')\n",
    "country_names_2_codes['DQ'] = 'DM'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "195\n",
      "442\n",
      "519\n"
     ]
    }
   ],
   "source": [
    "print(len(country_codes_2_names))\n",
    "country_codes_2_names.update({v: k for k,v in code2name.items()})\n",
    "print(len(country_codes_2_names))\n",
    "country_codes_2_names.update({v: k for k,v in country_codes_2_names.items()})\n",
    "print(len(country_codes_2_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pd.to_pickle(country_names_2_codes, \"data/combined_country_map.pkl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nationality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "nationality_2_codes = json.load(open(\"data/nationality_map.json\", \"r\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "nationality_2_codes.pop('(blank)', None)\n",
    "nationality_2_codes['domincan'] = 'DM'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pd.to_pickle(nationality_2_codes, \"data/nation_map.pkl\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}