ARTM_example_RU.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Модельный эксперимент с использованием BigARTM Python API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Автор - **Мурат Апишев** (great-mel@yandex.ru)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BigARTM &ndash; открытая библиотека для тематического моделирования текстовых коллекций, реализующая теорию аддитивной регуляризации тематических моделей (ARTM). Основной сайт проекта http://bigartm.org/.\n",
    "\n",
    "Здесь показан пример использования пользовательского Python API библиотеки. Мы проведём демонстрационный эксперимент.\n",
    "\n",
    "#### Описание:\n",
    "Построим две тематические модели коллекции текстовых документов, ARTM и PLSA, сравним качество полученных моделей.\n",
    "\n",
    "#### Функционалы качества:\n",
    "Важным функционалом является перплексия коллекции. Тем не менее это далеко не единственная величина, характеризующая качество обучения. В BigARTM реализованы следующие функционалы:\n",
    "\n",
    "- разреженность матрицы $\\Phi$ (слова-темы);\n",
    "- разреженность матрицы $\\Theta$ (темы-документы);\n",
    "- характеристики ядер тем (чистота, контрастность, размер);\n",
    "- перплексия;\n",
    "- веса тем;\n",
    "- доля фоновых слов;\n",
    "- когерентность топ-слов и слов из ядер тем.\n",
    "\n",
    "Мы будем использовать первые четыре. Более сильные разреженности матриц и более высокие средние значения чистоты и контрастности способствуют большей интерпретируемости модели.\n",
    "\n",
    "#### Цель эксперимента:\n",
    "Попробуем обучить модель ARTM таким образом, чтобы, в сравнении с PLSA, улучшить значения разреженностей и ядровых характеристик и не сильно ухудшить перплексию.\n",
    "\n",
    "Основным инструментом для корректирования процесса обучения являются регуляризаторы. Список имеющихся в BigARTM регуляризаторов:\n",
    "\n",
    "- разреживание матрицы $\\Phi$ (+ частичное обучение);\n",
    "- разреживание матрицы $\\Theta$ (+ частичное обучение);\n",
    "- декоррелирование тем в матрице $\\Phi$;\n",
    "- принудительное разреживание $\\Phi$;\n",
    "- балансирование классов (Label Regularization);\n",
    "- повышение когерентности.\n",
    "\n",
    "В этом эксперименте для обучения ARTM воспользуемся первыми тремя регуляризаторами. ARTM без регуляризации соответствует PLSA.\n",
    "\n",
    "#### Коллекция:\n",
    "Воспользуемся небольшой коллекцией 'kos', доступной в репозитории UCI https://archive.ics.uci.edu/ml/machine-learning-databases/bag-of-words/. Параметры коллекции следующие:\n",
    "\n",
    "- 3430 документов;\n",
    "- 6906 слов в словаре;\n",
    "- 467714 ненулевых счётчиков в \"мешке слов\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Для начала подключим все необходимые модули (убедитесь, что путь к Python API BigARTM находится в вашей переменной PATH):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import glob\n",
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import artm\n",
    "\n",
    "print artm.version()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Прежде всего необходимо подготовить входные данные. BigARTM имеет собственный формат документов для обработки, называемый батчами. В библиотеке присутствуют средства по созданию батчей из файлов в форматах Bag-Of-Words UCI и Vowpal Wabbit (подробности можно найти в http://docs.bigartm.org/en/latest/formats.html).\n",
    "\n",
    "В Python API, по аналогии с алгоритмами из scikit-learn, входные данные представлены одним классом BatchVectorizer. Объект этого класса принимает на вход батчи или файлы UCI / VW и подаётся на вход всем методам. В случае, если входные данные не являются батчами, он создаёт их и сохраняет на диск для последующего быстрого использования.\n",
    "\n",
    "Итак, создадим объект BatchVectorizer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "batch_vectorizer = None\n",
    "if len(glob.glob(os.path.join('kos', '*.batch'))) < 1:\n",
    "    batch_vectorizer = artm.BatchVectorizer(data_path='', data_format='bow_uci', collection_name='kos', target_folder='kos')\n",
    "else:\n",
    "    batch_vectorizer = artm.BatchVectorizer(data_path='kos', data_format='batches')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "ARTM &ndash; это класс, представляющий собой Python API BigARTM, и позволяющий использовать практически все возможности библиотеки в стиле scikit-learn. Создадим две тематические модели для нашего эксперимента. Наиболее важным параметром модели является число тем. Опционально можно указать списки регуляризаторов и функционалов качества, которые следует использовать для данной модели. Если этого не сделать, то регуляризаторы и функционалы всегда можно добавить позднее. Обратите внимание, что каждая модель задаёт своё пространство имён для названий регуляризаторов и функционалов качества."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dictionary = artm.Dictionary()\n",
    "\n",
    "model_plsa = artm.ARTM(topic_names=['topic_{}'.format(i) for i in xrange(15)],\n",
    "                       scores=[artm.PerplexityScore(name='PerplexityScore',\n",
    "                                                    use_unigram_document_model=False,\n",
    "                                                    dictionary=dictionary)],\n",
    "                       cache_theta=True)\n",
    "\n",
    "model_artm = artm.ARTM(topic_names=['topic_{}'.format(i) for i in xrange(15)],\n",
    "                       scores=[artm.PerplexityScore(name='PerplexityScore',\n",
    "                                                    use_unigram_document_model=False,\n",
    "                                                    dictionary=dictionary)],\n",
    "                       regularizers=[artm.SmoothSparseThetaRegularizer(name='SparseTheta', tau=-0.15)],\n",
    "                       cache_theta=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Следующий шаг — инициализация моделей. Сделаем это по словарю, что означает, что\n",
    "- будет создана матрица $\\Phi$ с именем 'pwt', число строк и столбцов в ней будет взято исходя из числа слов в словаре и заданного в модели числа тем;\n",
    "- эта матрица будет заполнена случайными значениями из диапазона (0, 1) и нормализована.\n",
    "\n",
    "Надо отметить, что этот шаг является опциональным, поскольку модель может быть автоматически инициализирована во время вызовов fit_offline() / fit_online().\n",
    "\n",
    "Словарь &ndash; это объект BigARTM, содержащий информацию о коллекции (словарь коллекции, различные величины и счётчики, связанные со словами). Создать словарь можно на основе папки с батчами. Затем собранный словарь можно сохранять на диск и позже подгрузить вновь."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "if not os.path.isfile('kos/dictionary.dict'):\n",
    "    dictionary.gather(data_path=batch_vectorizer.data_path)\n",
    "    dictionary.save(dictionary_path='kos/dictionary.dict')\n",
    "\n",
    "dictionary.load(dictionary_path='kos/dictionary.dict')\n",
    "dictionary.load(dictionary_path='kos/dictionary.dict')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Затем словари можно использовать, чтобы инициализировать модель:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_plsa.initialize(dictionary=dictionary)\n",
    "model_artm.initialize(dictionary=dictionary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как уже было сказано, ARTM предоставляет возможность использовать все функционалы качества, имеющиеся в BigARTM. Если функционал подключен к модели, то модель будет сохранять все его значения, полученные на момент каждого обновления матрицы $\\Phi$. Добавим функционалы качества, нужные для нашего эксперимента, которые отсутствовали в конструкторах:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_plsa.scores.add(artm.SparsityPhiScore(name='SparsityPhiScore'))\n",
    "model_plsa.scores.add(artm.SparsityThetaScore(name='SparsityThetaScore'))\n",
    "model_plsa.scores.add(artm.TopicKernelScore(name='TopicKernelScore', probability_mass_threshold=0.3))\n",
    "\n",
    "model_artm.scores.add(artm.SparsityPhiScore(name='SparsityPhiScore'))\n",
    "model_artm.scores.add(artm.SparsityThetaScore(name='SparsityThetaScore'))\n",
    "model_artm.scores.add(artm.TopicKernelScore(name='TopicKernelScore', probability_mass_threshold=0.3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Аналогично поступим с регуляризаторами для model_artm (зададим им стартовые коэффициенты регуляризации, которые можно будет позже изменить при необходимости):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_artm.regularizers.add(artm.SmoothSparsePhiRegularizer(name='SparsePhi', tau=-0.1))\n",
    "model_artm.regularizers.add(artm.DecorrelatorPhiRegularizer(name='DecorrelatorPhi', tau=1.5e+5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Теперь попробуем обучить модели в оффлайн-режиме (т. е. обновляя Фи раз за проход по коллекции). Инициируем пятнадцать проходов:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_plsa.num_document_passes = 1\n",
    "model_artm.num_document_passes = 1\n",
    "\n",
    "model_plsa.fit_offline(batch_vectorizer=batch_vectorizer, num_collection_passes=15)\n",
    "model_artm.fit_offline(batch_vectorizer=batch_vectorizer, num_collection_passes=15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Проверим результаты первой итерации обучения, сравнив финальные значения функционалов, а также графики перплексии (опишем печать в виде функции для возможности повторного использования):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sparsity Phi: 0.000 (PLSA) vs. 0.469 (ARTM)\n",
      "Sparsity Theta: 0.000 (PLSA) vs. 0.001 (ARTM)\n",
      "Kernel contrast: 0.466 (PLSA) vs. 0.525 (ARTM)\n",
      "Kernel purity: 0.215 (PLSA) vs. 0.359 (ARTM)\n",
      "Perplexity: 2058.027 (PLSA) vs. 1950.717 (ARTM)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuclVXd///Xm7MIDCAwKMhJYQQlkTyC5Hg+3aFZeai+\nhlp2l6ZlB7Xb0n6VimZpByszRc1S85BaCGgwouYBFUQFAUWOwgByloMw8/n9sa6Rzcyemb03e8+1\nr+HzfDyux+y99tr7eg/WXnOtda21ZGY455xz2WoRdwDnnHPJ5A2Ic865nHgD4pxzLifegDjnnMuJ\nNyDOOedy4g2Ic865nBS0AZE0SNJ0Sa9HP9dJukxSF0mTJM2RNFFSScp7rpY0T9JsSSellA+XNFPS\nXEm3FjK3c865xqmp5oFIagEsAY4ALgU+NLObJF0JdDGzqyQNAe4HDgN6A88AA83MJL0MXGpm0ySN\nB24zs4lNEt4551wdTdmFdQLwnpktBs4A7onK7wHOjB6PBh4ws+1mtgCYBxwuqSfQ0cymRfXuTXmP\nc865GDRlA3IO8LfocamZVQKY2XKgR1TeC1ic8p6lUVkvwtVLjSVRmXPOuZg0SQMiqTXh6uIfUVHt\nfjNfT8U55xKmVROd51TgNTNbFT2vlFRqZpVR99SKqHwpsG/K+3pHZfWV1yHJGyPnnMuBmSmb+k3V\nhXUe8PeU508AY6LHXwUeTyk/V1IbSf2B/YFXom6udZIOlyTg/JT31GFmiT2uvfba2DPsjtk9f/yH\n54/3yEXBr0AktScMoF+cUjwWeEjShcBC4GwAM5sl6SFgFrAN+Jbt+M0uAcYB7YDxZjah0NnjsGDB\ngrgj5CzJ2cHzx83zJ0/BGxAz2wR0r1W2mtCopKt/A3BDmvLXgKGFyOiccy57PhO9yIwZMybuCDlL\ncnbw/HHz/MnTZBMJm4oka26/k3POFZokrEgH0ZvUxsqP4o6Qs4qKirgj5CzJ2cHzx83zJ0+zbECW\nVLwbdwTnnGv2mmUX1guXP8iIW8+OO4pzziWGd2FFtr45N+4IzjnX7DXLBmTY7Rc3XqlIJbkfNcnZ\nwfPHzfMnT7NsQLqU9Wi8knPOuV3SLMdAmtvv5JxzheZjIM4555qMNyBFJsn9qEnODp4/bp4/ebwB\ncc45l5NmOQYyq+3BfHDYmRz/3HVxx3HOuUTIZQykqTaUalKDt77Bunf7xR3DOeeatWbbhdVtdTIn\nEya5HzXJ2cHzx83zJ0+zbUD6fPwu27dsjzuGc841W81yDGRpy97sU7WEBc+8S7/j94s7knPOFT2f\nBxJZXlIGwKr/JrMbyznnkqBZNiDdH7qdNW8u4dBrTok7StaS3I+a5Ozg+ePm+ZOnWd6Fte/xg+KO\n4JxzzV6zHANpbr+Tc84VWt7ngUhqB/wPMArYB9gMvAX828zezjWoc8655Kt3DETST4EXgKOAl4E/\nAQ8B24EbJT0t6VNNknI3kuR+1CRnB88fN8+fPA1dgbxiZtfW89qvJPUA+hQgU95s3VQF1dW07dA6\n7ijOOdfsNNsxkCkHfZsRb9/Bq5fdx8jbfH9055xrSF7HQCQ9CdTbupjZ6GxO1NS0Z3va8jEfvzkn\n7ijOOdcsNTQP5JfALcD7hMHzP0fHRuC9wkfbNS2HhFt5W7+frMmESe5HTXJ28Pxx8/zJU+8ViJk9\nCyDpFjM7NOWlJyW9WvBku6jksDIYB11W+BWIc84VQqNjIJJmA6eb2fzoeX9gvJkNzugEUglwJ3AQ\nUA1cCMwFHgT6AguAs81sXVT/6qjOduByM5sUlQ8HxgHtovN/p57zmZmx8u0VdD+olLWUUFK1BrXI\nqmvPOed2K4VaC+u7QIWkCknPAlOAtF/e9biNHQ3OwcA7wFXAM2ZWBkwGro5+gSHA2cBg4FTgdkk1\nv9AfgIvMbBAwSNLJDZ202+DurFVnWshYt3BtFnGdc85lotEGxMwmAAOBy4HLgDIzm5jJh0vqBIwy\ns7ujz9oeXWmcAdwTVbsHODN6PBp4IKq3AJgHHC6pJ9DRzKZF9e5NeU/6c7cQrRfNp1PVWjr375JJ\n3KKQ5H7UJGcHzx83z588jTYgktoDPwAuNbM3gD6S/ifDz+8PrJJ0t6TXJd0RfV6pmVUCmNlyoEdU\nvxewOOX9S6OyXsCSlPIlUVmD9uzdBeRdV845VwiZdGHdDXxMmJEO4Uv95xl+fitgOPB7MxsOfETo\nvqo98NK8JqPsgvLy8rgj5CzJ2cHzx83zJ08mq/HuZ2bnSDoPwMw2pYxLNGYJsNjMau7aeoTQgFRK\nKjWzyqh7akX0+lJg35T3947K6itPa8yYMfTr1w+Azp07M2zYsE/+49ZcZvpzf+7P/fnu/LyiooJx\n48YBfPJ9mTUza/AA/gvsAbwePd+PsMxJo++N6j8LDIoeXwuMjY4ro7IrgRujx0OA6UAbQvfXu+y4\nU+wl4HBAwHjglHrOZ0k2ZcqUuCPkLMnZzTx/3Dx/vKLvzoy+12uOTK5ArgUmAPtKuh8YCYzJoo26\nDLhfUmtgPnAB0BJ4SNKFwELCnVeY2SxJDwGzgG3At6JfDOASdr6Nd0ImJ9+8fhuLpi5g/5MG0LJN\nyyxiO+eca0iD80CirqrewCbgSMJf/y+Z2aqmiZe92vuBLGo1gD5V7/v+6M4514C8zwOJvonHm9mH\nZvZvM/tXMTce6azqNACAlc/7jHTnnMunTO7Cel3SYQVPUiAb9wlrYn00PRlrYtUMciVRkrOD54+b\n50+eTMZAjgC+LGkh4TZcES5OErGZlA0qg7ehxTy/AnHOuXzKZC2svunKzWxhQRLtotpjIK/+7CkO\n/clpvN7lOIav/k+MyZxzrnjlez+QDma2saGGoqZONidsaj1GlbG6xV5s37Mk7ijOOdesNDQG8rik\nWyR9RtKeNYWSBki6SNJE4JTCR9w1fY7pT9eqVRy++NG4o2Qkyf2oSc4Onj9unj95GtoP5HhJpwHf\nAEZK6kJYYn0O8G/gqxbWsSpuvhaWc84VRLPdE90551zmCrUfiHPOOVeHNyBFJsn9qEnODp4/bp4/\neXaLBmTblipmPTKbV26qiDuKc841G7vFGMiq2SvpNqQH6+hEp6q1vj+6c87V0mRjIJL+lcv74rJX\nWTfWqjMlrGfVrBWNv8E551yjcu3C+npeUxSYWogl7csAWDqluNfESnI/apKzg+ePm+dPnowaEElt\nJH1K0lBJbcxsWaGD5dva0rCo4oZXfU0s55zLh0zWwjod+CPwHmEhxf7AN8zsqcLHy15980AqTvg5\n5f/5MRWH/YDyV26KIZlzzhWvvK6FleIW4Fgzezc6yX6EmehF2YDUp8OoQ3jztVG03j/t2pDOOeey\nlEkX1oaaxiMyH9hQoDwFc+i1pzN0zVRG/u2SuKM0KMn9qEnODp4/bp4/eTK5AnlV0njgIcCALwLT\nJJ0FYGbJWKXQOedcXmUyBnJ3Ay+bmV2Y30i7xtfCcs657OV9DERSS2Cmmf16l5I555xrdhocAzGz\nKuC8JsriSHY/apKzg+ePm+dPnkzGQF6Q9DvgQcKe6ACY2esFS1UglW+uYM4dz9J2rw4ccd2pccdx\nzrlEy2QMZEqaYjOz4woTadc0NAYy7WcTOOwnpzK987EcsmZyEydzzrniVZB5IGZ2bO6RikuPo8Ns\n9J7ri3s5E+ecS4JG54FIKpX0F0lPRc+HSLqo8NHyr/fIvmylDXtXL2Xj8o1xx0kryf2oSc4Onj9u\nnj95MplIOA6YCOwTPZ8LfKdQgQqpZZuWLG6zPwBLpsyLOY1zziVbJmMg08zsMEnTzeyQqGyGmQ1r\nkoRZamweyEv7nMWRyx7jv9/+OyN+c24TJnPOueJVqLWwPpK0F2EWOpKOBNblkK8oVJ18Gi+8VkrX\nQ/rFHcU55xItky6sK4AngP0kvQDcC3w70xNIWiDpDUnTJb0SlXWRNEnSHEkTJZWk1L9a0jxJsyWd\nlFI+XNJMSXMl3Zrxb1jLyLu/xsiZf2DwBUfm+hEFleR+1CRnB88fN8+fPI02INF8j2OAEcA3gAPN\nbGYW56gGys3sEDM7PCq7CnjGzMqAycDVEAbogbOBwcCpwO2Sai6p/gBcZGaDgEGSTs4ig3POuTzL\nZAykHfAt4GhCN9ZzwB/NbEtGJ5DeBw41sw9Tyt4BjjGzSkk9gQozO0DSVYQ5JmOjek8B1wELgclm\nNiQqPzd6/zfTnM/XwnLOuSwVak/0e4EDgd8Cv4se35fFOQx4WtI0SV+LykrNrBLAzJYDPaLyXsDi\nlPcujcp6AUtSypdEZc4552KSySD6QTV/+UemSJqVxTlGmtkySd2BSZLmEA3Ip8jrJcOYMWPo168f\nAJ07d2bYsGGUl5cDO/opi/X5rbfemqi8qc9T+4CLIY/nL658nr+4nldUVDBu3DiAT74vs2ZmDR7A\nX4EjU54fAdzb2Pvq+axrge8BswlXIQA9gdnR46uAK1PqT4jO90mdqPxc4A/1nMMaM/0PL9rTR19n\nr948udG6TW3KlClxR8hZkrObef64ef54Rd+dWX2nZzIGMhsoAxZFRX2AOcD26ISfauC97YEWZrZR\n0p7AJOCnwPHAajMbK+lKoIuZXRUNot8fNRq9gKeBgWZmkl4CLgOmEbbU/Y2ZTUhzTmvsd5pywi84\n9j/XUHHo9ymfdnODdZ1zbndQqHkgp+SYB6AUeEySRee638wmSXoVeEjShYQB8rMBzGyWpIeAWcA2\n4FsprcElhFnx7YDx6RqPTLU9uAz+A3ss8TWxnHMuV5ncxruwoaOR975vZsMs3MI71MxujMpXm9kJ\nZlZmZieZ2dqU99xgZvub2WAzm5RS/lr0GQPN7PJd+aX3OjIsqth99Zxd+ZiCSO1HTZokZwfPHzfP\nnzyZ3IXV7PQuD+th7fvxe2zfsj3mNM45l0yNjoEkTabzQJa06kvvqkUsfHoufU8Y2ATJnHOueBVq\nDKRZWvCFH7BQMHjfznFHcc65RKq3C0vSBknrU44NqT+bMmQhHP3ApYz8+6V0Lesed5SdJLkfNcnZ\nwfPHzfMnT0NXIP8hzL94FHjAzBY1UNc559xupsExkGiV3LMIE/faAQ8SGpPVTRMve74WlnPOZS+X\nMZCMBtEltSA0Ir8BrjezX+UWsfC8AXHOuezlfTFFSSMk/RZ4nbCc++eKufFoDpLcj5rk7OD54+b5\nk6ehQfSFwO2EFXEvBu4i7E44XNLwJspXUJM//3ue73UO7z45O+4ozjmXOPV2YUmqYMcquQakXtqY\nmR1X2Gi5yaYL65P90S/7OyNu8/3RnXO7r3zPA7nKzF7axUxFbXPfMlgGH7/pa2I551y2GhoDub3J\nUsSk1eCwJlbr94tnTawk96MmOTt4/rh5/uRpqAHJ6lImiUoOCw1IlxV+BeKcc9lqaAxkLTC1vjea\n2ehChdoV2YyBrHpnFd0Gd2c9HelYtQ61aPZtpnPOpZXvMZCVwC27Fqm47TVoL1684A46HrIfQ6rN\nGxDnnMtCQ11YG8zs2fqOJktYQGohjrrr6xz07eNo0ao4VrZPcj9qkrOD54+b50+ehr41FzRVCOec\nc8mT9X4gkk4EfmhmJxYm0q7xpUyccy57eV3KRNKxkuZK2ijpr5KGRnuZ3wj8YVfDOuecS7aGurB+\nRVjCZC/gYeBFYJyZfdrMHm2KcLujJPejJjk7eP64ef7kaagBMTOrMLOtZvZPYKmZ/a6pgjWV6u3V\nvLD3F5jdbhjbN2+LO45zziVGQ/NA5gPfTym6GfhBzZNivQrJZQzE90d3zu3u8j0P5FngsynPp6Y8\nN8JOhc1CZUkZvVcvYsVzc7wBcc65DNXbhWVmF9R3AFc3YcaC27hPWNJk04z4lzRJcj9qkrOD54+b\n50+ejGfPSeos6SJJ/wGmFzBTk7NBZQDo3fgbEOecS4rG9kTfAzgD+BJwCNAROBOYambVTZIwS7mM\ngUz7+UQO+/EpTO9cziFrphQomXPOFa+87oku6W/AKGAS8AAwGXjXzPrvatBCyqUBWbdoHR889Qb7\nHHcAJQN7FCiZc84Vr3zviT4EWAPMBmabWRU7dihsVkr6lDD4G58pisYjyf2oSc4Onj9unj95GhpE\nHwacTei2ekbS80BHSaXZnkRSC0mvS3oiet5F0iRJcyRNlFSSUvdqSfMkzZZ0Ukr5cEkzo9nxt2ab\nwTnnXH5lvBaWpE8D5xEalSVmNiLjk0jfBT4NdDKz0ZLGAh+a2U2SrgS6mNlVkoYA9wOHAb2BZ4CB\nZmaSXgYuNbNpksYDt5nZxDTn8rWwnHMuS/nuwtqJmb1mZt8H+gJXZRGqN3AacGdK8RnAPdHjewgD\n8wCjgQfMbLuZLQDmAYdL6gl0NLNpUb17U97jnHMuBllvgmFBvTsVpvFrwgz21MuCUjOrjD5vOVAz\n+NALWJxSb2lU1gtYklK+JCrLu6rt8V69JLkfNcnZwfPHzfMnT0Mz0XeZpNOBSjObIam8gap5/dYe\nM2YM/fr1A6Bz584MGzaM8vJw+pr/yLWf99rQkxZnnUlFq5bs99TvG61fqOczZsxo0vP5c3/uz3fP\n5xUVFYwbNw7gk+/LbGW9H0hWHy5dD3wF2A7sQRiQfww4FCg3s8qoe2qKmQ2WdBXhImds9P4JwLXA\nwpo6Ufm5wDFm9s0058xpDGTl7FV0H+L7ozvndk8FHQNJOckZko7IpK6Z/cjM+pjZAOBcYLKZ/T/g\nSWBMVO2rwOPR4yeAcyW1kdQf2B94JermWifpcEkCzk95T150O6Abq9WVTmxg1VvL8/nRzjnXLOWy\nEfgRwDWSntqF894InChpDnB89BwzmwU8BMwCxgPfSrmcuAT4CzAXmGdmE3bh/HVIsLR9WBNr2bPx\nLWlSc4mZREnODp4/bp4/ebIeAzGzH+VyIjN7lrDCL2a2Gjihnno3ADekKX8NGJrLuTO1trQM5r/E\numlzgGMKeSrnnEu8RsdAJPUARgL7AJuBt4BXm9NaWDWmnHg9xz7zfzx39NWMeu76PCdzzrnile+1\nsI4lzPfoSlh9dwXQDhgE7EfY5vYWM1u/K6HzbVcakHXvr6aFjI799spzKuecK275HkQ/Dfi6mR1m\nZheb2TVm9n0zGw0cTGhUTtyFvEWnpH/X2BuPJPejJjk7eP64ef7kqXcMxMx+0MBr24F/FiSRc865\nRMhkDKQUuB7oZWanROtVHWVmf2mKgNnytbCccy57hZoHMg6YCOwdPZ8LfCe7aM4555qbTBqQbmb2\nEFANn3RfVRU0Vcw+WvMxH763NpZzJ7kfNcnZwfPHzfMnTyYNyEeS9iJar0rSkcC6gqaKUcUF42jX\ndQ9mffbKuKM451xRy2QMZDjwW+AgwhyQ7sAXzGxm4eNlb1fHQKb9YhKHXXMyMzqXM8z3R3fO7SZy\nGQNpdCa6mb0u6RigDBAwx8y25Zix6JWOCsuZ9Fw/J+YkzjlX3BrtwpJ0PvAlwo6Cw4HzorJmqdeR\n+7KFtvSsXsbGZRua/PxJ7kdNcnbw/HHz/MmTyRjIYSnHKOA6ws6BzVLLNi1Z1HYgAEsmx7eoonPO\nFbus9wOR1Jmw7ewphYm0a/IxD+SlXp9nv+XPs/SG+xj2w5PylMw554pXXtfCauAkrYG3zKwsqzc2\nkXw0INVbt9Gibes8JXLOueJXkImEkp6U9ER0/AuYQ9hVsNmKs/FIcj9qkrOD54+b50+eTPYD+WXK\n4+3AQjNbUqA8zjnnEqKge6LHwdfCcs657OV1HoikDUSzz2u/BJiZdcoyn3POuWak3jEQM+toZp3S\nHB13h8ZjfeVm3n7gTVa8vbJJz5vkftQkZwfPHzfPnzyZzANB0nBJl0n6tqRDCh2qGLwx4psceN6n\nmDO2Wd8v4JxzOctkLayfAF8EHo2KzgT+YWY/L3C2nORrDGTKSTdw7NM/4tlDv8cx037Z+Buccy7B\nCrIWFvBl4GAz2xKd5EZgBlCUDUi+tB06CJ6GPRb7mljOOZdOJl1YHwDtUp63BZYWJk7x2OuosKhi\n9zVNu5xJkvtRk5wdPH/cPH/yNHQX1m8Jd2GtA96W9HT0/ETglaaJF5/e5ftTjej98Xy2b95Gqz18\nZrpzzqWqdwxE0lcbeqOZ3VOQRLson/NA3up0FFWt2rHviw/Rtax7Xj7TOeeKUZOshVXsfCKhc85l\nL69rYUVrYH02Wjyx9msDJP1/ki7MJairX5L7UZOcHTx/3Dx/8jR0F9bXgSuAWyWtBlYSBtP7A+8C\nvzOzxwsf0TnnXDHKqAtLUj9gb2AzMNfMNmX04VJbYCrQhtBYPWxmP5XUBXgQ6AssAM42s3XRe64G\nLiQs3Hi5mU2KyocD4wiN2Hgz+0495/QuLOecy1JRjoFIam9mmyS1BF4ALgM+D3xoZjdJuhLoYmZX\nSRoC3E/Y/bA38Aww0MxM0svApWY2TdJ44DYzm5jmfN6AOOdclgqyH8iuSrlaaUu4CjHgDKDmLq57\nCLPbIWyV+4CZbTezBcA84HBJPYGOZjYtqndvynsK6q2/zWTqJQ+yaeVHTXG6RPejJjk7eP64ef7k\nKXgDIqmFpOnAcuDpqBEoNbNKADNbDvSIqvcCFqe8fWlU1gtI3YNkSVRWcG0u/DKfuf1cFk16pylO\n55xziZHJUia7xMyqgUMkdQIek3QgdZeJz2uf05gxY+jXrx8AnTt3ZtiwYZSXlwM7/krI9PmkDl35\nYCu0fWkOfPnTWb8/2+c1ZYX6/EI+Ly8vL6o8nr+48nn+4npeUVHBuHHjAD75vsxWTmMgkq4zs+ty\neN+PgU3A14ByM6uMuqemmNlgSVcR9hoZG9WfAFwLLKypE5WfCxxjZt9Mc468joFMPupHHPfSDTxb\nfi3HTLkub5/rnHPFpCnHQF7LpJKkbpJKosd7EJZBmQ08AYyJqn0VqLkd+AngXEltJPUH9gdeibq5\n1kk6XJKA81PeU1CthpQB0Pr9plkTq+YvhCRKcnbw/HHz/MmTUxeWmT2ZYdW9gXsktSA0Vg+a2XhJ\nLwEPRRMRFwJnR587S9JDwCxgG/CtlMuJS9j5Nt4JuWTPVslhg+Au6LLSV+V1zrlUmewHMgC4DTgK\nqAZeBL5rZvMLHy97+e7C+nDeahaNOIePBg3n6BfG5u1znXOumBRkHkh0tfB74O9R0bnAt83siJxS\nFpjPA3HOuewVagykvZndF83N2G5mf2Xn/UFcHiW5HzXJ2cHzx83zJ08mYyBPRXdHPUC43fYcYLyk\nrgBmtrqA+ZxzzhWpTLqw3m/gZTOzAfmNtGu8C8s557KX9zGQ6O6po8zshV0N11S8AXHOuezlfQwk\nmkX+u11K1Qxs+nAzFRffz+TP/rrg50pyP2qSs4Pnj5vnT55MxkD+I+nzwKO765/2ZvCZP/8/qmnB\n9s2X5md/9EWL4J13YO5cmDcv/Fy6FMaMgZRlTZxzrlhlMgayAdgTqCLsByLC2EenwsfLXqG6sBa1\n6k+fqgUsnDSHvicOavwNZrBsGXTqBB061H191Ch4/vm65a+9BsOH1y2/5hrYvBkOPBCGDAlHp6L8\nT+CcS6BcurAavQIxs465R2o+VpQMos/qBax8vp4GZNIkmDp1xxXFvHnw0UfwyCNw1ll16x95JLRs\nCQMHwqBB4WevXnDQQekDjBsXrlBS9e4NU6bA/vvv8u/nnHPZarQBidae+jLQ38x+JmlfYG8ze6Xg\n6YrIxl6DYPUk7JFH4aefrfP62rseofODd+xUtq1kLzYt2UBJms9b9O2bWbUKpJ2PBRMqGD26fOfK\nZqz7+W+pemsWbea9TZt3Z9H6vXdg6VI2lezDnmk+f/sXz6WqWyk2eAg2OFy1tOjWlVatQru1c9Bt\noeHbvBk2bdrxs0ULOOOMuh++ejX84Ad16lds3Ej59OlpwmyHadOge3fo1g1KSsIvW2QqUlZBTiLP\nH6+k589FJmMgtxOWMDkO+BmwkTAz/bAC5io6VjYY3gSqtqd9/d41o1lLKXMZxDwGMo+BrFnXlT+1\ng4vT1P/Zz+DOO+uWX3EFjB5dq1Di+y98jjvv/NwnRS2oog+LuPqx9lxc+wRr19Lq4Qfr/MedxWCe\n/9OsuvVXrkx75bNCpUwYdwbnn1/rhW3b4K676tRf3bI7jz6a5oJr5UoYMeKTp9tbtGZD225Ulgyi\n8oEKjjmmVv2tW/nvzS8wc1l3tnbsxraSbrRo25rWreHYY9NfpM2dG07Tti20aROOtm1Dm5WuB9E5\nt+syaUCOMLPh0aZQmNkaSW0KnKvoHHTjVxj/xhx6njky7etbTzidZzafjhm0qoYDLAyDlJam/7w+\nfWDYsFAn9TjqqPK09bt1g379oLo61KuubsnH1p89011+tGvHH09/kqVPz6KsehaDq96mzGazN8to\nle6/+J57sqzkAD5Y157N7MEmws9V1g37OE39zp255+g/M/H5netvqOrIpWvS/eNsZX73w2HlSrqx\nik7VG+iyeRnLN5cwbx51G5DFixnx4+MZkVK0lhJe5VBevOOZug3IkiWsP+UK5r/flq3sOJbQmwP+\ndEXdBnP9ev5yziSeea4t21u3w1q3pbpNO7a225uVN8AXv1j3V3j4YZg+Hdq33/kYMQL2269u/Q0b\nws/27dNc8RVI0v/69fzJk8kg+svACGBa1JB0ByaZ2SFNETBbPg+kHtXV8OGH4U/yNLZuDRcWVVWh\nanV1eNyhQ/gSrG3ZMli7dkf9qqpw9O2b/hSvvw7vvx96s6o3baHVug9h0yY+9fmBlJXVqjx3LqvO\nupjqypW037SS9ps/pIVVs3SvoSx6ciZHHVWr/htvhNa4lndaD+XN+2fWbRBmzoSDD65Tfw6DePHu\nOYwZU+uFykomHHcTk2eVsoIeVFJKJaUsY2+uv2tvLrig7u970UU7LtLatAn/hh06wC9/CeecU7f+\nxIkwf37o3Us9+vb1eyVc0yjIIDrwG+AxoFTSL4AvANfkkM9loGD9qC1a1Nt4QOjuads284/be+9w\npKqoqKB79/K09YcPT725rB0N7kg8aBDd3qrY8by6GtasodfmzfTqnaZ+nz7w4IOhFdyyJfzcupUD\nunThgDQRfJj/AAAUnUlEQVRXE3ToQNUZZ1G1eSu2eQu2ZSts2crcPUvqdh8CzJ/PKbN+xSm1ihd0\nOYQlA1+vW3/pUka/8Rf2bFPKko97sPzjUio/LqVybSlm6fvT7rkH/v739OV1uhCBG2+E//5358Zm\nzZoKLr+8vG6DnBBJH0NIev5cZHIX1v2SXgOOj4rONLPZhY3lXIoWLWCvvep/vUsXOPvszD9vwABa\n/vMRavcsdayooGvXNPV79YKxY2HFCqisDMeKFfQ7aAj9jk5T/513OOO1a6l9+8GWT49g2+lpFnVY\nvpyLS55l+Oh9WFy1Dws/3psVG9uzbh307Jn+V3j5ZXgyza48xx1H2gbkhz8M7+nRY+fj1FND16hz\nuchoS1tJw4GjCYspvmBmaf7sKg7eheViN2cO3HvvJw3NJz+PPhruu69u/SefrHvnROfO4W6Ev/yl\nbv3165n13Ie8+9HerNncjnXrYN260EN58cVhilBtJ5wA//lP3fJ//xtOO61u+RVXwKxZOzc2e+8N\nJ55Yf6Pmkq0gXViSfgJ8EXiEMInwbkn/MLOf5xbTuWaurAx+8YvM63ftCp//PHzwQThqBpi2p7/j\nj/HjGXLeeQypee8++4Tj1FNhyHfSvuWPf4TFi0M7lnoMHJj+FC++CC+9VLd88uT0Dcgdd8DGjaE3\ncd99w9GzZ7h4dM1XJoPoc4CDzWxL9HwPYIaZFWVPa9KvQJLcj5rk7FBE+c3CXJvt29Pfxve3v8GV\nV8Ly5Ts1MhWjR1P++ON16z/9dHjP/vuHY7/9ws/OneuN8OabOzc4lZVhHuvNN4fGobahQ+Gtt3Yu\na906NELpFlaYPz/0PHbuvGNKUNH8++co6fkLNYj+AWHUc0v0vC2wtP7qzrldIjU85vOlL4WjuhpW\nrdpx5fLBB+nrv/BCWMmgtiuvDKPxtW3bxtCDWjF0aObfJd/4Rui5W7w4HIsWhWj1dXeddBK89x7s\nuWdokPr0CXerDR5c/63vrvhkcgXyT8KkwacJYyAnAq8ASwDM7LICZ8xK0q9AnMu7t94Kt2y9++6O\n4733wo0Bl15at/7//R/8/vfhKmXgwB1XLqNGwYDMt//ZvBnatau76IAZfPrTYbWfjRt3fm316nBl\nUtsll4SrlQEDwgXUgAHh3oammmOzOyjUnuhfbeh1M7snmxMWmjcgzmXALHR/tU6zsvT556cf7P/j\nH8OlRm2zZoXBjgEDwmVEFhHWrt1xxbJkCfzv/9att3Ur7LFHqJ+qbVtYsya8VtumTennL7n6FaQB\nSZqkNyBJ7kdNcnbw/J8wC/1PqVcs8+bB974XLh1q+9zn4J//DJcDAwbAAQeEGwm+9rX09xRnmX/L\nFnjooXDRNH/+jp9SuN+gto0boWPH0H1Wc8Wy337hYupLX8ri3yFLSf/fT6HGQJxzuxMpTDrt3p26\n0/7TKC0Nk0kWLtyxEnXNrcnpGpCHHw6NTVlZ6Bpr5KqlXbv0kym3bKlbBmGwv3XrcI/B8uWh9w7C\nOEu6BmT9+jBENGBAOPr3T39V4+ryKxDnXH5s3hyuVubMCZulXXJJ+gGNsrKw+iXs6PoqK4Pbbw/f\n8nlQVRUaktSrlnbt4Cc/qVv31VfhsFpLw+6zT9jX7f778xInEZqkC0tSO+CzZvaPrN7YRLwBca7I\n/fCHYWB/zhxYsCDcTQZhJmS6pQBqGqJBg3YcaZcMyM2cOXDrraGhmT8/RNq+HU4+GSZMqFv/5ZfD\nWmc1g/k1R80FVVIVrAGR1BI4GTgPOAl4zsy+kFPKAkt6A5LkftQkZwfPH4utWz8ZZ6koKambf+vW\nMBpe08jU2GuvMPpegL6m7dvD1cvWraGtqu2++9J3qR19dAXPPVdep3zuXKioCPu/7btv+Jk6/6VY\n5H0MRNIxwJeA0wi37o4kbCy1KeeUzjlXo23bsE3zgQeGb9nazMKKknPn7ny0bl3/7VeDB+/Y6bPm\nKCtLv+5+Gq1ahVWQ63PWWWFPmprusZqjvt63qVPr3ry2556h7JZb6tZfvz50wRVjI1NbvVcgkpYA\ni4A/AP80sw2S3jez/k0ZMFtJvwJxzjWiZqZ+usmW9SzVT9++oW+qto0bw3bUNbMZu3fP+/orU6bA\nX/8ablOumWi5cWPoyRs7tm793/wGLr88XHilXrWceWY4ajPLT0OT7yuQh4EzgXOAKkmPEyYSZhOo\nN3AvUErY1fDPZvYbSV2AB4G+wALgbDNbF73nauBCYDtwuZlNisqHA+MIs+LHm1n6RX+cc81bQzP1\nDzwwdInVvmKpb8zknXfCOmQ12rYN39ajRsHdd9etn8O39bHHhiNVzVVGOlu3hr1jNm7cER/CjW7p\nGpDrrw9LzNReafmss8KM/4Iys3oPwuKJxwJ3EGaebwDOBjo09L6U9/cEhkWPOwBzgAOAscAPo/Ir\ngRujx0OA6YSGrR/wLjuukl4GDosejwdOrueclmRTpkyJO0LOkpzdzPPHLZb8M2aYjR5tNmyYWdeu\nOzYHPemk9PWnTjUrKTEbOtTs9NPNvvlNs+uvNxs/Pu/5160ze/ttswkTzO680+zVV9PXu+yy2vua\nhuPmm9PXv+Yas9LS8Cscf7zZeeeZ/eMfZtF3Z6Pf66lHg2MgZmbAFGCKpNbsGEi/HeiWQeO0HFge\nPd4oaTbQGzgDqNnI9B6gArgKGA08YGbbgQWS5gGHS1oIdDSzadF77iVcHU1sLINzztXr4IMhdQHK\njz4KfUy1B+1rLF4c1s5/881w1DjjDPhOmk6RqVPhpz8Nc2V69Njx88AD4cgjG4zWqVNYmj/d8vyp\nfv3rcHty6irLK1eG7ZbTWbZsx7Y2NQ44oOFz1Cfj23ijBuQgwkKKG8xsc1YnkvoRGoqDgMVm1iXl\ntdVm1lXSb4EXzexvUfmdhKuNhcANZnZSVH404Qqmzv5xPgbinCsYs3C78aJF4ahZh+XAA6m7FzJh\nX+OLLqpb/uUvh4GR2iZPht/9bkc/VE2DM2RIOEcefPxxWGggtbE56CAYNiyPYyCS/gj81szellQC\nvAhUAV2B7wNpNuCs97M6EMZULo+uRGp/w/s3vnOu+EnQrVs40q1TX9tpp4UN71M3FqushJEj09d/\n+2147LG65RdfDH/6U93yRx+F224LYzw1R5cuYQWB2gMvkTZtdmwhs6sa6sIaZWY1S5tdAMw1szMl\n9QSeIsMGRFIrQuNxn5nVXCtWSio1s8ro81ZE5UuB1N0Gekdl9ZWnNWbMGPpF+3R27tyZYcOGfXJ/\neUV0q2CxPr/11lsTlTf1ec3jYsnj+Ysr326Z/513oE0byr/ylcw+r1s3uO46yrt3h8pKKqZPhzVr\nKI+mytepP3EiTJ1KeXTemgTlV1wBxx5bt/7ll8O4cZSXllLRqhXj1q2Dtm3pd+ih5KS+wRFgesrj\nfwNj0r3W2EEYr/hVrbKxwJVW/yB6G6A/Ow+ivwQcThjYHw+cUs/56huTSoQkD4QmObuZ54+b58/B\n0qVmkyebPfKI2Z//bDZ2rNmVV5r961/p6//kJ+lH3H/605wG0RuaBzIFuIXwl/4U4AAzWx5dUbxl\nZo0Ou0gaCUwF3iR0UxnwI8KkxIcIVxULCbfxro3eczVwEbCNnW/j/TQ738Z7eT3ntPp+J+ec261t\n2xbWwF+9Ohw1j4cORYcckr+lTCQNAn5DuBX3VjMbF5WfDJxkZt/btd+kMLwBcc657OUykbDeKZdm\nNtfMTjGzYTWNR1Q+EVice0zXkNR+1KRJcnbw/HHz/MmT65z9K/KawjnnXOLktB+IpMVmtm/jNZue\nd2E551z28tqF1Qj/hnbOud1cvQ2IpA2S1kc/ax6vl7QByMMUFJdOkvtRk5wdPH/cPH/y1DuR0Mw6\nNmUQ55xzydLQbbztgP8F9gdmAndZWOSwqPkYiHPOZS/fYyD3AIcSJgGeRphU6JxzzgENNyBDzOwr\nZvYn4AvAqCbKtFtLcj9qkrOD54+b50+ehhqQbTUPktB15Zxzrmk1NAZSBXxU8xTYA9gUPTYz69Qk\nCbPkYyDOOZe9vO6JbmYtdz2Sc8655irXiYSuQJLcj5rk7OD54+b5k8cbEOeccznJaS2sYuZjIM45\nl72mXAvLOefcbs4bkCKT5H7UJGcHzx83z5883oA455zLiY+BOOec8zEQ55xzTccbkCKT5H7UJGcH\nzx83z5883oA455zLiY+BOOec8zEQ55xzTccbkCKT5H7UJGcHzx83z5883oA455zLiY+BOOec8zEQ\n55xzTaegDYikv0iqlDQzpayLpEmS5kiaKKkk5bWrJc2TNFvSSSnlwyXNlDRX0q2FzBy3JPejJjk7\neP64ef7kKfQVyN3AybXKrgKeMbMyYDJwNYCkIcDZwGDgVOB2STWXU38ALjKzQcAgSbU/s9mYMWNG\n3BFyluTs4Pnj5vmTp6ANiJk9D6ypVXwGcE/0+B7gzOjxaOABM9tuZguAecDhknoCHc1sWlTv3pT3\nNDtr166NO0LOkpwdPH/cPH/yxDEG0sPMKgHMbDnQIyrvBSxOqbc0KusFLEkpXxKVOeeci1ExDKL7\nLVMpFixYEHeEnCU5O3j+uHn+5Cn4bbyS+gJPmtmnouezgXIzq4y6p6aY2WBJVwFmZmOjehOAa4GF\nNXWi8nOBY8zsm/Wczxsk55zLQba38bYqVJAUio4aTwBjgLHAV4HHU8rvl/RrQhfV/sArZmaS1kk6\nHJgGnA/8pr6TZfsP4JxzLjcFbUAk/Q0oB/aStIhwRXEj8A9JFxKuLs4GMLNZkh4CZgHbgG+lzAi8\nBBgHtAPGm9mEQuZ2zjnXuGY3E90551zTKIZB9LyQdIqkd6LJhlfGnScbknpLmizpbUlvSros7ky5\nkNRC0uuSnog7S7YklUj6RzSJ9W1JR8SdKVOSvivprWiy7f2S2sSdqTHZTjIuJvVkvyn6384MSY9I\n6hRnxoaky5/y2vckVUvqmslnNYsGRFIL4HeESYsHAudJOiDeVFnZDlxhZgcCRwGXJCx/jcsJXZBJ\ndBuhe3QwcDAwO+Y8GZG0D/BtYHh0o0or4Nx4U2Uk40nGRShd9knAgWY2jDCHrVizQ/r8SOoNnEgY\nWshIs2hAgMOBeWa20My2AQ8QJiwmgpktN7MZ0eONhC+vRM11if7HdxpwZ9xZshX9tTjKzO4GiCaz\nro85VjZaAntKagW0Bz6IOU+jspxkXFTSZTezZ8ysOnr6EtC7yYNlqJ5/e4BfAz/I5rOaSwNSexJi\nYicbSuoHDANejjdJ1mr+x5fEQbX+wCpJd0ddcHdI2iPuUJkwsw+AW4BFhMm3a83smXhT5ay+ScZJ\ncyHwVNwhsiFpNLDYzN7M5n3NpQFpFiR1AB4GLo+uRBJB0ulAZXQVVfu27SRoBQwHfm9mw4FNhO6U\noiepM+Ev977APkAHSV+KN1XeJO6PEUn/B2wzs7/FnSVT0R9LPyLcJftJcSbvbS4NyFKgT8rz3lFZ\nYkTdDw8D95nZ443VLzIjgdGS5gN/B46VdG/MmbKxhPDX16vR84cJDUoSnADMN7PVZlYFPAqMiDlT\nriollQJEk4xXxJwnK5LGELpxk9aA7wf0A96Q9D7h+/M1SY1eATaXBmQasL+kvtEdKOcSJiYmyV3A\nLDO7Le4g2TKzH5lZHzMbQPi3n2xm58edK1NRt8liSYOiouNJzs0Ai4AjJbWLVq8+noTcAED9k4xh\n50nGxWin7JJOIXThjjazrbGlytwn+c3sLTPraWYDzKw/4Q+qQ8ys0Qa8WTQg0V9elxLuhHibsKpv\nUv5PhKSRwJeB4yRNj/rhT4k7127mMsJKCDMId2FdH3OejJjZK4QrpunAG4QvhTtiDZWBaJLxfwnb\nMyySdAFhkvGJkuYQGsIb48xYn3qy/xboADwd/f/39lhDNqCe/KmMDLuwfCKhc865nDSLKxDnnHNN\nzxsQ55xzOfEGxDnnXE68AXHOOZcTb0Ccc87lxBsQ55xzOfEGxDU7kjZEP/tKOi/Pn311refP5/Pz\nm1Ih/n3c7sUbENcc1Uxu6k+Wy0pIatlIlR/tdCKzo7P5/CKT9b+Pc6m8AXHN2Q3A0dHM4MujDa9u\nkvRytPHP1wEkHSNpqqTHCSsZIOkxSdOiDb6+FpXdAOwRfd59UdmGmpNJujmq/4aks1M+e0rKZlX3\npdS/MdoIaoakm2qHl7SnpLuijaJmSPpcVH5eVDZT0o0p9VOzfF7S3dHjuyXdJukFSe9KOivdv09e\n/sXdbqWge6I7F7OrgO+Z2WiAqMFYa2ZHRGumvSBpUlT3EMKGQIui5xeY2VpJ7YBpkh4xs6slXRKt\n2FvDos/+PPApMxsaLUI3TdKzUZ1hwBBgeXTOEcA7wJlmdkD0/nQ72P04yvupqE6JpL0JS3wcAqwl\nLJ0x2syeoO7qtanPe5rZSEmDCWtOPVr738e5bPkViNudnAScL2k6Yb+VrsDA6LVXUhoPgO9E62LV\nbA40kIaNJKxETLQIXQVwWMpnL7OwbtAMwsqn64DNku6Mriw2p/nME4Df1zwxs3XRZ06JVt+tBu4H\nPhNVaWj9on9GnzGb5O6z4YqMNyBudyLg22Z2SHTsl7L50kefVJKOAY4Djoi2KJ0BtEv5jEzPVSN1\nddYqoFW0AOjhhIUQ/weYkOYz6luorr4MqfXb1XotNUPS9mtxRcobENcc1XxBbgA6ppRPBL4V7b2C\npIGS2qd5fwmwxsy2KuxNf2TKax/XvL/WuZ4DzonGWboDo4BX6g0YztvZzCYAVwCfSlPtaeCSlPd0\njj7zM5K6RgP+5xGudgCWSyqT1AL4XH3npv5/H+ey4g2Ia45q/hKfCVRHS+RfbmZ/Juzz8bqkN4E/\nEvYTr20C0FrS24Rl3V9Mee0OYGbKYLgBmNlj0fneAJ4BflDPfgo12ToB/5L0BjAV+G6aur8AukYD\n89OB8mir16sIjcZ0YJqZ/SuqfzXwb+B5dt4Xvb6xkZ3+fdKc37kG+XLuzjnncuJXIM4553LiDYhz\nzrmceAPinHMuJ96AOOecy4k3IM4553LiDYhzzrmceAPinHMuJ96AOOecy8n/D119t6vO518xAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xbc529e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def print_measures(model_plsa, model_artm):\n",
    "    print 'Sparsity Phi: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['SparsityPhiScore'].last_value,\n",
    "        model_artm.score_tracker['SparsityPhiScore'].last_value)\n",
    "\n",
    "    print 'Sparsity Theta: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['SparsityThetaScore'].last_value,\n",
    "        model_artm.score_tracker['SparsityThetaScore'].last_value)\n",
    "\n",
    "    print 'Kernel contrast: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['TopicKernelScore'].last_average_contrast,\n",
    "        model_artm.score_tracker['TopicKernelScore'].last_average_contrast)\n",
    "\n",
    "    print 'Kernel purity: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['TopicKernelScore'].last_average_purity,\n",
    "        model_artm.score_tracker['TopicKernelScore'].last_average_purity)\n",
    "\n",
    "    print 'Perplexity: {0:.3f} (PLSA) vs. {1:.3f} (ARTM)'.format(\n",
    "        model_plsa.score_tracker['PerplexityScore'].last_value,\n",
    "        model_artm.score_tracker['PerplexityScore'].last_value)\n",
    "\n",
    "    plt.plot(xrange(model_plsa.num_phi_updates), model_plsa.score_tracker['PerplexityScore'].value, 'b--',\n",
    "             xrange(model_artm.num_phi_updates), model_artm.score_tracker['PerplexityScore'].value, 'r--', linewidth=2)\n",
    "    plt.xlabel('Iterations count')\n",
    "    plt.ylabel('PLSA perp. (blue), ARTM perp. (red)')\n",
    "    plt.grid(True)\n",
    "    plt.show()\n",
    "    \n",
    "print_measures(model_plsa, model_artm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Видно, что улучшения разреженностей и ядровых характеристик есть, а ухудшение перплексии невелико. Попробуем увеличить по модулю значения коэффициентов регуляризации при регуляризаторах:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_artm.regularizers['SparsePhi'].tau = -0.2\n",
    "model_artm.regularizers['SparseTheta'].tau = -0.2\n",
    "model_artm.regularizers['DecorrelatorPhi'].tau = 2.5e+5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Кроме того, подключим к каждой из моделей функционал TopTokensScore, который позволит взглянуть на самые вероятные слова в каждой теме:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model_plsa.scores.add(artm.TopTokensScore(name='TopTokensScore', num_tokens=6))\n",
    "model_artm.scores.add(artm.TopTokensScore(name='TopTokensScore', num_tokens=6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Продолжим обучение моделей, инициировав 25 проходов по коллекции, после чего снова посмотрим на значения функционалов качества:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_plsa.fit_offline(batch_vectorizer=batch_vectorizer, num_collection_passes=25)\n",
    "model_artm.fit_offline(batch_vectorizer=batch_vectorizer, num_collection_passes=25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sparsity Phi: 0.093 (PLSA) vs. 0.841 (ARTM)\n",
      "Sparsity Theta: 0.000 (PLSA) vs. 0.023 (ARTM)\n",
      "Kernel contrast: 0.640 (PLSA) vs. 0.740 (ARTM)\n",
      "Kernel purity: 0.674 (PLSA) vs. 0.822 (ARTM)\n",
      "Perplexity: 1619.031 (PLSA) vs. 1644.220 (ARTM)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVNX9//HXhwWkSJciRUHFXhAEJRhZNdbEkkRREwvR\nVPVri1EwBZMYu8YWyzfmJ+rXhibWICoJq9EoooCoIGABAQULRTrs7vv3x7kLw+7s7szsnZ256+f5\neMyDuWfvnfveq8zhnnPPOSYJ55xzLlvNCh3AOedcMnkF4pxzLidegTjnnMuJVyDOOedy4hWIc865\nnHgF4pxzLid5rUDMbGczm2ZmU6M/V5jZeWbWycyeN7PZZvacmXVIOWa0mc01s1lmdnhK+UAzm2Fm\nc8zspnzmds45Vz9rrHEgZtYMWAjsD5wLfCnpWjO7FOgkaZSZ7Q48AAwGegMTgf6SZGaTgXMlTTGz\n8cDNkp5rlPDOOedqaMwmrG8BH0haABwH3BuV3wscH70/FnhYUrmkecBcYIiZ9QDaSZoS7XdfyjHO\nOecKoDErkJOAB6P33SUtAZC0GOgWlfcCFqQcsygq60W4e6myMCpzzjlXII1SgZhZC8LdxaNRUfV2\nM59PxTnnEqZ5I53nKOBNSV9E20vMrLukJVHz1GdR+SKgT8pxvaOy2sprMDOvjJxzLgeSLJv9G6sJ\n6xTgoZTtp4CR0fszgCdTyk82s5Zm1g/YCXg9auZaYWZDzMyA01OOqUFS0b/GjBlT8AxNIaPn9JzF\n/kpKzlzk/Q7EzNoQOtB/mlJ8DTDOzM4E5gMjACTNNLNxwExgI3C2Nv9m5wBjgVbAeEkT8p09n+bN\nm1foCPVKQkbwnHHznPFKSs5c5L0CkbQG6FqtbCmhUkm3/1XAVWnK3wT2ykdG55xz2fOR6AUycuTI\nQkeoVxIygueMm+eMV1Jy5qLRBhI2FjNTU/udnHMu38wMFWknuqumrKys0BHqlYSM4Dnj5jnjlZSc\nufAKxDnnXE68Ccs555w3YTnnnGs8TbICUWXx34EkoV00CRnBc8bNc8YrKTlz0SQrEGuW1V2Yc865\nHHgfiHPOOe8Dcc4513i8AimQJLSLJiEjeM64ec54JSVnLrwCcc45l5Mm2Qcy5Zp/sd8lhxQ6inPO\nJYb3gUTWvPNBoSM451yT1yQrkMoVKwsdoV5JaBdNQkbwnHHznPFKSs5cNMkKhBVfFTqBc841eU2y\nD2TSoIsofeOGQkdxzrnE8D6QSMkqvwNxzrl8a5IVSO/rLyh0hHoloV00CRnBc8bNc8YrKTlz0SQr\nkH7f2aPQEZxzrslrkn0gTe13cs65fMulD6R5PR/YCvgO8E2gJ7AWeAf4p6R3cw3qnHMu+WptwjKz\n3wOvAEOBycBdwDigHLjazF4ws70bJWUTlIR20SRkBM8ZN88Zr6TkzEVddyCvSxpTy89uNLNuwHZ5\nyOSccy4BmmQfyOROhzNk6XOFjuKcc4mRSx9IrRWImT0N1Fq7SDo2u3iNw8wkYMPK9bTcumWh4zjn\nXCLEPZDweuAG4CNC5/lfo9cqoOhnK1z1aXHPh5WEdtEkZATPGTfPGa+k5MxFrX0gkl4EMLMbJO2X\n8qOnzeyNvCdroNWffkXn/l0KHcM555qsevtAzGwW8G1JH0bb/YDxknbL6ARmHYC7gT2BSuBMYA7w\nCLA9MA8YIWlFtP/oaJ9y4HxJz0flA4GxQKvo/GmHm1c1Yc159C12PsEfEnPOuUzkay6sC4EyMysz\nsxeBSUA2c4XczOYKZx/gPWAUMFHSLsC/gdHRL7A7MALYDTgKuN3Mqn6hO4CzJO0M7GxmR9R10nWf\n+XxYzjmXT/VWIJImAP2B84HzgF0kZfSIk5m1B74p6Z7os8qjO43jgHuj3e4Fjo/eHws8HO03D5gL\nDDGzHkA7SVOi/e5LOaaGt294nr7fLu7pTJLQLpqEjOA54+Y545WUnLmotwIxszbAr4BzJb0FbGdm\n38nw8/sBX5jZPWY21cz+N/q87pKWAEhaDHSL9u8FLEg5flFU1gtYmFK+MCpLa6+LDqP99p0yjOic\ncy4XmfSBPAK8CZwuac+oAvivpAH1frjZIOA1YKikN8zsz8BKQmXUOWW/LyV1MbNbgVclPRiV3w2M\nB+YDV0k6PCo/ELgk3aPEPheWc85lL/a5sCI7SjrJzE4BkLQmpV+iPguBBZKqntr6O6H/Y4mZdZe0\nJGqe+iz6+SKgT8rxvaOy2srTGjlyJH379gWgY8eODBgwgNLSUmDz7aRv+7Zv+/bXebusrIyxY8cC\nbPq+zJqkOl/Af4HWwNRoe0fCNCf1Hhvt/yKwc/R+DHBN9Lo0KrsUuDp6vzswDWhJaP56n813Sa8B\nQwAj3JUcWcv5lASTJk0qdIR6JSGj5Dnj5jnjlZSc0XdnRt/rVa9M7kDGABOAPmb2ADAMGJlFHXUe\n8ICZtQA+BH4ElADjzOxMQvPUiOibf6aZjQNmAhuBs6NfDOActnyMd0IWGZxzzsWszj6QqKmqN7AG\nOIDwr//XJH3ROPGyZ2Z6veNhfDX4UA59/tJCx3HOuUSIvQ9EksxsvKS9gH82KF0jGrz8Bf4720eh\nO+dcPmUykHCqmQ3Oe5KYtVhb3AMJqzqzilkSMoLnjJvnjFdScuYikz6Q/YEfmtl8YDWhGUuSinqe\nkJbri3syReecS7pMxoFsn65c0vy8JGqgqrmwZrfah13WTi90HOecS4RY+0DMbGtJq+qqKKr2yeaE\njaVVud+BOOdcPtXVB/Kkmd1gZgeZWduqQjPbwczOMrPngCPzHzF7068cz/p7Hip0jDoloV00CRnB\nc8bNc8YrKTlzUdd6IIea2dHAz4BhZtaJMMX6bMITWWcozGNVdAaMPqrQEZxzrslrkmuiN7XfyTnn\n8i1f64E455xzNXgFUiBJaBdNQkbwnHHznPFKSs5ceAXinHMuJ02yD+Rf+/6SznMns9X/3spup9S7\nbIlzzn3tNVofiJk9k8txjaX9x28zYNXLrHq/KB8Sc865JiHXJqyfxJoiZhtbtQ9/Li3e+bCS0C6a\nhIzgOePmOeOVlJy5yGQuLMysJbArIGC2pE/zmqqBytu0C38u9dHozjmXL5nMhfVt4E7gA8JEiv2A\nn0l6Nv/xsmdmmjTgfEqn30zZcTdS+sSFhY7knHNFL19rot8AHCzp/egkOxJGohdlBQLA1uEORCv8\nDsQ55/Ilkz6QlVWVR+RDoKi/mXe6YiTv3jaJ3a/7UaGj1CoJ7aJJyAieM26eM15JyZmLTO5A3jCz\n8cA4Qh/IicAUM/segKR/5DFfTnoP35Hew3csdAznnGvSMukDuaeOH0vSmfFGahifC8s557IXex+I\nmZUAMyT9uUHJnHPONTl19oFIqgBOaaQsXytJaBdNQkbwnHHznPFKSs5cZNIH8oqZ3QY8QlgTHQBJ\nU/OWyjnnXNHLpA9kUppiSTokP5Eaxsz06dRPWFT6Aza06sDQJU8UOpJzzhW9vIwDkXRw7pEKo6RF\nMwZ9VcbnK7sWOopzzjVZ9Y4DMbPuZvY3M3s22t7dzM7Kf7Tcte0RBhK2k8+F1RBJyAieM26eM15J\nyZmLTAYSjgWeA3pG23OAC/IVKA6tO7emnBJasZ6NazYWOo5zzjVJmfSBTJE02MymSdo3KpsuqSgX\n2qgaB7K8WSc6ajnL3v+STjt2LnQs55wravlaD2S1mXUhjELHzA4AVuSQr1GtahamdF/9afE2Yznn\nXJJlUoFcBDwF7GhmrwD3Af+T6QnMbJ6ZvWVm08zs9aisk5k9b2azzew5M+uQsv9oM5trZrPM7PCU\n8oFmNsPM5pjZTfWdd9VfH2bOfa/Rda8emUZtVEloF01CRvCccfOc8UpKzlxk8hTWVDMbDuxCmM59\ntqRsOhYqgVJJy1LKRgETJV1rZpcCo4FRZrY7MALYDegNTDSz/tHcJHcAZ0maYmbjzewISc/VdtJd\nfzQ0i4jOOeeylUkfSCvgbOBAQjPWf4A7Ja3L6ARmHwH7Sfoypew9YLikJWbWAyiTtKuZjSKMMbkm\n2u9Z4HJgPvBvSbtH5SdHx/8izfl8LiznnMtSvvpA7gP2AG4Fbove35/FOQS8YGZTzOzHUVl3SUsA\nJC0GukXlvYAFKccuisp6AQtTyhdGZc455wokk6lM9qz6l39kkpnNzOIcwyR9amZdgefNbDZRh3yK\nWG8ZRo4cSd++fQHo2LEjAwYMoLS0FNjcHlno7aqyYsmTbrt61kLnqW17+vTpXHDBBUWTp7Ztv55+\nPYshT9V2WVkZY8eOBdj0fZk1SXW+gP8DDkjZ3h+4r77javmsMcAvgVmEuxCAHsCs6P0o4NKU/SdE\n59u0T1R+MnBHLedQEkyaNKnQEeqVhIyS54yb54xXUnJG351Zfadn0gcyi9CB/nFUtB0wGyiPTrh3\nHce2AZpJWmVmbYHngd8DhwJLJV0TdaJ3klTVif5AVGn0Al4A+kuSmb0GnAdMISype4ukCWnOKUm8\neOa9dHrkDpYecwalD9foKnHOOZciX2uiH5ljHoDuwONmpuhcD0h63szeAMaZ2ZmEDvIRAJJmmtk4\nYCawEThbm2u4cwij4lsB49NVHqm0eAl7r5nMix8c2ID4zjnnalNvJ7qk+XW96jn2I0kDJO0raS9J\nV0flSyV9S9Iukg6XtDzlmKsk7SRpN0nPp5S/GX1Gf0nn15fbOoSBhLaqOJdvT22/LVZJyAieM26e\nM15JyZmLTJ7CSqTmncKEiiVri7MCcc65pKu3DyRpqvpAJv/mafb/07G83u3bDFnyTKFjOedcUcvX\nOJBEatkl3IG0XO93IM45lw+1ViBmttLMvkp5rUz9szFD5mLHE/blvbtfpusTdxc6SlpJaBdNQkbw\nnHHznPFKSs5c1PUU1r8I4y/+ATws6eM69i067ft0oP1Zwwodwznnmqw6+0CiWXK/Rxi41wp4hFCZ\nLG2ceNnzubCccy57ufSBZNSJbmbNCJXILcCVkm7MLWL+eQXinHPZi70T3cy+YWa3AlOBbwDfLebK\nI0mS0C6ahIzgOePmOeOVlJy5qLUPxMzmA8uAh4GfEqYuwcwGQlgnpDECOuecK061NmGZWRmbZ8kV\nYTGpKpJ0SH6j5Sa1CeuVbU+gy9K5dHj5n2w7uHeBkznnXPGKey6sUZJea2Cmguq27D36b3iXuQuW\ngVcgzjkXq7r6QG5vtBR5sq5FGEy47vPiG0yYhHbRJGQEzxk3zxmvpOTMRV0VSFa3MsVoQ8tQgWz4\noujHPTrnXOLU1QeyHHiptgMlHZuvUA2R2gfyaq8TGPrJ33n1wnEMvfHEAidzzrniFXcfyOfADQ2L\nVFjlbcIdSPkyvwNxzrm41dWEtVLSi7W9Gi1hA/T766/58LGp7HP59wodpYYktIsmISN4zrh5zngl\nJWcu6roDmddYIfKld+lOhY7gnHNNVtbrgZjZYcAlkg7LT6SG8alMnHMue7FOZWJmB5vZHDNbZWb/\nZ2Z7RWuZXw3c0dCwzjnnkq2uPpAbCVOYdAEeA14FxkoaJOkfjRGuKUtCu2gSMoLnjJvnjFdScuai\nrj4QSSqL3j9hZosk3dYImZxzziVAXeNAPgQuTim6DvhV1Uax3oWk9oHMemg6lWeexdJuu/HN+f9X\n4GTOOVe84h4H8iJwTMr2SynbIqxUWNwqKthj3VRmfe6d6s45F7da+0Ak/ai2FzC6ETPmrFXXMJCw\nTXnxDSRMQrtoEjKC54yb54xXUnLmos4FpVKZWUczO8vM/gVMy2Om2LTdtj0AbcqLbzJF55xLuvrW\nRG8NHAf8ANgXaAccD7wkqbJREmYptQ9k9Weradt9a9bSitZaW+BkzjlXvOIeB/IgMAc4DLgV6Ass\nk1RWrJVHdW22aUMFzWjNOsrXbix0HOeca1LqasLanbCk7SxglqQKNq9QmAjWzPjoocksfGEWzVqU\nFDrOFpLQLpqEjOA54+Y545WUnLmoqxN9ADCC0Gw10cxeBtqZWfdsT2Jmzcxsqpk9FW13MrPnzWy2\nmT1nZh1S9h1tZnPNbJaZHZ5SPtDMZkSj42/K9Nw7nbwfvb+1K82aZ9zd45xzLgMZz4VlZoOAUwiV\nykJJ38j4JGYXAoOA9pKONbNrgC8lXWtmlwKdJI0ys92BB4DBQG9gItBfksxsMnCupClmNh64WdJz\nac7lc2E551yWYu0DqU7Sm5IuBrYHRmURqjdwNHB3SvFxwL3R+3sJHfMAxwIPSyqXNA+YCwwxsx5A\nO0lTov3uSznGOedcAWTdrqOg1pUK0/gzYQR76m1Bd0lLos9bDHSLynsBC1L2WxSV9QIWppQvjMoS\nKwntoknICJ4zbp4zXknJmYu6RqI3mJl9G1giabqZldaxa6xtTiNHjqRv374AdOzYkQEDBlBaGk5f\n9R+z0NtViiVPkrenT59eVHmSvu3X8+txPcvKyhg7dizApu/LbGW9HkhWH252JXAqUA60JnTIPw7s\nB5RKWhI1T02StJuZjSLc5FwTHT8BGAPMr9onKj8ZGC7pF2nOuUUfyL9K/8h2/32IJb+4nANvHpG3\n39U555Isr30gKSc5zsz2z2RfSZdJ2k7SDsDJwL8lnQY8DYyMdjsDeDJ6/xRwspm1NLN+wE7A61Ez\n1wozG2JmBpyeckydmi/7nP4bZ1Gx4NNMf0XnnHMZyOXZ1v2B35jZsw0479XAYWY2Gzg02kbSTGAc\nMBMYD5ydcjtxDvA3wuDGuZImZHKiyq3DdCZaUVzzYVXdShazJGQEzxk3zxmvpOTMRdZ9IJIuy+VE\nkl4kzPCLpKXAt2rZ7yrgqjTlbwJ7ZX3iDqECYaXPh+Wcc3Gqtw/EzLoBw4CewFrgHeCNYp3OpHof\nyIsn38HwR87mpd1/xkHv3lnAZM45V7xiXQ/EzA4mjPfoTJh99zOgFWH8xY5m9hhwg6TiahuqpqRT\nuAMpWe13IM45F6e6+kCOBn4iabCkn0r6jaSLJR0L7EOoVA5rlJQNsNclRzHvqRnsPuHGQkfZQhLa\nRZOQETxn3DxnvJKSMxe13oFI+lUdPysHnshLoph16NeZDv06FzqGc841OZn0gXQHrgR6SToymq9q\nqKS/NUbAbPlcWM45l718jQMZCzwHbBttzwEuyC6ac865piaTCmQbSeOAStjUfFWR11RfA0loF01C\nRvCccfOc8UpKzlxkUoGsNrMuRPNVmdkBwIq8pnLOOVf0MukDGUhY0nZPwhiQrsAJkmbkP172qveB\nqFLMajOQrcuX03PVXJq3yuv8kc45l0i59IFkNJmimTUHdgEMmC2paBcYT9eJvtw60pEVrPhoKR36\ndipQMuecK1556UQ3s9OBHxBWFBwInBKVJcaaknYArF5cPIMJk9AumoSM4Dnj5jnjlZScucikPWdw\nyvtWhMkPpxJWBUyENSXtoQLWLC7qQfPOOZcoWa8HYmYdCcvOHpmfSA2Trgnrna0PYM/Vk3n7rv+y\n10+HFiiZc84Vr0ZZDwRYDfTL4biCWb9VmA9rwxd+B+Kcc3HJpA/kaTN7Kno9A8wmrCqYGD2fvotP\nX5rLXuccVOgomyShXTQJGcFzxs1zxispOXORSR/I9Snvy4H5khbmKU9ebPuNRN0wOedcIuR1TfRC\n8LmwnHMue3GvB7KSaPR59R8BktQ+y3zOOeeakFr7QCS1k9Q+zaudVx4Nl4R20SRkBM8ZN88Zr6Tk\nzEVG83pE05kcSLgjeVnStLymcs45V/QymQvrd8CJwD+iouOBRyVdkedsOUnXB/La7/5J96suYP7O\nh1P67l8KlMw554pXrH0gKX4I7CNpXXSSq4HpQFFWIGltLKdf+ft8/uWuhU7inHNNRiYDCT8hTGFS\nZStgUX7i5MdW24S5sFqu87mwspGEjOA54+Y545WUnLmo6ymsWwl9HiuAd83shWj7MOD1xokXj5bb\nhD7/rTYWTwXinHNJV2sfiJmdUdeBku7NS6IGStcH8uGzs9nh6F2Z12In+m6YW6BkzjlXvGLtAynW\nCiIXbXqEO5A2FX4H4pxzcam1DySaA+sYM2uR5mc7mNkfzOzM/MaLR9c9urFo4ixazJha6CibJKFd\nNAkZwXPGzXPGKyk5c1HXU1g/AS4CbjKzpcDnhM70fsD7wG2Snsx/xIYraVlCr0P9CSznnItTpkva\n9gW2BdYCcyStyejDzbYCXgJaEiqrxyT93sw6AY8A2wPzgBGSVkTHjAbOJEzceL6k56PygcBYQiU2\nXtIFtZzT58Jyzrks5W1N9IYwszaS1phZCfAKcB7wfeBLSdea2aVAJ0mjzGx34AHCKoi9gYlAf0ky\ns8nAuZKmmNl44GZJz6U5n1cgzjmXpcZaUCorKXcrWxHuQgQcB1R10t9LGN0OcCxhtcNySfOAucAQ\nM+sBtJM0JdrvvpRjEikJ7aJJyAieM26eM15JyZmLvFcgZtbMzKYBi4EXokqgu6QlAJIWA92i3XsB\nC1IOXxSV9QJS1yBZGJU555wrkIwmU2wISZXAvmbWHnjczPag5jTxsbY5jRw5kr59+wLQsWNHBgwY\ngP34fnb+cALP/OyX9D9pIKWlpcDmfx34ds3t0tLSospT13aVYsnj1zP/2349G7ZdVlbG2LFjATZ9\nX2Yrpz4QM7tc0uU5HPdbYA3wY6BU0pKoeWqSpN3MbBRhrZFrov0nAGOA+VX7ROUnA8Ml/SLNOdL2\ngbzW83sc8OnjvHbxoxxw3QnZRnfOuSatMftA3sxkJzPbxsw6RO9bE6ZBmQU8BYyMdjsDqHoc+Cng\nZDNraWb9gJ2A16NmrhVmNsTMDDg95ZiMbGwTBhOWLy2OwYTV/2VSjJKQETxn3DxnvJKSMxc5NWFJ\nejrDXbcF7jWzZoTK6hFJ483sNWBcNBBxPjAi+tyZZjYOmAlsBM5OuZ04hy0f452QTebKNmFCxcrl\nX2VzmHPOuVpksh7IDsDNwFCgEngVuFDSh/mPl73amrAmDfs1B//3SsoO/SOlE39TgGTOOVe88tWE\n9SAwDugB9AQeBR7KPl5hWbtwB8LK4mjCcs65pMukAmkj6f5obEa5pP9jy/VBEmHgnT/l8zfmc8A/\nf1voKEAy2kWTkBE8Z9w8Z7ySkjMXmfSBPBs9HfUw4XHbk4DxZtYZQNLSPOaLTfu+naFv50LHcM65\nJiOTPpCP6vixJO0Qb6SG8alMnHMue7HPhRU9PTVU0isNDddYvAJxzrnsxd6JHo0iv61BqVxaSWgX\nTUJG8Jxx85zxSkrOXGTSif4vM/t+NICvaXvuOTjoILjuOn9ayznn6pFJH8hKoC1QQVgPxAh9H+3z\nHy97tTVhfTbzC9bvtR/rS1qz04ZZ6Q9etAi23x4qKqBjRzj3XDjvPOjaNc+pnXOusPIyDkRSO0nN\nJLWQ1D7aLsrKoy6tO7WiT+V8tt34MSxbBukqzl694J574JvfhOXL4YorQoVy++2NH9g554pcvRWI\nBadGEyFiZn3MbEj+o8Wrbbe2VGK0ZQ0VPXry2WMvpd3vk0NPY9ZdL7HgoZdZc8h3YO1alnXekbVr\nq+24YQNMnQpPPw133gm//S386EfwixrzO6aVhHbRJGQEzxk3zxmvpOTMRSbjQG4nTGFyCPBHYBXw\nF8KqgYnRrMRYZh3ppGWUbFjHB/e9QrcTh9fY709/qrrhGAY8za7M4r1TduXWW0OL1iaffgqDBtU4\n/suSrhz5xh386lcwYkTKD1atgn/8g1f0Dd5YtiOLl4Sizp3Dq1cvqBos75xzSZBJBbK/pIHRolBI\nWmZmLfOcKy+mnnEzS8re49nuIznhx/3T7tO9O+yyC5SXw8aN8FX5bnTdCG3bVttx221h7715Z2lP\nXl3Ym0X0YhG9WFjRmzfeCC1gW3jtNTjjDIYB/enKx2zHF2zDExzCdVxSs4KKPPMMzJ4NffpA797h\nz223heZ5X8ll8xoCxc5zxstzxispOXORydfQxmg9cwGYWVfCHUniHHrPaQD8oI59fve78KpXy5bw\n1ltsvxI6r4T168Nrwwa4fG3oOtlCmzZw3HGsm/Rfun31Od34HIBmXTrzRHQHsoV//xvOO4+WGw/m\nlTkH8yLDWUqXcEyz0Gr2k59k9Gs751x+SKrzBfyQsE7HIuBPwGzgxPqOK9Qr/EpFrLJSmjdPk26/\nXXrmGWny5PT7/fa3Uujq3/Sa03pvndf+HplJEyemP+zCC6VjjpFGj5aeeEL68svco06aNCn3gxuR\n54yX54xXUnJG351Zfd/Wewci6QEzexM4NCo6XlItz8G6epmF25PddoO6bm1//Ws47DCYNAnKyuC/\n/6X/2hncfO0qrvtpuAtJ54UX4J13Qt9+1en23hvuuy/86ZxzccloSVszGwgcSGjGekXS1HwHy1WT\nncpk3TqYPBn694eePWv+/JJL4IsvWLT3UUzpeBhvvN+R//wndL1s2ACffw7bbFPzsIoKKCnJf3zn\nXHGLfS6s6EN/B5wI/J0wiPB44FFJV+QaNJ+abAVSl8rKUKksWRK2S0pg2DA45RTWnnAab73flgMO\nqHnY+vWwww5h1xNOgKOPhq23btzozrnikK8FpX4IDJZ0uaQxwAHAabkEdJvF+my4WWi7uvrqMBUL\nwEsvwbnn0nrjV2krDwh3J598Ao8+CiedFAbcf+97YTv2jHnkOePlOeOVlJy5yOQprE8IC0iti7a3\nInSou2JhBnvtFV6XXhqeIX7mGXj//fDMby2GD4ePPoJ//AMeewxefRUefzw0a514YiPmd84lUiZN\nWE8QBg2+QOgDOQx4HVgIIOm8PGfMyteyCStbL70EDz4IP/85DBiwqXjhwlCB7LILHH54zcM2bAhP\nLzvnmp589YGcUdfPJd2bzQnzzSuQDIwYsbmdavhwuOACOOaYenvTTzsN5syBs84KTV4dOjRCVudc\no8ilAin4uI24XxT7OJBIQZ8Nf/dd6bzzpHbtNo8z6ddPmjp1i91SM27YIPXosXn31q2l00+XysrC\n0JZCSspz9p4zXp4zXuQwDiSTTnTX1Oy+O9x8c2izuumm8CjWF1/AjjvWekiLFvDBB2E8SWkprF0b\n3h95JHz65jQ+AAAWA0lEQVT1VeNFd84Vj4zGgSSJN2HloKICZs2CPfes+bPKytBJX209sQ8+CDPf\nr1sH11/fSDmdc3mTlz6QNCdpBRwj6dGsDmwkXoHEbNw4+P3vw0yPp52W8UCR8ePh2WfhjDPCpMVf\ng/UsnUu0fI0DwcxKzOxoM7sfmA+clEtAt1kSng0vKyuDhx+GmTPh7LPDjI/nnx960utx++1w220w\neDD06xf66V98Mdzs5CVnAnjOeHnOwquzAjGz4WZ2FzAPOIvwCG8/SSc0QjZXDB55JLwOPDB0dtxy\nS3jOt56/FFdcEVYD7tED5s8PXS6lpeGuxDnXNNTahGVmC4GPgTuAJyStNLOPJPVrzIDZ8iasPJo2\nDf7yl3ArMXNm6FmvR2VlmMLr8cfDYPlXX4VWrWru99FH0LevN3U5Vyix9oGY2U2Eea/eAR4EngTe\nlrRDFoF6A/cB3QlriPxV0i1m1gl4BNiecHczQtKK6JjRwJlAOXC+pOej8oHAWMKo+PGSLqjlnF6B\n5Ft5efoVrZYsCf0lJ54Y1pXPcNWrpUuhS5ewmNdBB4WhKcOHh4fFapt12DkXr9jHgRAmTzwY+F/C\nyPOVwAhg60yeEQZ6AAOi91sT1hLZFbgGuCQqvxS4Onq/OzCNMMVKX+B9NldykwlzcgGMB46o5ZwN\nfh66MSTh2fCsM95+++aBIl27Sj/7mTRhgrRqVZ2HTZkide+++dCq18CBecpZIJ4zXp4zXsQ9DiT6\n3EmSfgr0A04BjiPcNWRSOS2WND16vwqYBfSOPqNqBPu9hDsdgGOBhyWVS5oHzAWGmFkPoJ2kKdF+\n96Uc44rFwQfD6NGw005h/vi77goDRX7/+zoP22+/sMT8e++FQ374w7B8b7qniiG0nl1/feiG8TEo\nzhVOxo/xmlkLYE/CRIorJa3N6kRmfYGy6DMWSOqU8rOlkjqb2a3Aq5IejMrvJtxtzAeuknR4VH4g\n4Q7m2DTnUaa/k8sTCWbMCNOlPPdc6FE/4oia+z3/fOj0GDwYOnas8RHr1kHr1jUP+/Of4aKLNm/3\n7x/mkfzBD+D734/5d3HuayKXJqxaG6nN7E7gVknvmlkH4FWgAugMXAw8lEWwrYHHCH0aq8ys+je8\nf+M3JWawzz7hdUUdy8aMGRPmlIcwCn7QoPA64wyse/e0lQfAvvuGeSDffBPeegvmzg2vgQPT7z9t\nGnz2WVgEsk8f76h3Li519XJ+U9LPo/c/AuZIOj5qTnqWDCsQM2tOqDzul/RkVLzEzLpLWhJ93mdR\n+SKgT8rhvaOy2srTGjlyJH379gWgY8eODBgwgNJo+diqZ7ILvV1VVix50m1Xzxrr50c95WXLlsGH\nH1L6wQfwwQeUjRsHvXpR+sMf1jz+888pe+cdMOOOO8LnvfBCGc8/P5199rmAffdNf77rroPx48N2\nq1ZlbLcdDBlSyi9+ARs2NJHrGeP29OnTueCCC4omT23bfj0bfv3Gjh0LsOn7Mmu1dY4A01Le/xMY\nme5n9b0I/RU3Viu7BrhUtXeityT0uaR2or8GDCF07I8HjqzlfDl3IjWmJHSsNVrGDRuk6dOlv/1N\nOv/89LMzVlaGyR87dZKGD5f+53+kv/5VmjxZkyZOrPPjr71WOuig0K+f2kn/+OPp93/wQemOO6R/\n/UtasECqqGj4rygl47+55DnjlpSc5NCJXtdjvJOAGwj/0p8E7CppcXRH8Y6kXeurnMxsGPAS8Dah\nmUrAZYT1RMYR7irmEx7jXR4dM5owaHEjWz7GO4gtH+M9v5ZzqrbfySXYkiXhud6lS7csb90aVq6s\nORV9ZSUsWBDarJptflbkyy/DtF+zZoUZ7Hv0qHmqb3wjjFep0qZNeC7gnntqbyZzLuniHgeyM3AL\n4VHcmySNjcqPAA6X9MuGxc0Pr0CaMCk8rjVjxuZXZWVYHKu6efPCHCqtW8POO8Ouu4ZOkAED4Ljj\n6jzNbbfB1KlhxpY5c8IDZQCzZ4ePqu6734VVq8Kkxn37bn7tu2/6QZPOFaNGWw8EuCCX4xrjhTdh\nxSYJGaVack6enH5wyaBB6T/kiy+kiROluXOldeu2+NGyZdLrr4eWtuoqK6UOHWqeBqT589PnfOyx\n0Dw2Z460Zk32v2++Jfq/exFKSk5yaMLKbKhwTRcBN+V4rHP5N2QILF4c1oefPXtzu1W6NisIbVbH\nHLN5u0cP2H57OOooOo4Zw+DBtZ/qtdfC9Pbz5m1+ffwx9OxZc18JTj01PKJcZZttQkvbSy+ln+x4\nxQpo396fHnPFJ6f1QMxsgaQ+9e/Z+LwJy+Vk4kT4wx/CzI+LFm2eNvj00+HeNKs2P/44XHxxGPHY\np0949egRHkM+8MBaT7NuHfz4x6F75uOPw5pe5eWhpW316pqVRHk5bLVVaArr2RO23TacZtttw1pg\nXqm4uDTKeiDRiT6WtF3WBzYCr0Bcg5WXwyefhG/4du3CeJbqbrwRfpmmG/DnP4c77qhZ/sQTcOed\n4Xaja9dNf1bsM5Alffbjs89C90x1S5aEDvxVq7Ys79w5PBBQ3Vdfha6fbt3Cq2vX8Gfv3nDZZZn9\n+u7rKe5O9JWEp6aqPrBqRwNaS8q1+SuvklKBlJWVbXo2u1glISMUKOfataGCWbAg3EYsWBCazIYP\nhxEjau7/pz9R9pvfUCPlxRfDddfV3P/uu+HWW8MI/U6d2LB1J1aWdGThHkcys88RrF8PI0em7L98\nOaxfz4dftGfHPVux+a9t0KdPiFvdJ5+EBwO6dAl1WpcuUF5exv77l3LVVTX3Ly8Pv2bHjtC2bWHv\ngPz/z3jFOhJdUruGR3KuiWrdOqyLsssume1/6qnhUeOePcP6859/Hv4cOjT9/vPnh6fMIi2BLkCX\nMR3Y55Q008LccguMGcMOQGVJCZVttmZjq3bMOvpiXtnv/BpPOfPCC/DiizRb2ZYzV7dl9eq2rP64\nLW+xD+8RKpYtKpC1a6Gyko8WtmbnXcNj0SUl0KFDqEz23BOefJIali0Li1q2bx/2bd8+vDp1CpWa\nS7a67kBaAT8HdgJmAP9PUnkjZstJUu5AnKvTF1+EO5tly7Z8DR0Kw4bV3P/KK8MkYV99BRs2bC6/\n+mq49NKa+48aBddcU6N4zulXMOXwX9OyZZiVf5Nf/zqcA1hPS9bSmnW04kou41bOY999w6PPmzzw\nADz1FEtXb8Xf/7kV6wmvZ/gOZRzMXnttUT+G+WZmz+aTL1py/S0tadG2Jc3btGTFNjuyoXsfdtoJ\nLrkkZf+VK2HtWlaua8HUt1vQun0LWrdrTpt2JbRtGx5GyHD1ZReJ9Q6EMEvuRuA/wNHAHkDawXvO\nuZhts014ZeqyyzZ3cmzYEDpNVq2q/Vv0iCNCG9Tq1Vu8dv7enuycbpiMWbjrWruWrdjAVmwAVnDV\n5Rs474dpliqeNg3GjaMz8JOU4tbbd+fL9gfTv3+1/R98EK6/np7AjSnFl3ANf+ESBg+uVoH88Y9w\n3XW0A4ZX2/86LmHQIHjjjZQfXHkl3H036yubs2hJcyqbtaCypDnP7HgBk3c+jd12g8svT9n/nnvg\nqadYt7GERYtLsBbNadaihE9LT2HFN45im23CLNKbPPMM/Oc/VFoJGytLKGkZXnb4YXDAATWv56uv\nhhq0pCQMdC0pCa9Bg8J4pepmzQqP9zVrtuVrp53S38otWhTucqv223lnaNmy5n4NVdvzvYTFo6re\nNwemZvuMcCFe+DiQ2CQho+Q541ZnzspKae3aMDjm00+lFSvS7zd9uvTQQ9LYsdJdd0m33CJdd10Y\nn5PO/fdLJ52kDd85XkuHHq0v9v2WPttjuF4652Hdeaf0yCPV9v/tbzWpfXuVt+ugNc3aaIO1kEBX\ndL5B3btLRx5Zbf8LL0w7WOdCbhBIw4Zltv9FXC+Qhg7NbP9LSq5Xq1aTMs5zz57X66STpN//vtr+\nF12Udv//HH+97rpLeuaZuvdfO/PD9Nc9BTGPA9mYUsmUmz8v6JwzC88U1zfEvmo25kydeiqceiot\ngE4pxV2Bb6bb/w9/gEMOoaS0lNRJm38t8et0X1W/+Q2cey6rV5Tz0dxy1q8pZ8OqjRy+dS/2b11j\nNQE480w48EA+WVDBP5+qoGJDBRUbylnfZj+O2CrNTcJ3vgPdujHvwwoeuLcClVdAZQUvVxzAuoqN\nrK2++MXQobBqFZ8uqmTC+ApKqKAZlYx7ZzeefSeszPm736Xsv8sucOSRLP2ykjenVNKMSkqo4PYn\n+vDoE2EB0G9/O2X/nj1hn31YtbKSeR9W0m5ZS7ZPdx0bqK4+kApgddUm0BpYE72XpPZ5yNNg3gfi\nnCsGUmhNXL8+vO/QoeY+y5bB9OmwcWPYt+rVpQscdljN/T/8EP72t837V/3Zv3+1Jr7I1Klw/vnw\n2GNhyei6NNo4kGLmFYhzzmUvlwqkziVtXf6krmVQrJKQETxn3DxnvJKSMxdegTjnnMuJN2E555zz\nJiznnHONxyuQAklCu2gSMoLnjJvnjFdScubCKxDnnHM58T4Q55xz3gfinHOu8XgFUiBJaBdNQkbw\nnHHznPFKSs5ceAXinHMuJ94H4pxzzvtAnHPONR6vQAokCe2iScgInjNunjNeScmZC69AnHPO5cT7\nQJxzznkfiHPOucaT1wrEzP5mZkvMbEZKWScze97MZpvZc2bWIeVno81srpnNMrPDU8oHmtkMM5tj\nZjflM3NjSUK7aBIygueMm+eMV1Jy5iLfdyD3AEdUKxsFTJS0C/BvYDSAme0OjAB2A44CbrfNC7Hf\nAZwlaWdgZzOr/pmJM3369EJHqFcSMoLnjJvnjFdScuYirxWIpJeBZdWKjwPujd7fCxwfvT8WeFhS\nuaR5wFxgiJn1ANpJmhLtd1/KMYm1fPnyQkeoVxIygueMm+eMV1Jy5qIQfSDdJC0BkLQY6BaV9wIW\npOy3KCrrBSxMKV8YlTnnnCugYuhE/1o+MjVv3rxCR6hXEjKC54yb54xXUnLmIu+P8ZrZ9sDTkvaO\ntmcBpZKWRM1TkyTtZmajAEm6JtpvAjAGmF+1T1R+MjBc0i9qOd/XskJyzrmGyvYx3ub5CpLColeV\np4CRwDXAGcCTKeUPmNmfCU1UOwGvS5KZrTCzIcAU4HTgltpOlu0FcM45l5u8ViBm9iBQCnQxs48J\ndxRXA4+a2ZmEu4sRAJJmmtk4YCawETg7ZUTgOcBYoBUwXtKEfOZ2zjlXvyY3Et0551zjKIZO9FiY\n2ZFm9l402PDSQuepjZnNM7O3zGyamb1e6DxVsh30WSi15BxjZgvNbGr0OrLAGXub2b/N7F0ze9vM\nzovKi+p6psn5P1F5sV3PrcxscvR35m0zGxOVF9v1rC1nUV3PKmbWLMrzVLSd9fVsEncgZtYMmAMc\nCnxC6Cs5WdJ7BQ2Whpl9CAySVH18TEGZ2YHAKuC+lAcergG+lHRtVCl3kjSqCHOOAVZKurGQ2apE\nD4f0kDTdzLYG3iSMf/oRRXQ968h5EkV0PQHMrI2kNWZWArwCnAd8nyK6nnXkPIoiu54AZnYhMAho\nL+nYXP6+N5U7kCHAXEnzJW0EHib8RShGRhFe9ywHfRZMLTlhywc1CkrSYknTo/ergFlAb4rsetaS\ns2qMVdFcTwBJa6K3WxH6bkWRXU+oNScU2fU0s97A0cDdKcVZX8+i+yLLUfVBiMU82FDAC2Y2xcx+\nUugw9aht0GcxOtfMppvZ3YVuykhlZn2BAcBrQPdivZ4pOSdHRUV1PaPmlmnAYuCFaGaKorueteSE\nIruewJ+BX7HlOLysr2dTqUCSZJikgYTa/5yoSSYpirW983ZgB0kDCH9xi6KpIGoWegw4P/oXfvXr\nVxTXM03Ooruekiol7Uu4kxtiZntQhNczTc7dKbLraWbfBpZEd5913RnVez2bSgWyCNguZbt3VFZ0\nJH0a/fk58Dih+a1YLTGz7rCpvfyzAudJS9LnKY98/xUYXMg8AGbWnPClfL+kqrFORXc90+UsxutZ\nRdJXQBlwJEV4Pauk5izC6zkMODbqj30IOMTM7gcWZ3s9m0oFMgXYycy2N7OWwMmEgYlFxczaRP/a\nw8zaAocD7xQ21RZqG/QJWw76LLQtckb/s1f5HsVxTf8fMFPSzSllxXg9a+QstutpZttUNfuYWWvg\nMEJ/TVFdz1pyvlds11PSZZK2k7QD4bvy35JOA54my+vZJJ7CgvAYL3AzoVL8m6SrCxypBjPrR7jr\nEKGD7YFiyWkpgz6BJYRBn08AjwJ9iAZ9Siro1KK15DyY0H5fCcwDflbVllsIZjYMeAl4m/DfWsBl\nwOvAOIrketaR8wcU1/Xci9Cp2yx6PSLpT2bWmeK6nrXlvI8iup6pzGw48MvoKaysr2eTqUCcc841\nrqbShOWcc66ReQXinHMuJ16BOOecy4lXIM4553LiFYhzzrmceAXinHMuJ16BuCbHzFZGf25vZqfE\n/Nmjq22/HOfnN6Z8XB/39eIViGuKqgY39SMMistYNA13XS7b4kRSkuYyqy7r6+NcKq9AXFN2FXBg\ntGjO+dFMqddGi/5Mr5oN2cyGm9lLZvYk8G5U9ng0Y/LbZvbjqOwqoHX0efdHZSurTmZm10X7v2Vm\nI1I+e5KZPWpms6qOi352tZm9E2W5tnp4M2trZv/PzGZE+3w3Kj8lKpthZlen7J+a5ftmdk/0/h4z\nu9nMXjGz983se+muTyxX3H2t5HVNdOcKbBTRNA0AUYWxXNL+0Zxpr5jZ89G++wJ7SPo42v6RpOVm\n1gqYYmZ/lzTazM6JZlOuouizvw/sLWkvM+sWHfNitM8AYHfCTKyvmNk3gPeA4yXtGh3fPk3+30Z5\nqxbO6mBm2wJXR3mXE5YGOFbSU9Q9O20PScPMbDfCHFL/qH59nMuW34G4r5PDgdOj9RomA52B/tHP\nXk+pPAAuMLPphHU8eqfsV5thhJlNkfQZYSbWqllXX5f0aTQj63SgL7ACWButD/FdYG2az/wW8Jeq\nDUkros+cJGmppErgAeCgaJe6puZ+IvqMWRTBuhmuafAKxH2dGPA/kvaNXjtKmhj9bPWmncIEc4cA\n+0drOEwHWqV8RqbnqrI+5X0F0FxSBWEq/8eA7wAT0nxGbRPV1ZYhdf9W1X6WmqGoVsdzyeUViGuK\nqr4gVwLtUsqfA86O1sDAzPqbWZs0x3cAlklab2a7Agek/GxD1fHVzvUf4KSon6Ur8E3C7LvpA4bz\ndpQ0AbgI2DvNbi8A56Qc0zH6zIPMrHPU4X8K4W4HwnoOu5hZM+C7tZ2b2q+Pc1nxCsQ1RVX/Ep8B\nVJrZNDM7X9JfgZnAVDN7G7gTSPfU1QSghZm9C1wJvJrys/8FZqR0hgtA0uPR+d4CJgK/ipqyasvW\nHnjGzN4iTKl+YZp9/wR0jjrmpwGl0VKjowiVxjRgiqRnov1HA/8EXgY+SXPO6ttbXJ8053euTj6d\nu3POuZz4HYhzzrmceAXinHMuJ16BOOecy4lXIM4553LiFYhzzrmceAXinHMuJ16BOOecy4lXIM45\n53Ly/wExWgTaHayOKQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xbd609b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print_measures(model_plsa, model_artm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Кроме того, для наглядности построим графики изменения разреженностей матриц по итерациям:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFMX5x/HPA16AwoIKGBBXQFHwWFTQCOp6At6KURGi\nq0k0XlG8QKNiPBIwapBojHiBRyTGeKD+gifrQVQQWBDkMtxyeIECoq7s8/ujemB2dmZ2ZrZnunv3\neb9e/dqtnpqeL706NVPVXSWqijHGGBPTKOgAxhhjwsUaBmOMMdVYw2CMMaYaaxiMMcZUYw2DMcaY\naqxhMMYYU03eGwYR6Ssic0VkvogMSfJ4kYg8JyIzROQDEema70zGGGNSy2vDICKNgPuAPkA3YICI\n7JVQ7QZguqruD5wHjMpnJmOMMenl+xtDT2CBqi5R1UpgHHBKQp2uwFsAqjoPKBaRnfOcyxhjTAr5\nbhjaAcviysu9ffFmAKcDiEhPoAPQPs+5jDHGpBCGwefhQEsRmQZcCkwHNgUbyRhjGq6t8nz8z3Df\nAGLae/s2U9V1wAWxsogsAhYmHkhEbFInY4zJgapKNvXz/Y1hCtBZRHYTkW2As4Hx8RVEpIWIbO39\n/hvgbVVdn+xgqhr6bdiwYYFnsJyWM6oZLaf/Wy7y+o1BVTeJyGXAa7hG6BFVnSMiF7mHdTSwNzBW\nRKqA2cCv8pkp3xYvXhx0hIxYTn9FIWcUMoLlDIN8dyWhqhOALgn7Hoz7/YPEx40xxgQnDIPP9UpZ\nWVnQETJiOf0VhZxRyAiWMwwk1z6oQhMRjUpWY4wJCxFBQzb43OCUl5cHHSEjltNfUcgZhYxgOcPA\nGgZjjDHVWFeSMcbUY9aVZIwxps6sYfBZVPodLae/opAzChnBcoaBNQzGGGOqsTEGY4ypx3IZY8j7\nnc/GGGN8NHkyPPUUrFoFhx0Gl13m+0tYV5LPotLvaDn9FYWcUcgIDTDnvHkwfDgMHgznnANHHw37\n7ANXXJG8/oIFMGoUPPMMTJrkT4YE9o3BGGP8tHIlTJwIn33mPtXHtu7d4a67atafMweuv77m/l13\nTX78nj3hnnugbVvYK3GlZH/YGIMxxtRGFb76ChYuhEWL3NaiBVx8cc26r70GffrU3H/44fD22zX3\nL1gADz0Ebdps2Vq3hp/9DHau+yrHuYwxWMNgjDHg3vwlyfvnlClw1FGwPmGZmAMOgKlTa9ZfuBCG\nDoV27WCXXdwn+7ZtoUOHvH3CT8f3wWcR+TkwCDgM2AXYCMwCXgGeVNVvcsxab5WXl1NaWhp0jFpZ\nTn9FIWcUMkIBcn7/Pbz/vvukPn/+lm377eGjj2rWb93aNQotWsDuu2/eyhs3JmnKjh1d/3+EpWwY\nROQ/wArgReAO4HNgO2BP4EjgRRG5R1XHpzqGMcYEYuNGWLoUuiRZ6mXNGvcNIFGTJlBVBY0SrsnZ\ndVf4+mto2bL6/ogMkuciZVeSiOykql+mfXJmdfoCI9mygtuIhMebA0/i1oZuDNytqmOSHMe6kowx\nNW3a5C7f/OQTmD3b/Vy0CLbaCr77zv2MpwrHHuve8PfYA/bc0/3s3BmaNQvm35BHoRtjEJFGwHzg\naNy3jynA2ao6N67O9UBzVb1eRHYC5gFtVPWnhGNZw2BMQ1VVBZ9+6t68Ez/Rq0KrVrB27ZZ9jRtD\np05usLdt28JmDRlfJ9ETkXUi8m2qLcPj9wQWqOoSVa0ExgGnJNRRYAfv9x2ArxIbhShpcNdg55nl\n9E8UMoKXc8YMeOQRd/NWr17QvLnrFvr005pPEHH1brnF9e3Pnu2+Kcybl9dGISrnMxcpxxhUdQcA\nEbkNWAk8AQgwEDcQnYl2wLK48nJcYxHvPmC8iKwAtgfOyvDYxpj66uKL3QBxvHbtYPVq1/WT6Lbb\nCpOrgcjkBreTVXX/uPIDIjIDuNmnDH2A6ap6lIh0Al4Xkf1UdX1ixbKyMoqLiwEoKiqipKRk89UL\nsdbbypmVY/vCkifq5di+sORJVY7PWvDX/+knSps3hylTKH/5ZZg/n9J774UTT6xWv7S0lPKuXaFJ\nE0r79IHu3SnfsAGKiig97LDg8icpx4QlT2lpKeXl5YwZMwZg8/tltmodYxCR/wL347qBFBgAXKqq\nh9Z6cJFDgFtUta9XHgpo/AC0iLwM/ElVJ3nlN4EhqvpRwrFsjMGYqLrjDrdt3Fh9/003wa23BpOp\ngcjXQj3nAGcCq73tF96+TEwBOovIbiKyDXA2kHh56xLgGAARaYO7HHZhhscPncRPEmFlOf0VhZx5\nz7h0qZveIZkdd3SNQpcucN55bq6fSZNgyJDC5/RJVHLmotauJFVdTM0B44yo6iYRuQx4jS2Xq84R\nkYvcwzoauB0YIyIzvaddp6pf5/J6xpgCWrXKzQn01lvu5//+B/37w7PP1qw7YACccQbstFPhc5qs\nZdKVtCfwAO4S0n1EZD/cuMPthQgYl8O6kowJi3ffdXP/xGve3L35P/JIMJlMUnm5j0FE3gauBR5U\n1e7evlmquk/OSXNgDYMxBabq5v3p1KnmYxs2QPv2cPDBcOSR7k7i7t1r3kxmApevMYamqjo5YV9k\n7zPIt6j0O1pOf0UhZ0YZN2yA8ePhoovcpG9dulS/cSymWTP48kuYMMGNE/To4VujEIVzCdHJmYtM\n/pJfepeRKoCInIG7r8EYU58MHOjGB378ccu+tm3dZHM9etSs37hx4bKZgsqkK6kjMBo4FFgDLAIG\nquqS/MerlsO6kozJp/PPh7FjXffQ8cfDCSdASUnNKShMpPg+xuDNdXSGqj4jIs2ARqq6ro45c2IN\ngzF1oAoffwzPPw/77QennVazztKlboZRHxaHMeHh+xiDqlYB13m/bwiqUYiSqPQ7Wk5/hTJnrDG4\n6Sbo0oXy/fd38wk9+GDy+h06hKJRCOW5TCIqOXORyRjDGyJyDfBPYENsp91rYEzIvfsuHHHElnLz\n5nDmmfCLXwSXyURCJmMMi5LsVlXtmJ9IKXNYV5Ix2fjpJ9hnH3e/wZlnQmmpXU7aAIVuPQY/WcNg\nTIJvv3XTTD/1lPuZrBso1TrGpsHwez2G3rW8WHMRKehNblEQlX5Hy+mvguXctAneeAMGDXKXkv7m\nN26JyaefTl4/rlGwc+mvqOTMRbrvlf1F5E5gAjAV+AK35nNn3JrPuwFX5z2hMWaL666De+7ZUi4t\nhbIyN0eRMT6p7XLVVkB/oBducZ6NwBzgFVV9ryAJt2SxriRjJk2Cc891M5Seey7kON++aThsjMGY\n+uC77+Cdd6Bv35qPqbrNbjozGcrXXEkmC1Hpd7Sc/vIl54oV8Pvfu/sJjj/eTWOdSCTnRqFBncsC\niErOXNi1a8YEbeZMuPNO+Oc/3SWmAAcdBGvWBJvLNFjWlWRM0C6/HO67z30TOP10uPJKOPRQu8zU\n+CKXrqScvjGISFtVXZVh3b7ASLas4DYi4fFrgIG42Vu3BvYGdlLVJHP9GlMPDRniZiq98kobTDah\nkOsYQ0ZLNHmT8N0H9AG6AQNEZK/4Oqp6l6p2V9UDgOuB8ig3ClHpd7Sc/soo59y5yfe3bw8jR+a9\nUahX5zIEopIzFzk1DKp6QoZVewILVHWJqlYC40i/fvQAIMWdOsZE1IcfQr9+sPfe8MEHQacxplYZ\njTGIyAFAb1x3zyRVnZbRwUX6A31U9UKvPAjoqaq/S1K3CbAc6JTsG4ONMZjIWbIEhg6FceNcuVkz\nuP9+dw+CMQWSlzEGEbkZ+AXwnLfrMRH5l6renkPGdE4C3kvXjVRWVkax93W7qKiIkpISSktLgS1f\n66xs5VCU77wTbryR0spK2HZbyk8/Hc46i9JTTglHPivX23J5eTljxowB2Px+mTVVTbsB84Dt4spN\ngHm1Pc+rewgwIa48FBiSou5zwNlpjqVRMHHixKAjZMRy+qtGzjVrVHfeWXXAANXFiwPJlCiy5zKk\nopLTe++s9f06fsvkqqQVuDmSvvfK2wKfZdjuTAE6i8huuHWiz8aNI1QjIi2AI3BXJxkTfUVFMGcO\n7Lhj0EmMyVom6zG8APQAXseNMRwLTMaNB6BJxgsSnt8XuJctl6sOF5GL3FN1tFfnPNxYxDlpjqO1\nZTWm4JYuhS+/hAMOCDqJMUnlZa4k7007JVUdm80L5soaBhMqqvDYY+7eg913h2nT3L0IxoRMXuZK\nUtWxsQ0YD0xP2GfixAaBws5y1sHKlXDSSfCrX8G6ddCpE+Wvvhp0qlqF8lwmYTmDV2vDICLl3qI8\nrYBpwEMick9tzzOmXnr2WejWDV55xY0jPPkk/Pvf0LRp0MmM8U0mXUnTVbW7iPwa2FVVh4nITFXd\nrzARN+ewriQTvLFj3cI4ffvCww9Du3ZBJzImrXzNlbSViOwCnAn8PqdkxtQX554LrVu7hsEmuTP1\nVCZTYtwKvAp8qqpTRKQjsCC/saIrKv2OljNHIm56i4RGIXQ5k4hCRrCcYZDJ4PO/VHU/Vb3EKy9U\nVVtg1tRvCxbASy8FncKYQNh6DMYkev55N45QWQmTJ8M++wSdyJic2dKextRFZSVce61bLOfbb12X\nUYcOQacypuCsYfBZVPodLWeCFSvg6KPhrrvcjWp33+0uTW3ePKOnR+F8RiEjWM4wSHlVkogMUtUn\nReSqZI+rqt3LYOqPr76Cjz6CXXaBZ56B3r2DTmRMYFKOMYjIRar6oIgMS/a4qv4hr8lq5rExBpNf\nr74KJSXQpk3QSYzxTV7mSgoLaxiMMSZ7eRl8FpGdReQGERktIo/Gttxj1m9R6Xds0DkrK30/ZBTO\nZxQyguUMg0wGn18EWgBvAK/EbcZEz4oVcPDB4K1wZYypKZO5kipUtaRAedLlsK4kUzezZ7tLUJct\ng65doaICtt466FTG5FW+7mN4WUSOzzGTMeFQXg69erlG4ec/h7fftkbBmBRSNgwisk5EvgWuwDUO\nG0Xk27j9GRGRviIyV0Tmi8iQFHVKRWS6iMwSkYnZ/zPCIyr9jg0q53PPQZ8+8M03cNpp8OabsNNO\ndT9unCiczyhkBMsZBinvY1DVHep6cBFpBNwHHI1bO3qKiLyoqnPj6rQA7geOU9XPRMTf/2ON6dIF\nmjSB3/4W7rnHVlozphbp7mNoDdwAdAZmAsNVNeNvCt4xDgGGqWo/rzwUt9bziLg6FwO7qOrNtRzL\nxhhM7pYtg113DTqFMQXn9xjD48AG4K/ADsCoHDK1A5bFlZd7++LtCbQSkYkiMkVEfpnD6xiTnjUK\nxmQs3UI9u6hqbGGeV0VkWh4zHAAcBTQD3heR91X108SKZWVlFBcXA1BUVERJSQmlpaXAlv6+oMux\nfWHJk6o8cuTIUJ4/O5/5K1dUVHDllVeGJk+qcuLfPug8qcphPZ/l5eWM8S7Hjr1fZk1Vk27ADKAl\n0MrbqpVTPS/hGIcAE+LKQ4EhCXWG4LqbYuWHgf5JjqVRMHHixKAjZKTe5rzlFtX7789LlnSicD6j\nkFHVcvrNe++s9f06fks3xrAYqAKS9U2pqnasrdERkcbAPNzg80pgMjBAVefE1dkL113VF9gW+BA4\nS1U/STiWpspqDAC33QY33+wGl+fOhc6dg05kTOB8XfNZVYvrGkhVN4nIZcBruPGMR1R1johc5B7W\n0ao6V0RexQ1wbwJGJzYKxtTqj390jUKjRvD449YoGFMHeV+PQVUnqGoXVd1DVYd7+x5U1dFxde5S\n1W7qlhD9a74z5VN8/2iY1aucd94Jv/+9W4d57Fg455y850oUhfMZhYxgOcPAFuox0fbVV25xHRF4\n7DEYNCjoRMZEnk27baLvk09g6lT4pV3pbEwiX9djEJFW6Z6oql9n80J1ZQ2DMcZkz+8b3L4EKoCP\nvG1q3PZRriHru6j0O1pOf0UhZxQyguUMg3Q3uI0CjgQmAU8D79lHdhM4VTeeYIzJm7RjDCIiQCkw\nAOiJu+z0AVVdVJB01bNYu9TQqcKFF0KnTjBkiDUQxmQgb2s+i0gRcDZwG3CDqj6UW8TcWcNg+Pvf\n4eKL3Uyps2ZBx1rvsTSmwfN1jEFEmonIOSLyIvB/wPbAgUE0ClESlX7HyOV8/3343e/c7w8/HLpG\nIQrnMwoZwXKGQboxhs+BBcA476cCB4nIQQCq+lz+4xkDrF4NZ5wBlZVwxRWB3MBmTEOS7nLVx9I8\nT1X1gvxESs66khqwM86Af/8bDjvMrb5mS3IakzFf50oCXrJvBSYURo6En35yYwzWKBiTd+nuY7ix\nYCnqkaj0O0YqZ/v28MIL0LZt0HFSisL5jEJGsJxhYHMlGWOMqSbdGMN3QI1V1HDrM6iq7pfPYEny\n2BiDMcZkye8xhkXASXWLZEwOPv8cdt7ZbmAzJiDpupJ+VNUlqbZMX0BE+orIXBGZLyJDkjx+hIis\nFZFp3hbpsY2o9DuGNmdlJZSWwlFHwcqV4c2ZIAo5o5ARLGcYpPvGMKmuBxeRRsB9uKU9VwBTRORF\nVZ2bUPUdVT25rq9n6oEHHoA5c1wD0SrtBL/GmDypba6kxkBLVf3SK28DlAGDVXXvWg8ucggwTFX7\neeWhuPGJEXF1jgCuUdW03VY2xtAAfPkl7LEHrF0L48fDSdaTaUxd+T0lxlnA18BMEXlbRI4DFgL9\ngIEZHr8dsCyuvNzbl+jnIlIhIq+ISNcMj23qm5tvdo3CccfBiScGncaYBivdGMNNuLmRfgYMBl4C\nLlbV01R1mo8ZpgIdVLUE1+30go/HLrio9DuGLuesWfDgg9C4MfzlL5sHnkOXM4Uo5IxCRrCcYZBu\njOFHVf0UQFWnicgCVX0py+N/BnSIK7f39m2mquvjfv+PiPxNRFolWyGurKyM4uJiAIqKiigpKaG0\ntBTY8kcKuhwTljypyhUVFaHKU75yJVx4IaWtWkHXrnY+81CuqKgIVZ6ol8N6PsvLyxkzZgzA5vfL\nbKW7j2E5cE/crqviy6p6T40n1TxGY2AebvB5JTAZGKCqc+LqtFHV1d7vPYFnVLU4ybFsjMEYY7Lk\n930MDwE7pChn9A6tqptE5DLcAj+NgEdUdY6IXOQe1tHAGSJyMVAJbATOyuYfYIwxxl8ZLdRT40ki\nPVR1Sh7ypHvNSHxjKC8v3/z1Lswsp7+ikDMKGcFy+s3vbwyJB++KW+JzALAWOCi7eMYYY6KgtvsY\nitnSGFQCuwEHqeriAmRLzBKJbwwmC9ddB/vv7xbesekvjMkLX9d8FpH3gea4FdzGqeoCEVmkqrvX\nPWr2rGGoZ95/Hw49FLbdFubPhw4dan+OMSZrvt7gBqzGDTa3AXb29tk7cy0SL7MMq0BzLlzoVmUD\nuOqqtI2CnU//RCEjWM4wSNkwqOqpwL64G9BuEZFFQEvvklJjcrN8ORx9NKxYAYcfDjfdFHQiY0yC\njK9KEpHWwJm48YYOqrprPoMleX3rSoo6VTdz6jvvwMEHw+uvww471Po0Y0zufB1jqOWFdstm6m0/\nWMNQT8ybB1dfDU88AS1bBp3GmHrP7zGGlArdKERJVPodA8vZpQu8/HLGjYKdT/9EISNYzjCwNZ+N\nMcZUk1NXUhCsKymCqqqgkX32MCZIBelKEpFLROQsEcn4rmnTAFVWuktSR4yova4xJlRy+TgnQG/g\nOZ+z1AtR6XfMa86qKrjgAnj+eRg+HFatyvlQdj79E4WMYDnDIOtP/ap6fz6CmHrk97+HJ5+EZs3g\nP/+Btm2DTmSMyUKtYwwichBwGPAz3LTYs4DXVXVN/uNVy2FjDFHw4IPw29+6ldheeQX69Ak6kTEN\nmt9zJZ0PXA4swt39/DmwHbAn0AvXQNykqkvrEjrjoNYwhN9337nLUZcvh4cfhl/9KuhExjR4fg8+\nNwV6qWp/Vf2jqj6sqvep6u9U9UDgL8AedQlcH0Wl3zEvOZs2hUmT4L77fGsUGvT59FkUMoLlDIN0\ncyXdr6ob0zxeoapv1vYCItJXROaKyHwRGZKmXg8RqRSR02uPbUKrQwe49NKgUxhj6iCTMYY9gQeA\nNqq6j4jsB5ysqrfXenCRRsB83JrPK4ApwNmqOjdJvddxYxiPqmqNK56sK8kYY7KXr/sYHgKuxy3U\ng6rOBM7O8Pg9gQWqukRVK3FrO5ySpN7lwLO4cQwTFVVVQScwxuRBJg1DU1WdnLDvpwyP3w5YFlde\n7u3bTER+Bpyqqg/g7pGItKj0O9Y5pyqcf767NDWP3+QazPksgChkBMsZBpncx/CliHTCW6RHRM4A\nVvqYYSQQP/YQ+cahQbjlFnj8cXevwgUXQKdOQScyxvgkk4bhUmA0sJeIfIa7fHVQhsf/DIhfnqu9\nty/eQcA4ERFgJ6CfiFSq6vjEg5WVlVFcXAxAUVERJSUllJaWAltabytnVo7ty+n5Y8dSfuutIELp\nM89Ap06B/3uCLsf2hSVPqnJ81jDkSVYuLS0NVZ505Ziw5ImduzFjxgBsfr/MVjYL9TQDGqnquowP\nLtIYmIcbfF4JTAYGqOqcFPUfA16ywecQe/ddtwJbZSX87W9w8cVBJzLGpJGXwWcRuVlEbgauBgbH\nlWulqpuAy4DXgNnAOFWdIyIXiciFyZ6SRfZQSvwkEVY55VSFwYNdo3DllQVpFOr1+SywKGQEyxkG\nmXQlbYj7fTvgRCDpJ/5kVHUC0CVh34Mp6l6Q6XFNAETcNBcjR8LttV6tbIyJqKzXYxCRbYFXVbU0\nL4lSv651JRljTJYKtbRnU9wgsjHGmHookzGGj0VkprfNxg0mj8x/tGiKSr+j5fRXFHJGISNYzjDI\nZIzhxLjffwJWq2qmN7iZKHv5ZXjzTbjrLjeNtjGmQUg37XardE9U1a/zkigFG2MosJkzoVcvWL/e\nLbozcGDQiYwxOchljCHdN4apuMtHkx1QgY7ZvJCJkFWr4KSTXKMwYACcc07QiYwxBZRu2u3dVbWj\n9zNxs0Yhhaj0O6bM+d13cMopsHQp/Pzn8Oij7jLVgET+fIZIFDKC5QyDjNZ89tZI6I37pvCuqr6Q\n11QmODfeCJMnQ3ExPP88bLdd0ImMMQWWyXoMfwM6A097u84C/qeqBV2NxcYYCmTNGrf62h13wN57\nB53GGFNHvq75HHfQucDesXdlb1Gd2apa0HcNaxiMMSZ7+brB7VOqz5C6q7fPJBGVfkfL6a8o5IxC\nRrCcYZByjEFEXsKNKewAzBGRyV75YNwsqcYYY+qhdPcxHJHuiar6dl4SpWBdSXmweDHcey+MGAHb\nbBN0GmNMHvh9H8M7tb0Ti71bR9fatXD88TBnDjRpAn/8Y9CJjDEhkW6MYaKIXC4i8eMLiMg2InKU\niIwFzstvvOiJRL/jjz9SfuSRrlHo1g2uuy7oRClF4nwSjZxRyAiWMwzSfWPoC1wAPC0iuwNrgSa4\nxuQ1YKSqTs9/ROMrVbjwQqiogLZt3foKRUVBpzLGhEhG6zGIyNa49Zg3qurarF5ApC9uNtZGwCOq\nOiLh8ZOB24AqoBIYrKqTkhzHeq38cP/9cNll0LQpvPMOHHhg0ImMMXmUl/sY6sK752E+bs3nFcAU\n4GxVnRtXp6mqfuf9vi/wTLJ7JKxh8MmGDXD22XDJJdCvX9BpjDF5VqiFerLRE1igqktUtRIYB5wS\nXyHWKHi2x31ziKzQ9zs2awbjx1PepEnQSTIS+vPpiULOKGQEyxkG+W4Y2gHL4srLvX3ViMipIjIH\neAk3rmHyKcBJ8Ywx4ZfRJHr55k3K94KI9AZuB45NVq+srIzi4mIAioqKKCkpobS0FNjSels5s3Js\nX1jyRL0c2xeWPKnK8VnDkCdZubS0NFR50pVjwpIndu7GjBkDsPn9Mls5jTGIyGhVvTCDeocAt6hq\nX688FNDEAeiE5/wP6JG4EJCNMeRg0iQYNQrGjHH3KhhjGpxCjjE8mGG9KUBnEdlNRLYBzgbGx1cQ\nkU5xvx8AbFPo1eH8lPhJIjAzZsAJJ8Azz7grkRKEJmctLKd/opARLGcYZNyVJCLNcZ/216nq1Eye\no6qbROQy3H0PsctV54jIRd6xRgP9ReRc4EdgI3Bm1v8KU92CBdCnD3zzDfTvD4MHB53IGBMhmUy7\n3QN4FDeZnuBudLsg08bBL9aVlKFly+Cww2DJEjjmGHj5Zdh226BTGWMC4vdcSTGPAJeo6rvei/QG\nHgP2yz6iybvhw12jcPDBbgU2axSMMVnKZIxhU6xRAFDV94Cf8hcp2gLvd7z7brjqKvjPf2D77VNW\nCzxnhiynf6KQESxnGGTyjeFtEXkQt7Sn4pb2LPcGilHVaXnMZ7K13XaucTDGmBxlMsYwMc3DqqpH\n+RspZQ4bYzDGmCyFbq4kP1nDkMSbb7qxhDRdRsaYhi0v9zGIyBUi0lych0Vkmogcl3vM+q1g/Y6P\nP+4uST3lFPjxx6yfHpX+UcvpnyhkBMsZBpkMPl+gqt8CxwE7Ar8Ehuc1lUnvr3+F886DTZugVy/Y\neuugExlj6pFMxhhmqup+InIvUK6qz4vIdFXtXpiIm3NYV1JVFfzhD3Drra58111w9dXBZjLGhFpe\nxhhE5DHcjKi7A/sDjXENREFXeLGGATfn0fnnu9lRR4+GX/866ETGmJDL11xJvwKG4ia2+w7YBjg/\nh3wNQl77HQcNclNcvPxynRuFqPSPWk7/RCEjWM4wqPU+BlWtAqbFlb8CvspnKJPCVlvBs88GncIY\nU8/Z5arGGFOPhXFpT5OLtWtdV9Hq1UEnMcY0QCkbBm+abUSkVbKtcBGjpc79jrNnQ8+e8Mgj8Nvf\n+pIpmaj0j1pO/0QhI1jOMEg3xvAP4ERgKm6OpPivIgp0zGOuhmncOPdNYcMGKCmBe+4JOpExpgGy\nMYYwUIUrrnA3rgGccw489BA0bRpsLmNM5OVtjEFE2onIoSJyeGzLIlRfEZkrIvNFZEiSx88RkRne\n9p6I7JvNP6BeEIEWLdwdzPffD08+aY2CMSYwmcyVNAKYBNwIXOtt12RycBFpBNwH9AG6AQNEZK+E\naguBw1V1f+B24KGM04dQzv2Ot9wC06fDJZe4hiLPotI/ajn9E4WMYDnDIJP1GE4FuqjqDzkcvyew\nQFWXAIj2KbwzAAAT4klEQVTIOOAUYG6sgqp+EFf/A9xd1g1P48bQrVvQKYwxJqMpMf4D/EJV12d9\ncJH+QB9VvdArDwJ6qurvUtS/BtgzVj/hsfoxxvD557B0KRx0UNBJjDENgK9rPovIX3FXH30HVIjI\nm8Dmbw2p3txzJSJH4qba6J2qTllZGcXFxQAUFRVRUlJCaWkpsOVrXajLM2ZQeuedsGkT5fffDzvu\nGK58VraylSNfLi8vZ8yYMQCb3y+zpqpJN+C8dFuq5yUc4xBgQlx5KDAkSb39gAVApzTH0iiYOHFi\nzZ2VlarDhqk2aqQKqr17q65YUeho1STNGUKW0z9RyKhqOf3mvXfW+n4dv6X8xqCqY0WkBOgMzFbV\nOTm0O1OAziKyG7ASOBsYEF9BRDoA/wZ+qar/y+E1wm3ZMhg4EN591w0q33CDG2i2NRSMMSGVcoxB\nRG7CLcozFTgY+JOqZn3FkIj0Be7FXQH1iKoOF5GLcK3YaBF5CDgdWIK7ia5SVXsmOY6myhpqb70F\nxxwDbdvCE0/A0UcHncgY04D4uh6DiMzGm2pbRHbEdQn18CFnTiLbMAA89RQcdxzsvHPQSYwxDYzf\nN7j9oG79hdhU2zbhXgZig0DVDBwYukYhac4Qspz+iUJGsJxhkO4+ho4iMt77XYBOcWVU9eS8Josa\nVZg6NegUxhhTZ+m6ko5I90RVfTsviVIIdVfS2rVw4YVuEZ033oCjjgo6kTHGAD7fx1DoN/7I+uAD\nGDAAFi+GHXaAb74JOpExxtSJjRvkqqoKRoyAww5zjcJBB8H06ZS3bBl0soxEpX/UcvonChnBcoaB\nNQy5+vprt17CTz/B1VfDpEnQqVPQqYwxps6yXo9BRLYDTlLVf+UnUsrXDd8Yw5tvwg8/wPHHB53E\nGGOS8vU+hoQDN8ZNnT0AOA54V1XPyClljkLZMBhjTMj5vlCPiBwhIg8Ci4FfAccCuxe6UQjc6tWw\naVNGVaPS72g5/RWFnFHICJYzDFI2DCKyHPgT8B7QVVX7AxtjN701GG+8AfvuC7ffHnQSY4wpiHT3\nMYzELdIzC/gH8CLwsap2LFy8ankK25VUVeUag1tucTev9ekDr7ziFtQxxpiI8H2MQUQEKMWNLRwP\ntMB1Kf2f5rBwT10UtGH44gsYNAhee83NiHrzzXDTTdYoGGMix/cxBm8674nqVlTbHddAnIIbc6i/\nrrjCNQo77QQTJrhvDRk2ClHpd7Sc/opCzihkBMsZBpms+RzvM2AwsC4PWcLj7rvh++9h1Cho3z7o\nNMYYU1Dpxhj+DvxVVWeLSAvgfWAT0Aq4RlWfLlxMu1zVGGNy4XdX0mGqOtv7/XxgvqruCxwIXJdj\nRmOMMSGXrmH4Me73Y4EXAFR1VTYvICJ9RWSuiMwXkSFJHu8iIv8Vke9F5Kpsjl1nr73mJsDL8B6F\nTESl39Fy+isKOaOQESxnGKRrGNaKyIki0h3oBUwAEJGtgCaZHFxEGgH34e6a7gYMEJG9Eqp9BVwO\n/DnL7LlThT//Gfr1g3Hj4PHHC/bSxhgTdunGGPYERgFtgZGqOsbb3wc4TlWvrvXgIocAw1S1n1ce\nirvYaUSSusOAdap6T4pj+TPGsGED/PrXrkEAdynqsGHQyOYTNMbUP36vxzAf6Jtk/6sisneGx28H\nLIsrLwd6ZhPQV59/7tZenjEDtt8enngCTj01sDjGGBNG2V6uGnMVMNLPIJkoKyujuLgYgKKiIkpK\nSigtLQW29PelLW/aROmOO8Iee1B+ww1QVESpd+yMnp9BObbPr+Plqzxy5Mjsz18A5di+sOSJ8vms\nqKjgyiuvDE2eVOXEv33QeVKVgz6f69dDjx6ltGlT8/yNGTMGYPP7ZdZUNesNWJZhvUOACXHlocCQ\nFHWHAVelOZb64ssvVdes8edYSUycODFvx/aT5fRXFHJGIaOq5UznhRdUzzlHdc89VUH10ktrf473\n3pnVe3zW6zEAiMhSVe2QQb3GwDzgaGAlMBkYoKpzktQdBqxX1btTHEtzyWqMMVGyYQNs3OgmXkh0\n881w223u9222gYED4dFH0x/P17mSRGQdoEDsgLGKAjRR1Yy6oUSkL3Av7gqoR1R1uIhchGvFRotI\nG+AjYAegCliPm811fcJxsmsYvv3Wra7WqlXmzzHGmALasAGmTYOpU7f8nDsXLr4Y7ruvZv2KCpg8\n2a0kvM8+rnGoTS4NQ05dSUFsZNOVtGiR6j77qB5xhOoPP2T+PB/Y12B/WU7/RCGjasPK+fTTrkso\nfttqK9Vzz617vhhy6EpK+anfW8Lzt0BnYCbwqKr+lFWrE4T333dXGn3+Oey1F3z5JfzsZ0GnMsY0\nEGvWwKxZ8PHHbps503VcvPRSzboHHQQHHOC2Aw902777wnbbFT53vHRdSf8EKoF3gX7AElW9ooDZ\nEvNoqqyb/eMfcMEFbh3mY4+FZ56BoqLCBDTGNHhz58LeSS7mb9kSvvrKzeJfaH6PMXysbm6k2N3O\nk1X1gLrHzE2tDcOrr0Jf77aLSy6Be++FrXK9GtcYY5yNG90b/iefwOzZbluxwvX1J77RV1ZC69bQ\nubMbA9hvP7ftu6/bHwRfxxiAaenKhd6obYzhp59UTz1VddSojPrd8qUh9Y8WguX0TxQyqoYrZ2Wl\n6nbb1RwHANV//3ti0udUVRU2Y23wc4wB2F9Evo01OkATryzeCzXPsuHKr8aN4bnngvmuZoyJDFX3\niT9+DODjj12nQ5s21etutZUbqvz+e+jWDbp2dT+7dYPVq5Mfvz68BeV0H0MQ7D4GY4wfevWC//63\n5v7XX4djjqm5v6oq2lOp+TpXUqgtWQJNmgTXaWeMCY1Fi1y//8KF8L//uZ/z58MDD8BRR9Ws3769\nGwzed98t/f+xsYBkotwo5CzbvqegNmJjDDNmqO6yi2qPHqrr1tW5/81vYeofTcdy+isKOaOQUbV6\nzqoq1VWrVP/7X9Vly5LXHzgw+RhAquHGjRv9GQeIyvnE5zGG8HnnHTj5ZPjmG+jSxdcFdowx4fDW\nWzBypPvkv3ChuzsY4C9/AW/OumoOPhi++AI6doROnbb83HPP5McP+h6BKIjWGMO227p7FM44w02Z\nbX9hY0Lvu+9cF8+iRe6NfvFi1xt88slw/vk1699xB9x445Zyy5buzf6SS9xtSiY79X+M4Ycf3H8d\no0a5q5CMMYGrqoLly10Hzm671Xz8b3+Da6+tub916+QNw2mnuU/7HTu6rWVL/zOb9KI1rHLrrW5m\nqRA3CvFzyYeZ5fRXFHL6lXHGDBg82H3i79oVmjZ1DcLNNyevv+eerue3b1/3ue7OO92kBIMHJ6//\n+efl/OIXbnqIMDcKUfib5ypa3xhuuinoBMbUexs3uqt8vv8eeveu+fjy5W4MIF7bttCsWfLjnXyy\n20x0RGuMISJZjYmSVavg4YfdTV4zZsCCBa57qEcPN+1DohUr4Mkn3QBv587u5/bbFz63yYyvcyWF\njTUMxuRG1U02/NlnbhbPRIsWub78mMaN3d2+hxziGgwTbbk0DHkfYxCRviIyV0Tmi8iQFHVGicgC\nEakQkZJ8Z8qnqPQ7Wk5/hSnnDz+4Vb5++Uvo2dP107dtC716lVNVVbP+brvBddfBY4+5xWLWr3fT\nRgfVKITpXKYTlZy5yGvDICKNgPuAPkA3YICI7JVQpx/QSVX3AC4C/p7PTPlWUVERdISMWE5/5Tvn\nN9+42T1ffx3GjHGXdF5yiWsEEm2zDYwY4bp7pkxxzy0qgtatK1izpmb9Ro1c/bIy6N49+KvA7W8e\nvHwPPvcEFqjqEgARGQecAsyNq3MK8DiAqn4oIi1EpI2qppiiKtzWrl0bdISMWE5/ZZvziy/cGlJf\nf+22r75y3T0XXww77FCz/t57w8qVNfdfdx0UF1ffJ+K+MTRv7q4G6tLFrR/8hz+sZccds4oZiPr6\nN4+SfDcM7YBlceXluMYiXZ3PvH01GoapU2u+wF57Jb8aYs4cd2NNoev/+GPNfUHmSVX/iy8a1vmc\nNcvdQZs4ccL++ycfOH3/ffdJu6rK3WD/449u69s3+SWUH34I11zjruj5/nv3c+NGd/VOsmv7e/d2\n8/kkOvFEdwlooo4dXYPRrp1bkLBdO7cla0Qg9aWgxmQk2zk0stmA/sDouPIgYFRCnZeAQ+PKbwAH\nJDlW0vlQPvoo+fwgBxyQfP6UfNc/4YTzQpUnVf1Wrc4LVZ58n8/u3f2pP3Vq8votWyY/n1OmJK/f\nr59qly6qhxyievzxqoMGqV59terSpcnr++G8887L38F9ZDn9RQ5zJeX1qiQROQS4RVX7euWhXsgR\ncXX+DkxU1X965bnAEZrQlSQidkmSMcbkQEM2JcYUoLOI7AasBM4GBiTUGQ9cCvzTa0jWJjYKkP0/\nzBhjTG7y2jCo6iYRuQx4DXcF1COqOkdELnIP62hV/T8ROV5EPgU2AElmTzHGGFMokbnBzRhjTGFE\nYhK9TG6SCwMRWSwiM0RkuogkmUwgGCLyiIisFpGZcftaishrIjJPRF4VkRZBZvQyJcs5TESWi8g0\nb+sbcMb2IvKWiMwWkY9F5Hfe/lCdzyQ5L/f2h+18bisiH3r/z3wsIsO8/WE7n6lyhup8epkaeVnG\ne+Wsz2XovzF4N8nNB44GVuDGLc5W1blpnxgAEVkIHKiqSW4jCo6I9AbWA4+r6n7evhHAV6p6p9fY\ntlTVoSHMOQxYp6r3BJktRkTaAm1VtUJEtgem4u7FOZ8Qnc80Oc8iROcTQESaqup3ItIYmAT8DndF\nY2jOZ5qc/Qjf+RwMHAg0V9WTc/l/PQrfGDbfJKeqlUDsJrkwEkJ4TlX1PSCxsToFGOv9PhY4taCh\nkkiRE9x5DQVVXaWqFd7v64E5QHtCdj5T5GznPRya8wmgqrE7UrbFjXsqITufkDInhOh8ikh74Hgg\nfkKTrM9l6N7Ekkh2k1y7FHWDpsDrIjJFRH4TdJhatI5d/aWqq4DWAedJ5zJvHq2Hg+5SiCcixUAJ\n8AHQJqznMy7nh96uUJ1Pr+tjOrAKeF1VpxDC85kiJ4TrfP4FuJYtjRbkcC6j0DBESS9VPQDXYl/q\ndY1ERVj7FP8GdFTVEtz/kKH4yu51zzwLXOF9Ik88f6E4n0lyhu58qmqVqnbHffPqKSLdCOH5TJKz\nKyE6nyJyArDa+6aY7ltMrecyCg3DZ0CHuHJ7b1/oqOpK7+cXwPPUnP4jTFaLSBvY3B/9ecB5klLV\nL3TLQNhDQI8g8wCIyFa4N9snVPVFb3fozmeynGE8nzGq+i1QDvQlhOczJj5nyM5nL+Bkb6zzaeAo\nEXkCWJXtuYxCw7D5JjkR2QZ3k9z4gDPVICJNvU9niEgz4DhgVrCpqhGqf4oYD5R5v58HvJj4hIBU\ny+n9hxxzOuE4p48Cn6jqvXH7wng+a+QM2/kUkZ1i3S8i0gQ4FjceEqrzmSLn3DCdT1W9QVU7qGpH\n3PvkW6r6S9y0Q2VetYzOZeivSgJ3uSpwL1tukhsecKQaRGR33LcExQ1MPRWWnCLyD6AU2BE3OeEw\n4AXgX8CuwBLgTFUNdLrIFDmPxPWPVwGLgYuS3RlfKCLSC3gH+Bj3t1bgBmAy8AwhOZ9pcp5DuM7n\nvrgB0Ube9k9VvUNEWhGu85kq5+OE6HzGiMgRwNXeVUlZn8tINAzGGGMKJwpdScYYYwrIGgZjjDHV\nWMNgjDGmGmsYjDHGVGMNgzHGmGqsYTDGGFONNQwmMkRknfdzNxFJXAmwrse+PqH8np/HL6R8nB/T\nsFjDYKIkdtPN7rgbtTLmTZWczg3VXkg1SvNcJcr6/BgTzxoGE0V/Anp7i5Fc4c16eae3kEpFbGZb\nETlCRN4RkReB2d6+573Zbz8WkV97+/4ENPGO94S3b13sxUTkz179GSJyZtyxJ4rIv0RkTux53mPD\nRWSWl+XOxPAi0kxEHhWRmV6d07z9A7x9M0VkeFz9+Cz9ReQx7/fHROReEZkkIp+KyOnJzo8vZ9w0\nLKpqm22R2IBvvZ9HAOPj9v8GuMH7fRvc/Fq7efXWAR3i6hZ5P7fDTRfRMv7YSV6rP/Cq93tr3JQC\nbbxjrwF2wc3t9F/gUKAVMDfuOM2T/DuGA/fElVt4x1niPb8R8CZwcmI2L8+j3u+P4aZmANgbt25J\njfNjm23ZbvaNwdQHxwHnenPlf4h7c93De2yyqi6Nq3uliFTg1lBoH1cvlV64mSpR1c9xs2rGZtCc\nrKorVVWBCqAY+AbY6M3NfxqwMckxjwHujxVU9RvvmBNV9WtVrQKeAg73qqSbQvkF7xhzCMGaBaZ+\nsIbB1AcCXK6q3b2tk6q+4T22YXMlN7HYUcDB6ubPr8B9c4gdI9PXivkh7vdNwFaqugk33fqzwInA\nhCTHSDVBWaoM8fW3S3gsPkNoVhIz0WYNg4mS2BvfOmCHuP2vApd46w8gInuISNMkz28BrFHVH0Rk\nL+CQuMd+jD0/4bXeBc7yxjF2Bg7DzaSaPKB73SJVnQBcBeyXpNrrwKVxzynyjnm4iLTyBsoH4L6d\ngJtPv4u49c9PS/XapD4/xmTFGgYTJbFPzjOBKhGZLiJXqOpDwCfANBH5GPg7kOwqpAnA1iIyG/gj\n8H7cY6OBmXGDyAqgqs97rzcDeAO41utSSpWtOfCyiMzATXs9OEndO4BW3oD2dKBU3ZKLQ3GNwXRg\niqq+7NW/HngFeA9YkeQ1E8vVzk+S1zcmLZt22xhjTDX2jcEYY0w11jAYY4ypxhoGY4wx1VjDYIwx\nphprGIwxxlRjDYMxxphqrGEwxhhTjTUMxhhjqvl/O5RW4uIE7LAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xf003f28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEPCAYAAACHuClZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8HuP9//HXO7Ek1oi1giSNSGqJYylRVLRCpCVKG0IR\nlFQtRWtJ0KjWEopIU5IQgtZWRexCJRRFSI4ECfEjWiqxfJEFkZx8fn/MHO7cOfd97mXmnplzPs/H\n436ce+a+Zu53rta5zlzXzHXJzHDOOeeq0SbpAM4557LPGxPnnHNV88bEOedc1bwxcc45VzVvTJxz\nzlXNGxPnnHNVi70xkdRP0mxJb0g6u0CZUZLmSKqXVBfu20zSE5JelTRT0qk55YdLelfStPDVL+5/\nh3POucJWifPkktoAo4EfAv8DpkqaaGazc8rsD3Qzs+6SdgXGAL2BZcAZZlYvaS3gJUmTco690syu\njDO/c8650sR9ZbILMMfM3jGzpcDtwIC8MgOAmwHM7HlgXUkbm9k8M6sP9y8CZgGdco5TzNmdc86V\nKO7GpBPw35ztd1mxQWiqzHv5ZSR1AeqA53N2nxx2i10vad2oAjvnnCtf6gfgwy6uu4Bfh1coANcA\n3zazOmAe4N1dzjmXoFjHTAiuMrbI2d4s3JdfZvOmykhahaAhucXMJjYWMLMPc8pfB9zf1JdL8onH\nnHOuAmZW1lBC3FcmU4EtJXWWtBpwGHBfXpn7gKMAJPUGPjWz+eFnNwCvmdnVuQdI2iRn82DglUIB\nzCz1r+HDhyeewXN6Rs/pORtflYj1ysTMGiSdDEwiaLjGm9ksSUOCj22cmT0kqb+kN4HFwGAASbsD\nRwAzJU0HDBhmZo8Al4W3EC8H5gJD4vx3xG3u3LlJRyiJ54xOFjKC54xaVnJWIu5uLsJf/j3y9o3N\n2z65ieOeAdoWOOdRUWZ0zjlXnWYbE0kbAbsDmwJfEHQpvWhmy2PO1moMHjw46Qgl8ZzRyUJG8JxR\ny0rOSqhQ/5ikvYFzgI7AdOADoB2wFdCNYGD8CjNbUJuo5ZNklfb/OedcayUJi3AAvj9wvJl918xO\nMLPzzOy3ZnYgsD1BA9O3irwuNGXKlKQjlMRzRicLGcFzRi0rOStRsJvLzM4s8tky4N5YEjnnnMuc\nYt1cZxQ70DIwL5Z3cznnXPkq6eYqNgC/dvizB/Bdvnk+5ADghfLjOeeca6kKjpmY2e/N7PcET6Tv\naGa/MbPfADux4lPtrkpZ6Uf1nNHJQkbwnFHLSs5KlPIE/MbAVznbX4X7nHPOOaDImMnXBaRzgYHA\nPeGug4A7zezimLNVzcdMnHMuNHcunH8+XHghdO1atGjUYyYAmNlFkh4G9gx3HWNm08v5Eueccwn5\n7DO45BIYORKWLAEz+OtfI/+aUid6XANYYMGEi+9KKt6subJkpR/Vc0YnCxnBc0at5jn/9jfYcksY\nMSJoSI44Ai6Op1OplOlUhgM7E9zVdSOwKvBXgilWnHPOpdWyZfDRR7DHHnDFFbDLLrF9VSljJvXA\nDsA0M9sh3DfDzHrFlioiPmbinGvVli+HSZNgv/1ApQ+BRD2dSqOvwt/IFn7JmuV8gXPOuZgtXx6M\nheRr0wb69SurIalUKY3JnZLGAh0kHQ88TrC6oYuI9/dGKws5s5ARPGfUYsk5bx7svz9ce2305y5D\nKXdz/UlSX2ABwbjJ78zssdiTOeecK+7hh2HwYPjgA3jtNTj2WGjXLpEoRcdMJLUFHjezvWsXKTo+\nZuKca5GWLIFhw+DKcIrEvfeGW26BTp0iOX3kYyZm1gAsl7RuVcmcc85F55e/DBqStm2DW30feyyy\nhqRSpYyZLCJYh328pFGNr7iDtSatur83BlnImYWM4DmjFlnOoUNhu+3g6aeD922bXOG8pkpZA/7u\n8OWccy4NttoK6uuDu7VSotnnTLLMx0ycc5m2dCksXAgdO9b0ayMdM5F0v6QDJK3axGfflnShpGMr\nCeqcc64Zzz4LO+4Ixx2XdJKSFLtGOp5gcsfZkqZKekjSE5LeBsYCL5nZDTVJ2cK1uv7emGUhZxYy\ngueMWkk5P/kEhgyB3XeHV16BGTPg449jz1atYmvAzwPOAs6S1AX4FvAF8IaZfV6TdM4515rccQec\nemrw3Miqq8JZZ8G550L79kkna5aPmTjnXFqcdx5cdBF8//swZgx85zuJxKhkzMQbE+ecS4svvoB7\n7oFBg2oyn1YhcU306GLWovp7UyALObOQETxn1JrN2b49HH54og1Jpbwxcc65Wlu8GGbPTjpFpCrq\n5pJ0gZldEH2caHk3l3MudV5/HQ45BBYsgGnTYIMNkk60klp2c71U4XHOOdd63Xkn7LwzvPoqrLFG\ncBtwC1FRY2Jm90cdpDVrMf29KZGFnFnICJ4zMosXwymnMOXQQ2HRIjj0UJg6Fbp3TzpZZJptTMKn\n3e+X9JGkDyRNlPTtWoRzzrkWob4eRo8OJmT885/htttg7bWTThWpUtaAfw74C3BbuOsw4BQz2zXm\nbFXzMRPnXGpcein07Qs77ZR0kmbF8pyJpBlm1itv38tmtn0FGWvKGxPnnCtfXAPwD0s6R1IXSZ0l\nnQU8JKmjpNpOZdlCpb6/N+Q5o5OFjOA5y/bSS3D11QU/Tk3OGJSynsnA8OeQvP2HAQb4+IlzrnVb\ntCiYQ2v06GB7jz0y0Z0VpdinU5HUDxhJcBU03sxGNFFmFLA/sBgYbGb1kjYDbgY2BpYD15nZqLD8\nesAdQGdgLjDQzD5r4rzezeWci9e0aXDYYTBnTjDAftppcMEFsNZaSSerWCzdXJJ+Jmnt8P15ku6W\ntEOJgdoAo4H9gG2AQZJ65pXZH+hmZt0Jrn7GhB8tA84ws22A3YCTco49B3jczHoATwBDS8njnHOR\nuvde6N07aEi22w5efBH+9KdMNySVKmXM5HwzWyhpD2AfYDzf/MJvzi7AHDN7x8yWArcDA/LKDCC4\nAsHMngfWlbSxmc0zs/pw/yJgFtAp55ibwvc3AQeVmCeVstKP6jmjk4WM4DmbtdtuwSqIJ50Ezz8P\ndXVFi2elPitRyphJQ/jzR8A4M3tQ0h9LPH8n4L852+8SNDDFyrwX7pvfuCNcT6UOeC7ctZGZzYdg\n3RVJG5WYxznnorPxxsHT7Ouvn3SSxJXSmLwnaSzQFxghaXVqOEGkpLWAu4Bfm9niAsUKDowMHjyY\nLl26ANChQwfq6uro06cP8M1fCb5d2nbjvrTkyfJ2nz59UpWn2HajtOTx+ox+e8qUKUyYMAHg69+X\n5SrlOZM1gH7ATDObI+lbwHZmNqnZk0u9gQvMrF+4fQ5guYPwksYAk83sjnB7NrCXmc2XtArwAPCw\nmV2dc8wsoE9YZpPw+JVWkfEBeOdcJGbNgr/8Jbjtt23bpNPELpYBeDP73MzuNrM54fb7pTQkoanA\nluHzKasR3E58X16Z+4Cj4OvG59PGLizgBuC13IYk55jB4fujgYkl5kml/L9Y0spzRicLGcFz8u9/\nBzP8brPNN41JFbJSn5UopZurYmbWIOlkYBLf3Bo8S9KQ4GMbZ2YPSeov6U3CW4MBJO0OHAHMlDSd\noCtrmJk9AowA7pR0LPAO3zwL45xz1fvXv2DoUHjmmWB7tdVg8GA49thEY6WZL9vrnHP5Jk6Egw6C\nDh3gxBPhlFPgW99KOlXN+Brwebwxcc5VZPlyuOGG4GHEVvjMSFwPLfaWNFXSIklfSWqQtKDymC5f\nVvpRPWd0spARWnjOhga4+eZgKpR8bdrAL34ReUOSlfqsRCm3+I4GBgFzgPbALwimpHfOuWz65z+D\nubOOPjp4Yt1VrZRbg180s51zp6KXNN3MSppSJUnezeWcW8GsWXDmmfDgg8H25psHjclAv4cnVyXd\nXKXczfV5eFtvvaTLgPep4UOLzjkXicb5sxoaglUOhw4NJmVs3z7pZC1CKY3CkWG5kwlu3d0cODjO\nUK1NVvpRPWd0spARWljO7t3hgANgyJCgYRk6tOYNSVbqsxKlXJkcFD40+CXwewBJvwaqe3rHOedq\n7a67WsUT7EkoZcxkmpntmLfPx0ycc+m1bBmsEusz2S1apLcGSxok6X6gq6T7cl5TgP+rMqtzzsVj\nyhTYait45ZWkk7QqxcZMngWuAGaHPxtfZxAsduUikpV+VM8ZnSxkhIzlNIOrroJ99oG33656Hq04\nZKU+K1HwOtDM3iGY92o3SZ2B7mb2uKT2BM+bLKxRRuecK+6LL2DQILjjjmD7rLPgoouSzdTKlDJm\ncjxwAtDRzLpJ6g6MMbMf1iJgNXzMxLlWYPnyYMXDF14InlifMCGY6ddVLK7nTE4iWB3xeYBwTRNf\n2dA5lw5t2gTL5n72GdxzD3xnpaWNXA2U8pzJEjP7qnEjXLDK/9yPUFb6UT1ndLKQETKUc4stoL4+\n9Q1JVuqzEqU0Jk9KGga0l9QX+Dtwf7yxnHOuTO3aJZ2gVStlzKQNcBywLyDgUeD6LAxG+JiJcy3M\n55/DtGmwxx5JJ2nRfD2TPN6YONeCvP8+HHhg8PzIk0/CLrsknajFims9k90lPSbpDUlvSXpb0luV\nx3T5stKP6jmjk4WMkKKcL78cNB4vvgibbhpM1JgjNTmbkZWclSjlbq7xwOnAS0BDvHGccy7PAw8E\nKx4uXgy77x7csbXhhkmncnlKGTN53sx2rVGeSHk3l3MZ9/HH0LUrLFwIRxwB11/vA+01EOmYiaTG\nyR0HAm2Bu4EljZ+b2bQKc9aMNybOtQD33ht0c/3ud6Cyfr+5CkU9ZtI4F9euwM7AxTn7fJ3LCGWl\nH9VzRicLGSElOQ86CIYPL9qQpCJnCbKSsxLFxkxGm9k/apbEOedcZhXr5lppHZOs8W4u5zLkueeC\n2X4HDUo6SasX19xczjkXrzvugKOPDiZt3Gor2GmnpBO5MhUbM+kpaUYTr5mSZtQsYSuQlX5Uzxmd\nLGSEGuQ0C6aKP+wwWLIEjjkGevUq+zRen8krdmXyNnBArYI451qZ5cvhxBNh3LhgcP1Pf4LTT/c7\ntjKq2JhJJtZ5L8bHTJxLsTfeCLqzli2D22+HAQOSTuRCUY+ZPFNlHuecK2yrrWDSJFi0CPr2TTqN\nq1LBMRMzO7mWQVqzrPSjes7oZCEj1CDnbrtF0pB4fSavlPVMnHPOuaJ8CnrnXPzMgilR6uqSTuJK\nEOmYiaSDix1oZneX80XOuVbKDM4/Hy65BG66CX7+86QTuRgU6+a6CzgP+HH4OiDn9eP4o7UeWelH\n9ZzRyUJGiCjnhRcGz5JIsc3426rqM6WK3c11MHAY0AuYCNxmZm/WJJVzrmW46CK44AJo0wb+9jf4\n6U+TTuRiUsp6JmsCA4BDgfWBc83syZK/QOoHjCS4ChpvZiOaKDMK2B9YDBxjZtPD/eMJroLmm1mv\nnPLDgeOBD8Jdw8zskSbO62MmziVlzJjgoUQJbrklWI/EZUIsy/YCXwKfAQuAtYCSr1MltQFGA/sB\n2wCDJPXMK7M/0M3MugNDgGtzPr4xPLYpV5rZjuFrpYbEOZew7beHjh3huuu8IWkFCjYmkn4gaRzB\ncr17A1ebWZ2ZPVrG+XcB5pjZO2a2FLid4Con1wDgZgAzex5YV9LG4fbTwCeFIpaRI9Wy0o/qOaOT\nhYxQZc7ddoM5c+C44yLLU0irqM+UK3Zl8jhBY/A0sDpwlKRRja8Sz98J+G/O9rvhvmJl3muiTFNO\nllQv6XpJ65aYxzlXSx07Jp3A1UixubmOLnagmd3U7MmlQ4D9zOyEcPvnwC5mdmpOmfuBS8zs2XD7\nceCsxmWBJXUG7s8bM9kQ+MjMTNIfgW+Z2Up//viYiXPOlS/qubl6mNmwKjO9B2yRs71ZuC+/zObN\nlFmBmX2Ys3kdcH+hsoMHD6ZLly4AdOjQgbq6Ovr06QN8c8np277t21Vuv/8+U8aNg732Skce3y5r\ne8qUKUyYMAHg69+XZTOzJl/AtEKflfoC2gJvAp2B1YB64Dt5ZfoDD4bvewPP5X3eBZiZt2+TnPen\nA7cW+H7LgsmTJycdoSSeMzpZyGhWYs5PPjHbbjszMLv11tgzNaVF1WcKhL87y/p9X+zKpK2k9Sgw\n0G1m/1dCQ9Ug6WRgEt/cGjxL0pAw7Dgze0hSf0lvEt4a3Hi8pFuBPsD6kv4DDDezG4HLJNUBy4G5\nBHeBOedq7Ysv4MADYeZM6NHDZ/9txYqNmSwh6G5qqjExM/t2nMGi4GMmzsVo2bLgIcSJE6FTJ3jm\nGejcOelULgJRj5m8ZhlfHMs5F6Mzzggakg4d4JFHvCFp5XwK+hRoHAhLO88ZnSxkhGZy/uIX0LUr\nPPggbLttzTI1pUXUZ8YVuzK5utAHkrYws//EkMc5lxW9esHrr8OqqyadxKVA0bm5JO1G8ADhU2b2\ngaRewDnAnma2ecEDU8LHTJxzrnyRzs0l6TLgBuAQ4MHw4cBJwPNA92qCOueca1mKjZn8GNjBzAYB\n+wKnAb3N7Goz+7Im6VqJrPSjes7oZCEj5OS8/HIYPTrRLMVkrj5boGJjJl82Nhpm9omkOWY2tzax\nnHOp8ec/w1lnBVPJ/+AHsPXWSSdyKVTsOZNPgadydn0/d9vMDow3WvV8zMS5Kl13HZxwQvB+7Nhv\n3rsWrZIxk2KNyV7FDrQyFshKijcmzlXhllvg6KODNdyvvhpOPbX5Y1yLEOkAvJk92dQLeItganoX\nkaz0o3rO6KQ+4+LFMHQoU8zg0ktT35Ckvj5DWclZiZIeWpS0oaRfSfoXMAXYONZUzrlkrbkmTJoE\nJ50EZ5+ddBqXAcW6udYGDgYOB7YC7gYONbPNahevOt7N5Zxz5Yt6zOQL4AXgPOBpMzNJb2VhgsdG\n3pg451z5Ih0zAYYSLNd7DTBUUrdqwrnCstKP6jmjk6qMy5fDww83+VGqchbhOZNXbAB+pJn1BgaE\nu+4FNpV0tqStapLOORevhgY47jjo3x9Gjkw6jcuwonNzrVRY2hYYRDB2smVsqSLi3VzOFfHVV3Dk\nkXDnnbDGGsF08vvsk3QqlwKRjpm0BN6YOFfAsmVw8MFw//2wzjrBNPJ77JF0KpcSUY+ZuBrJSj+q\n54xO4hkvvDBoSDp2hH/+s2BDknjOEnnO5Hlj4lxrNHAgbLcd3H037Lxz0mlcC+DdXM61Vg0N0LZt\n0ilcCtWkm0vSTZKuDQfjnXNZ5Q2Ji1Al3VyjgceBIyPO0mplpR/Vc0YnCxnBc0YtKzkrUWw9kyaZ\n2VRgKvCP6OM452LxySew3npJp3AtWHNrwG8GHAbsCWwKfAG8AjwIPGxmy2sRslI+ZuIcsHAh7Lgj\nfO97cM01wSSOzhVRyZhJwSsTSTcCnYAHgBHAB0A7gkkf+wHnSjrHzJ4qdA7nXAr86lfw5ptBI+Lj\nJC4mxcZMrjCzfc1slJk9a2ZvmtkrZna3mZ0C9AH+V5uYLVtW+lE9Z3RqlvHmm+Gvfw2ecL/9dmjX\nrqzDs1CX4DnToNjcXK80vpfUXlKPvM+/MrM34wznnKvCG28EVyUAo0ZBz57J5nEtWrPPmUg6ELgc\nWM3MukqqAy70NeCdS7nDDoM77oBDD4XbbgOV1QXuWrFIx0xyDCdYpncKgJnVS+pafjznXE2NHw+d\nO8OwYd6QuNiV8pzJUjP7LG+f/7kfoaz0o3rO6NQk45prwogRsO66FZ8iC3UJnjMNSrkyeVXS4UBb\nSd2BU4Fn443lnHMuS0oZM1kDOBfYFxDwKPAHM/sy/njV8TET55wrn69nkscbE9dqmAV3bB13HKy1\nVtJpXMbFMtGjpMmSnsh/VR7T5ctKP6rnjE7kGS+/HE47Dfr1CxqWiGShLsFzpkEpYya/zXnfDjgE\nWBZPHOdc2SZPhqFDg/dnneV3brlEVNTNJekFM9slhjyR8m4u1+K9+24w79aHHwa3AF90UdKJXAsQ\nVzdXx5zXBpL2A0q+11BSP0mzJb0h6ewCZUZJmiOpXtIOOfvHS5ovaUZe+fUkTZL0uqRHJVV+76Nz\nWfXVV/CznwUNyT77BEvxOpeQUp4zeQl4Mfz5b+A3wHGlnFxSG4L1T/YDtgEGSeqZV2Z/oJuZdQeG\nANfmfHxjeGy+c4DHzawH8AQwtJQ8aZWVflTPGZ1IMjY0QI8esPnmcOutsUzimIW6BM+ZBs2OmZhZ\nNU+77wLMMbN3ACTdDgwAZueUGQDcHH7X85LWlbSxmc03s6cldW7ivAOAvcL3NxE8nX9OFTmdy572\n7eHGG4Mrkw03TDqNa+UKjplIOrjYgWZ2d7Mnlw4B9jOzE8LtnwO7mNmpOWXuBy4xs2fD7ceBs8xs\nWrjdGbjfzHrlHPN/Ztax0HbOfh8zcc65MkU9N9cBRT4zoNnGpIa8xXDOuQQVbEzM7JgIzv8esEXO\n9mbhvvwymzdTJt/8xq4wSZsQLNzVpMGDB9OlSxcAOnToQF1dHX369AG+6b9MertxX1ryFNoeOXJk\nKusvi/WZn7Wk4ydPhi+/pM/++9csb319PaeddlrNvq/S7YrqM4HttNbnlClTmDBhAsDXvy/LZmZF\nX8D6wChgGsEg/NXA+s0dFx7bFngT6AysBtQD38kr0x94MHzfG3gu7/MuwMy8fSOAs8P3ZwOXFvh+\ny4LJkycnHaEknjM6FWW89FKznj3NXnst8jyFZKEuzTxn1MLfnc3+js99lTI312PAU8Bfw11HAH3M\nbJ9SGitJ/cIGqA0w3swulTQkDDsuLDOaYCngxcAx9s14ya0EKzquD8wHhpvZjZI6AncSXNG8Aww0\ns0+b+G5r7t/nXCY88QT07QvLl8N998EBxXqhnatOLHNzSXrFzLbN2zfTzLarIGNNeWPiWoTcBxPP\nPRf++MekE7kWLpaHFoFJkg6T1CZ8DSSYOdhFJLe/N808Z3RKzpj7YGLfvvD738eaK18W6hI8ZxoU\nHICXtJDgLikBpwG3hB+1BRax4pxdzrk43H03PPdcrA8mOhcFn4LeubS74QbYdlvYJfXT4bkWItIx\nE0ldzGxukS8T0MnM3i0rZQ15Y+Kcc+WLeszkckn/kHSUpG0kbSRpC0k/kPQH4BngO1UldkB2+lE9\nZ3SykBE8Z9SykrMSxR5a/JmkrQluBT4W+BbwOTALeAi4yDKwdK9zzrn4+ZiJc2lhBr/5DRx1FNTV\nJZ3GtWJx3RrsnKuF886Dq66C/v3hS7/od9nijUkKZKUf1XNGZ6WM48bBxRcHt/6OHw/t2iWSK18W\n6hI8Zxp4Y+Jc0h56CE48MXg/ZgyEEzk6lyU+ZuJckj74ALp1g0WLgm6uP/wh6UTOxTM3V4EvmmZm\nO5Z9YI15Y+Iy4dZbYfLkoKtLZf3361wsajYAn4WGJEuy0o/qOaOzQsbDD4frrktlQ5KFugTPmQbN\nrgEPEC5AtQvBXF1TzWxerKmcc85lSilT0P8C+B3wBMGkj3sBF5rZDfHHq453cznnXPniWs/kdeB7\nZvZxuL0+8KyZ9ag4aY14Y+JS55ZboEcPn7TRpVpcYyYfAwtztheG+1xEstKP6jmr9NxzcNxxsOee\nTLn11qTTlCS1dZnHcyavlDGTN4HnJU0kGDMZAMyQdAaAmV0ZYz7nWoYPPwwWuVq6FE49FTbdNOlE\nzkWqlG6u4cU+N7PaLv1WBu/mcqnQ0AD77Qf//Cd873vBbcCrrZZ0KucKiv05E0ltgLXMbEG54ZLg\njYlLhWHD4JJLYKONYNo06NQp6UTOFRXLmImkWyWtI2lN4BXgNUlnVhrSrSwr/aies0Lf/S6stx7c\nccfXDUnqMhbgOaOVlZyVKGUAfuvwSuQg4GGgK3BkrKmca0l+8hOYOxf69Ek6iXOxKWXM5FWgDrgV\nGG1mT0p62cy2r0XAang3l3POlS+uW4PHAnOBNYGnJHUGMjFm4pxzrjaabUzMbJSZdTKz/uGf+f8B\n9o4/WuuRlX5Uz1mijz5qtkjiGUvkOaOVlZyVKHuiRwssiyOMc5n3+OPQuTOMHp10EudqytczcS4q\nM2bAHnvAwoXw29/C5Zcnnci5itRsPZOs8MbE1cy770Lv3vDeezBwINx2G7TxhUxdNsW2nomkbSUN\nlHRU46uyiK4pWelH9ZwFfPYZ9O8fNCR77gk33dRsQ+J1GS3Pmbxm5+YKp1PpA2wNPATsDzwN3Bxr\nMuey4q23giuTHj3g3nuhXbukEzlXc6U8ZzIT2B6YbmbbS9oY+KuZ9a1FwGp4N5ermdmzYfXVoWvX\npJM4V7VKurlKmTX4CzNbLmmZpHWAD4DNK0roXEvVs2fSCZxLVCljJi9K6gBcB7wETAP+HWuqViYr\n/aieMzpZyAieM2pZyVmJZq9MzOxX4dsxkh4B1jGzGfHGci7FPvoINtgg6RTOpUopYyb/NLMfNrcv\njXzMxEXutddgt93gd7+DM84AldWt7FwmRDpmIqkdsAawgaT1gMYTrwP4ggyu9Vm2DI46ChYsgBdf\nTDqNc6lSbMxkCMEYSU+CcZKXwtdEoOS5IiT1kzRb0huSzi5QZpSkOZLqJdU1d6yk4ZLelTQtfPUr\nNU8aZaUftdXn/NOf4KWXYIstYNy4qq5KWn1dRsxzJq/glYmZXQ1cLekUM/tzJScPV2YcDfwQ+B8w\nVdJEM5udU2Z/oJuZdZe0KzAG6F3CsVf6+vOuZl57DYaHK1hfdx2svXayeZxLmVLGTNYETge2MLMT\nJHUHepjZA82eXOoNDDez/cPtcwjmihyRU2YMMNnM7gi3ZxE8JNm10LHhg5SLzOyKZr7fx0xcNPbb\nDyZNguOOg+uvTzqNc7GKazqVG4CvgO+F2+8Bfyzx/J2A/+Zsv8vK4y2FyjR37Mlht9j1ktYtMY9z\nlbn+ehg8GK4o+veLc61WKQ8tdjOzQyUNAjCzz6VYb2Ep5dzXABeamUn6I3AlcFxTBQcPHkyXLl0A\n6NChA3V1dfQJl09t7L9MertxX1ryFNoeOXJkKuuvJvW5+eZMOfpomD49kvPlZ43y3x/ldn19Paed\ndlpq8hTa9vqsvv4mTJgA8PXvy7KZWdEX8CzQHpgWbncDXmjuuLBsb+CRnO1zgLPzyowBDs3Zng1s\nXMqx4f50QyagAAAPaklEQVTOwIwC329ZMHny5KQjlMRzRicLGc08Z9SykjP83dns7/jcVyljJn2B\n8wgmepwE7A4MNrMpzTVUktoCrxMMor8PvAAMMrNZOWX6AyeZ2Y/CMZaRZta72LGSNjGzeeHxpwPf\nNbPDm/h+a+7f55xzbkWxzM1lZo9JmkZwpSDg12bW/LqkwbENkk4maITaAOPDxmBI8LGNM7OHJPWX\n9CawGDim2LHhqS8LbyFeTrA+/ZAy/s3ONe/TT2GddXxNEudKVOp/Ke2AT4AFwNaSvl/qF5jZI2bW\nw8y6m9ml4b6xZjYup8zJZralmW1vZtOKHRvuP8rMeplZnZkdZGbzS82TRrn9vWnWanIuXw4DBsA+\n+wRTy8eg1dRljXjO5JWynskI4FDgVYIrAQADnooxl3PJufZaeOop2GgjaN8+6TTOZUIpYyavA73M\nbEltIkXHx0xc2d5+G7bbDhYvhrvugkMOSTqRczUX13MmbwGrVhbJuQxZsgSOPDJoSH72M29InCtD\nwcZE0p8ljQI+B+oljQ3n0BoV7ncRyUo/aovPef318Mwz0KkTjC55+rmKtPi6rDHPmbxiYyaN06K+\nBNxXgyzOJevEE2H+fPjJT4LxEudcyQqOmUiaYGaDaxsnWj5m4pxz5Yt6zKRXlXmcc861EsUakzUk\n7SBpx6ZeNUvYCmSlH9VzRicLGcFzRi0rOStRbMykE3AFTU+8aMAPYknkXC0sXAgnnQSXXBIMuDvn\nqlJszGS6me1Q4zyR8jET16SGBjjoIHjgAdh7b3jiiaQTOZcqcT1n4lzLcuaZQUPSsSOMHZt0Guda\nhGKNSZPrtbvoZaUftUXkHDsWrroKVl0V7r4bunevWa5cLaIuU8RzJq9gY2Jmk2oZxLnYzZwZjJMA\njBsHe+2VbB7nWpBm5+bKMh8zcSswg4svhs8/h4suSjqNc6lVyZhJ2Y2JpHbAAWb297IOTIA3Js45\nV77YBuAltQ0XsLoFeIdgSnoXkaz0o3rO6GQhI3jOqGUlZyWKrmciaS/gcKA/wbK5uwNdzezzGmRz\nzjmXEcWeM3kX+A9wLXCvmS2U9LaZda1lwGp4N1crtnw5DBsGxx8P3bolnca5TIm6m+suYFOCLq0D\nJK1J8OS7c+l37rkwYgTsuy8sXZp0GudavGK3Bp8GdCWYUqUP8DqwoaSBktaqTbzWISv9qJnJecYZ\ncOml0LYt/OUvwTMlKZOZuvSckcpKzkoUHYC3wGQzO4GgYRkEDADm1iCbc+V74AEYOTJ4P3Ys9OuX\nbB7nWomSbw2WtCqwLfAesNDMvogzWBR8zKSV+X//D3r1Cp4jOf98uPDCpBM5l0mRPmciaQzwZzN7\nVdK6wL+BBqAj8Fszu63awHHzxqSVMYNrr4Xp04Mn3FXWfwvOuVDUA/B7mtmr4ftjgDfMbDtgJ+Cs\nCjO6JmSlHzX1OSX41a+YcsQRqW9IUl+XIc8ZrazkrESxxuSrnPd9gXsBzGxerImcc85lTrFurskE\nd3K9B0wGeprZPEmrAK+YWc/axayMd3M551z5KunmKvYE/BBgFLAJcFrOFckPgQcri+hchD75BD76\nKLFp5J1z3yj2nMkbZtbPzOrMbELO/keB/9YiXGuRlX7UVOU0g2OOgZ12gkkrrpaQqpwFZCEjeM6o\nZSVnJSpdafGMSFM4V66rroKJE4MHE7faKuk0zrV6Fa1nIum/ZrZ5DHki5WMmLdRzz8Gee8KyZXDv\nvTBgQNKJnGtRarkGvP+Gdsn4+GMYODBoSM44wxsS51KiYGMiaaGkBeHPxvcLJC0kmADSRSQr/aip\nyPncczB/PvTuHcy/1YRU5GxGFjKC54xaVnJWouDdXGa2di2DOFeSH/0Inn0WNtgglRM4OtdaFXvO\npB3wS2BLYAZwg5ktq2G2qvmYiXPOlS/qMZObgJ2BmQQrLV5RRTbnyjNrFvzyl8GzJM651CvWmGxt\nZj83s7HAT4E9K/kCSf0kzZb0hqSzC5QZJWmOpHpJdc0dK2k9SZMkvS7p0XAiyszKSj9q7DnN4Mkn\n4YADYOutgynkx4wp+zRZqM8sZATPGbWs5KxEscbk6+XpKu3ektQGGA3sB2wDDJLUM6/M/kA3M+tO\n8NT9mBKOPQd43Mx6AE8AQyvJlxb19fVJRyhJrDmffhp23BH69AnWJGnXLrgy+elPyz5VFuozCxnB\nc0YtKzkrUWw6le0lLQjfC2gfbotg3ax1Sjj/LsAcM3sHQNLtBItrzc4pMwC4meCkz0taV9LGBItx\nFTp2ALBXePxNwBSCBiaTPv3006QjlKSqnEuXwv/+B/Pmwa67rvz5aqtBfT2svz6cfDKcdBJsuGHt\nc9ZIFjKC54xaVnJWotjdXG0jOH8nVpx65V2CBqa5Mp2aOXZjM5sf5pwnaaOCCV56acXtNm1ghx1W\nLtfQEPwyy1eL8k2JO89XXwVXAEuXBu8bGoL9q6wCP//5yuWXLAnq8pprgvJLlwavtm3h7CZ6Lz/8\nEPbeGz79NHgtXhzs79wZ5s5dufz228Ojj8Iee8Aaa6z8uXMu1YpdmSSlkoUoCt+ytfPOK2wuZC3W\ntoUrl/v885XK1qr83KMPqX2eL7+EQ1b+3oWsxdoFGpO5DzwQNEA5FrA26zTVmLRvD6+++vVmA234\niA2of6cH+y1ZAquvvmL51VdH++278nkIhlKaUmjJkqOPnltW+XLPH0X5uXPnpipP4fJzueCCNOXx\n+qy2fGw3uJpZbC+gN/BIzvY5wNl5ZcYAh+ZszwY2LnYsMIvg6gSCWY1nFfh+85e//OUvf5X/Kvf3\nfdxXJlOBLSV1Bt4HDgMG5ZW5DzgJuENSb+BTM5sv6aMix94HDAZGAEcDE5v68nLvk3bOOVeZWBsT\nM2uQdDIwieDOsfFmNkvSkOBjG2dmD0nqL+lNYDHBEsEFjw1PPQK4U9KxwDvAwDj/Hc4554qraNZg\n55xzLlelswanWikPSqaBpLmSXpY0XdILSedpJGm8pPmSZuTsS92DogVyDpf0rqRp4atfkhnDTJtJ\nekLSq5JmSjo13J+qOm0i5ynh/tTUqaTVJT0f/jczU9LwcH/a6rJQztTUZS5JbcI894XbZddni7sy\nCR92fINgeeH/EYzbHGZms4semABJbwE7mVmq5gyRtAewCLjZzHqF+0YAH5vZZWEDvZ6ZJfpsT4Gc\nw4GFZnZlktlySdoE2MTM6iWtBbxE8KzUMaSoTovkPJQU1amkNczsc0ltgWeAU4FDSFFdFsm5Pymq\ny0aSTgd2AtYxswMr+e+9JV6ZfP2gpJktBRofdkwjkcL/DczsaSC/gRtA8IAo4c+DahqqCQVyQmW3\nl8fGzOaZWX34fhHB3YibkbI6LZCzU/hxaurUzD4P365OMO5rpKwuoWBOSFFdQnBFSjD/4vU5u8uu\nz9T9IotAoYcg08iAxyRNlXR80mGasVHug6JA4QdFk3dyOM/b9Ul3d+ST1AWoA54j7+FbUlSnOTmf\nD3elpk7DLpnpwDzgMTObSgrrskBOSFFdhq4CzuSbxg4qqM+W2Jhkye5mtiPBXwUnhd02WZHW/tFr\ngG+bWR3Bf8Sp6U4Iu47uAn4d/uWfX4epqNMmcqaqTs1suZntQHB1t4ukbUhhXTaRc2tSVpeSfgTM\nD69Ii10xNVufLbExeQ/YImd7s3Bf6pjZ++HPD4F7WHmqmTSZr2DOtMa+9Q8SztMkM/vQvhkIvA74\nbpJ5GklaheAX9C1m1vhcVOrqtKmcaa1TM1tAMC9fP1JYl41yc6awLncHDgzHb28DfiDpFmBeufXZ\nEhuTrx+UlLQawcOO9yWcaSWS1gj/AkTSmsC+wCvJplqBWPEvlcYHRaHIg6IJWCFn+H/8RgeTnjq9\nAXjNzK7O2ZfGOl0pZ5rqVNIGjV1DktoDfQnGdlJVlwVyzk5TXQKY2TAz28LMvk3wu/IJMzsSuJ8y\n67PF3c0Fwa3BwNV887Bj04uFJ0hSV4KrESMYnPtbWnJKuhXoA6wPzAeGA/cCfwc2J3xQ1MwSnQK1\nQM69Cfr6lwNzgSGNfb9JkbQ78BTBQnON01UMA14A7iQldVok5+GkpE4lbUcwINwmfN1hZhdJ6ki6\n6rJQzptJSV3mk7QX8Jvwbq6y67NFNibOOedqqyV2cznnnKsxb0ycc85VzRsT55xzVfPGxDnnXNW8\nMXHOOVc1b0ycc85VzRsT16JJWhj+7Cwpf5XPas89NG/76SjPX0tx1I9rXbwxcS1d44NUXQkevitZ\nOHV4McNW+CKzLM2tlq/s+nEulzcmrrW4BNgjXADo1+GMrpeFCxjVN87aLGkvSU9Jmgi8Gu67J5zZ\neaakX4T7LgHah+e7Jdy3sPHLJF0eln9Z0sCcc0+W9HdJsxqPCz+7VNIrYZbL8sNLWlPSDZJmhGV+\nEu4fFO6bIenSnPK5WQ6RdGP4/kZJV0t6RtKbkg5uqn4iqXHXqsS6BrxzKXIO4VQRAGHj8amZ7RrO\n4faMpElh2R2AbczsP+H2MWb2qaR2wFRJ/zCzoZJOCmd9bmThuQ8BepnZdpI2Co95MixTB2xNMGPs\nM5K+B8wGDjKznuHx6zSR//wwb+MiYOtK+hZwaZj3U4LlDA40s/soPovuJma2u6TvEMxpdXd+/ThX\nLr8yca3VvsBR4XoTzwMdge7hZy/kNCQAp0mqJ1iDZLOccoXsTjADK2b2AcGMsY2zw75gZu+HM8fW\nA12Az4AvwvUtfgJ80cQ59wH+0rhhZp+F55xsZv9nZsuBvwHfD4sUm0783vAcs0jBuh+uZfDGxLVW\nAk4xsx3CVzczezz8bPHXhYLJ734A7BquQVEPtMs5R6nf1WhJzvsGYBUzayBYfuAu4MfAI02co9Ak\neoUy5JZvl/dZboZUrfrnsssbE9fSNf6yXAisnbP/UeBX4fodSOouaY0mjl8X+MTMlkjqCfTO+eyr\nxuPzvutfwKHhuMyGwJ4EMwQ3HTD43g5m9ghwBtCriWKPASflHNMhPOf3JXUMbxYYRHAVBMF6FD0k\ntQF+Uui7KVw/zpXFGxPX0jX+hT4DWC5puqRfm9l1wGvANEkzgTFAU3dvPQKsKulV4GLg3zmfjQNm\n5AykG4CZ3RN+38vA48CZYXdXoWzrAA9IeplgCvjTmyh7EdAxHNSfDvQJl1M9h6ABmQ5MNbMHwvJD\ngQeBp4H/NfGd+dsr1E8T3+9cUT4FvXPOuar5lYlzzrmqeWPinHOuat6YOOecq5o3Js4556rmjYlz\nzrmqeWPinHOuat6YOOecq5o3Js4556r2/wGzal9fiBR69gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xf33c0b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(xrange(model_plsa.num_phi_updates), model_plsa.score_tracker['SparsityPhiScore'].value, 'b--',\n",
    "                 xrange(model_artm.num_phi_updates), model_artm.score_tracker['SparsityPhiScore'].value, 'r--', linewidth=2)\n",
    "plt.xlabel('Iterations count')\n",
    "plt.ylabel('PLSA Phi sp. (blue), ARTM Phi sp. (red)')\n",
    "plt.grid(True)\n",
    "plt.show()\n",
    "\n",
    "plt.plot(xrange(model_plsa.num_phi_updates), model_plsa.score_tracker['SparsityThetaScore'].value, 'b--',\n",
    "                 xrange(model_artm.num_phi_updates), model_artm.score_tracker['SparsityThetaScore'].value, 'r--', linewidth=2)\n",
    "plt.xlabel('Iterations count')\n",
    "plt.ylabel('PLSA Theta sp. (blue), ARTM Theta sp. (red)')\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Кажется, что достигнутых результатов достаточно. Регуляризация позволила добиться улучшения всех характеристик, ухудшив перплексию в пределах разумного. Взглянем на топ-слова слова моделей:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "topic_0:  [u'year', u'tax', u'jobs', u'america', u'president', u'issues']\n",
      "topic_1:  [u'people', u'war', u'service', u'military', u'rights', u'vietnam']\n",
      "topic_2:  [u'november', u'electoral', u'account', u'polls', u'governor', u'contact']\n",
      "topic_3:  [u'republican', u'gop', u'senate', u'senator', u'south', u'conservative']\n",
      "topic_4:  [u'people', u'time', u'country', u'speech', u'talking', u'read']\n",
      "topic_5:  [u'dean', u'democratic', u'edwards', u'primary', u'kerry', u'clark']\n",
      "topic_6:  [u'state', u'party', u'race', u'candidates', u'candidate', u'elections']\n",
      "topic_7:  [u'administration', u'president', u'years', u'bill', u'white', u'cheney']\n",
      "topic_8:  [u'campaign', u'national', u'media', u'local', u'late', u'union']\n",
      "topic_9:  [u'house', u'million', u'money', u'republican', u'committee', u'delay']\n",
      "topic_10:  [u'republicans', u'vote', u'senate', u'election', u'democrats', u'house']\n",
      "topic_11:  [u'iraq', u'war', u'american', u'iraqi', u'military', u'intelligence']\n",
      "topic_12:  [u'kerry', u'poll', u'percent', u'voters', u'polls', u'numbers']\n",
      "topic_13:  [u'news', u'time', u'asked', u'political', u'washington', u'long']\n",
      "topic_14:  [u'bush', u'general', u'bushs', u'kerry', u'oct', u'states']\n"
     ]
    }
   ],
   "source": [
    "for topic_name in model_plsa.topic_names:\n",
    "    print topic_name + ': ',\n",
    "    print model_plsa.score_tracker['TopTokensScore'].last_tokens[topic_name]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "topic_0:  [u'party', u'political', u'issue', u'tax', u'america', u'issues']\n",
      "topic_1:  [u'people', u'military', u'official', u'officials', u'service', u'public']\n",
      "topic_2:  [u'electoral', u'governor', u'account', u'contact', u'ticket', u'experience']\n",
      "topic_3:  [u'gop', u'convention', u'senator', u'debate', u'south', u'sen']\n",
      "topic_4:  [u'country', u'speech', u'bad', u'read', u'end', u'talking']\n",
      "topic_5:  [u'democratic', u'dean', u'john', u'edwards', u'primary', u'clark']\n",
      "topic_6:  [u'percent', u'race', u'candidates', u'candidate', u'win', u'nader']\n",
      "topic_7:  [u'administration', u'years', u'white', u'year', u'bill', u'jobs']\n",
      "topic_8:  [u'campaign', u'national', u'media', u'press', u'local', u'ads']\n",
      "topic_9:  [u'house', u'republican', u'million', u'money', u'elections', u'district']\n",
      "topic_10:  [u'november', u'poll', u'senate', u'republicans', u'vote', u'election']\n",
      "topic_11:  [u'iraq', u'war', u'american', u'iraqi', u'security', u'united']\n",
      "topic_12:  [u'bush', u'kerry', u'general', u'president', u'voters', u'bushs']\n",
      "topic_13:  [u'time', u'news', u'long', u'asked', u'washington', u'political']\n",
      "topic_14:  [u'state', u'states', u'people', u'oct', u'fact', u'ohio']\n"
     ]
    }
   ],
   "source": [
    "for topic_name in model_artm.topic_names:\n",
    "    print topic_name + ': ',\n",
    "    print model_artm.score_tracker['TopTokensScore'].last_tokens[topic_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Видно, что темы примерно одинаково интерпретируемы, но в модели ARTM они существенно разнообразнее."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Извлечём матрицу $\\Phi$ в виде pandas.DataFrame и напечатаем её (в случае необходимости, можно извлекать части матрицы с помощью метода ARTM.get_phi()):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                   topic_0   topic_1   topic_2   topic_3   topic_4   topic_5  \\\n",
      "parentheses       0.000000  0.000000  0.000000  0.000000  0.000000  0.000277   \n",
      "opinion           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "attitude          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "held              0.000000  0.000385  0.000000  0.000000  0.000000  0.000000   \n",
      "impeachment       0.000000  0.000115  0.000000  0.000000  0.000000  0.000000   \n",
      "platform          0.001717  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "msnbc             0.000000  0.000000  0.000000  0.000000  0.000000  0.000194   \n",
      "assault           0.000000  0.000000  0.000000  0.000000  0.001202  0.000000   \n",
      "tools             0.000000  0.000000  0.000000  0.000000  0.000502  0.000000   \n",
      "vance             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "contingent        0.000000  0.000544  0.000000  0.000000  0.000000  0.000000   \n",
      "drove             0.000000  0.000146  0.000000  0.000000  0.000374  0.000000   \n",
      "stripped          0.000000  0.000111  0.000000  0.000000  0.000410  0.000000   \n",
      "air               0.000000  0.000593  0.000000  0.000000  0.000124  0.000000   \n",
      "baby              0.000139  0.000003  0.000032  0.000000  0.000000  0.000000   \n",
      "highlight         0.000000  0.000000  0.000000  0.000000  0.000019  0.000000   \n",
      "honored           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kerry             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "nethercutt        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kansas            0.000007  0.000000  0.000000  0.000076  0.000000  0.000000   \n",
      "annual            0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "disaster          0.000076  0.000000  0.000000  0.000000  0.001151  0.000000   \n",
      "operatives        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "fluid             0.000030  0.000000  0.000000  0.000000  0.000000  0.000344   \n",
      "aug               0.000043  0.000000  0.000000  0.001610  0.003245  0.000000   \n",
      "monitoring        0.000000  0.000000  0.000000  0.000000  0.000000  0.000490   \n",
      "initially         0.000000  0.000113  0.000000  0.000000  0.000000  0.000000   \n",
      "kysen             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "action            0.000904  0.000107  0.000000  0.000000  0.000488  0.000147   \n",
      "wide              0.000000  0.000862  0.000000  0.000000  0.000000  0.000164   \n",
      "...                    ...       ...       ...       ...       ...       ...   \n",
      "drake             0.000000  0.000000  0.000000  0.000000  0.000000  0.000348   \n",
      "freeway           0.000000  0.000000  0.000000  0.000000  0.000297  0.000000   \n",
      "playbook          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "hoodies           0.000000  0.000000  0.002087  0.000000  0.000000  0.000000   \n",
      "vaantirepublican  0.000000  0.000000  0.002060  0.000000  0.000000  0.000000   \n",
      "suck              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "thirdplace        0.000000  0.000000  0.000000  0.000000  0.000000  0.000348   \n",
      "wink              0.000000  0.000000  0.000176  0.000000  0.000408  0.000000   \n",
      "inching           0.000000  0.000000  0.000000  0.000000  0.000000  0.000312   \n",
      "intangibles       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "sawyer            0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "composite         0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "geps              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "postiowa          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "brush             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "caucusgoers       0.000000  0.000000  0.000000  0.000000  0.000000  0.000667   \n",
      "countylevel       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "outreach          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "rncs              0.000297  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "uaw               0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "deanclark         0.000000  0.000000  0.000000  0.000000  0.000000  0.000348   \n",
      "samarra           0.000000  0.000000  0.000000  0.000000  0.000466  0.000000   \n",
      "postdebate        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "schwarz           0.000534  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "asserted          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "aft               0.000000  0.000000  0.000000  0.000000  0.000000  0.000419   \n",
      "spindizzy         0.000000  0.000000  0.000647  0.000000  0.000000  0.000000   \n",
      "barnes            0.000000  0.000000  0.000000  0.000792  0.000000  0.000000   \n",
      "barbour           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kroll             0.000000  0.000317  0.000000  0.000000  0.000000  0.000000   \n",
      "\n",
      "                   topic_6   topic_7   topic_8   topic_9  topic_10  topic_11  \\\n",
      "parentheses       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "opinion           0.000394  0.000000  0.000193  0.000000  0.000000  0.000355   \n",
      "attitude          0.000499  0.000000  0.000000  0.000000  0.000000  0.000187   \n",
      "held              0.000000  0.000000  0.000000  0.006317  0.000000  0.000790   \n",
      "impeachment       0.000000  0.000000  0.000000  0.000000  0.000094  0.000000   \n",
      "platform          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "msnbc             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "assault           0.000000  0.000000  0.000000  0.000000  0.000000  0.000471   \n",
      "tools             0.000000  0.000000  0.000329  0.000000  0.000000  0.000000   \n",
      "vance             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "contingent        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "drove             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "stripped          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "air               0.000000  0.000025  0.000000  0.000084  0.000000  0.000048   \n",
      "baby              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "highlight         0.000000  0.000590  0.000371  0.000000  0.000000  0.000000   \n",
      "honored           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kerry             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "nethercutt        0.000244  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kansas            0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "annual            0.000000  0.001851  0.000000  0.000000  0.000000  0.000000   \n",
      "disaster          0.000000  0.000000  0.000000  0.000000  0.000000  0.000804   \n",
      "operatives        0.000000  0.000008  0.000149  0.000000  0.000000  0.000431   \n",
      "fluid             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "aug               0.000183  0.000035  0.001008  0.000504  0.000000  0.000000   \n",
      "monitoring        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "initially         0.000000  0.000726  0.000314  0.000000  0.000000  0.000094   \n",
      "kysen             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "action            0.000000  0.001463  0.000212  0.002019  0.000000  0.000034   \n",
      "wide              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "...                    ...       ...       ...       ...       ...       ...   \n",
      "drake             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "freeway           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "playbook          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "hoodies           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "vaantirepublican  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "suck              0.000000  0.000000  0.000368  0.000000  0.000000  0.000000   \n",
      "thirdplace        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "wink              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "inching           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "intangibles       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "sawyer            0.000000  0.000243  0.000000  0.000000  0.000107  0.000000   \n",
      "composite         0.000391  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "geps              0.000535  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "postiowa          0.000000  0.000000  0.000471  0.000000  0.000000  0.000000   \n",
      "brush             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "caucusgoers       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "countylevel       0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "outreach          0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "rncs              0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "uaw               0.000000  0.000000  0.000334  0.000000  0.000000  0.000000   \n",
      "deanclark         0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "samarra           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "postdebate        0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "schwarz           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "asserted          0.000000  0.000178  0.000000  0.000000  0.000000  0.000000   \n",
      "aft               0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "spindizzy         0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "barnes            0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "barbour           0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "kroll             0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
      "\n",
      "                  topic_12      topic_13  topic_14  \n",
      "parentheses       0.000000  0.000000e+00  0.000000  \n",
      "opinion           0.002908  0.000000e+00  0.000346  \n",
      "attitude          0.000000  0.000000e+00  0.000000  \n",
      "held              0.000526  0.000000e+00  0.000000  \n",
      "impeachment       0.000000  0.000000e+00  0.000396  \n",
      "platform          0.000000  0.000000e+00  0.000000  \n",
      "msnbc             0.000000  0.000000e+00  0.002042  \n",
      "assault           0.000000  4.839016e-05  0.000000  \n",
      "tools             0.000000  3.548835e-04  0.000000  \n",
      "vance             0.000000  0.000000e+00  0.000803  \n",
      "contingent        0.000000  0.000000e+00  0.000000  \n",
      "drove             0.000000  0.000000e+00  0.000000  \n",
      "stripped          0.000000  0.000000e+00  0.000000  \n",
      "air               0.000000  4.631779e-05  0.010024  \n",
      "baby              0.000000  1.057148e-03  0.000000  \n",
      "highlight         0.000000  8.328939e-07  0.000000  \n",
      "honored           0.000000  4.265936e-04  0.000000  \n",
      "kerry             0.138169  0.000000e+00  0.000000  \n",
      "nethercutt        0.000143  0.000000e+00  0.000000  \n",
      "kansas            0.000100  0.000000e+00  0.000969  \n",
      "annual            0.000000  8.087117e-06  0.000000  \n",
      "disaster          0.000000  3.991339e-05  0.000000  \n",
      "operatives        0.000000  7.908887e-04  0.000000  \n",
      "fluid             0.000000  0.000000e+00  0.000000  \n",
      "aug               0.003553  0.000000e+00  0.000000  \n",
      "monitoring        0.000000  0.000000e+00  0.000000  \n",
      "initially         0.000000  1.245604e-04  0.000000  \n",
      "kysen             0.000000  7.598699e-04  0.000000  \n",
      "action            0.000000  0.000000e+00  0.000411  \n",
      "wide              0.000000  0.000000e+00  0.000487  \n",
      "...                    ...           ...       ...  \n",
      "drake             0.000000  0.000000e+00  0.000000  \n",
      "freeway           0.000000  0.000000e+00  0.000000  \n",
      "playbook          0.000378  0.000000e+00  0.000000  \n",
      "hoodies           0.000000  0.000000e+00  0.000000  \n",
      "vaantirepublican  0.000000  0.000000e+00  0.000000  \n",
      "suck              0.000000  0.000000e+00  0.000000  \n",
      "thirdplace        0.000000  0.000000e+00  0.000000  \n",
      "wink              0.000000  0.000000e+00  0.000000  \n",
      "inching           0.000000  0.000000e+00  0.000000  \n",
      "intangibles       0.000000  0.000000e+00  0.000310  \n",
      "sawyer            0.000000  0.000000e+00  0.000000  \n",
      "composite         0.000000  0.000000e+00  0.000000  \n",
      "geps              0.000000  0.000000e+00  0.000000  \n",
      "postiowa          0.000000  0.000000e+00  0.000000  \n",
      "brush             0.000000  2.548045e-04  0.000139  \n",
      "caucusgoers       0.000000  0.000000e+00  0.000000  \n",
      "countylevel       0.000000  4.599213e-04  0.000000  \n",
      "outreach          0.000000  0.000000e+00  0.000698  \n",
      "rncs              0.000000  0.000000e+00  0.000000  \n",
      "uaw               0.000000  0.000000e+00  0.000000  \n",
      "deanclark         0.000000  0.000000e+00  0.000000  \n",
      "samarra           0.000000  0.000000e+00  0.000000  \n",
      "postdebate        0.000086  0.000000e+00  0.000763  \n",
      "schwarz           0.000000  0.000000e+00  0.000000  \n",
      "asserted          0.000118  0.000000e+00  0.000000  \n",
      "aft               0.000000  0.000000e+00  0.000000  \n",
      "spindizzy         0.000000  0.000000e+00  0.000000  \n",
      "barnes            0.000000  0.000000e+00  0.000000  \n",
      "barbour           0.000000  0.000000e+00  0.000451  \n",
      "kroll             0.000000  0.000000e+00  0.000000  \n",
      "\n",
      "[6906 rows x 15 columns]\n"
     ]
    }
   ],
   "source": [
    "print model_artm.phi_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Дополнительно извлечём $\\Theta$ в виде pandas.DataFrame и напечатаем её:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              3001      3002      3003      3004      3005      3006  \\\n",
      "topic_0   0.076121  0.035934  0.069378  0.136764  0.083617  0.011377   \n",
      "topic_1   0.095387  0.029030  0.034951  0.043516  0.064787  0.022496   \n",
      "topic_2   0.004087  0.048284  0.012102  0.006649  0.015708  0.520682   \n",
      "topic_3   0.057739  0.029823  0.122427  0.085727  0.060414  0.016142   \n",
      "topic_4   0.122659  0.046273  0.084217  0.077540  0.064804  0.014840   \n",
      "topic_5   0.025646  0.010737  0.024132  0.051743  0.059580  0.017807   \n",
      "topic_6   0.044074  0.017331  0.072042  0.042680  0.036825  0.017084   \n",
      "topic_7   0.061899  0.059855  0.022353  0.029254  0.002857  0.005222   \n",
      "topic_8   0.049225  0.048494  0.043635  0.095823  0.025636  0.016371   \n",
      "topic_9   0.070107  0.492084  0.014166  0.131112  0.200206  0.025601   \n",
      "topic_10  0.056218  0.023523  0.158903  0.092290  0.059759  0.239756   \n",
      "topic_11  0.158816  0.031754  0.028020  0.024615  0.041939  0.006004   \n",
      "topic_12  0.054255  0.027387  0.191633  0.078728  0.125391  0.052281   \n",
      "topic_13  0.056038  0.056568  0.025291  0.041948  0.110980  0.032068   \n",
      "topic_14  0.067729  0.042923  0.096752  0.061612  0.047496  0.002269   \n",
      "\n",
      "              3007      3008      3009      3010    ...         991   \\\n",
      "topic_0   0.153216  0.031591  0.066032  0.036219    ...     0.053795   \n",
      "topic_1   0.041969  0.009980  0.024485  0.055780    ...     0.119861   \n",
      "topic_2   0.010361  0.000000  0.012824  0.004855    ...     0.201033   \n",
      "topic_3   0.140113  0.095807  0.090738  0.080190    ...     0.015301   \n",
      "topic_4   0.028386  0.042167  0.044592  0.068055    ...     0.048995   \n",
      "topic_5   0.039975  0.080582  0.077005  0.067891    ...     0.014937   \n",
      "topic_6   0.186580  0.291586  0.159585  0.068007    ...     0.005185   \n",
      "topic_7   0.033841  0.019672  0.033055  0.071855    ...     0.035326   \n",
      "topic_8   0.021736  0.015335  0.030470  0.059429    ...     0.030472   \n",
      "topic_9   0.158547  0.344387  0.276789  0.279663    ...     0.025226   \n",
      "topic_10  0.032311  0.000000  0.048914  0.031847    ...     0.080713   \n",
      "topic_11  0.024322  0.002047  0.013785  0.018584    ...     0.272355   \n",
      "topic_12  0.044409  0.017861  0.013088  0.019473    ...     0.023370   \n",
      "topic_13  0.032618  0.034591  0.043202  0.063385    ...     0.040395   \n",
      "topic_14  0.051617  0.014395  0.065438  0.074767    ...     0.033035   \n",
      "\n",
      "              992       993       994       995       996       997   \\\n",
      "topic_0   0.057835  0.078560  0.037073  0.170305  0.023683  0.094523   \n",
      "topic_1   0.121933  0.180116  0.112602  0.032622  0.034981  0.143123   \n",
      "topic_2   0.000000  0.057726  0.002886  0.000632  0.007803  0.014957   \n",
      "topic_3   0.040474  0.066918  0.036634  0.043454  0.012304  0.051498   \n",
      "topic_4   0.071596  0.133850  0.082915  0.049283  0.021307  0.059609   \n",
      "topic_5   0.017675  0.021395  0.015159  0.030125  0.042343  0.039248   \n",
      "topic_6   0.009005  0.029403  0.040560  0.104222  0.106431  0.031618   \n",
      "topic_7   0.018613  0.005084  0.109385  0.062973  0.036562  0.036915   \n",
      "topic_8   0.185597  0.034914  0.050408  0.026470  0.023620  0.130383   \n",
      "topic_9   0.012500  0.009456  0.067889  0.088528  0.025921  0.030995   \n",
      "topic_10  0.040815  0.000000  0.018810  0.124785  0.104889  0.031240   \n",
      "topic_11  0.281438  0.232306  0.255559  0.022806  0.116224  0.068760   \n",
      "topic_12  0.014712  0.000000  0.058669  0.136899  0.352839  0.069516   \n",
      "topic_13  0.054435  0.081143  0.079657  0.036140  0.027119  0.173720   \n",
      "topic_14  0.073375  0.069129  0.031795  0.070757  0.063973  0.023897   \n",
      "\n",
      "              998       999       1000  \n",
      "topic_0   0.077524  0.081823  0.066142  \n",
      "topic_1   0.185704  0.103474  0.126820  \n",
      "topic_2   0.008666  0.007206  0.016567  \n",
      "topic_3   0.043518  0.090904  0.129180  \n",
      "topic_4   0.077166  0.076402  0.115954  \n",
      "topic_5   0.032495  0.027419  0.030624  \n",
      "topic_6   0.019817  0.011535  0.043325  \n",
      "topic_7   0.060655  0.054204  0.052605  \n",
      "topic_8   0.059885  0.065536  0.083408  \n",
      "topic_9   0.031605  0.023796  0.024791  \n",
      "topic_10  0.011905  0.001742  0.000308  \n",
      "topic_11  0.196675  0.146919  0.169095  \n",
      "topic_12  0.011672  0.079587  0.036432  \n",
      "topic_13  0.134102  0.135609  0.059862  \n",
      "topic_14  0.048609  0.093843  0.044887  \n",
      "\n",
      "[15 rows x 3430 columns]\n"
     ]
    }
   ],
   "source": [
    "theta_matrix = model_artm.get_theta()\n",
    "print theta_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Можно использовать модель для определения векторов $\\theta_d$ для новых документов с помощью метода ARTM.transform():"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Empty DataFrame\n",
      "Columns: []\n",
      "Index: []\n"
     ]
    }
   ],
   "source": [
    "test_batch_vectorizer = artm.BatchVectorizer(data_format='batches', data_path='kos_test', batches=['test_docs.batch'])\n",
    "test_theta_matrix = model_artm.transform(batch_vectorizer=test_batch_vectorizer)\n",
    "print test_theta_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Выводы\n",
    "\n",
    "Задача построения тематической модели имеет бесконечно большое множество решений. Это даёт большую свободу действий, и регуляризаторы позволяют использовать её для получения результата, удовлетворяющего сразу нескольким требованиям (разреженность, интерпретируемость, удовлетворительное значение перплексии и т. п.).\n",
    "\n",
    "Приведённый выше пример является демонстрационным, можно пробовать более гибкие стратегии регуляризации для получения ещё более хорошего результата. По аналогичной схеме можно производить эксперименты с более крупными коллекциями."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}