diff --git a/elephantscale/notebooks/feedforward-credit-card-fraud-pipeline.ipynb b/elephantscale/notebooks/feedforward-credit-card-fraud-pipeline.ipynb index ccf223e..dba172b 100644 --- a/elephantscale/notebooks/feedforward-credit-card-fraud-pipeline.ipynb +++ b/elephantscale/notebooks/feedforward-credit-card-fraud-pipeline.ipynb @@ -4,27 +4,17 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Feedforward Network with Credit Card Fraud\n", + "# Feedforward Network with Credit Card Fraud (Pipeline API version)\n", "\n", - "Credit Card Transactions Fraud Detection Example:\n", - "\n", - "The notebook demonstrates how to develop a fraud detection application with the BigDL deep learning library on Apache Spark. We'll try to introduce some techniques that can be used for training a fraud detection model, but some advanced skills is not applicable since the dataset is highly simplified.\n", - "\n", - "Dataset: Credit Card Fraud Detection https://www.kaggle.com/dalpozz/creditcardfraud\n", - "\n", - "This dataset presents transactions that occurred in two days, where we got 492 frauds out of 284,807 transactions. The dataset is highly unbalanced, the positive class (frauds) account for 0.172% of all transactions.\n", - "\n", - "It contains only numerical input variables which are the result of a PCA transformation. \n", - "\n", - "Unfortunately, due to confidentiality issues, we cannot find the original features and more background information about the data. Features V1, V2, ... V28 are the principal components obtained with PCA, the only features which have not been transformed with PCA are 'Time' and 'Amount'. Feature 'Time' contains the seconds elapsed between each transaction and the first transaction in the dataset. The feature 'Amount' is the transaction Amount, this feature can be used for example-dependant cost-senstive learning. Feature 'Class' is the response variable and it takes value 1 in case of fraud and 0 otherwise.\n", - "\n", - "Contact: yuhao.yang@intel.com\n" + "Please view the credit-card-fraud example, which will show how to do this in with an RDD-based approach. Here, we will follow the same approach with a dataframe-based pipeline API based on Spark MLLib.\n" ] }, { "cell_type": "code", "execution_count": 36, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "%matplotlib inline\n", @@ -124,7 +114,9 @@ { "cell_type": "code", "execution_count": 38, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "cc_training = spark.read.csv(\"../data/creditcardfraud/creditcard.csv\", header=True, inferSchema=\"true\", mode=\"DROPMALFORMED\")" @@ -161,7 +153,9 @@ { "cell_type": "code", "execution_count": 40, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "cc_training = cc_training.select([col(c).cast(\"double\") for c in cc_training.columns])\n", @@ -206,7 +200,9 @@ { "cell_type": "code", "execution_count": 42, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "# get the time to split the data.\n", @@ -273,7 +269,9 @@ { "cell_type": "code", "execution_count": 43, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "assembler = VectorAssembler(inputCols=cols, outputCol=\"assembled\")\n", @@ -289,23 +287,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### step 2. split the dataset into training and validation dataset.\n", - "\n", - "Unlike some other training dataset, where the data does not have a time of occurance. For this case, we can know the sequence of the transactions from the Time column. Thus randomly splitting the data into training and validation does not make much sense, since in real world applications, we can only use the history transactions for training and use the latest transactions for validation. Thus we'll split the dataset according the time of occurance." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### step 3. Build the model with BigDL\n", + "## step 2. Build the model with BigDL\n", "From the research community and industry feedback, a simple neural network turns out be the perfect candidate for the fraud detection training. We will quickly build a multiple layer Perceptron with linear layers.\n", "```\n", " val bigDLModel = Sequential()\n", @@ -380,12 +362,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ + "# Evaluate our Model\n", "\n", - "Now we have finished the training of our first model (which is certainly not the best, keep reading!).\n", - "\n", - "We'll need to think about how do evaluate the trained model:\n", - "\n", - "Given the class imbalance ratio, we recommend measuring the accuracy using the Area Under the Precision-Recall Curve (AUPRC). Confusion matrix accuracy is not meaningful for unbalanced classification. Since even if the model predicts all the records as normal transactions, it will still get an accuracy above 99%." + "Now we are goin to do the Precision, Recall, and AUC. To evaluate our model." ] }, { @@ -433,7 +412,9 @@ { "cell_type": "code", "execution_count": 47, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "y_pred = np.array(predictionDF.select('prediction').collect())\n", @@ -443,7 +424,9 @@ { "cell_type": "code", "execution_count": 48, - "metadata": {}, + "metadata": { + "scrolled": true + }, "outputs": [ { "name": "stdout", @@ -474,74 +457,33 @@ "sn.heatmap(df_cm, annot=True,fmt='d');" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To this point, we have finished the training and evaluation with a simple BigDL model. We can see that even though the recall and precision are high, the area under precision-recall curve is not optimistic. That's because we haven't really apply any technique to handle the imbalanced training data.\n", - "\n", - "Next we'll try to optimize the training process.\n", - "\n", - "### step 4. handle the data imbalance\n", - "There are several ways to approach this classification problem taking into consideration this unbalance.\n", - "\n", - "Collect more data? Nice strategy but not applicable in this case.\n", - "\n", - "Resampling the dataset Essentially this is a method that will process the data to have an approximate 50-50 ratio. One way to achieve this is by OVER-sampling, which is adding copies of the under-represented class (better when there're little data) Another is UNDER-sampling, which deletes instances from the over-represented class (better when there are lots of data)\n", - "Apart from under and over sampling, there is a very popular approach called SMOTE (Synthetic Minority Over-Sampling Technique), which is a combination of oversampling and undersampling, but the oversampling approach is not by replicating minority class but constructing new minority class data instance via an algorithm.\n", - "\n", - "We'll start with Resampling.\n", - "\n", - "Since there're 492 frauds out of 284,807 transactions, to build a reasonable training dataset, we'll use UNDER-sampling for normal transactions and use OVER-sampling for fraud transactions. By using the sampling rate as fraud -> 10, normal -> 0.05, we can get a training dataset of (5K fraud + 14K normal) transactions. We can use the training data to fit a model.\n", - "\n", - "Yet we'll soon find that since there're only 5% of all the normal transactions are included in the training data, the model can only cover 5% of all the normal transactions, which is obviousely not optimistic. So how can we get a better converage for the normal transactions without breaking the ideal ratio in the training dataset?\n", - "\n", - "An immediate improvement would be to train multiple models. For each model, we will run the resampling from the original dataset and get a new training data set. After training, we can select best voting strategy for all the models to make the prediction.\n", - "\n", - "We'll use Ensembling of neural networks. That's where a Bagging classifier becomes handy. Bagging is an Estimator we developed for ensembling of multiple other Estimator.\n", - "\n", - "```\n", - "package org.apache.spark.ml.ensemble\n", - "\n", - "class Bagging[M <: Model[M]](override val uid: String)\n", - " extends Estimator[BaggingModel[M]]\n", - " with BaggingParams[M] {\n", - "For usage, user need to set the specific Estimator to use and the number of models to be trained:\n", - " val estimator = new Bagging()\n", - " .setPredictor(dlClassifier)\n", - " .setLabelCol(\"Class\")\n", - " .setIsClassifier(true)\n", - " .setNumModels(10)\n", - "```\n", - "\n", - "Internally, Bagging will train $(numModels) models. Each model is trained with the resampled data from the original dataset." - ] - }, { "cell_type": "code", "execution_count": null, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { - "display_name": "Python 2", + "display_name": "Python 3", "language": "python", - "name": "python2" + "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", - "version": 2 + "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", - "pygments_lexer": "ipython2", - "version": "2.7.14" + "pygments_lexer": "ipython3", + "version": "3.5.4" } }, "nbformat": 4, diff --git a/elephantscale/notebooks/feedforward-credit-card-fraud.ipynb b/elephantscale/notebooks/feedforward-credit-card-fraud.ipynb index 712d290..43a100b 100644 --- a/elephantscale/notebooks/feedforward-credit-card-fraud.ipynb +++ b/elephantscale/notebooks/feedforward-credit-card-fraud.ipynb @@ -8,6 +8,8 @@ "\n", "Let us look at a BigDL example with Credit Card Fraud. We will train a simple, feedforward neural network with the credit card dataset.\n", "\n", + "This is using RDDs. The feedforward-credit-card-fraud-pipeline uses the Spark.ML pipeline interface using dataframes.\n", + "\n", "Let's look briefly at the credit card dataset\n", "\n", "| Time\" | \"V1\" | \"V2\" | \"V3\" | \"V4\" | \"V5\" | \"V6\" | \"V7\" | \"V8\" | \"V9\" | \"V10\" | \"V11\" | \"V12\" | \"V13\" | \"V14\" | \"V15\" | \"V16\" | \"V17\" | \"V18\" | \"V19\" | \"V20\" | \"V21\" | \"V22\" | \"V23\" | \"V24\" | \"V25\" | \"V26\" | \"V27\" | \"V28\" | \"Amount\" | \"Class\" | \n", @@ -25,7 +27,9 @@ { "cell_type": "code", "execution_count": 1, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "%matplotlib inline\n", @@ -58,7 +62,9 @@ { "cell_type": "code", "execution_count": null, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "## Parameteters\n", @@ -69,10 +75,12 @@ { "cell_type": "code", "execution_count": 2, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ - "learning_rate = 0.1\n", + "learning_rate = 0.01\n", "training_epochs = 100\n", "batch_size = 10000\n", "display_step = 1\n", @@ -137,7 +145,9 @@ { "cell_type": "code", "execution_count": 2, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "cc_training = spark.read.csv(\"../data/creditcardfraud/creditcard.csv\", header=True, inferSchema=\"true\", mode=\"DROPMALFORMED\")" @@ -174,7 +184,9 @@ { "cell_type": "code", "execution_count": 3, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "cc_training = cc_training.select([col(c).cast(\"double\") for c in cc_training.columns])\n", @@ -183,46 +195,14 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": null, "metadata": { + "collapsed": true, "scrolled": true }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "+----+------------------+-------------------+------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------------------+-------------------+------------------+------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------+-----+\n", - "|Time| V1| V2| V3| V4| V5| V6| V7| V8| V9| V10| V11| V12| V13| V14| V15| V16| V17| V18| V19| V20| V21| V22| V23| V24| V25| V26| V27| V28|Amount|Class|\n", - "+----+------------------+-------------------+------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------------------+-------------------+------------------+------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------+-----+\n", - "| 0.0| -1.3598071336738|-0.0727811733098497| 2.53634673796914| 1.37815522427443| -0.338320769942518| 0.462387777762292| 0.239598554061257| 0.0986979012610507| 0.363786969611213| 0.0907941719789316|-0.551599533260813|-0.617800855762348| -0.991389847235408| -0.311169353699879| 1.46817697209427| -0.470400525259478| 0.207971241929242| 0.0257905801985591| 0.403992960255733| 0.251412098239705| -0.018306777944153| 0.277837575558899| -0.110473910188767| 0.0669280749146731| 0.128539358273528| -0.189114843888824| 0.133558376740387|-0.0210530534538215|149.62| 1.0|\n", - "| 0.0| 1.19185711131486| 0.26615071205963| 0.16648011335321| 0.448154078460911| 0.0600176492822243|-0.0823608088155687| -0.0788029833323113| 0.0851016549148104|-0.255425128109186| -0.166974414004614| 1.61272666105479| 1.06523531137287| 0.48909501589608| -0.143772296441519| 0.635558093258208| 0.463917041022171| -0.114804663102346| -0.183361270123994| -0.145783041325259|-0.0690831352230203| -0.225775248033138| -0.638671952771851| 0.101288021253234| -0.339846475529127| 0.167170404418143| 0.125894532368176|-0.00898309914322813| 0.0147241691924927| 2.69| 1.0|\n", - "| 1.0| -1.35835406159823| -1.34016307473609| 1.77320934263119| 0.379779593034328| -0.503198133318193| 1.80049938079263| 0.791460956450422| 0.247675786588991| -1.51465432260583| 0.207642865216696| 0.624501459424895| 0.066083685268831| 0.717292731410831| -0.165945922763554| 2.34586494901581| -2.89008319444231| 1.10996937869599| -0.121359313195888| -2.26185709530414| 0.524979725224404| 0.247998153469754| 0.771679401917229| 0.909412262347719| -0.689280956490685| -0.327641833735251| -0.139096571514147| -0.0553527940384261|-0.0597518405929204|378.66| 1.0|\n", - "| 1.0|-0.966271711572087| -0.185226008082898| 1.79299333957872| -0.863291275036453|-0.0103088796030823| 1.24720316752486| 0.23760893977178| 0.377435874652262| -1.38702406270197|-0.0549519224713749|-0.226487263835401| 0.178228225877303| 0.507756869957169| -0.28792374549456| -0.631418117709045| -1.0596472454325| -0.684092786345479| 1.96577500349538| -1.2326219700892| -0.208037781160366| -0.108300452035545|0.00527359678253453| -0.190320518742841| -1.17557533186321| 0.647376034602038| -0.221928844458407| 0.0627228487293033| 0.0614576285006353| 123.5| 1.0|\n", - "| 2.0| -1.15823309349523| 0.877736754848451| 1.548717846511| 0.403033933955121| -0.407193377311653| 0.0959214624684256| 0.592940745385545| -0.270532677192282| 0.817739308235294| 0.753074431976354|-0.822842877946363| 0.53819555014995| 1.3458515932154| -1.11966983471731| 0.175121130008994| -0.451449182813529| -0.237033239362776|-0.0381947870352842| 0.803486924960175| 0.408542360392758|-0.00943069713232919| 0.79827849458971| -0.137458079619063| 0.141266983824769| -0.206009587619756| 0.502292224181569| 0.219422229513348| 0.215153147499206| 69.99| 1.0|\n", - "| 2.0|-0.425965884412454| 0.960523044882985| 1.14110934232219| -0.168252079760302| 0.42098688077219|-0.0297275516639742| 0.476200948720027| 0.260314333074874| -0.56867137571251| -0.371407196834471| 1.34126198001957| 0.359893837038039| -0.358090652573631| -0.137133700217612| 0.517616806555742| 0.401725895589603| -0.0581328233640131| 0.0686531494425432|-0.0331937877876282| 0.0849676720682049| -0.208253514656728| -0.559824796253248|-0.0263976679795373| -0.371426583174346| -0.232793816737034| 0.105914779097957| 0.253844224739337| 0.0810802569229443| 3.67| 1.0|\n", - "| 4.0| 1.22965763450793| 0.141003507049326|0.0453707735899449| 1.20261273673594| 0.191880988597645| 0.272708122899098|-0.00515900288250983| 0.0812129398830894| 0.464959994783886|-0.0992543211289237| -1.41690724314928|-0.153825826253651| -0.75106271556262| 0.16737196252175| 0.0501435942254188| -0.443586797916727| 0.00282051247234708| -0.61198733994012|-0.0455750446637976| -0.21963255278686| -0.167716265815783| -0.270709726172363| -0.154103786809305| -0.780055415004671| 0.75013693580659| -0.257236845917139| 0.0345074297438413|0.00516776890624916| 4.99| 1.0|\n", - "| 7.0|-0.644269442348146| 1.41796354547385| 1.0743803763556| -0.492199018495015| 0.948934094764157| 0.428118462833089| 1.12063135838353| -3.80786423873589| 0.615374730667027| 1.24937617815176|-0.619467796121913| 0.291474353088705| 1.75796421396042| -1.32386521970526| 0.686132504394383|-0.0761269994382006| -1.2221273453247| -0.358221569869078| 0.324504731321494| -0.156741852488285| 1.94346533978412| -1.01545470979971| 0.057503529867291| -0.649709005559993| -0.415266566234811|-0.0516342969262494| -1.20692108094258| -1.08533918832377| 40.8| 1.0|\n", - "| 7.0| -0.89428608220282| 0.286157196276544|-0.113192212729871| -0.271526130088604| 2.6695986595986| 3.72181806112751| 0.370145127676916| 0.851084443200905|-0.392047586798604| -0.410430432848439|-0.705116586646536|-0.110452261733098| -0.286253632470583| 0.0743553603016731| -0.328783050303565| -0.210077268148783| -0.499767968800267| 0.118764861004217| 0.57032816746536| 0.0527356691149697| -0.0734251001059225| -0.268091632235551| -0.204232669947878| 1.0115918018785| 0.373204680146282| -0.384157307702294| 0.0117473564581996| 0.14240432992147| 93.2| 1.0|\n", - "| 9.0| -0.33826175242575| 1.11959337641566| 1.04436655157316| -0.222187276738296| 0.49936080649727| -0.24676110061991| 0.651583206489972| 0.0695385865186387|-0.736727316364109| -0.366845639206541| 1.01761446783262| 0.836389570307029| 1.00684351373408| -0.443522816876142| 0.150219101422635| 0.739452777052119| -0.540979921943059| 0.47667726004282| 0.451772964394125| 0.203711454727929| -0.246913936910008| -0.633752642406113| -0.12079408408185| -0.385049925313426|-0.0697330460416923| 0.0941988339514961| 0.246219304619926| 0.0830756493473326| 3.68| 1.0|\n", - "|10.0| 1.44904378114715| -1.17633882535966| 0.913859832832795| -1.37566665499943| -1.97138316545323| -0.62915213889734| -1.4232356010359| 0.0484558879088564| -1.72040839292037| 1.62665905834133| 1.1996439495421|-0.671439778462005| -0.513947152539479|-0.0950450453999549| 0.230930409124119| 0.0319674667862076| 0.253414715863197| 0.854343814324194| -0.221365413645481| -0.387226474431156|-0.00930189652490052| 0.313894410791098| 0.0277401580170247| 0.500512287104917| 0.25136735874921| -0.129477953726618| 0.0428498709381461| 0.0162532619375515| 7.8| 1.0|\n", - "|10.0| 0.38497821518095| 0.616109459176472|-0.874299702595052|-0.0940186259679115| 2.92458437838817| 3.31702716826156| 0.470454671805879| 0.53824722837695|-0.558894612428441| 0.30975539423728|-0.259115563735702|-0.326143233995877|-0.0900467227020648| 0.362832368569793| 0.928903660629178| -0.129486811402759| -0.809978925963589| 0.359985390219981| 0.70766382644648| 0.12599157561542| 0.049923685888971| 0.238421512225103|0.00912986861262866| 0.996710209581086| -0.767314827174801| -0.492208295340017| 0.042472441919027|-0.0543373883732122| 9.99| 1.0|\n", - "|10.0| 1.249998742053| -1.22163680921816| 0.383930151282291| -1.23489868766892| -1.48541947377961| -0.753230164566149| -0.689404975426345| -0.227487227519552| -2.09401057344842| 1.32372927445937| 0.227666231237246|-0.242681998944186| 1.20541680770748| -0.317630527025074| 0.725674990179153| -0.815612186027305| 0.873936447614439| -0.847788598847099| -0.683192626267037| -0.102755941505071| -0.231809239223849| -0.483285330117712| 0.0846676908596583| 0.392830885335013| 0.161134553588505| -0.354990039673962| 0.0264155490776107| 0.0424220887282304| 121.5| 1.0|\n", - "|11.0| 1.0693735878819| 0.287722129331455| 0.828612726634281| 2.71252042961718| -0.178398016248009| 0.337543730282968| -0.0967168617395962| 0.115981735546597|-0.221082566236194| 0.460230444301678|-0.773656930526689| 0.32338724546722|-0.0110758870883779| -0.178485175177916| -0.65556427824926| -0.19992517131173| 0.1240054151819| -0.980496201537345| -0.982916082135047| -0.153197231044512| -0.0368755317335273| 0.0744124028162195|-0.0714074332998586| 0.104743752596029| 0.548264725394119| 0.104094153162781| 0.0214910583643189| 0.021293311477486| 27.5| 1.0|\n", - "|12.0| -2.7918547659339| -0.327770756658658| 1.64175016056605| 1.76747274389883| -0.136588446465306| 0.80759646826532| -0.422911389711497| -1.90710747624096| 0.755712908314791| 1.1510869876677| 0.844555470974377| 0.7929439518176| 0.370448092803246| -0.734975105820311| 0.406795710431001| -0.303057623825763| -0.155868714793874| 0.778265457041536| 2.22186801373788| -1.58212204356551| 1.15166304848789| 0.222181966098225| 1.02058620426601| 0.0283166513238872| -0.232746324289105| -0.23555721754117| -0.16477751177654|-0.0301536365592253| 58.8| 1.0|\n", - "|12.0|-0.752417042956605| 0.345485415344747| 2.05732291276727| -1.46864329840046| -1.1583936804082|-0.0778498291166733| -0.608581418236123|0.00360348436201849|-0.436166983515744| 0.747730827192802|-0.793980602837221|-0.770406728847129| 1.04762699748088| -1.06660368148653| 1.10695345662141| 1.66011355713381| -0.279265373246772| -0.419994141181313| 0.432535348618175| 0.263450864446125| 0.499624954671111| 1.35365048557231| -0.256573280448308|-0.0650837078816517|-0.0391243535426488|-0.0870864732146962| -0.180997500092721| 0.129394059390202| 15.99| 1.0|\n", - "|12.0| 1.10321543528383|-0.0402962145973447| 1.2673320885949| 1.28909146962552| -0.735997163604068| 0.288069162976262| -0.586056786337461| 0.189379713679593| 0.782332891785191| -0.267975066537173|-0.450311279515466| 0.936707714991982| 0.708380406186981| -0.468647287707221| 0.354574063407955| -0.246634655717582|-0.00921237772707382| -0.595912405700819| -0.57568162226261| -0.113910176982092| -0.0246120063374677| 0.196001952806192| 0.0138016541409422| 0.103758331023198| 0.364297540595235| -0.382260574113217| 0.092809187460487| 0.0370505169810008| 12.99| 1.0|\n", - "|13.0|-0.436905071360625| 0.918966212909322| 0.92459077438817| -0.727219053596792| 0.915678718106307| -0.127867352079254| 0.707641607333935| 0.0879623554672504| -0.66527135413364| -0.737979823596458| 0.32409781346169| 0.277192107214981| 0.252624256310781| -0.291896460370468| -0.184520169327133| 1.14317370716197| -0.92870926272403| 0.680469592634687| 0.0254364616880793|-0.0470212823165035| -0.194795823794671| -0.672637997017793| -0.156857514491897| -0.888386320943716| -0.342413218776576| -0.049026728633951| 0.0796923991551505| 0.131023789452311| 0.89| 1.0|\n", - "|14.0| -5.40125766315825| -5.45014783420644| 1.18630463143652| 1.73623880012095| 3.04910587764025| -1.76340557365201| -1.55973769907953| 0.160841747266769| 1.23308974041888| 0.345172827050629| 0.917229867699146| 0.970116716069048| -0.266567764915222| -0.479129929276704| -0.526608502569153| 0.47200411177674| -0.725480944982201| 0.075081351540202| -0.406866573198217| -2.19684802485647| -0.503600328973703| 0.984459785590244| 2.45858857639219| 0.0421188969891572| -0.481630823956716| -0.621272013713977| 0.392053289557744| 0.949594245504846| 46.8| 1.0|\n", - "|15.0| 1.4929359769862| -1.02934573189487| 0.45479473374366| -1.43802587991702| -1.55543410136344| -0.720961147043557| -1.08066413038614|-0.0531271179483221| -1.9786815953872| 1.63807603690446| 1.07754241162743| -0.63204651464934| -0.41695716661602| 0.0520105153724404|-0.0429789228232019| -0.166432496451972| 0.304241418614353| 0.554432499062278| 0.0542295152184719| -0.387910172646258| -0.177649846438814| -0.175073809074822| 0.0400022190621329| 0.295813862676508| 0.33293059939425| -0.220384850672322| 0.0222984359135846|0.00760225559997897| 5.0| 1.0|\n", - "+----+------------------+-------------------+------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------------------+-------------------+------------------+------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------+-----+\n", - "only showing top 20 rows\n", - "\n" - ] - } - ], + "outputs": [], "source": [ - "cc_training.show()" + "cc_training.show(1)" ] }, { @@ -274,7 +254,9 @@ { "cell_type": "code", "execution_count": 6, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "# get the time to split the data.\n", @@ -478,22 +460,12 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": null, "metadata": { + "collapsed": true, "scrolled": false }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAugAAAK7CAYAAACgQHrJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3XucXXV97//XZ2ZygwTCZYCE+zUK\niBEQUaskXgEVvNWCrYK9UM/R369aa9XiT63Wo22t2ovVQ48c8VSRVKtNFRW8jOgRq6AJ90uIQWBC\nAoTcrzPz+f2x1yR7JjPJJOyZvdbK6/l47Mfs/V1rr/3dn+yZvOc73/VdkZlIkiRJKoeOdndAkiRJ\n0g4GdEmSJKlEDOiSJElSiRjQJUmSpBIxoEuSJEklYkCXJEmSSsSALkn7iIhYFhEvaXc/JEm7ZkCX\nJEmSSsSALkmSJJWIAV2S9jERMSUiPh0RvcXt0xExpdh2aER8MyJWR8SqiPhxRHQU294TEY9ExLqI\nuDciXtzedyJJ9dTV7g5IkibclcC5wFwggf8A3g/8f8C7gIeB7mLfc4GMiDnA24FnZ2ZvRBwHdE5s\ntyVp3+AIuiTte34X+HBmrszMx4C/BN5UbNsGzAKOzcxtmfnjzEygH5gCnBoRkzJzWWY+0JbeS1LN\nGdAlad8zG3iw6fGDRRvA3wJLgBsiYmlEvBcgM5cA7wA+BKyMiK9ExGwkSS1nQJekfU8vcGzT42OK\nNjJzXWa+KzNPAF4F/OngXPPM/HJm/lbx3AT+emK7LUn7BgO6JO17rgXeHxHdEXEo8AHgXwEi4pUR\ncVJEBLCWxtSW/oiYExEvKk4m3QxsKrZJklrMgC5J+56/Am4BbgNuB35ZtAGcDHwPWA/cDPxzZvbQ\nmH/+ceBx4FHgMOAvJrTXkrSPiMa5P5IkSZLKwBF0SZIkqUQM6JIkSVKJGNAlSZKkEjGgS5IkSSXS\n1e4OjKeZM2fmSSed1O5u1MaGDRvYf//9292NWrCWrWMtW8t6to61bB1r2TrWsrWa63nrrbc+npnd\nrThurQP64Ycfzi233NLubtRGT08P8+bNa3c3asFato61bC3r2TrWsnWsZetYy9ZqrmdEPLjrvcfO\nKS6SJElSiRjQJUmSpBIxoEuSJEklYkCXJEmSSsSALkmSJJWIAV2SJEkqEQO6JEmSVCIGdEmSJKlE\nDOiSJElSiRjQJUmSpBIxoEuSJEklYkBX5Vz789/wuR890O5uSJIkjYuudndA2lPv+/fbAXjreSe2\nuSeSJEmt5wi6JEmSVCIGdEmSJKlEDOiSJElSiRjQJUmSpBIxoEuSJEklYkCXJEmSSsSALkmSJJWI\nAV2SJEkqEQO6JEmSVCIGdEmSJKlEDOiSJElSiRjQJUmSpBJpW0CPiKsjYmVE3NHUdl1ELCpuyyJi\nUdF+XERsatr2uXb1W5IkSRpPXW187S8A/wR8cbAhM39n8H5E/B2wpmn/BzJz7oT1TpIkSWqDtgX0\nzLwpIo4baVtEBPAG4EUT2SdJkiSp3SIz2/fijYD+zcw8fVj7C4FPZubZTfvdCdwHrAXen5k/HuWY\nVwBXAHR3d5+1YMGCcer9vmf9+vVMnz693d3g8u9sAOAL5+/f5p7svbLUsg6sZWtZz9axlq1jLVvH\nWrZWcz3nz59/62B2faraOcVlVy4Frm16vBw4JjOfiIizgG9ExGmZuXb4EzPzKuAqgDlz5uS8efMm\nor/7hJ6eHkpRz+98C6AcfdlLpallDVjL1rKerWMtW8dato61bK3xqmfpVnGJiC7gtcB1g22ZuSUz\nnyju3wo8AJzSnh5KkiRJ46d0AR14CXBPZj482BAR3RHRWdw/ATgZWNqm/kmSJEnjpp3LLF4L3AzM\niYiHI+IPik2XMHR6C8ALgdsiYjHwVeCtmblq4norSZIkTYx2ruJy6Sjtl4/Q9jXga+PdJ0mSJKnd\nyjjFRZIkSdpnGdAlSZKkEjGgS5IkSSViQJckSZJKxIAuSZIklYgBXZIkSSoRA7okSZJUIgZ0SZIk\nqUQM6JIkSVKJGNAlSZKkEjGgS5IkSSViQJckSZJKxIAuSZIklYgBXZIkSSoRA7okSZJUIgZ0SZIk\nqUQM6JIkSVKJGNAlSZKkEjGgS5IkSSViQJckSZJKxIAuSZIklYgBXZIkSSoRA7okSZJUIgZ0SZIk\nqUQM6JIkSVKJGNAlSZKkEjGgS5IkSSXS1oAeEVdHxMqIuKOp7UMR8UhELCpuFzZte19ELImIeyPi\n5e3ptSRJkjR+2j2C/gXg/BHaP5WZc4vb9QARcSpwCXBa8Zx/jojOCeupJEmSNAHaGtAz8yZg1Rh3\nvxj4SmZuycxfA0uAc8atc5IkSVIbtHsEfTRvj4jbiikwBxVtRwIPNe3zcNEmSZIk1UZkZns7EHEc\n8M3MPL14fDjwOJDAR4BZmfn7EfEZ4ObM/Ndiv88D12fm14Yd7wrgCoDu7u6zFixYMFFvpfbWr1/P\n9OnT290NLv/OBgC+cP7+be7J3itLLevAWraW9Wwda9k61rJ1rGVrNddz/vz5t2bm2a04blcrDtJK\nmbli8H5E/AvwzeLhw8DRTbseBfSO8PyrgKsA5syZk/PmzRu3vu5renp6KEU9v/MtgHL0ZS+VppY1\nYC1by3q2jrVsHWvZOtaytcarnqWb4hIRs5oevgYYXOFlIXBJREyJiOOBk4GfT3T/JEmSpPHU1hH0\niLgWmAccGhEPAx8E5kXEXBpTXJYBfwyQmXdGxALgLqAPeFtm9rej35IkSdJ4aWtAz8xLR2j+/C72\n/yjw0fHrkaokM4mIdndDkiSppUo3xUUaqzaf3yxJkjQuDOiqLPO5JEmqIwO6KqvdS4RKkiSNBwO6\nJEmSVCIGdFWW4+eSJKmODOiqLGe4SJKkOjKgq7LSMXRJklRDBnRVliPokiSpjgzokiRJUokY0FVZ\njqBLkqQ6MqCrspyDLkmS6siArspyBF2SJNWRAV2VZT6XJEl1ZEBXZaVD6JIkqYYM6Kos47kkSaoj\nA7oqywF0SZJURwZ0SZIkqUQM6KouR9AlSVINGdBVWa6DLkmS6siArspyDrokSaojA7oqy3wuSZLq\nyICuynIddEmSVEcGdFWW8VySJNWRAV2V5QC6JEmqIwO6KstVXCRJUh0Z0FVd5nNJklRDBnRVlvlc\nkiTVUdsCekRcHRErI+KOpra/jYh7IuK2iPh6RMws2o+LiE0Rsai4fa5d/VZ5OAddkiTVUTtH0L8A\nnD+s7Ubg9Mw8A7gPeF/Ttgcyc25xe+sE9VGSJEmaUG0L6Jl5E7BqWNsNmdlXPPwZcNSEd0yV4Umi\nkiSpjso8B/33gW83PT4+In4VET+KiBe0q1MqD6e4SJKkOop2Xo0xIo4DvpmZpw9rvxI4G3htZmZE\nTAGmZ+YTEXEW8A3gtMxcO8IxrwCuAOju7j5rwYIF4/wu9h3r169n+vTp7e4Gl39nAwB/d940DplW\n5t8xR1eWWtaBtWwt69k61rJ1rGXrWMvWaq7n/Pnzb83Ms1tx3K5WHKSVIuIy4JXAi7P47SEztwBb\nivu3RsQDwCnALcOfn5lXAVcBzJkzJ+fNmzdBPa+/np4eSlHP73wLgOecey5HHbRfmzuzd0pTyxqw\nlq1lPVvHWraOtWwda9la41XPUg0/RsT5wHuAizJzY1N7d0R0FvdPAE4GlranlyoLp7hIkqQ6atsI\nekRcC8wDDo2Ih4EP0li1ZQpwY0QA/KxYseWFwIcjog/oB96amatGPLAkSZJUYW0L6Jl56QjNnx9l\n368BXxvfHkmSJEntV6opLtKecIqLJEmqIwO6JEmSVCIGdFWWFyqSJEl1ZECXJEmSSsSArspyDrok\nSaojA7okSZJUIgZ0VZYD6JIkqY4M6JIkSVKJGNBVWekkdEmSVEMGdEmSJKlEDOiqLMfPJUlSHRnQ\nJUmSpBIxoEuSJEklYkBXZXmOqCRJqiMDuiRJklQiBnRVmEPokiSpfgzokiRJUokY0FVZzkGXJEl1\nZECXJEmSSsSArspyAF2SJNWRAV2SJEkqEQO6Kss56JIkqY4M6JIkSVKJGNBVWeksdEmSVEMGdEmS\nJKlEDOiqLOegS5KkOjKgS5IkSSViQFdlOYIuSZLqqK0BPSKujoiVEXFHU9vBEXFjRNxffD2oaI+I\n+IeIWBIRt0XEme3ruSRJkjQ+2j2C/gXg/GFt7wW+n5knA98vHgNcAJxc3K4APjtBfVRJuYqLJEmq\no7YG9My8CVg1rPli4Jri/jXAq5vav5gNPwNmRsSsiempJEmSNDEi2zyRNyKOA76ZmacXj1dn5sym\n7U9m5kER8U3g45n5k6L9+8B7MvOWYce7gsYIO93d3WctWLBgYt7IPmD9+vVMnz693d3g8u9sAOAv\nnzeVYw/obHNv9k5ZalkH1rK1rGfrWMvWsZatYy1bq7me8+fPvzUzz27FcbtacZAJEiO07fTbRWZe\nBVwFMGfOnJw3b944d2vf0dPTQynq+Z1vAXDWWWdz+pEHtrkze6c0tawBa9la1rN1rGXrWMvWsZat\nNV71bPcc9JGsGJy6UnxdWbQ/DBzdtN9RQO8E902SJEkaV2UM6AuBy4r7lwH/0dT+5mI1l3OBNZm5\nvB0dlCRJksZLW6e4RMS1wDzg0Ih4GPgg8HFgQUT8AfAb4LeL3a8HLgSWABuBt0x4h1UqroMuSZLq\nqK0BPTMvHWXTi0fYN4G3jW+PJEmSpPYq4xQXaUxcB12SJNWRAV2SJEkqEQO6Kss56JIkqY4M6JIk\nSVKJGNAlSZKkEjGgq7Kc4SJJkurIgC5JkiSViAFdlZWeJSpJkmrIgC5JkiSViAFdleX4uSRJqiMD\nuiRJklQiBnRVllPQJUlSHRnQJUmSpBIxoKvCHEKXJEn1Y0CXJEmSSsSArspyDrokSaojA7okSZJU\nIgZ0VZYD6JIkqY4M6JIkSVKJGNBVWc5BlyRJdWRAV6WkqVySJNWcAV2VZViXJEl1ZEBXpZjJJUlS\n3RnQVVlmdUmSVEcGdFWKoVySJNWdAV2V5XQXSZJURwZ0VYonhkqSpLrrancHRhIRc4DrmppOAD4A\nzAT+CHisaP+LzLx+grunkkgnvEiSpBoqZUDPzHuBuQAR0Qk8AnwdeAvwqcz8RBu7pzYykkuSpLqr\nwhSXFwMPZOaD7e6ISsa0LkmSaijKPqc3Iq4GfpmZ/xQRHwIuB9YCtwDvyswnh+1/BXAFQHd391kL\nFiyY2A7X2Pr165k+fXpb+9A3kPzhDRsB+PNnT+XUQzrb2p+9VYZa1oW1bC3r2TrWsnWsZetYy9Zq\nruf8+fNvzcyzW3HcUgf0iJgM9AKnZeaKiDgceJzG2OlHgFmZ+fujPX/OnDl57733Tkxn9wE9PT3M\nmzevrX3Y2jfAKe//NgBf+sPn8PyTDm1rf/ZWGWpZF9aytaxn61jL1rGWrWMtW6u5nhHRsoBe9iku\nF9AYPV8BkJkrMrM/MweAfwHOaWvvNOE8MVSSJNVd2QP6pcC1gw8iYlbTttcAd0x4jyRJkqRxVMpV\nXAAiYj/gpcAfNzX/TUTMpTHFZdmwbdoHNM/IKvHsLEmSpL1W2oCemRuBQ4a1valN3ZEkSZImRNmn\nuEijcj66JEmqIwO6JEmSVCIGdFWKc9AlSVLdGdAlSZKkEjGgq1Ka5507gC5JkurIgK5K6RswlkuS\npHozoKtS+vqbRtCLSei3LFvFus3b2tUlSZKkljKgq1K29Q8Mebxm4zZe/7mb+X+u/VWbeiRJktRa\nBnRVSnNAT2BtMXJ+/4r1beqRJElSaxnQVSnb+ofOQd+0rR+AaZM729EdSZKkljOgq1L6mqe4JGza\n2gjo+xnQJUlSTRjQVSmjjaBP7TKgS5KkejCgq1KGzkHPHQHdEXRJklQTBnRVSt/A0FVctgzOQZ/k\nR1mSJNWDqUaVsrWveR102Lh1MKA7gi5JkurBgK5KGT6CPjjlpavTj7IkSaoHU40qZcgc9GzcADoj\n2tQjSZKk1jKgq1KGr+LSXyT0Dj/JkiSpJow1qpThVxIdGGgE9HAEXZIk1YQBXZXSN3wEvQjoHeZz\nSZJUEwZ0VcrQOejJYF7vcARdkiTVhAFdlZLDHg9sH0E3oEuSpHowoKtacujdwZNEzeeSJKkuDOiq\nlGS0OegmdEmSVA8GdFVK5tD7A54kKkmSasaArkoZPgd9xxQXE7okSaoHA7oqJYcl9IHhiV2SJKni\nutrdgdFExDJgHdAP9GXm2RFxMHAdcBywDHhDZj7Zrj5q4g2dg57bp7jk8OQuSZJUUWUfQZ+fmXMz\n8+zi8XuB72fmycD3i8fahwzP4YNTXMznkiSpLsoe0Ie7GLimuH8N8Oo29kVt0DxS3nySqPlckiTV\nRZR1akBE/Bp4kkb2+p+ZeVVErM7MmU37PJmZBw173hXAFQDd3d1nLViwYCK7XWvr169n+vTpbe3D\n9x7cxr/evRWAt82dwpLV/Xx3WR8vO7aLNz59Slv7tifKUMu6sJatZT1bx1q2jrVsHWvZWs31nD9/\n/q1Nsz6ektLOQQeen5m9EXEYcGNE3DOWJ2XmVcBVAHPmzMl58+aNYxf3LT09PbS7ng/+dBncfScA\np512GhuXrYJlyzjyqKOYN++0tvZtT5ShlnVhLVvLeraOtWwda9k61rK1xquepZ3ikpm9xdeVwNeB\nc4AVETELoPi6sn09VDsM/4vPjpNE29EbSZKk1itlQI+I/SNixuB94GXAHcBC4LJit8uA/2hPD9Uu\nQ9ZwyR0niUqSJNVFWae4HA58vbj4TBfw5cz8TkT8AlgQEX8A/Ab47Tb2UW2w0youA4PtBnVJklQP\npQzombkUeOYI7U8AL574Hqkshq6Cnq7iIkmSaqeUU1yk0QwfKXcddEmSVDcGdFXW0HXQTeiSJKke\nDOiqlOEj5QOOoEuSpJoxoKtSmkfKE+gvHg4Y0CVJUk0Y0FUpO42gb0/mJnRJklQPBnRVytB10JN+\nL1QkSZJqxoCuStlpHXTnoEuSpJoxoKtShq/W4ioukiSpbgzoqhRH0CVJUt0Z0FVZmbBl20Djfpv7\nIkmS1CoGdFXK8CuJbunrL9rb0RtJkqTWM6CrUpqDeJJs3j6CbkKXJEn1YEBXpQyP4YMj6OZzSZJU\nFwZ0VcqQEfSELX3OQZckSfViQFelDJ/Ksn2Ki5PQJUlSTRjQVSk7j6AXJ4m2qT+SJEmtZkBXpey8\nisvgCHo7eiNJktR6BnRVSnMOH8hkaxHQB0zokiSpJgzoqpTmHD44eg5OcZEkSfVhQFelNJ8k2j8w\nZFF0SZKkWjCgq1KaR9Cbp7V4oSJJklQXBnRVytA56E3t5nNJklQTBnRVytBlFnPEdkmSpCozoKtS\nmqeyOMVFkiTVkQFd1dKUw/sHmprN55IkqSYM6KqU4eugj9QuSZJUZQZ0VcrQeefOQZckSfVTuoAe\nEUdHxA8j4u6IuDMi/qRo/1BEPBIRi4rbhe3uqybe0GUWh2yZ6K5IkiSNi652d2AEfcC7MvOXETED\nuDUibiy2fSozP9HGvqnNmmN484WKHEGXJEl1UbqAnpnLgeXF/XURcTdwZHt7pbIYdZnFNvRFkiRp\nPESWeOgxIo4DbgJOB/4UuBxYC9xCY5T9yRGecwVwBUB3d/dZCxYsmKDe1t/69euZPn16W/vwpbu3\ncOODfQC88oRJfHPpNgDOOLSTPz17aju7tkfKUMu6sJatZT1bx1q2jrVsHWvZWs31nD9//q2ZeXYr\njlu6EfRBETEd+BrwjsxcGxGfBT5CY7D0I8DfAb8//HmZeRVwFcCcOXNy3rx5E9bnuuvp6aHd9exZ\neyc8uAyAo485BpY+AMBBBx/MvHnntLFne6YMtawLa9la1rN1rGXrWMvWsZatNV71LN1JogARMYlG\nOP9SZv47QGauyMz+zBwA/gWoThrTuHCZRUmSVEelC+gREcDngbsz85NN7bOadnsNcMdE903tN9rS\nimWeqiVJkrQnyjjF5fnAm4DbI2JR0fYXwKURMZfGYOky4I/b0z2102iruEiSJNVF6QJ6Zv4EiBE2\nXT/RfVH5DF0H3WUWJUlS/ZRuiou0K8koU1ychS5JkmrCgK5KcQRdkiTVnQFdldI87dwriUqSpDoy\noKtikijOUBhwioskSaohA7oqJRM6ioQ+2pKLkiRJVWZAV6Vk7ljixwsVSZKkOjKgq1KS3D6C3j8w\nZIMkSVItGNBVKZlsH0JvnuIy4BwXSZJUEwZ0VUoCHUVA73eKiyRJqiEDuiqlMQe9kdCHrOLiCLok\nSaoJA7oqpTEHvXF/cFpLhCPokiSpPgzoqpaEGLbMYmeEyyxKkqTaMKCrUprOEd1+JdGOjnAEXZIk\n1YYBXZWSufOVRDsjvFKRJEmqDQO6KiVpnuLSaOvsiCEnjEqSJFWZAV2VkslOJ4lO7uqgz4QuSZJq\nwoCuSmkeQd8e0Ds72DbksqKSJEnVZUBXpWTuWGZx8CTRyV0d9BnQJUlSTRjQVSnFyueN+8Wslsld\nHWzrd4qLJEmqBwO6KqOvfwBGmoPuFBdJklQjXe3ugDQW2/oHOPnKbwNw+AFTAE8SlSRJ9eQIuiph\nw5a+7fc7Bk8SLQbNp3Q5gi5JkurDgK5K2LStf/v9jVsb95tH0A3okiSpLgzoqoTN23YE8DWbtgE7\nAvqUrg76PElUkiTVhAFdlbBpa/9ObQNNq7j0DSSZhnRJklR9BnRVwua+kQL6jlVcAJdalCRJtWBA\nVyVsHnEEfcccdIC+AeehS5Kk6qtcQI+I8yPi3ohYEhHvbXd/NDGaTxIddMcja4EdAd0RdEmSVAeV\nCugR0Ql8BrgAOBW4NCJObW+vNBGaTxIdbnJnJ4AruUiSpFqo2oWKzgGWZOZSgIj4CnAxcNdIO6/a\nnHz0W3ex/5Qupk/pIiIIIILia2y/z5BtjfaO4j7N+w/u07Qfw7cRjec2Djzk9QYN3ovt+xWvVWwM\nGpeyT7LpOTueP+QgIz8c8nojbYcd00Qmde74XW34uZaDfbjj8X46739shO1Dbdraz5a+fg7ab/KQ\nOsVInRhh0Ls/k//141/zo/se47XPOpJnHXsQ9z26boTeNwyOoH/79uVMm9zF1r4BZkzt2qmuOx4z\n5PFgS1dH87/n0H+Poc+L7Y9He421m7Zx34r1POOoA7av297stsf64N6Vo76n0V6nrJq/FybaXU/0\nM2nJ49sfb+sfYMOWfg7ab1J7OjSSkv/7Nbv7iX6mPPDEqNtzpG/anXcakz35m9fwn7lVcM+qfqYu\nbdTSc9j3XPO/dXMttbM9+Xzt7ntce+aJTeMzOBhVWvkiIl4PnJ+Zf1g8fhPwnMx8+0j77z/75Ox+\n86cnsosaR087Ygaf+O1nsmlbP9OndPHJG+9jycr1vOMlJ/MnX1nU7u5JkqR9zOtPmcQnfv9lAETE\nrZl5diuOW7UR9JHGTob8hhERVwBXAHR3d3P1y/ejbwAGZ0gM/j6Sg0/Mwfu5/f727bnja/MLZY5y\nf8jxIDOH7DdSx7P5wbC2ISOnuxm13unwu9l/UBTbBobtMHyUKoCNmzax37Rpox5nUGdH4/HWkWqe\nIx97uMP3C6Z2Bau3JJ0BG7YlB0/r5/H7fwXABuB3jwGOgXzyPv7mhdPoK16vI6B/yD/KsH+jJoO/\noDbXIHf1vGHtQ/YZdvBJHTDa79WbNm5i2n4j13K012m3ZPRvwHb+nr9p0yamDftcbhto1L8MdlWa\nkb4f2m3jxp3rOdxY+tzqt9Xuz9neGOmzqb1jLXcY7WfxWFnL1to/N9HT09Py41YtoD8MHN30+Cig\nt3mHzLwKuApgzpw5+aL58yeudzXX09PDvHnz2t2NWrCWrWMtW8t6to61bB1r2TrWsrXGq54lGWMa\ns18AJ0fE8RExGbgEWNjmPkmSJEktU6kR9Mzsi4i3A98FOoGrM/PONndLkiRJaplKBXSAzLweuL7d\n/ZAkSZLGQ9WmuEiSJEm1ZkCXJEmSSsSALkmSJJWIAV2SJEkqEQO6JEmSVCIGdEmSJKlEDOiSJElS\niRjQJUmSpBKJzGx3H8ZNRKwD7m13P2rkUODxdneiJqxl61jL1rKerWMtW8dato61bK3meh6bmd2t\nOGjlriS6h+7NzLPb3Ym6iIhbrGdrWMvWsZatZT1bx1q2jrVsHWvZWuNVT6e4SJIkSSViQJckSZJK\npO4B/ap2d6BmrGfrWMvWsZatZT1bx1q2jrVsHWvZWuNSz1qfJCpJkiRVTd1H0CVJkqRKMaBLkiRJ\nJVLbgB4R50fEvRGxJCLe2+7+lF1EHB0RP4yIuyPizoj4k6L9QxHxSEQsKm4XNj3nfUV9742Il7ev\n9+UTEcsi4vaiZrcUbQdHxI0RcX/x9aCiPSLiH4pa3hYRZ7a39+USEXOaPn+LImJtRLzDz+bYRMTV\nEbEyIu5oatvjz2JEXFbsf39EXNaO99Juo9TybyPinqJeX4+ImUX7cRGxqenz+bmm55xV/HxYUtQ7\n2vF+2m2Ueu7x97X/349ay+ua6rgsIhYV7X42d2EXeWhif25mZu1uQCfwAHACMBlYDJza7n6V+QbM\nAs4s7s8A7gNOBT4E/NkI+59a1HUKcHxR7852v4+y3IBlwKHD2v4GeG9x/73AXxf3LwS+DQRwLvBf\n7e5/WW/F9/ajwLF+NsdcsxcCZwJ3NLXt0WcROBhYWnw9qLh/ULvfW0lq+TKgq7j/1021PK55v2HH\n+Tnw3KLO3wYuaPd7K1E99+j72v/vR6/lsO1/B3yguO9nc9e1HC0PTejPzbqOoJ8DLMnMpZm5FfgK\ncHGb+1Rqmbk8M39Z3F8H3A0cuYunXAx8JTO3ZOavgSU06q7RXQxcU9y/Bnh1U/sXs+FnwMyImNWO\nDlbAi4EHMvPBXezjZ7NJZt4ErBrWvKefxZcDN2bmqsx8ErgROH/8e18uI9UyM2/IzL7i4c+Ao3Z1\njKKeB2Tmzdn4X/yL7Kj/PmWUz+ZoRvu+9v97dl3LYhT8DcC1uzqGn82GXeShCf25WdeAfiTwUNPj\nh9l12FSTiDgOeBbwX0XT24s/21w9+CcdrPHuJHBDRNwaEVcUbYdn5nJo/AAADivareXYXcLQ/2T8\nbO6dPf0sWtOx+X0aI2mDjo/yM1LxAAAgAElEQVSIX0XEjyLiBUXbkTTqN8ha7mxPvq/9bO7eC4AV\nmXl/U5ufzTEYlocm9OdmXQP6SHOmXE9yDCJiOvA14B2ZuRb4LHAiMBdYTuPPZGCNd+f5mXkmcAHw\ntoh44S72tZZjEBGTgYuAfyua/Gy23mi1s6a7ERFXAn3Al4qm5cAxmfks4E+BL0fEAVjL3dnT72vr\nuXuXMnRgw8/mGIyQh0bddYS2p/zZrGtAfxg4uunxUUBvm/pSGRExicaH8UuZ+e8AmbkiM/szcwD4\nF3ZMFbDGu5CZvcXXlcDXadRtxeDUleLrymJ3azk2FwC/zMwV4GfzKdrTz6I13YXi5K9XAr9bTA2g\nmIrxRHH/VhrzpE+hUcvmaTDWsslefF/72dyFiOgCXgtcN9jmZ3P3RspDTPDPzboG9F8AJ0fE8cWo\n2yXAwjb3qdSKOWqfB+7OzE82tTfPhX4NMHiG+ELgkoiYEhHHAyfTOLlknxcR+0fEjMH7NE4iu4NG\nzQbP4r4M+I/i/kLgzcWZ4OcCawb/jKYhhowC+dl8Svb0s/hd4GURcVAx5eBlRds+LyLOB94DXJSZ\nG5vauyOis7h/Ao3P4dKinusi4tzi5+6b2VH/fd5efF/7//2uvQS4JzO3T13xs7lro+UhJvrn5nie\nCdvOG42zau+j8Zvhle3uT9lvwG/R+NPLbcCi4nYh8H+A24v2hcCspudcWdT3XvbBM713UcsTaKwk\nsBi4c/DzBxwCfB+4v/h6cNEewGeKWt4OnN3u91C2G7Af8ARwYFObn82x1e5aGn/S3kZjROcP9uaz\nSGN+9ZLi9pZ2v68S1XIJjXmmgz83P1fs+7ri+38x8EvgVU3HOZtG8HwA+CeKq3rva7dR6rnH39f+\nfz9yLYv2LwBvHbavn81d13K0PDShPzejOIAkSZKkEqjrFBdJkiSpkgzokiRJUokY0CVJkqQSMaBL\nkiRJJWJAlyRJkkrEgC5JkiSViAFdkiRJKhEDuiRJklQiBnRJkiSpRAzokiRJUokY0CVJkqQSMaBL\nkiRJJWJAlyRJkkrEgC5JkiSViAFdkiRJKhEDuiRJklQiBnRJkiSpRAzokiRJUokY0CVJkqQSMaBL\nkiRJJWJAlyRJkkrEgC5JkiSViAFdkiRJKhEDuiRJklQiBnRJkiSpRAzokiRJUokY0CVJkqQSMaBL\nkiRJJWJAl6QJFBHLIuIlLTrW5Ij4anHMjIh5rTiuJKm9DOiSVG0/AX4PeLTdHdmdiOhsdx8kqQoM\n6JI0QSLi/wDHAP8ZEesj4s+L9osi4s6IWB0RPRHx9KbnLIuI90XEXRHxZET874iYCpCZWzPz05n5\nE6B/DK//loi4OyLWRcTSiPjjYdsvjohFEbE2Ih6IiPOL9oOL1+0t+vCNov3yiPjJsGNkRJxU3P9C\nRHw2Iq6PiA3A/Ih4RUT8qniNhyLiQ8Oe/1sR8dOiFg8Vr/HsiFgREV1N+70uIhbtQfklqTIM6JI0\nQTLzTcBvgFdl5vTM/JuIOAW4FngH0A1cTyPAT2566u8CLwdOBE4B3r+XXVgJvBI4AHgL8KmIOBMg\nIs4Bvgi8G5gJvBBYVjzv/wD7AacBhwGf2oPXfCPwUWAGjdH+DcCbi9d4BfDfIuLVRR+OAb4N/CON\nWswFFmXmL4AngJc2Hff3in5JUu0Y0CWpvX4H+FZm3piZ24BPANOA5zXt80+Z+VBmrqIRdi/dmxfK\nzG9l5gPZ8CPgBuAFxeY/AK4u+jGQmY9k5j0RMQu4AHhrZj6ZmduK547Vf2Tm/y2OuTkzezLz9uLx\nbTR+OTmv2Pd3ge9l5rXF6zyRmYOj5NfQCOVExME0fmH58t7UQZLKzoAuSe01G3hw8EFmDgAPAUc2\n7fNQ0/0Hi+fssYi4ICJ+FhGrImI1cCFwaLH5aOCBEZ52NLAqM5/cm9dkaN+JiOdExA8j4rGIWAO8\ndQx9APhX4FURMR14A/DjzFy+l32SpFIzoEvSxMphj3uBYwcfRETQCKqPNO1zdNP9Y4rn7JGImAJ8\njcYI/eGZOZPGdJoodnmIxhSa4R4CDo6ImSNs20Bj6svgaxwxwj7D3++XgYXA0Zl5IPC5MfSBzHwE\nuBl4DfAmnN4iqcYM6JI0sVYAJzQ9XgC8IiJeHBGTgHcBW4CfNu3ztog4qpja8RfAdYMbImLK4Emj\nwOSImFqE/OEmA1OAx4C+iLgAeFnT9s8Dbyn60RERR0bE04pR6m8D/xwRB0XEpIh4YfGcxcBpETG3\n6MOHxvD+Z9AYkd9czHt/Y9O2LwEviYg3RERXRBwSEXObtn8R+HPgGcDXx/BaklRJBnRJmlgfA95f\nrFLyZ5l5L4251f8IPA68isZJpFubnvNlGvPFlxa3v2radi+wicaUmO8W949lmMxcB/y/NH4heJJG\nMF7YtP3nFCeOAmuAHzUd503ANuAeGieavqN4zn3Ah4HvAffTOAl0d/478OGIWAd8oOjPYB9+Q2Pa\nzbuAVcAi4JlNz/160aevZ+aGMbyWJFVSZA7/66MkqSwiYhnwh5n5vXb3pQwi4gHgj62HpDpzBF2S\nVAkR8Toac9p/0O6+SNJ46tr9LpIktVdE9ACnAm8qVrqRpNpyioskSZJUIk5xkSRJkkqk1lNcZs6c\nmSeddFK7u1EbGzZsYP/99293N2rDeraW9Wwda9la1rO1rGfrWMvWuvXWWx/PzO5WHKvWAf3www/n\nlltuaXc3aqOnp4d58+a1uxu1YT1by3q2jrVsLevZWtazdaxla0XEg7vfa2yc4iJJkiSViAFdkiRJ\nKhEDuiRJklQiBnRJkiSpRAzokiRJUokY0CVJkqQSMaBLkiRJJWJAlyRJkkrEgC5JkiSViAFdkiRJ\nKhEDuiRJklQiBvSa2NY/wCv/8cfceNeKdndFkiRJT4EBvSYeXbOZOx5Zy8+WPtHurkiSJOkpMKDX\nxCOrNwHQW3yVJElSNRnQa2L5miKgr9nc5p5IkiTpqTCg10Tv6s3FV0fQJUmSqsyAXhODU1weW7eF\nLX39be6NJEmS9pYBvSaWN42cr1izpY09kSRJ0lNhQK+J3tWbmTGlC9gxmi5JkqTqMaDXRO/qTTzr\n2IO235ckSVI1jSmgR8T5EXFvRCyJiPeOsP3YiPh+RNwWET0RcVTTtr+OiDuK2+80tb8oIn5ZtF8T\nEV1Fe0TEPxSvdVtEnNn0nP6IWFTcFj61t14fazdvY92WPs46phHQB1d0kSRJUvXsNqBHRCfwGeAC\n4FTg0og4ddhunwC+mJlnAB8GPlY89xXAmcBc4DnAuyPigIjoAK4BLsnM04EHgcuKY10AnFzcrgA+\n2/Q6mzJzbnG7aG/ecB0tL1ZwOaF7fw7ZfzKPrHapRUmSpKoaywj6OcCSzFyamVuBrwAXD9vnVOD7\nxf0fNm0/FfhRZvZl5gZgMXA+cAiwJTPvK/a7EXhdcf9iGmE/M/NnwMyImLUX722fMTilZfbMacya\nOdUpLpIkSRXWNYZ9jgQeanr8MI3R8GaLaQTsvwdeA8yIiEOK9g9GxCeB/YD5wF3A48CkiDg7M28B\nXg8cvYvXOxJYDkyNiFuAPuDjmfmN4Z2NiCtojLzT3d1NT0/PGN5itfX8ZhsAD979KyZv28qS3nXj\n8r7Xr1+/T9RzoljP1rKerWMtW8t6tpb1bB1rWV5jCegxQlsOe/xnwD9FxOXATcAjQF9m3hARzwZ+\nCjwG3Fy0Z0RcAnwqIqYAN9AI3bt7vWMyszciTgB+EBG3Z+YDQ3bMvAq4CmDOnDk5b968MbzFavvF\nd++h856lXPyy+dy29S7+7ZaHOO+884gYqZR7r6enh32hnhPFeraW9Wwda9la1rO1rGfrWMvyGssU\nl4fZMboNcBTQ27xDZvZm5msz81nAlUXbmuLrR4s54y+lEb7vL9pvzswXZOY5NEL9/bt7vcwc/LoU\n6AGeNfa3Wl+9qzdzxAFT6ewIZs+cyoat/azd3Lf7J0qSJKl0xhLQfwGcHBHHR8Rk4BJgyAoqEXFo\nceInwPuAq4v2zmKqCxFxBnAGjdFyIuKw4usU4D3A54rnLwTeXKzmci6wJjOXR8RBxb5ExKHA82lM\nl9nn9a7exOyZU4HGPHRwJRdJkqSq2u0Ul8zsi4i3A98FOoGrM/POiPgwcEtmLgTmAR+LiKQxGv62\n4umTgB8XUy3WAr+XmYNDu++OiFfS+CXhs5n5g6L9euBCYAmwEXhL0f504H9GxEDxnI9npgEd6F2z\niTOLJRYHA3rv6k087YgD2tktSZIk7YWxzEEnM6+nEZyb2z7QdP+rwFdHeN5mGiu5jHTMdwPvHqE9\n2RHwm9t/CjxjLP3dl/QPJI+u2bw9mM8+sPHVpRYlSZKqySuJVtzj67ewrT+ZfWBjikv3jCl0dQTL\nXWpRkiSpkgzoFde8BjpAZ0dwxIGuhS5JklRVBvSK6y2msgwGdGhMc+l1ioskSVIlGdArbvsI+oFN\nAX3mVHpdxUWSJKmSDOgV17tmE/tP7uSAaTvO9509cxqPrtlM/8Dw60lJkiSp7AzoFddYA33akKuG\nzpo5jb6B5LF1W9rYM0mSJO0NA3rF9a7ezKym+ecARxYXLXKaiyRJUvUY0Ctu+ZpN2wP5oOaLFUmS\nJKlaDOgVtnlbP4+v3zrkBFGAWQca0CVJkqrKgF5hy9fsvMQiwAFTu5g+pculFiVJkirIgF5hg1cL\nnTVsiktENJZaHMcR9M3b+sft2JIkSfsyA3qFPVIE8COHjaBDY5rLeJ0kesOdj/KMD32XRQ+tHpfj\nS5Ik7csM6BU2OIXliAOn7rRt9sxpLB+HKS4r123mvf9+O9v6k6/e+lDLjy9JkrSvM6BX2PI1mzh0\n+hSmdHXutO3ImVN5YsPWlk5FyUze89Xb2LCljzOPmcm3blvOtv6Blh1fkiRJBvRKe2T1zkssDhqP\nlVx6Hurjh/c+xvsueBpvPe9Enty4jZ8sebxlx5ckSZIBvdIGryI6ksH2wZVenqqlj63n2nu38oKT\nD+XNzz2O8+Z0c8DULv5zUW9Lji9JkqQGA3pFZSbL12zePlI+3OCJo4+0YAS9r3+Ady5YzKQO+NvX\nP5OOjmBKVycXnD6L7975qCu6SJIktZABvaLWbNrGxq39zB5lisvhB04BWjPF5TM/fIDFD63mslOn\nDDkh9aK5s9mwtZ8f3LPyKb+GJEmSGgzoFbWrJRYBpnR10j1jylNeyWXRQ6v5hx/cz6vnzuacWV1D\ntp17wiF0z5jCQqe5SJIktYwBvaIGg/esUQI6NOahP5W10Ddu7eOd1y3i8BlT+MuLT99pe2dH8Ipn\nzOIH965k7eZte/06kiRJ2qFr97uojAaD92hTXABmHziVe1es2+Vx7l6+lhvvWjHitsUPrebXj2/g\ny3/0HA6cNmnEfS6aO5sv/HQZ373jUX777KPH2HtJkqT6+M4dj7b0eAb0inpk9SYmd3Zw6P5TRt1n\n9sxp9Nz7GJlJRIy4z5Vfv51f/mbkK4JGwJ+8+GSed+Kho77Gs46eydEHT2Ph4l4DuiRJ2uc8umYz\n7/nabS09pgG9onpXb+aIA6fS0TFy8IZGQN+0rZ/VG7dx0P6Td9r+0KqN/PI3q3n3y+fw1vNOHPEY\nnbs4PkBEcNEzZ/O5Hy3l8fVbOHT66L8wSJIk1cnAQPLury5ma19rL9zoHPSKWr560y6nt0BjiguM\nvtTif97WOLnzomfOprMjRryNxUXPPJL+geT625fvwTuQJEmqti/evIwf3/8473/l01t6XAN6Re3q\nIkWDdnexooWLennWMTM5+uD9nlJf5hwxgzmHz3A1F0mStM9YsnIdH/v2PbzoaYfxxnOOaemxDegV\n1Nc/wKNrNzN7lIsUDRoM6COthX7/inXc8+g6Lnrm7Jb06aK5s7nlwSd5+MmNLTmeJElSWW3tG+Ad\n1y1i/yldfPx1zxj1XL+9ZUCvoJXrtjCQ7HYE/ZD9JzO5s2PEgL5wcS8dAa84Y1ZL+vSqMxpB/z8X\nO81FkiTV2z98/37ueGQt/+M1z+CwGbuecrw3DOgVNBi4dzcHvaMjmDVzKr3DprhkJgsX9/LcEw9p\n2YfqmEP2Y+7RM1m42GkukiSpvm59cBX/3LOE3z7rKM4//YhxeQ0DegXt7iqizWYfOG2nEfTbHl7D\ng09sbNn0lkEXPXM2dy9fy5KVu157XZIkqYrWb+njndct5siDpvHBi04bt9cxoFfQ4Emfu7qK6KBZ\nM6eyfFhAX7i4l0mdwfmntWZ6y6BXnjGLjsCTRSVJUi391Tfv4qEnN/LJN8xl+pTxW63cddBL6Af3\nrODqnyzjHy59FgePsH557+pNHDC1a0wfjCNnTuPRtZvp6x+gq7OD/oHkm7f1ct4ph3HgfiNfHXRv\nHXbAVJ574iF8fdEjvO1FJzGlq7Olx5ckSbt2410r+B/X3z2mdbk3b97M1J/9YAJ6VR+PrN7Ef5t3\nIs8+7uBxfR0Desk8umYz77xuMWs2beP937idz7zxzJ3ODB7LEouDZs+cxkDCinVbOHLmNH7+61Ws\nWLuFK1/R2uktg6544YlcdvXP+eQN9/G+C1u7JqgkSRrd8jWbeNeCRXTPmMK5Jxyy2/0fffRRjjhi\n9/tph1kHTuX/ffHJ4/46YwroEXE+8PdAJ/C/MvPjw7YfC1wNdAOrgN/LzIeLbX8NvKLY9SOZeV3R\n/iLgE8Bk4FbgDzKzLxpp9O+BC4GNwOWZ+cviOZcB7y+O9VeZec1eveuSykz+/Gu3saWvn999zjF8\n6b9+wzcWPcJrnnXUkP16V28ec0CfVVysaPnqTRw5cxoLF/cybVInL3n6YS3vP8B5p3Tzxuccw1U/\nXsqLnnYYzxnDDwhJkvTUDAwkf/Zvi+kbSD5/2bM57tD9d/ucnp4nmTfvmRPQO+2p3c5Bj4hO4DPA\nBcCpwKURceqw3T4BfDEzzwA+DHyseO4rgDOBucBzgHdHxAER0QFcA1ySmacDDwKXFce6ADi5uF0B\nfLY41sHAB4vjnAN8MCIO2sv3XUr/+rMHuem+x7jywqfz4YtP5+xjD+ID37hzpyuB9q7Z/VVEBw2e\nSPrI6k1s7Rvg23cs56WnHs5+k8fvjydXXvh0jjl4P/50wWLWbd42bq8jSZIarrl5Gf93yRO8/xWn\njimcq9zGcpLoOcCSzFyamVuBrwAXD9vnVOD7xf0fNm0/FfhRZvZl5gZgMXA+cAiwJTPvK/a7EXhd\ncf9iGmE/M/NnwMyImAW8HLgxM1dl5pPFc87fw/dbWg88tp6PXn83Lzylm98791g6O4JPvmEuA5n8\n2YLFDAwkABu39rF647axj6Bvv1jRZn6y5DFWb9zW8tVbhtt/SheffMNclq/ZxIf/865xfS1JkvZ1\nS1au4+PFFS0vPefodndHLTCWYdQjgYeaHj9MYxS72WIaAfvvgdcAMyLikKL9gxHxSWA/YD5wF/A4\nMCkizs7MW4DXA4OfqJFe78hdtA8REVfQGHmnu7ubnp6eMbzF9uobSD76X5vpZIDXzF7Pj370o+3b\n3nBKJ//7jie48ovf4+XHTaJ3feOkj9W9y+jpeXhMx9+vC265awk3LU72nwQ8ehc9K+/e436uX79+\nj+r5iuMn8W+3PswRA49x1uGe7jDcntZTu2Y9W8datpb1bC3rOVTfQPJXP9vMpBjgolnrhmSI3bGW\n5TWW1DTStUtz2OM/A/4pIi4HbgIeAfoy84aIeDbwU+Ax4OaiPSPiEuBTETEFuAHo283rjaUfZOZV\nwFUAc+bMyXnz5u363ZXAp793H79ecz+feeOZO13Z87xMHvriLXzt/sd5ywXn0rFmM/zk57zkuWdy\nzvFjO4P4mEU3sXnyZG57dDWvmns0L3nRGXvVz56eHvakns/7rQF+/dn/y5fu28ybLjx3XK60VWV7\nWk/tmvVsHWvZWtaztaznUJ+84V6WrV3C537vTM4/fc+WT7aW5TWWKS4Ps2N0G+AoYMhC15nZm5mv\nzcxnAVcWbWuKrx/NzLmZ+VIaIfv+ov3mzHxBZp5DI9Tfv5vX220/qmjxQ6v5xx8s4dVzZ+8UzgEi\ngo+99gxmTOnindct4sFVG4HdX0W02ZEzp/HTB55gw9b+cZ/e0mxyVwefesNc1m/p431fu53MnX6f\nkiRJe+mXv3mSz/Q8wOvOPGqPw7nKbSwB/RfAyRFxfERMBi4BFjbvEBGHFid+AryPxoouRERnMdWF\niDgDOIPGaDkRcVjxdQrwHuBzxfMXAm+OhnOBNZm5HPgu8LKIOKg4OfRlRVtlbdrazzuvW8RhM6bw\nlxefPup+3TOm8LHXPoM7e9fy99+7jwg4/ICxB/RZM6eSCYfNmDLhq6qcfPgM3nv+0/j+PSv5yi8e\n2v0TJEnSbm3c2sefXreIIw6YygcvGr52h6put1NciqUP304jDHcCV2fmnRHxYeCWzFwIzAM+FhFJ\nYzT8bcXTJwE/LtbxXktj+cXBqSzvjohX0vgl4bOZObhS/vU0llhcQmOZxbcU/VgVER+h8QsDwIcz\nc9Xev/X2+/i372bp4xv40h8+hwOn7fqiQS877QjecPZRLLjlYY44YCqTOsd+EdjBE0pfccYsOjtG\nmik0vi5/3nF8/54VfOSbd/G8Ew/h2EM8u1ySpN1Zs3EbSx9fP+K2L//Xb3hw1Uau/aNzOWBqay88\nqPYb05l7mXk9jeDc3PaBpvtfBb46wvM201jJZaRjvht49wjtyY6AP3zb1RSj81W3ZOV6rrn5QS5/\n3nE8/6RDx/ScD7zqNG5e+sT2pRPH6oRDpwPw6rk7nVM7ITo6gr99/TN5+adv4p3XLWLBHz+Xrj34\nBUOSpH3Nqg1bOf/TN7Fy3ZZR9/mjFxw/pgsSqXpcWqNNFi56hI6A/z7/xDE/Z/qULv79vz1/j+dy\nv+zUw7nxnS/k5MNn7Gk3W2b2zGl85OLTecd1i/ifNy3lbfNPaltfJEkqs8zkff9+G6s3buPvL5nL\nASP8lX3apE7OGefLzat9DOhtkJksXNzLc088ZI9XNumeMWWPX6+jI9oazgddPHc237t7BZ+68T5e\neHI3zzjqwHZ3SZKk0vnqrQ/z3TtX8L4LnsbFbfrrt9rLeQZtcNvDa1j2xMYJXVGlDCKCv3r16Rw6\nfQrvuO5XbN7W3+4uSZJUKg+t2shf/uddnHP8wfzhC05od3fUJgb0Nli4uJdJncH5p+17SyLN3G8y\nf/vbZ/DAYxv4+LfvaXd3JEkqjf6B5F0LFgPwyTc8sy0LO6gcDOgTrH8g+eZtvZx3ymEcuN++edb1\nC07u5vLnHccXfrqMm+57rN3dkSSpFP7lx0v5+bJV/OVFp3HUQfu1uztqIwP6BPv5r1exYu0WLp67\nb01vGe69FzyNkw6bzru/upjVG7e2uzuSJLXVnb1r+Lsb7uXCZxzBa8903vm+zoA+wRYu7mW/yZ28\n5OmHt7srbTV1Uief/p25PLF+K1d+4w6vMipJ2mdt3ta4cOFB+03mo69+BsX1Y7QPM6BPoK19A3z7\njuW89NTDmTa5s93dabvTjzyQd770FL5123L+Y1Fvu7sjSVJbfOK793LfivX8zevP4KD9J7e7OyoB\nl1mcQD++/zH+//buPDyu+srz//uoSi553yQ27wRjMMEYbCCEkIjQCUsWGkgHmKydEKanQz/dJPD7\nhcnODCFJEzKZSTrd9C90N/mFAIGQdroJkBAEYTOWjQUGY7wgGVvGthYvkqylVGf+qCtTFmWrJN3S\nreXzep56XLpbnTrPler41Pd+756uvrKbveVI/ut7j+ePr+7i6/++jnPeMZOjpwxv2kkREZGwtXb0\n8KPHNtKbTOX9tXr7U/x6zXY+fc48ahcdlffXk+KgAn0MrWhoZur4Ss5bWBN1KAUjHqvgGx9ezKU/\neZrntrRqvlcREYnc3Su3ctezTRw9Zfj3HhmJ8xZWc9PFJ4/Ja0lxUIE+Rg709vP7V3Zy6dLjGBfX\nyKJMMyelv87rGYNOhYiIyJEM3EzwrPkzuO+vzok6HClTqhTHyB/W76Srt5+PaHjL2yTi6fH4Pbpx\nkYiIROzVN/ezcVcHHynz2dYkWirQx8iKhmaOnpLg7AUzow6l4FRVpk9DddBFRCRqKxqaiVUYl7zz\nmKhDkTKmAn0M7O3q44kNu/nwkuN0V7AsBjro3eqgi4hIhNyd3zY0854Tqpk5aWzGn4tkowJ9DDz8\n8g56+1OaveUwKmNGhamDLiIi0VqzdQ/b2g/o81oipwJ9DKxoaGbezAksmT016lAKkpmRiMfUQRcR\nkUj9tqGZRLyCD55S3jcTlOipQM+zXfu7eXZzKx897TjdGewIqior1EEXEZHIJPtT/MeLO3j/SUcx\nuaoy6nCkzKlAz7P/fHEHKUdflw1BHXQREYnSc1vaaOno0ee1FAQV6Hn2u5fe5KRjJrPw6MlRh1LQ\nEuqgi4hIhFY0bGdSIs75J+lunhI9Feh59uqb+1g+f3rUYRS8KnXQRUQkIj3Jfn637k0+uPhoqipj\nUYcjogI9n/Z09bKvO8n8mROjDqXgqYMuIiJReWLDbvZ3J3VzIikYKtDzqLG1C4B5KtCHpA66iIhE\nZUVDM9MnVPKeE6qjDkUEUIGeV02tnQDMmzkh4kgKnzroIiIShc6eJH9Yv5NLTj2WypjKIikMOhPz\nqCnooM+doQJ9KOlZXFSgi4jI2PrD+p109+lmglJYVKDnUWNrJ8dMqdIFJzlId9A1xEVERMbWbxua\nOXZqFWfOnxF1KCIHqUDPo62tXRrekqOqeIweddBFRGQM7enq5YnXdvPhJcdSUaGbCUrhUIGeR42t\nXZrBJUfqoIuIyFh7eN2b9PU7Hz1tVtShiBxCBXqedPQkaenoYa466Dmp0hh0EREZYysamllQPZF3\nzpoSdSgih1CBnidbgwtE1UHPjTroIiIylvYe6OO5La18eMmxmGl4ixQWFeh5oikWh6cqHqOv3+lP\nedShiIhIGViztZ2Uw216fA0AACAASURBVDnHz4w6FJG3UYGeJ01twRSLKtBzkqhMn4rqoouIyFio\nb2wjVmEsnTst6lBE3kYFep40tXYyc+I4plRVRh1KUaiKBwW6xqGLiMgYqG9s55TjpjBhXDzqUETe\nJqcC3cwuMrMNZrbJzL6SZf08M3vMzF40szozm52x7ntmti54XJmx/AIzW2Nma83sKTM7IYdj9Qfb\nrzWzFaN76/nV2NKl7vkwJIK54rvVQRcRkTzrTaZo2LaHZfOmRx2KSFZDFuhmFgN+AlwMLAauNrPF\ngza7DbjL3ZcANwO3Bvt+CDgDWAqcDdxoZgOXSv8U+IS7LwXuBr52pGMFDrj70uDx0WG/2zG0tU1T\nLA5HVaU66CIiMjZebt5Ld19KNyeSgpVLB/0sYJO7b3H3XuAe4NJB2ywGHgueP56xfjHwhLsn3b0T\naAAuCtY5MFCsTwWahzhW0eju66d57wFdIDoMibg66CIiMjZWN7UDsFwddClQuQy8mgW8kfHzNtLd\n8EwNwBXAj4DLgMlmNjNY/k0zux2YAJwPvBLscw3wkJkdAPYB7zrSsdy9Fagys3ogCXzX3X8zOFgz\nuxa4FqCmpoa6uroc3mK4mjtSuEPXrq3U1TUPvUOR6OjoyFs+X9uVBOCZlat4c2osL69RaPKZz3Kk\nfIZHuQyX8hmuMPL5uxe6qRlvvLLmuYNFSTnSuVm4cinQs00OOnguvBuAH5vZZ4Enge1A0t0fNbMz\ngWeA3cCzpItrgOuBS9x9pZndCNxOumjPeqxgn7nu3mxmxwN/NLOX3H3zIYG53wHcAbBo0SKvra3N\n4S2G67H1O+Gpei56zzLOmFs6/zuvq6sjX/ms3NQCa1ZyyqlLObtMprzKZz7LkfIZHuUyXMpnuEab\nT3fnhqf+wHtPOoba2qXhBVaEdG4WrlwK9G3AnIyfZ/PWcBQA3L0ZuBzAzCYBV7j73mDdLcAtwbq7\ngY1mVgOc5u4rg0PcCzycw7Gag3+3mFkdcDpwSIFeCBqDmxTNm6EhLrk6OAY9qTHoIiKSP02tXbR0\n9LJsfuk00KT05DIGfRWw0MwWmNk44CrgkBlUzKzazAaOdRNwZ7A8Fgx1wcyWAEuAR4F2YKqZnRjs\n8wFg/RDHmm5miYFtgHOhML+Z2trayeREnBkTx0UdStE4OAa9T2PQRUQkf1Y1tgGwfJ4uEJXCNWQH\n3d2TZnYd8AgQA+5095fN7Gag3t1XALXArWbmpIelfDHYvRL4U3AL3X3AJ909CWBmXwAeMLMU6YL9\nc8E+hzvWycA/BdtXkB6DXpAFemNrF/OqJ+jWwcOgDrqIiIyF1U3tTKmKs/CoSVGHInJYOc3O7+4P\nAQ8NWvaNjOf3A/dn2a+b9Kws2Y75IPBgluWHO9YzwKm5xBu1ptZOTjluatRhFBV10EVEZCzUN7Wz\nbN50KirURJPCpTuJhizZn2Jbu6ZYHK5EXB10ERHJr/bOXjbt6mC55j+XAqcCPWTNe7pJplwF+jAd\nvJOoOugiIpInmv9cioUK9JA1tXUCME93ER0WddBFRCTf6pvaqYwZp82ZFnUoIkekAj1kA1MszleB\nPiwHC3R10EVEJE/qG9s45bipVFWWxw3xpHipQA9ZU0sniXgFR01ORB1KUTEzEvEKddBFRCQvepL9\nvLh9L2dq/nMpAirQQ9bU1sW8mRN0dfgIVFXGNAZdRETyYt32vfQmUyzT/OdSBFSgh6yptZO5MzS8\nZSTUQRcRkXxZ1Zi+QHSZLhCVIqACPUSplLO1rYv5msFlRNRBFxGRfKlvbGdB9URqNARVioAK9BDt\n2t9Dd1+KedXqoI+EOugiIpIP7s7qpjZ1z6VoqEAPUWNrMMXiDHXQR0IddBERyYfNuztp7+rTBaJS\nNFSgh2irplgcFXXQRUQkH1Y3tQHoAlEpGirQQ9TY2km8wjhuWlXUoRQlddBFRCQfVjW2M31CJe+o\nUQNNioMK9BA1tXUxe/p44jGldSTUQRcRkXxY3dTOsnkzMNMUyFIcVEmGqKm1k3ka3jJiVZUxFegi\nIhKqlo4eXm/pZLnGn0sRUYEeEnenqSV9kyIZmUS8QkNcREQkVPXB/OfLNYOLFJF41AGUivauPvb3\nJNVBH4WEOugiIjICew/08eCabfT2v/0z5E8bWxgXr+DU2VMjiExkZFSgh0RTLI6eOugiIjJc7s7f\n3vMCdRt2H3abPzv5KBLx2BhGJTI6KtBDcnCKxWoV6COlMegiIjJcv1i5lboNu/n6hxdz1Zlzsm4z\nvlLFuRQXFeghaWztxAxmT1eBPlKJeAW9yRSplFNRoSvtRUTkyLbs7uCW/1zPeQur+ct3z9dnh5QM\nXSQakqbWLo6dUkWV/pc+YgO5yzaGUEREJFNff4rr713LuHgFt/3FaSrOpaSoQA+JplgcvUQ8fTpq\nHLqIiAzlJ49vomHbXr5z2akcPUU3CJTSogI9JE2tmmJxtBKV6dNR49BFRORIXtjazv/54yYuO30W\nH1pybNThiIROBXoI9nf30drZqw76KFUFV9irgy4iIofTk3S+dF8Dx0yp4tuXnhJ1OCJ5oYtEQ9A0\nMIOLOuijog66iIgM5Z4NvTS2Jrn7mncxpaoy6nBE8kId9BAMFOhzVaCPijroIiJyJI+/uovH30hy\nzXsWcM47ZkYdjkjeqIMegqa24CZFGuIyKuqgi4iUN3fnlv9cz+MbdmVdv2NvN7MnGTdcuGiMIxMZ\nWyrQQ7BzbzdTquJMSiidozEwzaI66CIi5en+1dv4/556nXe/YybTJ4572/rT5kzjrIltuiuolDxV\nlCFo6eylelIi6jCK3sA0iz196qCLiJSbN9q6+PZvX+HsBTP4+efPJnaYec3r6urGNjCRCGgMegha\nO3qYOent/9OX4TnYQU+qgy4iUk76U86X72vAgB98/LTDFuci5UIFeghaO3qZOVEd9NFSB11EpDzd\n8eQWnm9s49uXnsLs6ZpwQUQFeghaOnqonqwO+mipgy4iUn5ebt7L7b/fwCWnHsNlp8+KOhyRgqAC\nfZSS/Snau/rUQQ+BOugiIuWlu6+f6+9dy/QJ47jlz0/FTENbRCDHAt3MLjKzDWa2ycy+kmX9PDN7\nzMxeNLM6M5udse57ZrYueFyZsfwCM1tjZmvN7CkzOyGHY33GzDYGj8+M7q2Ho62rF4BqjUEfNXXQ\nRUTKy98/soHXdnbw/Y8tyTpri0i5GrJAN7MY8BPgYmAxcLWZLR602W3AXe6+BLgZuDXY90PAGcBS\n4GzgRjObEuzzU+AT7r4UuBv42hDHmgF8MzjOWcA3zWz6SN50mFo70gX6TM3iMmrjYuqgi4iUi6c3\ntfCzp17n0+fMo3bRUVGHI1JQcplm8Sxgk7tvATCze4BLgVcytlkMXB88fxz4TcbyJ9w9CSTNrAG4\nCLgPcGCgWJ8KNA9xrAuB37t7WxDH74Nj/TKnd5onLR09AJpmMQQVFca4eIU66CIiJWLz7g7aO3vf\ntjyZcm74VQPHV0/kpotPjiAykcKWS4E+C3gj4+dtpLvYmRqAK4AfAZcBk81sZrD8m2Z2OzABOJ+3\nCvtrgIfM7ACwD3jXEMfKFkfkV5O81UHXV3NhSMQr1EEXESkB//FiM9fd/cJh18crjAf+27sZP043\nHRIZLJcCPdsVGz7o5xuAH5vZZ4Enge1A0t0fNbMzgWeA3cCzQDLY53rgEndfaWY3AreTLtqzHivH\nODCza4FrAWpqavJ+Q4OVjX0AvLp2FW9UlvbFLR0dHXnPZ4X307h1G3V1u/P6OoVgLPJZTpTP8CiX\n4SrHfLZ3p/ja0wc4fmoFly/M3sCqGW+0b15L3ebhHbsc85kvymXhyqVA3wbMyfh5Nm8NRwHA3ZuB\nywHMbBJwhbvvDdbdAtwSrLsb2GhmNcBp7r4yOMS9wMNHOpaZbQNqB8VRNzhYd78DuANg0aJFXltb\nO3iTUK18+FUqN27hkj+rLfmrz+vq6sh3Piev/CMzamZQW7s0r69TCMYin+VE+QyPchmucsunu/Pp\nO58nRS8/+8J5LKieGOrxyy2f+aRcFq5cZnFZBSw0swVmNg64CliRuYGZVZvZwLFuAu4MlseC4SmY\n2RJgCfAo0A5MNbMTg30+AKw/0rGAR4APmtn04OLQDwbLItWyv4eZExMlX5yPlarKGD1JDXERESlW\nP3+uiT9tbOGrHzo59OJcpFwM2UF396SZXUe6GI4Bd7r7y2Z2M1Dv7itId7ZvNTMnPSzli8HulcCf\nguJ1H/DJ4IJRzOwLwANmliJdsH8u2Cfrsdy9zcz+B+n/MADcPHDBaJRaO3s1/jxEiXgF3X26SFRE\npBht3t3Bdx5aT+2iGj5x9tyowxEpWrkMccHdHwIeGrTsGxnP7wfuz7JfN+lZWbId80HgwSzLsx4r\nWHcnb3XUC0JrR4+mWAyROugiIsWprz/Fl+5dS1VljO9fsUTfLIuMgu4kOkotHb1U6+YKoVEHXUSk\nOP34j5to2LaX71x2KkdNqYo6HJGipgJ9FNydlo4eqiergx6WRLxCHXQRkSKz9o09/PjxTVx++iwu\nOfXYqMMRKXoq0Eehs7efnmSKmeqgh6aqMqYOuohIETnQ28+X7l3L0ZMTfOvSU6IOR6Qk5DQGXbJr\nDe4iqjHo4VEHXURk9Oob2/iPF3eMyWtt3LWfLS2d3H3N2UypqhyT1xQpdSrQR6EluItotWZxCY06\n6CIio7OtvYu//JdV9PanqKrM/106zeDLHziRd59QnffXEikXKtBHoSXooFergx4addBFREYulXK+\nfF8DKXf+8KX3MWfGhKhDEpER0Bj0UWgNOuiaBz086qCLiIzcz556nZWvt/HNj56i4lykiKlAH4WB\nMegzdJFoaAY66O4edSgiIkVl/Y59/P0jG7jwlKP5i2Wzow5HREZBBfootHb2MqUqTiKe/zF+5SIR\njJfUMBcRkdz1JPu5/t61TBlfyXcuO1U3CRIpcirQR2F3R4/Gn4csEU+fkirQRURy94NHX+PVN/fz\n/Y+dqpnFREqACvRRaO3o0fjzkA3MONCjcegiIjl5dnMr//ynLfyXs+fy/pOOjjocEQmBCvRRaO3o\nZeZEdSrCpA66iEju9nX3ccOvGpg3YwJf+9DJUYcjIiFRgT4KrZ29VE9WBz1MAx10zeQiIjK0b/37\ny7y5r5sfXrmUCeM0c7JIqVCBPkLJ/hTtXeqgh00ddBGR3Dy9qYVfv7CdL55/AqfPnR51OCISIhXo\nI9TW1Yu77iIaNnXQRURy8/iruxgXr+Cva98RdSgiEjIV6CP01k2K1EEPkzroIiK5qW9q57TZUw82\nNkSkdKhAH6GBAl3TLIZLHXQRkaEd6O1n3fa9LJ8/I+pQRCQPVKCPUEtwF1FNsxiuRKU66CIiQ2nY\ntodkylk+T2PPRUqRCvQRGijQq3WRaKiq4gN3ElUHXUTkcFY3tQOwTAW6SElSgT5CrZ29VMaMKeM1\nrVWYBjro3X3qoIuIHM6qxjYWHjWJaRP0La5IKVKBPkKtHT3MnJjAzKIOpaQc7KBrDLqISFaplLOm\nqZ3l89U9FylVKtBHqLWjV+PP8+BgB11j0EVEstq4q4N93UmWzdMFoiKlSgX6CLV09GiKxTxIHOyg\nq0AXEclmVWMbAGeqgy5SslSgj1BLRy/VE9VBD1uswqiMGd26SFREJKvVTe1UT0owd8aEqEMRkTxR\ngT4C7k5rZw/Vk9VBz4dEPKYOuojIYaxqbGP5vOm6BkqkhKlAH4Gu3n66+1LMVAc9L6oqK9RBFxHJ\nYue+bra1H9AFoiIlTgX6CLx1kyJ10PNBHXQRkezqG9Pzn+sOoiKlTQX6CLR09AK6i2i+JNRBFxHJ\nalVjG1WVFZxy3JSoQxGRPFKBPgKtQQe9Rh30vFAHXUQku9VN7SydM43KmD6+RUqZfsNHoLVTHfR8\nqqqsoEcddBGRQ3T2JHllxz6Wa/5zkZKnAn0EWvanO+gzdJFoXiTiFeqgi4gMsvaNPfSnXBeIipQB\nFegj0NrZy+Sq+MGb6ki4qipjGoMuIjJIfWM7ZnDGPBXoIqUupwLdzC4ysw1mtsnMvpJl/Twze8zM\nXjSzOjObnbHue2a2LnhcmbH8AjNbY2ZrzewpMzshWD7XzB43sxeC410SLJ9vZgeC7dea2T+O/u2P\nTEtHj8af55E66CIib1ff1Maioyczpaoy6lBEJM+GLNDNLAb8BLgYWAxcbWaLB212G3CXuy8BbgZu\nDfb9EHAGsBQ4G7jRzAYuPf8p8Al3XwrcDXwtWP414D53Px24CviHjNfZ7O5Lg8dfDfvdhqS1o1fj\nz/NIHXQRkUP1p5wXtu7R8BaRMpFLB/0sYJO7b3H3XuAe4NJB2ywGHgueP56xfjHwhLsn3b0TaAAu\nCtY5MFCsTwWah1heMFo6epg5UR30fFEHXUTkUK++uY+OnqQuEBUpE7kU6LOANzJ+3hYsy9QAXBE8\nvwyYbGYzg+UXm9kEM6sGzgfmBNtdAzxkZtuATwHfDZZ/C/hksPwh4G8yXmdBMPTlCTM7L5c3mA+t\nneqg55M66CIih1rdlL5B0TKNPxcpC/EctrEsy3zQzzcAPzazzwJPAtuBpLs/amZnAs8Au4FngWSw\nz/XAJe6+0sxuBG4nXbRfDfyru//AzM4Bfm5m7wR2AHPdvdXMlgG/MbNT3H3fIcGaXQtcC1BTU0Nd\nXV0ObzF3/SmnvbOXjpYd1NW1hnrsQtfR0RF6PrPZtaOHrp7kmLxWlMYqn+VC+QyPchmuMPL5n2u7\nmZ4wNjWsZLNl+1guHzo/w6NcFq5cCvRtvNX1BpjNoGEn7t4MXA5gZpOAK9x9b7DuFuCWYN3dwEYz\nqwFOc/eVwSHuBR4Onn+eYBiMuz9rZlVAtbvvAnqC5avNbDNwIlA/KJY7gDsAFi1a5LW1tTm8xdzt\n3t+DP/oHlr/zRGrPmR/qsQtdXV0dYeczm9W9G3ikaRPve9/7sBL+IBqrfJYL5TM8ymW4wsjnf3/2\nMd69aDrnn39GOEEVMZ2f4VEuC1cuQ1xWAQvNbIGZjSN94eaKzA3MrNrMBo51E3BnsDwWDHXBzJYA\nS4BHgXZgqpmdGOzzAWB98HwrcEGwz8lAFbDbzGqCC1Yxs+OBhcCW4b/l0WkJ7iI6U7O45E0iXoE7\n9PUP/qJGRKT8bN9zgOa93SzX8BaRsjFkB93dk2Z2HfAIEAPudPeXzexmoN7dVwC1wK1m5qSHuHwx\n2L0S+FPQBd0HfNLdkwBm9gXgATNLkS7YPxfs82Xgn83setJDaT7r7m5m7wVuNrMk0A/8lbu3jT4F\nw9PaEdxFVDcpypuqyvT88t3JfsbFNVW/iJS3+sb0R92Z83WBqEi5yGWIC+7+EOkLNjOXfSPj+f3A\n/Vn26yY9k0u2Yz4IPJhl+SvAuVmWPwA8kEu8+dTame6gV09WBz1fEkFR3tOXSn9/IiJSxlY3tTNh\nXIyTjpkcdSgiMkZyKtDlLS1BB71a0yzmTSLooPdoJhcRKRP9KWffgb6s655/vY3T504jHtM3iiLl\nQgX6MLV09BCvMKaMV+ryZaCD3q250EWkTPz1L1bzyMs7D7v+by9YOIbRiEjUVGUOU2tHDzMnjSvp\n2UWiVqUOuoiUkb7+FE++1sJ5C6u54KSj3rY+FqvgI0uOjSAyEYmKCvRhau3opVozuOSVOugiUk7W\n79jHgb5+Pr58Dh857biowxGRAqABbcPU0tmrKRbzLBFXB11EyseqxvRdQpfP1zSKIpKmAn2YWvb3\nUK0pFvOqqjJjFhcRkRK3uqmNWdPGc+zU8VGHIiIFQgX6MLg7rZ3pMeiSP+qgi0i5cHfqG9vVPReR\nQ6hAH4au3n66+1Iag55nAx10jUEXkVL3RtsBdu3v0V1CReQQKtCH4eBdRFWg55XmQReRclHflL5L\n6HLdJVREMqhAH4bdHem7iGqIS35VaRYXESkT9U3tTE7EOfFo3SVURN6iAn0YWoMCXXcRzS910EWk\nXKxubOf0edOJVejeGiLyFhXow9DamR7iUj1ZHfR8UgddRMrB3q4+Nuzcz5kafy4ig6hAH4aBDvoM\nTbOYV/FYBbEKUwddREramq3p+c+XaQYXERlEBfowtHT0MrkqfnAaQMmfqniFOugiUtLqm9qIVxhL\n50yLOhQRKTAq0IehpaNHUyyOkURlTB10ESlpqxrbOeW4KUwYF486FBEpMCrQh6G1o5dqzeAyJtRB\nF5FS1ptM0fDGHpbN0/SKIvJ2KtCHobWzh5mawWVMpDvoKtBFpDS93LyXnmSKMzX+XESyUIE+DC0d\nvZoDfYwk4hV092mIi4iUpvpGXSAqIoenAj1Hyf4U7V29uovoGFEHXURKWX1TG3NnTOCoyVVRhyIi\nBUgFeo7au/pwhxp10MdElTroIlKi3J3VTe0sV/dcRA5DBXqOWoI50NVBHxvqoItIqWps7aKlo5fl\nukBURA5DBXqO1u/YB8CC6okRR1IequIV9KiDLiIlqL6xDUAddBE5LBXoOapvamdyIs6JR0+OOpSy\noA66iJSq1U3tTB1fyQk1k6IORUQKlAr0HK1ubOeMedOJVVjUoZQFddBFpFStamxj2bzpVOjzREQO\nQwV6DvZ29bFh536Wz9PXkWMlUVlBtzroIlJi2jt72by7U8NbROSIVKDnYM1WzVc71hLxmDroIlJy\nVjelP090gaiIHIkK9BysamwjXmEsnTMt6lDKRpU66CJSglY1tVEZM5bMnhp1KCJSwFSg56C+qZ1T\njpvChHHxqEMpG4l4jP6Uk+xXkS4ipWN1YzunzppKVWUs6lBEpICpQB9CbzJFwxt7WKavI8dUVWX6\n1FQXXURKRXdfPy9u28vy+fo8EZEjU4E+hHXNe+lJpjhT48/HVCKe7i5pHLqIlIp12/fS259imSYc\nEJEhqEAfwupGXSAaBXXQRaTUPPjCdszQjGAiMiQV6EOob2pj7owJHDW5KupQyoo66CJSSp54bTe/\nWLmVz527gJmTElGHIyIFTgX6Ebg79Y3tmq82Agc76H3qoItIcevodW78VQMLj5rEjRcuijocESkC\nORXoZnaRmW0ws01m9pUs6+eZ2WNm9qKZ1ZnZ7Ix13zOzdcHjyozlF5jZGjNba2ZPmdkJwfK5Zva4\nmb0QHO+SjH1uCmLYYGYXju6tD62xtYvWzl7NVxuBgx30pDroIlK83J27XumhrbOXH165VLO3iEhO\nhizQzSwG/AS4GFgMXG1miwdtdhtwl7svAW4Gbg32/RBwBrAUOBu40cymBPv8FPiEuy8F7ga+Fiz/\nGnCfu58OXAX8Q3CsxcHPpwAXAf8QxJY39Y1tAOqgRyChDrqIlIAVDc08/2Y/13/gRN45S3Ofi0hu\ncumgnwVscvct7t4L3ANcOmibxcBjwfPHM9YvBp5w96S7dwINpItrAAcGivWpQPMQyy8F7nH3Hnd/\nHdgUxJY39Y3tTB1fyQk1k/L5MpKFOugiUuya9xzg679ZxwnTKviv7z0+6nBEpIjkcuedWcAbGT9v\nI90Nz9QAXAH8CLgMmGxmM4Pl3zSz24EJwPnAK8E+1wAPmdkBYB/wrmD5t4BHzexvgInAn2XE8dyg\nOGYNDtbMrgWuBaipqaGuri6Ht5jdk+u7mD+pgieffGLExyglHR0do8rncDTtSxfmq9e+CDtK8wZR\nY5nPcqB8hke5HL2UO7fVd9PTl+ITpzhP/enJqEMqGTo/w6NcFq5cKh/LsswH/XwD8GMz+yzwJLAd\nSLr7o2Z2JvAMsBt4FkgG+1wPXOLuK83sRuB20kX71cC/uvsPzOwc4Odm9s4c48Dd7wDuAFi0aJHX\n1tbm8Bbfrq2zlx0P/55PnXcCtbUnjOgYpaauro6R5nO4Nu3qgGee4IRFJ1O79G3/DysJY5nPcqB8\nhke5HL1/efp1Xml9he9cdirHHdiifIZI52d4lMvClcsQl23AnIyfZ/PWsBMA3L3Z3S8Pxo1/NVi2\nN/j3Fndf6u4fIF1kbzSzGuA0d18ZHOJe4N3B888D9wX7PgtUAdW5xBGm1U3p+c91gWg0BmZx6dEY\ndBEpMpt27ee7v3uV9590FFefNWfoHUREBsmlQF8FLDSzBWY2jvSFmisyNzCzajMbONZNwJ3B8lgw\n1AUzWwIsAR4F2oGpZnZisM8HgPXB863ABcE+J5Mu0HcHr3mVmSXMbAGwEHh++G85N/VNbVTGjCWz\ndVFPFDQGXUSKUbI/xfX3NjBhXIzvXnEqZtm+/BURObIhh7i4e9LMrgMeAWLAne7+spndDNS7+wqg\nFrjVzJz0EJcvBrtXAn8K/kDtAz7p7kkAM/sC8ICZpUgX7J8L9vky8M9mdj3pISyfdXcHXjaz+0iP\nYU8CX3T3vFVv9Y3tnDprqqbEiojmQReRYrRm6x5e2r6X2/7iNN3gTkRGLKer79z9IeChQcu+kfH8\nfuD+LPt1k57JJdsxHwQezLL8FeDcw+xzC3BLLjGPRndfPy9t28tnz52f75eSw1AHXUSKUX1Tenre\n9590VMSRiEgx051Es1i3fS+9/SmWzdP851GpjBkVpg66iBSX1Y3tHF8zkRkTx0UdiogUMRXoWaxq\nHLhAVAV6VMyMRDymDrqIFI1UyqlvaudMTS4gIqOkAj2L1U1tHF89kZmTElGHUtaqKivUQReRorF5\ndwd7D/SxTHefFpFRUoE+SCrlrG5q1/CWAqAOuogUk/omffsqIuFQgT7IlpYO2rv6OHO+vqKMWlVl\nBT1JddBFpDisamxj5sRxLKieGHUoIlLkVKAPUh+MP9dXlNFLxGN096mDLiLFYeDbV819LiKjpQJ9\nkPqmdmZMHMfx6oBELqEOuogUiV37u2lq7WK5mjsiEoKc5kEvNTf/9hUefGFb1nX7u5PULjpKHZAC\nUKUOuogUidUDs39peKSIhKAsC/TT5kwlmcremTXgY8vmjG1AklWisoKOnmTUYYiIDKm+qZ1EvIJ3\nHjc16lBEpASUwCaIQQAADjpJREFUZYF+6dJZXLp0VtRhyBAS8RgtHb1RhyEiMqT6xjZOmz2NcXGN\nHBWR0dNfEilY6THoGuIiIoXtQG8/Lzfv0/hzEQmNCnQpWFXxGD26UZGIFLi1b+whmXIV6CISGhXo\nUrDUQReRYlDf2AbAsrm6QFREwqECXQpWehYXddBFpLDVN7Vz4tGTmDqhMupQRKREqECXgqUOuogU\nuv6Us2ZrO8vmqXsuIuFRgS4Fqyoeo6/f6U951KGIiGT12s797O9OcqbGn4tIiFSgS8FKVKZPT3XR\nRaRQ1TcFNyhSB11EQqQCXQpWVTCfsMahi0ihWt3YRs3kBHNmjI86FBEpISrQpWAlKmOAOugiUrhW\nNbZz5vzpmFnUoYhICVGBLgWrqlIddBEpXDv2HmD7ngO6QFREQqcCXQpWIq4OuogUrvrGgfHnukBU\nRMKlAl0KljroIlLIVje1M74yxuLjpkQdioiUGBXoUrAOdtD71EEXkcJT39TG0jnTqIzpo1REwqW/\nKlKwDnbQk+qgi0hh6ehJ8krzPpZr/nMRyQMV6FKw1EEXkUK1duseUg7L5+sCUREJnwp0KVjqoItI\noapvasMMTp87LepQRKQEqUCXgqUOuogUqudfb+OkY6Ywpaoy6lBEpASpQJeClQjuJNqjDrqIFJA/\nvrqTZza3cuEpR0cdioiUKBXoUrAG7iTarQ66iBSI1o4e/p/7X+KkYybz32rfEXU4IlKi4lEHIHI4\n6qCLSCFxd2769UvsO9DH/3/NWQeH4YmIhE0ddClYBwt0ddBFpAD8avU2Hn1lJzdeuIiTjtHNiUQk\nf1SgS8EyMxLxCnXQRSRyb7R18e0VL3PO8TP5/HsWRB2OiJS4nAp0M7vIzDaY2SYz+0qW9fPM7DEz\ne9HM6sxsdsa675nZuuBxZcbyC8xsjZmtNbOnzOyEYPkPg2Vrzew1M9uTsU9/xroVo3vrUgyqKmMa\ngy4ikepPOdffu5YKM277+GlUVFjUIYlIiRtyDLqZxYCfAB8AtgGrzGyFu7+SsdltwF3u/m9m9n7g\nVuBTZvYh4AxgKZAAnjCz37n7PuCnwKXuvt7M/hr4GvBZd78+47X/Bjg943UOuPvS0bxhKS7qoItI\n1P7pyc3UN7XzwytPY9a08VGHIyJlIJcO+lnAJnff4u69wD3ApYO2WQw8Fjx/PGP9YuAJd0+6eyfQ\nAFwUrHNgYBDfVKA5y2tfDfwylzcipUkddBGJ0rrte/nh71/jQ0uO5c+Xzoo6HBEpE+buR97A7GPA\nRe5+TfDzp4Cz3f26jG3uBla6+4/M7HLgAaAaWAZ8k3T3fQLwPPATd/+BmZ0H/AY4AOwD3hV01geO\nOQ94Dpjt7v3BsiSwFkgC33X332SJ91rgWoCamppl99133/CzIll1dHQwadKkMX3N//5UF8dNrOC6\n06vG9HXHQhT5LGXKZ3iKOZc9/Uf+TBuOZApuWXmArj74n+eOZ9K4kQ1tKeZ8FiLlMzzKZbjOP//8\n1e6+PIxj5TLNYra/SIP/At4A/NjMPgs8CWwHku7+qJmdCTwD7AaeJV1cA1wPXOLuK83sRuB24JqM\nY14F3D9QnAfmunuzmR0P/NHMXnL3zYcE5n4HcAfAokWLvLa2Noe3KLmoq6tjrPM586WnmDxpHLW1\nZ43p646FKPJZypTP8BRrLm/69Yv88vk3Qj/uv33uLN53Ys2I9y/WfBYq5TM8ymXhyqVA3wbMyfh5\nNoOGo7h7M3A5gJlNAq5w973BuluAW4J1dwMbzawGOM3dVwaHuBd4eNDrXgV8Mcvr4O5bzKyO9Pj0\nzUjJ0hh0EcnFv6/dzi+ff4PLTp/FomMmh3bcRUdPHlVxLiIyErkU6KuAhWa2gHRn/Crgv2RuYGbV\nQJu7p4CbgDuD5TFgmru3mtkSYAnwaLDbVDM70d1fIz0EZn3G8RYB00l33AeWTQe63L0neL1zge+P\n4D1LEamqjNHVmxx6QxEpWzv2HuDrv1nHGXOn8fcfW0I8phmERaS4DVmgu3vSzK4DHgFiwJ3u/rKZ\n3QzUu/sKoBa41cyc9BCXgc53JfAnM4P0OPNPunsSwMy+ADxgZimgHfhcxsteDdzjhw6QPxn4p2D7\nCtJj0DNnkpESlIhX0N6lDrqIZJdKOTf8qoFkyrn940tVnItIScilg467PwQ8NGjZNzKe3w/cn2W/\nbtIzuWQ75oPAg4dZ960sy54BTs0lXikdmsVFRI7kX59p5OlNrdx6+anMr54YdTgiIqFQq0EKmsag\ni8jhbNy5n+8+/CoXnHQUV505Z+gdRESKhAp0KWiJyhjdfSrQReRQvckUf3fvWiYn4nz3iiUEQylF\nREpCTkNcRKKS7qBriIuIHOp//eE1Xm7exx2fWkbN5ETU4YiIhEoddCloVZUxetRBF5EMqxrb+Mcn\nNnPl8jl88JRjog5HRCR06qBLQUvEK+jtT/EvT7+e9Y5ZxWxjUx+NT78edRglQ/kMT6Hn8mdPv87s\n6RP4+keyzkEgIlL0VKBLQZtfPQGAb/+2RGfUXF+i7ysqymd4CjiXE8bFuOtzZzEpoY8wESlN+usm\nBe2y02fz/pOOJpXyoTcuMk8//TTnnntu1GGUDOUzPIWey6rKGOPHxaIOQ0Qkb1SgS8GbOr4y6hDy\nYtI4Y/rEcVGHUTKUz/AolyIi0dJFoiIiIiIiBUQFuoiIiIhIAVGBLiIiIiJSQFSgi4iIiIgUEBXo\nIiIiIiIFRAW6iIiIiEgBUYEuIiIiIlJAVKCLiIiIiBQQFegiIiIiIgVEBbqIiIiISAFRgS4iIiIi\nUkDM3aOOIW/MbD+wIeo4Skg10BJ1ECVE+QyX8hke5TJcyme4lM/wKJfhWuTuk8M4UDyMgxSwDe6+\nPOogSoWZ1Suf4VE+w6V8hke5DJfyGS7lMzzKZbjMrD6sY2mIi4iIiIhIAVGBLiIiIiJSQEq9QL8j\n6gBKjPIZLuUzXMpneJTLcCmf4VI+w6Nchiu0fJb0RaIiIiIiIsWm1DvoIiIiIiJFRQW6iIiIiEgB\nKdkC3cwuMrMNZrbJzL4SdTyFzszmmNnjZrbezF42s78Nln/LzLab2drgcUnGPjcF+d1gZhdGF31h\nMrNGM3spyFt9sGyGmf3ezDYG/04PlpuZ/e8gny+a2RnRRl9YzGxRxjm41sz2mdnf6fzMnZndaWa7\nzGxdxrJhn49m9plg+41m9pko3kvUDpPLvzezV4N8PWhm04Ll883sQMY5+o8Z+ywL/kZsCvJtUbyf\nqB0mn8P+3dbnftph8nlvRi4bzWxtsFzn5xEcoTbK/99Ody+5BxADNgPHA+OABmBx1HEV8gM4Fjgj\neD4ZeA1YDHwLuCHL9ouDvCaABUG+Y1G/j0J6AI1A9aBl3we+Ejz/CvC94PklwO8AA94FrIw6/kJ9\nBL/fbwLzdH4OK2/vBc4A1mUsG9b5CMwAtgT/Tg+eT4/6vRVILj8IxIPn38vI5fzM7QYd53ngnCDP\nvwMujvq9FVA+h/W7rc/9I+dz0PofAN8Inuv8PHIuD1cb5f1vZ6l20M8CNrn7FnfvBe4BLo04poLm\n7jvcfU3wfD+wHph1hF0uBe5x9x53fx3YRDrvcmSXAv8WPP834M8zlt/lac8B08zs2CgCLAIXAJvd\nvekI2+j8HMTdnwTaBi0e7vl4IfB7d29z93bg98BF+Y++sGTLpbs/6u7J4MfngNlHOkaQzynu/qyn\nP8Hv4q38l5XDnJuHc7jfbX3uB46Uz6AL/nHgl0c6hs7PtCPURnn/21mqBfos4I2Mn7dx5GJTMpjZ\nfOB0YGWw6Lrgq5o7B77GQTnOhQOPmtlqM7s2WHa0u++A9C8+cFSwXPnM3VUc+uGi83Pkhns+Kq+5\n+RzpLtqABWb2gpk9YWbnBctmkc7fAOXy7Ybzu61zMzfnATvdfWPGMp2fORhUG+X9b2epFujZxklp\nPskcmNkk4AHg79x9H/BT4B3AUmAH6a/GQDnOxbnufgZwMfBFM3vvEbZVPnNgZuOAjwK/Chbp/MyP\nw+VPeR2CmX0VSAK/CBbtAOa6++nAl4C7zWwKyuVQhvu7rXzm5moObXDo/MxBltrosJtmWTai87NU\nC/RtwJyMn2cDzRHFUjTMrJL0CfgLd/81gLvvdPd+d08B/8xbwwSU4yG4e3Pw7y7gQdK52zkwdCX4\nd1ewufKZm4uBNe6+E3R+hmC456PyegTBhV8fBj4RDAsgGIrRGjxfTXqc9Imkc5k5DEa5zDCC322d\nm0MwszhwOXDvwDKdn0PLVhsxBn87S7VAXwUsNLMFQcftKmBFxDEVtGBc2s+A9e5+e8byzHHQlwED\nV4WvAK4ys4SZLQAWkr6gRAAzm2hmkweek76AbB3pvA1cvf0Z4N+D5yuATwdXgL8L2Dvw9Zkc4pDu\nj87PURvu+fgI8EEzmx4MOfhgsKzsmdlFwP8LfNTduzKW15hZLHh+POlzcUuQz/1m9q7g7++neSv/\nZW8Ev9v63B/anwGvuvvBoSs6P4/scLURY/G3c6yuhB3rB+kraV8j/b/Br0YdT6E/gPeQ/rrlRWBt\n8LgE+DnwUrB8BXBsxj5fDfK7gTK8unuIfB5PehaBBuDlgXMQmAk8BmwM/p0RLDfgJ0E+XwKWR/0e\nCu0BTABagakZy3R+5p6/X5L+OruPdDfn8yM5H0mPr94UPP4y6vdVQLncRHqM6cDfz38Mtr0i+BvQ\nAKwBPpJxnOWkC8/NwI8J7u5dbo/D5HPYv9v63D98PoPl/wr81aBtdX4eOZeHq43y/rfTgp1ERERE\nRKQAlOoQFxERERGRoqQCXURERESkgKhAFxEREREpICrQRUREREQKiAp0EREREZECogJdRERERKSA\nqEAXERERESkg/xeCBzSYfQK7xQAAAABJRU5ErkJggg==\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "\n", "loss = np.array(train_summary.read_scalar(\"Loss\"))\n", @@ -544,7 +516,9 @@ { "cell_type": "code", "execution_count": 17, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "predictions = trained_model.predict(cc_rdd_test).collect()\n", @@ -596,21 +570,21 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 2", + "display_name": "Python 3", "language": "python", - "name": "python2" + "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", - "version": 2 + "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", - "pygments_lexer": "ipython2", - "version": "2.7.14" + "pygments_lexer": "ipython3", + "version": "3.5.4" } }, "nbformat": 4, diff --git a/elephantscale/notebooks/feedforward-iris-pipeline.ipynb b/elephantscale/notebooks/feedforward-iris-pipeline.ipynb index b481de8..878eff7 100644 --- a/elephantscale/notebooks/feedforward-iris-pipeline.ipynb +++ b/elephantscale/notebooks/feedforward-iris-pipeline.ipynb @@ -4,10 +4,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Feedforward Neural Network on the Iris Dataset\n", - "# Using Pipeline API\n", + "# Feedforward Neural Network on the Iris Dataset Using Pipeline API\n", "\n", - "Fisher's Iris dataset \n", + "This notebook shows how to do the iris classification on the Spark mllib dataframe-based pipeline API (as opposed to the RDD API). For introduction, first see the feedforward-iris.ipynb.\n", + "\n", + "### Fisher's Iris dataset \n", "\n", "We are going to try the same Iris dataset as on feedforward-iris.ipynb, but this time using\n", "Spark MLLib's Pipeline-based API.\n", @@ -41,7 +42,9 @@ { "cell_type": "code", "execution_count": 1, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "%matplotlib inline\n", @@ -103,10 +106,12 @@ { "cell_type": "code", "execution_count": 2, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ - "learning_rate = 0.1\n", + "learning_rate = 0.01\n", "training_epochs = 100\n", "batch_size = 16\n", "display_step = 1\n", @@ -194,7 +199,9 @@ { "cell_type": "code", "execution_count": 4, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "iris_training = spark.read.csv(\"../data/iris/iris_training.csv\", header=True, inferSchema=\"true\", mode=\"DROPMALFORMED\")\n", @@ -204,7 +211,9 @@ { "cell_type": "code", "execution_count": 5, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "iris_training = iris_training.select([col(c).cast(\"double\") for c in iris_training.columns])\n", @@ -227,7 +236,9 @@ { "cell_type": "code", "execution_count": 6, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "assembler = VectorAssembler(inputCols=['c1','c2','c3','c4'], outputCol=\"assembled\")\n", @@ -450,7 +461,9 @@ { "cell_type": "code", "execution_count": 33, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "y_pred = np.array(predictionDF.select('prediction').collect())\n", @@ -496,28 +509,30 @@ { "cell_type": "code", "execution_count": null, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { - "display_name": "Python 2", + "display_name": "Python 3", "language": "python", - "name": "python2" + "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", - "version": 2 + "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", - "pygments_lexer": "ipython2", - "version": "2.7.14" + "pygments_lexer": "ipython3", + "version": "3.5.4" } }, "nbformat": 4, diff --git a/elephantscale/notebooks/feedforward-iris.ipynb b/elephantscale/notebooks/feedforward-iris.ipynb index 7872678..dc1c6ce 100644 --- a/elephantscale/notebooks/feedforward-iris.ipynb +++ b/elephantscale/notebooks/feedforward-iris.ipynb @@ -6,7 +6,9 @@ "source": [ "# Feedforward Neural Network on the Iris Dataset\n", "\n", - "Fisher's Iris dataset \n", + "This notebook shows how to do a simple feedforward neural Network on BigDL using the RDD based API. If you are interested in seeing the same example using the dataframe-based Spark MLLib pipeline, please see feedforward-iris-pipeline.ipynb.\n", + "\n", + "### Fisher's Iris dataset \n", "\n", "This dataset contains 150 samples, with 4 dimensions, as follows:\n", "\n", @@ -36,7 +38,9 @@ { "cell_type": "code", "execution_count": 1, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "%matplotlib inline\n", @@ -96,7 +100,9 @@ { "cell_type": "code", "execution_count": 2, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "learning_rate = 0.1\n", @@ -186,7 +192,9 @@ { "cell_type": "code", "execution_count": 4, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "iris_training = spark.read.csv(\"../data/iris/iris_training.csv\", header=True, inferSchema=\"true\", mode=\"DROPMALFORMED\")\n", @@ -201,7 +209,9 @@ { "cell_type": "code", "execution_count": 5, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "iris_k_train = iris_training.rdd.map(list)\n", @@ -591,7 +601,9 @@ { "cell_type": "code", "execution_count": 14, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "predictions = trained_model.predict(iris_rdd_test).collect()\n", @@ -645,7 +657,9 @@ { "cell_type": "code", "execution_count": 16, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "def map_predict_label(outputs, threshold):\n", @@ -659,7 +673,9 @@ { "cell_type": "code", "execution_count": 17, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "true_positive_rate = list()\n", @@ -673,28 +689,30 @@ { "cell_type": "code", "execution_count": null, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { - "display_name": "Python 2", + "display_name": "Python 3", "language": "python", - "name": "python2" + "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", - "version": 2 + "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", - "pygments_lexer": "ipython2", - "version": "2.7.14" + "pygments_lexer": "ipython3", + "version": "3.5.4" } }, "nbformat": 4, diff --git a/elephantscale/notebooks/lstm-20news.ipynb b/elephantscale/notebooks/lstm-20news.ipynb index 04cb4be..7df4484 100644 --- a/elephantscale/notebooks/lstm-20news.ipynb +++ b/elephantscale/notebooks/lstm-20news.ipynb @@ -6,13 +6,20 @@ "source": [ "# LSTM Example: 20 Newsgroup Classification\n", "\n", - "This shows how to do an LSTM classification on the 20 newsgroup collection" + "This shows how to do an LSTM classification on the 20 newsgroup collection\n", + "\n", + "## Requirements\n", + "\n", + "Running this notebook requires a minimum of 16GB executor memory. Starting it up in an ordinary jupyter notebook \n", + "won't work as you'll get an out of memory exception. Quit jupyter and make sure you start this with at least --executor-memory 16gb" ] }, { "cell_type": "code", "execution_count": 1, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "import itertools\n", @@ -32,7 +39,9 @@ { "cell_type": "code", "execution_count": 2, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "\n", @@ -90,7 +99,9 @@ { "cell_type": "code", "execution_count": null, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "\n", @@ -112,7 +123,9 @@ { "cell_type": "code", "execution_count": 3, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "\n", @@ -132,7 +145,9 @@ { "cell_type": "code", "execution_count": 4, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "\n", @@ -283,28 +298,30 @@ { "cell_type": "code", "execution_count": null, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { - "display_name": "Python 2", + "display_name": "Python 3", "language": "python", - "name": "python2" + "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", - "version": 2 + "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", - "pygments_lexer": "ipython2", - "version": "2.7.14" + "pygments_lexer": "ipython3", + "version": "3.5.4" } }, "nbformat": 4, diff --git a/elephantscale/notebooks/lstm-stocks.ipynb b/elephantscale/notebooks/lstm-stocks.ipynb index 506fe16..ba97dea 100644 --- a/elephantscale/notebooks/lstm-stocks.ipynb +++ b/elephantscale/notebooks/lstm-stocks.ipynb @@ -6,13 +6,17 @@ "source": [ "# LSTM Stock Market Example\n", "\n", - "Wouldn't you like to predict the stock market? (Don't try this at home). Here's an example of how we can use LSTMs to predict patterns in the S&P 500." + "Wouldn't you like to predict the stock market? (Don't try this at home). Here's an example of how we can use LSTMs to predict patterns in the S&P 500.\n", + "\n", + "What is an LSTM? An LSTM (Long Short Term Memory) is a recurrent neural network that will help us to predict patterns in time-series data." ] }, { "cell_type": "code", "execution_count": 3, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "\n", @@ -33,7 +37,9 @@ { "cell_type": "code", "execution_count": 3, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "learning_rate = 0.1\n", @@ -60,7 +66,9 @@ { "cell_type": "code", "execution_count": 8, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "#Step 1 Load Data\n", @@ -70,7 +78,9 @@ { "cell_type": "code", "execution_count": 9, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "X_train_rdd = sc.parallelize(X_train).map(lambda x: x.flatten())\n", @@ -82,7 +92,9 @@ { "cell_type": "code", "execution_count": 11, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [ "x_train_sample = X_train_rdd.zip(y_train_rdd).map(lambda x:Sample.from_ndarray(x[0], x[1]))\n", @@ -139,7 +151,7 @@ "source": [ "# Training The Model\n", "\n", - "Notice that the model is a regression, not classificaiton model" + "Notice that the model is a regression, not a classification model. This is why we are using the MSECriterion. What we are trying to do is predict the next value of the S&P500 given the input sequence of S&P 500 values." ] }, { @@ -178,13 +190,16 @@ "source": [ "# Making Predictions\n", "\n", - "Here we are going to make predictions based on the model. We will use sliding windows of length 50." + "Here we are going to make predictions based on the model. We will use sliding windows of length 50.\n", + "\n", + "Here's how this works: We're going to Take sequences of length 50, and then use those 50 steps as inputs to the model. Then we will get the output of the model for the next step, and continue on making predictions." ] }, { "cell_type": "code", "execution_count": 43, "metadata": { + "collapsed": true, "scrolled": false }, "outputs": [], @@ -212,10 +227,20 @@ "predictions = prediction_seqs" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Plots\n", + "\n", + "Let us go ahead and plot some of our predictions." + ] + }, { "cell_type": "code", "execution_count": null, "metadata": { + "collapsed": true, "scrolled": false }, "outputs": [], @@ -228,16 +253,30 @@ { "cell_type": "code", "execution_count": null, - "metadata": {}, + "metadata": { + "collapsed": true + }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { - "display_name": "Python 2", + "display_name": "Python 3", "language": "python", - "name": "python2" + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.4" } }, "nbformat": 4, diff --git a/elephantscale/notebooks/sp500.csv b/elephantscale/notebooks/sp500.csv new file mode 100644 index 0000000..f770a07 --- /dev/null +++ b/elephantscale/notebooks/sp500.csv @@ -0,0 +1,4171 @@ +1455.219971 +1399.420044 +1402.109985 +1403.449951 +1441.469971 +1457.599976 +1438.560059 +1432.25 +1449.680054 +1465.150024 +1455.140015 +1455.900024 +1445.569946 +1441.359985 +1401.530029 +1410.030029 +1404.089966 +1398.560059 +1360.160034 +1394.459961 +1409.280029 +1409.119995 +1424.969971 +1424.369995 +1424.23999 +1441.719971 +1411.709961 +1416.829956 +1387.119995 +1389.939941 +1402.050049 +1387.670044 +1388.26001 +1346.089966 +1352.170044 +1360.689941 +1353.430054 +1333.359985 +1348.050049 +1366.420044 +1379.189941 +1381.76001 +1409.170044 +1391.280029 +1355.619995 +1366.699951 +1401.689941 +1395.069946 +1383.619995 +1359.150024 +1392.140015 +1458.469971 +1464.469971 +1456.630005 +1493.869995 +1500.640015 +1527.349976 +1527.459961 +1523.859985 +1507.72998 +1508.52002 +1487.920044 +1498.579956 +1505.969971 +1494.72998 +1487.369995 +1501.339966 +1516.349976 +1504.459961 +1500.589966 +1467.170044 +1440.51001 +1356.560059 +1401.439941 +1441.609985 +1427.469971 +1434.540039 +1429.859985 +1477.439941 +1460.98999 +1464.920044 +1452.430054 +1468.25 +1446.290039 +1415.099976 +1409.569946 +1432.630005 +1424.170044 +1412.140015 +1383.050049 +1407.810059 +1420.959961 +1452.359985 +1466.040039 +1447.800049 +1437.209961 +1406.949951 +1400.719971 +1373.859985 +1399.050049 +1381.52002 +1378.02002 +1422.449951 +1420.599976 +1448.810059 +1477.26001 +1467.630005 +1457.839966 +1471.359985 +1461.670044 +1456.949951 +1446 +1469.439941 +1470.540039 +1478.72998 +1464.459961 +1486 +1475.949951 +1479.130005 +1452.180054 +1441.47998 +1455.310059 +1450.550049 +1454.819946 +1442.390015 +1454.599976 +1469.540039 +1446.22998 +1456.670044 +1478.900024 +1475.619995 +1480.880005 +1492.920044 +1495.839966 +1509.97998 +1510.48999 +1493.73999 +1481.959961 +1495.569946 +1480.189941 +1464.290039 +1474.469971 +1452.420044 +1449.619995 +1419.890015 +1430.829956 +1438.099976 +1438.699951 +1452.560059 +1462.930054 +1479.319946 +1482.800049 +1472.869995 +1460.25 +1471.839966 +1491.560059 +1484.430054 +1479.849976 +1496.069946 +1491.719971 +1499.47998 +1498.130005 +1505.969971 +1508.310059 +1506.449951 +1514.089966 +1509.839966 +1502.589966 +1517.680054 +1520.77002 +1507.079956 +1492.25 +1502.51001 +1494.5 +1489.26001 +1481.98999 +1484.910034 +1480.869995 +1465.810059 +1444.51001 +1459.900024 +1451.339966 +1449.050049 +1448.719971 +1439.030029 +1427.209961 +1426.569946 +1458.290039 +1436.51001 +1436.22998 +1426.459961 +1434.319946 +1436.280029 +1408.98999 +1402.030029 +1387.02002 +1364.589966 +1329.780029 +1374.170044 +1374.619995 +1349.969971 +1342.130005 +1388.76001 +1396.930054 +1395.780029 +1398.130005 +1364.900024 +1364.439941 +1379.579956 +1398.660034 +1429.400024 +1421.219971 +1428.319946 +1426.689941 +1432.189941 +1431.869995 +1409.280029 +1400.140015 +1365.97998 +1351.26001 +1382.949951 +1389.810059 +1372.319946 +1367.719971 +1342.619995 +1347.349976 +1322.359985 +1341.77002 +1348.969971 +1336.089966 +1341.930054 +1314.949951 +1315.22998 +1324.969971 +1376.540039 +1351.459961 +1343.550049 +1369.890015 +1380.199951 +1371.180054 +1359.98999 +1340.930054 +1312.150024 +1322.73999 +1305.599976 +1264.73999 +1274.859985 +1305.949951 +1315.189941 +1328.920044 +1334.219971 +1320.280029 +1283.27002 +1347.560059 +1333.339966 +1298.349976 +1295.859985 +1300.800049 +1313.27002 +1326.819946 +1318.550049 +1326.650024 +1329.469971 +1347.969971 +1342.540039 +1342.900024 +1360.400024 +1364.300049 +1357.51001 +1354.949951 +1364.170044 +1373.72998 +1366.01001 +1373.469971 +1349.469971 +1354.310059 +1352.26001 +1340.890015 +1332.530029 +1314.76001 +1330.310059 +1318.800049 +1315.920044 +1326.609985 +1301.530029 +1278.939941 +1255.27002 +1252.819946 +1245.859985 +1267.650024 +1257.939941 +1239.939941 +1241.22998 +1234.180054 +1241.410034 +1253.800049 +1261.890015 +1264.73999 +1233.420044 +1180.160034 +1197.660034 +1166.709961 +1173.560059 +1150.530029 +1170.810059 +1142.619995 +1122.140015 +1117.579956 +1139.829956 +1152.689941 +1182.170044 +1153.290039 +1147.949951 +1160.329956 +1145.869995 +1106.459961 +1103.25 +1151.439941 +1128.430054 +1137.589966 +1168.380005 +1165.890015 +1183.5 +1179.680054 +1191.810059 +1238.160034 +1253.689941 +1242.97998 +1224.359985 +1209.469971 +1228.75 +1234.52002 +1253.050049 +1249.459961 +1266.439941 +1267.430054 +1248.579956 +1266.609985 +1263.51001 +1261.199951 +1255.540039 +1255.180054 +1245.670044 +1248.920044 +1249.439941 +1284.98999 +1288.48999 +1291.959961 +1312.829956 +1309.380005 +1289.050049 +1293.170044 +1277.890015 +1267.930054 +1248.079956 +1255.819946 +1260.670044 +1267.109985 +1283.569946 +1270.030029 +1276.959961 +1264.959961 +1254.390015 +1255.849976 +1241.599976 +1219.869995 +1214.359985 +1208.430054 +1212.579956 +1223.140015 +1237.040039 +1225.349976 +1218.599976 +1216.76001 +1211.069946 +1226.199951 +1224.380005 +1236.719971 +1234.449951 +1219.23999 +1190.589966 +1198.780029 +1181.52002 +1180.180054 +1208.140015 +1215.680054 +1202.449951 +1214.439941 +1207.709961 +1215.02002 +1210.849976 +1191.030029 +1171.650024 +1190.48999 +1202.930054 +1205.819946 +1204.52002 +1211.22998 +1215.930054 +1220.75 +1214.349976 +1200.47998 +1204.400024 +1183.530029 +1183.430054 +1190.160034 +1191.290039 +1186.72998 +1178.02002 +1181.660034 +1161.969971 +1171.410034 +1157.26001 +1165.310059 +1162.089966 +1184.930054 +1179.209961 +1161.51001 +1148.560059 +1129.030029 +1133.579956 +1132.939941 +1131.73999 +1106.400024 +1085.780029 +1092.540039 +1038.77002 +1032.73999 +1016.099976 +984.539978 +965.799988 +1003.450012 +1012.27002 +1007.039978 +1018.609985 +1040.939941 +1038.550049 +1051.329956 +1072.280029 +1069.630005 +1071.380005 +1062.439941 +1056.75 +1080.98999 +1097.430054 +1091.650024 +1089.97998 +1097.540039 +1077.089966 +1068.609985 +1073.47998 +1089.900024 +1084.780029 +1085.199951 +1100.089966 +1104.609985 +1078.300049 +1059.790039 +1059.780029 +1084.099976 +1087.199951 +1102.839966 +1118.859985 +1115.800049 +1118.540039 +1120.310059 +1118.329956 +1139.089966 +1141.209961 +1142.23999 +1138.650024 +1151.060059 +1142.660034 +1137.030029 +1150.339966 +1157.420044 +1149.5 +1128.52002 +1140.199951 +1139.449951 +1129.900024 +1144.800049 +1170.349976 +1167.099976 +1158.310059 +1139.930054 +1136.76001 +1137.069946 +1119.380005 +1123.089966 +1134.359985 +1142.920044 +1149.560059 +1139.930054 +1144.890015 +1144.650024 +1149.369995 +1157.130005 +1161.02002 +1148.079956 +1154.670044 +1165.27002 +1172.51001 +1164.890015 +1160.709961 +1155.140015 +1156.550049 +1145.599976 +1138.410034 +1146.189941 +1127.569946 +1138.880005 +1127.579956 +1119.310059 +1128.180054 +1132.150024 +1133.280029 +1133.060059 +1100.640015 +1113.569946 +1130.199951 +1122.199951 +1094.439941 +1090.02002 +1083.51001 +1080.170044 +1096.219971 +1111.939941 +1107.5 +1118.51001 +1116.47998 +1104.180054 +1083.339966 +1097.97998 +1080.949951 +1089.839966 +1109.430054 +1109.380005 +1109.890015 +1106.72998 +1131.780029 +1153.839966 +1146.140015 +1162.77002 +1157.540039 +1164.310059 +1168.26001 +1165.579956 +1154.089966 +1153.040039 +1166.160034 +1165.550049 +1170.290039 +1151.849976 +1153.589966 +1148.699951 +1131.869995 +1138.48999 +1144.579956 +1147.390015 +1146.540039 +1136.76001 +1125.400024 +1126.339966 +1122.72998 +1125.290039 +1117.800049 +1130.469971 +1103.689941 +1111.01001 +1102.550049 +1128.369995 +1126.069946 +1124.469971 +1125.170044 +1107.829956 +1100.959961 +1093.140015 +1091.47998 +1076.319946 +1065.449951 +1076.920044 +1086.459961 +1084.560059 +1073.430054 +1052.670044 +1049.48999 +1088.849976 +1073.01001 +1054.98999 +1074.560059 +1097.280029 +1091.069946 +1098.22998 +1106.589966 +1091.880005 +1079.880005 +1086.02002 +1097.079956 +1083.819946 +1074.550049 +1067.660034 +1064.660034 +1067.140015 +1040.680054 +1040.689941 +1049.900024 +1029.150024 +1027.530029 +1030.73999 +1013.599976 +1020.26001 +1009.559998 +1007.27002 +1036.170044 +1037.140015 +1019.98999 +1006.289978 +989.140015 +992.719971 +976.140015 +973.530029 +990.640015 +989.820007 +968.650024 +948.090027 +953.98999 +989.030029 +976.97998 +952.830017 +920.469971 +927.369995 +921.390015 +917.929993 +900.940002 +906.039978 +881.559998 +847.75 +819.849976 +797.700012 +843.429993 +838.679993 +852.840027 +898.960022 +902.780029 +911.619995 +884.659973 +864.23999 +834.599976 +859.570007 +876.77002 +905.460022 +908.640015 +903.799988 +884.210022 +919.619995 +930.25 +928.77002 +950.700012 +937.429993 +949.359985 +962.700012 +940.859985 +947.950012 +934.820007 +917.869995 +917.799988 +916.070007 +878.02002 +893.400024 +879.150024 +893.919983 +902.960022 +909.580017 +909.450012 +886.909973 +889.809998 +891.099976 +873.52002 +869.460022 +843.320007 +845.390015 +833.700012 +819.289978 +839.659973 +854.950012 +827.369995 +815.280029 +847.909973 +827.909973 +818.950012 +800.580017 +785.280029 +798.549988 +776.76001 +803.919983 +835.320007 +841.440002 +881.27002 +860.02002 +879.200012 +884.390015 +899.719971 +890.159973 +896.140015 +882.5 +897.650024 +890.22998 +882.150024 +890.710022 +885.76001 +900.960022 +908.349976 +915.390015 +923.76001 +902.650024 +894.73999 +876.190002 +882.950012 +882.530029 +904.27002 +909.830017 +900.359985 +896.73999 +914.150024 +933.76001 +930.549988 +932.869995 +913.309998 +938.869995 +936.309998 +934.530029 +920.75 +917.580017 +906.549988 +912.22998 +892 +904.450012 +904.960022 +901.580017 +889.47998 +910.400024 +902.98999 +891.119995 +884.25 +895.76001 +897.380005 +892.469971 +889.659973 +875.400024 +879.390015 +879.820007 +909.030029 +908.590027 +929.01001 +922.929993 +909.929993 +927.570007 +927.570007 +926.26001 +931.659973 +918.219971 +914.599976 +901.780029 +887.619995 +878.359985 +887.340027 +861.400024 +847.47998 +858.539978 +864.359985 +844.609985 +855.700012 +860.320007 +848.200012 +843.590027 +838.150024 +829.690002 +835.969971 +829.200012 +818.679993 +817.369995 +834.890015 +851.169983 +845.130005 +837.099976 +848.169983 +832.580017 +838.570007 +827.549988 +837.280029 +841.150024 +834.809998 +821.98999 +829.849976 +822.099976 +828.890015 +807.47998 +800.72998 +804.190002 +831.900024 +833.27002 +862.789978 +866.450012 +874.02002 +875.669983 +895.789978 +864.22998 +874.73999 +869.950012 +868.52002 +863.5 +848.179993 +858.47998 +880.900024 +876.450012 +878.849976 +879.929993 +878.289978 +865.98999 +871.580017 +868.299988 +885.22998 +890.809998 +879.909973 +893.580017 +892.01001 +911.369995 +919.02002 +911.429993 +898.809998 +914.840027 +917.840027 +916.919983 +916.299988 +930.080017 +926.549988 +934.390015 +929.619995 +920.27002 +933.409973 +945.109985 +942.299988 +939.280029 +946.669983 +944.299988 +920.77002 +919.72998 +923.419983 +931.869995 +933.219971 +951.47998 +953.219971 +949.640015 +963.590027 +967 +971.559998 +986.23999 +990.140015 +987.76001 +975.929993 +984.840027 +997.47998 +998.51001 +988.609985 +1010.73999 +1011.659973 +1010.090027 +994.700012 +995.690002 +981.640015 +983.450012 +975.320007 +985.820007 +976.219971 +974.5 +982.320007 +993.75 +985.700012 +1004.419983 +1007.840027 +1002.210022 +988.700012 +998.140015 +1003.859985 +1000.419983 +994.090027 +981.72998 +993.320007 +978.799988 +988.109985 +988.609985 +981.599976 +998.679993 +996.52002 +989.280029 +987.48999 +990.309998 +980.150024 +982.820007 +965.460022 +967.080017 +974.119995 +977.590027 +980.590027 +990.349976 +984.030029 +990.51001 +990.669983 +999.73999 +1002.349976 +1000.299988 +1003.27002 +993.059998 +993.710022 +996.72998 +996.789978 +1002.840027 +1008.01001 +1021.98999 +1026.27002 +1027.969971 +1021.390015 +1031.640015 +1023.169983 +1010.919983 +1016.419983 +1018.630005 +1014.809998 +1029.319946 +1025.969971 +1039.579956 +1036.300049 +1022.820007 +1029.030029 +1009.380005 +1003.27002 +996.849976 +1006.580017 +995.969971 +1018.219971 +1020.23999 +1029.849976 +1034.349976 +1039.25 +1033.780029 +1038.72998 +1038.060059 +1045.349976 +1049.47998 +1046.76001 +1050.069946 +1039.319946 +1044.680054 +1046.030029 +1030.359985 +1033.77002 +1028.910034 +1031.130005 +1046.790039 +1048.109985 +1046.939941 +1050.709961 +1059.02002 +1053.25 +1051.810059 +1058.050049 +1053.209961 +1047.109985 +1046.569946 +1058.530029 +1058.410034 +1050.349976 +1043.630005 +1034.150024 +1042.439941 +1033.650024 +1035.280029 +1052.079956 +1053.890015 +1058.449951 +1058.199951 +1070.119995 +1066.619995 +1064.72998 +1069.719971 +1061.5 +1069.300049 +1060.180054 +1059.050049 +1071.209961 +1074.140015 +1068.040039 +1075.130005 +1076.47998 +1089.180054 +1088.660034 +1092.939941 +1096.02002 +1094.040039 +1095.890015 +1109.47998 +1109.640015 +1111.920044 +1108.47998 +1122.219971 +1123.670044 +1126.329956 +1131.920044 +1121.859985 +1127.22998 +1121.219971 +1130.52002 +1132.050049 +1139.829956 +1138.77002 +1147.619995 +1143.939941 +1141.550049 +1155.369995 +1144.050049 +1128.47998 +1134.109985 +1131.130005 +1135.26001 +1136.030029 +1126.52002 +1128.589966 +1142.76001 +1139.810059 +1145.540039 +1157.76001 +1152.109985 +1145.810059 +1156.98999 +1151.819946 +1147.060059 +1144.109985 +1140.98999 +1139.089966 +1143.670044 +1144.910034 +1144.939941 +1155.969971 +1149.099976 +1151.030029 +1154.869995 +1156.859985 +1147.199951 +1140.579956 +1123.890015 +1106.780029 +1120.569946 +1104.48999 +1110.699951 +1123.75 +1122.319946 +1109.780029 +1095.400024 +1093.949951 +1091.329956 +1109.189941 +1108.060059 +1122.469971 +1127 +1126.209961 +1132.170044 +1141.810059 +1150.569946 +1148.160034 +1140.530029 +1139.319946 +1145.199951 +1129.439941 +1128.170044 +1128.839966 +1134.609985 +1135.819946 +1118.150024 +1124.089966 +1139.930054 +1140.599976 +1135.530029 +1138.109985 +1122.410034 +1113.890015 +1107.300049 +1117.48999 +1119.550049 +1121.530029 +1113.98999 +1098.699951 +1087.119995 +1095.449951 +1097.280029 +1096.439941 +1095.699951 +1084.099976 +1091.48999 +1088.680054 +1089.189941 +1093.560059 +1095.410034 +1113.050049 +1114.939941 +1121.280029 +1120.680054 +1121.199951 +1124.98999 +1116.640015 +1122.5 +1140.420044 +1142.180054 +1131.329956 +1136.469971 +1125.290039 +1132.01001 +1133.560059 +1132.050049 +1135.02002 +1130.300049 +1134.410034 +1144.060059 +1140.650024 +1134.430054 +1133.349976 +1136.199951 +1140.839966 +1128.939941 +1125.380005 +1116.209961 +1118.329956 +1109.109985 +1112.810059 +1114.349976 +1115.140015 +1111.469971 +1106.689941 +1101.390015 +1100.900024 +1108.670044 +1093.880005 +1096.839966 +1086.199951 +1084.069946 +1094.829956 +1095.420044 +1100.430054 +1101.719971 +1106.619995 +1099.689941 +1098.630005 +1080.699951 +1063.969971 +1065.219971 +1079.040039 +1075.790039 +1063.22998 +1064.800049 +1079.339966 +1081.709961 +1095.170044 +1091.22998 +1098.349976 +1095.680054 +1096.189941 +1104.959961 +1105.089966 +1107.77002 +1099.150024 +1104.23999 +1105.910034 +1118.310059 +1113.630005 +1121.300049 +1116.27002 +1118.380005 +1123.920044 +1125.819946 +1128.329956 +1120.369995 +1123.5 +1128.550049 +1122.199951 +1129.300049 +1113.560059 +1108.359985 +1110.109985 +1103.52002 +1110.060059 +1114.800049 +1114.579956 +1131.5 +1135.170044 +1134.47998 +1142.050049 +1130.650024 +1122.140015 +1124.390015 +1121.839966 +1113.650024 +1103.290039 +1108.199951 +1114.02002 +1103.22998 +1103.660034 +1106.48999 +1095.73999 +1094.800049 +1111.089966 +1125.400024 +1127.439941 +1130.199951 +1130.51001 +1130.560059 +1143.199951 +1161.670044 +1166.170044 +1164.890015 +1164.079956 +1162.910034 +1173.47998 +1184.170044 +1183.810059 +1175.430054 +1181.939941 +1183.550049 +1170.339966 +1177.23999 +1176.939941 +1181.76001 +1182.650024 +1178.569946 +1173.819946 +1191.369995 +1190.329956 +1191.170044 +1190.25 +1177.069946 +1182.810059 +1189.23999 +1188 +1198.680054 +1203.380005 +1205.719971 +1203.209961 +1194.199951 +1194.650024 +1205.449951 +1209.569946 +1210.130005 +1204.920044 +1213.540039 +1213.449951 +1213.550049 +1211.920044 +1202.079956 +1188.050049 +1183.73999 +1187.890015 +1186.189941 +1190.25 +1182.98999 +1187.699951 +1177.449951 +1184.52002 +1195.97998 +1184.630005 +1175.410034 +1167.869995 +1163.75 +1168.410034 +1174.069946 +1174.550049 +1171.359985 +1181.27002 +1189.410034 +1193.189941 +1189.890015 +1203.030029 +1201.719971 +1202.300049 +1191.98999 +1197.01001 +1205.300049 +1206.140015 +1210.119995 +1210.339966 +1200.75 +1201.589966 +1184.160034 +1190.800049 +1200.199951 +1211.369995 +1203.599976 +1210.410034 +1210.079956 +1210.469971 +1222.119995 +1225.310059 +1219.430054 +1207.01001 +1209.25 +1200.079956 +1206.829956 +1197.75 +1188.069946 +1190.209961 +1189.650024 +1183.780029 +1171.709961 +1172.530029 +1171.420044 +1174.280029 +1165.359985 +1181.410034 +1180.589966 +1172.920044 +1176.119995 +1181.390015 +1184.069946 +1191.140015 +1181.199951 +1181.209961 +1187.76001 +1173.790039 +1162.050049 +1142.619995 +1145.97998 +1152.780029 +1137.5 +1159.949951 +1152.119995 +1162.099976 +1151.829956 +1156.380005 +1143.219971 +1156.849976 +1162.160034 +1161.170044 +1175.650024 +1172.630005 +1171.349976 +1178.839966 +1166.219971 +1171.109985 +1159.359985 +1154.050049 +1165.689941 +1173.800049 +1185.560059 +1191.079956 +1189.280029 +1193.859985 +1194.069946 +1190.01001 +1197.619995 +1198.780029 +1191.5 +1202.219971 +1204.290039 +1196.02002 +1197.51001 +1197.26001 +1194.670044 +1200.930054 +1198.109985 +1200.819946 +1203.910034 +1206.579956 +1210.959961 +1216.959961 +1216.099976 +1213.609985 +1213.880005 +1200.72998 +1191.569946 +1190.689941 +1201.569946 +1199.849976 +1191.329956 +1194.439941 +1204.98999 +1194.939941 +1197.869995 +1211.859985 +1219.439941 +1222.209961 +1223.290039 +1226.5 +1227.920044 +1221.130005 +1229.349976 +1235.199951 +1227.040039 +1233.680054 +1229.030029 +1231.160034 +1236.790039 +1243.719971 +1234.180054 +1235.349976 +1244.119995 +1245.040039 +1235.859985 +1226.420044 +1223.130005 +1231.380005 +1229.130005 +1237.810059 +1230.390015 +1233.869995 +1219.339966 +1220.23999 +1219.02002 +1219.709961 +1221.72998 +1217.589966 +1209.589966 +1212.369995 +1205.099976 +1212.280029 +1208.410034 +1220.329956 +1221.589966 +1218.02002 +1233.390015 +1236.359985 +1231.670044 +1241.47998 +1240.560059 +1231.199951 +1227.160034 +1227.72998 +1237.910034 +1231.02002 +1221.339966 +1210.199951 +1214.619995 +1215.290039 +1215.630005 +1215.660034 +1216.890015 +1227.680054 +1228.810059 +1226.699951 +1214.469971 +1196.390015 +1191.48999 +1195.900024 +1187.329956 +1184.869995 +1177.680054 +1176.839966 +1186.569946 +1190.099976 +1178.140015 +1195.76001 +1177.800049 +1179.589966 +1199.380005 +1196.540039 +1191.380005 +1178.900024 +1198.410034 +1207.01001 +1202.76001 +1214.76001 +1219.939941 +1220.140015 +1222.810059 +1218.589966 +1220.650024 +1230.959961 +1234.719971 +1233.76001 +1229.01001 +1231.209961 +1242.800049 +1248.27002 +1254.849976 +1261.22998 +1265.609985 +1268.25 +1257.459961 +1257.47998 +1249.47998 +1264.670044 +1265.079956 +1262.089966 +1263.699951 +1257.369995 +1255.839966 +1259.369995 +1260.430054 +1267.430054 +1272.73999 +1270.939941 +1267.319946 +1259.920044 +1259.619995 +1262.790039 +1268.119995 +1268.660034 +1256.540039 +1258.170044 +1254.420044 +1248.290039 +1268.800049 +1273.459961 +1273.47998 +1285.449951 +1290.150024 +1289.689941 +1294.180054 +1286.060059 +1287.609985 +1282.930054 +1277.930054 +1285.040039 +1261.48999 +1263.819946 +1266.859985 +1264.680054 +1273.829956 +1283.719971 +1285.189941 +1280.079956 +1282.459961 +1270.839966 +1264.030029 +1265.02002 +1254.780029 +1265.650024 +1263.780029 +1266.98999 +1262.859985 +1275.530029 +1280 +1289.380005 +1287.23999 +1283.030029 +1292.670044 +1287.790039 +1289.430054 +1294.119995 +1280.660034 +1291.23999 +1289.140015 +1287.22998 +1278.26001 +1275.880005 +1278.469971 +1272.22998 +1281.420044 +1284.130005 +1297.47998 +1303.02002 +1305.329956 +1307.25 +1305.079956 +1297.22998 +1305.040039 +1301.670044 +1302.949951 +1301.609985 +1293.22998 +1302.890015 +1300.25 +1294.869995 +1297.810059 +1305.930054 +1311.560059 +1309.040039 +1295.5 +1296.619995 +1286.569946 +1288.119995 +1289.119995 +1285.329956 +1307.280029 +1309.930054 +1311.459961 +1311.280029 +1308.109985 +1301.73999 +1305.410034 +1309.719971 +1310.609985 +1305.189941 +1313.209961 +1308.119995 +1312.25 +1325.76001 +1324.660034 +1325.140015 +1322.849976 +1305.920044 +1291.23999 +1294.5 +1292.079956 +1270.319946 +1261.810059 +1267.030029 +1262.069946 +1256.579956 +1258.569946 +1272.880005 +1280.160034 +1259.869995 +1270.089966 +1285.709961 +1288.219971 +1265.290039 +1263.849976 +1256.150024 +1257.930054 +1252.300049 +1237.439941 +1223.689941 +1230.040039 +1256.160034 +1251.540039 +1240.130005 +1240.119995 +1252.199951 +1245.599976 +1244.5 +1250.560059 +1239.199951 +1246 +1272.869995 +1270.199951 +1280.189941 +1270.910034 +1274.079956 +1265.47998 +1267.339966 +1272.430054 +1258.599976 +1242.280029 +1236.199951 +1234.48999 +1236.859985 +1259.810059 +1249.130005 +1240.290039 +1260.910034 +1268.880005 +1268.400024 +1263.199951 +1278.550049 +1276.660034 +1270.920044 +1277.410034 +1280.27002 +1279.359985 +1275.77002 +1271.47998 +1265.949951 +1271.810059 +1266.73999 +1268.209961 +1285.579956 +1295.430054 +1297.47998 +1302.300049 +1297.52002 +1298.819946 +1292.98999 +1296.060059 +1295.089966 +1301.780029 +1304.280029 +1305.369995 +1303.819946 +1311.01001 +1313.25 +1300.26001 +1294.02002 +1298.920044 +1299.540039 +1313 +1318.069946 +1316.280029 +1319.660034 +1321.180054 +1317.640015 +1325.180054 +1318.030029 +1314.780029 +1326.369995 +1336.349976 +1336.589966 +1338.880005 +1335.849976 +1331.319946 +1334.109985 +1350.199951 +1353.219971 +1349.589966 +1350.660034 +1353.420044 +1349.949951 +1362.829956 +1365.619995 +1369.060059 +1364.050049 +1365.800049 +1366.959961 +1368.599976 +1377.02002 +1377.380005 +1382.219971 +1389.079956 +1377.339966 +1377.930054 +1377.939941 +1367.810059 +1367.339966 +1364.300049 +1379.780029 +1382.839966 +1385.719971 +1378.329956 +1380.900024 +1384.420044 +1393.219971 +1396.569946 +1399.76001 +1401.199951 +1400.5 +1402.810059 +1406.089966 +1400.949951 +1381.959961 +1386.719971 +1399.47998 +1400.630005 +1396.709961 +1409.119995 +1414.76001 +1412.900024 +1407.290039 +1409.839966 +1413.040039 +1411.560059 +1413.209961 +1425.48999 +1427.089966 +1422.47998 +1425.550049 +1423.530029 +1418.300049 +1410.76001 +1416.900024 +1426.839966 +1424.72998 +1418.300049 +1416.599976 +1418.339966 +1409.709961 +1412.839966 +1412.109985 +1414.849976 +1423.819946 +1430.72998 +1431.900024 +1430.619995 +1426.369995 +1430.5 +1422.949951 +1427.98999 +1440.130005 +1423.900024 +1422.180054 +1420.619995 +1428.819946 +1438.23999 +1445.939941 +1448.390015 +1446.98999 +1448 +1450.02002 +1448.310059 +1438.060059 +1433.369995 +1444.26001 +1455.300049 +1456.810059 +1455.540039 +1459.680054 +1457.630005 +1456.380005 +1451.189941 +1449.369995 +1399.040039 +1406.819946 +1403.170044 +1387.170044 +1374.119995 +1395.410034 +1391.969971 +1401.890015 +1402.839966 +1406.599976 +1377.949951 +1387.170044 +1392.280029 +1386.949951 +1402.060059 +1410.939941 +1435.040039 +1434.540039 +1436.109985 +1437.5 +1428.609985 +1417.22998 +1422.530029 +1420.859985 +1424.550049 +1437.77002 +1439.369995 +1443.76001 +1444.609985 +1448.390015 +1438.869995 +1447.800049 +1452.849976 +1468.329956 +1471.47998 +1472.5 +1470.72998 +1484.349976 +1480.930054 +1480.410034 +1495.420044 +1494.25 +1494.069946 +1482.369995 +1486.300049 +1495.920044 +1502.390015 +1505.619995 +1509.47998 +1507.719971 +1512.579956 +1491.469971 +1505.849976 +1503.150024 +1501.189941 +1514.140015 +1512.75 +1522.75 +1525.099976 +1524.119995 +1522.280029 +1507.51001 +1515.72998 +1518.109985 +1530.22998 +1530.619995 +1536.339966 +1539.180054 +1530.949951 +1517.380005 +1490.719971 +1507.670044 +1509.119995 +1493 +1515.670044 +1522.969971 +1532.910034 +1531.050049 +1533.699951 +1512.839966 +1522.189941 +1502.560059 +1497.73999 +1492.890015 +1506.339966 +1505.709961 +1503.349976 +1519.430054 +1524.869995 +1525.400024 +1530.439941 +1531.849976 +1510.119995 +1518.76001 +1547.699951 +1552.5 +1549.52002 +1549.369995 +1546.170044 +1553.079956 +1534.099976 +1541.569946 +1511.040039 +1518.089966 +1482.660034 +1458.949951 +1473.910034 +1455.27002 +1465.810059 +1472.199951 +1433.060059 +1467.670044 +1476.709961 +1497.48999 +1453.089966 +1453.640015 +1452.920044 +1426.540039 +1406.699951 +1411.27002 +1445.939941 +1445.550049 +1447.119995 +1464.069946 +1462.5 +1479.369995 +1466.790039 +1432.359985 +1463.76001 +1457.640015 +1473.98999 +1489.420044 +1472.290039 +1478.550049 +1453.550049 +1451.699951 +1471.48999 +1471.560059 +1483.949951 +1484.25 +1476.650024 +1519.780029 +1529.030029 +1518.75 +1525.75 +1517.72998 +1517.209961 +1525.420044 +1531.380005 +1526.75 +1547.040039 +1546.630005 +1539.589966 +1542.839966 +1557.589966 +1552.579956 +1565.150024 +1562.469971 +1554.410034 +1561.800049 +1548.709961 +1538.530029 +1541.23999 +1540.079956 +1500.630005 +1506.329956 +1519.589966 +1515.880005 +1514.400024 +1535.280029 +1540.97998 +1531.02002 +1549.380005 +1508.439941 +1509.650024 +1502.170044 +1520.27002 +1475.619995 +1474.77002 +1453.699951 +1439.180054 +1481.050049 +1470.579956 +1451.150024 +1458.73999 +1433.27002 +1439.699951 +1416.77002 +1440.699951 +1407.219971 +1428.22998 +1469.02002 +1469.719971 +1481.140015 +1472.420044 +1462.790039 +1485.01001 +1507.339966 +1504.660034 +1515.959961 +1477.650024 +1486.589966 +1488.410034 +1467.949951 +1445.900024 +1454.97998 +1453 +1460.119995 +1484.459961 +1496.449951 +1497.660034 +1476.27002 +1478.48999 +1468.359985 +1447.160034 +1447.160034 +1411.630005 +1416.180054 +1390.189941 +1409.130005 +1420.329956 +1401.02002 +1416.25 +1380.949951 +1373.199951 +1333.25 +1325.189941 +1310.5 +1338.599976 +1352.069946 +1330.609985 +1353.959961 +1362.300049 +1355.810059 +1378.550049 +1395.420044 +1380.819946 +1336.640015 +1326.449951 +1336.910034 +1331.290039 +1339.130005 +1348.859985 +1367.209961 +1348.859985 +1349.98999 +1348.780029 +1360.030029 +1342.530029 +1353.109985 +1371.800049 +1381.290039 +1380.02002 +1367.680054 +1330.630005 +1331.339966 +1326.75 +1333.699951 +1304.339966 +1293.369995 +1273.369995 +1320.650024 +1308.77002 +1315.47998 +1288.140015 +1276.599976 +1330.73999 +1298.420044 +1329.51001 +1349.880005 +1352.98999 +1341.130005 +1325.76001 +1315.219971 +1322.699951 +1370.180054 +1367.530029 +1369.310059 +1370.400024 +1372.540039 +1365.540039 +1354.48999 +1360.550049 +1332.829956 +1328.319946 +1334.430054 +1364.709961 +1365.560059 +1390.329956 +1388.170044 +1375.939941 +1379.930054 +1388.819946 +1397.839966 +1396.369995 +1390.939941 +1385.589966 +1409.339966 +1413.900024 +1407.48999 +1418.26001 +1392.569946 +1397.680054 +1388.280029 +1403.579956 +1403.040039 +1408.660034 +1423.569946 +1425.349976 +1426.630005 +1413.400024 +1390.709961 +1394.349976 +1375.930054 +1385.349976 +1390.839966 +1398.26001 +1400.380005 +1385.670044 +1377.650024 +1377.199951 +1404.050049 +1360.680054 +1361.76001 +1358.439941 +1335.48999 +1339.869995 +1360.030029 +1360.140015 +1350.930054 +1337.810059 +1342.829956 +1317.930054 +1318 +1314.290039 +1321.969971 +1283.150024 +1278.380005 +1280 +1284.910034 +1261.52002 +1262.900024 +1252.310059 +1273.699951 +1244.689941 +1253.390015 +1239.48999 +1228.300049 +1214.910034 +1245.359985 +1260.319946 +1260.680054 +1260 +1277 +1282.189941 +1252.540039 +1257.76001 +1234.369995 +1263.199951 +1284.26001 +1267.380005 +1260.310059 +1249.01001 +1284.880005 +1289.189941 +1266.069946 +1296.319946 +1305.319946 +1289.589966 +1285.829956 +1292.930054 +1298.199951 +1278.599976 +1266.689941 +1274.540039 +1277.719971 +1292.199951 +1266.839966 +1271.51001 +1281.660034 +1300.680054 +1282.829956 +1277.579956 +1274.97998 +1236.829956 +1242.310059 +1267.790039 +1224.51001 +1232.040039 +1249.050049 +1251.699951 +1192.699951 +1213.599976 +1156.390015 +1206.51001 +1255.079956 +1207.089966 +1188.219971 +1185.869995 +1209.180054 +1213.27002 +1106.420044 +1166.359985 +1161.060059 +1114.280029 +1099.22998 +1056.890015 +996.22998 +984.940002 +909.919983 +899.219971 +1003.349976 +998.01001 +907.840027 +946.429993 +940.549988 +985.400024 +955.049988 +896.780029 +908.109985 +876.77002 +848.919983 +940.51001 +930.090027 +954.090027 +968.75 +966.299988 +1005.75 +952.77002 +904.880005 +930.98999 +919.210022 +898.950012 +852.299988 +911.289978 +873.289978 +850.75 +859.119995 +806.580017 +752.440002 +800.030029 +851.809998 +857.390015 +887.679993 +896.23999 +816.210022 +848.809998 +870.73999 +845.219971 +876.070007 +909.700012 +888.669983 +899.23999 +873.590027 +879.72998 +868.570007 +913.179993 +904.419983 +885.280029 +887.880005 +871.630005 +863.159973 +868.150024 +872.799988 +869.419983 +890.640015 +903.25 +931.799988 +927.450012 +934.700012 +906.650024 +909.72998 +890.349976 +870.26001 +871.789978 +842.619995 +843.73999 +850.119995 +805.219971 +840.23999 +827.5 +831.950012 +836.570007 +845.710022 +874.090027 +845.140015 +825.880005 +825.440002 +838.51001 +832.22998 +845.849976 +868.599976 +869.890015 +827.159973 +833.73999 +835.190002 +826.840027 +789.169983 +788.419983 +778.940002 +770.049988 +743.330017 +773.140015 +764.900024 +752.830017 +735.090027 +700.820007 +696.330017 +712.869995 +682.549988 +683.380005 +676.530029 +719.599976 +721.359985 +750.73999 +756.549988 +753.890015 +778.119995 +794.349976 +784.039978 +768.539978 +822.919983 +806.119995 +813.880005 +832.859985 +815.940002 +787.530029 +797.869995 +811.080017 +834.380005 +842.5 +835.47998 +815.549988 +825.159973 +856.559998 +858.72998 +841.5 +852.059998 +865.299988 +869.599976 +832.390015 +850.080017 +843.549988 +851.919983 +866.22998 +857.51001 +855.159973 +873.640015 +872.809998 +877.52002 +907.23999 +903.799988 +919.530029 +907.390015 +929.22998 +909.23999 +908.349976 +883.919983 +893.070007 +882.880005 +909.710022 +908.130005 +903.469971 +888.330017 +887 +910.330017 +893.059998 +906.830017 +919.140015 +942.869995 +944.73999 +931.76001 +942.460022 +940.090027 +939.140015 +942.429993 +939.150024 +944.890015 +946.210022 +923.719971 +911.969971 +910.710022 +918.369995 +921.22998 +893.039978 +895.099976 +900.940002 +920.26001 +918.900024 +927.22998 +919.320007 +923.330017 +896.419983 +898.719971 +881.030029 +879.559998 +882.679993 +879.130005 +901.049988 +905.840027 +932.679993 +940.73999 +940.380005 +951.130005 +954.580017 +954.070007 +976.289978 +979.26001 +982.179993 +979.619995 +975.150024 +986.75 +987.47998 +1002.630005 +1005.650024 +1002.719971 +997.080017 +1010.47998 +1007.099976 +994.349976 +1005.809998 +1012.72998 +1004.090027 +979.72998 +989.669983 +996.460022 +1007.369995 +1026.130005 +1025.569946 +1028 +1028.119995 +1030.97998 +1028.930054 +1020.619995 +998.039978 +994.75 +1003.23999 +1016.400024 +1025.390015 +1033.369995 +1044.140015 +1042.72998 +1049.339966 +1052.630005 +1068.76001 +1065.48999 +1068.300049 +1064.660034 +1071.660034 +1060.869995 +1050.780029 +1044.380005 +1062.97998 +1060.609985 +1057.079956 +1029.849976 +1025.209961 +1040.459961 +1054.719971 +1057.579956 +1065.47998 +1071.48999 +1076.189941 +1073.189941 +1092.02002 +1096.560059 +1087.680054 +1097.910034 +1091.060059 +1081.400024 +1092.910034 +1079.599976 +1066.949951 +1063.410034 +1042.630005 +1066.109985 +1036.189941 +1042.880005 +1045.410034 +1046.5 +1066.630005 +1069.300049 +1093.079956 +1093.01001 +1098.51001 +1087.23999 +1093.47998 +1109.300049 +1110.319946 +1109.800049 +1094.900024 +1091.380005 +1106.23999 +1105.650024 +1110.630005 +1091.48999 +1095.630005 +1108.859985 +1109.23999 +1099.920044 +1105.97998 +1103.25 +1091.939941 +1095.949951 +1102.349976 +1106.410034 +1114.109985 +1107.930054 +1109.180054 +1096.079956 +1102.469971 +1114.050049 +1118.02002 +1120.589966 +1126.47998 +1127.780029 +1126.199951 +1126.420044 +1115.099976 +1132.98999 +1136.52002 +1137.140015 +1141.689941 +1144.97998 +1146.97998 +1136.219971 +1145.680054 +1148.459961 +1136.030029 +1150.22998 +1138.040039 +1116.47998 +1091.76001 +1096.780029 +1092.170044 +1097.5 +1084.530029 +1073.869995 +1089.189941 +1103.319946 +1097.280029 +1063.109985 +1066.189941 +1056.73999 +1070.52002 +1068.130005 +1078.469971 +1075.51001 +1094.869995 +1099.51001 +1106.75 +1109.170044 +1108.01001 +1094.599976 +1105.23999 +1102.939941 +1104.48999 +1115.709961 +1118.310059 +1118.790039 +1122.969971 +1138.699951 +1138.5 +1140.449951 +1145.609985 +1150.23999 +1149.98999 +1150.51001 +1159.459961 +1166.209961 +1165.829956 +1159.900024 +1165.810059 +1174.170044 +1167.719971 +1165.72998 +1166.589966 +1173.219971 +1173.27002 +1169.430054 +1178.099976 +1187.439941 +1189.439941 +1182.449951 +1186.439941 +1194.369995 +1196.47998 +1197.300049 +1210.650024 +1211.670044 +1192.130005 +1197.52002 +1207.170044 +1205.939941 +1208.670044 +1217.280029 +1212.050049 +1183.709961 +1191.359985 +1206.780029 +1186.689941 +1202.26001 +1173.599976 +1165.869995 +1128.150024 +1110.880005 +1159.72998 +1155.790039 +1171.670044 +1157.439941 +1135.680054 +1136.939941 +1120.800049 +1115.050049 +1071.589966 +1087.689941 +1073.650024 +1074.030029 +1067.949951 +1103.060059 +1089.410034 +1070.709961 +1098.380005 +1102.829956 +1064.880005 +1050.469971 +1062 +1055.689941 +1086.839966 +1091.599976 +1089.630005 +1115.22998 +1114.609985 +1116.040039 +1117.51001 +1113.199951 +1095.310059 +1092.040039 +1073.689941 +1076.76001 +1074.569946 +1041.23999 +1030.709961 +1027.369995 +1022.580017 +1028.060059 +1060.27002 +1070.25 +1077.959961 +1078.75 +1095.339966 +1095.170044 +1096.47998 +1064.880005 +1071.25 +1083.47998 +1069.589966 +1093.670044 +1102.660034 +1115.01001 +1113.839966 +1106.130005 +1101.530029 +1101.599976 +1125.859985 +1120.459961 +1127.23999 +1125.810059 +1121.640015 +1127.790039 +1121.060059 +1089.469971 +1083.609985 +1079.25 +1079.380005 +1092.540039 +1094.160034 +1075.630005 +1071.689941 +1067.359985 +1051.869995 +1055.329956 +1047.219971 +1064.589966 +1048.920044 +1049.329956 +1080.290039 +1090.099976 +1104.51001 +1091.839966 +1098.869995 +1104.180054 +1109.550049 +1121.900024 +1121.099976 +1125.069946 +1124.660034 +1125.589966 +1142.709961 +1139.780029 +1134.280029 +1124.829956 +1148.670044 +1142.160034 +1147.699951 +1144.72998 +1141.199951 +1146.23999 +1137.030029 +1160.75 +1159.969971 +1158.060059 +1165.150024 +1165.319946 +1169.77002 +1178.099976 +1173.810059 +1176.189941 +1184.709961 +1165.900024 +1178.170044 +1180.26001 +1183.079956 +1185.619995 +1185.640015 +1182.449951 +1183.780029 +1183.26001 +1184.380005 +1193.569946 +1197.959961 +1221.060059 +1225.849976 +1223.25 +1213.400024 +1218.709961 +1213.540039 +1199.209961 +1197.75 +1178.339966 +1178.589966 +1196.689941 +1199.72998 +1197.839966 +1180.72998 +1198.349976 +1189.400024 +1187.76001 +1180.550049 +1206.069946 +1221.530029 +1224.709961 +1223.119995 +1223.75 +1228.280029 +1233 +1240.400024 +1240.459961 +1241.589966 +1235.22998 +1242.869995 +1243.910034 +1247.079956 +1254.599976 +1258.839966 +1256.77002 +1257.540039 +1258.51001 +1259.780029 +1257.880005 +1257.640015 +1271.869995 +1270.199951 +1276.560059 +1273.849976 +1271.5 +1269.75 +1274.47998 +1285.959961 +1283.76001 +1293.23999 +1295.02002 +1281.920044 +1280.26001 +1283.349976 +1290.839966 +1291.180054 +1296.630005 +1299.540039 +1276.339966 +1286.119995 +1307.589966 +1304.030029 +1307.099976 +1310.869995 +1319.050049 +1324.569946 +1320.880005 +1321.869995 +1329.150024 +1332.319946 +1328.01001 +1336.319946 +1340.430054 +1343.01001 +1315.439941 +1307.400024 +1306.099976 +1319.880005 +1327.219971 +1306.329956 +1308.439941 +1330.969971 +1321.150024 +1310.130005 +1321.819946 +1320.02002 +1295.109985 +1304.280029 +1296.390015 +1281.869995 +1256.880005 +1273.719971 +1279.209961 +1298.380005 +1293.77002 +1297.540039 +1309.660034 +1313.800049 +1310.189941 +1319.439941 +1328.26001 +1325.829956 +1332.410034 +1332.869995 +1332.630005 +1335.540039 +1333.51001 +1328.170044 +1324.459961 +1314.160034 +1314.410034 +1314.52002 +1319.680054 +1305.140015 +1312.619995 +1330.359985 +1337.380005 +1335.25 +1347.23999 +1355.660034 +1360.47998 +1363.609985 +1361.219971 +1356.619995 +1347.319946 +1335.099976 +1340.199951 +1346.290039 +1357.160034 +1342.079956 +1348.650024 +1337.77002 +1329.469971 +1328.97998 +1340.680054 +1343.599976 +1333.27002 +1317.369995 +1316.280029 +1320.469971 +1325.689941 +1331.099976 +1345.199951 +1314.550049 +1312.939941 +1300.160034 +1286.170044 +1284.939941 +1279.560059 +1289 +1270.97998 +1271.829956 +1287.869995 +1265.420044 +1267.640015 +1271.5 +1278.359985 +1295.52002 +1287.140015 +1283.5 +1268.449951 +1280.099976 +1296.670044 +1307.410034 +1320.640015 +1339.670044 +1337.880005 +1339.219971 +1353.219971 +1343.800049 +1319.48999 +1313.640015 +1317.719971 +1308.869995 +1316.140015 +1305.439941 +1326.72998 +1325.839966 +1343.800049 +1345.02002 +1337.430054 +1331.939941 +1304.890015 +1300.670044 +1292.280029 +1286.939941 +1254.050049 +1260.339966 +1200.069946 +1199.380005 +1119.459961 +1172.530029 +1120.76001 +1172.640015 +1178.810059 +1204.48999 +1192.76001 +1193.890015 +1140.650024 +1123.530029 +1123.819946 +1162.349976 +1177.599976 +1159.27002 +1176.800049 +1210.079956 +1212.920044 +1218.890015 +1204.420044 +1173.969971 +1165.23999 +1198.619995 +1185.900024 +1154.22998 +1162.27002 +1172.869995 +1188.680054 +1209.109985 +1216.01001 +1204.089966 +1202.089966 +1166.76001 +1129.560059 +1136.430054 +1162.949951 +1175.380005 +1151.060059 +1160.400024 +1131.420044 +1099.22998 +1123.949951 +1144.030029 +1164.969971 +1155.459961 +1194.890015 +1195.540039 +1207.25 +1203.660034 +1224.579956 +1200.859985 +1225.380005 +1209.880005 +1215.390015 +1238.25 +1254.189941 +1229.050049 +1242 +1284.589966 +1285.089966 +1253.300049 +1218.280029 +1237.900024 +1261.150024 +1253.22998 +1261.119995 +1275.920044 +1229.099976 +1239.699951 +1263.849976 +1251.780029 +1257.810059 +1236.910034 +1216.130005 +1215.650024 +1192.97998 +1188.040039 +1161.790039 +1158.670044 +1192.550049 +1195.189941 +1246.959961 +1244.579956 +1244.280029 +1257.079956 +1258.469971 +1261.01001 +1234.349976 +1255.189941 +1236.469971 +1225.72998 +1211.819946 +1215.75 +1219.660034 +1205.349976 +1241.300049 +1243.719971 +1254 +1265.329956 +1265.430054 +1249.640015 +1263.02002 +1257.599976 +1277.060059 +1277.300049 +1281.060059 +1277.810059 +1280.699951 +1292.079956 +1292.47998 +1295.5 +1289.089966 +1293.670044 +1308.040039 +1314.5 +1315.380005 +1316 +1314.650024 +1326.060059 +1318.430054 +1316.329956 +1313.01001 +1312.410034 +1324.089966 +1325.540039 +1344.900024 +1344.329956 +1347.050049 +1349.959961 +1351.949951 +1342.640015 +1351.77002 +1350.5 +1343.22998 +1358.040039 +1361.22998 +1362.209961 +1357.660034 +1363.459961 +1365.73999 +1367.589966 +1372.180054 +1365.680054 +1374.089966 +1369.630005 +1364.329956 +1343.359985 +1352.630005 +1365.910034 +1370.869995 +1371.089966 +1395.949951 +1394.280029 +1402.599976 +1404.170044 +1409.75 +1405.52002 +1402.890015 +1392.780029 +1397.109985 +1416.51001 +1412.52002 +1405.540039 +1403.280029 +1408.469971 +1419.040039 +1413.380005 +1398.959961 +1398.079956 +1382.199951 +1358.589966 +1368.709961 +1387.569946 +1370.26001 +1369.569946 +1390.780029 +1385.140015 +1376.920044 +1378.530029 +1366.939941 +1371.969971 +1390.689941 +1399.97998 +1403.359985 +1397.910034 +1405.819946 +1402.310059 +1391.569946 +1369.099976 +1369.579956 +1363.719971 +1354.579956 +1357.98999 +1353.390015 +1338.349976 +1330.660034 +1324.800049 +1304.859985 +1295.219971 +1315.98999 +1316.630005 +1318.859985 +1320.680054 +1317.819946 +1332.420044 +1313.319946 +1310.329956 +1278.040039 +1278.180054 +1285.5 +1315.130005 +1314.98999 +1325.660034 +1308.930054 +1324.180054 +1314.880005 +1329.099976 +1342.839966 +1344.780029 +1357.97998 +1355.689941 +1325.51001 +1335.02002 +1313.719971 +1319.98999 +1331.849976 +1329.040039 +1362.160034 +1365.51001 +1374.02002 +1367.579956 +1354.680054 +1352.459961 +1341.469971 +1341.449951 +1334.76001 +1356.780029 +1353.640015 +1363.670044 +1372.780029 +1376.51001 +1362.660034 +1350.52002 +1338.310059 +1337.890015 +1360.02002 +1385.969971 +1385.300049 +1379.319946 +1375.319946 +1365 +1390.98999 +1394.22998 +1401.349976 +1402.219971 +1402.800049 +1405.869995 +1404.109985 +1403.930054 +1405.530029 +1415.51001 +1418.160034 +1418.130005 +1413.170044 +1413.48999 +1402.079956 +1411.130005 +1410.439941 +1409.300049 +1410.48999 +1399.47998 +1406.579956 +1404.939941 +1403.439941 +1432.119995 +1437.920044 +1429.079956 +1433.560059 +1436.560059 +1459.98999 +1465.77002 +1461.189941 +1459.319946 +1461.050049 +1460.26001 +1460.150024 +1456.890015 +1441.589966 +1433.319946 +1447.150024 +1440.670044 +1444.48999 +1445.75 +1450.98999 +1461.400024 +1460.930054 +1455.880005 +1441.47998 +1432.560059 +1432.839966 +1428.589966 +1440.130005 +1454.920044 +1460.910034 +1457.339966 +1433.189941 +1433.819946 +1413.109985 +1408.75 +1412.969971 +1411.939941 +1412.160034 +1427.589966 +1414.199951 +1417.26001 +1428.390015 +1394.530029 +1377.51001 +1379.849976 +1380.030029 +1374.530029 +1355.48999 +1353.329956 +1359.880005 +1386.890015 +1387.810059 +1391.030029 +1409.150024 +1406.290039 +1398.939941 +1409.930054 +1415.949951 +1416.180054 +1409.459961 +1407.050049 +1409.280029 +1413.939941 +1418.069946 +1418.550049 +1427.839966 +1428.47998 +1419.449951 +1413.579956 +1430.359985 +1446.790039 +1435.810059 +1443.689941 +1430.150024 +1426.660034 +1419.829956 +1418.099976 +1402.430054 +1426.189941 +1462.420044 +1459.369995 +1466.469971 +1461.890015 +1457.150024 +1461.02002 +1472.119995 +1472.050049 +1470.680054 +1472.339966 +1472.630005 +1480.939941 +1485.97998 +1492.560059 +1494.810059 +1494.819946 +1502.959961 +1500.180054 +1507.839966 +1501.959961 +1498.109985 +1513.170044 +1495.709961 +1511.290039 +1512.119995 +1509.390015 +1517.930054 +1517.01001 +1519.430054 +1520.329956 +1521.380005 +1519.790039 +1530.939941 +1511.949951 +1502.420044 +1515.599976 +1487.849976 +1496.939941 +1515.98999 +1514.680054 +1518.199951 +1525.199951 +1539.790039 +1541.459961 +1544.26001 +1551.180054 +1556.219971 +1552.47998 +1554.52002 +1563.22998 +1560.699951 +1552.099976 +1548.339966 +1558.709961 +1545.800049 +1556.890015 +1551.689941 +1563.77002 +1562.849976 +1569.189941 +1562.170044 +1570.25 +1553.689941 +1559.97998 +1553.280029 +1563.069946 +1568.609985 +1587.72998 +1593.369995 +1588.849976 +1552.359985 +1574.569946 +1552.01001 +1541.609985 +1555.25 +1562.5 +1578.780029 +1578.790039 +1585.160034 +1582.23999 +1593.609985 +1597.569946 +1582.699951 +1597.589966 +1614.420044 +1617.5 +1625.959961 +1632.689941 +1626.670044 +1633.699951 +1633.77002 +1650.339966 +1658.780029 +1650.469971 +1667.469971 +1666.290039 +1669.160034 +1655.349976 +1650.51001 +1649.599976 +1660.060059 +1648.359985 +1654.410034 +1630.73999 +1640.420044 +1631.380005 +1608.900024 +1622.560059 +1643.380005 +1642.810059 +1626.130005 +1612.52002 +1636.359985 +1626.72998 +1639.040039 +1651.810059 +1628.930054 +1588.189941 +1592.430054 +1573.089966 +1588.030029 +1603.26001 +1613.199951 +1606.280029 +1614.959961 +1614.079956 +1615.410034 +1631.890015 +1640.459961 +1652.319946 +1652.619995 +1675.02002 +1680.189941 +1682.5 +1676.26001 +1680.910034 +1689.369995 +1692.089966 +1695.530029 +1692.390015 +1685.939941 +1690.25 +1691.650024 +1685.329956 +1685.959961 +1685.72998 +1706.869995 +1709.670044 +1707.140015 +1697.369995 +1690.910034 +1697.47998 +1691.420044 +1689.469971 +1694.160034 +1685.390015 +1661.319946 +1655.829956 +1646.060059 +1652.349976 +1642.800049 +1656.959961 +1663.5 +1656.780029 +1630.47998 +1634.959961 +1638.170044 +1632.969971 +1639.77002 +1653.079956 +1655.079956 +1655.170044 +1671.709961 +1683.98999 +1689.130005 +1683.420044 +1687.98999 +1697.599976 +1704.76001 +1725.52002 +1722.339966 +1709.910034 +1701.839966 +1697.420044 +1692.77002 +1698.670044 +1691.75 +1681.550049 +1695 +1693.869995 +1678.660034 +1690.5 +1676.119995 +1655.449951 +1656.400024 +1692.560059 +1703.199951 +1710.140015 +1698.060059 +1721.540039 +1733.150024 +1744.5 +1744.660034 +1754.670044 +1746.380005 +1752.069946 +1759.77002 +1762.109985 +1771.949951 +1763.310059 +1756.540039 +1761.640015 +1767.930054 +1762.969971 +1770.48999 +1747.150024 +1770.609985 +1771.890015 +1767.689941 +1782 +1790.619995 +1798.180054 +1791.530029 +1787.869995 +1781.369995 +1795.849976 +1804.76001 +1802.47998 +1802.75 +1807.22998 +1805.810059 +1800.900024 +1795.150024 +1792.810059 +1785.030029 +1805.089966 +1808.369995 +1802.619995 +1782.219971 +1775.5 +1775.319946 +1786.540039 +1781 +1810.650024 +1809.599976 +1818.319946 +1827.98999 +1833.319946 +1842.02002 +1841.400024 +1841.069946 +1848.359985 +1831.97998 +1831.369995 +1826.77002 +1837.880005 +1837.48999 +1838.130005 +1842.369995 +1819.199951 +1838.880005 +1848.380005 +1845.890015 +1838.699951 +1843.800049 +1844.859985 +1828.459961 +1790.290039 +1781.560059 +1792.5 +1774.199951 +1794.189941 +1782.589966 +1741.890015 +1755.199951 +1751.640015 +1773.430054 +1797.02002 +1799.839966 +1819.75 +1819.26001 +1829.829956 +1838.630005 +1840.76001 +1828.75 +1839.780029 +1836.25 +1847.609985 +1845.119995 +1845.160034 +1854.290039 +1859.449951 +1845.72998 +1873.910034 +1873.810059 +1877.030029 +1878.040039 +1877.170044 +1867.630005 +1868.199951 +1846.339966 +1841.130005 +1858.829956 +1872.25 +1860.77002 +1872.01001 +1866.52002 +1857.439941 +1865.619995 +1852.560059 +1849.040039 +1857.619995 +1872.339966 +1885.52002 +1890.900024 +1888.77002 +1865.089966 +1845.040039 +1851.959961 +1872.180054 +1833.079956 +1815.689941 +1830.609985 +1842.97998 +1862.310059 +1864.849976 +1871.890015 +1879.550049 +1875.390015 +1878.609985 +1863.400024 +1869.430054 +1878.329956 +1883.949951 +1883.680054 +1881.140015 +1884.660034 +1867.719971 +1878.209961 +1875.630005 +1878.47998 +1896.650024 +1897.449951 +1888.530029 +1870.849976 +1877.859985 +1885.079956 +1872.829956 +1888.030029 +1892.48999 +1900.530029 +1911.910034 +1909.780029 +1920.030029 +1923.569946 +1924.969971 +1924.23999 +1927.880005 +1940.459961 +1949.439941 +1951.27002 +1950.790039 +1943.890015 +1930.109985 +1936.160034 +1937.780029 +1941.98999 +1956.97998 +1959.47998 +1962.869995 +1962.609985 +1949.97998 +1959.530029 +1957.219971 +1960.959961 +1960.22998 +1973.319946 +1974.619995 +1985.439941 +1977.650024 +1963.709961 +1972.829956 +1964.680054 +1967.569946 +1977.099976 +1973.280029 +1981.569946 +1958.119995 +1978.219971 +1973.630005 +1983.530029 +1987.01001 +1987.97998 +1978.339966 +1978.910034 +1969.949951 +1970.069946 +1930.670044 +1925.150024 +1938.98999 +1920.209961 +1920.23999 +1909.569946 +1931.589966 +1936.920044 +1933.75 +1946.719971 +1955.180054 +1955.060059 +1971.73999 +1981.599976 +1986.51001 +1992.369995 +1988.400024 +1997.920044 +2000.02002 +2000.119995 +1996.73999 +2003.369995 +2002.280029 +2000.719971 +1997.650024 +2007.709961 +2001.540039 +1988.439941 +1995.689941 +1997.449951 +1985.540039 +1984.130005 +1998.97998 +2001.569946 +2011.359985 +2010.400024 +1994.290039 +1982.77002 +1998.300049 +1965.98999 +1982.849976 +1977.800049 +1972.290039 +1946.160034 +1946.170044 +1967.900024 +1964.819946 +1935.099976 +1968.890015 +1928.209961 +1906.130005 +1874.73999 +1877.699951 +1862.48999 +1862.76001 +1886.76001 +1904.01001 +1941.280029 +1927.109985 +1950.819946 +1964.579956 +1961.630005 +1985.050049 +1982.300049 +1994.650024 +2018.050049 +2017.810059 +2012.099976 +2023.569946 +2031.209961 +2031.920044 +2038.26001 +2039.680054 +2038.25 +2039.329956 +2039.819946 +2041.319946 +2051.800049 +2048.719971 +2052.75 +2063.5 +2069.409912 +2067.030029 +2072.830078 +2067.560059 +2053.439941 +2066.550049 +2074.330078 +2071.919922 +2075.370117 +2060.310059 +2059.820068 +2026.140015 +2035.329956 +2002.329956 +1989.630005 +1972.73999 +2012.890015 +2061.22998 +2070.649902 +2078.540039 +2082.169922 +2081.879883 +2088.77002 +2090.570068 +2080.350098 +2058.899902 +2058.199951 +2020.579956 +2002.609985 +2025.900024 +2062.139893 +2044.810059 +2028.26001 +2023.030029 +2011.27002 +1992.670044 +2019.420044 +2022.550049 +2032.119995 +2063.149902 +2051.820068 +2057.090088 +2029.550049 +2002.160034 +2021.25 +1994.98999 +2020.849976 +2050.030029 +2041.51001 +2062.52002 +2055.469971 +2046.73999 +2068.590088 +2068.530029 +2088.47998 +2096.98999 +2100.340088 +2099.679932 +2097.449951 +2110.300049 +2109.659912 +2115.47998 +2113.860107 +2110.73999 +2104.5 +2117.389893 +2107.780029 +2098.530029 +2101.040039 +2071.26001 +2079.429932 +2044.160034 +2040.23999 +2065.949951 +2053.399902 +2081.189941 +2074.280029 +2099.5 +2089.27002 +2108.100098 +2104.419922 +2091.5 +2061.050049 +2056.149902 +2061.02002 +2086.23999 +2067.889893 +2059.689941 +2066.959961 +2080.620117 +2076.330078 +2081.899902 +2091.179932 +2102.060059 +2092.429932 +2095.840088 +2106.629883 +2104.98999 +2081.179932 +2100.399902 +2097.290039 +2107.959961 +2112.929932 +2117.689941 +2108.919922 +2114.76001 +2106.850098 +2085.51001 +2108.290039 +2114.48999 +2089.459961 +2080.149902 +2088 +2116.100098 +2105.330078 +2099.120117 +2098.47998 +2121.100098 +2122.72998 +2129.199951 +2127.830078 +2125.850098 +2130.820068 +2126.060059 +2104.199951 +2123.47998 +2120.790039 +2107.389893 +2111.72998 +2109.600098 +2114.070068 +2095.840088 +2092.830078 +2079.280029 +2080.149902 +2105.199951 +2108.860107 +2094.110107 +2084.429932 +2096.290039 +2100.439941 +2121.23999 +2109.98999 +2122.850098 +2124.199951 +2108.580078 +2102.310059 +2101.48999 +2057.639893 +2063.110107 +2077.419922 +2076.780029 +2068.76001 +2081.340088 +2046.680054 +2051.310059 +2076.620117 +2099.600098 +2108.949951 +2107.399902 +2124.290039 +2126.639893 +2128.280029 +2119.209961 +2114.149902 +2102.149902 +2079.649902 +2067.639893 +2093.25 +2108.570068 +2108.629883 +2103.840088 +2098.040039 +2093.320068 +2099.840088 +2083.560059 +2077.570068 +2104.179932 +2084.070068 +2086.050049 +2083.389893 +2091.540039 +2102.439941 +2096.919922 +2079.610107 +2035.72998 +1970.890015 +1893.209961 +1867.609985 +1940.51001 +1987.660034 +1988.869995 +1972.180054 +1913.849976 +1948.859985 +1951.130005 +1921.219971 +1969.410034 +1942.040039 +1952.290039 +1961.050049 +1953.030029 +1978.089966 +1995.310059 +1990.199951 +1958.030029 +1966.969971 +1942.73999 +1938.76001 +1932.23999 +1931.339966 +1881.77002 +1884.089966 +1920.030029 +1923.819946 +1951.359985 +1987.050049 +1979.920044 +1995.829956 +2013.430054 +2014.890015 +2017.459961 +2003.689941 +1994.23999 +2023.859985 +2033.109985 +2033.660034 +2030.77002 +2018.939941 +2052.51001 +2075.149902 +2071.179932 +2065.889893 +2090.350098 +2089.409912 +2079.360107 +2104.050049 +2109.790039 +2102.310059 +2099.929932 +2099.199951 +2078.580078 +2081.719971 +2075 +2045.969971 +2023.040039 +2053.189941 +2050.439941 +2083.580078 +2081.23999 +2089.169922 +2086.590088 +2089.139893 +2088.870117 +2090.110107 +2080.409912 +2102.629883 +2079.51001 +2049.620117 +2091.689941 +2077.070068 +2063.590088 +2047.619995 +2052.22998 +2012.369995 +2021.939941 +2043.410034 +2073.070068 +2041.890015 +2005.550049 +2021.150024 +2038.969971 +2064.290039 +2060.98999 +2056.5 +2078.360107 +2063.360107 +2043.939941 +2012.660034 +2016.709961 +1990.26001 +1943.089966 +1922.030029 +1923.670044 +1938.680054 +1890.280029 +1921.839966 +1880.329956 +1881.329956 +1859.329956 +1868.98999 +1906.900024 +1877.079956 +1903.630005 +1882.949951 +1893.359985 +1940.23999 +1939.380005 +1903.030029 +1912.530029 +1915.449951 +1880.050049 +1853.439941 +1852.209961 +1851.859985 +1829.079956 +1864.780029 +1895.579956 +1926.819946 +1917.829956 +1917.780029 +1945.5 +1921.27002 +1929.800049 +1951.699951 +1948.050049 +1932.22998 +1978.349976 +1986.449951 +1993.400024 +1999.98999 +2001.76001 +1979.26001 +1989.26001 +1989.569946 +2022.189941 +2019.640015 +2015.930054 +2027.219971 +2040.589966 +2049.580078 +2051.600098 +2049.800049 +2036.709961 +2035.939941 +2037.050049 +2055.01001 +2063.949951 +2059.73999 +2072.780029 +2066.129883 +2045.170044 +2066.659912 +2041.910034 +2047.599976 +2041.98999 +2061.719971 +2082.419922 +2082.780029 +2080.72998 +2094.340088 +2100.800049 +2102.399902 +2091.47998 +2091.580078 +2087.790039 +2091.699951 +2095.149902 +2075.810059 +2065.300049 +2081.429932 +2063.370117 +2051.120117 +2050.629883 +2057.139893 +2058.689941 +2084.389893 +2064.459961 +2064.110107 +2046.609985 +2066.659912 +2047.209961 +2047.630005 +2040.040039 +2052.320068 +2048.040039 +2076.060059 +2090.540039 +2090.100098 +2099.060059 +2096.949951 +2099.330078 +2105.26001 +2099.129883 +2109.409912 +2112.129883 +2119.120117 +2115.47998 +2096.070068 +2079.060059 +2075.320068 +2071.5 +2077.98999 +2071.219971 +2083.25 +2088.899902 +2085.449951 +2113.320068 +2037.410034 +2000.540039 +2036.089966 +2070.77002 +2098.860107 +2102.949951 +2088.550049 +2099.72998 +2097.899902 +2129.899902 +2137.159912 +2152.139893 +2152.429932 +2163.75 +2161.73999 +2166.889893 +2163.780029 +2173.02002 +2165.169922 +2175.030029 +2168.47998 +2169.179932 +2166.580078 +2170.060059 +2173.600098 +2170.840088 diff --git a/elephantscale/scripts/README.md b/elephantscale/scripts/README.md new file mode 100644 index 0000000..4af92eb --- /dev/null +++ b/elephantscale/scripts/README.md @@ -0,0 +1,4 @@ +# Scripts Directory + +This contains scripts. For video 3, run the run-transfer.sh script. You will need to start with plenty +of memory (16gb minimum). diff --git a/run-bigdl-docker.sh b/run-bigdl-docker.sh new file mode 100755 index 0000000..6d01f68 --- /dev/null +++ b/run-bigdl-docker.sh @@ -0,0 +1,36 @@ +#!/bin/bash + +## Usage +# ./run-bigdl-docker.sh [optional command] +# +# ./run-bigdl-docker.sh xxx/yyy +# +# or during developing, give a local docker image id +# ./run-bigdl-docker.sh abcd1234 +# +# provide an optional command, here is an example of simple bash +# ./run-bigdl-docker.sh abcd1234 bash + +if [ -z "$1" ] ; then + echo "Usage: $0 [optional command]" + echo "Missing Docker image id. exiting" + exit -1 +fi + +image_id="$1" +cmd="$2" +name="bigdl" + +## remove any previously running containers +docker rm -f "$name" + +# mount the current directory at /work +this="${BASH_SOURCE-$0}" +mydir=$(cd -P -- "$(dirname -- "$this")" && pwd -P) + +docker run -it --name "$name" \ + -p 8888:8888 \ + -p 6006:6006 \ + -v"$mydir:/work" \ + "$image_id" \ + ${cmd} diff --git a/run-bigdl-native.sh b/run-bigdl-native.sh new file mode 100755 index 0000000..11abd84 --- /dev/null +++ b/run-bigdl-native.sh @@ -0,0 +1,66 @@ +#!/bin/bash + +## Runs BigDL natively + +## Usage +# BIGDL_HOME=~/apps/BigDL SPARK_HOME=~/apps/spark ./run-bigdl.sh + + +# Check environment variables +if [ -z "${BIGDL_HOME}" ]; then + echo "Please set BIGDL_HOME environment variable" + exit 1 +fi + +if [ -z "${SPARK_HOME}" ]; then + echo "Please set SPARK_HOME environment variable" + exit 1 +fi + +# activate py35 environment +#source activate py35 +# activate py27 environment +source activate py27 +conda info -e + + +#setup paths +export PYSPARK_PYTHON=$(which python) +export PYSPARK_DRIVER_PYTHON=$(which jupyter) +echo "### PYSPARK_PYTHON=$PYSPARK_PYTHON" +echo "### PYSPARK_DRIVER_PYTHON=$PYSPARK_DRIVER_PYTHON" +# this starts the notebook without a security token +#export PYSPARK_DRIVER_PYTHON_OPTS="notebook --notebook-dir=./ --ip=* --no-browser --NotebookApp.token=''" +export PYSPARK_DRIVER_PYTHON_OPTS="notebook --notebook-dir=./ --ip=* --no-browser" +export BIGDL_JAR_NAME=`ls ${BIGDL_HOME}/lib/ | grep jar-with-dependencies.jar` +export BIGDL_JAR="${BIGDL_HOME}/lib/$BIGDL_JAR_NAME" +export BIGDL_PY_ZIP_NAME=`ls ${BIGDL_HOME}/lib/ | grep python-api.zip` +export BIGDL_PY_ZIP="${BIGDL_HOME}/lib/$BIGDL_PY_ZIP_NAME" +export BIGDL_CONF=${BIGDL_HOME}/conf/spark-bigdl.conf + +# Check files +if [ ! -f ${BIGDL_CONF} ]; then + echo "Cannot find ${BIGDL_CONF}" + exit 1 +fi + +if [ ! -f ${BIGDL_PY_ZIP} ]; then + echo ${BIGDL_PY_ZIP} + echo "Cannot find ${BIGDL_PY_ZIP}" + exit 1 +fi + +if [ ! -f $BIGDL_JAR ]; then + echo "Cannot find $BIGDL_JAR" + exit 1 +fi + +${SPARK_HOME}/bin/pyspark \ + --master local[4] \ + --driver-memory 4g \ + --properties-file ${BIGDL_CONF} \ + --py-files ${BIGDL_PY_ZIP} \ + --jars ${BIGDL_JAR} \ + --conf spark.driver.extraClassPath=${BIGDL_JAR} \ + --conf spark.executor.extraClassPath=${BIGDL_JAR} \ + --conf spark.sql.catalogImplementation='in-memory'