global_mode.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "brutal-universal",
   "metadata": {},
   "source": [
    "# Bitcoin Price Prediction - Global Mode"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "precious-rogers",
   "metadata": {},
   "source": [
    "* **Description**: COMP4103(Big Data)--Group Project\n",
    "* **Author**: Aaron\n",
    "* **Version**: 1.2 (The whole workflow with results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "reflected-coral",
   "metadata": {},
   "source": [
    "**Updates:**\n",
    "1. Add the final results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dramatic-ontario",
   "metadata": {},
   "source": [
    "**Issues:**\n",
    "N/A"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "valid-observer",
   "metadata": {},
   "source": [
    "**To be done**:\n",
    "1. Statistical Analysis of Residuals - plot histogram of residuals and a QQplot to check for normality(Linear Regression)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "changing-lingerie",
   "metadata": {},
   "source": [
    "**Questions:**\n",
    "N/A"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "assigned-nylon",
   "metadata": {},
   "source": [
    "**Future Work:**\n",
    "1. The **autoTuning()** and **tsCrossValidation()** cost much time because the procedure is not distributed. The possible way is to overwrite the following code.\n",
    "    * CrossValidator: https://spark.apache.org/docs/3.1.1/api/python/_modules/pyspark/ml/tuning.html#CrossValidator\n",
    "    * TrainValidationSplit: https://spark.apache.org/docs/3.1.1/api/python/_modules/pyspark/ml/tuning.html#TrainValidationSplit\n",
    "2. Could add more algorithms of sklearn by Spark UDFs.\n",
    "    * Apache Arrow: https://spark.apache.org/docs/latest/api/python/user_guide/arrow_pandas.html\n",
    "3. Could consider Linear model Trees: https://medium.com/convoy-tech/the-best-of-both-worlds-linear-model-trees-7c9ce139767d"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sexual-vaccine",
   "metadata": {},
   "source": [
    "****Unified DataSet Format****    \n",
    "**TimeZone**: `UTC`\n",
    "1. `id`: Represent the order of this rows\n",
    "2. `Timestamp` : Represent the time of this record\n",
    "3. `Close`: Represent the original close price without shifting\n",
    "4. `NEXT_BTC_CLOSE`: Represent the close price with shifting. It's our label."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "stainless-humor",
   "metadata": {},
   "source": [
    "## 1. Data loading and Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "liable-velvet",
   "metadata": {},
   "source": [
    "### 1.1. Load related packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "renewable-volunteer",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Apache Spark\n",
    "from pyspark.sql import SparkSession\n",
    "import pyspark.sql.functions as F\n",
    "\n",
    "from pyspark.ml import Pipeline\n",
    "from pyspark.ml.feature import VectorAssembler,StandardScaler\n",
    "from pyspark.ml.regression import LinearRegression, GeneralizedLinearRegression, DecisionTreeRegressor, RandomForestRegressor, GBTRegressor\n",
    "from pyspark.ml.evaluation import RegressionEvaluator\n",
    "\n",
    "# Python\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from itertools import product\n",
    "import time\n",
    "\n",
    "# Graph packages\n",
    "# https://plotly.com/python/getting-started/#jupyterlab-support\n",
    "# https://plotly.com/python/time-series/\n",
    "import plotly.express as px\n",
    "\n",
    "# Scikit-learn\n",
    "from sklearn.metrics import mean_absolute_percentage_error"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "interstate-darkness",
   "metadata": {},
   "source": [
    "### 1.2. Create a SparkSession"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "refined-punishment",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start a SparkSession\n",
    "spark = SparkSession \\\n",
    "    .builder \\\n",
    "    .master(\"local[*]\") \\\n",
    "    .appName(\"Bitcoin Prediction\") \\\n",
    "    .getOrCreate()\n",
    "\n",
    "sc = spark.sparkContext"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "revolutionary-taste",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "            <div>\n",
       "                <p><b>SparkSession - in-memory</b></p>\n",
       "                \n",
       "        <div>\n",
       "            <p><b>SparkContext</b></p>\n",
       "\n",
       "            <p><a href=\"http://wvd-stud2prd-0.ad.nottingham.ac.uk:4041\">Spark UI</a></p>\n",
       "\n",
       "            <dl>\n",
       "              <dt>Version</dt>\n",
       "                <dd><code>v3.1.1</code></dd>\n",
       "              <dt>Master</dt>\n",
       "                <dd><code>local[*]</code></dd>\n",
       "              <dt>AppName</dt>\n",
       "                <dd><code>Bitcoin Prediction</code></dd>\n",
       "            </dl>\n",
       "        </div>\n",
       "        \n",
       "            </div>\n",
       "        "
      ],
      "text/plain": [
       "<pyspark.sql.session.SparkSession at 0x25445850a90>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show the URL of SparkSession\n",
    "spark"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adopted-submission",
   "metadata": {},
   "source": [
    "### 1.3. DataSet Checking"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "directed-cabin",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read csv file\n",
    "filename = \"bitcoin_10y_1min_interpolate.csv\"\n",
    "\n",
    "dataset = spark.read.format(\"csv\") \\\n",
    "          .option(\"inferSchema\",'True') \\\n",
    "          .option(\"header\",True) \\\n",
    "          .load(filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "starting-browser",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- Timestamp: string (nullable = true)\n",
      " |-- Open: double (nullable = true)\n",
      " |-- High: double (nullable = true)\n",
      " |-- Low: double (nullable = true)\n",
      " |-- Close: double (nullable = true)\n",
      " |-- Volume_(BTC): double (nullable = true)\n",
      " |-- Volume_(Currency): double (nullable = true)\n",
      " |-- Weighted_Price: double (nullable = true)\n",
      " |-- NEXT_BTC_CLOSE: double (nullable = true)\n",
      " |-- id: integer (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Check the schema of loaded dataSet\n",
    "dataset.printSchema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "private-mustang",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check the num of partitions\n",
    "dataset.rdd.getNumPartitions()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "popular-intent",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Row(id=4856403, Timestamp='2021-03-30 23:55:00', Close=58714.31, NEXT_BTC_CLOSE=58686.0),\n",
       " Row(id=4856404, Timestamp='2021-03-30 23:56:00', Close=58686.0, NEXT_BTC_CLOSE=58685.81),\n",
       " Row(id=4856405, Timestamp='2021-03-30 23:57:00', Close=58685.81, NEXT_BTC_CLOSE=58723.84),\n",
       " Row(id=4856406, Timestamp='2021-03-30 23:58:00', Close=58723.84, NEXT_BTC_CLOSE=58760.59),\n",
       " Row(id=4856407, Timestamp='2021-03-30 23:59:00', Close=58760.59, NEXT_BTC_CLOSE=58778.18)]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Have a look on the original and shifted label column\n",
    "dataset.select(\"id\",\"Timestamp\",\"Close\",\"NEXT_BTC_CLOSE\").tail(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "amber-spanish",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check if there are any \"nan\", \"Null\" or empty in numerical columns\n",
    "all_col = dataset.columns\n",
    "#dataset.select([F.count(F.when(F.isnan(cols) | dataset[cols].isNull() | (dataset[cols] == \"\"), cols)).alias(cols) for cols in all_col]).show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "dense-spank",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4856408"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Total rows\n",
    "dataset.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "falling-learning",
   "metadata": {},
   "source": [
    "## 2. Feature Engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "viral-extent",
   "metadata": {},
   "source": [
    "### 2.1. Transform to MLlib required format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "affecting-nashville",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[None, None, None]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# labels and features\n",
    "feature_cols = dataset.columns\n",
    "# Gain the column list of features\n",
    "non_feature_cols  = ['id',\"NEXT_BTC_CLOSE\",'Timestamp']\n",
    "[feature_cols.remove(non_feature) for non_feature in non_feature_cols]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "prepared-affiliation",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Open',\n",
       " 'High',\n",
       " 'Low',\n",
       " 'Close',\n",
       " 'Volume_(BTC)',\n",
       " 'Volume_(Currency)',\n",
       " 'Weighted_Price']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "korean-favorite",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Form a column contains all the features - VectorAssembler\n",
    "vector_assembler = VectorAssembler(inputCols=feature_cols, outputCol=\"features\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "imposed-suspension",
   "metadata": {},
   "source": [
    "### 2.2. Feature Standardization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "friendly-choir",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Standardize the \"features\" column\n",
    "# Use this column \"scaledFeatures\" if the algorithm didn't have the build-in param: Standardization\n",
    "standard_scaler = StandardScaler(inputCol=\"features\", outputCol=\"scaledFeatures\", withStd=True, withMean=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "handled-ethnic",
   "metadata": {},
   "source": [
    "### 2.3. Train-test Split(Time series)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "wrapped-israeli",
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Description: Split and keep the original time-series order\n",
    "Args:\n",
    "    dataSet: The dataSet which needs to be splited\n",
    "    proportion: A number represents the split proportion\n",
    "\n",
    "Return: \n",
    "    train_data: The train dataSet\n",
    "    test_data: The test dataSet\n",
    "'''\n",
    "def trainSplit(dataSet, proportion):\n",
    "    records_num = dataset.count()\n",
    "    split_point = round(records_num * proportion)\n",
    "    \n",
    "    train_data = dataset.filter(F.col(\"id\") < split_point)\n",
    "    test_data = dataset.filter(F.col(\"id\") >= split_point)\n",
    "    \n",
    "    return (train_data,test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "bound-refund",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data: 3399486\n",
      "Test data: 1456922\n"
     ]
    }
   ],
   "source": [
    "# Split the dataSet: Train(70%), test(30%)\n",
    "proportion = 0.7\n",
    "train_data,test_data = trainSplit(dataset, proportion)\n",
    "\n",
    "# Cache it\n",
    "train_data.cache()\n",
    "test_data.cache()\n",
    "\n",
    "# Number of train and test dataSets\n",
    "print(f\"Training data: {train_data.count()}\\nTest data: {test_data.count()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "living-radiation",
   "metadata": {},
   "source": [
    "## 3. Model Building"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "virtual-payment",
   "metadata": {},
   "source": [
    "### 3.1. LinearRegression(Simple workflow demo)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "better-christianity",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train a LinearRegression model\n",
    "lr = LinearRegression(featuresCol=\"features\",labelCol=\"NEXT_BTC_CLOSE\",maxIter=5, regParam=0.0, elasticNetParam=0.8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "statutory-language",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Chain assembler and forest in a Pipeline\n",
    "pipeline = Pipeline(stages=[vector_assembler, lr])\n",
    "pipeline_model = pipeline.fit(train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "enormous-thomas",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------------------+--------------+-----------------+\n",
      "|          Timestamp|NEXT_BTC_CLOSE|       prediction|\n",
      "+-------------------+--------------+-----------------+\n",
      "|2018-06-23 05:58:00|       6100.82| 6104.74202062437|\n",
      "|2018-06-23 05:59:00|       6105.39|6101.757430469564|\n",
      "|2018-06-23 06:00:00|        6085.6|6104.490407662455|\n",
      "|2018-06-23 06:01:00|       6074.08|6087.143770105451|\n",
      "|2018-06-23 06:02:00|        6073.6|6072.667359567475|\n",
      "+-------------------+--------------+-----------------+\n",
      "only showing top 5 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Make predictions\n",
    "predictions = pipeline_model.transform(test_data)\n",
    "predictions.select(\"Timestamp\",\"NEXT_BTC_CLOSE\",\"prediction\").show(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "molecular-drama",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Model': 'Linear Regression',\n",
       " 'Proportion': 0.7,\n",
       " 'RMSE': 23.08776704182093,\n",
       " 'MAPE': 0.000680338742099268,\n",
       " 'MAE': 9.368862815098803,\n",
       " 'Variance': 127357630.08188389,\n",
       " 'R2': 0.9999958145719638,\n",
       " 'Adjusted_R2': 0.9999958145374901}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute test error\n",
    "rmse_evaluator = RegressionEvaluator(labelCol=\"NEXT_BTC_CLOSE\", predictionCol=\"prediction\", metricName='rmse')\n",
    "mae_evaluator = RegressionEvaluator(labelCol=\"NEXT_BTC_CLOSE\", predictionCol=\"prediction\", metricName='mae')\n",
    "r2_evaluator = RegressionEvaluator(labelCol=\"NEXT_BTC_CLOSE\", predictionCol=\"prediction\", metricName='r2')\n",
    "var_evaluator = RegressionEvaluator(labelCol=\"NEXT_BTC_CLOSE\", predictionCol=\"prediction\", metricName='var')\n",
    "\n",
    "rmse = rmse_evaluator.evaluate(predictions)\n",
    "mae = mae_evaluator.evaluate(predictions)\n",
    "var = var_evaluator.evaluate(predictions)\n",
    "r2 = r2_evaluator.evaluate(predictions)\n",
    "\n",
    "predictions_pd = predictions.select(\"NEXT_BTC_CLOSE\",\"prediction\").toPandas()\n",
    "mape = mean_absolute_percentage_error(predictions_pd[\"NEXT_BTC_CLOSE\"], predictions_pd[\"prediction\"])\n",
    "\n",
    "# Adjusted R-squared\n",
    "n = predictions.count()\n",
    "p = len(predictions.columns)\n",
    "adj_r2 = 1-(1-r2)*(n-1)/(n-p-1)\n",
    "\n",
    "# Use dict to store each result\n",
    "results = {\n",
    "    \"Model\": \"Linear Regression\",\n",
    "    \"Proportion\": proportion,\n",
    "    \"RMSE\": rmse,\n",
    "    \"MAPE\": mape,\n",
    "    \"MAE\": mae,\n",
    "    \"Variance\": var,\n",
    "    \"R2\": r2,\n",
    "    \"Adjusted_R2\": adj_r2,\n",
    "}\n",
    "\n",
    "results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "magnetic-index",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DataFrame[Timestamp: string, Open: double, High: double, Low: double, Close: double, Volume_(BTC): double, Volume_(Currency): double, Weighted_Price: double, NEXT_BTC_CLOSE: double, id: int]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Release Cache\n",
    "train_data.unpersist()\n",
    "test_data.unpersist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "senior-conditions",
   "metadata": {},
   "source": [
    "## 4. Parameter Tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "individual-amber",
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Description: Use Grid Search to tune the Model \n",
    "Args:\n",
    "    dataSet: The dataSet which needs to be splited\n",
    "    proportion_lst: A list represents the split proportion\n",
    "    feature_col: The column name of features\n",
    "    label_col: The column name of label\n",
    "    ml_model: The module to use\n",
    "    params: Parameters which want to test \n",
    "    assembler: An assembler to dataSet\n",
    "    scaler: A scaler to dataSet\n",
    "Return: \n",
    "    results_df: The best result in a pandas dataframe\n",
    "'''\n",
    "def autoTuning(dataSet, proportion_lst, feature_col, label_col, ml_model, params, assembler, scaler):\n",
    "    \n",
    "    # Initialize the best result for comparison\n",
    "    result_best = {\"RMSE\": float('inf')}\n",
    "    \n",
    "    # Try different proportions \n",
    "    for proportion in proportion_lst:\n",
    "        # Split the dataSet\n",
    "        train_data,test_data = trainSplit(dataSet, proportion)\n",
    "    \n",
    "        # Cache it\n",
    "        train_data.cache()\n",
    "        test_data.cache()\n",
    "    \n",
    "        # ALL combination of params\n",
    "        param_lst = [dict(zip(params, param)) for param in product(*params.values())]\n",
    "    \n",
    "        for param in param_lst:\n",
    "            # Chosen Model\n",
    "            if ml_model == \"LinearRegression\":\n",
    "                model = LinearRegression(featuresCol=feature_col, \\\n",
    "                                         labelCol=label_col, \\\n",
    "                                         maxIter=param['maxIter'], \\\n",
    "                                         regParam=param['regParam'], \\\n",
    "                                         elasticNetParam=param['elasticNetParam'])\n",
    "            \n",
    "            elif ml_model == \"GeneralizedLinearRegression\":\n",
    "                model = GeneralizedLinearRegression(featuresCol=feature_col, \\\n",
    "                                                    labelCol=label_col, \\\n",
    "                                                    maxIter=param['maxIter'], \\\n",
    "                                                    regParam=param['regParam'], \\\n",
    "                                                    family=param['family'], \\\n",
    "                                                    link=param['link'])\n",
    "            \n",
    "            elif ml_model == \"DecisionTree\":\n",
    "                model = DecisionTreeRegressor(featuresCol=feature_col, \\\n",
    "                                              labelCol=label_col, \\\n",
    "                                              maxDepth = param[\"maxDepth\"], \\\n",
    "                                              seed=0)\n",
    "            \n",
    "            elif ml_model == \"RandomForest\":\n",
    "                model = RandomForestRegressor(featuresCol=feature_col, \\\n",
    "                                              labelCol=label_col, \\\n",
    "                                              numTrees = param[\"numTrees\"], \\\n",
    "                                              maxDepth = param[\"maxDepth\"], \\\n",
    "                                              seed=0)\n",
    "            \n",
    "            elif ml_model == \"GBTRegression\":\n",
    "                model = GBTRegressor(featuresCol=feature_col, \\\n",
    "                                     labelCol=label_col, \\\n",
    "                                     maxIter = param['maxIter'], \\\n",
    "                                     maxDepth = param['maxDepth'], \\\n",
    "                                     stepSize = param['stepSize'], \\\n",
    "                                     seed=0)\n",
    "            \n",
    "            # Chain assembler and model in a Pipeline\n",
    "            pipeline = Pipeline(stages=[assembler, model])\n",
    "            # Train a model and calculate running time\n",
    "            start = time.time()\n",
    "            pipeline_model = pipeline.fit(train_data)\n",
    "            end = time.time()\n",
    "\n",
    "            # Make predictions\n",
    "            predictions = pipeline_model.transform(test_data)\n",
    "\n",
    "            # Compute test error by several evaluators\n",
    "            # https://spark.apache.org/docs/3.1.1/mllib-evaluation-metrics.html#regression-model-evaluation\n",
    "            # https://spark.apache.org/docs/3.1.1/api/scala/org/apache/spark/ml/evaluation/RegressionEvaluator.html\n",
    "            rmse_evaluator = RegressionEvaluator(labelCol=label_col, predictionCol=\"prediction\", metricName='rmse')\n",
    "            mae_evaluator = RegressionEvaluator(labelCol=label_col, predictionCol=\"prediction\", metricName='mae')\n",
    "            r2_evaluator = RegressionEvaluator(labelCol=label_col, predictionCol=\"prediction\", metricName='r2')\n",
    "            var_evaluator = RegressionEvaluator(labelCol=label_col, predictionCol=\"prediction\", metricName='var')\n",
    "            \n",
    "            predictions_pd = predictions.select(\"NEXT_BTC_CLOSE\",\"prediction\").toPandas()\n",
    "            mape = mean_absolute_percentage_error(predictions_pd[\"NEXT_BTC_CLOSE\"], predictions_pd[\"prediction\"])\n",
    "            \n",
    "            rmse = rmse_evaluator.evaluate(predictions)\n",
    "            mae = mae_evaluator.evaluate(predictions)\n",
    "            var = var_evaluator.evaluate(predictions)\n",
    "            r2 = r2_evaluator.evaluate(predictions)\n",
    "            # Adjusted R-squared\n",
    "            n = predictions.count()\n",
    "            p = len(predictions.columns)\n",
    "            adj_r2 = 1-(1-r2)*(n-1)/(n-p-1)\n",
    "        \n",
    "            # Use dict to store each result\n",
    "            results = {\n",
    "                \"Model\": ml_model,\n",
    "                \"Proportion\": proportion,\n",
    "                \"Parameters\": [list(param.values())],\n",
    "                \"RMSE\": rmse,\n",
    "                \"MAPE\":mape,\n",
    "                \"MAE\": mae,\n",
    "                \"Variance\": var,\n",
    "                \"R2\": r2,\n",
    "                \"Adjusted_R2\": adj_r2,\n",
    "                \"Time\": end - start,\n",
    "                \"Predictions\": predictions.select(\"NEXT_BTC_CLOSE\",\"prediction\",'Timestamp')\n",
    "            }\n",
    "            \n",
    "            # Only store the lowest RMSE\n",
    "            if results['RMSE'] < result_best['RMSE']:\n",
    "                result_best = results\n",
    "                \n",
    "        # Release Cache\n",
    "        train_data.unpersist()\n",
    "        test_data.unpersist()\n",
    "        \n",
    "    # Transform dict to pandas dataframe\n",
    "    results_df = pd.DataFrame(result_best)\n",
    "    return results_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "decimal-thickness",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a function to plot line-like graph\n",
    "# https://plotly.com/python/time-series/#time-series-with-range-selector-buttons\n",
    "'''\n",
    "Description: Plot the line graph by plotly(custom design)\n",
    "Args:\n",
    "    data: The data(pandas dataframe) which you want to ploy by line\n",
    "    graph_title: The title of the graph\n",
    "Return: None\n",
    "'''\n",
    "def line_plot(data,graph_title):\n",
    "    plot = px.line(data,title=graph_title)\n",
    "    plot.update_xaxes(\n",
    "        rangeslider_visible=True,\n",
    "        rangeselector=dict(\n",
    "            buttons=list([\n",
    "                dict(count=7, label=\"1w\", step=\"day\", stepmode=\"backward\"),\n",
    "                dict(count=1, label=\"1m\", step=\"month\", stepmode=\"backward\"),\n",
    "                dict(count=6, label=\"6m\", step=\"month\", stepmode=\"backward\"),\n",
    "                dict(count=1, label=\"1y\", step=\"year\", stepmode=\"backward\"),\n",
    "                dict(step=\"all\")\n",
    "            ])\n",
    "        )\n",
    "    )\n",
    "    plot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "deadly-leonard",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Draw the Prediction Graph\n",
    "'''\n",
    "Description: Plot a line graph of the Prediction\n",
    "Args:\n",
    "    result: The result from the \"autoTuning\" Func\n",
    "    graph_title: The title of the graph\n",
    "Return: None\n",
    "'''\n",
    "def drawPrediction(result, graph_title):\n",
    "    result_pd = result['Predictions'][0].withColumn(\"Time\", F.to_timestamp(\"Timestamp\", 'yyyy-MM-dd HH:mm:ss')) \\\n",
    "                                        .drop(\"Timestamp\") \\\n",
    "                                        .toPandas() \\\n",
    "                                        .set_index('Time')\n",
    "    # Display the info of the best trained Model\n",
    "    print(result.iloc[0,:-1])\n",
    "    # Draw by plotly\n",
    "    line_plot(result_pd, graph_title)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "flexible-rubber",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameter choosing - only use the most popular/useful params\n",
    "# https://spark.apache.org/docs/3.1.1/ml-classification-regression.html#regression\n",
    "# The explanations of params are form the above document"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "narrative-sharp",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split proportion list\n",
    "proportion_lst = [0.6, 0.7, 0.8, 0.9]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "primary-chuck",
   "metadata": {},
   "source": [
    "### 4.1. Linear Regression Model - Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "informed-liberia",
   "metadata": {},
   "outputs": [],
   "source": [
    "# LinearRegression\n",
    "lr_params = {\n",
    "    'maxIter' : [5, 10, 50, 80, 100], # max number of iterations (>=0), default:100\n",
    "    'regParam' : np.arange(0,1,0.2).round(decimals=2),# regularization parameter (>=0), default:0.0\n",
    "    'elasticNetParam' : np.arange(0,1,0.2).round(decimals=2) # the ElasticNet mixing parameter, [0, 1], default:0.0 \n",
    "}\n",
    "result_lr = autoTuning(dataset, proportion_lst, \"features\", \"NEXT_BTC_CLOSE\", \"LinearRegression\", lr_params, vector_assembler ,standard_scaler)\n",
    "# Virtualization\n",
    "drawPrediction(result_lr,\"Predict by LinearRegression\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "conservative-feeling",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generalized linear regression\n",
    "glr_params = {\n",
    "    'maxIter' : [5, 10, 50, 80], # max number of iterations (>=0), default:25\n",
    "    'regParam' : [0, 0.1, 0.2],# regularization parameter (>=0), default:0.0\n",
    "    'family': ['gaussian', 'gamma'], # The name of family which is a description of the error distribution to be used in the model.\n",
    "    'link': ['identity', 'inverse'] # which provides the relationship between the linear predictor and the mean of the distribution function.\n",
    "}\n",
    "result_glr = autoTuning(dataset, proportion_lst, \"features\", \"NEXT_BTC_CLOSE\", \"GeneralizedLinearRegression\", glr_params, vector_assembler ,standard_scaler)\n",
    "# Virtualization\n",
    "drawPrediction(result_glr,\"Predict by GeneralizedLinearRegression\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "downtown-printer",
   "metadata": {},
   "source": [
    "### 4.2. Tree based Model - Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "warming-genetics",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model          RandomForest\n",
      "Proportion              0.7\n",
      "Parameters          [5, 10]\n",
      "RMSE                9335.58\n",
      "MAPE                0.16048\n",
      "MAE                 3553.66\n",
      "Variance        2.29335e+07\n",
      "R2                  0.31568\n",
      "Adjusted_R2        0.315675\n",
      "Time                23.4275\n",
      "Name: 0, dtype: object\n"
     ]
    }
   ],
   "source": [
    "# RandomForest\n",
    "rf_params = {\n",
    "    'numTrees' : [3, 5, 10, 20, 30],# Number of trees to train, >=1, default:20\n",
    "    'maxDepth' : [3, 5, 10] # Maximum depth of the tree, <=30, default:5\n",
    "}\n",
    "result_rf = autoTuning(dataset, proportion_lst, \"features\", \"NEXT_BTC_CLOSE\", \"RandomForest\", rf_params, vector_assembler ,standard_scaler)\n",
    "# Virtualization\n",
    "drawPrediction(result_rf,\"Predict by RandomForest\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "aging-wagner",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model          GBTRegression\n",
      "Proportion               0.7\n",
      "Parameters      [40, 5, 0.3]\n",
      "RMSE                 8090.42\n",
      "MAPE                0.128075\n",
      "MAE                  2977.23\n",
      "Variance         2.78414e+07\n",
      "R2                  0.486053\n",
      "Adjusted_R2         0.486048\n",
      "Time                 168.555\n",
      "Name: 0, dtype: object\n"
     ]
    }
   ],
   "source": [
    "# GBTRegression\n",
    "gb_params = {\n",
    "    'maxIter' : [20, 40, 60], # max number of iterations (>=0), default:20\n",
    "    'maxDepth' : [5, 8, 10], # Maximum depth of the tree (>=0), <=30, default:5\n",
    "    'stepSize': [0.1, 0.3, 0.5, 0.7] # learning rate, [0,1], default:0.1\n",
    "}\n",
    "result_gb = autoTuning(dataset, proportion_lst, \"features\", \"NEXT_BTC_CLOSE\", \"GBTRegression\", gb_params, vector_assembler ,standard_scaler)\n",
    "# Virtualization\n",
    "drawPrediction(result_gb,\"Predict by GBTRegression\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "moral-karma",
   "metadata": {},
   "source": [
    "## 5. Time Series Cross Validation \n",
    "> https://machinelearningmastery.com/backtest-machine-learning-models-time-series-forecasting/     \n",
    "> https://hub.packtpub.com/cross-validation-strategies-for-time-series-forecasting-tutorial/        \n",
    "> https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "unsigned-identity",
   "metadata": {},
   "source": [
    "### 5.1. Multiple Splits Time Series Cross Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "therapeutic-conference",
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Description: Multiple Splits Cross Validation on Time Series data\n",
    "Args:\n",
    "    num: Number of DataSet\n",
    "    n_splits: Split times\n",
    "Return: \n",
    "    split_position_df: All set of splits position in a Pandas dataframe\n",
    "'''\n",
    "def mulTsCrossValidation(num, n_splits):\n",
    "    split_position_lst = []\n",
    "    # Calculate the split position for each time \n",
    "    for i in range(1, n_splits+1):\n",
    "        # Calculate train size and test size\n",
    "        train_size = i * num // (n_splits + 1) + num % (n_splits + 1)\n",
    "        test_size = num //(n_splits + 1)\n",
    "\n",
    "        # Calculate the start/split/end point for each fold\n",
    "        start = 0\n",
    "        split = train_size\n",
    "        end = train_size + test_size\n",
    "        \n",
    "        # Avoid to beyond the whole number of dataSet\n",
    "        if end > num:\n",
    "            end = num\n",
    "        split_position_lst.append((start,split,end))\n",
    "        \n",
    "    # Transform the split position list to a Pandas Dataframe\n",
    "    split_position_df = pd.DataFrame(split_position_lst,columns=['start','split','end'])\n",
    "    return split_position_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "consecutive-stopping",
   "metadata": {},
   "source": [
    "### 5.2. Blocked Time Series Cross Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "related-preservation",
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Description: Blocked Time Series Cross Validation\n",
    "Args:\n",
    "    num: Number of DataSet\n",
    "    n_splits: Split times\n",
    "Return: \n",
    "    split_position_df: All set of splits position in a Pandas dataframe\n",
    "'''\n",
    "def blockedTsCrossValidation(num, n_splits):\n",
    "    kfold_size = num // n_splits\n",
    "\n",
    "    split_position_lst = []\n",
    "    # Calculate the split position for each time \n",
    "    for i in range(n_splits):\n",
    "        # Calculate the start/split/end point for each fold\n",
    "        start = i * kfold_size\n",
    "        end = start + kfold_size\n",
    "        # Manually set train-test split proportion in each fold\n",
    "        split = int(0.8 * (end - start)) + start\n",
    "        split_position_lst.append((start,split,end))\n",
    "        \n",
    "    # Transform the split position list to a Pandas Dataframe\n",
    "    split_position_df = pd.DataFrame(split_position_lst,columns=['start','split','end'])\n",
    "    return split_position_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "serious-lesson",
   "metadata": {},
   "source": [
    "### 5.3. Walk Forward Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "competitive-money",
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Description: Walk Forward Validation on Time Series data\n",
    "Args:\n",
    "    num: Number of DataSet\n",
    "    min_obser: Minimum Number of Observations\n",
    "    expand_window: Sliding or Expanding Window\n",
    "Return: \n",
    "    split_position_df: All set of splits position in a Pandas dataframe\n",
    "'''\n",
    "def wfTsCrossValidation(num, min_obser, expand_window):\n",
    "    split_position_lst = []\n",
    "    # Calculate the split position for each time \n",
    "    for i in range(min_obser,num,expand_window):\n",
    "        # Calculate the start/split/end point for each fold\n",
    "        start = 0\n",
    "        split = i\n",
    "        end = split + expand_window\n",
    "        \n",
    "        # Avoid to beyond the whole number of dataSet\n",
    "        if end > num:\n",
    "            end = num\n",
    "        split_position_lst.append((start,split,end))\n",
    "        \n",
    "    # Transform the split position list to a Pandas Dataframe\n",
    "    split_position_df = pd.DataFrame(split_position_lst,columns=['start','split','end'])\n",
    "    return split_position_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dense-alias",
   "metadata": {},
   "source": [
    "### 5.4. Time Series Cross Validation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "running-video",
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Description: Cross Validation on Time Series data\n",
    "Args:\n",
    "    dataSet: The dataSet which needs to be splited\n",
    "    feature_col: The column name of features\n",
    "    label_col: The column name of label\n",
    "    ml_model: The module to use\n",
    "    params: Parameters which want to test \n",
    "    assembler: An assembler to dataSet\n",
    "    scaler: A scaler to dataSet\n",
    "    cv_info: The type of Cross Validation\n",
    "Return: \n",
    "    tsCv_df: All the splits performance of each model in a pandas dataframe\n",
    "'''\n",
    "def tsCrossValidation(dataSet, feature_col, label_col, ml_model, params, assembler, scaler, cv_info):\n",
    "    \n",
    "    # Get the number of samples\n",
    "    num = dataSet.count()\n",
    "    \n",
    "    # Save results in a list\n",
    "    result_lst = []\n",
    "    \n",
    "    # ALL combination of params\n",
    "    param_lst = [dict(zip(params, param)) for param in product(*params.values())]\n",
    "\n",
    "    for param in param_lst:\n",
    "        # Chosen Model\n",
    "        if ml_model == \"LinearRegression\":\n",
    "            model = LinearRegression(featuresCol=feature_col, \\\n",
    "                                     labelCol=label_col, \\\n",
    "                                     maxIter=param['maxIter'], \\\n",
    "                                     regParam=param['regParam'], \\\n",
    "                                     elasticNetParam=param['elasticNetParam'])\n",
    "\n",
    "        elif ml_model == \"GeneralizedLinearRegression\":\n",
    "            model = GeneralizedLinearRegression(featuresCol=feature_col, \\\n",
    "                                                labelCol=label_col, \\\n",
    "                                                maxIter=param['maxIter'], \\\n",
    "                                                regParam=param['regParam'], \\\n",
    "                                                family=param['family'], \\\n",
    "                                                link=param['link'])\n",
    "\n",
    "        elif ml_model == \"DecisionTree\":\n",
    "            model = DecisionTreeRegressor(featuresCol=feature_col, \\\n",
    "                                          labelCol=label_col, \\\n",
    "                                          maxDepth = param[\"maxDepth\"], \\\n",
    "                                          seed=0)\n",
    "\n",
    "        elif ml_model == \"RandomForest\":\n",
    "            model = RandomForestRegressor(featuresCol=feature_col, \\\n",
    "                                          labelCol=label_col, \\\n",
    "                                          numTrees = param[\"numTrees\"], \\\n",
    "                                          maxDepth = param[\"maxDepth\"], \\\n",
    "                                          seed=0)\n",
    "\n",
    "        elif ml_model == \"GBTRegression\":\n",
    "            model = GBTRegressor(featuresCol=feature_col, \\\n",
    "                                 labelCol=label_col, \\\n",
    "                                 maxIter = param['maxIter'], \\\n",
    "                                 maxDepth = param['maxDepth'], \\\n",
    "                                 stepSize = param['stepSize'], \\\n",
    "                                 seed=0)\n",
    "            \n",
    "        \n",
    "        \n",
    "        # Identify the type of Cross Validation \n",
    "        if cv_info['cv_type'] == 'mulTs':\n",
    "            split_position_df = mulTsCrossValidation(num, cv_info['kSplits'])\n",
    "        elif cv_info['cv_type'] == 'blkTs':\n",
    "            split_position_df = blockedTsCrossValidation(num, cv_info['kSplits'])\n",
    "        elif cv_info['cv_type'] == 'wfTs':\n",
    "            split_position_df = wfTsCrossValidation(num, cv_info['min_obser'], cv_info['expand_window'])\n",
    "            \n",
    "\n",
    "        for position in split_position_df.itertuples():\n",
    "            # Get the start/split/end position from a kind of Time Series Cross Validation\n",
    "            start = getattr(position, 'start')\n",
    "            splits = getattr(position, 'split')\n",
    "            end = getattr(position, 'end')\n",
    "            idx  = getattr(position, 'Index')\n",
    "            \n",
    "            # Train/Test size\n",
    "            train_size = splits - start\n",
    "            test_size = end - splits\n",
    "\n",
    "            # Get training data and test data\n",
    "            train_data = dataSet.filter(F.col(\"id\").between(start, splits-1))\n",
    "            test_data = dataSet.filter(F.col(\"id\").between(splits, end-1))\n",
    "\n",
    "            # Cache it\n",
    "            train_data.cache()\n",
    "            test_data.cache()\n",
    "\n",
    "            # Chain assembler and model in a Pipeline\n",
    "            pipeline = Pipeline(stages=[assembler, model])\n",
    "            # Train a model and calculate running time\n",
    "            start = time.time()\n",
    "            pipeline_model = pipeline.fit(train_data)\n",
    "            end = time.time()\n",
    "\n",
    "            # Make predictions\n",
    "            predictions = pipeline_model.transform(test_data)\n",
    "\n",
    "            # Compute test error by several evaluator\n",
    "            rmse_evaluator = RegressionEvaluator(labelCol=label_col, predictionCol=\"prediction\", metricName='rmse')\n",
    "            mae_evaluator = RegressionEvaluator(labelCol=label_col, predictionCol=\"prediction\", metricName='mae')\n",
    "            r2_evaluator = RegressionEvaluator(labelCol=label_col, predictionCol=\"prediction\", metricName='r2')\n",
    "            var_evaluator = RegressionEvaluator(labelCol=label_col, predictionCol=\"prediction\", metricName='var')\n",
    "            \n",
    "            predictions_pd = predictions.select(\"NEXT_BTC_CLOSE\",\"prediction\").toPandas()\n",
    "            mape = mean_absolute_percentage_error(predictions_pd[\"NEXT_BTC_CLOSE\"], predictions_pd[\"prediction\"])\n",
    "\n",
    "            rmse = rmse_evaluator.evaluate(predictions)\n",
    "            mae = mae_evaluator.evaluate(predictions)\n",
    "            var = var_evaluator.evaluate(predictions)\n",
    "            r2 = r2_evaluator.evaluate(predictions)\n",
    "            # Adjusted R-squared\n",
    "            n = predictions.count()\n",
    "            p = len(predictions.columns)\n",
    "            adj_r2 = 1-(1-r2)*(n-1)/(n-p-1)\n",
    "\n",
    "            # Use dict to store each result\n",
    "            results = {\n",
    "                \"Model\": ml_model,\n",
    "                'CV_type': cv_info['cv_type'],\n",
    "                \"Splits\": idx + 1,\n",
    "                \"Train&Test\": (train_size,test_size),\n",
    "                \"Parameters\": list(param.values()),\n",
    "                \"RMSE\": rmse,\n",
    "                \"MAPE\": mape,\n",
    "                \"MAE\": mae,\n",
    "                \"Variance\": var,\n",
    "                \"R2\": r2,\n",
    "                \"Adjusted_R2\": adj_r2,\n",
    "                \"Time\": end - start\n",
    "            }\n",
    "            \n",
    "            # Store each splits result\n",
    "            result_lst.append(results)\n",
    "            \n",
    "            # Release Cache\n",
    "            train_data.unpersist()\n",
    "            test_data.unpersist()\n",
    "\n",
    "    # Transform dict to pandas dataframe\n",
    "    tsCv_df = pd.DataFrame(result_lst)\n",
    "    return tsCv_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "passive-slovak",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Cross Validation Parameter\n",
    "# Multiple Splits Time Series Cross Validation\n",
    "mul_cv = {'cv_type':'mulTs',\n",
    "          'kSplits': 5}\n",
    "\n",
    "# Blocked Time Series Cross Validation\n",
    "blk_cv = {'cv_type':'blkTs',\n",
    "          'kSplits': 10}\n",
    "\n",
    "# Walk Forward Validation, Last 50 steps\n",
    "wf_cv = {'cv_type':'wfTs',\n",
    "         'min_obser': 4856359,\n",
    "         'expand_window': 1}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "endless-launch",
   "metadata": {},
   "source": [
    "#### 5.4.1. LinearRegression Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "incorporate-billion",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(809403, 809401)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>1.366073</td>\n",
       "      <td>0.001256</td>\n",
       "      <td>0.612516</td>\n",
       "      <td>5.248168e+04</td>\n",
       "      <td>0.999964</td>\n",
       "      <td>0.999964</td>\n",
       "      <td>4.625429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(1618804, 809401)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>0.935522</td>\n",
       "      <td>0.001143</td>\n",
       "      <td>0.248898</td>\n",
       "      <td>1.863908e+04</td>\n",
       "      <td>0.999953</td>\n",
       "      <td>0.999953</td>\n",
       "      <td>4.872537</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(2428206, 809401)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>13.591950</td>\n",
       "      <td>0.000985</td>\n",
       "      <td>5.412886</td>\n",
       "      <td>1.896515e+07</td>\n",
       "      <td>0.999990</td>\n",
       "      <td>0.999990</td>\n",
       "      <td>5.327785</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(3237607, 809401)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>8.830537</td>\n",
       "      <td>0.000657</td>\n",
       "      <td>4.810164</td>\n",
       "      <td>5.626686e+06</td>\n",
       "      <td>0.999986</td>\n",
       "      <td>0.999986</td>\n",
       "      <td>6.095136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(4047008, 809400)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>30.032420</td>\n",
       "      <td>0.000733</td>\n",
       "      <td>13.404001</td>\n",
       "      <td>1.858179e+08</td>\n",
       "      <td>0.999995</td>\n",
       "      <td>0.999995</td>\n",
       "      <td>6.843345</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Model CV_type  Splits         Train&Test     Parameters  \\\n",
       "0  LinearRegression   mulTs       1   (809403, 809401)  [5, 0.0, 0.8]   \n",
       "1  LinearRegression   mulTs       2  (1618804, 809401)  [5, 0.0, 0.8]   \n",
       "2  LinearRegression   mulTs       3  (2428206, 809401)  [5, 0.0, 0.8]   \n",
       "3  LinearRegression   mulTs       4  (3237607, 809401)  [5, 0.0, 0.8]   \n",
       "4  LinearRegression   mulTs       5  (4047008, 809400)  [5, 0.0, 0.8]   \n",
       "\n",
       "        RMSE      MAPE        MAE      Variance        R2  Adjusted_R2  \\\n",
       "0   1.366073  0.001256   0.612516  5.248168e+04  0.999964     0.999964   \n",
       "1   0.935522  0.001143   0.248898  1.863908e+04  0.999953     0.999953   \n",
       "2  13.591950  0.000985   5.412886  1.896515e+07  0.999990     0.999990   \n",
       "3   8.830537  0.000657   4.810164  5.626686e+06  0.999986     0.999986   \n",
       "4  30.032420  0.000733  13.404001  1.858179e+08  0.999995     0.999995   \n",
       "\n",
       "       Time  \n",
       "0  4.625429  \n",
       "1  4.872537  \n",
       "2  5.327785  \n",
       "3  6.095136  \n",
       "4  6.843345  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# LinearRegression\n",
    "lr_params = {\n",
    "    'maxIter' : [5], # max number of iterations (>=0), default:100\n",
    "    'regParam' : [0.0],# regularization parameter (>=0), default:0.0\n",
    "    'elasticNetParam' : [0.8] # the ElasticNet mixing parameter, [0, 1], default:0.0 \n",
    "}\n",
    "lr_mul_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"LinearRegression\", lr_params, vector_assembler ,standard_scaler, mul_cv)\n",
    "lr_mul_cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "accessory-crest",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>0.010439</td>\n",
       "      <td>0.000255</td>\n",
       "      <td>0.002908</td>\n",
       "      <td>4.676590e-01</td>\n",
       "      <td>0.999767</td>\n",
       "      <td>0.999767</td>\n",
       "      <td>3.317110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>0.329218</td>\n",
       "      <td>0.000937</td>\n",
       "      <td>0.139805</td>\n",
       "      <td>9.453162e+02</td>\n",
       "      <td>0.999885</td>\n",
       "      <td>0.999885</td>\n",
       "      <td>3.700262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>0.767360</td>\n",
       "      <td>0.000973</td>\n",
       "      <td>0.441272</td>\n",
       "      <td>5.216484e+03</td>\n",
       "      <td>0.999887</td>\n",
       "      <td>0.999887</td>\n",
       "      <td>3.786622</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>0.260701</td>\n",
       "      <td>0.000577</td>\n",
       "      <td>0.146041</td>\n",
       "      <td>6.426075e+02</td>\n",
       "      <td>0.999894</td>\n",
       "      <td>0.999894</td>\n",
       "      <td>3.677978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>2.425555</td>\n",
       "      <td>0.004829</td>\n",
       "      <td>0.520071</td>\n",
       "      <td>2.306649e+03</td>\n",
       "      <td>0.997453</td>\n",
       "      <td>0.997452</td>\n",
       "      <td>3.343266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>6</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>4.338398</td>\n",
       "      <td>0.001176</td>\n",
       "      <td>2.787380</td>\n",
       "      <td>8.450683e+04</td>\n",
       "      <td>0.999777</td>\n",
       "      <td>0.999777</td>\n",
       "      <td>3.182063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>7</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>8.156550</td>\n",
       "      <td>0.000688</td>\n",
       "      <td>5.547771</td>\n",
       "      <td>9.696180e+05</td>\n",
       "      <td>0.999931</td>\n",
       "      <td>0.999931</td>\n",
       "      <td>3.380068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>8</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>7.433761</td>\n",
       "      <td>0.000621</td>\n",
       "      <td>3.821281</td>\n",
       "      <td>1.649642e+06</td>\n",
       "      <td>0.999966</td>\n",
       "      <td>0.999966</td>\n",
       "      <td>3.441067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>9</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>11.280208</td>\n",
       "      <td>0.001000</td>\n",
       "      <td>6.802791</td>\n",
       "      <td>1.568764e+06</td>\n",
       "      <td>0.999919</td>\n",
       "      <td>0.999919</td>\n",
       "      <td>3.521520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>10</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>68.313149</td>\n",
       "      <td>0.001010</td>\n",
       "      <td>47.464780</td>\n",
       "      <td>7.598891e+07</td>\n",
       "      <td>0.999939</td>\n",
       "      <td>0.999939</td>\n",
       "      <td>3.468341</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Model CV_type  Splits       Train&Test     Parameters  \\\n",
       "0  LinearRegression   blkTs       1  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "1  LinearRegression   blkTs       2  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "2  LinearRegression   blkTs       3  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "3  LinearRegression   blkTs       4  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "4  LinearRegression   blkTs       5  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "5  LinearRegression   blkTs       6  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "6  LinearRegression   blkTs       7  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "7  LinearRegression   blkTs       8  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "8  LinearRegression   blkTs       9  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "9  LinearRegression   blkTs      10  (388512, 97128)  [5, 0.0, 0.8]   \n",
       "\n",
       "        RMSE      MAPE        MAE      Variance        R2  Adjusted_R2  \\\n",
       "0   0.010439  0.000255   0.002908  4.676590e-01  0.999767     0.999767   \n",
       "1   0.329218  0.000937   0.139805  9.453162e+02  0.999885     0.999885   \n",
       "2   0.767360  0.000973   0.441272  5.216484e+03  0.999887     0.999887   \n",
       "3   0.260701  0.000577   0.146041  6.426075e+02  0.999894     0.999894   \n",
       "4   2.425555  0.004829   0.520071  2.306649e+03  0.997453     0.997452   \n",
       "5   4.338398  0.001176   2.787380  8.450683e+04  0.999777     0.999777   \n",
       "6   8.156550  0.000688   5.547771  9.696180e+05  0.999931     0.999931   \n",
       "7   7.433761  0.000621   3.821281  1.649642e+06  0.999966     0.999966   \n",
       "8  11.280208  0.001000   6.802791  1.568764e+06  0.999919     0.999919   \n",
       "9  68.313149  0.001010  47.464780  7.598891e+07  0.999939     0.999939   \n",
       "\n",
       "       Time  \n",
       "0  3.317110  \n",
       "1  3.700262  \n",
       "2  3.786622  \n",
       "3  3.677978  \n",
       "4  3.343266  \n",
       "5  3.182063  \n",
       "6  3.380068  \n",
       "7  3.441067  \n",
       "8  3.521520  \n",
       "9  3.468341  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_blk_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"LinearRegression\", lr_params, vector_assembler ,standard_scaler, blk_cv)\n",
    "lr_blk_cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "after-slovak",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(4856359, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>6.410132</td>\n",
       "      <td>0.000109</td>\n",
       "      <td>6.410132</td>\n",
       "      <td>41.089788</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.713154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(4856360, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>2.135334</td>\n",
       "      <td>0.000036</td>\n",
       "      <td>2.135334</td>\n",
       "      <td>4.559651</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.805584</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(4856361, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>28.938096</td>\n",
       "      <td>0.000493</td>\n",
       "      <td>28.938096</td>\n",
       "      <td>837.413416</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.877965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(4856362, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>35.871947</td>\n",
       "      <td>0.000611</td>\n",
       "      <td>35.871947</td>\n",
       "      <td>1286.796570</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.024721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(4856363, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>57.297181</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>57.297181</td>\n",
       "      <td>3282.966920</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.112156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>6</td>\n",
       "      <td>(4856364, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>12.726607</td>\n",
       "      <td>0.000217</td>\n",
       "      <td>12.726607</td>\n",
       "      <td>161.966535</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.866153</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>7</td>\n",
       "      <td>(4856365, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>1.909822</td>\n",
       "      <td>0.000033</td>\n",
       "      <td>1.909822</td>\n",
       "      <td>3.647420</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.654250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>8</td>\n",
       "      <td>(4856366, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>7.355302</td>\n",
       "      <td>0.000125</td>\n",
       "      <td>7.355302</td>\n",
       "      <td>54.100463</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.010690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>9</td>\n",
       "      <td>(4856367, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>6.841876</td>\n",
       "      <td>0.000117</td>\n",
       "      <td>6.841876</td>\n",
       "      <td>46.811273</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.833155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>10</td>\n",
       "      <td>(4856368, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>21.152901</td>\n",
       "      <td>0.000361</td>\n",
       "      <td>21.152901</td>\n",
       "      <td>447.445234</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.956287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>11</td>\n",
       "      <td>(4856369, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>37.335059</td>\n",
       "      <td>0.000637</td>\n",
       "      <td>37.335059</td>\n",
       "      <td>1393.906614</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.754557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>12</td>\n",
       "      <td>(4856370, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>15.826054</td>\n",
       "      <td>0.000270</td>\n",
       "      <td>15.826054</td>\n",
       "      <td>250.463980</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.784153</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>13</td>\n",
       "      <td>(4856371, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>25.924992</td>\n",
       "      <td>0.000442</td>\n",
       "      <td>25.924992</td>\n",
       "      <td>672.105211</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.633149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>14</td>\n",
       "      <td>(4856372, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>24.634869</td>\n",
       "      <td>0.000420</td>\n",
       "      <td>24.634869</td>\n",
       "      <td>606.876749</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.703892</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>15</td>\n",
       "      <td>(4856373, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>20.704843</td>\n",
       "      <td>0.000353</td>\n",
       "      <td>20.704843</td>\n",
       "      <td>428.690515</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.653950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>16</td>\n",
       "      <td>(4856374, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>39.172257</td>\n",
       "      <td>0.000667</td>\n",
       "      <td>39.172257</td>\n",
       "      <td>1534.465710</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.998795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>17</td>\n",
       "      <td>(4856375, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>7.184019</td>\n",
       "      <td>0.000122</td>\n",
       "      <td>7.184019</td>\n",
       "      <td>51.610127</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.758154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>18</td>\n",
       "      <td>(4856376, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>27.686442</td>\n",
       "      <td>0.000471</td>\n",
       "      <td>27.686442</td>\n",
       "      <td>766.539073</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.017116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>19</td>\n",
       "      <td>(4856377, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>1.032495</td>\n",
       "      <td>0.000018</td>\n",
       "      <td>1.032495</td>\n",
       "      <td>1.066046</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.623888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>20</td>\n",
       "      <td>(4856378, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>8.209809</td>\n",
       "      <td>0.000140</td>\n",
       "      <td>8.209809</td>\n",
       "      <td>67.400956</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9.101103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>21</td>\n",
       "      <td>(4856379, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>21.658920</td>\n",
       "      <td>0.000369</td>\n",
       "      <td>21.658920</td>\n",
       "      <td>469.108828</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.032086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>22</td>\n",
       "      <td>(4856380, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>49.425394</td>\n",
       "      <td>0.000842</td>\n",
       "      <td>49.425394</td>\n",
       "      <td>2442.869525</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.899149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>23</td>\n",
       "      <td>(4856381, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>24.535962</td>\n",
       "      <td>0.000418</td>\n",
       "      <td>24.535962</td>\n",
       "      <td>602.013414</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.786549</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>24</td>\n",
       "      <td>(4856382, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>56.836666</td>\n",
       "      <td>0.000969</td>\n",
       "      <td>56.836666</td>\n",
       "      <td>3230.406651</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.552687</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>25</td>\n",
       "      <td>(4856383, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>39.960240</td>\n",
       "      <td>0.000682</td>\n",
       "      <td>39.960240</td>\n",
       "      <td>1596.820775</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.340163</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>26</td>\n",
       "      <td>(4856384, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>14.438882</td>\n",
       "      <td>0.000246</td>\n",
       "      <td>14.438882</td>\n",
       "      <td>208.481300</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.014159</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>27</td>\n",
       "      <td>(4856385, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>40.345751</td>\n",
       "      <td>0.000688</td>\n",
       "      <td>40.345751</td>\n",
       "      <td>1627.779589</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.785308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>28</td>\n",
       "      <td>(4856386, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>17.997994</td>\n",
       "      <td>0.000307</td>\n",
       "      <td>17.997994</td>\n",
       "      <td>323.927787</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.744155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>29</td>\n",
       "      <td>(4856387, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>1.993140</td>\n",
       "      <td>0.000034</td>\n",
       "      <td>1.993140</td>\n",
       "      <td>3.972607</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.897159</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>30</td>\n",
       "      <td>(4856388, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>1.748922</td>\n",
       "      <td>0.000030</td>\n",
       "      <td>1.748922</td>\n",
       "      <td>3.058725</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.948405</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>31</td>\n",
       "      <td>(4856389, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>6.008873</td>\n",
       "      <td>0.000103</td>\n",
       "      <td>6.008873</td>\n",
       "      <td>36.106550</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.899360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>32</td>\n",
       "      <td>(4856390, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>10.399502</td>\n",
       "      <td>0.000177</td>\n",
       "      <td>10.399502</td>\n",
       "      <td>108.149632</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.722106</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>33</td>\n",
       "      <td>(4856391, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>7.936927</td>\n",
       "      <td>0.000135</td>\n",
       "      <td>7.936927</td>\n",
       "      <td>62.994808</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.814153</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>34</td>\n",
       "      <td>(4856392, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>1.508807</td>\n",
       "      <td>0.000026</td>\n",
       "      <td>1.508807</td>\n",
       "      <td>2.276498</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.127162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>35</td>\n",
       "      <td>(4856393, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>67.454859</td>\n",
       "      <td>0.001150</td>\n",
       "      <td>67.454859</td>\n",
       "      <td>4550.157988</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.914156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>36</td>\n",
       "      <td>(4856394, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>33.108827</td>\n",
       "      <td>0.000564</td>\n",
       "      <td>33.108827</td>\n",
       "      <td>1096.194448</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.123162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>37</td>\n",
       "      <td>(4856395, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>22.965371</td>\n",
       "      <td>0.000391</td>\n",
       "      <td>22.965371</td>\n",
       "      <td>527.408280</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.769149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>38</td>\n",
       "      <td>(4856396, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>32.513714</td>\n",
       "      <td>0.000554</td>\n",
       "      <td>32.513714</td>\n",
       "      <td>1057.141613</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.888173</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>39</td>\n",
       "      <td>(4856397, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>45.905748</td>\n",
       "      <td>0.000782</td>\n",
       "      <td>45.905748</td>\n",
       "      <td>2107.337681</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.952802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>40</td>\n",
       "      <td>(4856398, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>12.388048</td>\n",
       "      <td>0.000211</td>\n",
       "      <td>12.388048</td>\n",
       "      <td>153.463731</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.082202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>41</td>\n",
       "      <td>(4856399, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>2.554572</td>\n",
       "      <td>0.000044</td>\n",
       "      <td>2.554572</td>\n",
       "      <td>6.525839</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.123799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>42</td>\n",
       "      <td>(4856400, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>0.671454</td>\n",
       "      <td>0.000011</td>\n",
       "      <td>0.671454</td>\n",
       "      <td>0.450850</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.122158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>43</td>\n",
       "      <td>(4856401, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>40.938363</td>\n",
       "      <td>0.000697</td>\n",
       "      <td>40.938363</td>\n",
       "      <td>1675.949562</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.125157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>44</td>\n",
       "      <td>(4856402, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>26.556362</td>\n",
       "      <td>0.000452</td>\n",
       "      <td>26.556362</td>\n",
       "      <td>705.240375</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.333664</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>45</td>\n",
       "      <td>(4856403, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>28.090093</td>\n",
       "      <td>0.000479</td>\n",
       "      <td>28.090093</td>\n",
       "      <td>789.053340</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.082335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>46</td>\n",
       "      <td>(4856404, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>0.282511</td>\n",
       "      <td>0.000005</td>\n",
       "      <td>0.282511</td>\n",
       "      <td>0.079812</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.024898</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>47</td>\n",
       "      <td>(4856405, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>38.199571</td>\n",
       "      <td>0.000650</td>\n",
       "      <td>38.199571</td>\n",
       "      <td>1459.207261</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.086836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>48</td>\n",
       "      <td>(4856406, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>38.868192</td>\n",
       "      <td>0.000661</td>\n",
       "      <td>38.868192</td>\n",
       "      <td>1510.736331</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.742578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>49</td>\n",
       "      <td>(4856407, 1)</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>18.168039</td>\n",
       "      <td>0.000309</td>\n",
       "      <td>18.168039</td>\n",
       "      <td>330.077638</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.807346</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Model CV_type  Splits    Train&Test     Parameters       RMSE  \\\n",
       "0   LinearRegression    wfTs       1  (4856359, 1)  [5, 0.0, 0.8]   6.410132   \n",
       "1   LinearRegression    wfTs       2  (4856360, 1)  [5, 0.0, 0.8]   2.135334   \n",
       "2   LinearRegression    wfTs       3  (4856361, 1)  [5, 0.0, 0.8]  28.938096   \n",
       "3   LinearRegression    wfTs       4  (4856362, 1)  [5, 0.0, 0.8]  35.871947   \n",
       "4   LinearRegression    wfTs       5  (4856363, 1)  [5, 0.0, 0.8]  57.297181   \n",
       "5   LinearRegression    wfTs       6  (4856364, 1)  [5, 0.0, 0.8]  12.726607   \n",
       "6   LinearRegression    wfTs       7  (4856365, 1)  [5, 0.0, 0.8]   1.909822   \n",
       "7   LinearRegression    wfTs       8  (4856366, 1)  [5, 0.0, 0.8]   7.355302   \n",
       "8   LinearRegression    wfTs       9  (4856367, 1)  [5, 0.0, 0.8]   6.841876   \n",
       "9   LinearRegression    wfTs      10  (4856368, 1)  [5, 0.0, 0.8]  21.152901   \n",
       "10  LinearRegression    wfTs      11  (4856369, 1)  [5, 0.0, 0.8]  37.335059   \n",
       "11  LinearRegression    wfTs      12  (4856370, 1)  [5, 0.0, 0.8]  15.826054   \n",
       "12  LinearRegression    wfTs      13  (4856371, 1)  [5, 0.0, 0.8]  25.924992   \n",
       "13  LinearRegression    wfTs      14  (4856372, 1)  [5, 0.0, 0.8]  24.634869   \n",
       "14  LinearRegression    wfTs      15  (4856373, 1)  [5, 0.0, 0.8]  20.704843   \n",
       "15  LinearRegression    wfTs      16  (4856374, 1)  [5, 0.0, 0.8]  39.172257   \n",
       "16  LinearRegression    wfTs      17  (4856375, 1)  [5, 0.0, 0.8]   7.184019   \n",
       "17  LinearRegression    wfTs      18  (4856376, 1)  [5, 0.0, 0.8]  27.686442   \n",
       "18  LinearRegression    wfTs      19  (4856377, 1)  [5, 0.0, 0.8]   1.032495   \n",
       "19  LinearRegression    wfTs      20  (4856378, 1)  [5, 0.0, 0.8]   8.209809   \n",
       "20  LinearRegression    wfTs      21  (4856379, 1)  [5, 0.0, 0.8]  21.658920   \n",
       "21  LinearRegression    wfTs      22  (4856380, 1)  [5, 0.0, 0.8]  49.425394   \n",
       "22  LinearRegression    wfTs      23  (4856381, 1)  [5, 0.0, 0.8]  24.535962   \n",
       "23  LinearRegression    wfTs      24  (4856382, 1)  [5, 0.0, 0.8]  56.836666   \n",
       "24  LinearRegression    wfTs      25  (4856383, 1)  [5, 0.0, 0.8]  39.960240   \n",
       "25  LinearRegression    wfTs      26  (4856384, 1)  [5, 0.0, 0.8]  14.438882   \n",
       "26  LinearRegression    wfTs      27  (4856385, 1)  [5, 0.0, 0.8]  40.345751   \n",
       "27  LinearRegression    wfTs      28  (4856386, 1)  [5, 0.0, 0.8]  17.997994   \n",
       "28  LinearRegression    wfTs      29  (4856387, 1)  [5, 0.0, 0.8]   1.993140   \n",
       "29  LinearRegression    wfTs      30  (4856388, 1)  [5, 0.0, 0.8]   1.748922   \n",
       "30  LinearRegression    wfTs      31  (4856389, 1)  [5, 0.0, 0.8]   6.008873   \n",
       "31  LinearRegression    wfTs      32  (4856390, 1)  [5, 0.0, 0.8]  10.399502   \n",
       "32  LinearRegression    wfTs      33  (4856391, 1)  [5, 0.0, 0.8]   7.936927   \n",
       "33  LinearRegression    wfTs      34  (4856392, 1)  [5, 0.0, 0.8]   1.508807   \n",
       "34  LinearRegression    wfTs      35  (4856393, 1)  [5, 0.0, 0.8]  67.454859   \n",
       "35  LinearRegression    wfTs      36  (4856394, 1)  [5, 0.0, 0.8]  33.108827   \n",
       "36  LinearRegression    wfTs      37  (4856395, 1)  [5, 0.0, 0.8]  22.965371   \n",
       "37  LinearRegression    wfTs      38  (4856396, 1)  [5, 0.0, 0.8]  32.513714   \n",
       "38  LinearRegression    wfTs      39  (4856397, 1)  [5, 0.0, 0.8]  45.905748   \n",
       "39  LinearRegression    wfTs      40  (4856398, 1)  [5, 0.0, 0.8]  12.388048   \n",
       "40  LinearRegression    wfTs      41  (4856399, 1)  [5, 0.0, 0.8]   2.554572   \n",
       "41  LinearRegression    wfTs      42  (4856400, 1)  [5, 0.0, 0.8]   0.671454   \n",
       "42  LinearRegression    wfTs      43  (4856401, 1)  [5, 0.0, 0.8]  40.938363   \n",
       "43  LinearRegression    wfTs      44  (4856402, 1)  [5, 0.0, 0.8]  26.556362   \n",
       "44  LinearRegression    wfTs      45  (4856403, 1)  [5, 0.0, 0.8]  28.090093   \n",
       "45  LinearRegression    wfTs      46  (4856404, 1)  [5, 0.0, 0.8]   0.282511   \n",
       "46  LinearRegression    wfTs      47  (4856405, 1)  [5, 0.0, 0.8]  38.199571   \n",
       "47  LinearRegression    wfTs      48  (4856406, 1)  [5, 0.0, 0.8]  38.868192   \n",
       "48  LinearRegression    wfTs      49  (4856407, 1)  [5, 0.0, 0.8]  18.168039   \n",
       "\n",
       "        MAPE        MAE     Variance   R2  Adjusted_R2      Time  \n",
       "0   0.000109   6.410132    41.089788 -inf          NaN  7.713154  \n",
       "1   0.000036   2.135334     4.559651 -inf          NaN  7.805584  \n",
       "2   0.000493  28.938096   837.413416 -inf          NaN  7.877965  \n",
       "3   0.000611  35.871947  1286.796570 -inf          NaN  8.024721  \n",
       "4   0.000977  57.297181  3282.966920 -inf          NaN  8.112156  \n",
       "5   0.000217  12.726607   161.966535 -inf          NaN  7.866153  \n",
       "6   0.000033   1.909822     3.647420 -inf          NaN  8.654250  \n",
       "7   0.000125   7.355302    54.100463 -inf          NaN  8.010690  \n",
       "8   0.000117   6.841876    46.811273 -inf          NaN  7.833155  \n",
       "9   0.000361  21.152901   447.445234 -inf          NaN  7.956287  \n",
       "10  0.000637  37.335059  1393.906614 -inf          NaN  7.754557  \n",
       "11  0.000270  15.826054   250.463980 -inf          NaN  7.784153  \n",
       "12  0.000442  25.924992   672.105211 -inf          NaN  7.633149  \n",
       "13  0.000420  24.634869   606.876749 -inf          NaN  7.703892  \n",
       "14  0.000353  20.704843   428.690515 -inf          NaN  7.653950  \n",
       "15  0.000667  39.172257  1534.465710 -inf          NaN  7.998795  \n",
       "16  0.000122   7.184019    51.610127 -inf          NaN  7.758154  \n",
       "17  0.000471  27.686442   766.539073 -inf          NaN  8.017116  \n",
       "18  0.000018   1.032495     1.066046 -inf          NaN  7.623888  \n",
       "19  0.000140   8.209809    67.400956 -inf          NaN  9.101103  \n",
       "20  0.000369  21.658920   469.108828 -inf          NaN  8.032086  \n",
       "21  0.000842  49.425394  2442.869525 -inf          NaN  7.899149  \n",
       "22  0.000418  24.535962   602.013414 -inf          NaN  8.786549  \n",
       "23  0.000969  56.836666  3230.406651 -inf          NaN  8.552687  \n",
       "24  0.000682  39.960240  1596.820775 -inf          NaN  8.340163  \n",
       "25  0.000246  14.438882   208.481300 -inf          NaN  8.014159  \n",
       "26  0.000688  40.345751  1627.779589 -inf          NaN  7.785308  \n",
       "27  0.000307  17.997994   323.927787 -inf          NaN  7.744155  \n",
       "28  0.000034   1.993140     3.972607 -inf          NaN  7.897159  \n",
       "29  0.000030   1.748922     3.058725 -inf          NaN  7.948405  \n",
       "30  0.000103   6.008873    36.106550 -inf          NaN  7.899360  \n",
       "31  0.000177  10.399502   108.149632 -inf          NaN  7.722106  \n",
       "32  0.000135   7.936927    62.994808 -inf          NaN  7.814153  \n",
       "33  0.000026   1.508807     2.276498 -inf          NaN  8.127162  \n",
       "34  0.001150  67.454859  4550.157988 -inf          NaN  7.914156  \n",
       "35  0.000564  33.108827  1096.194448 -inf          NaN  8.123162  \n",
       "36  0.000391  22.965371   527.408280 -inf          NaN  7.769149  \n",
       "37  0.000554  32.513714  1057.141613 -inf          NaN  7.888173  \n",
       "38  0.000782  45.905748  2107.337681 -inf          NaN  7.952802  \n",
       "39  0.000211  12.388048   153.463731 -inf          NaN  8.082202  \n",
       "40  0.000044   2.554572     6.525839 -inf          NaN  8.123799  \n",
       "41  0.000011   0.671454     0.450850 -inf          NaN  8.122158  \n",
       "42  0.000697  40.938363  1675.949562 -inf          NaN  8.125157  \n",
       "43  0.000452  26.556362   705.240375 -inf          NaN  8.333664  \n",
       "44  0.000479  28.090093   789.053340 -inf          NaN  8.082335  \n",
       "45  0.000005   0.282511     0.079812 -inf          NaN  8.024898  \n",
       "46  0.000650  38.199571  1459.207261 -inf          NaN  8.086836  \n",
       "47  0.000661  38.868192  1510.736331 -inf          NaN  7.742578  \n",
       "48  0.000309  18.168039   330.077638 -inf          NaN  7.807346  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_wf_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"LinearRegression\", lr_params, vector_assembler ,standard_scaler, wf_cv)\n",
    "lr_wf_cv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "portable-michael",
   "metadata": {},
   "source": [
    "#### 5.4.2. Generalized linear regression Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "married-malaysia",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(809403, 809401)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>1.366073</td>\n",
       "      <td>0.001256</td>\n",
       "      <td>0.612516</td>\n",
       "      <td>5.248168e+04</td>\n",
       "      <td>0.999964</td>\n",
       "      <td>0.999964</td>\n",
       "      <td>3.211656</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(1618804, 809401)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>0.935522</td>\n",
       "      <td>0.001143</td>\n",
       "      <td>0.248898</td>\n",
       "      <td>1.863908e+04</td>\n",
       "      <td>0.999953</td>\n",
       "      <td>0.999953</td>\n",
       "      <td>4.256966</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(2428206, 809401)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>13.591950</td>\n",
       "      <td>0.000985</td>\n",
       "      <td>5.412886</td>\n",
       "      <td>1.896515e+07</td>\n",
       "      <td>0.999990</td>\n",
       "      <td>0.999990</td>\n",
       "      <td>4.251082</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(3237607, 809401)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>8.830537</td>\n",
       "      <td>0.000657</td>\n",
       "      <td>4.810164</td>\n",
       "      <td>5.626686e+06</td>\n",
       "      <td>0.999986</td>\n",
       "      <td>0.999986</td>\n",
       "      <td>4.835094</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(4047008, 809400)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>30.032420</td>\n",
       "      <td>0.000733</td>\n",
       "      <td>13.404001</td>\n",
       "      <td>1.858179e+08</td>\n",
       "      <td>0.999995</td>\n",
       "      <td>0.999995</td>\n",
       "      <td>4.872096</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         Model CV_type  Splits         Train&Test  \\\n",
       "0  GeneralizedLinearRegression   mulTs       1   (809403, 809401)   \n",
       "1  GeneralizedLinearRegression   mulTs       2  (1618804, 809401)   \n",
       "2  GeneralizedLinearRegression   mulTs       3  (2428206, 809401)   \n",
       "3  GeneralizedLinearRegression   mulTs       4  (3237607, 809401)   \n",
       "4  GeneralizedLinearRegression   mulTs       5  (4047008, 809400)   \n",
       "\n",
       "                    Parameters       RMSE      MAPE        MAE      Variance  \\\n",
       "0  [10, 0, gaussian, identity]   1.366073  0.001256   0.612516  5.248168e+04   \n",
       "1  [10, 0, gaussian, identity]   0.935522  0.001143   0.248898  1.863908e+04   \n",
       "2  [10, 0, gaussian, identity]  13.591950  0.000985   5.412886  1.896515e+07   \n",
       "3  [10, 0, gaussian, identity]   8.830537  0.000657   4.810164  5.626686e+06   \n",
       "4  [10, 0, gaussian, identity]  30.032420  0.000733  13.404001  1.858179e+08   \n",
       "\n",
       "         R2  Adjusted_R2      Time  \n",
       "0  0.999964     0.999964  3.211656  \n",
       "1  0.999953     0.999953  4.256966  \n",
       "2  0.999990     0.999990  4.251082  \n",
       "3  0.999986     0.999986  4.835094  \n",
       "4  0.999995     0.999995  4.872096  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generalized linear regression\n",
    "glr_params = {\n",
    "    'maxIter' : [10], # max number of iterations (>=0), default:25\n",
    "    'regParam' : [0],# regularization parameter (>=0), default:0.0\n",
    "    'family': ['gaussian'], # The name of family which is a description of the error distribution to be used in the model.\n",
    "    'link': ['identity'] # which provides the relationship between the linear predictor and the mean of the distribution function.\n",
    "}\n",
    "glr_mul_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"GeneralizedLinearRegression\", glr_params, vector_assembler ,standard_scaler, mul_cv)\n",
    "glr_mul_cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "naval-replication",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>0.010439</td>\n",
       "      <td>0.000255</td>\n",
       "      <td>0.002908</td>\n",
       "      <td>4.676590e-01</td>\n",
       "      <td>0.999767</td>\n",
       "      <td>0.999767</td>\n",
       "      <td>2.655176</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>0.329218</td>\n",
       "      <td>0.000937</td>\n",
       "      <td>0.139805</td>\n",
       "      <td>9.453162e+02</td>\n",
       "      <td>0.999885</td>\n",
       "      <td>0.999885</td>\n",
       "      <td>2.912059</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>0.767360</td>\n",
       "      <td>0.000973</td>\n",
       "      <td>0.441272</td>\n",
       "      <td>5.216484e+03</td>\n",
       "      <td>0.999887</td>\n",
       "      <td>0.999887</td>\n",
       "      <td>3.014059</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>0.260701</td>\n",
       "      <td>0.000577</td>\n",
       "      <td>0.146041</td>\n",
       "      <td>6.426075e+02</td>\n",
       "      <td>0.999894</td>\n",
       "      <td>0.999894</td>\n",
       "      <td>2.916116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>2.425555</td>\n",
       "      <td>0.004829</td>\n",
       "      <td>0.520071</td>\n",
       "      <td>2.306649e+03</td>\n",
       "      <td>0.997453</td>\n",
       "      <td>0.997452</td>\n",
       "      <td>2.719054</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>6</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>4.338398</td>\n",
       "      <td>0.001176</td>\n",
       "      <td>2.787380</td>\n",
       "      <td>8.450683e+04</td>\n",
       "      <td>0.999777</td>\n",
       "      <td>0.999777</td>\n",
       "      <td>2.672051</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>7</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>8.156550</td>\n",
       "      <td>0.000688</td>\n",
       "      <td>5.547771</td>\n",
       "      <td>9.696180e+05</td>\n",
       "      <td>0.999931</td>\n",
       "      <td>0.999931</td>\n",
       "      <td>2.715182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>8</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>7.433761</td>\n",
       "      <td>0.000621</td>\n",
       "      <td>3.821281</td>\n",
       "      <td>1.649642e+06</td>\n",
       "      <td>0.999966</td>\n",
       "      <td>0.999966</td>\n",
       "      <td>2.911060</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>9</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>11.280208</td>\n",
       "      <td>0.001000</td>\n",
       "      <td>6.802791</td>\n",
       "      <td>1.568764e+06</td>\n",
       "      <td>0.999919</td>\n",
       "      <td>0.999919</td>\n",
       "      <td>2.848171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>10</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>68.313149</td>\n",
       "      <td>0.001010</td>\n",
       "      <td>47.464780</td>\n",
       "      <td>7.598891e+07</td>\n",
       "      <td>0.999939</td>\n",
       "      <td>0.999939</td>\n",
       "      <td>2.944083</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         Model CV_type  Splits       Train&Test  \\\n",
       "0  GeneralizedLinearRegression   blkTs       1  (388512, 97128)   \n",
       "1  GeneralizedLinearRegression   blkTs       2  (388512, 97128)   \n",
       "2  GeneralizedLinearRegression   blkTs       3  (388512, 97128)   \n",
       "3  GeneralizedLinearRegression   blkTs       4  (388512, 97128)   \n",
       "4  GeneralizedLinearRegression   blkTs       5  (388512, 97128)   \n",
       "5  GeneralizedLinearRegression   blkTs       6  (388512, 97128)   \n",
       "6  GeneralizedLinearRegression   blkTs       7  (388512, 97128)   \n",
       "7  GeneralizedLinearRegression   blkTs       8  (388512, 97128)   \n",
       "8  GeneralizedLinearRegression   blkTs       9  (388512, 97128)   \n",
       "9  GeneralizedLinearRegression   blkTs      10  (388512, 97128)   \n",
       "\n",
       "                    Parameters       RMSE      MAPE        MAE      Variance  \\\n",
       "0  [10, 0, gaussian, identity]   0.010439  0.000255   0.002908  4.676590e-01   \n",
       "1  [10, 0, gaussian, identity]   0.329218  0.000937   0.139805  9.453162e+02   \n",
       "2  [10, 0, gaussian, identity]   0.767360  0.000973   0.441272  5.216484e+03   \n",
       "3  [10, 0, gaussian, identity]   0.260701  0.000577   0.146041  6.426075e+02   \n",
       "4  [10, 0, gaussian, identity]   2.425555  0.004829   0.520071  2.306649e+03   \n",
       "5  [10, 0, gaussian, identity]   4.338398  0.001176   2.787380  8.450683e+04   \n",
       "6  [10, 0, gaussian, identity]   8.156550  0.000688   5.547771  9.696180e+05   \n",
       "7  [10, 0, gaussian, identity]   7.433761  0.000621   3.821281  1.649642e+06   \n",
       "8  [10, 0, gaussian, identity]  11.280208  0.001000   6.802791  1.568764e+06   \n",
       "9  [10, 0, gaussian, identity]  68.313149  0.001010  47.464780  7.598891e+07   \n",
       "\n",
       "         R2  Adjusted_R2      Time  \n",
       "0  0.999767     0.999767  2.655176  \n",
       "1  0.999885     0.999885  2.912059  \n",
       "2  0.999887     0.999887  3.014059  \n",
       "3  0.999894     0.999894  2.916116  \n",
       "4  0.997453     0.997452  2.719054  \n",
       "5  0.999777     0.999777  2.672051  \n",
       "6  0.999931     0.999931  2.715182  \n",
       "7  0.999966     0.999966  2.911060  \n",
       "8  0.999919     0.999919  2.848171  \n",
       "9  0.999939     0.999939  2.944083  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glr_blk_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"GeneralizedLinearRegression\", glr_params, vector_assembler ,standard_scaler, blk_cv)\n",
    "glr_blk_cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "dress-material",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(4856359, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>6.410132</td>\n",
       "      <td>0.000109</td>\n",
       "      <td>6.410132</td>\n",
       "      <td>41.089788</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.072119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(4856360, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>2.135334</td>\n",
       "      <td>0.000036</td>\n",
       "      <td>2.135334</td>\n",
       "      <td>4.559650</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.927115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(4856361, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>28.938096</td>\n",
       "      <td>0.000493</td>\n",
       "      <td>28.938096</td>\n",
       "      <td>837.413416</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.173125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(4856362, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>35.871947</td>\n",
       "      <td>0.000611</td>\n",
       "      <td>35.871947</td>\n",
       "      <td>1286.796569</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.451129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(4856363, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>57.297181</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>57.297181</td>\n",
       "      <td>3282.966922</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.226120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>6</td>\n",
       "      <td>(4856364, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>12.726607</td>\n",
       "      <td>0.000217</td>\n",
       "      <td>12.726607</td>\n",
       "      <td>161.966537</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.313744</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>7</td>\n",
       "      <td>(4856365, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>1.909822</td>\n",
       "      <td>0.000033</td>\n",
       "      <td>1.909822</td>\n",
       "      <td>3.647420</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.240277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>8</td>\n",
       "      <td>(4856366, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>7.355302</td>\n",
       "      <td>0.000125</td>\n",
       "      <td>7.355302</td>\n",
       "      <td>54.100463</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.173120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>9</td>\n",
       "      <td>(4856367, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>6.841876</td>\n",
       "      <td>0.000117</td>\n",
       "      <td>6.841876</td>\n",
       "      <td>46.811273</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.924188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>10</td>\n",
       "      <td>(4856368, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>21.152901</td>\n",
       "      <td>0.000361</td>\n",
       "      <td>21.152901</td>\n",
       "      <td>447.445231</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.179151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>11</td>\n",
       "      <td>(4856369, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>37.335059</td>\n",
       "      <td>0.000637</td>\n",
       "      <td>37.335059</td>\n",
       "      <td>1393.906614</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.925171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>12</td>\n",
       "      <td>(4856370, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>15.826054</td>\n",
       "      <td>0.000270</td>\n",
       "      <td>15.826054</td>\n",
       "      <td>250.463980</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.861115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>13</td>\n",
       "      <td>(4856371, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>25.924992</td>\n",
       "      <td>0.000442</td>\n",
       "      <td>25.924992</td>\n",
       "      <td>672.105210</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.990122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>14</td>\n",
       "      <td>(4856372, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>24.634869</td>\n",
       "      <td>0.000420</td>\n",
       "      <td>24.634869</td>\n",
       "      <td>606.876748</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.162174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>15</td>\n",
       "      <td>(4856373, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>20.704843</td>\n",
       "      <td>0.000353</td>\n",
       "      <td>20.704843</td>\n",
       "      <td>428.690514</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.151120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>16</td>\n",
       "      <td>(4856374, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>39.172257</td>\n",
       "      <td>0.000667</td>\n",
       "      <td>39.172257</td>\n",
       "      <td>1534.465710</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.100572</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>17</td>\n",
       "      <td>(4856375, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>7.184019</td>\n",
       "      <td>0.000122</td>\n",
       "      <td>7.184019</td>\n",
       "      <td>51.610127</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.367123</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>18</td>\n",
       "      <td>(4856376, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>27.686442</td>\n",
       "      <td>0.000471</td>\n",
       "      <td>27.686442</td>\n",
       "      <td>766.539073</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.960170</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>19</td>\n",
       "      <td>(4856377, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>1.032495</td>\n",
       "      <td>0.000018</td>\n",
       "      <td>1.032495</td>\n",
       "      <td>1.066046</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.978186</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>20</td>\n",
       "      <td>(4856378, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>8.209809</td>\n",
       "      <td>0.000140</td>\n",
       "      <td>8.209809</td>\n",
       "      <td>67.400956</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.362132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>21</td>\n",
       "      <td>(4856379, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>21.658920</td>\n",
       "      <td>0.000369</td>\n",
       "      <td>21.658920</td>\n",
       "      <td>469.108829</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.325189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>22</td>\n",
       "      <td>(4856380, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>49.425394</td>\n",
       "      <td>0.000842</td>\n",
       "      <td>49.425394</td>\n",
       "      <td>2442.869526</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.972475</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>23</td>\n",
       "      <td>(4856381, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>24.535962</td>\n",
       "      <td>0.000418</td>\n",
       "      <td>24.535962</td>\n",
       "      <td>602.013416</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.337849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>24</td>\n",
       "      <td>(4856382, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>56.836666</td>\n",
       "      <td>0.000969</td>\n",
       "      <td>56.836666</td>\n",
       "      <td>3230.406651</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.519900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>25</td>\n",
       "      <td>(4856383, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>39.960240</td>\n",
       "      <td>0.000682</td>\n",
       "      <td>39.960240</td>\n",
       "      <td>1596.820775</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.163129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>26</td>\n",
       "      <td>(4856384, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>14.438882</td>\n",
       "      <td>0.000246</td>\n",
       "      <td>14.438882</td>\n",
       "      <td>208.481300</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.653253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>27</td>\n",
       "      <td>(4856385, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>40.345751</td>\n",
       "      <td>0.000688</td>\n",
       "      <td>40.345751</td>\n",
       "      <td>1627.779591</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.710096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>28</td>\n",
       "      <td>(4856386, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>17.997994</td>\n",
       "      <td>0.000307</td>\n",
       "      <td>17.997994</td>\n",
       "      <td>323.927787</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.628933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>29</td>\n",
       "      <td>(4856387, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>1.993140</td>\n",
       "      <td>0.000034</td>\n",
       "      <td>1.993140</td>\n",
       "      <td>3.972606</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.557732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>30</td>\n",
       "      <td>(4856388, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>1.748922</td>\n",
       "      <td>0.000030</td>\n",
       "      <td>1.748922</td>\n",
       "      <td>3.058726</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.214162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>31</td>\n",
       "      <td>(4856389, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>6.008873</td>\n",
       "      <td>0.000103</td>\n",
       "      <td>6.008873</td>\n",
       "      <td>36.106550</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.364576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>32</td>\n",
       "      <td>(4856390, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>10.399502</td>\n",
       "      <td>0.000177</td>\n",
       "      <td>10.399502</td>\n",
       "      <td>108.149634</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.287099</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>33</td>\n",
       "      <td>(4856391, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>7.936927</td>\n",
       "      <td>0.000135</td>\n",
       "      <td>7.936927</td>\n",
       "      <td>62.994807</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.946145</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>34</td>\n",
       "      <td>(4856392, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>1.508807</td>\n",
       "      <td>0.000026</td>\n",
       "      <td>1.508807</td>\n",
       "      <td>2.276498</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.120160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>35</td>\n",
       "      <td>(4856393, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>67.454859</td>\n",
       "      <td>0.001150</td>\n",
       "      <td>67.454859</td>\n",
       "      <td>4550.157985</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.097341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>36</td>\n",
       "      <td>(4856394, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>33.108827</td>\n",
       "      <td>0.000564</td>\n",
       "      <td>33.108827</td>\n",
       "      <td>1096.194448</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.996175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>37</td>\n",
       "      <td>(4856395, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>22.965371</td>\n",
       "      <td>0.000391</td>\n",
       "      <td>22.965371</td>\n",
       "      <td>527.408280</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.298218</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>38</td>\n",
       "      <td>(4856396, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>32.513714</td>\n",
       "      <td>0.000554</td>\n",
       "      <td>32.513714</td>\n",
       "      <td>1057.141615</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.139663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>39</td>\n",
       "      <td>(4856397, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>45.905748</td>\n",
       "      <td>0.000782</td>\n",
       "      <td>45.905748</td>\n",
       "      <td>2107.337683</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.373858</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>40</td>\n",
       "      <td>(4856398, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>12.388048</td>\n",
       "      <td>0.000211</td>\n",
       "      <td>12.388048</td>\n",
       "      <td>153.463730</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.284315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>41</td>\n",
       "      <td>(4856399, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>2.554572</td>\n",
       "      <td>0.000044</td>\n",
       "      <td>2.554572</td>\n",
       "      <td>6.525839</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.033320</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>42</td>\n",
       "      <td>(4856400, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>0.671454</td>\n",
       "      <td>0.000011</td>\n",
       "      <td>0.671454</td>\n",
       "      <td>0.450850</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.207150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>43</td>\n",
       "      <td>(4856401, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>40.938363</td>\n",
       "      <td>0.000697</td>\n",
       "      <td>40.938363</td>\n",
       "      <td>1675.949569</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.965341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>44</td>\n",
       "      <td>(4856402, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>26.556362</td>\n",
       "      <td>0.000452</td>\n",
       "      <td>26.556362</td>\n",
       "      <td>705.240375</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.990914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>45</td>\n",
       "      <td>(4856403, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>28.090093</td>\n",
       "      <td>0.000479</td>\n",
       "      <td>28.090093</td>\n",
       "      <td>789.053341</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.073121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>46</td>\n",
       "      <td>(4856404, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>0.282511</td>\n",
       "      <td>0.000005</td>\n",
       "      <td>0.282511</td>\n",
       "      <td>0.079812</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.184126</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>47</td>\n",
       "      <td>(4856405, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>38.199571</td>\n",
       "      <td>0.000650</td>\n",
       "      <td>38.199571</td>\n",
       "      <td>1459.207261</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.424275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>48</td>\n",
       "      <td>(4856406, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>38.868192</td>\n",
       "      <td>0.000661</td>\n",
       "      <td>38.868192</td>\n",
       "      <td>1510.736332</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.121252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>49</td>\n",
       "      <td>(4856407, 1)</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>18.168039</td>\n",
       "      <td>0.000309</td>\n",
       "      <td>18.168039</td>\n",
       "      <td>330.077639</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.212124</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          Model CV_type  Splits    Train&Test  \\\n",
       "0   GeneralizedLinearRegression    wfTs       1  (4856359, 1)   \n",
       "1   GeneralizedLinearRegression    wfTs       2  (4856360, 1)   \n",
       "2   GeneralizedLinearRegression    wfTs       3  (4856361, 1)   \n",
       "3   GeneralizedLinearRegression    wfTs       4  (4856362, 1)   \n",
       "4   GeneralizedLinearRegression    wfTs       5  (4856363, 1)   \n",
       "5   GeneralizedLinearRegression    wfTs       6  (4856364, 1)   \n",
       "6   GeneralizedLinearRegression    wfTs       7  (4856365, 1)   \n",
       "7   GeneralizedLinearRegression    wfTs       8  (4856366, 1)   \n",
       "8   GeneralizedLinearRegression    wfTs       9  (4856367, 1)   \n",
       "9   GeneralizedLinearRegression    wfTs      10  (4856368, 1)   \n",
       "10  GeneralizedLinearRegression    wfTs      11  (4856369, 1)   \n",
       "11  GeneralizedLinearRegression    wfTs      12  (4856370, 1)   \n",
       "12  GeneralizedLinearRegression    wfTs      13  (4856371, 1)   \n",
       "13  GeneralizedLinearRegression    wfTs      14  (4856372, 1)   \n",
       "14  GeneralizedLinearRegression    wfTs      15  (4856373, 1)   \n",
       "15  GeneralizedLinearRegression    wfTs      16  (4856374, 1)   \n",
       "16  GeneralizedLinearRegression    wfTs      17  (4856375, 1)   \n",
       "17  GeneralizedLinearRegression    wfTs      18  (4856376, 1)   \n",
       "18  GeneralizedLinearRegression    wfTs      19  (4856377, 1)   \n",
       "19  GeneralizedLinearRegression    wfTs      20  (4856378, 1)   \n",
       "20  GeneralizedLinearRegression    wfTs      21  (4856379, 1)   \n",
       "21  GeneralizedLinearRegression    wfTs      22  (4856380, 1)   \n",
       "22  GeneralizedLinearRegression    wfTs      23  (4856381, 1)   \n",
       "23  GeneralizedLinearRegression    wfTs      24  (4856382, 1)   \n",
       "24  GeneralizedLinearRegression    wfTs      25  (4856383, 1)   \n",
       "25  GeneralizedLinearRegression    wfTs      26  (4856384, 1)   \n",
       "26  GeneralizedLinearRegression    wfTs      27  (4856385, 1)   \n",
       "27  GeneralizedLinearRegression    wfTs      28  (4856386, 1)   \n",
       "28  GeneralizedLinearRegression    wfTs      29  (4856387, 1)   \n",
       "29  GeneralizedLinearRegression    wfTs      30  (4856388, 1)   \n",
       "30  GeneralizedLinearRegression    wfTs      31  (4856389, 1)   \n",
       "31  GeneralizedLinearRegression    wfTs      32  (4856390, 1)   \n",
       "32  GeneralizedLinearRegression    wfTs      33  (4856391, 1)   \n",
       "33  GeneralizedLinearRegression    wfTs      34  (4856392, 1)   \n",
       "34  GeneralizedLinearRegression    wfTs      35  (4856393, 1)   \n",
       "35  GeneralizedLinearRegression    wfTs      36  (4856394, 1)   \n",
       "36  GeneralizedLinearRegression    wfTs      37  (4856395, 1)   \n",
       "37  GeneralizedLinearRegression    wfTs      38  (4856396, 1)   \n",
       "38  GeneralizedLinearRegression    wfTs      39  (4856397, 1)   \n",
       "39  GeneralizedLinearRegression    wfTs      40  (4856398, 1)   \n",
       "40  GeneralizedLinearRegression    wfTs      41  (4856399, 1)   \n",
       "41  GeneralizedLinearRegression    wfTs      42  (4856400, 1)   \n",
       "42  GeneralizedLinearRegression    wfTs      43  (4856401, 1)   \n",
       "43  GeneralizedLinearRegression    wfTs      44  (4856402, 1)   \n",
       "44  GeneralizedLinearRegression    wfTs      45  (4856403, 1)   \n",
       "45  GeneralizedLinearRegression    wfTs      46  (4856404, 1)   \n",
       "46  GeneralizedLinearRegression    wfTs      47  (4856405, 1)   \n",
       "47  GeneralizedLinearRegression    wfTs      48  (4856406, 1)   \n",
       "48  GeneralizedLinearRegression    wfTs      49  (4856407, 1)   \n",
       "\n",
       "                     Parameters       RMSE      MAPE        MAE     Variance  \\\n",
       "0   [10, 0, gaussian, identity]   6.410132  0.000109   6.410132    41.089788   \n",
       "1   [10, 0, gaussian, identity]   2.135334  0.000036   2.135334     4.559650   \n",
       "2   [10, 0, gaussian, identity]  28.938096  0.000493  28.938096   837.413416   \n",
       "3   [10, 0, gaussian, identity]  35.871947  0.000611  35.871947  1286.796569   \n",
       "4   [10, 0, gaussian, identity]  57.297181  0.000977  57.297181  3282.966922   \n",
       "5   [10, 0, gaussian, identity]  12.726607  0.000217  12.726607   161.966537   \n",
       "6   [10, 0, gaussian, identity]   1.909822  0.000033   1.909822     3.647420   \n",
       "7   [10, 0, gaussian, identity]   7.355302  0.000125   7.355302    54.100463   \n",
       "8   [10, 0, gaussian, identity]   6.841876  0.000117   6.841876    46.811273   \n",
       "9   [10, 0, gaussian, identity]  21.152901  0.000361  21.152901   447.445231   \n",
       "10  [10, 0, gaussian, identity]  37.335059  0.000637  37.335059  1393.906614   \n",
       "11  [10, 0, gaussian, identity]  15.826054  0.000270  15.826054   250.463980   \n",
       "12  [10, 0, gaussian, identity]  25.924992  0.000442  25.924992   672.105210   \n",
       "13  [10, 0, gaussian, identity]  24.634869  0.000420  24.634869   606.876748   \n",
       "14  [10, 0, gaussian, identity]  20.704843  0.000353  20.704843   428.690514   \n",
       "15  [10, 0, gaussian, identity]  39.172257  0.000667  39.172257  1534.465710   \n",
       "16  [10, 0, gaussian, identity]   7.184019  0.000122   7.184019    51.610127   \n",
       "17  [10, 0, gaussian, identity]  27.686442  0.000471  27.686442   766.539073   \n",
       "18  [10, 0, gaussian, identity]   1.032495  0.000018   1.032495     1.066046   \n",
       "19  [10, 0, gaussian, identity]   8.209809  0.000140   8.209809    67.400956   \n",
       "20  [10, 0, gaussian, identity]  21.658920  0.000369  21.658920   469.108829   \n",
       "21  [10, 0, gaussian, identity]  49.425394  0.000842  49.425394  2442.869526   \n",
       "22  [10, 0, gaussian, identity]  24.535962  0.000418  24.535962   602.013416   \n",
       "23  [10, 0, gaussian, identity]  56.836666  0.000969  56.836666  3230.406651   \n",
       "24  [10, 0, gaussian, identity]  39.960240  0.000682  39.960240  1596.820775   \n",
       "25  [10, 0, gaussian, identity]  14.438882  0.000246  14.438882   208.481300   \n",
       "26  [10, 0, gaussian, identity]  40.345751  0.000688  40.345751  1627.779591   \n",
       "27  [10, 0, gaussian, identity]  17.997994  0.000307  17.997994   323.927787   \n",
       "28  [10, 0, gaussian, identity]   1.993140  0.000034   1.993140     3.972606   \n",
       "29  [10, 0, gaussian, identity]   1.748922  0.000030   1.748922     3.058726   \n",
       "30  [10, 0, gaussian, identity]   6.008873  0.000103   6.008873    36.106550   \n",
       "31  [10, 0, gaussian, identity]  10.399502  0.000177  10.399502   108.149634   \n",
       "32  [10, 0, gaussian, identity]   7.936927  0.000135   7.936927    62.994807   \n",
       "33  [10, 0, gaussian, identity]   1.508807  0.000026   1.508807     2.276498   \n",
       "34  [10, 0, gaussian, identity]  67.454859  0.001150  67.454859  4550.157985   \n",
       "35  [10, 0, gaussian, identity]  33.108827  0.000564  33.108827  1096.194448   \n",
       "36  [10, 0, gaussian, identity]  22.965371  0.000391  22.965371   527.408280   \n",
       "37  [10, 0, gaussian, identity]  32.513714  0.000554  32.513714  1057.141615   \n",
       "38  [10, 0, gaussian, identity]  45.905748  0.000782  45.905748  2107.337683   \n",
       "39  [10, 0, gaussian, identity]  12.388048  0.000211  12.388048   153.463730   \n",
       "40  [10, 0, gaussian, identity]   2.554572  0.000044   2.554572     6.525839   \n",
       "41  [10, 0, gaussian, identity]   0.671454  0.000011   0.671454     0.450850   \n",
       "42  [10, 0, gaussian, identity]  40.938363  0.000697  40.938363  1675.949569   \n",
       "43  [10, 0, gaussian, identity]  26.556362  0.000452  26.556362   705.240375   \n",
       "44  [10, 0, gaussian, identity]  28.090093  0.000479  28.090093   789.053341   \n",
       "45  [10, 0, gaussian, identity]   0.282511  0.000005   0.282511     0.079812   \n",
       "46  [10, 0, gaussian, identity]  38.199571  0.000650  38.199571  1459.207261   \n",
       "47  [10, 0, gaussian, identity]  38.868192  0.000661  38.868192  1510.736332   \n",
       "48  [10, 0, gaussian, identity]  18.168039  0.000309  18.168039   330.077639   \n",
       "\n",
       "     R2  Adjusted_R2      Time  \n",
       "0  -inf          NaN  6.072119  \n",
       "1  -inf          NaN  5.927115  \n",
       "2  -inf          NaN  6.173125  \n",
       "3  -inf          NaN  6.451129  \n",
       "4  -inf          NaN  6.226120  \n",
       "5  -inf          NaN  6.313744  \n",
       "6  -inf          NaN  6.240277  \n",
       "7  -inf          NaN  6.173120  \n",
       "8  -inf          NaN  5.924188  \n",
       "9  -inf          NaN  6.179151  \n",
       "10 -inf          NaN  5.925171  \n",
       "11 -inf          NaN  5.861115  \n",
       "12 -inf          NaN  5.990122  \n",
       "13 -inf          NaN  6.162174  \n",
       "14 -inf          NaN  6.151120  \n",
       "15 -inf          NaN  6.100572  \n",
       "16 -inf          NaN  6.367123  \n",
       "17 -inf          NaN  5.960170  \n",
       "18 -inf          NaN  5.978186  \n",
       "19 -inf          NaN  6.362132  \n",
       "20 -inf          NaN  6.325189  \n",
       "21 -inf          NaN  5.972475  \n",
       "22 -inf          NaN  6.337849  \n",
       "23 -inf          NaN  6.519900  \n",
       "24 -inf          NaN  6.163129  \n",
       "25 -inf          NaN  6.653253  \n",
       "26 -inf          NaN  6.710096  \n",
       "27 -inf          NaN  6.628933  \n",
       "28 -inf          NaN  6.557732  \n",
       "29 -inf          NaN  6.214162  \n",
       "30 -inf          NaN  7.364576  \n",
       "31 -inf          NaN  6.287099  \n",
       "32 -inf          NaN  5.946145  \n",
       "33 -inf          NaN  6.120160  \n",
       "34 -inf          NaN  6.097341  \n",
       "35 -inf          NaN  5.996175  \n",
       "36 -inf          NaN  6.298218  \n",
       "37 -inf          NaN  6.139663  \n",
       "38 -inf          NaN  6.373858  \n",
       "39 -inf          NaN  6.284315  \n",
       "40 -inf          NaN  6.033320  \n",
       "41 -inf          NaN  6.207150  \n",
       "42 -inf          NaN  5.965341  \n",
       "43 -inf          NaN  5.990914  \n",
       "44 -inf          NaN  6.073121  \n",
       "45 -inf          NaN  6.184126  \n",
       "46 -inf          NaN  6.424275  \n",
       "47 -inf          NaN  6.121252  \n",
       "48 -inf          NaN  6.212124  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glr_wf_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"GeneralizedLinearRegression\", glr_params, vector_assembler ,standard_scaler, wf_cv)\n",
    "glr_wf_cv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "beautiful-johnston",
   "metadata": {},
   "source": [
    "#### 5.4.3. RandomForest Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "heated-remark",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(809403, 809401)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>351.515235</td>\n",
       "      <td>0.504572</td>\n",
       "      <td>281.271832</td>\n",
       "      <td>7.989460e+04</td>\n",
       "      <td>-1.354177</td>\n",
       "      <td>-1.354211</td>\n",
       "      <td>10.700240</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(1618804, 809401)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>17.682341</td>\n",
       "      <td>0.041240</td>\n",
       "      <td>12.998234</td>\n",
       "      <td>1.994027e+04</td>\n",
       "      <td>0.983227</td>\n",
       "      <td>0.983226</td>\n",
       "      <td>12.395513</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(2428206, 809401)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>5258.263045</td>\n",
       "      <td>0.459848</td>\n",
       "      <td>3061.772203</td>\n",
       "      <td>9.371884e+06</td>\n",
       "      <td>-0.458168</td>\n",
       "      <td>-0.458189</td>\n",
       "      <td>15.274308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(3237607, 809401)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>1104.861100</td>\n",
       "      <td>0.119616</td>\n",
       "      <td>843.623513</td>\n",
       "      <td>6.615315e+06</td>\n",
       "      <td>0.783033</td>\n",
       "      <td>0.783029</td>\n",
       "      <td>20.716533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(4047008, 809400)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10989.855879</td>\n",
       "      <td>0.139607</td>\n",
       "      <td>4745.627951</td>\n",
       "      <td>3.764583e+07</td>\n",
       "      <td>0.350035</td>\n",
       "      <td>0.350025</td>\n",
       "      <td>23.449244</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Model CV_type  Splits         Train&Test Parameters          RMSE  \\\n",
       "0  RandomForest   mulTs       1   (809403, 809401)    [5, 10]    351.515235   \n",
       "1  RandomForest   mulTs       2  (1618804, 809401)    [5, 10]     17.682341   \n",
       "2  RandomForest   mulTs       3  (2428206, 809401)    [5, 10]   5258.263045   \n",
       "3  RandomForest   mulTs       4  (3237607, 809401)    [5, 10]   1104.861100   \n",
       "4  RandomForest   mulTs       5  (4047008, 809400)    [5, 10]  10989.855879   \n",
       "\n",
       "       MAPE          MAE      Variance        R2  Adjusted_R2       Time  \n",
       "0  0.504572   281.271832  7.989460e+04 -1.354177    -1.354211  10.700240  \n",
       "1  0.041240    12.998234  1.994027e+04  0.983227     0.983226  12.395513  \n",
       "2  0.459848  3061.772203  9.371884e+06 -0.458168    -0.458189  15.274308  \n",
       "3  0.119616   843.623513  6.615315e+06  0.783033     0.783029  20.716533  \n",
       "4  0.139607  4745.627951  3.764583e+07  0.350035     0.350025  23.449244  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_params = {\n",
    "    'numTrees' : [5],# Number of trees to train, >=1, default:20\n",
    "    'maxDepth' : [10] # Maximum depth of the tree, <=30, default:5\n",
    "}\n",
    "rf_mul_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"RandomForest\", rf_params, vector_assembler ,standard_scaler, mul_cv)\n",
    "rf_mul_cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "official-conducting",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>0.245460</td>\n",
       "      <td>0.015894</td>\n",
       "      <td>0.187042</td>\n",
       "      <td>6.553693e-01</td>\n",
       "      <td>0.871169</td>\n",
       "      <td>0.871153</td>\n",
       "      <td>8.148999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12.881155</td>\n",
       "      <td>0.039150</td>\n",
       "      <td>6.914689</td>\n",
       "      <td>5.521326e+02</td>\n",
       "      <td>0.824489</td>\n",
       "      <td>0.824467</td>\n",
       "      <td>8.678176</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>13.150895</td>\n",
       "      <td>0.021935</td>\n",
       "      <td>9.041445</td>\n",
       "      <td>5.671155e+03</td>\n",
       "      <td>0.966855</td>\n",
       "      <td>0.966851</td>\n",
       "      <td>9.991845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>3.576121</td>\n",
       "      <td>0.008668</td>\n",
       "      <td>2.388849</td>\n",
       "      <td>7.033641e+02</td>\n",
       "      <td>0.980104</td>\n",
       "      <td>0.980102</td>\n",
       "      <td>9.457256</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>74.163257</td>\n",
       "      <td>0.094393</td>\n",
       "      <td>61.015094</td>\n",
       "      <td>3.694015e+03</td>\n",
       "      <td>-1.381557</td>\n",
       "      <td>-1.381852</td>\n",
       "      <td>8.710910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>6</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>633.485863</td>\n",
       "      <td>0.226869</td>\n",
       "      <td>573.634702</td>\n",
       "      <td>3.307825e+05</td>\n",
       "      <td>-3.748733</td>\n",
       "      <td>-3.749320</td>\n",
       "      <td>8.489157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>7</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>127.021944</td>\n",
       "      <td>0.013337</td>\n",
       "      <td>104.172232</td>\n",
       "      <td>1.025163e+06</td>\n",
       "      <td>0.983359</td>\n",
       "      <td>0.983357</td>\n",
       "      <td>8.851895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>8</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>333.969051</td>\n",
       "      <td>0.047057</td>\n",
       "      <td>258.135086</td>\n",
       "      <td>1.775488e+06</td>\n",
       "      <td>0.932385</td>\n",
       "      <td>0.932377</td>\n",
       "      <td>9.140229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>9</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>592.967194</td>\n",
       "      <td>0.055913</td>\n",
       "      <td>325.248486</td>\n",
       "      <td>9.888972e+05</td>\n",
       "      <td>0.775863</td>\n",
       "      <td>0.775835</td>\n",
       "      <td>9.684478</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>10</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12659.077342</td>\n",
       "      <td>0.203978</td>\n",
       "      <td>10727.303241</td>\n",
       "      <td>1.147586e+08</td>\n",
       "      <td>-1.108708</td>\n",
       "      <td>-1.108969</td>\n",
       "      <td>9.309010</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Model CV_type  Splits       Train&Test Parameters          RMSE  \\\n",
       "0  RandomForest   blkTs       1  (388512, 97128)    [5, 10]      0.245460   \n",
       "1  RandomForest   blkTs       2  (388512, 97128)    [5, 10]     12.881155   \n",
       "2  RandomForest   blkTs       3  (388512, 97128)    [5, 10]     13.150895   \n",
       "3  RandomForest   blkTs       4  (388512, 97128)    [5, 10]      3.576121   \n",
       "4  RandomForest   blkTs       5  (388512, 97128)    [5, 10]     74.163257   \n",
       "5  RandomForest   blkTs       6  (388512, 97128)    [5, 10]    633.485863   \n",
       "6  RandomForest   blkTs       7  (388512, 97128)    [5, 10]    127.021944   \n",
       "7  RandomForest   blkTs       8  (388512, 97128)    [5, 10]    333.969051   \n",
       "8  RandomForest   blkTs       9  (388512, 97128)    [5, 10]    592.967194   \n",
       "9  RandomForest   blkTs      10  (388512, 97128)    [5, 10]  12659.077342   \n",
       "\n",
       "       MAPE           MAE      Variance        R2  Adjusted_R2      Time  \n",
       "0  0.015894      0.187042  6.553693e-01  0.871169     0.871153  8.148999  \n",
       "1  0.039150      6.914689  5.521326e+02  0.824489     0.824467  8.678176  \n",
       "2  0.021935      9.041445  5.671155e+03  0.966855     0.966851  9.991845  \n",
       "3  0.008668      2.388849  7.033641e+02  0.980104     0.980102  9.457256  \n",
       "4  0.094393     61.015094  3.694015e+03 -1.381557    -1.381852  8.710910  \n",
       "5  0.226869    573.634702  3.307825e+05 -3.748733    -3.749320  8.489157  \n",
       "6  0.013337    104.172232  1.025163e+06  0.983359     0.983357  8.851895  \n",
       "7  0.047057    258.135086  1.775488e+06  0.932385     0.932377  9.140229  \n",
       "8  0.055913    325.248486  9.888972e+05  0.775863     0.775835  9.684478  \n",
       "9  0.203978  10727.303241  1.147586e+08 -1.108708    -1.108969  9.309010  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_blk_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"RandomForest\", rf_params, vector_assembler ,standard_scaler, blk_cv)\n",
    "rf_blk_cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "religious-scene",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(4856359, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10309.198323</td>\n",
       "      <td>0.175588</td>\n",
       "      <td>10309.198323</td>\n",
       "      <td>1.062796e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25.597913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(4856360, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10090.824991</td>\n",
       "      <td>0.171891</td>\n",
       "      <td>10090.824991</td>\n",
       "      <td>1.018247e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25.329065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(4856361, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10962.858509</td>\n",
       "      <td>0.186654</td>\n",
       "      <td>10962.858509</td>\n",
       "      <td>1.201843e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24.511330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(4856362, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11895.726341</td>\n",
       "      <td>0.202668</td>\n",
       "      <td>11895.726341</td>\n",
       "      <td>1.415083e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.054279</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(4856363, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11494.661753</td>\n",
       "      <td>0.196033</td>\n",
       "      <td>11494.661753</td>\n",
       "      <td>1.321272e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.610012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>6</td>\n",
       "      <td>(4856364, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>9987.145637</td>\n",
       "      <td>0.170365</td>\n",
       "      <td>9987.145637</td>\n",
       "      <td>9.974308e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.959577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>7</td>\n",
       "      <td>(4856365, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10758.576541</td>\n",
       "      <td>0.183533</td>\n",
       "      <td>10758.576541</td>\n",
       "      <td>1.157470e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27.444233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>8</td>\n",
       "      <td>(4856366, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11738.851146</td>\n",
       "      <td>0.200235</td>\n",
       "      <td>11738.851146</td>\n",
       "      <td>1.378006e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27.178553</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>9</td>\n",
       "      <td>(4856367, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11012.023003</td>\n",
       "      <td>0.187817</td>\n",
       "      <td>11012.023003</td>\n",
       "      <td>1.212647e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.042655</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>10</td>\n",
       "      <td>(4856368, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>8011.526013</td>\n",
       "      <td>0.136583</td>\n",
       "      <td>8011.526013</td>\n",
       "      <td>6.418455e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27.845516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>11</td>\n",
       "      <td>(4856369, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10197.592505</td>\n",
       "      <td>0.173969</td>\n",
       "      <td>10197.592505</td>\n",
       "      <td>1.039909e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25.940131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>12</td>\n",
       "      <td>(4856370, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11899.411984</td>\n",
       "      <td>0.202953</td>\n",
       "      <td>11899.411984</td>\n",
       "      <td>1.415960e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.862306</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>13</td>\n",
       "      <td>(4856371, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10234.334052</td>\n",
       "      <td>0.174491</td>\n",
       "      <td>10234.334052</td>\n",
       "      <td>1.047416e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.101527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>14</td>\n",
       "      <td>(4856372, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11439.382599</td>\n",
       "      <td>0.194966</td>\n",
       "      <td>11439.382599</td>\n",
       "      <td>1.308595e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.505017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>15</td>\n",
       "      <td>(4856373, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10300.272442</td>\n",
       "      <td>0.175496</td>\n",
       "      <td>10300.272442</td>\n",
       "      <td>1.060956e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.212100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>16</td>\n",
       "      <td>(4856374, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10281.758708</td>\n",
       "      <td>0.175065</td>\n",
       "      <td>10281.758708</td>\n",
       "      <td>1.057146e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.138532</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>17</td>\n",
       "      <td>(4856375, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>9367.081357</td>\n",
       "      <td>0.159517</td>\n",
       "      <td>9367.081357</td>\n",
       "      <td>8.774221e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27.350984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>18</td>\n",
       "      <td>(4856376, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10418.865048</td>\n",
       "      <td>0.177347</td>\n",
       "      <td>10418.865048</td>\n",
       "      <td>1.085527e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.562823</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>19</td>\n",
       "      <td>(4856377, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>8461.149006</td>\n",
       "      <td>0.144032</td>\n",
       "      <td>8461.149006</td>\n",
       "      <td>7.159104e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27.194874</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>20</td>\n",
       "      <td>(4856378, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10882.032388</td>\n",
       "      <td>0.185270</td>\n",
       "      <td>10882.032388</td>\n",
       "      <td>1.184186e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27.196202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>21</td>\n",
       "      <td>(4856379, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11768.392023</td>\n",
       "      <td>0.200293</td>\n",
       "      <td>11768.392023</td>\n",
       "      <td>1.384951e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.846672</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>22</td>\n",
       "      <td>(4856380, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12733.720589</td>\n",
       "      <td>0.216912</td>\n",
       "      <td>12733.720589</td>\n",
       "      <td>1.621476e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.089569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>23</td>\n",
       "      <td>(4856381, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12181.572074</td>\n",
       "      <td>0.207421</td>\n",
       "      <td>12181.572074</td>\n",
       "      <td>1.483907e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.290534</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>24</td>\n",
       "      <td>(4856382, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10370.577911</td>\n",
       "      <td>0.176762</td>\n",
       "      <td>10370.577911</td>\n",
       "      <td>1.075489e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.150938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>25</td>\n",
       "      <td>(4856383, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>6237.292015</td>\n",
       "      <td>0.106387</td>\n",
       "      <td>6237.292015</td>\n",
       "      <td>3.890381e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30.264950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>26</td>\n",
       "      <td>(4856384, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12341.410357</td>\n",
       "      <td>0.210452</td>\n",
       "      <td>12341.410357</td>\n",
       "      <td>1.523104e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.917187</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>27</td>\n",
       "      <td>(4856385, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10546.091305</td>\n",
       "      <td>0.179967</td>\n",
       "      <td>10546.091305</td>\n",
       "      <td>1.112200e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.504777</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>28</td>\n",
       "      <td>(4856386, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12136.373984</td>\n",
       "      <td>0.207047</td>\n",
       "      <td>12136.373984</td>\n",
       "      <td>1.472916e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.069534</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>29</td>\n",
       "      <td>(4856387, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11244.534647</td>\n",
       "      <td>0.191833</td>\n",
       "      <td>11244.534647</td>\n",
       "      <td>1.264396e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.340592</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>30</td>\n",
       "      <td>(4856388, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12027.627943</td>\n",
       "      <td>0.205192</td>\n",
       "      <td>12027.627943</td>\n",
       "      <td>1.446638e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30.749720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>31</td>\n",
       "      <td>(4856389, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12006.942041</td>\n",
       "      <td>0.204825</td>\n",
       "      <td>12006.942041</td>\n",
       "      <td>1.441667e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.707644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>32</td>\n",
       "      <td>(4856390, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12743.357526</td>\n",
       "      <td>0.217432</td>\n",
       "      <td>12743.357526</td>\n",
       "      <td>1.623932e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.306292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>33</td>\n",
       "      <td>(4856391, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>19191.545424</td>\n",
       "      <td>0.327501</td>\n",
       "      <td>19191.545424</td>\n",
       "      <td>3.683154e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.869169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>34</td>\n",
       "      <td>(4856392, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>15938.819592</td>\n",
       "      <td>0.271994</td>\n",
       "      <td>15938.819592</td>\n",
       "      <td>2.540460e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.383972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>35</td>\n",
       "      <td>(4856393, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>18020.398048</td>\n",
       "      <td>0.307167</td>\n",
       "      <td>18020.398048</td>\n",
       "      <td>3.247347e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.114867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>36</td>\n",
       "      <td>(4856394, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>18127.919653</td>\n",
       "      <td>0.308808</td>\n",
       "      <td>18127.919653</td>\n",
       "      <td>3.286215e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30.362515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>37</td>\n",
       "      <td>(4856395, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11609.756207</td>\n",
       "      <td>0.197856</td>\n",
       "      <td>11609.756207</td>\n",
       "      <td>1.347864e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.953801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>38</td>\n",
       "      <td>(4856396, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11831.599219</td>\n",
       "      <td>0.201752</td>\n",
       "      <td>11831.599219</td>\n",
       "      <td>1.399867e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.180984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>39</td>\n",
       "      <td>(4856397, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11875.559250</td>\n",
       "      <td>0.202346</td>\n",
       "      <td>11875.559250</td>\n",
       "      <td>1.410289e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.304842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>40</td>\n",
       "      <td>(4856398, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>12842.250096</td>\n",
       "      <td>0.218778</td>\n",
       "      <td>12842.250096</td>\n",
       "      <td>1.649234e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.055995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>41</td>\n",
       "      <td>(4856399, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10376.704734</td>\n",
       "      <td>0.176778</td>\n",
       "      <td>10376.704734</td>\n",
       "      <td>1.076760e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.853883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>42</td>\n",
       "      <td>(4856400, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>9394.789348</td>\n",
       "      <td>0.160052</td>\n",
       "      <td>9394.789348</td>\n",
       "      <td>8.826207e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.623255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>43</td>\n",
       "      <td>(4856401, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10813.767421</td>\n",
       "      <td>0.184096</td>\n",
       "      <td>10813.767421</td>\n",
       "      <td>1.169376e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27.076667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>44</td>\n",
       "      <td>(4856402, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>9423.982555</td>\n",
       "      <td>0.160506</td>\n",
       "      <td>9423.982555</td>\n",
       "      <td>8.881145e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.383296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>45</td>\n",
       "      <td>(4856403, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>5544.778697</td>\n",
       "      <td>0.094482</td>\n",
       "      <td>5544.778697</td>\n",
       "      <td>3.074457e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.759749</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>46</td>\n",
       "      <td>(4856404, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>8836.500940</td>\n",
       "      <td>0.150573</td>\n",
       "      <td>8836.500940</td>\n",
       "      <td>7.808375e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.561355</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>47</td>\n",
       "      <td>(4856405, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>16798.477692</td>\n",
       "      <td>0.286059</td>\n",
       "      <td>16798.477692</td>\n",
       "      <td>2.821889e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.752142</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>48</td>\n",
       "      <td>(4856406, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>8998.039865</td>\n",
       "      <td>0.153131</td>\n",
       "      <td>8998.039865</td>\n",
       "      <td>8.096472e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27.114531</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>49</td>\n",
       "      <td>(4856407, 1)</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>10548.091562</td>\n",
       "      <td>0.179456</td>\n",
       "      <td>10548.091562</td>\n",
       "      <td>1.112622e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26.981726</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Model CV_type  Splits    Train&Test Parameters          RMSE  \\\n",
       "0   RandomForest    wfTs       1  (4856359, 1)    [5, 10]  10309.198323   \n",
       "1   RandomForest    wfTs       2  (4856360, 1)    [5, 10]  10090.824991   \n",
       "2   RandomForest    wfTs       3  (4856361, 1)    [5, 10]  10962.858509   \n",
       "3   RandomForest    wfTs       4  (4856362, 1)    [5, 10]  11895.726341   \n",
       "4   RandomForest    wfTs       5  (4856363, 1)    [5, 10]  11494.661753   \n",
       "5   RandomForest    wfTs       6  (4856364, 1)    [5, 10]   9987.145637   \n",
       "6   RandomForest    wfTs       7  (4856365, 1)    [5, 10]  10758.576541   \n",
       "7   RandomForest    wfTs       8  (4856366, 1)    [5, 10]  11738.851146   \n",
       "8   RandomForest    wfTs       9  (4856367, 1)    [5, 10]  11012.023003   \n",
       "9   RandomForest    wfTs      10  (4856368, 1)    [5, 10]   8011.526013   \n",
       "10  RandomForest    wfTs      11  (4856369, 1)    [5, 10]  10197.592505   \n",
       "11  RandomForest    wfTs      12  (4856370, 1)    [5, 10]  11899.411984   \n",
       "12  RandomForest    wfTs      13  (4856371, 1)    [5, 10]  10234.334052   \n",
       "13  RandomForest    wfTs      14  (4856372, 1)    [5, 10]  11439.382599   \n",
       "14  RandomForest    wfTs      15  (4856373, 1)    [5, 10]  10300.272442   \n",
       "15  RandomForest    wfTs      16  (4856374, 1)    [5, 10]  10281.758708   \n",
       "16  RandomForest    wfTs      17  (4856375, 1)    [5, 10]   9367.081357   \n",
       "17  RandomForest    wfTs      18  (4856376, 1)    [5, 10]  10418.865048   \n",
       "18  RandomForest    wfTs      19  (4856377, 1)    [5, 10]   8461.149006   \n",
       "19  RandomForest    wfTs      20  (4856378, 1)    [5, 10]  10882.032388   \n",
       "20  RandomForest    wfTs      21  (4856379, 1)    [5, 10]  11768.392023   \n",
       "21  RandomForest    wfTs      22  (4856380, 1)    [5, 10]  12733.720589   \n",
       "22  RandomForest    wfTs      23  (4856381, 1)    [5, 10]  12181.572074   \n",
       "23  RandomForest    wfTs      24  (4856382, 1)    [5, 10]  10370.577911   \n",
       "24  RandomForest    wfTs      25  (4856383, 1)    [5, 10]   6237.292015   \n",
       "25  RandomForest    wfTs      26  (4856384, 1)    [5, 10]  12341.410357   \n",
       "26  RandomForest    wfTs      27  (4856385, 1)    [5, 10]  10546.091305   \n",
       "27  RandomForest    wfTs      28  (4856386, 1)    [5, 10]  12136.373984   \n",
       "28  RandomForest    wfTs      29  (4856387, 1)    [5, 10]  11244.534647   \n",
       "29  RandomForest    wfTs      30  (4856388, 1)    [5, 10]  12027.627943   \n",
       "30  RandomForest    wfTs      31  (4856389, 1)    [5, 10]  12006.942041   \n",
       "31  RandomForest    wfTs      32  (4856390, 1)    [5, 10]  12743.357526   \n",
       "32  RandomForest    wfTs      33  (4856391, 1)    [5, 10]  19191.545424   \n",
       "33  RandomForest    wfTs      34  (4856392, 1)    [5, 10]  15938.819592   \n",
       "34  RandomForest    wfTs      35  (4856393, 1)    [5, 10]  18020.398048   \n",
       "35  RandomForest    wfTs      36  (4856394, 1)    [5, 10]  18127.919653   \n",
       "36  RandomForest    wfTs      37  (4856395, 1)    [5, 10]  11609.756207   \n",
       "37  RandomForest    wfTs      38  (4856396, 1)    [5, 10]  11831.599219   \n",
       "38  RandomForest    wfTs      39  (4856397, 1)    [5, 10]  11875.559250   \n",
       "39  RandomForest    wfTs      40  (4856398, 1)    [5, 10]  12842.250096   \n",
       "40  RandomForest    wfTs      41  (4856399, 1)    [5, 10]  10376.704734   \n",
       "41  RandomForest    wfTs      42  (4856400, 1)    [5, 10]   9394.789348   \n",
       "42  RandomForest    wfTs      43  (4856401, 1)    [5, 10]  10813.767421   \n",
       "43  RandomForest    wfTs      44  (4856402, 1)    [5, 10]   9423.982555   \n",
       "44  RandomForest    wfTs      45  (4856403, 1)    [5, 10]   5544.778697   \n",
       "45  RandomForest    wfTs      46  (4856404, 1)    [5, 10]   8836.500940   \n",
       "46  RandomForest    wfTs      47  (4856405, 1)    [5, 10]  16798.477692   \n",
       "47  RandomForest    wfTs      48  (4856406, 1)    [5, 10]   8998.039865   \n",
       "48  RandomForest    wfTs      49  (4856407, 1)    [5, 10]  10548.091562   \n",
       "\n",
       "        MAPE           MAE      Variance   R2  Adjusted_R2       Time  \n",
       "0   0.175588  10309.198323  1.062796e+08 -inf          NaN  25.597913  \n",
       "1   0.171891  10090.824991  1.018247e+08 -inf          NaN  25.329065  \n",
       "2   0.186654  10962.858509  1.201843e+08 -inf          NaN  24.511330  \n",
       "3   0.202668  11895.726341  1.415083e+08 -inf          NaN  26.054279  \n",
       "4   0.196033  11494.661753  1.321272e+08 -inf          NaN  28.610012  \n",
       "5   0.170365   9987.145637  9.974308e+07 -inf          NaN  26.959577  \n",
       "6   0.183533  10758.576541  1.157470e+08 -inf          NaN  27.444233  \n",
       "7   0.200235  11738.851146  1.378006e+08 -inf          NaN  27.178553  \n",
       "8   0.187817  11012.023003  1.212647e+08 -inf          NaN  28.042655  \n",
       "9   0.136583   8011.526013  6.418455e+07 -inf          NaN  27.845516  \n",
       "10  0.173969  10197.592505  1.039909e+08 -inf          NaN  25.940131  \n",
       "11  0.202953  11899.411984  1.415960e+08 -inf          NaN  26.862306  \n",
       "12  0.174491  10234.334052  1.047416e+08 -inf          NaN  26.101527  \n",
       "13  0.194966  11439.382599  1.308595e+08 -inf          NaN  28.505017  \n",
       "14  0.175496  10300.272442  1.060956e+08 -inf          NaN  28.212100  \n",
       "15  0.175065  10281.758708  1.057146e+08 -inf          NaN  28.138532  \n",
       "16  0.159517   9367.081357  8.774221e+07 -inf          NaN  27.350984  \n",
       "17  0.177347  10418.865048  1.085527e+08 -inf          NaN  28.562823  \n",
       "18  0.144032   8461.149006  7.159104e+07 -inf          NaN  27.194874  \n",
       "19  0.185270  10882.032388  1.184186e+08 -inf          NaN  27.196202  \n",
       "20  0.200293  11768.392023  1.384951e+08 -inf          NaN  26.846672  \n",
       "21  0.216912  12733.720589  1.621476e+08 -inf          NaN  28.089569  \n",
       "22  0.207421  12181.572074  1.483907e+08 -inf          NaN  29.290534  \n",
       "23  0.176762  10370.577911  1.075489e+08 -inf          NaN  29.150938  \n",
       "24  0.106387   6237.292015  3.890381e+07 -inf          NaN  30.264950  \n",
       "25  0.210452  12341.410357  1.523104e+08 -inf          NaN  28.917187  \n",
       "26  0.179967  10546.091305  1.112200e+08 -inf          NaN  29.504777  \n",
       "27  0.207047  12136.373984  1.472916e+08 -inf          NaN  29.069534  \n",
       "28  0.191833  11244.534647  1.264396e+08 -inf          NaN  29.340592  \n",
       "29  0.205192  12027.627943  1.446638e+08 -inf          NaN  30.749720  \n",
       "30  0.204825  12006.942041  1.441667e+08 -inf          NaN  29.707644  \n",
       "31  0.217432  12743.357526  1.623932e+08 -inf          NaN  29.306292  \n",
       "32  0.327501  19191.545424  3.683154e+08 -inf          NaN  29.869169  \n",
       "33  0.271994  15938.819592  2.540460e+08 -inf          NaN  28.383972  \n",
       "34  0.307167  18020.398048  3.247347e+08 -inf          NaN  29.114867  \n",
       "35  0.308808  18127.919653  3.286215e+08 -inf          NaN  30.362515  \n",
       "36  0.197856  11609.756207  1.347864e+08 -inf          NaN  29.953801  \n",
       "37  0.201752  11831.599219  1.399867e+08 -inf          NaN  28.180984  \n",
       "38  0.202346  11875.559250  1.410289e+08 -inf          NaN  29.304842  \n",
       "39  0.218778  12842.250096  1.649234e+08 -inf          NaN  28.055995  \n",
       "40  0.176778  10376.704734  1.076760e+08 -inf          NaN  28.853883  \n",
       "41  0.160052   9394.789348  8.826207e+07 -inf          NaN  28.623255  \n",
       "42  0.184096  10813.767421  1.169376e+08 -inf          NaN  27.076667  \n",
       "43  0.160506   9423.982555  8.881145e+07 -inf          NaN  28.383296  \n",
       "44  0.094482   5544.778697  3.074457e+07 -inf          NaN  26.759749  \n",
       "45  0.150573   8836.500940  7.808375e+07 -inf          NaN  26.561355  \n",
       "46  0.286059  16798.477692  2.821889e+08 -inf          NaN  28.752142  \n",
       "47  0.153131   8998.039865  8.096472e+07 -inf          NaN  27.114531  \n",
       "48  0.179456  10548.091562  1.112622e+08 -inf          NaN  26.981726  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_wf_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"RandomForest\", rf_params, vector_assembler ,standard_scaler, wf_cv)\n",
    "rf_wf_cv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "little-substitute",
   "metadata": {},
   "source": [
    "#### 5.4.4. GBTRegression Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "coupled-address",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(809403, 809401)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>304.124379</td>\n",
       "      <td>0.411859</td>\n",
       "      <td>234.294797</td>\n",
       "      <td>5.946904e+04</td>\n",
       "      <td>-0.762192</td>\n",
       "      <td>-0.762218</td>\n",
       "      <td>52.052768</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(1618804, 809401)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>16.935725</td>\n",
       "      <td>0.040170</td>\n",
       "      <td>12.673021</td>\n",
       "      <td>1.974642e+04</td>\n",
       "      <td>0.984613</td>\n",
       "      <td>0.984613</td>\n",
       "      <td>63.991497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(2428206, 809401)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>5046.938453</td>\n",
       "      <td>0.409534</td>\n",
       "      <td>2864.826508</td>\n",
       "      <td>8.310177e+06</td>\n",
       "      <td>-0.343318</td>\n",
       "      <td>-0.343338</td>\n",
       "      <td>121.272183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(3237607, 809401)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>954.675730</td>\n",
       "      <td>0.100484</td>\n",
       "      <td>705.534389</td>\n",
       "      <td>6.473980e+06</td>\n",
       "      <td>0.838009</td>\n",
       "      <td>0.838007</td>\n",
       "      <td>176.767860</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(4047008, 809400)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>11035.333384</td>\n",
       "      <td>0.133965</td>\n",
       "      <td>4677.757736</td>\n",
       "      <td>3.771877e+07</td>\n",
       "      <td>0.344645</td>\n",
       "      <td>0.344635</td>\n",
       "      <td>243.220430</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Model CV_type  Splits         Train&Test    Parameters  \\\n",
       "0  GBTRegression   mulTs       1   (809403, 809401)  [40, 5, 0.3]   \n",
       "1  GBTRegression   mulTs       2  (1618804, 809401)  [40, 5, 0.3]   \n",
       "2  GBTRegression   mulTs       3  (2428206, 809401)  [40, 5, 0.3]   \n",
       "3  GBTRegression   mulTs       4  (3237607, 809401)  [40, 5, 0.3]   \n",
       "4  GBTRegression   mulTs       5  (4047008, 809400)  [40, 5, 0.3]   \n",
       "\n",
       "           RMSE      MAPE          MAE      Variance        R2  Adjusted_R2  \\\n",
       "0    304.124379  0.411859   234.294797  5.946904e+04 -0.762192    -0.762218   \n",
       "1     16.935725  0.040170    12.673021  1.974642e+04  0.984613     0.984613   \n",
       "2   5046.938453  0.409534  2864.826508  8.310177e+06 -0.343318    -0.343338   \n",
       "3    954.675730  0.100484   705.534389  6.473980e+06  0.838009     0.838007   \n",
       "4  11035.333384  0.133965  4677.757736  3.771877e+07  0.344645     0.344635   \n",
       "\n",
       "         Time  \n",
       "0   52.052768  \n",
       "1   63.991497  \n",
       "2  121.272183  \n",
       "3  176.767860  \n",
       "4  243.220430  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gb_params = {\n",
    "    'maxIter' : [40], # max number of iterations (>=0), default:20\n",
    "    'maxDepth' : [5], # Maximum depth of the tree (>=0), <=30, default:5\n",
    "    'stepSize': [0.3] # learning rate, [0,1], default:0.1\n",
    "}\n",
    "gb_mul_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"GBTRegression\", gb_params, vector_assembler ,standard_scaler, mul_cv)\n",
    "gb_mul_cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "enormous-reminder",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>0.216869</td>\n",
       "      <td>0.013450</td>\n",
       "      <td>0.157667</td>\n",
       "      <td>5.860867e-01</td>\n",
       "      <td>0.899433</td>\n",
       "      <td>0.899421</td>\n",
       "      <td>39.846949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>10.149855</td>\n",
       "      <td>0.031160</td>\n",
       "      <td>5.358995</td>\n",
       "      <td>8.316581e+02</td>\n",
       "      <td>0.891028</td>\n",
       "      <td>0.891015</td>\n",
       "      <td>41.014925</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>11.780647</td>\n",
       "      <td>0.019741</td>\n",
       "      <td>8.321268</td>\n",
       "      <td>5.470287e+03</td>\n",
       "      <td>0.973402</td>\n",
       "      <td>0.973399</td>\n",
       "      <td>43.392517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>3.531250</td>\n",
       "      <td>0.008578</td>\n",
       "      <td>2.361131</td>\n",
       "      <td>7.012716e+02</td>\n",
       "      <td>0.980600</td>\n",
       "      <td>0.980598</td>\n",
       "      <td>42.093172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>72.701613</td>\n",
       "      <td>0.092485</td>\n",
       "      <td>59.733615</td>\n",
       "      <td>3.539424e+03</td>\n",
       "      <td>-1.288609</td>\n",
       "      <td>-1.288892</td>\n",
       "      <td>42.091254</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>6</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>628.012678</td>\n",
       "      <td>0.224381</td>\n",
       "      <td>567.701285</td>\n",
       "      <td>3.237684e+05</td>\n",
       "      <td>-3.667031</td>\n",
       "      <td>-3.667608</td>\n",
       "      <td>43.951389</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>7</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>128.678271</td>\n",
       "      <td>0.013624</td>\n",
       "      <td>106.632673</td>\n",
       "      <td>1.024729e+06</td>\n",
       "      <td>0.982922</td>\n",
       "      <td>0.982920</td>\n",
       "      <td>43.213658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>8</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>257.940747</td>\n",
       "      <td>0.034981</td>\n",
       "      <td>194.151209</td>\n",
       "      <td>1.679424e+06</td>\n",
       "      <td>0.959666</td>\n",
       "      <td>0.959661</td>\n",
       "      <td>43.327354</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>9</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>580.910108</td>\n",
       "      <td>0.054826</td>\n",
       "      <td>319.164390</td>\n",
       "      <td>9.947693e+05</td>\n",
       "      <td>0.784885</td>\n",
       "      <td>0.784859</td>\n",
       "      <td>43.267266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>10</td>\n",
       "      <td>(388512, 97128)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>11501.536760</td>\n",
       "      <td>0.183975</td>\n",
       "      <td>9681.641599</td>\n",
       "      <td>9.766690e+07</td>\n",
       "      <td>-0.740701</td>\n",
       "      <td>-0.740916</td>\n",
       "      <td>43.379082</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Model CV_type  Splits       Train&Test    Parameters          RMSE  \\\n",
       "0  GBTRegression   blkTs       1  (388512, 97128)  [40, 5, 0.3]      0.216869   \n",
       "1  GBTRegression   blkTs       2  (388512, 97128)  [40, 5, 0.3]     10.149855   \n",
       "2  GBTRegression   blkTs       3  (388512, 97128)  [40, 5, 0.3]     11.780647   \n",
       "3  GBTRegression   blkTs       4  (388512, 97128)  [40, 5, 0.3]      3.531250   \n",
       "4  GBTRegression   blkTs       5  (388512, 97128)  [40, 5, 0.3]     72.701613   \n",
       "5  GBTRegression   blkTs       6  (388512, 97128)  [40, 5, 0.3]    628.012678   \n",
       "6  GBTRegression   blkTs       7  (388512, 97128)  [40, 5, 0.3]    128.678271   \n",
       "7  GBTRegression   blkTs       8  (388512, 97128)  [40, 5, 0.3]    257.940747   \n",
       "8  GBTRegression   blkTs       9  (388512, 97128)  [40, 5, 0.3]    580.910108   \n",
       "9  GBTRegression   blkTs      10  (388512, 97128)  [40, 5, 0.3]  11501.536760   \n",
       "\n",
       "       MAPE          MAE      Variance        R2  Adjusted_R2       Time  \n",
       "0  0.013450     0.157667  5.860867e-01  0.899433     0.899421  39.846949  \n",
       "1  0.031160     5.358995  8.316581e+02  0.891028     0.891015  41.014925  \n",
       "2  0.019741     8.321268  5.470287e+03  0.973402     0.973399  43.392517  \n",
       "3  0.008578     2.361131  7.012716e+02  0.980600     0.980598  42.093172  \n",
       "4  0.092485    59.733615  3.539424e+03 -1.288609    -1.288892  42.091254  \n",
       "5  0.224381   567.701285  3.237684e+05 -3.667031    -3.667608  43.951389  \n",
       "6  0.013624   106.632673  1.024729e+06  0.982922     0.982920  43.213658  \n",
       "7  0.034981   194.151209  1.679424e+06  0.959666     0.959661  43.327354  \n",
       "8  0.054826   319.164390  9.947693e+05  0.784885     0.784859  43.267266  \n",
       "9  0.183975  9681.641599  9.766690e+07 -0.740701    -0.740916  43.379082  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gb_blk_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"GBTRegression\", gb_params, vector_assembler ,standard_scaler, blk_cv)\n",
    "gb_blk_cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "israeli-republican",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Splits</th>\n",
       "      <th>Train&amp;Test</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>1</td>\n",
       "      <td>(4856359, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>2890.492341</td>\n",
       "      <td>0.049231</td>\n",
       "      <td>2890.492341</td>\n",
       "      <td>8.354946e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>311.532754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>2</td>\n",
       "      <td>(4856360, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>6838.085423</td>\n",
       "      <td>0.116482</td>\n",
       "      <td>6838.085423</td>\n",
       "      <td>4.675941e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>295.566624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>3</td>\n",
       "      <td>(4856361, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4419.999577</td>\n",
       "      <td>0.075255</td>\n",
       "      <td>4419.999577</td>\n",
       "      <td>1.953640e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>303.142009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>4</td>\n",
       "      <td>(4856362, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>3971.490656</td>\n",
       "      <td>0.067662</td>\n",
       "      <td>3971.490656</td>\n",
       "      <td>1.577274e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>300.848879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>5</td>\n",
       "      <td>(4856363, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>9411.128135</td>\n",
       "      <td>0.160500</td>\n",
       "      <td>9411.128135</td>\n",
       "      <td>8.856933e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>299.834889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>6</td>\n",
       "      <td>(4856364, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>2643.751920</td>\n",
       "      <td>0.045098</td>\n",
       "      <td>2643.751920</td>\n",
       "      <td>6.989424e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>295.209719</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>7</td>\n",
       "      <td>(4856365, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>7388.959484</td>\n",
       "      <td>0.126050</td>\n",
       "      <td>7388.959484</td>\n",
       "      <td>5.459672e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>294.508795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>8</td>\n",
       "      <td>(4856366, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>9479.654503</td>\n",
       "      <td>0.161699</td>\n",
       "      <td>9479.654503</td>\n",
       "      <td>8.986385e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>294.856564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>9</td>\n",
       "      <td>(4856367, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>6095.821747</td>\n",
       "      <td>0.103968</td>\n",
       "      <td>6095.821747</td>\n",
       "      <td>3.715904e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>290.341172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>10</td>\n",
       "      <td>(4856368, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>1360.662367</td>\n",
       "      <td>0.023197</td>\n",
       "      <td>1360.662367</td>\n",
       "      <td>1.851402e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>295.397956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>11</td>\n",
       "      <td>(4856369, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4500.210429</td>\n",
       "      <td>0.076773</td>\n",
       "      <td>4500.210429</td>\n",
       "      <td>2.025189e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>295.614616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>12</td>\n",
       "      <td>(4856370, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>1702.355620</td>\n",
       "      <td>0.029035</td>\n",
       "      <td>1702.355620</td>\n",
       "      <td>2.898015e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>307.977090</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>13</td>\n",
       "      <td>(4856371, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>9779.904613</td>\n",
       "      <td>0.166743</td>\n",
       "      <td>9779.904613</td>\n",
       "      <td>9.564653e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>296.542670</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>14</td>\n",
       "      <td>(4856372, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4808.948050</td>\n",
       "      <td>0.081961</td>\n",
       "      <td>4808.948050</td>\n",
       "      <td>2.312598e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>292.320524</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>15</td>\n",
       "      <td>(4856373, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4813.378438</td>\n",
       "      <td>0.082010</td>\n",
       "      <td>4813.378438</td>\n",
       "      <td>2.316861e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>294.337876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>16</td>\n",
       "      <td>(4856374, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>5777.516323</td>\n",
       "      <td>0.098373</td>\n",
       "      <td>5777.516323</td>\n",
       "      <td>3.337969e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>291.905100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>17</td>\n",
       "      <td>(4856375, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4470.528528</td>\n",
       "      <td>0.076131</td>\n",
       "      <td>4470.528528</td>\n",
       "      <td>1.998563e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>290.095710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>18</td>\n",
       "      <td>(4856376, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>3790.024173</td>\n",
       "      <td>0.064513</td>\n",
       "      <td>3790.024173</td>\n",
       "      <td>1.436428e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>284.257610</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>19</td>\n",
       "      <td>(4856377, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>2870.950548</td>\n",
       "      <td>0.048872</td>\n",
       "      <td>2870.950548</td>\n",
       "      <td>8.242357e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>284.686734</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>20</td>\n",
       "      <td>(4856378, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>5828.333470</td>\n",
       "      <td>0.099229</td>\n",
       "      <td>5828.333470</td>\n",
       "      <td>3.396947e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>288.534291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>21</td>\n",
       "      <td>(4856379, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4580.880159</td>\n",
       "      <td>0.077965</td>\n",
       "      <td>4580.880159</td>\n",
       "      <td>2.098446e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>288.233802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>22</td>\n",
       "      <td>(4856380, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>8860.605518</td>\n",
       "      <td>0.150935</td>\n",
       "      <td>8860.605518</td>\n",
       "      <td>7.851033e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>289.803948</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>23</td>\n",
       "      <td>(4856381, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>7187.734218</td>\n",
       "      <td>0.122389</td>\n",
       "      <td>7187.734218</td>\n",
       "      <td>5.166352e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>292.525831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>24</td>\n",
       "      <td>(4856382, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>3160.822292</td>\n",
       "      <td>0.053875</td>\n",
       "      <td>3160.822292</td>\n",
       "      <td>9.990798e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>284.446757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>25</td>\n",
       "      <td>(4856383, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>5462.377893</td>\n",
       "      <td>0.093170</td>\n",
       "      <td>5462.377893</td>\n",
       "      <td>2.983757e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>286.942437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>26</td>\n",
       "      <td>(4856384, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4374.417742</td>\n",
       "      <td>0.074595</td>\n",
       "      <td>4374.417742</td>\n",
       "      <td>1.913553e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>287.640108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>27</td>\n",
       "      <td>(4856385, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>5064.566521</td>\n",
       "      <td>0.086426</td>\n",
       "      <td>5064.566521</td>\n",
       "      <td>2.564983e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>287.210467</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>28</td>\n",
       "      <td>(4856386, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>10627.483048</td>\n",
       "      <td>0.181305</td>\n",
       "      <td>10627.483048</td>\n",
       "      <td>1.129434e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>281.985315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>29</td>\n",
       "      <td>(4856387, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>6260.019939</td>\n",
       "      <td>0.106797</td>\n",
       "      <td>6260.019939</td>\n",
       "      <td>3.918785e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>284.984795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>30</td>\n",
       "      <td>(4856388, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>12362.291340</td>\n",
       "      <td>0.210902</td>\n",
       "      <td>12362.291340</td>\n",
       "      <td>1.528262e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>283.577603</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>31</td>\n",
       "      <td>(4856389, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>11897.893655</td>\n",
       "      <td>0.202964</td>\n",
       "      <td>11897.893655</td>\n",
       "      <td>1.415599e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>286.552008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>32</td>\n",
       "      <td>(4856390, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>12273.847435</td>\n",
       "      <td>0.209421</td>\n",
       "      <td>12273.847435</td>\n",
       "      <td>1.506473e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>291.079360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>33</td>\n",
       "      <td>(4856391, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>20569.589902</td>\n",
       "      <td>0.351017</td>\n",
       "      <td>20569.589902</td>\n",
       "      <td>4.231080e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>283.851434</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>34</td>\n",
       "      <td>(4856392, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>12322.607683</td>\n",
       "      <td>0.210283</td>\n",
       "      <td>12322.607683</td>\n",
       "      <td>1.518467e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>302.742665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>35</td>\n",
       "      <td>(4856393, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>19203.138592</td>\n",
       "      <td>0.327327</td>\n",
       "      <td>19203.138592</td>\n",
       "      <td>3.687605e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>295.040581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>36</td>\n",
       "      <td>(4856394, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>19243.588718</td>\n",
       "      <td>0.327814</td>\n",
       "      <td>19243.588718</td>\n",
       "      <td>3.703157e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>280.195135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>37</td>\n",
       "      <td>(4856395, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>6432.062594</td>\n",
       "      <td>0.109617</td>\n",
       "      <td>6432.062594</td>\n",
       "      <td>4.137143e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>287.925472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>38</td>\n",
       "      <td>(4856396, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>7258.970776</td>\n",
       "      <td>0.123780</td>\n",
       "      <td>7258.970776</td>\n",
       "      <td>5.269266e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>288.422990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>39</td>\n",
       "      <td>(4856397, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>1662.189842</td>\n",
       "      <td>0.028322</td>\n",
       "      <td>1662.189842</td>\n",
       "      <td>2.762875e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>291.357783</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>40</td>\n",
       "      <td>(4856398, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4446.971699</td>\n",
       "      <td>0.075758</td>\n",
       "      <td>4446.971699</td>\n",
       "      <td>1.977556e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>283.032385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>41</td>\n",
       "      <td>(4856399, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>7724.985398</td>\n",
       "      <td>0.131603</td>\n",
       "      <td>7724.985398</td>\n",
       "      <td>5.967540e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>282.927552</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>42</td>\n",
       "      <td>(4856400, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>4245.043004</td>\n",
       "      <td>0.072319</td>\n",
       "      <td>4245.043004</td>\n",
       "      <td>1.802039e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>282.510642</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>43</td>\n",
       "      <td>(4856401, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>5983.155347</td>\n",
       "      <td>0.101858</td>\n",
       "      <td>5983.155347</td>\n",
       "      <td>3.579815e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>289.587059</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>44</td>\n",
       "      <td>(4856402, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>2364.887957</td>\n",
       "      <td>0.040278</td>\n",
       "      <td>2364.887957</td>\n",
       "      <td>5.592695e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>288.201697</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>45</td>\n",
       "      <td>(4856403, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>5464.092489</td>\n",
       "      <td>0.093107</td>\n",
       "      <td>5464.092489</td>\n",
       "      <td>2.985631e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>286.559751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>46</td>\n",
       "      <td>(4856404, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>3971.075233</td>\n",
       "      <td>0.067667</td>\n",
       "      <td>3971.075233</td>\n",
       "      <td>1.576944e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>293.234954</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>47</td>\n",
       "      <td>(4856405, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>15540.037728</td>\n",
       "      <td>0.264629</td>\n",
       "      <td>15540.037728</td>\n",
       "      <td>2.414928e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>297.980945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>48</td>\n",
       "      <td>(4856406, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>6422.018998</td>\n",
       "      <td>0.109291</td>\n",
       "      <td>6422.018998</td>\n",
       "      <td>4.124233e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>319.166888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>49</td>\n",
       "      <td>(4856407, 1)</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>2528.460281</td>\n",
       "      <td>0.043017</td>\n",
       "      <td>2528.460281</td>\n",
       "      <td>6.393111e+06</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>307.627870</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Model CV_type  Splits    Train&Test    Parameters          RMSE  \\\n",
       "0   GBTRegression    wfTs       1  (4856359, 1)  [40, 5, 0.3]   2890.492341   \n",
       "1   GBTRegression    wfTs       2  (4856360, 1)  [40, 5, 0.3]   6838.085423   \n",
       "2   GBTRegression    wfTs       3  (4856361, 1)  [40, 5, 0.3]   4419.999577   \n",
       "3   GBTRegression    wfTs       4  (4856362, 1)  [40, 5, 0.3]   3971.490656   \n",
       "4   GBTRegression    wfTs       5  (4856363, 1)  [40, 5, 0.3]   9411.128135   \n",
       "5   GBTRegression    wfTs       6  (4856364, 1)  [40, 5, 0.3]   2643.751920   \n",
       "6   GBTRegression    wfTs       7  (4856365, 1)  [40, 5, 0.3]   7388.959484   \n",
       "7   GBTRegression    wfTs       8  (4856366, 1)  [40, 5, 0.3]   9479.654503   \n",
       "8   GBTRegression    wfTs       9  (4856367, 1)  [40, 5, 0.3]   6095.821747   \n",
       "9   GBTRegression    wfTs      10  (4856368, 1)  [40, 5, 0.3]   1360.662367   \n",
       "10  GBTRegression    wfTs      11  (4856369, 1)  [40, 5, 0.3]   4500.210429   \n",
       "11  GBTRegression    wfTs      12  (4856370, 1)  [40, 5, 0.3]   1702.355620   \n",
       "12  GBTRegression    wfTs      13  (4856371, 1)  [40, 5, 0.3]   9779.904613   \n",
       "13  GBTRegression    wfTs      14  (4856372, 1)  [40, 5, 0.3]   4808.948050   \n",
       "14  GBTRegression    wfTs      15  (4856373, 1)  [40, 5, 0.3]   4813.378438   \n",
       "15  GBTRegression    wfTs      16  (4856374, 1)  [40, 5, 0.3]   5777.516323   \n",
       "16  GBTRegression    wfTs      17  (4856375, 1)  [40, 5, 0.3]   4470.528528   \n",
       "17  GBTRegression    wfTs      18  (4856376, 1)  [40, 5, 0.3]   3790.024173   \n",
       "18  GBTRegression    wfTs      19  (4856377, 1)  [40, 5, 0.3]   2870.950548   \n",
       "19  GBTRegression    wfTs      20  (4856378, 1)  [40, 5, 0.3]   5828.333470   \n",
       "20  GBTRegression    wfTs      21  (4856379, 1)  [40, 5, 0.3]   4580.880159   \n",
       "21  GBTRegression    wfTs      22  (4856380, 1)  [40, 5, 0.3]   8860.605518   \n",
       "22  GBTRegression    wfTs      23  (4856381, 1)  [40, 5, 0.3]   7187.734218   \n",
       "23  GBTRegression    wfTs      24  (4856382, 1)  [40, 5, 0.3]   3160.822292   \n",
       "24  GBTRegression    wfTs      25  (4856383, 1)  [40, 5, 0.3]   5462.377893   \n",
       "25  GBTRegression    wfTs      26  (4856384, 1)  [40, 5, 0.3]   4374.417742   \n",
       "26  GBTRegression    wfTs      27  (4856385, 1)  [40, 5, 0.3]   5064.566521   \n",
       "27  GBTRegression    wfTs      28  (4856386, 1)  [40, 5, 0.3]  10627.483048   \n",
       "28  GBTRegression    wfTs      29  (4856387, 1)  [40, 5, 0.3]   6260.019939   \n",
       "29  GBTRegression    wfTs      30  (4856388, 1)  [40, 5, 0.3]  12362.291340   \n",
       "30  GBTRegression    wfTs      31  (4856389, 1)  [40, 5, 0.3]  11897.893655   \n",
       "31  GBTRegression    wfTs      32  (4856390, 1)  [40, 5, 0.3]  12273.847435   \n",
       "32  GBTRegression    wfTs      33  (4856391, 1)  [40, 5, 0.3]  20569.589902   \n",
       "33  GBTRegression    wfTs      34  (4856392, 1)  [40, 5, 0.3]  12322.607683   \n",
       "34  GBTRegression    wfTs      35  (4856393, 1)  [40, 5, 0.3]  19203.138592   \n",
       "35  GBTRegression    wfTs      36  (4856394, 1)  [40, 5, 0.3]  19243.588718   \n",
       "36  GBTRegression    wfTs      37  (4856395, 1)  [40, 5, 0.3]   6432.062594   \n",
       "37  GBTRegression    wfTs      38  (4856396, 1)  [40, 5, 0.3]   7258.970776   \n",
       "38  GBTRegression    wfTs      39  (4856397, 1)  [40, 5, 0.3]   1662.189842   \n",
       "39  GBTRegression    wfTs      40  (4856398, 1)  [40, 5, 0.3]   4446.971699   \n",
       "40  GBTRegression    wfTs      41  (4856399, 1)  [40, 5, 0.3]   7724.985398   \n",
       "41  GBTRegression    wfTs      42  (4856400, 1)  [40, 5, 0.3]   4245.043004   \n",
       "42  GBTRegression    wfTs      43  (4856401, 1)  [40, 5, 0.3]   5983.155347   \n",
       "43  GBTRegression    wfTs      44  (4856402, 1)  [40, 5, 0.3]   2364.887957   \n",
       "44  GBTRegression    wfTs      45  (4856403, 1)  [40, 5, 0.3]   5464.092489   \n",
       "45  GBTRegression    wfTs      46  (4856404, 1)  [40, 5, 0.3]   3971.075233   \n",
       "46  GBTRegression    wfTs      47  (4856405, 1)  [40, 5, 0.3]  15540.037728   \n",
       "47  GBTRegression    wfTs      48  (4856406, 1)  [40, 5, 0.3]   6422.018998   \n",
       "48  GBTRegression    wfTs      49  (4856407, 1)  [40, 5, 0.3]   2528.460281   \n",
       "\n",
       "        MAPE           MAE      Variance   R2  Adjusted_R2        Time  \n",
       "0   0.049231   2890.492341  8.354946e+06 -inf          NaN  311.532754  \n",
       "1   0.116482   6838.085423  4.675941e+07 -inf          NaN  295.566624  \n",
       "2   0.075255   4419.999577  1.953640e+07 -inf          NaN  303.142009  \n",
       "3   0.067662   3971.490656  1.577274e+07 -inf          NaN  300.848879  \n",
       "4   0.160500   9411.128135  8.856933e+07 -inf          NaN  299.834889  \n",
       "5   0.045098   2643.751920  6.989424e+06 -inf          NaN  295.209719  \n",
       "6   0.126050   7388.959484  5.459672e+07 -inf          NaN  294.508795  \n",
       "7   0.161699   9479.654503  8.986385e+07 -inf          NaN  294.856564  \n",
       "8   0.103968   6095.821747  3.715904e+07 -inf          NaN  290.341172  \n",
       "9   0.023197   1360.662367  1.851402e+06 -inf          NaN  295.397956  \n",
       "10  0.076773   4500.210429  2.025189e+07 -inf          NaN  295.614616  \n",
       "11  0.029035   1702.355620  2.898015e+06 -inf          NaN  307.977090  \n",
       "12  0.166743   9779.904613  9.564653e+07 -inf          NaN  296.542670  \n",
       "13  0.081961   4808.948050  2.312598e+07 -inf          NaN  292.320524  \n",
       "14  0.082010   4813.378438  2.316861e+07 -inf          NaN  294.337876  \n",
       "15  0.098373   5777.516323  3.337969e+07 -inf          NaN  291.905100  \n",
       "16  0.076131   4470.528528  1.998563e+07 -inf          NaN  290.095710  \n",
       "17  0.064513   3790.024173  1.436428e+07 -inf          NaN  284.257610  \n",
       "18  0.048872   2870.950548  8.242357e+06 -inf          NaN  284.686734  \n",
       "19  0.099229   5828.333470  3.396947e+07 -inf          NaN  288.534291  \n",
       "20  0.077965   4580.880159  2.098446e+07 -inf          NaN  288.233802  \n",
       "21  0.150935   8860.605518  7.851033e+07 -inf          NaN  289.803948  \n",
       "22  0.122389   7187.734218  5.166352e+07 -inf          NaN  292.525831  \n",
       "23  0.053875   3160.822292  9.990798e+06 -inf          NaN  284.446757  \n",
       "24  0.093170   5462.377893  2.983757e+07 -inf          NaN  286.942437  \n",
       "25  0.074595   4374.417742  1.913553e+07 -inf          NaN  287.640108  \n",
       "26  0.086426   5064.566521  2.564983e+07 -inf          NaN  287.210467  \n",
       "27  0.181305  10627.483048  1.129434e+08 -inf          NaN  281.985315  \n",
       "28  0.106797   6260.019939  3.918785e+07 -inf          NaN  284.984795  \n",
       "29  0.210902  12362.291340  1.528262e+08 -inf          NaN  283.577603  \n",
       "30  0.202964  11897.893655  1.415599e+08 -inf          NaN  286.552008  \n",
       "31  0.209421  12273.847435  1.506473e+08 -inf          NaN  291.079360  \n",
       "32  0.351017  20569.589902  4.231080e+08 -inf          NaN  283.851434  \n",
       "33  0.210283  12322.607683  1.518467e+08 -inf          NaN  302.742665  \n",
       "34  0.327327  19203.138592  3.687605e+08 -inf          NaN  295.040581  \n",
       "35  0.327814  19243.588718  3.703157e+08 -inf          NaN  280.195135  \n",
       "36  0.109617   6432.062594  4.137143e+07 -inf          NaN  287.925472  \n",
       "37  0.123780   7258.970776  5.269266e+07 -inf          NaN  288.422990  \n",
       "38  0.028322   1662.189842  2.762875e+06 -inf          NaN  291.357783  \n",
       "39  0.075758   4446.971699  1.977556e+07 -inf          NaN  283.032385  \n",
       "40  0.131603   7724.985398  5.967540e+07 -inf          NaN  282.927552  \n",
       "41  0.072319   4245.043004  1.802039e+07 -inf          NaN  282.510642  \n",
       "42  0.101858   5983.155347  3.579815e+07 -inf          NaN  289.587059  \n",
       "43  0.040278   2364.887957  5.592695e+06 -inf          NaN  288.201697  \n",
       "44  0.093107   5464.092489  2.985631e+07 -inf          NaN  286.559751  \n",
       "45  0.067667   3971.075233  1.576944e+07 -inf          NaN  293.234954  \n",
       "46  0.264629  15540.037728  2.414928e+08 -inf          NaN  297.980945  \n",
       "47  0.109291   6422.018998  4.124233e+07 -inf          NaN  319.166888  \n",
       "48  0.043017   2528.460281  6.393111e+06 -inf          NaN  307.627870  "
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gb_wf_cv = tsCrossValidation(dataset, \"features\", \"NEXT_BTC_CLOSE\", \"GBTRegression\", gb_params, vector_assembler ,standard_scaler, wf_cv)\n",
    "gb_wf_cv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "streaming-marine",
   "metadata": {},
   "source": [
    "# 6.Summary  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "premier-apparatus",
   "metadata": {},
   "source": [
    "## 6.1. Model Comparison Table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "protected-simpson",
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Description: Apply calculations on Time Series Cross Validation results to form the final Model Comparison Table\n",
    "Args:\n",
    "    cv_result: The results from tsCrossValidation()\n",
    "    model_info: The model information which you would like to show\n",
    "    evaluator_lst: The evaluator metrics which you would like to show\n",
    "Return: \n",
    "    comparison_df: A pandas dataframe of a model on a type of Time Series Cross Validation\n",
    "'''\n",
    "def modelComparison(cv_result, model_info, evaluator_lst):\n",
    "    # Calculate mean of all splits on chosen evaluator \n",
    "    col_mean_df = cv_result[evaluator_lst].mean().to_frame().T\n",
    "    # Extract model info\n",
    "    model_info_df = cv_result[model_info][:1]\n",
    "    # Concatenate by row\n",
    "    comparison_df = pd.concat([model_info_df,col_mean_df],axis=1)\n",
    "    return comparison_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "primary-excitement",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define what model_info and evaluators in the Model Comparison Table\n",
    "model_info = ['Model','CV_type','Parameters']\n",
    "evaluator_lst = ['RMSE','MAPE','MAE','Variance','R2','Adjusted_R2','Time']\n",
    "\n",
    "# The the Cross Validation results would like to compare\n",
    "comparison_lst = [lr_mul_cv,lr_blk_cv,lr_wf_cv,glr_mul_cv,glr_blk_cv,glr_wf_cv,rf_mul_cv,rf_blk_cv,rf_wf_cv,gb_mul_cv,gb_blk_cv,gb_wf_cv]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "sporting-lighter",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>CV_type</th>\n",
       "      <th>Parameters</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAPE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>Variance</th>\n",
       "      <th>R2</th>\n",
       "      <th>Adjusted_R2</th>\n",
       "      <th>Time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>10.951301</td>\n",
       "      <td>0.000955</td>\n",
       "      <td>4.897693</td>\n",
       "      <td>4.209617e+07</td>\n",
       "      <td>0.999978</td>\n",
       "      <td>0.999978</td>\n",
       "      <td>5.552847</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>10.331534</td>\n",
       "      <td>0.001206</td>\n",
       "      <td>6.767410</td>\n",
       "      <td>8.027055e+06</td>\n",
       "      <td>0.999642</td>\n",
       "      <td>0.999642</td>\n",
       "      <td>3.481830</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>[5, 0.0, 0.8]</td>\n",
       "      <td>22.281872</td>\n",
       "      <td>0.000380</td>\n",
       "      <td>22.281872</td>\n",
       "      <td>7.883044e+02</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.992324</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>10.951301</td>\n",
       "      <td>0.000955</td>\n",
       "      <td>4.897693</td>\n",
       "      <td>4.209617e+07</td>\n",
       "      <td>0.999978</td>\n",
       "      <td>0.999978</td>\n",
       "      <td>4.285379</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>10.331534</td>\n",
       "      <td>0.001206</td>\n",
       "      <td>6.767410</td>\n",
       "      <td>8.027055e+06</td>\n",
       "      <td>0.999642</td>\n",
       "      <td>0.999642</td>\n",
       "      <td>2.830701</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GeneralizedLinearRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>[10, 0, gaussian, identity]</td>\n",
       "      <td>22.281872</td>\n",
       "      <td>0.000380</td>\n",
       "      <td>22.281872</td>\n",
       "      <td>7.883044e+02</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.219181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>3544.435520</td>\n",
       "      <td>0.252977</td>\n",
       "      <td>1789.058747</td>\n",
       "      <td>1.074657e+07</td>\n",
       "      <td>0.060790</td>\n",
       "      <td>0.060776</td>\n",
       "      <td>16.507168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>1445.053828</td>\n",
       "      <td>0.072719</td>\n",
       "      <td>1206.804086</td>\n",
       "      <td>1.188896e+07</td>\n",
       "      <td>0.009523</td>\n",
       "      <td>0.009400</td>\n",
       "      <td>9.046196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RandomForest</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>[5, 10]</td>\n",
       "      <td>11352.125410</td>\n",
       "      <td>0.193476</td>\n",
       "      <td>11352.125410</td>\n",
       "      <td>1.356186e+08</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>28.045077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>mulTs</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>3471.601534</td>\n",
       "      <td>0.219202</td>\n",
       "      <td>1699.017290</td>\n",
       "      <td>1.051643e+07</td>\n",
       "      <td>0.212351</td>\n",
       "      <td>0.212340</td>\n",
       "      <td>131.460948</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>blkTs</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>1319.545880</td>\n",
       "      <td>0.067720</td>\n",
       "      <td>1094.522383</td>\n",
       "      <td>1.017001e+07</td>\n",
       "      <td>0.077560</td>\n",
       "      <td>0.077446</td>\n",
       "      <td>42.557757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GBTRegression</td>\n",
       "      <td>wfTs</td>\n",
       "      <td>[40, 5, 0.3]</td>\n",
       "      <td>6945.673721</td>\n",
       "      <td>0.118392</td>\n",
       "      <td>6945.673721</td>\n",
       "      <td>6.861013e+07</td>\n",
       "      <td>-inf</td>\n",
       "      <td>NaN</td>\n",
       "      <td>292.099180</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         Model CV_type                   Parameters  \\\n",
       "0             LinearRegression   mulTs                [5, 0.0, 0.8]   \n",
       "0             LinearRegression   blkTs                [5, 0.0, 0.8]   \n",
       "0             LinearRegression    wfTs                [5, 0.0, 0.8]   \n",
       "0  GeneralizedLinearRegression   mulTs  [10, 0, gaussian, identity]   \n",
       "0  GeneralizedLinearRegression   blkTs  [10, 0, gaussian, identity]   \n",
       "0  GeneralizedLinearRegression    wfTs  [10, 0, gaussian, identity]   \n",
       "0                 RandomForest   mulTs                      [5, 10]   \n",
       "0                 RandomForest   blkTs                      [5, 10]   \n",
       "0                 RandomForest    wfTs                      [5, 10]   \n",
       "0                GBTRegression   mulTs                 [40, 5, 0.3]   \n",
       "0                GBTRegression   blkTs                 [40, 5, 0.3]   \n",
       "0                GBTRegression    wfTs                 [40, 5, 0.3]   \n",
       "\n",
       "           RMSE      MAPE           MAE      Variance        R2  Adjusted_R2  \\\n",
       "0     10.951301  0.000955      4.897693  4.209617e+07  0.999978     0.999978   \n",
       "0     10.331534  0.001206      6.767410  8.027055e+06  0.999642     0.999642   \n",
       "0     22.281872  0.000380     22.281872  7.883044e+02      -inf          NaN   \n",
       "0     10.951301  0.000955      4.897693  4.209617e+07  0.999978     0.999978   \n",
       "0     10.331534  0.001206      6.767410  8.027055e+06  0.999642     0.999642   \n",
       "0     22.281872  0.000380     22.281872  7.883044e+02      -inf          NaN   \n",
       "0   3544.435520  0.252977   1789.058747  1.074657e+07  0.060790     0.060776   \n",
       "0   1445.053828  0.072719   1206.804086  1.188896e+07  0.009523     0.009400   \n",
       "0  11352.125410  0.193476  11352.125410  1.356186e+08      -inf          NaN   \n",
       "0   3471.601534  0.219202   1699.017290  1.051643e+07  0.212351     0.212340   \n",
       "0   1319.545880  0.067720   1094.522383  1.017001e+07  0.077560     0.077446   \n",
       "0   6945.673721  0.118392   6945.673721  6.861013e+07      -inf          NaN   \n",
       "\n",
       "         Time  \n",
       "0    5.552847  \n",
       "0    3.481830  \n",
       "0    7.992324  \n",
       "0    4.285379  \n",
       "0    2.830701  \n",
       "0    6.219181  \n",
       "0   16.507168  \n",
       "0    9.046196  \n",
       "0   28.045077  \n",
       "0  131.460948  \n",
       "0   42.557757  \n",
       "0  292.099180  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show the Comparison Table\n",
    "pd.concat([modelComparison(cv_result,model_info,evaluator_lst) for cv_result in comparison_lst])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "young-disease",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}