DL_univariate_time_series.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DL_univariate_time_series.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyNUkLvBWLKaRDmzU7NSFslM",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/wenxuan0923/My-notes/blob/master/DL_univariate_time_series.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-4M5zdGwkWei",
        "colab_type": "text"
      },
      "source": [
        "## Time Series forecasting\n",
        "### - Temperature prediction\n",
        "\n",
        "This note will cover topics about:\n",
        "-  Convert univariate time series data into DL dataset using self-defined generator\n",
        "- One-step v.s. Multi-step forecasting\n",
        "- Regularization methods for RNN model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dyvax9_Qkcw9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "883450b4-68fd-4434-8fee-d51a1a830098"
      },
      "source": [
        "import os\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from keras import layers\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Dense, GRU"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "twxYAYkxfZgf",
        "colab_type": "text"
      },
      "source": [
        "The temperature dataset used in this note can be found [here](https://s3.amazonaws.com/keras-datasets/jena_climate_2009_2016.csv.zip). It has 420,551 lines of data, each line is a timestep: a record of a date and 14 weather-related values. Note the temperature is recorded evey 10 minutes. For a single hour you will have 6 observations. Similarly, a single day will contain 144 (24\\*6) observations. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cVjC4VUmfYnU",
        "colab_type": "code",
        "outputId": "d3303d49-0745-4ba1-96c8-439570dd0668",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 453
        }
      },
      "source": [
        "temperature_df = pd.read_csv('jena_climate_2009_2016.csv')\n",
        "temperature_df = temperature_df.set_index('Date Time')\n",
        "temperature_df.index = pd.to_datetime(temperature_df.index, \n",
        "                                      format=\"%d.%m.%Y %H:%M:%S\")\n",
        "temperature_df.head()"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>p (mbar)</th>\n",
              "      <th>T (degC)</th>\n",
              "      <th>Tpot (K)</th>\n",
              "      <th>Tdew (degC)</th>\n",
              "      <th>rh (%)</th>\n",
              "      <th>VPmax (mbar)</th>\n",
              "      <th>VPact (mbar)</th>\n",
              "      <th>VPdef (mbar)</th>\n",
              "      <th>sh (g/kg)</th>\n",
              "      <th>H2OC (mmol/mol)</th>\n",
              "      <th>rho (g/m**3)</th>\n",
              "      <th>wv (m/s)</th>\n",
              "      <th>max. wv (m/s)</th>\n",
              "      <th>wd (deg)</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Date Time</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2009-01-01 00:10:00</th>\n",
              "      <td>996.52</td>\n",
              "      <td>-8.02</td>\n",
              "      <td>265.40</td>\n",
              "      <td>-8.90</td>\n",
              "      <td>93.3</td>\n",
              "      <td>3.33</td>\n",
              "      <td>3.11</td>\n",
              "      <td>0.22</td>\n",
              "      <td>1.94</td>\n",
              "      <td>3.12</td>\n",
              "      <td>1307.75</td>\n",
              "      <td>1.03</td>\n",
              "      <td>1.75</td>\n",
              "      <td>152.3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2009-01-01 00:20:00</th>\n",
              "      <td>996.57</td>\n",
              "      <td>-8.41</td>\n",
              "      <td>265.01</td>\n",
              "      <td>-9.28</td>\n",
              "      <td>93.4</td>\n",
              "      <td>3.23</td>\n",
              "      <td>3.02</td>\n",
              "      <td>0.21</td>\n",
              "      <td>1.89</td>\n",
              "      <td>3.03</td>\n",
              "      <td>1309.80</td>\n",
              "      <td>0.72</td>\n",
              "      <td>1.50</td>\n",
              "      <td>136.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2009-01-01 00:30:00</th>\n",
              "      <td>996.53</td>\n",
              "      <td>-8.51</td>\n",
              "      <td>264.91</td>\n",
              "      <td>-9.31</td>\n",
              "      <td>93.9</td>\n",
              "      <td>3.21</td>\n",
              "      <td>3.01</td>\n",
              "      <td>0.20</td>\n",
              "      <td>1.88</td>\n",
              "      <td>3.02</td>\n",
              "      <td>1310.24</td>\n",
              "      <td>0.19</td>\n",
              "      <td>0.63</td>\n",
              "      <td>171.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2009-01-01 00:40:00</th>\n",
              "      <td>996.51</td>\n",
              "      <td>-8.31</td>\n",
              "      <td>265.12</td>\n",
              "      <td>-9.07</td>\n",
              "      <td>94.2</td>\n",
              "      <td>3.26</td>\n",
              "      <td>3.07</td>\n",
              "      <td>0.19</td>\n",
              "      <td>1.92</td>\n",
              "      <td>3.08</td>\n",
              "      <td>1309.19</td>\n",
              "      <td>0.34</td>\n",
              "      <td>0.50</td>\n",
              "      <td>198.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2009-01-01 00:50:00</th>\n",
              "      <td>996.51</td>\n",
              "      <td>-8.27</td>\n",
              "      <td>265.15</td>\n",
              "      <td>-9.04</td>\n",
              "      <td>94.1</td>\n",
              "      <td>3.27</td>\n",
              "      <td>3.08</td>\n",
              "      <td>0.19</td>\n",
              "      <td>1.92</td>\n",
              "      <td>3.09</td>\n",
              "      <td>1309.00</td>\n",
              "      <td>0.32</td>\n",
              "      <td>0.63</td>\n",
              "      <td>214.3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                     p (mbar)  T (degC)  ...  max. wv (m/s)  wd (deg)\n",
              "Date Time                                ...                         \n",
              "2009-01-01 00:10:00    996.52     -8.02  ...           1.75     152.3\n",
              "2009-01-01 00:20:00    996.57     -8.41  ...           1.50     136.1\n",
              "2009-01-01 00:30:00    996.53     -8.51  ...           0.63     171.6\n",
              "2009-01-01 00:40:00    996.51     -8.31  ...           0.50     198.0\n",
              "2009-01-01 00:50:00    996.51     -8.27  ...           0.63     214.3\n",
              "\n",
              "[5 rows x 14 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oCaKqinl059i",
        "colab_type": "text"
      },
      "source": [
        "## Univariate Time Series\n",
        "Let's try only use the temperature **T (degC)** itself for prediction in this note:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n-Fx5NkjhoMN",
        "colab_type": "code",
        "outputId": "60e50dc9-f84f-40ae-821c-60a9f194235e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 272
        }
      },
      "source": [
        "uni_ts = temperature_df['T (degC)']\n",
        "uni_ts.plot()\n",
        "plt.show()"
      ],
      "execution_count": 94,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD/CAYAAAAUnaZMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2dd3hUZfbHvycBQieU0BI0dESqRAQVQYqiqFgRdW2ry+paltVV8aeuXVkbFiygoqxtdcUGCAhIEZASkBaQIoReQgktQEhyfn/ceyeTycxkbn3fufN+nidPZu7ccubOveee97ynEDNDoVAoFP4kSbQACoVCoXAPpeQVCoXCxyglr1AoFD5GKXmFQqHwMUrJKxQKhY9RSl6hUCh8jGNKnoiSieg3Ipqkv29ORIuIaCMRfUlEVZw6lkKhUChiw0lL/u8A1ga9/zeAUczcCsBBAHc4eCyFQqFQxIAjSp6IMgAMAvCB/p4A9AXwtb7KeABXOnEshUKhUMROJYf28zqAhwHU0t/XB5DPzEX6++0A0sNtSETDAAwDgBo1anRr166dQyIpFApFYrB06dJ9zJwW7jPbSp6ILgOwl5mXElEfs9sz81gAYwEgKyuLs7Oz7YqkUCgUCQURbYn0mROW/HkAriCiSwFUBVAbwBsAUomokm7NZwDY4cCxFAqFQmEC2z55Zn6UmTOYORPAUAA/M/NNAGYBuFZf7VYA39s9lkKhUCjM4Wac/CMAHiCijdB89B+6eCyFQqFQhMGpiVcAADPPBjBbf70JQHcn969QKBQKc6iMV4VCofAxSskrFAqpOXisEMcLi0WLEbc46q5RKBQKp+n67HQAQO7IQYIliU+UJa9QKBQ+Ril5hUIRN+QXFOL1GetRUqJ6U8eKUvIKhSJuePy71Xh9xgbM2ZAnWpS4QSl5RcJw4lQxMkdMxpg5f4gWRWGRAn0CtrhYWfKxopS8ImGYvW4vAODFKb8LlkTji8Vb0feV2aLFUPgcFV0TRzAztuwvQGaDGqJFiUv2HS0ULUIZHv1mlWgRFAmAsuTjiDFzN6HPK7Oxdtdh0aIoFEIhEi1B/KCUfByxcNN+AMDuQycESxKf7BdsyW/dX4BPF5avCJude0CANPHHqOnrRYsQYO76PCzdUv53KyouwUfzN6OwqESAVOFRSj6OKNbDxopV+JglcnYeEnr8IWN+xePfrcaJU2WzN69971dBEsUXb8zcIFqEALeMW4xr3i3/u32xZBuenrgGo2dtFCBVeJSSjyNOFWvWwZ3/ycbqHWIVlsI8BwvkmhMAgJ9yduPlaWUnorcdKBAkTcUwlxo478zeKN0oKO+wNsp+U6IHklLyccTCTaUX9Cql5E0jy/iHZREEwLBPluLtWaUhpbPW7UWvl2bhx1W7BEpVMfuPFeKlqeukGwUdPyVfjR2l5GOgpITxycIt5YbZish8snAL+r82R7QYUhEPk4WrtmvGg6xGxKx1WhLUw1+vFCxJ/KBCKGNg0qpdeOK71diZfxyPDJSj0bjs+uKJ71aLFiHAuS/ORMGpYpydWU+0KNJjuBkWb5bLDRIvyDhdppR8DBScLAIAHJAoznr/MXlkkZ2dejSSTG4SWSnStVTekZOCJYlPjp4oEi1COZS7JgaS9HF2sdISCgdgaWYHFE6Tf1w+40sp+RhIStKUfIlESr5BzSqiRZCaxZsP4M7x2VJVKyTpnWyliH4Q3fj+Qrw7O/5qDE3L2SNahHIoJR8Duo4XqjBkUlbxwLBPsjFj7R7kHz8VWMYSPaRl5BeJKjsu+GM//j1VjhpDVnlnthyx8krJx0BykuGuESfDtJzd4g4eZ5w4VYz8Ak25J0lkPEcLrxOd4FZUXIKbP1wsVIZwbN0vb8x+Rbw0dZ0UOQe2lTwRVSWixUS0gohyiOhpfXlzIlpERBuJ6Esiilv/ghGfPnHFTmEy5OxU9WrC8cXirVixLb/MsqvfWRB4TRLGLf6++4hoEcoh43kCgMJiOcoDnDhVjGcmrsHRk2UnVp+emIPXZ6xHfoREt5XbxYeiOmHJnwTQl5k7A+gCYCAR9QDwbwCjmLkVgIMA7nDgWEL4YvFW0SLg2992lHkfT/5dN3n0m1UY/Pb8MsvWBBVwC7bkZXHWHJEwAkMRnU8XbsG4+Zvxdki5go/m5+L1GRvQ5ZnpYbeT4dlpW8mzxlH9bWX9jwH0BfC1vnw8gCvtHksGRMV/h6bEi54YixeSItxlK7fnCxtKh5sbkEAXlEGephxyyBHPdaMc8ckTUTIRLQewF8B0AH8AyGdmw2TZDiA9wrbDiCibiLLz8uSZ+InEJ2GqCLrJR/M346UwE1C/bNjnqRzBbDtQgI175XI5rIngzooU733F6Pno9dIsN0WKa3YeOoEHvlwuWoy4R4aHtyNKnpmLmbkLgAwA3QHEnBbKzGOZOYuZs9LS0pwQx1c8PXEN3pn9R6DtmcGMtVqo1nkjf8b4BbmeytTrpVno/9rciJ+LKLN66Zu/hF0ebHfJMgkbyf+95/AJZI6YjBlr5AjD+ybERSgGOX40q/a7L9w1wTBzPoBZAHoCSCUiI6M2A4AMV4xvMNwQO/KP48kfcgRLU5YXp6wVLUKA4HtMlizhSKGcRt0YEXNAJ4vkrMu069Bx0SLYQobeD05E16QRUar+uhqAAQDWQlP21+qr3Qrge7vHUpRl/1Hxqecb9hzBlJCKheuCokeWbjnomSxb9h8rtyxYnf62Nb/c5yKQ0asrY8QPAPzlP9miRQBgfTwxYZl429YJS74JgFlEtBLAEgDTmXkSgEcAPEBEGwHUB/ChA8dSBJEnWMkfOFaIAaPm4u7PlpWxBIOHqNe8u8CzG7X3y7M9OY5bFOhx9CIUbqQ8MdFJeCdOyRFCaRVfuGuYeSUzd2XmTszcgZmf0ZdvYubuzNyKma9jZvFmp0k27DlSrqGCopS/fbY08Pryt+YBAJZvyy/XZm+6hz7mcNa8jKwLUeSD3pqHE/q8y6GgLF2viORCGhnnWadOYfVR55c4eV9xqOAUrnpnPrYdKMCAUXPLNFTwmmhp+AWFxcIr3u0NilxZv0eLor3y7flCh/5fLN5W5r2spQxC3Vhrdx0OtLfzYuL64/mbkTlicuB9pLM0Yel212Ux+HrpdmSOmIxDBd4/5JxgSZguVWc2rS1AkrIkvJIvLmGM/nkDDp/QLqzJq3bht6350tSdiIZsE64yIGtjlzIPGwYe+25VuXV25GuTjF5keT41cU2Z95HcMl4+IsfN2wwA2HZQfCmAUGLxulwXpktV7j7xI8uEV/I/5ezGKz+tx4s/ahEhRpjdpjzxP05F5Iu2eCQ0kmWqFBpMcBYuIF9texnEkUGGWDAzOjxWKN7oSHglbxSNOq7/GIYVtSgOOuMYlp+XbMo7Gngt400pm/I0KArKIJUxWznSw/GAgLBTGSYroxFvSa8Jr+SNH8yoGf/6DHm6rMuosHYfLo373SzBULQiZDmFwXIcPi5h7RpZTpSkGKdn7+ET2HtEfOy7GRJayTNzoGa1kVwUi+UiOqxMKFG++vJtkePQ35vzB+aul7dsxb6jJyNWErTDr3/sR2FRSZkh/kNfr3D8OHZJ5EvaDN8t34meL/4sWgxTJLSSX7fnSKC2SbKJMeK4+ZvdEqkMItwxFRFNF4yMkuk6csrvuGWc9/XK+782J6b1sp6bEbGSoFVW7ziEG95fiBd+XFvmvJ2SpvhXKdHmMk4Vl+C5SWtcN25kjYQCZCmuYI2EbuRdEhTEkGTicbcz35vhWmjlSRmIdh8adfdFEurvFqE3Ln3jF1yXlYFWDWsCADbuPSql6y2YaOL1eGEm9h8rxNGTRRh5TSfPZFI4Q0Jb8sEKwUzK+6x1e90QpxyyKYb//JqLb5Z5Fzcdr6zZdRhPh4QoymylAtEteaPmz7aDBZi5dk9Ut5wTrNgmPoEoFLl/vegktJIPJrTjSzTcnHC8fsyv+F+2ltAj24X1r+9zbFcmdLset2hdGk6ZyxhNU44oIl7VVasSPuCMRrhjfDauDGnS4gR7j5wIJNEVFEo4MR3HKCWvI1o5GCzafAAPfb0SgPzWnxWKSuSqRfJHUEioE3y5ZFvY5bL/knOiTIrXTNG8uovDZHQ6xdpdchZIM4hnn7xS8jpFJSX4KsINGg4zlr9VRCsGZo67B82s38250q4fs9DR4weHmBrN1wlUrh9ArPz8+x5c9tYvro+APo7Sk8CISfhxlTvN5I+eLCpznZn9rnd8vMRpkRxl6NjymbBeopS8zp7DJ/HwhJUxr3/Lh4tclEZDtH69fsxCNH/0R7FCmGSnyfrdJ10sg/DpQq0uPIMxavp6S/t48KsVWL3jsJCiZV6wfFs+Ojw5DdNySovYmb3sZ/6+19XIn05PTcOLU6wXahMdkJBwSv6nnN3Yut9+bYxlntQmj/3C7f3yLFw/xlmLIXh4vl3CeiJO4MVz9OjJYsuTlUYXKZHlGnYcLB/K61QbzJXbtfMyf2NpO0srX7XAxYf14ThvvJ5wSn7YJ0vR77XZAMRbytH4cslWXPNu7Ep7y/4CLNp8APNc6v0aWj7YKrKd86Mni7Bgo7v9clfYiEYxainZOW9b9h/Df37Ntbz9hr3l5y2camhvfK/gNBUrD7R49pm7TUIpeaM+jVPJKJ8t2uJa27RHJpSvUhgLf3LJjVQsm3Z2kB9W7BQtQkQMS97O3Mh17/2Kf32fY7lC59YD3o7i3J5/SDQSSsn3eqk0HdnsBF04Hvt2Nd6aKX9JYruc++JMXP3OAkf2Zaf41IlTxVjwh/NWt8zPL8OSt6P33PLnD3w9cjN3s2wJcqG+ZmH+QuKfUDgJpeT3Bbkcbv94CUb/bF9B281KHb8gF+2emAIA2OaxxRQrZiczo2FHoT727Wrc+P6iMpUwnUDmUQrBvk/+pEtNSGTqC/vGDGsT24lAQin5UKbm2A8Js9vF58kfcgJ9LGVpWiwrE/Rs2+COVE7wrc0Er2CcdjUYIx8n9nqwoBBfLtnqmotRJO//shn3fr4MRR40XIk3ElrJO8H/HGyPVrtqZcf25UfSaqUAAKpXSXZ0v04q5n0OTVAbJDngkzfo+eLPeGTCKrR9fKrtfRnc/tFiT9oVxsKklbuwbo8zowsv8mC8wraSJ6JmRDSLiNYQUQ4R/V1fXo+IphPRBv1/Xfvi+hwVIhAVo2KoW0k5MmJY8l4VxTPLrHV55ZqSxzuLNx9AhyeneVajym2csOSLADzIzO0B9ABwDxG1BzACwExmbg1gpv5eobDNgWPOumucxOny0Ebo6k0fWMvMfWumPE1wwhGpDIRVyAFLyWiyvnDTftv7kgHbSp6ZdzHzMv31EQBrAaQDGAxgvL7aeABX2j2W31GGfGwY3hUZs0DX7Dxc8UomMCZcrYb9vmox09YMdgqwhfa+tYsTrQN/26op+SVx0AI0Fhz1yRNRJoCuABYBaMTMu/SPdgNo5OSxFAoZJ9mcrvUje79TWVm8+QCyLRZUm7tBK9bmZFb7qu2HAu5Gr3FMyRNRTQATAAxn5jKPZ9au/LBXPxENI6JsIsrOy5O3PZzbOKkcpq7eVfFKgnBSB5KEGtDpYMwkG99x72E5/fhu8uMq7dofMuZXXPuetTIfbnTuunz0PAwYFVuXMqdxRMkTUWVoCv4zZv5GX7yHiJronzcBEHYWg5nHMnMWM2elpaU5IU5c4uRk4l2fLnNsXzLy9dLt2O1g7L6TOB1CabV65b6jJ9H9hZmOyhIPzHOgRIVbGbf5BWLci05E1xCADwGsZebXgj76AcCt+utbAXxv91h+5p7Pl/kqbCsSTiUejfgm9oqhXiJLk/dYGtI7xfyN/pig9CtOWPLnAbgZQF8iWq7/XQpgJIABRLQBQH/9vSIKOQ5P2snI19nORFPMXpcXc5NuLxFZLVIUWw+41ynNLJI8Y6XCdiNvZp6HyIEh/ezuX2GfgsIiVK8iR8/2Q8edG614aa3GyjGL7pVQMkdMxpLH+lve3stnjUyusxXb8nEw6LooKWEkJVmf11jvUHKVSFTGq0954ce1AIAluQfQ/l/TMFuixI52T0zB2Ll/iBZDejbsjQ8FM2ud+YCJVdsPudYx6Z//WxF4vc9mTsVFo5wrwiYKpeR9yti5mwAA2blazO+vJhM7rJalrYhRM9bjxKkSvPCj9U47icL0NXsqXikCEgYeleGx71a51jEpPzh/QrlvlJL3M3lHTuLfU3VlavJi35OA4Xey4XRilUy46U5S9ejLopS8QNxukv3ubOUSiWe22GhTmYDzvwGCWy2aHcH6EaXkBTJ1tbyFtrxQEkZ/T0V4dvtwNLXr0HFkjpiMVTsOeXK8wxKWvvAapeQdZNuBgkDdi1jYHqZBspPYqSlSVOJ+yQCrza0TFTO9aGXwye/IP14ub2Cxx/VgZBvQLNq03/URfChKyTtIr5dm4SoTbfImr3K3/MBH83MDr81cVqu2HwpM2LrJpjx54qutMnPtHnR55ifXJqqDufGD2Pv3inbXbN1fgPNG/ow3BFfBlO0au37sQkxc6W3ZEaXkBeKlJWsmE/Py0fMw4htrjcTNIIO1aZcXflyL/IJT2H5QztaNoth1SBulutGT1wwfL8gVevxweN3mUyn5BMGJmh5O40Ttb0V4RD9ADZMi9DeWsaic31FKPkFwui2dE9hIRPSUD+dtxsQVO0WLEZ9I9BsfOXEK+46KbzjjtU9ejlz3BIOZMXeDt5a1DBe3HUQW/np20hoAwOWdm0ZcR7QPPJhNeUc9z9Rcs/Mw2jetHXgv0/kw6PjUTwCA3JGDBEviLcqSF8CMtXtx67jFosUQjpmaIi3+70cXJSnP0xNzMGd9fPY36Puq94XbLn3zF+wMan1oRHbJYsh/v3yHaBECeP0AVEreBQ6fiB6b68f4Zyscd6iYlxt8ND834oN416Hj5bpSiXI1FxWXCOs4FMrw/y4vt0wWF/wvHo+co7F+79FAbSkvUEreAT6evxm5+0pDtR6d4H5kih/4ZOEW0SKYJr+gED1f/BlPT1wjWhQAwHOT1+Ls52fg0PFTOHGqGE9PzBEmy6ng3AqJ3DUPfLUcXy/dLlqMABNX7MTYuZs8izxSPnkHeGriGlQOejLvPaIs9VgpLCrBr5v2o3eb+OgKdlgvlTwrpKqnCB/0RaPmYP2eowCA2ev24sCxwjK5EV7DrJX6/TJ7Gy7t0ARA+egarycdAeCbZeFdNbPW7UXNlEro0iwVJcxIqZRcxuXkNmt2Hsa5LRu4fhxlyTtEcF/IeAkTO1lULLwZ9sgpv+PWcYuxzESmsAwYP7FdnXVZpyaWtnt71saAggeARyasRGGR+Mbmf/pgET5ftBVHdJelzLfC7R8twXXv/YoLXpqFto9PRXEJ49yRP3t2/MLiEjw3aQ0OuVx6QSl5F5D4ui5D28en4ob3F5ZZtsjjgk6b92mKKr8gcoinCOsvVjbts5dReeM5p1na7uVp68q8ZxbfFSlYoRuiLPhDzgJhwRnKu/SmJ1539fp22Q58MG8zXp7mbtltXyv5vUdOIHPEZHy1xJmWc7Eis/USyhK9fMGGPUdw+MQpjNHr0HuFUctk39FCFJcwRk75vVy4p2jlFUykekBWf3Mnh+uiWw8yI2DhSPxcBgC0e2JquWVe37ZF+oVdVOzuyfK1ks/dp6UP/2+pt0pehmGzGU4WFWPAqLkYOmYhOqTX8fTYqdWrAADqVa+Cuevz8N6cP/DEd6vLrDNGoi5ShvJKolBfswBhgo8PZ3MJKifbU3l2iuOJwo6btVm9aqa32ayPApndzQPxtZIXxbKt8ldXPFRQ6gds+7hm1azZddjzLNQd+kRXchIFLJtTIfMEL01dV247txi/IBczbHRkEobD7pqJ951vehui0oddJFlEPwyjYScU9aL2jS1v+2X2NlfzQFR0TYJy87jYKxp6giQurid/iB6CKLGOQrGDGrRd49oVrxQCM3D0pBZ9dCjKHIus5O63Pr9Sq6q8qtQRS56IxhHRXiJaHbSsHhFNJ6IN+v+6ThzLCktyD2Lw2/NFHb4c+wWUGBjUsWwUx8rt4Zs2nDglxtVEkHuCFQBem74+8FqSZ1IABjt2/jqGcdk1rJViah+LPK4bL5q7ercULUJEnHLXfAxgYMiyEQBmMnNrADP198JYIVGDitdneF9jO9ba9e/NEeP/LusPlU2Farw5c0NAkToRJlsrxTnrj9m53qaf3NEdAPD3fq0Dy85vXfEEMRFQt3plAMCkkJrpBYVFWLX9EF6a6s8G7lUrJ4sWISKOXGXMPJeIMkMWDwbQR389HsBsAI84cTy3uaBNGto0rInxv+aWiX9XuIecar08pSV0BR2fOewDhuGcT96YDG9Uu2pgWaUYJmuihXHe9/lvmPn73vAfKlzFzYnXRsxsPM53A2gUbiUiGkZE2USUnZfnbEEoq8PXwZ2b4vHL2peLoDBDQWGR5W3jnaFnNzO9TTyFnTpB+ya18fGfzza93bgIGa3M7GoIZfUqsdmDxyN0yMreIk+ym+gEQK/xJLqGNW0b9gpk5rHMnMXMWWlpcqS2t2lUC0Bs1ksk+rtYCdBKluSwC1q4IEl42jauZXobAkk9qWkQSY+alf2V6zqj2+n1yixLqVTx7WiUPQ4liQg5O51tjm32wbt8W37E8GGnszrH3NzN8rZ5EebEho5dGHZ5vOOmkt9DRE0AQP/v2Vjt6Yk5yBwx2bLftG4Nza8YPFw1y85D7tWv+VOP001vY3bizA5WzvqfP16CeXqlQCeteudrh+vq3KaM4azudc9dYmufTvfl7Xa6sFiJCrn4TOshi+t2H3FQEvlxU8n/AOBW/fWtAL538VhlMIo0WXXXpKdqiQ2T7j8ftVIqYeaDvZ0SzTZ/69MSPVrUR+7IQRhxSbuYt3NrYqhJHe1BOH9E38AyM3XiDQqLSwJVKac7GKfuVsRO6Dc0+43tiDV7XXl7qaiEcdLhJLxGtUqNHBkiny7Qi9i9cl1nW/uZZ7Ps8IIRfTHjAXl0QkU4FUL5BYBfAbQlou1EdAeAkQAGENEGAP3193GBMQKoXqUSVj19MVqm1RQskUb/Mxrh4YGlij2tZuzW+R95RyteyQLGvEVwxl6NGP23XmDoJqdGB8b+/sizV7Mmkv/81RgU2G0fLbF17Fipo0fKyELz+tWRO3IQru2WYWs/H8zbbHnbOQ/1QdPUamjVsFQnfHbnObbkcRtHlDwz38DMTZi5MjNnMPOHzLyfmfsxc2tm7s/MngfO5uw87PUhy/DhvM3YuNe5oWGoorr6rHRc1D7sfHY53CpBm6xb7cFK68qu6Whg4gEUDiMmXQYLMhZ+MxmiG0nJpwpWrLP+2SfsctGVVe/v1xoPDSw7cu2eWS/C2u7RuE55F+55rdwvF2wHX5c1+GnNbsf2ZWWy89lJazDw9V8ckyE55EYjIlza0VqpWqcwPDPBoXPJSYRhFzS3td83Z2q5BD843EC7Qc0qptb/fNHWMu8jPXIe/nplTPszXIEyFV0zWPxYPzRvUCPsZyIftl8O64EHBrRBzZC8ghev6ei5LKH3YDwgz7jaBY5byN40bsJQrFoyRQ7ezeGKPoksBDXy6o7omFEHY+Zswmn1quOXhy9ESmXNbghtFmGV/Uetpcff3acl8gtOBUY/913YCpd2aoK0min4aH4uRs/aGNN+/u/bsl2+dthsKtGodoq+j9LfLaNuNWw/6F2zinD8tXcLNKwVPdCgXeNa+F3ApOU5LeqHXV67qvejnkrJFdvFqdUr47u/nYc+r8x2X6AY8JWSZ2bsOVwaHuVklqsMz+9w0Q4ivRlDu2u10N+8oSsAoFm96oHPRFchfCRoaB8aYdOotnVX0mcLt1a8UhRqBKzR0itq8v29otbTd5vBXZri0UvOqHC9ZK+r11WA6GssEpn1ayAzwohIBL5y14xfkIseL860tY9IBnuHdK1gk4g2dWedlqr/L6/k7YR5uokTfsqTRcXSJUkdPWkv3vvVIZ0xvH/rwG8KAHWqVcbp9TWlIOL7Xp9VcfLa+a2tX/fxMq9iljeGdsFHt5VPaLu7j1x1bHyl5J3oQhPpJquUpJ2qSD5LN4nmKpJ10qdONftDaTulX91i4abS+IETEbI7IzHm5m5oWKsqhvdvI3wiswwxiDIgxgn+cFjV8VP+3qtMaG4ohn+8amUxamxwl3Rc2K5hueVWY/gPHnNnNOcrJe8EkSx1Y6Rq1yoxu/3zV3Uo3dbWkb3FCeOtuIQxbr71cDe3OV5oTsnbSeBxk2iGy2n1qmOQHnRg9blktQRyg5opEefIAKB+zRQ8e2UH/PxgH2uCScaU1c4FigTjK5+8E1zWqWnY5Sl6MlGVGFLPozH659gm/AxuOud0fKt3m/fpqDci7/+yCdsOOD8h2aqh+bIL4ahks3tSOE6r5/1IsX6NyHMUcx++MPDaqcn0WInF536zhexvWdnkVi6LK3sVxOx11gqcff6X0mSGHhFm8q/tloH7+7bC8P5tLB0DAEZO+R2vBtUkjxWZRvYGN3Q3X4TMLJ/anOSMRM+W4X9juzhR+KpVw5p4bYi9jE6zuD2harVwWkqyvOV73WCzzabwkfCVki+0cJNVTqZA5mjLtMhWVOXkJDxwUdug6IjYMSpSWq3Vfp0+MZbZoHoFa3pHiwbRs4Blrq8NOOPHDVVdTswJAeETbtygT9s0LHmsv+tK3uoItFoVua+hUD7/yzmYNvyCwPtBJnNr3CrF7Bslb7W076d3nBOYBHPLG3LSYreldno1xyFZzZA7clCFccwA0EKS0K20Win45I7uWPp4f9GihKVtI/sum1Dl9alee8cudkpcm6Fx7apIs1C4LrO+OWPDqpJ3s3SyG5zbskGZCqyjb+iKCXf3FCiRhm+UvNWco5pVKwXcIbFeU3Me6mPtYCaxkuwRrTjY+j1HcM/nyxxxK8QyN9GrdRrq2yxv4BYf397d8X06VSDMKyX/r8vbm1rfEOuNoV3Ro0XsJQXO+NdUU8cxiDclHwoRBaLyRCJeAoewGvXCXHpTxXpvnV6/BhY/1s/S8Uxh4V4/s2nkBszD/7sck1fuspW12LROVQzv3xo36IlQ8UrdGl8egm4AAB3jSURBVObKG4TjmYlla7uv3hG5nvv395wX836re+SmiLURiEErvVBfzaqV8P4tWW6IVCbSR7bSDxdYyJHpmF4H91zY0nZRNTuo6Bpow8+/XtAikMEZC7G4TgzmbsjDBD1Cxgyx6vjnruyAZyauwVd39URm/er4fnn4ei+GZbTfRjxu1crJpiefv7vnPCzZfADP/7jW8nFlZMKy7WXeRzqv79+Shc7NUsN+Fg4rTVe84PmrOuLqszJcrcoaPD8gUxJV/zMaBjK7zZCURHjo4nbILyjE10u3V7yBCyglD21Y9eilFad1W+Xv/13u2r4BrYmI0Ugk2o1xSnfT3DpusfWDWRhddGmWio7pdTxT8laKybmJE4lhMlCtSnJMDb3tYFy//do1RC0BtWki0Tkj1fTIJ5jU6lXQuVmqo6VWYsU37hqFRrRMyhMWJ4CdwIuyJ0mk1akZfeNZMa3/3p/Owrd/O9cVWR4fdAYqW4yjlzBi1nMeuMh6qLIbNI2SlBUrLqRVxIRS8hLTIb2Oo/srFujk9CKN36zfc2CHJugaph6QE9zZqwWev0orhWs29FWqkgceYyQjmnGHesHVZ6Xb3odXE+qh+MZdc/BY7IWjKiVRoASwRG6/Mjx6STv8+Xx7NdlD2X3Ygb6zkp4vwH42stMMyWqGITEU/wrFC1VwRpPIE/QiGd6/Ne7s1VwqVw3gzIM3e4uzPXhjRa67wgZrd8feBWrtswMDMeiylivtmFEHlWOoXe01Mp6tgXpNmLaN5VRcMtA3pJCWFxnLViAi6RS8UzxxmbmQVaeQT4tYxMxMfOXkJFfqjijEUE/v9uSXX9SNUf3AkOJofjlX8cT5girG+kbJ/7bV+1nrRERG5ZCqR6+EtoeLVzzxySew3z/RcP2uIKKBAN4AkAzgA2Ye6cZxzNb2NirqyeiTv+3cTJzT3J0iWnaxU1fcLe7v1xpN6lTFFZ3DVxBVlHdLKhWfOLhqyRNRMoC3AVwCoD2AG4jIFceU2cARmQ2Zp644U7pWawbto2TUiqJq5WTc3DMzakmHaOQ8fTF+CSqpKxNOZVbLaMwkGvVNNpF3CrfdNd0BbGTmTcxcCOC/AAa7cSCzdS7u6q216JKpF6OsjLre29K3XlMjpVLU5hQiccoF5XcdX0XCIIVQGgiq4+T2mUkHsC3o/XZ9meOYVfKXdmyC3JGDbGUjTrrv/DLv5z0ipzVol6u6lsaf160uxhqJREMLVRTDIfPIzglC74/QaBs7vHKdeCNg+gMXVLySSdY9N9DxfYpA+OOPiIYRUTYRZeflWWv6AcCVDkIV0T4k1jijrjz13g1Oq+eMTEYMepfTYq/BEkqtqs5PATmV3yVbAtLL13ZCnWqVkVLJmWJlho5PT62G3JGDHMngNHCq+Na7N8WWqRwOoxG6k8hQQdIJ3P4WOwAEB+Rm6MsCMPNYZs5i5qy0NOsd4edt3Gd5W6tIphfCcm/fVo7sx4nhcKeMijN4Zz7Y29Q+ZSpi5STXZTXDiicvknZuBgBG39gVo280X7QrEpd0dKbmUDXJG9Z4jdtKfgmA1kTUnIiqABgK4AeXj+kZsll/BlP+3ivw+jqHrCzjm9rRqbFsa7bMbpNUudLfZcU49U5espd1ahqxJ7JIInWIe+jitqb2I+fdbR5XlTwzFwG4F8A0AGsBfMXMOW4eU1H2Rrb7IAq4WDy64qM1lTYIbt1X1SF3RkX89YIWnhzHNfQnrKR2iaPcdm4mAKB1w7IlkVOrm5t/88u5ct3pxMw/MnMbZm7JzM+7fTwF0MjB4k6h8w52wjRiuWnM1p/xwlnTu00aLjcRg++0cnjuyg6Wt03Rz6fsTq2PbjvbsX0Zyj20V25JjBM4DWpWwfrnLpF2pG4WX8ws5BdYb4LhFL89MUC0CAHq1qji2CSn4WIZ3EVTcik2GmA75T4P3o8XLeLmrM8z1Zg81eH68X/qcTo+vt2aEuzRQkuqk3nqolHtFFzoYLSPocsrhcxnxDpJT0QBY2PZEwPw66N9HZNNBL5Q8mYmXZ0KuQvFaCdXz4G2ck4w88HemHC3Viu9uY1cgBZp2rZPX9EBy/81wJSyC+XCttFvZCsJSZ0zrEf7mKF2tdgfmm7o0z5tG+IvvcxXJTWMUSMR547znK1saoUaIfMu4//sbL9d48FfKSRYoFHt2Ea4915YGqxQr0YVNKkjZw5FrPhCyVdkpfQ/ozQV/7/Dergqyw/3xt7L06B6lWR0yqiDSfedjzE3d3NEjoa1qqLb6Vqt9GcHWx/uHz6hlXBOTiKk2oyRv7MCJRWLtZmeWg2f3nlO4P1jg9zr6BWMmfrmblnNjw0qnyz+VJhm3AtG9MWFbdPw+vVdcJZeL79lWk3kjhyE21xS8rf0PD3mdYuCTOqZD/ZGO4eqh6bpBpzhogodUVWNcRR6q+7TdwMR9ZX8UdGpAh69tB1mrN0DAGjhYn9KQIuV73paqqmCaWueKU26cLpRCAC0a2K9Z+jr1zsXIleRjzOWss8vXdsJZ2fWC7yXsRyzVzSuXRW3ndccT4U0FG+aWg0f3a5ZxyUljEs6NEbrRu72jTXTGq9909qB+8PJfrHf/u1cLN1yEIM6NsH+Y4W4sks6/hfUV/XMps7fW2YRERGbEHeI1/7I+pK4bAysplO3aljT00Ycsbi6vChDcVfvlrZS0L2K3Z86vFeF6yQlkesKHgD6tI09x6Wfg/73YDLqVsfgLumolJyEu3q3RI2UUrdQ7shBAUs/Gm7PtQ5o37jilRzGF0o+2i3VuZk3PttgRNWNdppmdb3zReaOHBS1WURG3WpY/9wlntSYGXFJO5xpoxCbFyq+TrXKtt1nTmJM8MZCcYn5fAgr1EyphEpJZCrre8EIdydZR17T0dX9h8MX7pq9UdraFZeUoJqLF9TEe89Hg1plb7YaPqlr3ruN9QzkSEy4+1zsyD+O+7/4zdR2P9x7fplRxeODzkD35vWibCEQD7R8rOGAXpKcRGX6CLdIq4FNecfKrZfZoDoWjOiLgkJz5cHNQkTY+MKlprZxe5JVhHvRF9royyXbIn5WXKJN1o27LauML9cpOoZJ1ZfB92eX2f/sg9PrO1+Lp9vpddHt9LpYtT0f7/+yOebtQl05d/ZyNznJzrDdizZvxRFcQm7UB4qVZCIUBz3hWjTQlHy7xrXw++4jAIDrs5rhis5NQURIlazUk1/LIfjeXWP4R/u2a+RZ70gZa66bJaNuNVeTQR4b1B6rnrrItf2LIuv0uhhytvv9U7/6a8/A62cGnxl43cYD/3skQi+XC9s1xNLH++PSoJo0Q7s3kzbJyK8tQX2h5KP5wIslHNYqNCI9dEPjqEWQZFERDezgzcRacBTWLT0zA6+fDBNS6RWhXcMy6lZH/ZopZSJK7JT2doOJ956Pl67phOH9W+Pru84VLY4r+ELJGwk74ZC5ip/MeGVtDerYBG1DrE8ZLL1nLZYScKq0s1VEJuO9OqQz5gdNXHbX3aPBv6fbIcwVETqx2jGjDoac3QzD+7dB28biRkGAe642Xyj5aBFrvVr7I9LFr7x901mY9o+yDR+cTHG3iqydoipCpFGTUim5zHkzdLvVUZEbNE2thkwX5prsYLjb3DpLvlDyofVL3r7RevMBhXheua6TrRBGq0y893xMG26uw9D0fzjfkcgOJEGB3NAHjfF24Jnex4iHQ6aHDoBAA3q3RrC+UPKhlvygTk0ifqaQn5RKyZ4mYRl0zKgTdsge6k4KJjTRyO2Jz5kP9saMByI3VpHBOxkqQn09scxOVzFHEXyOnr7izIpXchBfhFB6UYnQLbxIColHDh0/JVqEAGayOWPJqrRDRWUAZJjPMDBEubprOpKo1GIVjWhL/tZzM/HkD6VtNYzRl1t1bXyh5ONYx0sru2hVIfpGDObhge1iXld0rLUMlryBobySkghXn+VMhzInkOEcpadWw458rS91neqV8figM3CRSyUPfOGuiWbJS6pDpUf0eZPgPgxgZjIzSbAGkcGSd6PVoJPIYECE1h26s1cLnObShLAvlPyGvUdFi2CZWCovJiIS3IdxiQxWqoFEopRBhgehV4mZgE+U/O+7D0f8TFZ3iOyIvg1kiBKJR2RQYAay3nrynCFv8IWSV4rcOkZjkWAW/18/CdwOQg8ft8hw3l6+thPSU6uVa78nC0m+0Hqx44uJ13C0algTG+PAjVNPcLnYcP7mhjG2SUtUljzWH2c/P0O0GGGRQa1efVaGVBOtocjgkwe0tofHXa7ECdi05InoOiLKIaISIsoK+exRItpIROuI6GJ7YlYkR/llQ/UiUbL6vNvp8dh3B/WTFEGyJBd8KE3qyPugcTtM0gpGKK4sCkxmZHFp9W6T5kmtI7sDl9UArgYwN3ghEbUHMBTAmQAGAniHiDyNLTN+SNlcOUZTbUO+yoKHtLJW3ruya7poEeIKI8JMKfmKkdSL5Bq2lDwzr2XmdWE+Ggzgv8x8kpk3A9gIwNmW7EHUCNNfUtbfMbglmQwYRaQ+vv1swZKURUTD43jGKLaqdHzFJNqD0K0piHQAwZ08tuvLykFEw4gom4iy8/LyLB3s3T91s7SdCGSKGpl8//m458JWmHz/+ejTVnxRsGDOMdFOTgSvXtcZn995Tpllk+8/X5A0wHNXdkC9GlVQJYEbm8dKolnyFZpLRDQDQDjH0WPM/L1dAZh5LICxAJCVlWXJuSKyvKpdRHqTmjeogaQkkrKTlYz34dThvXDgaCEA4Jpu5ScWRfrqh2Q1w5As95uV+IF+ZzTCktyDosXwjAqVPDP3t7DfHQCCr7gMfZlnGCMylswpL5NcMo0q4oF2jaNXxmxYS97JYkUptT1MRJIBt8Z2PwAYSkQpRNQcQGsAi106VhmMqBXDOu3mQl9XO8ikVmV2TcosmyK+SbRry9bsFhFdBeAtAGkAJhPRcma+mJlziOgrAGsAFAG4h5ndDwhFadx39+b1sPj/+qmY7yjIfLGrUYbCLZRP3gTM/C2AbyN89jyA5+3s3wrBiktmBd+9eT38vvsI2jQS1w5NZkUq8wNIEd8kWnSN7+LU4uUHvKprOv7WpxUaC0z6STSLRqEA5EmG8grfKXnpCbrARCp4IPEudrNMG34BduQXiBZD4TBdZelQ5RFKyQtCfGyN3Ja8DCOyto1rhW0HqIhvKuqu5Td8lzkhXjVERyb5ZLbkZX4AKRTxhO+UvOws35YPANh35KRgSeTGTDcm0aSnVhMtgkIREd+5a+rGSfbrkRNFokWQGplHGaH89I8LcOKUJxHCCoVpfKfkrw2Tbi4TnTLqYOX2Q8jUq1Eq4p8aKZVQQxVUiyv+d1dPNKgpX8loN/Cdu0bW+ugGHdK1TNzU6t6mVqvCVQpFKWdn1guU/fY7vjM/ZNPxd/dpiXdn/xF4/+Tl7XF113TPZ/iTkgBU4FF456azAnXJFQqFP/CheSeXlv/nRW2RUbd0Yi6lUjKyBNTTiSW79dKOTXBZp6YeSKNQKLzCh0peLpKTCPMe6StaDHTM0NxEWWEadysUCv/iO3dNHEXeecoN3Zth8eYDuK9fa9SvUQUrtueLFkmhUHiA75R8j5ZydxQSxZVd0tEpIzUwF2BMACsUCn/jO3eNDOnwMkJECZfOrVAofKjkFQqFQlGK75S8igdXKBSKUnynEatU8t1XUigUCsv4buJVFmY80BsL/tgXeN//jEaYsXaPQIkUCkUiopS8S7RqWBOtGpZOdI69uZsUNeQVCkVioZS8RySpAH6FQiEAWw5sInqZiH4nopVE9C0RpQZ99igRbSSidUR0sX1RFQqFQmEWu7OU0wF0YOZOANYDeBQAiKg9gKEAzgQwEMA7RJRs81iKBOWFqzqKFkGhiFtsKXlm/omZje4XCwEYxdwHA/gvM59k5s0ANgLobudYisTF67LMCoWfcDLe8M8Apuiv0wFsC/psu75MoTBN8AS2QqEwR4UTr0Q0A0DjMB89xszf6+s8BqAIwGdmBSCiYQCGAcBpp51mdnNFAlBJTVorFJapUMkzc/9onxPRbQAuA9CPOdBxYgeAZkGrZejLwu1/LICxAJCVlaWiDBXlKC5Rl4VCYRW70TUDATwM4ApmLgj66AcAQ4kohYiaA2gNYLGdYykSl8qqVIVCYRm7cfKjAaQAmE5a9ceFzHwXM+cQ0VcA1kBz49zDzKqdvcIUq566CPM37ldNzxUKG9hS8szcKspnzwN43s7+FYlNraqVMbBDuOkghUIRK2ocrFAoFD5GKXmFQqHwMUrJKxQKhY9RSl6hUCh8jG+qUC57YgBOFZeIFkOhUCikwjdKvl6NKqJFUCgUCulQ7hqFQqHwMUrJKxQKhY9RSl6hUCh8jFLyCoVC4WOUklcoFAofo5S8QqFQ+BgqLQEvHiLKA7DF4uYNAOxzUBynUHKZQ1a5AHllU3KZw49ync7MaeE+kErJ24GIspk5S7QcoSi5zCGrXIC8sim5zJFocil3jUKhUPgYpeQVCoXCx/hJyY8VLUAElFzmkFUuQF7ZlFzmSCi5fOOTVygUCkV5/GTJKxQKhSIEpeQVCoXCxyglr1AoFD4mLpU8EZFoGYIhomT9v1RyAQARSfcby3iegNLfUUaIqI7+X5rfk4ga6/+l+j2J6EwiqipajnAQ0XlE1NLLY0pzwUSDiHoS0ZtEdBsAsCSzxfoPNh7A40RUTyK5uhPR/QDAzNK0y9Lleh/AI0QUNjtPBESURUSfAPiX1zdgNIgoiYhqE9EkAG8CcvyeRNSViGYCeBaQ6n7sRETzADwHoL5oeYIhorOI6CcAPwOo4+WxpVfyRHQtgNEAlgDoR0TPEVEHwWKBiFoAeAfALACnA3iWiAaJlQogouEAvoX24LlEXybUQiWiZCJ6EVqI2HwAZwF4kogaCZYriYhGAxgDYCaAJgCeIqLqIuUy0BX6EQCVAaQT0fWAOGueNEYB+A+A8cz8FxFyROFxAF8z81XMvAMQP8ogospENAbatf8mgGkA+uifefI7Sq/kAZwJ4Btm/gTAQwDOAXAdEaWKFQvdAKxl5o8BPAhgOYDLiKiZUKmAjQAuA3A3gEcBgJmLBV/sSQC2Ahiin6/hAHoAqCZQJkOJ/gygny7XSwAYQJFIuUJoB62eyesAbiKiWsxcIuL31C32mgB+Y+b/AAARtRTtQtIf1i0AHGXm1/VlA3QdIdqVmgJgDoBezDwJwAQAZxBRJa9GZdIpeSIaQkQPEFFPfdEBAFWJqA4z7wawB5rl3DPiTtyRqwcRtQlatARABhE1Y+aD0CzUfABXC5ZrMoCV+v+jhtsG+sUuSK4SAF8w83oiSmHmnQC2QyvI5Cmh54uZv2HmfCIaACAbmjX/AhGdIVK2IKW0EUAhgM36361EdJpXLpIw19eDAM4hoieIaD6AlwF8TETdvJAnnFy6stwHoBcRDSKi7wD8E5rl/JC+jmcupZBzdoyZP2fm4/r7SgCKmbko4Sx5fUj/LwCP6IveJ6KLASwG0BDAB0T0FTRldQRAI307V5/QRJRKRJMBTAcwhIhq6h+dADAPwBD9/ToAawDU82LSJ4xcNYyPmLmYmU8AeBXAHUTUgJk9sU7DnS9dnnwAYOaTRFQLQHMAO72QKYJcNfTlxvVzEMCNzDwAwDFoytQTd1I42YKUUhaAw8ycAyAHwJMA3tXdAK7dv5HOFzMfBvA2gGuhjRRvALALwDVezLNUINdH0OYJxjHzxQA+ANCDiHq4LVck2ZiZdTeX8VvNAXAVEdVNOEuemYsBtAXwIDO/BuApaFbDEWgX09cApjLzDQAWAbhE387tJ3QNaH60+/TXF+jL8wAsBNCRiLrr8u8AcJ6uYN0mrFwhF85sXcb7AG3iU4BcvcKscw6AHGbeSUQ1iai1ALmM88X6/2xm/lFfdwqArgAKPJAromw6WwHUIqIvATwMYCmA9cx8ymUlEVEmZn4TQB9mnsvMJwF8B+1h5MX5inauJgHIBFBXf58NbeR/0gO5IsrGGiW6os/V1+ntkUxilTwR3UJEvYP863sA1NX9VV8D2ABgKDMfYOYvmXmcvl5baBeW23LV1idwxgL4Cpr13p2I0nWl/iuA3wCM0i38MwFsdWvirgK5ziGipvp6BAQenM9Bi2Y5BOAsN0Y+JuSqpG+SCmAbEd0Oze3VxWmZzMgVhm7QrFPXRj8mZKsLIA3AbmgPnrsBtHXDnWTmfOkuSoNu0NxvxU7LFKNc6bpMK6G5Z+4logYA/gSgA4D9bsgVo2yBe1J/KKfom54wlrslW0BGr6Of9C/VGMDn0Hy1f0B76v0VwP3QfFZv6n7SttBO2EBm3kVE/aD52TYDuJuZt3kg19+ZeZ++znnQ3DPZ+kSwse1rADKgzRXcwszrBMm1hJk/1ZclAWgBbQhbCGA4M68SLZe+/BMANwEYD2CUfnMKlYuIakMbYbwATaE+yMzrnZLLgmyBa0x3txmf1wRQhZkPCJAp+HylQJsXewXaA9HR82XzfnwA2rXfGsA/mHmNU3JZkC34nCXrQRCfAtjIzE85KVdEmNmzPwDJ+v82AD41lgF4F8A4aBbeVGjDnOr651/qJw8AWgK4ykO53oIW2RO87j+gWcd1ANQKWreWRHIZ564hgAslkas2gJr6sqEArpVErjoAqurL+gMY7LRcNmWrEbRukiQyVdOXnQvgSonOVa2g5ZUl+x2rBy2v4oZsEWX25CDaSXgBwL+h+aIuhxZnG/z5XmjujlugxZ9fr3/2GYBzBMmVBM2q6x20rCa0cLbF0NxLTSWVK0MyuZbocjWRVC7Hf0dZrzFZz5eM5yoeZKvoz3WfPBH1hjZZVBdaONizAE4BuNCYCGTNd/w0gJdZi7/9CcAtRPQbNPeNY24Gk3KVQJsAfipo00EA/gZgBYCOrIUDyijXdsnkWq7LtUtSuRyP9JHxGpP1fMl4ruJBtphw+ykCLbri5qD370CbPLoNwNKgp2BjaBE0zfRljQG0kESurwBk6ssGA7hAyaXkikfZZJRJZrlkly0m+V0/AFAd2oyy4cu6CcCL+uvlAO7TX2dBS5jx5osruZRcCSibjDLJLJfsssXy57q7hpkLmPkkay4ZABgALcYcAG6HluI7CcAXAJa5LY8dubwId1Jy+UMuWWWTUSaZ5ZJdtlioVPEqzkBakSyGlqn6g774CID/gxbLupn1okJeYkYu1h/XSi4lV7zLJqNMMsslu2zR8DIZqgRaNb19ADrpT74nAJQw8zwRCl7JpeRKYNlklElmuWSXLTJe+oagVR4sgVbz5Q4vj63kUnIp2eSXSWa5ZJct0p+nGa9ElAHgZgCvsVbzQgqUXOZQcplHRtlklAmQVy5Abtki4XlZA4VCoVB4hzRVKBUKhULhPErJKxQKhY9RSl6hUCh8jFLyCoVC4WOUklcoFAofo5S8whcQUTERLSeiHCJaQUQPUgU9UIkok4huNHGM+voxlhPRbiLaEfS+OxG9af+bKBTOokIoFb6AiI4yc039dUNoXXvmM/OTUbbpA+CfzHyZheM9BeAoM79iTWKFwhuUJa/wHcy8F8AwaL0+SbfYfyGiZfrfufqqIwH00i3xfxBRMhG9TERLiGglEf011mMSUR89zR1E9BQRjdePuYWIriail4hoFRFNJaLK+nrdiGgOES0lomlE1MTpc6FQKCWv8CXMvAlaN5+G0LqODWDmswBcD61PMACMAPALM3dh5lEA7gBwiJnPBnA2gL8QUXOLIrQE0BfAFQA+BTCLmTsCOA5gkK7o34LWBrEbtPaXz1s8lkIREc+qUCoUAqkMYDQRdQFQDK0/ZzguglZ46lr9fR1ozaA3WzjmFGY+RUSroD1spurLVwHIBNAWWuXC6XpV2mRoDbEVCkdRSl7hS4ioBTSFvhfAk9B6bHaGNno9EWkzaA0gpjkgwklAawtHRKe4dPKrBNp9RwBymLmnA8dSKCKi3DUK30FEaQDeAzBaV651AOxirQ/nzdCsZkCrBV4raNNpAO4O8pm3IaIaLom5DkAaEfXUj1WZiM506ViKBEZZ8gq/UI2IlkNzzRQB+ATAa/pn7wCYQES3QHObHNOXrwRQTEQrAHwM4A1orpRlemefPABXuiEsMxfqbqE3iagOtHvxdQA5bhxPkbioEEqFQqHwMcpdo1AoFD5GKXmFQqHwMUrJKxQKhY9RSl6hUCh8jFLyCoVC4WOUklcoFAofo5S8QqFQ+Jj/BzflHKHsoss2AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zGRalvtDk1jF",
        "colab_type": "text"
      },
      "source": [
        "### Prepare the data\n",
        "\n",
        "To get the data ready for modelling, we need to:\n",
        "* Normalize the data\n",
        "* Convert the dataframe into a 2D tensor of shape (timesteps, features)\n",
        "* Get ready a training and validation set\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k9Tl9_Pe4VvC",
        "colab_type": "text"
      },
      "source": [
        "A supervised learning algorithm requires that data is provided as a collection of samples, where each sample has an input (x) and an output (y). We need to transform the time series data into a supervised learning dataset.\n",
        "\n",
        "**For example**: To make one-step prediction using past 3 days data for time series = [1, 2, 3, 4, 5, 6, 7, 8, 9]\n",
        "\n",
        "We want the data to be in the format:\n",
        "\n",
        "[1,2,3] $\\Rightarrow$ [4] <br>\n",
        "[2,3,4] $\\Rightarrow$ [5] <br>\n",
        "[3,4,5] $\\Rightarrow$ [6] <br>\n",
        "[4,5,6] $\\Rightarrow$ [7] <br>\n",
        "$\\vdots$ \n",
        "\n",
        "Similarly, To make three-step prediction using past 2 days data for the same time series, we want the data to be in the format:\n",
        "\n",
        "[1,2] $\\Rightarrow$ [5] <br>\n",
        "[2,3] $\\Rightarrow$ [6] <br>\n",
        "[3,4] $\\Rightarrow$ [7] <br>\n",
        "[4,5] $\\Rightarrow$ [8] <br>\n",
        "$\\vdots$ \n",
        "\n",
        "To do this, we need to define a generator function:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qi_utuf1sgTt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def generator(data, look_back, pred_step, min_index, max_index,\n",
        "                shuffle=False, batch_size=1, step=1):\n",
        "    if max_index is None:\n",
        "        max_index = len(data) - pred_step\n",
        "    # i represents batch\n",
        "    i = min_index + look_back\n",
        "    while True:\n",
        "        if shuffle:\n",
        "            rows = np.random.randint(\n",
        "                    min_index + look_back, max_index, size=batch_size)\n",
        "        else:\n",
        "            if i + batch_size > max_index:\n",
        "                i = min_index + look_back    # start over\n",
        "            rows = np.arange(i, min(i + batch_size, max_index))\n",
        "            i += len(rows)\n",
        "\n",
        "        samples = np.zeros((len(rows),\n",
        "                            look_back // step,\n",
        "                            data.shape[-1]))\n",
        "        targets = np.zeros((len(rows),))\n",
        "\n",
        "        # j represents sample\n",
        "        for j, row in enumerate(rows):\n",
        "            indices = range(rows[j] - look_back, rows[j], step)\n",
        "            samples[j] = data[indices]\n",
        "            targets[j] = data[rows[j] + pred_step][0]\n",
        "        yield samples, targets"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X1RjptxiJbPa",
        "colab_type": "text"
      },
      "source": [
        "Note: in `range()` and `lst[:]` the right limit is not included.\n",
        " \n",
        "\n",
        "- **data**: The original array of floating-point data of shape (timesteps, features)\n",
        "- **look_back**: How many timesteps back the input data should go\n",
        "- **pred_step**: How many timesteps in the future the target should be\n",
        "- **min_index** and **max_index**: Indices in the data array that delimit which timesteps to draw from. This is useful for keeping a segment of the data for validation and another for testing.\n",
        "- **shuffle**: Whether to shuffle the samples or draw them in chronological order\n",
        "- **batch_size**: The number of samples per batch\n",
        "- **step**: The period, in timesteps, at which you sample data. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yOpT60txEoEc",
        "colab_type": "text"
      },
      "source": [
        "**This function can be illustrated by the image below:**\n",
        "\n",
        "<center><img src='https://drive.google.com/uc?id=1SCd0rbpJL4DQL871mZf2BVLwAhW5TJDu' width=400></img></center>\n",
        "\n",
        "At index t:\n",
        "- input data is: $[x_{(t-\\text{look_back})},...,x_{t-2},x_{t-1}]$\n",
        "- output target is: $[x_{(t+\\text{pred_step})}]$\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "#### Example:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZUF-dG05sk7c",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        },
        "outputId": "31363927-7a26-44a5-e739-a8b63e58af4c"
      },
      "source": [
        "example_ts = np.array([1,2,3,4,5,6,7,8,9])\n",
        "example_ts = example_ts.reshape(-1,1)   # univatriate data: only 1 feature\n",
        "pred_step = 2   # 3-step prediction\n",
        "look_back = 3   # look back 3 data points for each prediction\n",
        "example_gen = generator(example_ts, look_back, pred_step, \n",
        "                        min_index=0, max_index=None)\n",
        "i = 0\n",
        "while i <= 3:\n",
        "  display(next(example_gen))\n",
        "  i += 1"
      ],
      "execution_count": 115,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "(array([[[1.],\n",
              "         [2.],\n",
              "         [3.]]]), array([6.]))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "(array([[[2.],\n",
              "         [3.],\n",
              "         [4.]]]), array([7.]))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "(array([[[3.],\n",
              "         [4.],\n",
              "         [5.]]]), array([8.]))"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "(array([[[4.],\n",
              "         [5.],\n",
              "         [6.]]]), array([9.]))"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2-7cTA3cPDKN",
        "colab_type": "text"
      },
      "source": [
        "Illustration for this example:\n",
        "\n",
        "<center><img src='https://drive.google.com/uc?id=1LFkGPk8iwLY6ZsOCP8eWoaTej4iKXuHP' width=600></img></center>\n",
        "\n",
        "Let's use the first 300,000 rows of the data as training dataset, and there remaining as the validation dataset. \n",
        "\n",
        "This amounts to ~2100 days worth of training data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HFYcuoJ1Prld",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "TRAIN_SPLIT = 300000\n",
        "# Convert pd.Series into numpt array of floating value\n",
        "uni_data = uni_ts.values \n",
        "# Only use the training data for normalization\n",
        "uni_train_mean = uni_data[:TRAIN_SPLIT].mean()\n",
        "uni_train_std = uni_data[:TRAIN_SPLIT].std()\n",
        "uni_data = (uni_data-uni_train_mean)/uni_train_std"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f7JakGh00FIV",
        "colab_type": "text"
      },
      "source": [
        "Let's now create the data for the univariate model. For this univariate model, the model will be given the last 20 recorded temperature observations, and needs to learn to predict the temperature at the next time step."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zEB-stIKy-OK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "uni_data = uni_data.reshape(-1, 1)\n",
        "look_back = 20\n",
        "pred_step = 0\n",
        "BATCH_SIZE = 256\n",
        "\n",
        "train_gen = generator(uni_data, \n",
        "                      look_back, \n",
        "                      pred_step, \n",
        "                      min_index=0, \n",
        "                      max_index=TRAIN_SPLIT,\n",
        "                      batch_size=BATCH_SIZE)\n",
        "\n",
        "val_gen = generator(uni_data, \n",
        "                    look_back, \n",
        "                    pred_step, \n",
        "                    min_index=TRAIN_SPLIT+1,\n",
        "                    max_index=None,\n",
        "                    batch_size=BATCH_SIZE)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BVxzCwmTz7l4",
        "colab_type": "text"
      },
      "source": [
        "Let's take a look at a single data point in your training set:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SpUNdN5F0xpn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "8087e93b-3550-4c65-90a6-952f3b24224b"
      },
      "source": [
        "first_train_data = next(train_gen) # whole batch: 256 data points\n",
        "print(len(first_train_data[0]))  # input data \n",
        "print(len(first_train_data[1]))  # target value data"
      ],
      "execution_count": 109,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "256\n",
            "256\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g-nDtkTr0203",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 381
        },
        "outputId": "4e7bce74-3bd8-44f9-b4de-002365b55b68"
      },
      "source": [
        "first_train_data[0][0]  # each sample has 20 data points (size=look_back)"
      ],
      "execution_count": 110,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-1.99766294],\n",
              "       [-2.04281897],\n",
              "       [-2.05439744],\n",
              "       [-2.0312405 ],\n",
              "       [-2.02660912],\n",
              "       [-2.00113649],\n",
              "       [-1.95134907],\n",
              "       [-1.95134907],\n",
              "       [-1.98492663],\n",
              "       [-2.04513467],\n",
              "       [-2.08334362],\n",
              "       [-2.09723778],\n",
              "       [-2.09376424],\n",
              "       [-2.09144854],\n",
              "       [-2.07176515],\n",
              "       [-2.07176515],\n",
              "       [-2.07639653],\n",
              "       [-2.08913285],\n",
              "       [-2.09260639],\n",
              "       [-2.10418486]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 110
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7H549C3cy-fd",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "outputId": "51b701f4-eb94-4be1-84a0-af938d8ced9f"
      },
      "source": [
        "plt.plot(range(len(first_train_data[0][0])), first_train_data[0][0], label='historical data')\n",
        "plt.plot(len(first_train_data[0][0]) + pred_step, first_train_data[1][0],\n",
        "         color='red', marker='+', label='target value')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": 111,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deVhU59n48e89rCKLgoACKriLCKhojJqYGJMYazRbY9JsNott07RJ3ze2tskv65s2bbqkWZqtpllqs1fNYpq4ZDdGMEGDOygGRAVBEFD25/fHDJaYAQZmhbk/18XlzDnPnHM72z3Pec65HzHGoJRSyn9ZvB2AUkop79JEoJRSfk4TgVJK+TlNBEop5ec0ESillJ8L9HYA3TFgwACTnJzs7TCUUqpH2bx58xFjTOypy3tkIkhOTiYnJ8fbYSilVI8iIvvtLddDQ0op5ec0ESillJ/TRKCUUn6uR44RKKU8r7GxkeLiYurq6rwdiupEaGgoSUlJBAUFOdReE4FSyiHFxcVERESQnJyMiHg7HNUOYwzl5eUUFxeTkpLi0GP00JBSyiF1dXXExMRoEvBxIkJMTEyXem6aCJRSDtMk0DN09XXSQ0PKLeoam3n2s33UNTR36/EWi3DllCHER4a6ODKl1Kk0ESi3eDfvIH/4zy4AuvMj0hioqG3gvgVpLo5M9VSFhYXMmzePvLy876y76667OPPMM5k9e7bdx65cuZJRo0aRmprapX0++eSThIWFce2113Y53vDwcGpqajpsc8899xAeHs7tt9/ebpvuxt4VmgiUW2QXHiUiJJDcu88jwNL1TPDTf33JO1sPcte8VAID9Aim6th9993X4fqVK1cyb968Ln2ZNjU18eMf/9jZ0JzWndi7Sj9hyi2y91UwcWj/biUBgAUZCZTXNvBZQbmLI1M9WXNzMzfddBPjxo3jvPPO48SJEwAsWrSI119/HYClS5eSmppKeno6t99+Oxs2bODNN99kyZIlZGZmUlBQQG5uLlOnTiU9PZ2LL76Yo0ePAnDWWWdx2223kZWVxV//+lfuuece/vjHPwKQn5/P7NmzycjIYOLEiRQUFFBTU8M555zDxIkTGT9+PKtWrer0//DAAw8watQoZsyYwa5du04uf+aZZ5g8eTIZGRlceumlHD9+3G7s9to5S3sEyuWO1jawp7SGiyYkdnsbM0fHEhkayKrcA8wc9Z0aWcrL7n1rG9tLjrl0m6kJkdx94bgO2+zZs4eXXnqJZ555hssvv5w33niDq6+++uT68vJyVqxYwc6dOxERKisr6devH/Pnz2fevHlcdtllAKSnp/Poo48yc+ZM7rrrLu69914efvhhABoaGk7WMrvnnntObvuqq65i6dKlXHzxxdTV1dHS0kJwcDArVqwgMjKSI0eOMHXqVObPn9/uYO3mzZt5+eWXyc3NpampiYkTJzJp0iQALrnkEm666SYA7rzzTpYtW8bPfvaz78Ter18/u+2coT0C5XI5+62/rrKG9u/2NkICA7ggbRDv5R2irrF7A86q90lJSSEzMxOASZMmUVhY+K31UVFRhIaGcsMNN/Dvf/+bsLCw72yjqqqKyspKZs6cCcB1113Hxx9/fHL9woULv/OY6upqDhw4wMUXXwxYL9gKCwvDGMNvfvMb0tPTmT17NgcOHODw4cPtxv/JJ59w8cUXExYWRmRkJPPnzz+5Li8vjzPOOIPx48ezfPlytm3bZncbjrbrCu0RKJfLKawgOMBCxuB+Tm1nQWYCr+QUsX5nKXPHD3JRdMoVOvvl7i4hISEnbwcEBJw8NNQqMDCQTZs2sW7dOl5//XUee+wx1q9f36V99O3b1+G2y5cvp6ysjM2bNxMUFERycnK3r7xetGgRK1euJCMjg+eee44PP/zQqXZdoT0C5XKbCisYnxRFaFCAU9s5bVgMcREhrMo94KLIVG9XU1NDVVUVc+fO5S9/+QtbtmwBICIigurqasDaa+jfvz+ffPIJAC+++OLJ3kF7IiIiSEpKYuXKlQDU19dz/PhxqqqqiIuLIygoiA8++ID9++1WeT7pzDPPZOXKlZw4cYLq6mreeuutk+uqq6sZNGgQjY2NLF++/Fv7bo29o3bO0ESgXOpEQzN5B6qYnBzt9LYCLMK89AQ+2FlG1YlGF0Snervq6mrmzZtHeno6M2bM4M9//jMAV1xxBQ899BATJkygoKCA559/niVLlpCenk5ubi533XVXp9t+8cUXeeSRR0hPT2fatGkcOnSIq666ipycHMaPH88LL7zAmDFjOtzGxIkTWbhwIRkZGVxwwQVMnjz55Lr777+f0047jenTp39rO6fG3l47Z4gxpvsPFvk+cA8wFphijLE7W4yI3ArcBAjwjDHm4Tbrfgb8FGgG3jHG/LKz/WZlZRmdmMY3fV5QzpXPbGTZdVmcMzbe6e1tKapkweOf8YdL07l88mAXRKi6a8eOHYwdO9bbYSgH2Xu9RGSzMSbr1LbO9gjygEuAj9trICJpWJPAFCADmCciI2zrzgYWABnGmHHAH52MR3lZTmEFAFlDne8RAKQnRZEcE8abW0pcsj2l1Hc5lQiMMTuMMbs6aTYW+MIYc9wY0wR8hDV5APwEeNAYU2/bXqkz8Sjv21RYwej4CKLCHCt/2xkRYX5GAhsKjlB6TMsfK+UOnhgjyAPOEJEYEQkD5gKtffxRtnVfiMhHIjK5vY2IyGIRyRGRnLKyMg+ErbqqqbmFL/cfZXJK908btWd+ZgItBt7eetCl21VKWXWaCERkrYjk2flb4MgOjDE7gN8D7wP/AXKxjgeA9fTVaGAqsAR4Vdq5EsMY87QxJssYkxUbqxcY+aKdh6qpbWh2yUBxWyPiIhiXEKmHh5Ryk06vIzDG2K/i1AXGmGXAMgAR+S1QbFtVDPzbWEesN4lICzAA0J/8PVC2bXzA1YkAYH5GAr97dyf7y2sZGuP4ed5Kqc555PRREYmz/TsE6/jAv2yrVgJn29aNAoKBI56ISbledmEFif36kNCvj8u3fWFGAgBv5mqvQClXcyoRiMjFIlIMnA68IyLv2ZYniMjqNk3fEJHtwFvAT40xlbblzwLDRCQPeBm4zjhzPqvyGmMM2YVHmZzs2vGBVgn9+jAlJZpVW0rQt4h/qqys5G9/+5tH9rVy5Uq2b9/ukm21LVznq5w9a2iFMSbJGBNijIk3xpxvW15ijJnbpt0ZxphUY0yGMWZdm+UNxpirjTFpxpiJxpiuXQuufMb+8uOUVdeT5YbDQq3mZySQX1rDjoPVnTdWvqNN4TZndCcRGGNoaWnp8r5cmQh6Ar2yWLlE6/jAlBT3JYK54wcRaBFWbdGSEz3Kvfe6ZDNLly6loKCAzMxMlixZ0m4J6MLCQkaPHs21115LWloaRUVF3H///YwePZoZM2Zw5ZVXnvyFXlBQwJw5c5g0aRJnnHEGO3futFv6uVVVVRVDhw49mVxqa2sZPHgwjY2NDpWHPuuss05WNj1y5AjJycmAtbz2kiVLmDx5Munp6Tz11FMuec4cpUXnlEtkF1bQLyyIEbHhbttHdN9gzhwVy1u5Jfzq/DFYujnXgXKB226D3FzH2591VudtMjPh4YfbXf3ggw+Sl5dHrm2/TU1NdktAg7Vc9fPPP8/UqVPJzs7mjTfeYMuWLTQ2Nn6r9PPixYt58sknGTlyJF988QU333wz69ev/07p51ZRUVFkZmby0UcfcfbZZ/P2229z/vnnExQU1G4ZaUcsW7aMqKgosrOzqa+vZ/r06Zx33nmkpKQ49HhnaSJQLpFTeJSsof3d/uW8IDOB9TtL2fzNUbecnaRcpLAQ2hZg++gj679Dh4LtV7CzWktAf/zxx1gslm+VgB46dChTp04F4LPPPmPBggWEhoYSGhrKhRdeCFgL1G3YsIHvf//7J7dZX1/f6X4XLlzIK6+8wtlnn83LL7/MzTffDFjLQ995551UVlZSU1PD+eef7/D/5f3332fr1q0nJ9epqqpiz549mghUz1FWXc/eI7Us9EAtoNlj4wkNsrAq94AmAm/q4Jf7d4hYJ6F2sY5KQDtSSrqlpYV+/fqd7GE4av78+fzmN7+hoqKCzZs3M2vWLMCx8tCBgYEnDyu1LVdtjOHRRx/tUvJwJR0jUE7bvN9WX8gDX8x9QwI5N3Ug72w9SGNz1wcBVc91ajlmR0tAT58+nbfeeou6ujpqamp4++23AYiMjCQlJYXXXnsNsH4Z2ytbfarw8HAmT57Mrbfeyrx58wgIsJZbd6Q8dHJyMps3bwY4+esf4Pzzz+eJJ56gsdFaZXf37t3U1tY6/Nw4SxOBctqmfUcJCbQwPjHKI/tbkJHA0eONfLpHLznpEe6+2yWbiYmJYfr06aSlpbFkyRKHS0BPnjyZ+fPnk56ezgUXXMD48eOJirK+V5cvX86yZcvIyMhg3LhxJwecTy39fKqFCxfyz3/+81uzmTlSHvr222/niSeeYMKECRw58t/374033khqaioTJ04kLS2NH/3oRzQ1NXX7ueoqp8pQe4uWofYt8x/7lD5BAbzyo9M9sr+GphYmP7CWWWPi+MvCTI/sU/XsMtQ1NTWEh4dz/PhxzjzzTJ5++mkmTpzo7bDcypNlqJWfq61vYlvJMbeeNnqq4EALc8cP5L1thzjRoPMZq84tXryYzMxMJk6cyKWXXtrrk0BX6WCxcspX31TS3GI8Mj7Q1vyMRF7aVMTaHYdPlp9Qqj3/+te/Om/kx7RHoJyyqbACi8DEIc5NVN9VU1KiiY8M0YqkHtYTDyX7o66+TpoIlFNyCisYOyiSiFDXTETjqACLcGF6Ah/uKqXquM5n7AmhoaGUl5drMvBxxhjKy8sJDQ11+DF6aEh1W2NzC199U+mR6wfsWZCZyN8/3ce7eQe5YsoQr8TgT5KSkiguLkYnhvJ9oaGhJCUlOdxeE4Hqtm0lxzjR6PqJaByVlhjJsAF9WZVboonAA4KCgjx2pavyLD00pLote1/rRDTuKT3dGRHhwowENu4r57DOZ6xUt2kiUN2WXVjB0Jgw4iIdPxbpavMzEzAG3tJBY6W6TROB6hZjDDn7vV/4bXhsOOMTo/TsIaWcoIlAdUtBWS0VtQ1eOyzU1vyMBLYWV7HviOdqsyjVmzg7VeX3RWSbiLSIyHcuW27T7lYRybO1va3N8kwR2SgiuSKSIyJTnIlHeY47J6rvqnkZgxDR+YyV6i5newR5WCej/7i9BiKSBtwETAEygHkiMsK2+g/AvcaYTOAu233VA2QXVjAgPJiUAZ2X+3W3QVF9OC0lmlVbDug57kp1g7NzFu8wxuzqpNlY4AtjzHFjTBPwEdbkAWCASNvtKEB/0vUQ2YUVZA2NRsQ3Zgmbn5HI3rJatpUc83YoSvU4nhgjyAPOEJEYEQkD5gKtVyDdBjwkIkXAH4Fft7cREVlsO3yUoxe0eNehqjqKKk6Q5QPjA60uSBtIUIDooLFS3dBpIhCRtbbj+6f+LXBkB8aYHcDvgfeB/wC5QGvJyJ8AvzDGDAZ+ASzrYDtPG2OyjDFZsbGxjuxauYknJqrvqv59g5k5KpY3c0toadHDQ0p1RaeJwBgz2xiTZudvlaM7McYsM8ZMMsacCRwFdttWXQf823b7NazjCMrH5RRWEBYcQOqgyM4be9D8zEQOHatjky1RKaUc45HTR0UkzvbvEKzjA601YUuAmbbbs4A9nohHOWdT4VEmDulPYIBvnX08e2wcfYICWKVnDynVJc6ePnqxiBQDpwPviMh7tuUJIrK6TdM3RGQ78BbwU2NMpW35TcCfRGQL8FtgsTPxKPc7VtfIzkPHfGp8oFVYcCDnjYvn3byDNDTpfMZKOcqponPGmBXACjvLS7AOCrfeP6Odx38KTHImBuVZX+4/ijEwxQeuH7BnQWYCq3JL+GRPGeeMjfd2OEr1CL7Vt1c+L7uwgkCLkOnhiWgcdcbIWPqFBfHO1we9HYpSPYYmAtUl2YVHGZcYRViwb1YwDwqwkDU0mq3FVd4ORakeQxOBclh9UzO5RZVMHup74wNtpSVGUlBWw/GGJm+HolSPoIlAOSzvQBUNTS1M9qHrB+xJS4jCGNhxUK8yVsoRmgiUwzbtOwpAls/3CKIAyDugiUApR2giUA7LKaxgeGxfYsJDvB1Kh+IjQxgQHkzeAR0nUMoRmgiUQ1pafGMiGkeICOMSosjTAnRKOUQTgXLIntIaqk409ohEANYB4z2Hq6lrbO68sVJ+ThOBcsgmH5qIxhFpCVE0tRh2H672dihK+TxNBMohOYUVxEeGMDi6j7dDcYgOGCvlOE0EyiHZ+yrISvadiWg6k9S/D5GhgeSV6ICxUp3RRKA6daDyBCVVdT5bX8geESEtMYpteuaQUp3SRKA6lb3POj7gixVHO5KWGMWOQ9U0NmslUqU6oolAdSq7sIKIkEDGDPStiWg6My4hkoamFvYcrvF2KEr5NE0EqlPZhRVMHNqfAEvPGB9odXLAWMcJlOqQJgLVocrjDew+XONT8xM7KiWmL32DA3ScQKlO+FUiWJV7gPvf3u7tMHqUnMKeUV/IHotFrzBWyhFOJwIReUhEdorIVhFZISJ2ZywRkTkisktE8kVkaZvlKSLyhW35KyIS7GxM7dlzuIbnNhRSXdforl30Otn7KwgOsJAx2DcnounMuMRItpcco7nFeDsUpXyWK3oEa4A0Y0w6sBv49akNRCQAeBy4AEgFrhSRVNvq3wN/McaMAI4CN7ggJrumDY+hucWQbbtKVnUue18F45OiCA0K8HYo3ZKWEMWJxmb2HdEBY6Xa43QiMMa8b4xpnQFkI5Bkp9kUIN8Ys9cY0wC8DCwQ69VJs4DXbe2eBy5yNqb2TBzan5BAC5/ll7trF71KXWMzXx+o6jFlJezRK4yV6pyrxwiuB961szwRKGpzv9i2LAaobJNIWpd/h4gsFpEcEckpKyvrVnChQQFkJffns/wj3Xq8v8ktqqSx2TC5h10/0Nbw2L6EBFq0JLVSHXAoEYjIWhHJs/O3oE2bO4AmYLk7AjXGPG2MyTLGZMXGxnZ7O9OGD2DnoWrKa+pdGF3v9PHuMkRgUg8cKG4VGGBh7KBIPYVUqQ44NAO5MWZ2R+tFZBEwDzjHGGNvVO4AMLjN/STbsnKgn4gE2noFrcvdZtrwGAA+31vOvPQEd+6qR6utb2L5F98we2w8/cLcNn7vEWmJkaz6qoSWFoOlh10LoZQnuOKsoTnAL4H5xpjj7TTLBkbazhAKBq4A3rQljQ+Ay2ztrgNWORtTR8YnRhEREqjjBJ14NaeIqhON/HjmMG+H4rS0hCiq65soOtre21Mp/+aKMYLHgAhgjYjkisiTACKSICKrAWy/9m8B3gN2AK8aY7bZHv8r4H9EJB/rmMEyF8TUrsAAC6cNi+bzAh0naE9jcwt//2Qfk5P7M2lozx0obqUDxkp1zKFDQx2xnfZpb3kJMLfN/dXAajvt9mI9q8hjpg0fwNodpRyoPEFiv55RX9+T3tl6kAOVJ7h3/jhvh+ISI+PDCQoQ8kqq+F76IG+Ho5TP8asri1tNHzEAQM8essMYw5MfFTAiLpxZY+K8HY5LhAQGMCo+Qs8cUqodfpkIRsWHMyA8mA2aCL7j4z1H2HmomsVnDutVA6tpCVFsKzmG/XMZlPJvfpkIRITThw9gQ0G5fjGc4qmPCoiPDOGiTLuXc/RYaYmRVNQ2cLCqztuhKOVz/DIRAEwfHkNpdT0FZVp6oNXW4ko2FJRzw4wUggN711tj3MkBYz08pNSpetenvQv+O06gp5G2euqjvUSEBHLllCHeDsXlxg6MxCJoJVKl7PDbRDA4Ooyk/n3YoKeRAlB4pJZ38w5y1dShRIQGeTscl+sTHMCIuHCdm0ApO/w2EQBMHz6AzwvKtUQx8PdP9xJosXD99GRvh+I2aQlRWmpCKTv8OhFMGxHDsbomtvn5l8ORmnpeyynmkomJxEWGejsctxmXGMXhY/WUVuuAsVJt+XUiON1Wd8jfxwme31BIQ3MLN53Z88tJdCQtIRKAbTpOoNS3+HUiiIsIZVR8uF+PE9TWN/HC5/s5d2w8w2PDvR2OW6W2JgIdJ1DqW/w6EYC13ER2YQX1Tc3eDsUrXs62FZc7a7i3Q3G7iNAgUgb01ZpDSp3C7xPB9BEDqGts4atvKr0disc1Nrew7JO9TEmOZuKQnjvnQFeMS9C5CZQ6ld8ngikp0VgENhT43zjB21tLKKmq40e9oNS0o9ISoyg+eoLK4w3eDkUpn+H3iSCqTxDjk/r5Xd0hYwxPfbSXkXHhnD26dxSXc0RagvUKYx0wVuq//D4RgLXcRG5RJbX1TZ037iU+3F3GzkPV/Gjm8F5VXK4z42wDxlpqQqn/0kSAdcC4qcWwqbDC26F4zFMfFTAwMpT5Gf41XWf/vsEk9uujpSaUakMTAZCV3J/gQIvfHB7KLapk496KXllczhFpiZF6CqlSbTj1LSAiD4nIThHZKiIrRKRfO+3miMguEckXkaVtli+3Lc8TkWdFxCtFbkKDApg0pL/fXFj29McFRIQGcsWUwd4OxSvSEqLYe6SW6rpGb4eilE9w9ufgGiDNGJMO7AZ+fWoDEQkAHgcuAFKBK0Uk1bZ6OTAGGA/0AW50Mp5umz4ihu0Hj1FR27vPJtl3pJZ38w5xTS8tLueI1jmMt+vhIaUAJxOBMeZ928T0ABuBJDvNpgD5xpi9xpgG4GVgge3xq40NsKmdx3vE6cOtZak37u3dvYJnPtlLkMXCol5cXK4z4xJtA8aaCJQCXDtGcD3wrp3liUBRm/vFtmUn2Q4JXQP8p72Ni8hiEckRkZyysjIXhPttGUlRhIcE9up5jMuq63l9czGXTkokLqL3FpfrTFxEKHERITpOoJRNYGcNRGQtMNDOqjuMMatsbe4AmrAe6umOvwEfG2M+aa+BMeZp4GmArKwsl9eNDgywcFpKdK++sOz5DYU0Nrdw0xn+cwFZe9IStSS1Uq06TQTGmNkdrReRRcA84BxjfwLgA0DbUckk27LWx98NxAI/ciBetzp9eAzrdpZSUnmChH59vB2OS9XUN/HC54WcnzqQYb28uJwj0hIi+XBXKScamukTHODtcJTyKmfPGpoD/BKYb4w53k6zbGCkiKSISDBwBfCm7fE3AucDVxpjWpyJxRVap6/sjb2Clzd9w7G6Jr8qJ9GRcYlRtBjYcUjHCZRydozgMSACWCMiuSLyJICIJIjIagDbYPItwHvADuBVY8w22+OfBOKBz22Pv8vJeJwyOj6CmL7Bve56gsbmFpZ9uo8pKdFM8JPicp1pPXNIxwmUcuDQUEeMMSPaWV4CzG1zfzWw2k47p/bvahaLMHV4DJ8VHMEYg0jvKL3w1pYSDlbV8cDFad4OxWckRIXSPyxIS1IrhV5Z/B3Thw/g8LF69h6p9XYoLtFaXG50fIRfFZfrjIjogLFSNpoITjF9hHX6yt5yeOjDXWXsOlzN4jOH9ZoejquMS4hi9+Fqv52USKlWmghOMSQ6jMR+fXp0uQljDJv3V3D7a1u4efmXJESFcqGfFZdzRFpiJI3Nhj2Ha7wdilJe5VPH6H2BiDBteAzvbz9MS4vpUSWaj9Y28MaXxbySXcSe0hr6Bgdw0YQEbjxjmF8Wl+vMeNuAcd6BqpODx0r5I00EdkwfMYDXNhez/eAxn/+CaGkxbNxbzkvZRbyXd4iG5hYyB/fjwUvGc2FGAn1D9CVuz5DoMCJCA3WcQPk9/ZawY9pw6zjBZ/lHfDYRlB6r47XNxbyaU8T+8uNEhgbyg9OGsHDyYMYOivR2eD2CiFjnMNYzh5Sf00RgR1xkKCPiwvmsoJwfzRzu7XBOam4xfLS7lJc3FbFuZynNLYbTUqL5xexRzEkbSGiQXiHbVWkJUby4cT9NzS0EBujhM+WfNBG0Y/rwGF7NKaahqcXrx9cPVp3gpU1FvJZTxMGqOmL6BnPjjBQWTh6s5SKclJYYRX1TCwVltYweGOHtcJTyCk0E7Zg2YgDPf76f3KJKpqREey2OY3WNzHvkUyqON3DGyFjumpfKOWPjvZ6ceou0xP/OYayJQPkrTQTtmJoSg0Ws4wTeTAT/+LSQ8toG3vjJ6Uwa6r04equUAeH0CQogr6SKSyd5bToMpbxKf1a2IyosiLTEKD73YgG6quON/P3TvZyXGq9JwE0CLEJqQiTbdMBY+TFNBB2YNnwAXxUd5XhDU+eN3WDZp3uprmvittmjvLJ/f5GWEMm2kipaWlw+zYVSPYImgg5MHxFDY7Nh074Kj+/7aG0Dz35WyNzxA0lN0NNB3WlcYhS1Dc0UlveO+lJKdZUmgg5kDY0mOMDilcNDT3+yl9qGJm49R3sD7paWYLvCWOcwVn5KE0EH+gQHMGFIPz4r8GwBuiM19Ty/oZB56Ql6JosHjIwPJzjAonMTKL+liaAT00cMYFvJMSqPN3hsn09/vJe6xmZuPWekx/bpz4ICLIwZFKGlJpTf0kTQiWnDYzAGjx0eKq2u44XPC1mQmciIOL1YzFPGJUSRd+AY9qfdVqp3czoRiMhDIrJTRLaKyAoR6ddOuzkisktE8kVkqZ31j4iIz9UDzhjcj77BAR6bx/jJD/fS2Gz4ufYGPCotMZKqE40UHz3h7VCU8jhX9AjWAGnGmHRgN/DrUxuISADwOHABkApcKSKpbdZnAT45mW5QgIUpKdEeGSc4fKyOf36xn4snJJIyoK/b96f+q3XAeJseHlJ+yOlEYIx53zZBPcBGwN7lmVOAfGPMXmNMA/AysABOJomHgF86G4u7TBs+gL1ltRyqqnPrfv72QT4tLYafz9LegKeNHhhBgEW0EqnyS64eI7geeNfO8kSgqM39YtsygFuAN40xBzvasIgsFpEcEckpKytzSbCOmtY6faUbewUlldbCct/PSmJITJjb9qPsCw0KYGRcuA4YK7/kUCIQkbUikmfnb0GbNncATcByR3cuIgnA94FHO2trjHnaGJNljMmKjY11dBcuMXZgJNF9g906feXjH+RjMPz07BFu24fqWFpiFHkHqnTAWPkdh4rOGWNmd7ReRBYB84BzjP1P0QFgcJv7SbZlE6RsMcYAABlTSURBVIARQL5tYvUwEck3xvjUt6HFIpw+LIYNBUcwxrh8EviiiuO8mlPEwsmDSeqvvQFvSUuI5PXNxZRW1xMfGertcJTyGFecNTQH6/H9+caY4+00ywZGikiKiAQDV2A9HPSOMWagMSbZGJMMHPe1JNBqxsgBHKyq458b97t8249/kI8g2hvwsrQ2cxgr5U9cMUbwGBABrBGRXBF5EqyHfURkNYBtMPkW4D1gB/CqMWabC/btMZdOTGL22Dj+36ptPPfZPpdtd395La9tLuYHpw1hUFQfl21Xdd3YQZGIoAPGyu84PR9Be7/gjTElwNw291cDqzvZls9eQRUcaOFvV03iln99yT1vbaepxXDjGcOc3u4j6/IJtAg/Oct3psT0V31DAhk2oC9fa49A+Rm9srgLggMtPH7VROaOH8j/vbODpz4qcGp7e8tqWPFVMVdPHarHpH3E+MQocosqtSS18iuaCLooKMDCI1dM4MKMBH737k4e/yC/29t6ZN0eggMt/Him9gZ8xZmjYjlSU89W7RUoP6JTVXZDYICFv1yeQYDAQ+/torml6yUh8kurWbWlhMVnDCM2IsRNkaqumjUmjgCLsGb7ITIH262WolSvoz2CbgoMsPCnyzO5ZGIif16zmz+v2d2l888fXruHPkEBLD7T+XEG5Tr9woKZnNyfNdsPezsUpTxGE4ETAizCQ5dlcHlWEo+s28Of3ncsGew8dIx3vj7IomnJxIRrb8DXnJs6kN2Ha9ivM5YpP6GJwEkBFuHBS9K5cspgHvsgn9//Z1enyeCva/fQNzhQewM+6tyx8QDaK1B+QxOBC1gswgMXjefqqUN48qMCHnhnR7vJYFtJFe/mHeL6GSn0Cwv2cKTKEUNiwhgdH6GJQPkNHSx2EYtFuH9BGoEWC3//dB9NLYa7L0z9TjmKh9fuISI0kBtmpHgpUuWIc1Pj+duH+RytbaB/X03YqnfTHoELiQh3X5jKDTNSeG5DIXet2vat89G/Lq5izfbD3HTGMKL6BHkxUtWZc1PjaTGwfmept0NRyu20R+BiIsKd3xtLoEV46uO9NBvD/y1Iw2IR/rJ2N1F9gvjh9GRvh6k6MT4xivjIENZsP8ylk+xNsaFU76GJwA1EhKUXjCHAIvztwwKamw2XTx7M+p2lLDl/NBGh2hvwdRaLMHtsPCu+OkBdYzOhQQHeDkkpt9FDQ24iIiw5fzQ/nzWCV3KKuO7ZTUT3Dea6acneDk056NzUeI43NPO5h+arVspbNBG4kYjwP+eN5hezR1FT38RPZg4nPEQ7YT3F6cNj6BscwPt69pDq5fRbyQNunT2SeRmDGKYT0vcoIYEBzBwdy9odh3mgxTrOo1RvpD0CDxkeG+7ymc2U+52bGk9ZdT1biiu9HYpSbqOJQKkOnD26tQidHh5SvZcmAqU60C8smCnJ0ZoIVK/mVCIQkYdEZKeIbBWRFSJit26viMwRkV0iki8iS9ssFxF5QER2i8gOEfm5M/Eo5Q7npsazp7SGwiNahE71Ts72CNYAacaYdGA38OtTG4hIAPA4cAGQClwpIqm21YuAwcAYY8xY4GUn41HK5c5NtRahW7tDewWqd3IqERhj3rdNTA+wEbB3CeYUIN8Ys9cY04D1y36Bbd1PgPuMMS227en1/MrnDI4OY8zACD2NVPVarhwjuB54187yRKCozf1i2zKA4cBCEckRkXdFpN1pvkRksa1dTllZmcuCVsoR56bGk1NYQUVtg7dDUcrlOk0EIrJWRPLs/C1o0+YOoAlY3sX9hwB1xpgs4Bng2fYaGmOeNsZkGWOyYmNju7gbpZyjRehUb9bpBWXGmNkdrReRRcA84Bxjvwj/AazjAK2SbMvA2jv4t+32CuAfncWjlDeMT4xiYGQoa7Yf4jItQqd6GWfPGpoD/BKYb4w53k6zbGCkiKSISDBwBfCmbd1K4Gzb7ZlYB5yV8jkiwuzUOD7efYS6xmZvh6OUSzk7RvAYEAGsEZFcEXkSQEQSRGQ1gG0w+RbgPWAH8KoxZpvt8Q8Cl4rI18DvgBudjEcpt5k9Np4Tjc1sKDji7VCUcimnag0ZY0a0s7wEmNvm/mpgtZ12lcD3nIlBKU85fXgM4SGBrNl+mFlj4r0djlIuo1cWK+WgkMAAZo6KZe2O0m/NPKdUT6eJQKkuaC1Cl6tF6FQvoolAqS7QInSqN9JEoFQXRIUFcVqKFqFTvYsmAqW6aPbYePJLa9inRehUL6GJQKkuOlmETnsFqpfQRKBUF7UWodPDQ6q30ESgVDeclxpPzn4tQqd6B528XqluODd1II+sz2fdjsN8P2tw5w/opT7ZU8aj6/O7fV1FaFAAc9IGsiAzgYjQIBdHpxylPQKluiEtMdJWhM5/Dw8dq2vkF69soajiOCFBlm79HT5Wx50r85jywDqWvLaFzfuPYr92pXIn7REo1Q2tReje2HyAusZmQoMCvB2Sxz28Zg/ltfWs+ul00pPszlLbKWMMW4ureDn7G97MLeG1zcWMig9n4eQhXDIhkf59g10ctbJHewRKddO5qQM50djMZ/n+V4Ru56FjPP95IVdOGdLtJADWhJoxuB+/uySdTXfM5sFLxhMWHMj9b2/ntN+u4+cvfcWG/CNa0sPNtEegVDdNHRZ9sgjdOWP9pwidMYa7Vm4jIjSQJeeNdtl2+4YEcsWUIVwxZQg7Dh7jlewi/v1lMW9uKWFoTBgLJw/msklJxEWEumyfykp7BEp1U0hgADNH+18RulW5JWwqrOCX549x26GbsYMiuWf+ODbdMZuHF2YyMDKUP/xnF6f/bj2LX8jhg52lNPvRc+5u2iNQygnnpcbzztaDfFVUyaSh/b0djttV1zXywOodpCdFsXCy+8+WCg0K4KIJiVw0IZG9ZTW8klPEG5uLeX/7YRKiQvnj9zOYNmKA2+Po7bRHoJQTzhoVR6AfFaH769o9HKmp574FaQRYxKP7HhYbzq8vGMuGpefwxFUT6RsSyI0v5PDVN0c9Gkdv5HQiEJGHRGSniGwVkRUiYnfkSETmiMguEckXkaVtlp8jIl/aZjj7VETsTnajlC+KCgtiSko0a7Yf8nYobrf7cDX/2FDIwqzBZA7u/gCxs4IDLVwwfhDLbzqN2IgQfvhcNnsOV3stnt7AFT2CNUCaMSYd65zDvz61gYgEAI8DFwCpwJUikmpb/QRwlTEmE/gXcKcLYlLKY85NjaegrJa9ZTXeDsVtjDHctSqP8JBAfjlnjLfDASAuIpQXrz+N4AAL1yzbRPHR9qZNV51xOhEYY963zUsMsBFIstNsCpBvjNlrjGkAXgYWtG4CiLTdjgJKnI1JKU86WYRuR+89PPTW1oNs3FvB7eePJtqHzu0fEhPGCzdM4XhDE9cs28SRmnpvh9QjuXqM4HrgXTvLE4GiNveLbcvAOmH9ahEpBq7BOqG9Uj1GUv8wxg6K7LXjBDX1TTzwznbSEiP5wZQh3g7nO8YMjOQfP5zMwaoTXPfsJo7VNXo7pB7HoUQgImtFJM/O34I2be4AmoDlXYzhF8BcY0wS8A/gz+3EsFhEckQkp6ysrIu7UMq9zk2NZ/P+o5T3wl+kj67bw+Fj9dw73/MDxI6aNDSaJ6+exK5D1dz4fA51jc3eDqlHcSgRGGNmG2PS7PytAhCRRcA8rMf67Z3cewBoe65ZEnBARGKBDGPMF7blrwDT2onhaWNMljEmKzY21rH/nVIecl5qPC0G1u0s9XYoLpVfWs2yT/fx/UlJPn967Fmj4/jzwkyyCyu45V9f0dTc4u2QegxXnDU0B/glMN8Y095oTTYwUkRSRCQYuAJ4EzgKRInIKFu7c4EdzsaklKeNS4hkUFTvKkJnjOHuN7cRFhzAry7wjQHizszPSOC+BWms3XGYX73xtV9d6OcMV1xQ9hgQAqwREYCNxpgfi0gC8HdjzFxjTJOI3AK8BwQAzxpjtgGIyE3AGyLSgjUxXO+CmJTyKBFh9th4XttcRHlNPTHhId4OyWmrvz7EZ/nl3LdgHAN60P/nmqlDOVrbwJ/X7KZfWBB3fm8stu8m1Q6nE4Exxu55/8aYEmBum/urgdV22q0AVjgbh1Le9oPThvBqThE/enEzy286jZDAnluRtLa+if97ZzupgyK56rSh3g6ny342awRHjzew7NN9RPcN5qdn6+VJHdEri5VykbGDIvnT5Rnk7D/K0je+7tF19R/7IJ+DVXXcf9E4nx0g7oiI8P++l8rFExJ56L1d/HPjfm+H5NO01pBSLjQvPYF9ZbX8ac1uhg3oy8/OGentkLqsoKyGv3+yl0snJjFpaLS3w+k2i0X4w2XpHDvRyP9blUe/sCDmpSd4OyyfpD0CpVzsllkjuHhCIn9as5u3t/as6yONMdzz5jZCgwJY2kMGiDsSFGDh8asmMnloNL94JZePduup5/ZoIlDKxUSEBy8dT9bQ/vzvq1t6VFG097Yd4pM9R/ifc0cRG9FzBog7EhoUwN8XZTEiLoIfv7iZL3vQ6+EpmgiUcoOQwACeumYS8ZGh3PTC5h5RB+dEQzP3v72DMQMjuGZqzxsg7khkaBAvXD+F+MgQfviPbHYd0iJ1bWkiUMpNYsJDeHZRFvVNzdz4fA7VPl764PEP8jlQeYL7FqQRGND7vhpiI0J48YbTCA2ycM2yL9hecszbIfmM3vdqK+VDRsRF8MRVk9hTWsPPX/Ldq133Hanl6Y/3cvGERKak9NwB4s4Mjg7jhetPo76phbmPfMJZD33AfW9t59M9R2ho8s3XxhOkJ57ilpWVZXJycrwdhlIO++fG/dy5Mo8fTk/m7gvHeTucbzHG8MPnsskpPMr6/51JXGTvnxO49Fgd7207xLqdpWwoKKehqYXwkEBmjBjArLFxnD06rteMkbQlIpuNMVmnLtfTR5XygKunDmVvWS3PfraPYbHhPnMM/kRDM29uOcCHu8q483tj/SIJAMRFhnLN6clcc3oyxxua2JBfzvpdpazfUcp/tlknGcpIimLWmHhmjYljXEIklh54PYWjtEeglIc0txgWv5DDh7vLeHbRZGaO8mzxxJr6JraXHOPrA1VsO1BFXkkV+aU1tBgYMzCCt342g6BeODbQFcYYdhysZv3Ow6zfWcpXRZUYA3ERIZw9Oo5ZY+OYMWIAfUN65m/o9noEmgiU8qCa+iYue2IDB46e4I2bpzEqPsIt+6k60Xjyyz7vwDHyDlSxr7yW1o97fGQIaQlRjEuMYnxiFFOHRRMRGuSWWHqy8pp6PtpdxrqdpXy8u4zquiaCAyxcmJHAby9J63FlRDQRKOUjSipPsODxzwgJtLDqp9OdLlDX0NRCzv4Kcosq2XbA+ov/m4r/nq6a2K8P4xIiSbN96Y9LjCQuwj8OAblSY3MLOYVHeTfvIC98vp+zR8fyxNWTCA3qOclAE4FSPmRLUSWXP/U5aYlRLL/xtC5/mZRV1/PhrlLW7yzlkz1HqKm3zhY7JDqMtMRIxiXYvvQTIntFJVRf868vvuE3K75m5qhYnrqm5yQDTQRK+ZjVXx/k5uVfsiAzgYcXZnZYKtkYw7aSY6zbUcr6XaVsKaoErId4Wgc0pyRHExWmh3c85dXsIn71761MHz6AZ67Nok+w7ycDPWtIKR8zd/wglpw/mofe28WwAeHcOvvbBepq65v4LP8I63eW8sGuUg4fq0cEMpL68b/njmLW2DhSB0VqrX0vuXzyYCwWYcnrW7j+uWyWLcoiLLhnfqX2zKiV6iVuPms4e8tq+cva3aTE9mXC4H6s31nKup2lbCwop6G5hYiQQM4YNYBZY+I5a3Rsj5okpre7bFISgRbhf17NZdE/svnHosk98oyinhexUr2IiPDbS9IoqjjOrS9/dfKsnmED+nLt6UOZNSaOrORoggP9+7ROX3bRhEQCLMJtr+Ry3bObeO76KYT3sGTgVLQi8hBwIdAAFAA/NMZU2mn3LNbJ7UuNMWltlkdjnbA+GSgELjfGaGlA5VdCAgN48ppJ/OE/OxkZH8GsMXGkDOjr7bBUF1yYkUCARfj5S19x7bIveO76KUT2oNNxnf2ZsQZIM8akA7uBX7fT7jlgjp3lS4F1xpiRwDrbfaX8TnTfYB68NJ0bZqRoEuih5o4fxGM/mMjW4iquWbaJqhO+XWSwLacSgTHmfWNMk+3uRiCpnXYfAxV2Vi0Anrfdfh64yJl4lFLKm+akDeSJqyexvaSKq//+BZXHG7wdkkNceeDxeuDdLj4m3hhz0Hb7EBDvwniUUsrjzk2N56lrJrHrUDU/eOYLjtb6fjLoNBGIyFoRybPzt6BNmzuAJmB5dwMx1gsa2r2oQUQWi0iOiOSUlel0c0op3zVrTDxPXzuJ/LIarnxmI+U19d4OqUOdJgJjzGxjTJqdv1UAIrII60DwVabrV6cdFpFBtu0MAko7iONpY0yWMSYrNtazxbqUUqqrzhodx7Lrsth3pJYrn9lIWbXvJgOnDg2JyBzgl8B8Y0x35uJ7E7jOdvs6YJUz8SillC85Y2Qs/1g0mW8qjnPlMxspPVbn/Ebvucf5bZzC2TGCx4AIYI2I5IrIkwAikiAiq1sbichLwOfAaBEpFpEbbKseBM4VkT3AbNt9pZTqNaaNGMBzP5xCSeUJrnh6I4eqnEwG997rmsDa0FpDSinlAZv2VfDDf2wiNiKEV350OvHdnQRIBLr5vd1erSG9XFEppTxgSko0L9wwhRFxEUSEdvFa3nvusSaA1rpSrbdddJhIewRKKdWTaI9AKaWUq2kiUEqpnuTuu12+SU0ESinVk/jg6aNKKaV6OE0ESinl5zQRKKWUn9NEoJRSfk4TgVJK+bkeeUGZiJQB+7v58AHAEReG4yoaV9doXF2jcXWNr8YFzsU21BjznfLNPTIROENEcuxdWedtGlfXaFxdo3F1ja/GBe6JTQ8NKaWUn9NEoJRSfs4fE8HT3g6gHRpX12hcXaNxdY2vxgVuiM3vxgiUUkp9mz/2CJRSSrWhiUAppfxcr00EIjJHRHaJSL6ILLWzPkREXrGt/0JEkj0Q02AR+UBEtovINhG51U6bs0SkyjYHdK6I3OXuuGz7LRSRr237/M6sP2L1iO352ioiEz0Q0+g2z0OuiBwTkdtOaeOR50tEnhWRUhHJa7MsWkTWiMge27/923nsdbY2e0TkOg/E9ZCI7LS9TitEpF87j+3wNXdDXPeIyIE2r9Xcdh7b4WfXDXG90iamQhHJbeex7ny+7H43eOw9ZozpdX9AAFAADAOCgS1A6iltbgaetN2+AnjFA3ENAibabkcAu+3EdRbwthees0JgQAfr5wLvAgJMBb7wwmt6COsFMR5/voAzgYlAXptlfwCW2m4vBX5v53HRwF7bv/1tt/u7Oa7zgEDb7d/bi8uR19wNcd0D3O7A69zhZ9fVcZ2y/k/AXV54vux+N3jqPdZbewRTgHxjzF5jTAPwMrDglDYLgOdtt18HzhFpnRDUPYwxB40xX9puVwM7gER37tOFFgAvGKuNQD8RGeTB/Z8DFBhjuntFuVOMMR8DFacsbvseeh64yM5DzwfWGGMqjDFHgTXAHHfGZYx53xjTZLu7EUhy1f6cictBjnx23RKX7fN/OfCSq/bnqA6+GzzyHuutiSARKGpzv5jvfuGebGP70FQBMR6JDrAdipoAfGFn9ekiskVE3hWRcR4KyQDvi8hmEVlsZ70jz6k7XUH7H1BvPF8A8caYg7bbh4B4O228/bxdj7UnZ09nr7k73GI7ZPVsO4c5vPl8nQEcNsbsaWe9R56vU74bPPIe662JwKeJSDjwBnCbMebYKau/xHr4IwN4FFjpobBmGGMmAhcAPxWRMz20306JSDAwH3jNzmpvPV/fYqx9dJ86F1tE7gCagOXtNPH0a/4EMBzIBA5iPQzjS66k496A25+vjr4b3Pke662J4AAwuM39JNsyu21EJBCIAsrdHZiIBGF9oZcbY/596npjzDFjTI3t9mogSEQGuDsuY8wB27+lwAqsXfS2HHlO3eUC4EtjzOFTV3jr+bI53Hp4zPZvqZ02XnneRGQRMA+4yvYF8h0OvOYuZYw5bIxpNsa0AM+0sz9vPV+BwCXAK+21cffz1c53g0feY701EWQDI0UkxfZr8grgzVPavAm0jq5fBqxv7wPjKrZjkMuAHcaYP7fTZmDrWIWITMH6Grk1QYlIXxGJaL2NdbAx75RmbwLXitVUoKpNl9Xd2v2l5o3nq42276HrgFV22rwHnCci/W2HQs6zLXMbEZkD/BKYb4w53k4bR15zV8fVdkzp4nb258hn1x1mAzuNMcX2Vrr7+ergu8Ez7zF3jID7wh/Ws1x2Yz0D4Q7bsvuwfjgAQrEeasgHNgHDPBDTDKxdu61Aru1vLvBj4Me2NrcA27CeLbERmOaBuIbZ9rfFtu/W56ttXAI8bns+vwayPPQ69sX6xR7VZpnHny+siegg0Ij1GOwNWMeU1gF7gLVAtK1tFvD3No+93vY+ywd+6IG48rEeM259j7WeHZcArO7oNXdzXC/a3jtbsX7BDTo1Ltv973x23RmXbflzre+pNm09+Xy1993gkfeYlphQSik/11sPDSmllHKQJgKllPJzmgiUUsrPaSJQSik/p4lAKaX8nCYCpZTyc5oIlFLKz/1//QSmNpooW4cAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GWi2zlRr-2fw",
        "colab_type": "text"
      },
      "source": [
        "### A GRU model for time series forecast"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d-c6xiRDjmJD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = Sequential()\n",
        "model.add(layers.GRU(64,\n",
        "                    dropout=0.2,\n",
        "                    recurrent_dropout=0.2,\n",
        "                    input_shape=(None, uni_data.shape[-1])))\n",
        "model.add(layers.Dense(1))\n",
        "model.add(Dense(1))\n",
        "model.compile(optimizer='adam', loss='mae')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FB9jc9W12G8-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 563
        },
        "outputId": "a486269c-72f4-4778-adbe-7d05fcd9ed61"
      },
      "source": [
        "history = model.fit_generator(train_gen,\n",
        "                              epochs=15,\n",
        "                              steps_per_epoch=TRAIN_SPLIT//BATCH_SIZE,\n",
        "                              validation_data=val_gen,\n",
        "                              validation_steps=(len(uni_data)-TRAIN_SPLIT)//BATCH_SIZE)"
      ],
      "execution_count": 127,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.2203 - val_loss: 0.2838\n",
            "Epoch 2/15\n",
            "1171/1171 [==============================] - 69s 59ms/step - loss: 0.1894 - val_loss: 0.2201\n",
            "Epoch 3/15\n",
            "1171/1171 [==============================] - 69s 59ms/step - loss: 0.1847 - val_loss: 0.1882\n",
            "Epoch 4/15\n",
            "1171/1171 [==============================] - 69s 59ms/step - loss: 0.1816 - val_loss: 0.1691\n",
            "Epoch 5/15\n",
            "1171/1171 [==============================] - 70s 59ms/step - loss: 0.1796 - val_loss: 0.2098\n",
            "Epoch 6/15\n",
            "1171/1171 [==============================] - 75s 64ms/step - loss: 0.1809 - val_loss: 0.1999\n",
            "Epoch 7/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.1793 - val_loss: 0.2106\n",
            "Epoch 8/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.1802 - val_loss: 0.2083\n",
            "Epoch 9/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.1798 - val_loss: 0.2095\n",
            "Epoch 10/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.1807 - val_loss: 0.1973\n",
            "Epoch 11/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.1798 - val_loss: 0.2214\n",
            "Epoch 12/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.1793 - val_loss: 0.2153\n",
            "Epoch 13/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.1790 - val_loss: 0.2082\n",
            "Epoch 14/15\n",
            "1171/1171 [==============================] - 76s 65ms/step - loss: 0.1781 - val_loss: 0.2074\n",
            "Epoch 15/15\n",
            "1171/1171 [==============================] - 70s 60ms/step - loss: 0.1797 - val_loss: 0.2206\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eZwq_lvZ9nbN",
        "colab_type": "text"
      },
      "source": [
        "By setting:\n",
        "\n",
        " `steps_per_epoch = TotalTrainingSamples // TrainingBatchSize`\n",
        "\n",
        "`validation_steps = TotalvalidationSamples // ValidationBatchSize`\n",
        "\n",
        "It makes sure that at each epoch:\n",
        "\n",
        "- You train on exactly your entire training set\n",
        "- You validate on exactly your entire validation set"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7RPcqaPijl9J",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 271
        },
        "outputId": "1da30610-d703-4b13-99dd-9f5e855fb288"
      },
      "source": [
        "ax = pd.DataFrame(history.history)[['loss', 'val_loss']].plot(marker='.')\n",
        "ax.legend(bbox_to_anchor=(1.5, 1.05))\n",
        "plt.show()"
      ],
      "execution_count": 128,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhwAAAD+CAYAAABx7dzcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXiU1d3/8fd3ZrISCAECYQkBBERQ1gi47xY3rNaqFRRbxVZr26e2trW29qldfq22fdqqrVvdCtUqomJFLUWruAACsu8iS9jCvoWsc35/nAmEQEIgM5lk8nldV66Zue97znwTQuYz55z73OacQ0RERCSWAvEuQERERBKfAoeIiIjEnAKHiIiIxJwCh4iIiMScAoeIiIjEXCjeBYiIiDRms2fPbh8KhZ4ETkYf1GsTBhaWl5ffOmTIkMLqOxU4REREahEKhZ7Myck5KTs7e0cgENBaEjUIh8O2ZcuWvps2bXoSGFl9v5KaiIhI7U7Ozs7erbBRu0Ag4LKzs3fhe4IO39/A9YiIiDQ1AYWNuon8nI6YLRQ4REREJOYUOERERBq59PT0QfGuob4a3aTRdu3auW7dusW7DBGRJmX27NlbnXPZ8a5DpCaNLnB069aNWbNmxbsMEZEmxczWxLsGOeiDlVtbfLhya8szerbbc2bPdvui1W44HOb222/v8s4772Sambv77rs3jh07dseaNWuSvvSlL/XYu3dvsKKiwh566KE1F1544d7rrruu2/z581uYmRs1atTWn/3sZ4edrtpQGl3gEBERaazunjAvd/mmPem1HbOvpDzw2dZ96c7Bo+99xgntWhS1SAmFazq+d07LogevGbCuLq//3HPPtV6wYEHakiVLFm3cuDE0dOjQky6++OK9Tz31VJsLLrhg129/+9tN5eXl7NmzJ/Dxxx+nb9y4MWnFihWLALZu3Ro8tu82ujSHQ0REJIr2lpSHXOScFuf842i1PW3atJbXXnvt9lAoRG5ubvmwYcP2fvDBB+nDhw/f9/zzz7e76667Os2cOTMtKysr3KdPn5J169aljBkzJnfChAmtsrKyKqJVx/FQD4eIiEgd1aUn4oOVW1t87ZlPepdXhAOhYCD8u2sHrormsMqRXHLJJXvff//9ZS+//HLm1772te533nnn5jvvvHPbwoULF7/yyiutHn300ex//vOfbV566aXVsayjNgocIiIiUXRmz3b7nrr51OWxmMNx9tln73niiSey77zzzm2FhYWhmTNnZvz5z39et3z58uQePXqUfu9739taUlJic+bMSd+4ceOulJSU8M0337yzX79+xTfeeGOPaNVxPBQ4REREouzMnu32xaJX48Ybb9z50UcfZZx00kn9zMz9/Oc/L+jatWv5Qw891PbPf/5zTigUcunp6RXjx4//fPXq1Um33HJLt3A4bAD3339/QbTrORbmXONaPC0/P9/pLBURkWNjZrOdc/nxriMRzZs3b/WAAQO2xruOpmLevHntBgwY0K369sSaNLpuJkz7vb8VERGRRiNxhlRW/Aeevw5cGIIpMGYS5A6Nd1UiIiJCIvVwFMyAcLkPHBWlsHpavCsSERGRiMQJHD0vAszfDyZDt7PiWo6IiIgclDiBI3co9DgHUjI1nCIiItLIJE7gADjhfCjZBW1OiHclIiIiUkWdAoeZjTCzZWa20sx+dIT9d5nZYjObb2ZTzSyvyr4HzGyRmS0xsz+bmUXzGzhEx4H+duPcmL2EiIiIHLujBg4zCwKPAJcAfYGvmFnfaod9CuQ75/oDE4AHIs89HTgD6A+cDJwKnBO16qvrOMDfKnCIiEgzlp6ePqimfcuWLUvu1atXv4asB+rWwzEUWOmcW+WcKwVeAK6seoBz7l3nXFHk4XSgS+UuIBVIBlKAJGBzNAo/orTWkNUdNihwiIhIHK36bwv+8785rPpvi3iX0ljUZR2OzkDVi9UUAMNqOf4W4E0A59zHZvYusBF/CsnDzrkl1Z9gZrcBtwF07dq1bpXXpOMA2DCnfm2IiIgcyavfzKVwca2Xp6dkb4BtK9LBwQd/hLa9ikjJqPHy9LTvW8QXH6n1onB33HFH59zc3NJ77rlnC8Bdd93VKRQKuWnTprXctWtXsLy83O67774No0eP3nks305RUZHddNNNefPnz08PBoM88MAD66644oo9s2bNSv3qV7/avayszMLhMC+//PJneXl5ZSNHjuyxcePG5HA4bD/4wQ82jB07dkddXyuqk0bNbDSQDzwYedwTOAnf49EZON/MDjtf1Tn3uHMu3zmXn52dXb8iOg2EnWuhaHv92hERETkepXtCvoMfwEUe18+oUaO2T5w4sU3l49deey3rtttu2/rGG2+sXLx48ZL33ntv+Y9//OMu4XDNueZIfvvb37Y3M5YvX774H//4x6rbbrutW1FRkT300EPZd9xxx+alS5cunj9//pLu3buXTpw4sVVOTk7ZsmXLFq9YsWLR1VdfvftYXqsuP4T1QG6Vx10i2w5hZhcC9wLnOOdKIpuvAqY75/ZGjnkTOA2I3apcByaOzoMTzovZy4iISDN0lJ4IwA+njL+2N+GyAIGkMFc9uooe59brQm5nnHHG/m3btoVWr16dtHHjxlBmZmZFbm5u+dixY3OnT5+eEQgEKCwsTC4oKAh17dq1vK7tfvTRRxnf+ta3CgEGDRpU3KlTp9IFCxaknnbaaft+97vfdSwoKEi+/vrrd5xyyiklgwcP3n/vvffm3n777Z2vvPLKXSNGjNh7LN9DXXo4PgF6mVl3M0sGrgcmVT3AzAYBjwEjnXOFVXatBc4xs5CZJeEnjB42pBJVmjgqIiLx1OPcfYx6cTlnfGc9o15cXt+wUWnkyJE7xo0blzV+/Pg2V1999fbHHnuszbZt20ILFixYsnTp0sVt27Yt279/f1RGLr7xjW9sf+2111ampaWFL7/88l6TJk1q2b9//5I5c+YsPuWUU/b/9Kc/7fz973+/47G0edQeDudcuZndCbwNBIGnnHOLzOx+YJZzbhJ+CCUDeCly1uta59xI/Bkr5wML8P1LbznnXj+WAo9Zehto3VUTR0VEJH56nLsvWkGj0ujRo7ePHTu2244dO0Lvvffesueeey6rXbt2ZSkpKe71119vuWHDhuRjbfOMM87YO27cuDYjR47cM3/+/JSNGzcm9+/fv3jx4sXJJ510Ukm/fv0K165dmzx37ty0/v37F7dv3778jjvu2J6VlVXxt7/9rd2xvFadxpWcc5OBydW23Vfl/oU1PK8C+PqxFBQVHQf6IRUREZEEkZ+fX7xv375Ahw4dSvPy8spuvfXW7ZdccknP3r179+3fv39R9+7di4+1zR/84AeFN910U17v3r37BoNBHnvssdVpaWlu3LhxbV588cW2oVDIZWdnl/3iF7/Y+MEHH7S45557ugQCAUKhkPvLX/6y5lhey5xzRz+qAeXn57tZs2bVr5Fpv4ep98MP1/hTZUVEEpyZzXbO5ce7jkQ0b9681QMGDNga7zqainnz5rUbMGBAt+rbE2tp80oH5nGol0NERKQxqPepOo1Sx8gCaxvn+gu6iYiINDMzZ85Mu+mmm7pX3ZacnByeP3/+0njUk5iBo0VbyMxVD4eIiERDOBwOWyAQaFxzEI5i6NCh+5cuXbq4IV8zHA4bcMTFQBJzSAUiK47qTBUREam3hVu2bMmMvJlKDcLhsG3ZsiUTWHik/YnZwwH+TJWl/4LiXZCaGe9qRESkiSovL79106ZNT27atOlkEvmDev2FgYXl5eW3Hmln4gaOTpEVRzctgG5nxrcWERFpsoYMGVIIjIx3HU1d4ia1yjNVNKwiIiISd4kbODLaQ8tOWuJcRESkEUjcwAF+WEU9HCIiInGX2IGj40DYthJK9sS7EhERkWYtsQNHp4GA8xNHRUREJG4SO3Bo4qiIiEijkNiBo2UOZORo4qiIiEicJXbgAD+soiXORURE4irxA0fHgbB1OZTui3clIiIizVYzCBwDwIU1cVRERCSOEj9wVC5xromjIiIicZP4gaNlR2jRXvM4RERE4ijxA4eZH1bRmSoiIiJxk/iBA/ywypalUFoU70pERESapeYRODoO9BNHNy+MdyUiIiLNUvMIHJUTRzWPQ0REJC6aR+Bo1RnS2+pMFRERkThpHoHDzA+raOKoiIhIXNQpcJjZCDNbZmYrzexHR9h/l5ktNrP5ZjbVzPKq7OtqZv82syWRY7pFr/xj0GkgFC6BsuK4vLyIiEhzdtTAYWZB4BHgEqAv8BUz61vtsE+BfOdcf2AC8ECVfc8BDzrnTgKGAoXRKPyYdRwIrgI2L4rLy4uIiDRndenhGAqsdM6tcs6VAi8AV1Y9wDn3rnOu8pzT6UAXgEgwCTnnpkSO21vluIZVean6jZ/G5eVFRESas7oEjs7AuiqPCyLbanIL8Gbkfm9gp5lNNLNPzezBSI/JIczsNjObZWaztmzZUtfaj03rrpCWpYmjIiIicRDVSaNmNhrIBx6MbAoBZwHfB04FegA3V3+ec+5x51y+cy4/Ozs7miVVLS4ycVSnxoqIiDS0ugSO9UBulcddItsOYWYXAvcCI51zJZHNBcDcyHBMOfAqMLh+JddDxwF+4mh5ydGPFRERkaipS+D4BOhlZt3NLBm4HphU9QAzGwQ8hg8bhdWe29rMKrstzgcW17/s49RpIITLNHFURESkgR01cER6Ju4E3gaWAC865xaZ2f1mNjJy2INABvCSmc01s0mR51bgh1OmmtkCwIAnYvB91E3HyhVHNY9DRESkIYXqcpBzbjIwudq2+6rcv7CW504B+h9vgVGV1Q1SMzWPQ0REpIE1j5VGK1Veql5nqoiIiDSo5hU4wA+rFC6G8tJ4VyIiItJsNL/A0WkgVJT60CEiIiINovkFjo66VL2IiEhDa36BI6s7pLTSmSoiIiINqPkFjkBAE0dFREQaWPMLHOADx+ZFUFEW70pERESaheYZODoNgooS2LI03pWIiIg0C80zcFReql7DKiIiIg2ieQaONidAcktNHBUREWkgzTNwBALQsb9OjRUREWkgzTNwgB9W2bQQKsrjXYmIiEjCa8aBYyCU74ety+JdiYiISMJrvoGjU2TFUU0cFRERibnmGzja9oSkFprHISIi0gCab+AIBCHnFJ2pIiIi0gCab+AAP6yyaQGEK+JdiYiISEJr3oGj40AoK4Kty+NdiYiISEJr3oGjky5VLyIi0hCad+Bo2wtCaTpTRUREJMaad+AIhjRxVEREpAE078ABflhl43wIh+NdiYiISMJS4Og4AMr2wbaV8a5EREQkYSlwdKycOKphFRERkVhR4MjuA6FUTRwVERGJoToFDjMbYWbLzGylmf3oCPvvMrPFZjbfzKaaWV61/a3MrMDMHo5W4VETDEGHk3VqrIiISAwdNXCYWRB4BLgE6At8xcz6VjvsUyDfOdcfmAA8UG3/L4D3619ujHQc4AOHJo6KiIjERF16OIYCK51zq5xzpcALwJVVD3DOveucK4o8nA50qdxnZkOADsC/o1NyDHQaCKV7YPuqeFciIiKSkOoSODoD66o8Lohsq8ktwJsAZhYAfg98/3gLbBCaOCoiIhJTUZ00amajgXzgwcimO4DJzrmCozzvNjObZWaztmzZEs2S6qb9SRBMVuAQERGJkVAdjlkP5FZ53CWy7RBmdiFwL3COc64ksvk04CwzuwPIAJLNbK9z7pCJp865x4HHAfLz890xfxf1FUyCDv10poqIiEiM1CVwfAL0MrPu+KBxPXBD1QPMbBDwGDDCOVdYud05N6rKMTfjJ5YedpZLo9BxICycCM6BWbyrERERSShHHVJxzpUDdwJvA0uAF51zi8zsfjMbGTnsQXwPxktmNtfMJsWs4ljpNBBKdmniqIiISAzUpYcD59xkYHK1bfdVuX9hHdp4Bnjm2MprQB0H+NuN86DtCfGtRUREJMFopdFK7ftCIEkTR0VERGJAgaNSKAU69NXEURERkRhQ4Kiq40A/pOIa/kQZERGRRKbAUVXHAVC8E3auiXclIiIiCUWBo6pOkRVHNawiIiISVQocVbXvB4GQJo6KiIhEmQJHVUmpfplzXapeREQkqhQ4qus4wA+paOKoiIhI1ChwVNdxIOzfDrvWHf1YERERqRMFjuo6DfK3mjgqIiISNQoc1XXoBxbUPA4REZEoUuCoLikNsvvoTBUREZEoUuA4kk4DNXFUREQkihQ4jqTjQCjaCrvXx7sSERGRhKDAcSRVL1UvIiIi9abAcSQ5p4AFdKaKiIhIlChwHElyOrQ7URNHRUREokSBoyaaOCoiIhI1Chw16TgA9hXCnk3xrkRERKTJU+CoScfIpeo1rCIiIlJvChw1yTkFME0cFRERiQIFjpqkZEC73jo1VkREJAoUOGrTcYCGVERERKJAgaM2nQbCno2wZ3O8KxEREWnSFDhqo4mjIiIiUaHAUZucU/yt5nGIeOtmwrTf+1sRkWNQp8BhZiPMbJmZrTSzHx1h/11mttjM5pvZVDPLi2wfaGYfm9miyL7rov0NxFRqK2jbU2eqiIAPGU9fAlPvh2cugzUfx7uixKIwJwnuqIHDzILAI8AlQF/gK2bWt9phnwL5zrn+wATggcj2IuAm51w/YATwRzNrHa3iq3t70Sb+b8pyZq/ZEb1GM3Nh9TT9ERBZ9CqEy/39ilL45yj4dDxUlMW3rkSwbiY8fSlM/QU8O1J/byQh1aWHYyiw0jm3yjlXCrwAXFn1AOfcu865osjD6UCXyPblzrkVkfsbgEIgO1rFV/XG/A18/e+z+fPUFYx6cnp0Qse6mbD6AyjZDc9eoT8C0rxtXuBvLQjBZEjNhNfugD8PghmPQ9n++NbXlM14FMJlgPNhbvW0eFckEnV1CRydgXVVHhdEttXkFuDN6hvNbCiQDHx2hH23mdksM5u1ZcuWOpR0uNXbfN5xQFl5mOmrth1XO4c2Og1chb9fXqI/AtJ8bZgLn78PA0fB+ffCzW/At+bAqAnQqjO8eTf88RT44P+geHe8q21alr3pe48scDDMdTsr3lWJRF0omo2Z2WggHzin2vaOwN+BMc65cPXnOeceBx4HyM/PP66rpQ3v0ZakoFFW4QgGAgzv0fZ4mjlUt7MgmALlkU9ueWfWv02RpuidX0JaFoz4f75no1Kvi6DnhbDmIz//4D//60PH0K/DsG9Aiyj8P0xkn70DL97k1/y54D7YMMf/3ckdGu/KRKKuLj0c64HcKo+7RLYdwswuBO4FRjrnSqpsbwW8AdzrnJtev3JrNiQvi2e/OpTkUIBhPdowJC+r/o3mDoUxk6DPZYADs/q3KdLUrPkIVk6BM797aNioZAbdzoAbJ8LYd6H72fD+A/DHk+GtH8PuDQ1fc1Ow+kN4/ga/ovHol+GE8+Cs7ylsSMKqS+D4BOhlZt3NLBm4HphU9QAzGwQ8hg8bhVW2JwOvAM855yZEr+wjO71nO24Y2pUZq7azbW/J0Z9QF7lD4arHIKkFzH42Om2KNBXO+bNSMnLg1LFHP77zYLhuHNwxA04a6ecm/GkAvP4d2L4q9vU2FQWz4B/XQutcuPFVSG8T74pEYu6ogcM5Vw7cCbwNLAFedM4tMrP7zWxk5LAHgQzgJTOba2aVgeRa4Gzg5sj2uWY2MPrfxkGjhnWltCLMS7MLotdoSks4+WpYNFHj09K8rPwPrP0YzrkbktPr/rz2feDqx+Dbc2DQaJj7D3hoCLw8FjYvjl29TcHG+TDuamjRDm56DTJiMo9epNEx545rykTM5Ofnu1mzZtWrjese+5iNu4r57/fPJRCI0jBIwSx48gK47A9w6i3RaVOkMQuH4fFzoHgX3DkLQsnH39bujfDxwzDraSjbB30uh7Pugs5DoldvU1C4FJ65FEJp8NXJkJUXtabNbLZzLj9qDYpEWUKuNDpqeB5rtxcxbeXW6DXaeQi07wdzNKwizcSS12DTfDjv3vqFDYBWHeELv4LvLoRzfuRPN3/ifHjuSn/2y9oZib/o1bbP/PcbCPm5YVEMGyJNQUIGjhH9cmjbIplx09dEr1EzGDLGL3OulUcl0VWUwzu/guyT4JRrotduehs47x4fPC663w+vPHsFPD0Cpv4ycRe92rnOh42KUj+M0vaEeFck0uASMnAkhwJce2ouU5dsZuOuKC5G1P9aCKWql0MS37znYdsKOP8nEAhGv/2UlnDGd+B/5kPvS8CFgbA/BX3Rq9F/vXjaswmeG+nnf934CrQ/Kd4VicRFQgYOgBuGdsUBz89cd9Rj6ywtC/peCfNfgtJ90WtXpDEpL4H//sYPI/a5LLavlZTm53KEUoHIfKsZj8KbP4Ki7bF97Yawb6vv2dizGUZPgE4xnTMv0qglbODIbZPOub2zeWHmWsoqDltr7PgNHgOle2DRK9FrU6QxmfUU7C7wC1E1xNozuUNhzOtwwU/hhhdh8I0w8zG/ZPr0R5vutVr274C/fxF2rIYb/qn1NaTZS9jAATBqWB6Fe0qYumRz9BrNOx3a9tKaHJKYSvbC+7/zi3f1OLfhXjd3qF/0qvcX4Io/wden+d6At34IfzkNlr/t1wRpKkr2wLhrYMsyuH48dNdS5SIJHTjO69OeTpmpjJu+NnqNmsHgm6BgJhQuiV67Io3BjL9C0VY4/7741pFzsl8Q6yv/BJxfJOvvVzWNNTxKi+Af18GGT+HLz/il30UksQNHMGB8ZWhXPli5lc+3RnHOxcAbIJCkXg5JLEXb4cOH4MTLIPfUeFfjw/2JI+D2j2HEb/wb+KNnwOv/A3uP7yKPMVdeAv8c5ZeDv/rx2M+BEWlCEjpwAFx3ai6hgPGPGVE8RbZFO/+HZP4LUFYcvXYlOtbNTPw1HWLhwz9ByW5/NdjGJJQMw2+Hb3/ql1ef8xw8NNjXWx6lSxhEQ0UZvHSzvyDblQ9H93RikQSQ8IGjfatULu7XgZdmF1BcVhG9hoeM8ZPClrwevTal/tbN9Os6TP0lPHs5rInZ9QITy55NMOMxOOXL0KFfvKs5svQ2cOkDcMd06HoaTLkPHhkKi1+L//yOcAVMvA2WTYZLf+eXcxeRQyR84AAYPSyPnUVlTF6wMXqNdj8XWuc1nzU5mkqvwexnobwYv6ZDiQ8dT17ou+FnPgFrpze+6+E0hp/t+7+DcJlflKuxy+4No16E0RP9EuEv3gTPXBa/BfnCYZj0LX+tpYvuh6F1uMidSDMUincBDeG0E9rSo10Lxk1fw9WDu0Sn0UDATx595xd+yeJEXjlwxX9gfKR7OJTql2VujKf4ffI3f5EwzI//W9APfe3b6t8MZj998NjWeZBziv803+Fkf5vV3f+7xlJZMezdfPBr7XS/7oQLQzAlPj/bHath9jP+97lNj4Z97froeQF0P8eH/nd/BY+f6+dXnf9Tv5R6Q3AO3rwb5o6Hc+/xi5mJyBE1i8BhZtwwrCu/fGMJizfspm+nVtFpeNBoePfX/g/eRfdHp83GaM6zQKTLunw/rJzSuAJHuAL+/ROY/hfo9QUYfgdsmA3dzjpYp3Owez1sXgSbFsDmhf7+ssmRVS6BpBbQoa8PIDkn+9v2fSE18vuybiasnnZou5Vt798Bewth7yZ/u2fTwVCxZ9PBfcW7av4+Kkp9+w39s/3vb/xqomff3bCvGw3BkL+Y4inX+F6a6X/1K5We+V04/U6/sFisOAdTfgqfPOmDxjk/jN1riSSAhLxa7JHsLCpl2K+ncs2QLvzqqlOi1/DzN/hTZL+7uP4XuGqsnr7Uz7oHwEHLTn7VxMYw1l+yBybcAive9kHj4l8e21LcpUWwZenBALJpIWxecGgwaJ0HmV1g3QwfbiwAXYf7IZu9hT5UVBxh8mIoFTI6QMscf5vRAVpGbjNy/P3dG/yQQEUpYHDtc9B3ZL1/LHVWuMSvc3H6nf5n19RtX+Xndix5HVp18T0epXt9T0iPc/y/SbQWM3v3/8F7v4Ght8ElDzTMImm10NVipbFrNoED4HsvzuOthRuZce+FZKREqXNn+dt+jYBrn/PLniea0iJ4oAf0uhg6DYDU1vDeb/0b/RV/hv5fjl9tO9fB89f7N81LH/SfdKOhsjdk08JIEFkIn0/z61NUatHe94Zk5EBG+2qhIrItpVXd3oTWzfTXLpn3Tz8xcsykhhva+Odo+Oy/8J150KJtw7xmQ1j9AUz6Nmz/7NDtgSRIzfS9VimtqtzPrLY9su+Q+5H9Gz71vUKfTfW9nFc8FPuhuDpQ4JDGrlkMqVQaPbwrL88p4NVP1zN6eJQuDd3zQmjV2U9WTMTAseq/fhgl/6twwnl+W5/L4KWvwsRboeAT/8m4oXt3Cmb7sFFeDKNe8uP50WLmezQyu/h1IODg2S8VZRBM9qtHRmvoI3eo/xo8xi+F/fSlfqnvdr2i035N1s/2PQHn/jixwgZAtzN978Y7vwLCgPnf35z+/tTf4t2+F6tkN2wtPHi/dG/dX8OCMHB0owgbIk1BswocA3Nb07djK8ZNX8OoYV2xaHSBBoL+U857D8CONZAVpSDTWCx9w3+y63bmwW0tc/yn8Ck/g+mPwMa58OVnG26i3qJX4JVv+N6EMa9D+z6xf83K630caQ5HtHQaCDe/4S/29fSl/jLmHfpG/3UqTf0FpLeF0+6I3WvEU/ezIfQ7P1wVTPaTOo/27xauiASSXT6UVA8ny970IbxyTtPajyDvtFh/JyIJoVlFczNj9PA8lm7aw5y1O6PXcOU595+Oi16bjUG4Apa/Bb0vhmDSofuCSTDi13DN037o4bGzfTd2LDkH7z/oF1fqOBDGvtMwYaNS5fU+Yjmps0M/uHmynycSy1M9P38fVr0LZ97lLxWfiHKH+mB8/r11P/snEPRXhc7qBh37+6Dd51IY+BUY9nU478eReSBBH2K66RopInXVrAIHwJUDO5GREmL89CiuPNq6q+/S/3QcVJRHr914WzfTz1s48dKajzn5av/Gn5oJz46Ejx6KzSJM5SW+V+OdX0L/6/wbSIt20X+dxiC7N3x1MiS3gOdGQkGU5zQ5B1Pv90OBp94a3bYbm2iHxOMJMSICNMPA0SIlxAJdFz4AABxKSURBVFWDOvOvBRvZsa80eg0PHgN7NsDK/0SvzXhb+i8/ye5oF59q38eHjj6X+tNTXxrjJ5VGy75tfphh/gtw3k/gqscglBK99hujtif40JGWBc99EdZ8HL22l7/l596c8wNISo1eu81FQ/R0iSSgZhc4AEYN70ppeZgJswui1+iJl/gzFxJl5VHn/BoVPc45uA5FbVJbwbV/9+uRLHkdnjjfX5q7vrYsgyfP92cGXPM0nHN33E8/bDCtu8JX3/RzZsZdHZk7UE/hsJ+70aYHDBxV//ZEROqoWQaOPjmtyM/LYvyMNYTDUer+Dyb5WfHL34bdUVxCPV62LPNrGtQ2nFKdmV8A6cZX/ZVHnzjfL8J0vD57B568CEr3+cmUJ199/G01Va06+Z6OrG4w/lpYMaV+7S2aCIWL4Lx7D5+XIyISQ80ycACMHp7H6m1FfPTZtug1OvgmcBUwNwEmjy79l789lsBRqcc58PX3IbuPH155+95jn9vyyd9g3DX+1NSx70CXZry8QEZ7GPMvyD4Rnv+KP3PoeFSU+TkwHU6Gfs0wvIlIXDXbwDHi5Byy0pMYF83Jo21P8LPW5/zdd103ZcsmQ+chx3+qa2Zn/8n81Fvh44f9HIy9hUd/XrgC3roH3rjLT8T92lt+aKG5a9HWT1LsOMCvTLpw4rG38ek42PG5v9aI1o4QkQbWbP/qpCYFuTY/lylLNrN5d3H0Gh5yM+xcA5//N3ptNrTdG/2iUMfTu1FVKAUu+72f5Ll+tj91du2Mmo8v2QMv3OCviTLsdrj++brNH2ku0rLgxlegy6nw8i0w9/m6P7dsv18rpstQ6P2F2NUoIlKDOgUOMxthZsvMbKWZ/egI++8ys8VmNt/MpppZXpV9Y8xsReRrTDSLr68bhnWlIux4Yea66DXa53L/xjC7CU8eXTbZ3/a5PDrtDbgebp3iA8gzl8KMxw8/dXbnOnhqhJ+jcNnv4ZLf+AtzyaFSW8Hol31P2qu3w6ynj/4c8BcY27MBLriv+Uy6FZFG5aiBw8yCwCPAJUBf4CtmVn35w0+BfOdcf2AC8EDkuW2AnwHDgKHAz8wsK3rl109e2xac1asdL3yylvKKKA2BJKXCgMg4+76tRz++MVo22Z/FkH1i9NrMOQVu+68/xfbNu2HibX4yKPhlyp84H3auhVEvJv7aEPWV3AJu+Cf0ugj+9T8w47Hajy/eDdP+ACecD921UJWIxEddejiGAiudc6ucc6XAC8AhFw1xzr3rnCuKPJwOdInc/wIwxTm33Tm3A5gCjIhO6dExengeG3cV887SOswvqKvBYyBcBnP/Eb02G0rxblj1nh9OifYn4bQsP0xy3k9gwUvw19PhH9fB0yP8ZcRvmXL0NT/ES0qD68b5Xqg3fwAf/qnmY6f/BfZv93M3RETipC6BozNQdcyhILKtJrcAbx7ncxvcBX3ak9MqlXEz1kav0fZ9IHcYzHkuNqtuxtLK//iwFK3hlOoCAb+Wxhd+BTtW+0WowuVw6e8adpnyRBBKgS8/4884mXKfn6NR/fdt3zb46GE46QroPDguZYqIQJQnjZrZaCAfePAYn3ebmc0ys1lbtmyJZklHFQoGuH5oLu8v38LabUVHf0JdDR4D21bAmo+i12ZDWDYZ0tvFfhXF8mJ/vRAADDYviO3rJapgEnzpST+M9+6v/JLlVUPHB3+Asn2+V0lEJI7qEjjWA7lVHneJbDuEmV0I3AuMdM6VHMtznXOPO+fynXP52dnZda09aq4/tSvBgDF+ZhRPke33RUhp1bRWHq0og+X/9pdkDwRj+1rdzoJgii6CFQ2BIFz5F3+G1Ad/gLd/7EPHrvUw8wnof716j0Qk7upyGsAnQC8z644PC9cDN1Q9wMwGAY8BI5xzVSdDvA38uspE0YuBe+pddZTlZKZy4UnteWlWAXdd1JuUUBTebJNbwClfhrnj4ZLf+vkLjd3qD6BkF5x4Wexfq/IiWLG83HtzEgjA5X/0VzKd/hc/AXf3Bj9cde4P412diMjRezicc+XAnfjwsAR40Tm3yMzuN7ORkcMeBDKAl8xsrplNijx3O/ALfGj5BLg/sq3RGTUsj+37Snlr4aboNTpkjB86mP9i9NqMpaVvQCgNepzbMK+ni2BFlxmM+A2ccq1fKXbDHL+9LguuiYjEWJ0WOnDOTQYmV9t2X5X7NZ5a4Jx7CnjqeAtsKGf2bEde23TGTV/DlQOjNK+14wDoONCvyTH0tsa9/oFzsOxNv7pncnq8q5HjZeaXlMeAyFyO1dMU6kQk7prtSqPVBQLGqGFd+WT1DpZtiuKl1YeM8RfLWj87em3GwsZ5sLug/quLSvx1P8sPrWh+jIg0IgocVVwzJJfkUIDxM6I4efTkayApHWY/E702Y2HpG/6skd6NapkUOR6V82POv9ffqndDRBoBBY4q2rRI5rJTOjJxznr2lRzj1U1rktrKX1Z94UR/rZDGatlk6Hqav0iYNH2aHyMijYwCRzWjhnVlb0k5k+ZtiF6jg2/2ayEsmBC9NqNpx2rYvFDDKSIiEjMKHNUMycuiT05Lxk1fg4vWKqFd8qF938a7JsfSyou1KXCIiEhsKHBUY+Ynjy7asJt5Bbui1ahfeXTDp7BxfnTajKZlk30gatMj3pWIiEiCUuA4gi8O6kx6cpBx06M4ebT/tX5lzcbWy1G0HdZ8qOEUERGJKQWOI2iZmsQXB3Xm9Xkb2FVUFp1G09tA3yth/ktQGsVrttTX8rfBhaFPA6wuKiIizZYCRw1GDetKSXmYCXMKotfokDF+6fDFr0avzfpa9ga07ASdBsW7EhERSWAKHDXo1ymTQV1bM35GFCeP5p0BbXv6lUcbg7L9sHIqnHhJ414FVUREmjwFjlqMGpbHqi37+HjVtug0aAaDb4J106FwaXTarI9V70FZkYZTREQk5hQ4anF5/460SA7y89cXM3vNjug0OuAGCCTBnOei0159LP0XpLTS0tciIhJzChy1WLRhN8XlYZZt2sO1j33MpLnr699oRrZf72Le81BeUv/2jle4Apa/Bb0uglBy/OoQEZFmQYGjFtNXbTswf6Mi7Pj2C3P54iMfMm76mvqdvTJ4DOzfDq98A9bNjFK1x6hgFuzbotNhRUSkQShw1GJ4j7YkhwIEDVJCAW4+PY/9pRX85NWFnPrr//DNf8zh3WWFlFeEj63hpBaAwaKJ8OzI+ISOpf/yQzu9Lmr41xYRkWYnFO8CGrMheVmMv3U401dtY3iPtgzJy8I5x6INu5kwu4BX567njfkbad8yhasGd+aawV3o1aHl0Rte++HB++UlsHpaw19ka9lkfxnz1MyGfV0REWmWFDiOYkheFkPysg48NjNO7pzJyZ0zuefSPry7tJAJs9fz5LTPeey9VQzoksk1Q7pwxYBOtE6vYW5Et7MglArl+4EwdDi5Yb6ZSluWw7aVMOwbDfu6IiLSbFnU1piIkvz8fDdr1qx4l3HMtuwp4bW565kwu4Clm/aQHAxwYd/2XDOkC2f3yiYUrDZ6tW6mv2T9J0/4eRTX/b3hip32B5j6c/juYsjs3HCvKyIxY2aznXP58a5DpCbq4YiS7JYp3HpWD245szuLNuzm5TkFvDZ3A5MXbCK7ZQpXDerMlwZ34cScyJBL7lD/ldHev/kvegX6XdUwxS6b7FcWVdgQEZEGoh6OGCotD/PuskImzC7g3aWFlIcd/btk8qXBXchrm86iDbsZ3i2TIVO+DDvXwTdnQIt2sS1qzyb4/Ylw3k/gnLtj+1oi0mDUwyGNnXo4Yig5FOAL/XL4Qr8ctu4tYdLcDUyYXcDPJi06cEwoYDx84U/5wgfXYZPvhi8/Hduilr3pb7W6qIiINCCdFttA2mWk8LUzuzP5O2cx5rQ8Kq9cUh52fOPfxTxUcTUsmsjklx5n9prtlJYf46m2dbVsMmR1g/YnxaZ9ERGRI1APRxyMHNiZf85aR1l5mFAwwB3n9WTr7ttZseATTl34Sy6a3YripEwGd81iaPc2DO3ehsFds0hNCtbvhUv2wKr/wqljdbE2ERFpUAoccXCk9T0AOO053OPn8kbvN3gy+0fM/Hw7f5q6AucgKWgM6NKaYT3aMLS7f05GyjH+862cChWlGk4REZEGp0mjjc27v4b3fgtfeQFOvIRd+8uYvWY7M1ZtZ8bn21mwfhcVYUcwYJzcqRVDu7dhWPe2nNqtDZnpScxes+PwIFPp5bGw8j/w/RUQVNYUSSSaNCqNnQJHY1NeCo+fC0Xb4JvTIe3Q0LCvpJw5a3cw83MfQuau20lpRRgz6JqVTsHO/YTDjqRQgKdvPpUzekbOeqkogwdPgBMvg6v+2vDfl4jElAKHNHZ1+phrZiOAPwFB4Enn3G+q7T8b+CPQH7jeOTehyr4HgMvwE1SnAN9xjS3lNCahZPjiX+CJ8+Hte/39KlqkhDirVzZn9coGoLisgrnrdjLz8+28PLuAirD/0ZaWhxn15Aw6t06jZ/sMLkpdyujiXaxsew7Z+8vITEtq8G+tqvKKMFOXFjJ91TbO79P+wPcj0VFrT5eISBwcNXCYWRB4BLgIKAA+MbNJzrnFVQ5bC9wMfL/ac08HzsAHEYAPgHOA/9a38ITWaSCc+T8w7fd+MbBaLrCWmhRkeI+2DO/RljN6tmPUk9MpLQ8TDBjXDO5CUVkFKzbvJbz6DYotiSsmJ7F/8r9p3zKFXh0y6JmdQc8OLenVPoOe7TNo2yIZq8eEUuccu4vL2by7mE27itm8239t2l3Mpl0lB+5v2VNy4DlPf7iajJQg3dtlkNsmjdw26eRmpZPbJp2ubdLp3DqN5FD8T6iK1Zt4Xdt1zlFUWsGOolJ2FpX5r/2l7CgqY1eRv91ZVMaabfuYvXYHzkHAYGj3NnRvl0FWehJZ6clktUgmKz2J1unJB7a1SksiGNBEYhGJnbr0cAwFVjrnVgGY2QvAlcCBwOGcWx3ZV/1cTgekAsmAAUnA5npX3Ryc80NY+gZM+rYfWqnDRdZqnIzqHO6P32B/m/N5eOiZrCjcy8rCvawo3MuE2QXsK6040EZWehK92rfkhPYZ9Gqf4UNJ+ww27NjPR6u20btDS9plpBwSJDbvitzuLmHTrmL2l1UcVlvr9CRyWqXSvlUqJ3VsybrtRUxftR2H/8U4ITuDzPRklmzcw38WF1Ja5Qq8ZpDTKvVACMltk3ZIIGnfMoVAlTfLuryBO+fYU1LOrqKyA2/gVd/I/f1Sdu4vY0dRGZt37WfT7oMhKbtlMq1Sk0hNCpISCpASCpKSFCA1clu5LTUpsi8U8PurHh8KULBjPw++vYyyijChoHFtfi4tUkLsLKoMEpFa9vv7pbVcmTg9OUhWejJlFWEq+xDDDpZv3suKzXvZub/sQA9YdWaQmebDR+v0Q28PhpNktuwpZtXWfeTnZTE4L4u0pCBpyUFSQ8FD/g2OVXPvkSmrCDNl0WY+Wb2dywd0apY/A0l8R53DYWbXACOcc7dGHt8IDHPO3XmEY58B/lVtSOV3wK3495WHnXP3HuF5twG3AXTt2nXImjVrjvsbSijrZ8OTF8Kg0TDyoeNvZ+N8eOwsGPkwDL7xkF3OOTbtLmbF5r0HgsjKwj2sKNzLzqKyozadHAzQvlUKOa1S6RD5yslM8betUsnJ9Nuqn9I7e80ORj05nbLyMEmhAONvHX7gj2w47Ni8p5i124pYt2M/67YX+a8dRazbvp9Nu4sPrSEUoEtr3zOSmhRg6pLCAxNrLz0lh+RQMBIkStlRVMqu/T5UlNfw5gvQMjVU5Y3Xv9Eu2bgH8L/IfTu1olvbFpSUV1BSHqakLExxeQUlZeED24rLDt7W8lKH/0xDgQM9D1VDQOsDb/7+fuu0JLJa+NvM9CRSQsFaf7aVvU+VYaYyVO3YV1Ztm7/dsc9vO1KArKnu1FCAtOQgaUlBUiNf/n7gQDBJTa6yLSnI1r0ljJ++lvKwIxQ0vn1+L07MaUlSKEByMEBSMEBS0EgKBkgOHXx8YF8osj8QOCz0NJYg45xj697SyO+w/1q73f8+r91exIad+6n8FUlNOvT/Q11pDoc0djE9VcHMegInAV0im6aY2VnOuWlVj3POPQ48Dn7SaCxralI6D4HTvwUf/gn6fhF6XnB87Sx9AzDoPeKwXWZGx8w0OmamcXbvg/MonHNs21fKis17+dsHq5i6pPBAb8TVgzvztTO7k9MqlTbHOQRTY28MEAgcrGnYEZ5bXFbB+p2RIFItkKzYvPdAkCgPuwPXsql84z4xpyWZaQeHEg59I488Tks67GJ71d/E77/y5GN6QyivCFNcHqYkEkIqg8i8gp3c9+oiysNhkoJ+ou/pPeu3vH1NP1szIzMticy0JPLa1r294rIKdhaV8eh7n/Hcx6sJR4ZqLu6bw2kntGV/WQXFZRXsL/OBa39pxSHbissq2Lq3/MD94rKKA8dUD2JlFY7fT1l+3N97KGAHAokZ7NpfDvh6T+2WRbe2GbRO9wGtddrBf+/MKiEuPTlY6+90TSFmX0n5gVC8dnvRYUG5enDLbplCblYap3bLYvPutAM9fmXlYaav2qZeDkk4dQkc64HcKo+7RLbVxVXAdOfcXgAzexM4DZhW67PkoHN/7Jcjf/07cMfHkNLy2NtY9gZ0HQ4ZdZ+YaWa0y0ihXUYKyaEAH6zceuDN9oZhefTrdPQhnqMZkpd1XH9UU5OCnJCdwQnZGYftm71mBzc8MZ2yijDJwQDjxx77J8Waaq0pINVFKBggIxg4bO2Ukzq2olf7llH/FH68P9sjSU0KkpMZ5IoBnXjhk7UHfg/Gnt2jXq/hnKOswjHj823c+uwsyip86PrN1f3p1SGD0oowZeVhyiocZRVh/7jyq9wd+jhyTOX90vIwn67dwbyCXYAfWvpsyz5WbytiR1FZrSv5JgWNzCphpHV60oHHRSXlvBSZnB0MGMN6tGVvSTkF24vYtq/0kHZaJAfJbZNOXtsWnNkzm66R+Uld26TTJSudtOSDvX7VA+3wHseQCEWaiLoMqYSA5cAF+KDxCXCDc27REY59hipDKmZ2HTAWGIH/cPwW8Efn3Os1vV6zPy32SNbNhL9dDPlfhcv/79ieu2MN/Kk/XPQLOOPbx11CY+maroumVGtTE++Js8faZk3DdpW9Njv3H5y3s6vy/v5qj4vKIsNwpYfMdwI/7+Xkzq0OhIiubQ7OLcpKTzqm3r/6/gw0pCKNXZ3W4TCzS/GnvQaBp5xzvzKz+4FZzrlJZnYq8AqQBRQDm5xz/SJnuPwFOBs/gfQt59xdtb2WAkcN3r4XPn4YbpoEPc6p+/OmPwpv/RC+NQfanhC7+kQaoWgHmRmrtnHTUzN9D1ro+OZaxIoChzR2WvirqSgtgkfPgHA53P4xpBw+nHBEz14BewvhmzNiW59IM9FYe9AUOKSxi//iBlI3yelw5SOwcx1M/XndnlO0HVZ/CCdeGtvaRJqRIXlZfPO8no0qbIg0BQocTUne6TDs6zDzcVj9wdGPXzEFXAX0uTz2tYmIiNRCgaOpueA+yOoGr93ph1lqs+wNyMiBToMapDQREZGaKHA0Nckt/AJeOz6Hd35R83FlxbDiP3DiJRDQP7OIiMSX3omaou5nwam3wvS/wtrpRz7m8/ehbJ+GU0REpFFQ4GiqLvw5ZObCa9+Esv2H71/6L0hu6cOJiIhInClwNFUpGXDlQ7BtJbz7q0P3hcOw/C2/FHooJT71iYiIVKHA0ZT1OBeG3AwfPwLrPjm4ff1s2LtZwykiItJoKHA0dRf9Alp2gtfu8BNFwQ+nBELQ66L41iYiIhKhwNHUpbaCkX+Crcvhvd/4bcsmQ7czIa11fGsTERGJiOnl6aWB9LwQBo2GD/8M7fv68HHq2HhXJSIicoB6OBLFxb+CjPYw8ev+ccuO8a1HRESkCgWORJHWGoZ/Ewj7xxPH+svai4iINAIKHIkkXAqYv19RCqunxbUcERGRSgociaTbWRBKBQtCMNk/FhERaQQ0aTSR5A6FMZN8z0a3s/xjERGRRkCBI9HkDlXQEBGRRkdDKiIiIhJzChwiIiIScwocIiIiEnMKHCIiIhJzChwiIiIScwocIiIiEnPmnIt3DYcwsy3Amno00Q7YGqVyYq0p1QpNq96mVCs0rXqbUq3QtOqtT615zrnsaBYjEk2NLnDUl5nNcs7lx7uOumhKtULTqrcp1QpNq96mVCs0rXqbUq0ix0pDKiIiIhJzChwiIiISc4kYOB6PdwHHoCnVCk2r3qZUKzSteptSrdC06m1KtYock4SbwyEiIiKNTyL2cIiIiEgjo8AhIiIiMZcwgcPMRpjZMjNbaWY/inc9tTGzXDN718wWm9kiM/tOvGs6GjMLmtmnZvaveNdyNGbW2swmmNlSM1tiZqfFu6aamNl3I78DC83seTNLjXdNVZnZU2ZWaGYLq2xrY2ZTzGxF5DYrnjVWqqHWByO/B/PN7BUzax3PGqs6Ur1V9n3PzJyZtYtHbSKxkBCBw8yCwCPAJUBf4Ctm1je+VdWqHPiec64vMBz4ZiOvF+A7wJJ4F1FHfwLecs71AQbQSOs2s87At4F859zJQBC4Pr5VHeYZYES1bT8CpjrnegFTI48bg2c4vNYpwMnOuf7AcuCehi6qFs9weL2YWS5wMbC2oQsSiaWECBzAUGClc26Vc64UeAG4Ms411cg5t9E5Nydyfw/+DbFzfKuqmZl1AS4Dnox3LUdjZpnA2cDfAJxzpc65nfGtqlYhIM3MQkA6sCHO9RzCOfc+sL3a5iuBZyP3nwW+2KBF1eBItTrn/u2cK488nA50afDCalDDzxbg/4AfAJrRLwklUQJHZ2BdlccFNOI38KrMrBswCJgR30pq9Uf8H8BwvAupg+7AFuDpyBDQk2bWIt5FHYlzbj3wO/wn2Y3ALufcv+NbVZ10cM5tjNzfBHSIZzHH4GvAm/EuojZmdiWw3jk3L961iERbogSOJsnMMoCXgf9xzu2Odz1HYmaXA4XOudnxrqWOQsBg4K/OuUHAPhpPl/8hInMfrsSHpE5ACzMbHd+qjo3z59U3+k/iZnYvfihzfLxrqYmZpQM/Bu6Ldy0isZAogWM9kFvlcZfItkbLzJLwYWO8c25ivOupxRnASDNbjR+qOt/MxsW3pFoVAAXOucoeown4ANIYXQh87pzb4pwrAyYCp8e5prrYbGYdASK3hXGup1ZmdjNwOTDKNe6Fh07Ah895kf9vXYA5ZpYT16pEoiRRAscnQC8z625myfiJd5PiXFONzMzwcwyWOOf+EO96auOcu8c518U51w3/c33HOddoP4U75zYB68zsxMimC4DFcSypNmuB4WaWHvmduIBGOsG1mknAmMj9McBrcaylVmY2Aj8cONI5VxTvemrjnFvgnGvvnOsW+f9WAAyO/E6LNHkJETgik8LuBN7G/8F+0Tm3KL5V1eoM4EZ8b8HcyNel8S4qgXwLGG9m84GBwK/jXM8RRXphJgBzgAX4/4+NamlrM3se+Bg40cwKzOwW4DfARWa2At9L85t41liphlofBloCUyL/zx6Na5FV1FCvSMLS0uYiIiIScwnRwyEiIiKNmwKHiIiIxJwCh4iIiMScAoeIiIjEnAKHiIiIxJwCh4iIiMScAoeIiIjE3P8HE9zR20uiMg8AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VuNdfRPN5hEy",
        "colab_type": "text"
      },
      "source": [
        "Useful Reference:\n",
        "\n",
        "1. [How to Use the TimeseriesGenerator for Time Series Forecasting in Keras](\n",
        "https://machinelearningmastery.com/how-to-use-the-timeseriesgenerator-for-time-series-forecasting-in-keras/)\n",
        "\n",
        "2. [TensorFlow tutorial](https://www.tensorflow.org/tutorials/structured_data/time_series)\n",
        "\n",
        "3. [steps_per_epoch v.s. validation_steps](https://stackoverflow.com/questions/45943675/meaning-of-validation-steps-in-keras-sequential-fit-generator-parameter-list)"
      ]
    }
  ]
}