Custom_ANN_Models.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.0"
    },
    "colab": {
      "name": "Custom ANN Models.ipynb",
      "provenance": []
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "388t_8gVvncM"
      },
      "source": [
        "### The Reuters dataset\n",
        "\n",
        "- _Reuters dataset_, is a set of short newswires and their topics, published by Reuters in 1986.  \n",
        "- There are 46 different topics some topics are more represented than others, but each topic has at least 10 examples in the training set.\n",
        "- The dataset can be loaded using the following script. Specifically, it retrieves 8,982 training examples and 2,246 test examples. The argument `num_words=10000` restricts the data to the 10,000 most frequently occurring words found in the data.\n",
        "\n",
        "Note, here we only use Keras to download the dataset. The deep learning model should be done in PyTorch."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_HNW8xSSG06F",
        "outputId": "b75d708c-ad76-4e84-8fc1-b00461da0c3a"
      },
      "source": [
        "!pip install torchinfo"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting torchinfo\n",
            "  Downloading torchinfo-1.5.4-py3-none-any.whl (19 kB)\n",
            "Installing collected packages: torchinfo\n",
            "Successfully installed torchinfo-1.5.4\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5uE31_rCvncN"
      },
      "source": [
        "from IPython.core.interactiveshell import InteractiveShell\n",
        "InteractiveShell.ast_node_interactivity = \"all\"\n",
        "\n",
        "import tensorflow.keras\n",
        "from tensorflow.keras.datasets import reuters\n",
        "import pandas as pd\n",
        "\n",
        "from torchinfo import summary\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "\n",
        "# add your import statement\n",
        "import numpy as np\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "from torch.utils.data import Dataset, DataLoader, WeightedRandomSampler"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Cd61I8rBvncN",
        "outputId": "a24540c8-246f-4166-bc4f-72c8712a3821"
      },
      "source": [
        "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(num_words=10000)\n",
        "\n",
        "# show the sizes training/testing samples \n",
        "len(train_data)\n",
        "len(test_data)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/reuters.npz\n",
            "2113536/2110848 [==============================] - 0s 0us/step\n",
            "2121728/2110848 [==============================] - 0s 0us/step\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8982"
            ]
          },
          "metadata": {},
          "execution_count": 3
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2246"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QliRhYRbvncO"
      },
      "source": [
        "Each example is a list of integers, each corresponds to one word (word indices).  The following shows one example. Also, you can decode it back to words, in case you are curious:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 139
        },
        "id": "Une1Qn0xvncP",
        "outputId": "fe226774-713e-49a9-f824-642ac8ad0099"
      },
      "source": [
        "# show an example from the training dataset\n",
        "print(train_data[10])\n",
        "\n",
        "# retrieve the mapping from words to indices\n",
        "word_index = reuters.get_word_index()\n",
        "\n",
        "# reverse mapping\n",
        "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
        "\n",
        "# Note that our indices were offset by 3\n",
        "# because 0, 1 and 2 are reserved indices for \"padding\", \"start of sequence\", and \"unknown\".\n",
        "decoded_newswire = ' '.join([reverse_word_index[i - 3] for i in train_data[0] if (i-3) in reverse_word_index])\n",
        "\n",
        "decoded_newswire"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[1, 245, 273, 207, 156, 53, 74, 160, 26, 14, 46, 296, 26, 39, 74, 2979, 3554, 14, 46, 4689, 4329, 86, 61, 3499, 4795, 14, 61, 451, 4329, 17, 12]\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/reuters_word_index.json\n",
            "557056/550378 [==============================] - 0s 0us/step\n",
            "565248/550378 [==============================] - 0s 0us/step\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'said as a result of its december acquisition of space co it expects earnings per share in 1987 of 1 15 to 1 30 dlrs per share up from 70 cts in 1986 the company said pretax net should rise to nine to 10 mln dlrs from six mln dlrs in 1986 and rental operation revenues to 19 to 22 mln dlrs from 12 5 mln dlrs it said cash flow per share this year should be 2 50 to three dlrs reuter 3'"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mbu6gwauvncP"
      },
      "source": [
        "The label associated with an example is an integer between 0 and 45: a topic index. In total, there are `46` topics.\n",
        "\n",
        "\n",
        "The code below shows the label for one sample and also the distribution of labels. This dataset is very imbalanced. Some labels only have a few training samples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-8K5mUTIvncQ",
        "outputId": "ace9015f-0665-4f2e-c0bc-88dcbeecc5a2"
      },
      "source": [
        "train_labels[10]\n",
        "\n",
        "# distribution of labels\n",
        "\n",
        "pd.Series(train_labels).value_counts().sort_values(ascending = False)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 5
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3     3159\n",
              "4     1949\n",
              "19     549\n",
              "16     444\n",
              "1      432\n",
              "11     390\n",
              "20     269\n",
              "13     172\n",
              "8      139\n",
              "10     124\n",
              "9      101\n",
              "21     100\n",
              "25      92\n",
              "2       74\n",
              "18      66\n",
              "24      62\n",
              "0       55\n",
              "34      50\n",
              "36      49\n",
              "12      49\n",
              "6       48\n",
              "28      48\n",
              "30      45\n",
              "23      41\n",
              "31      39\n",
              "17      39\n",
              "40      36\n",
              "32      32\n",
              "41      30\n",
              "14      26\n",
              "39      24\n",
              "26      24\n",
              "43      21\n",
              "15      20\n",
              "29      19\n",
              "37      19\n",
              "38      19\n",
              "45      18\n",
              "5       17\n",
              "7       16\n",
              "27      15\n",
              "22      15\n",
              "42      13\n",
              "44      12\n",
              "33      11\n",
              "35      10\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YEw_YiP6vncQ"
      },
      "source": [
        "### Prepare data\n",
        "First, we'll follow the \"bad-of-words\" approach to generating sample representations. You can vectorize the data with the following code. Specifically,\n",
        "- The training/test datasets is represented as a matrix\n",
        "- Each sample is treated as a row, and each word is a column. \n",
        "- If a word (say $j$) appears in a sample (say $i$), 1 is recorded in the cell ($i, j$), and 0 otherwise."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d6XJW3XPvncQ",
        "outputId": "016528c8-4f75-4ca5-9d30-ef9d3a6686a8"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "def vectorize_sequences(sequences, dimension=10000):\n",
        "    results = np.zeros((len(sequences), dimension))\n",
        "    for i, sequence in enumerate(sequences):\n",
        "        results[i, sequence] = 1.\n",
        "    return results\n",
        "\n",
        "# Our vectorized training data\n",
        "x_train = vectorize_sequences(train_data)\n",
        "# Our vectorized test data\n",
        "x_test = vectorize_sequences(test_data)\n",
        "\n",
        "# Get Y ready\n",
        "y_train = np.array(train_labels)\n",
        "y_test = np.array(test_labels)\n",
        "\n",
        "x_train.shape\n",
        "x_test.shape\n",
        "\n",
        "y_train.shape\n",
        "y_test.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(8982, 10000)"
            ]
          },
          "metadata": {},
          "execution_count": 6
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2246, 10000)"
            ]
          },
          "metadata": {},
          "execution_count": 6
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(8982,)"
            ]
          },
          "metadata": {},
          "execution_count": 6
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2246,)"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NQtrltXCvncR"
      },
      "source": [
        "### Create seqential models in different architectures\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KqA3X-swvncR",
        "outputId": "4e82a5da-1dab-4b8e-815a-407385aa43fe"
      },
      "source": [
        "# fix random number\n",
        "\n",
        "random_seed = 124\n",
        "#seed(random_seed)\n",
        "\n",
        "torch.manual_seed(random_seed)\n",
        "torch.backends.cudnn.deterministic = True"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<torch._C.Generator at 0x7f1f1cfdb0b0>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HRVA6U6XvncS"
      },
      "source": [
        "# Create dataset class\n",
        "\n",
        "class news_dataset(Dataset):\n",
        "    \n",
        "    # add your code\n",
        "\n",
        "    def __init__(self, x, y):\n",
        "        self.x = torch.from_numpy(x).float()\n",
        "        self.y = y #torch.from_numpy(y).float()\n",
        "        \n",
        "    def __getitem__(self, index):\n",
        "        return self.x[index], self.y[index]\n",
        "        \n",
        "    def __len__ (self):\n",
        "        return len(self.x)\n",
        "\n",
        "\n",
        "train_dataset = news_dataset(x_train, y_train)\n",
        "test_dataset = news_dataset(x_test, y_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PW9vNbn9vncS"
      },
      "source": [
        "# Define your models: A, B, C\n",
        "        \n",
        "\n",
        "class linear_model(nn.Module):\n",
        "    def __init__(self, h1, h2=None):  # add more parameters if needsed\n",
        "        \n",
        "       # add your code\n",
        "        super(linear_model, self).__init__()\n",
        "        if h2:\n",
        "           self.classifier = nn.Sequential(\n",
        "                nn.Linear(in_features=10000, out_features=h1),\n",
        "                nn.ReLU(),\n",
        "                nn.Linear(in_features=h1, out_features=h2),\n",
        "                nn.ReLU(),\n",
        "                nn.Linear(in_features=h2, out_features=46)\n",
        "           )\n",
        "        else:\n",
        "            self.classifier = nn.Sequential(\n",
        "                nn.Linear(in_features=10000, out_features=h1),\n",
        "                nn.ReLU(),\n",
        "                nn.Linear(in_features=h1, out_features=46)\n",
        "            )\n",
        "    \n",
        "    def forward(self, x):\n",
        "        output = self.classifier(x)\n",
        "        return output\n",
        "        "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-gYLE2ttvncS"
      },
      "source": [
        "# Define a train function\n",
        "\n",
        "\n",
        "\n",
        "def train_model(model, train_dataset, test_dataset, device, lr=0.0002, epochs=30, batch_size=256, l2= 0):  # add more parameters if needed\n",
        "    \n",
        "    history = {'train_loss': [], #train_\n",
        "               'train_acc': [],\n",
        "               'test_loss': [],\n",
        "               'test_acc': []}\n",
        "    \n",
        "    # add your code\n",
        "\n",
        "    train_loader = DataLoader(train_dataset, batch_size=batch_size)\n",
        "    test_loader = DataLoader(test_dataset, batch_size=batch_size)\n",
        "\n",
        "  \n",
        "    model = model.to(device)   \n",
        "\n",
        "\n",
        "    criterion = nn.CrossEntropyLoss()\n",
        "    optimizer = torch.optim.RMSprop(model.parameters(), lr=lr, weight_decay= l2) \n",
        "\n",
        "  \n",
        "    print('Training Start')\n",
        "    for epoch in range(epochs):\n",
        "        model.train()\n",
        "        train_loss = 0\n",
        "        train_acc = 0\n",
        "        test_loss = 0\n",
        "        test_acc = 0\n",
        "        for x, y in train_loader:\n",
        "            \n",
        "            x = x.to(device)\n",
        "            y = y.to(device)\n",
        "          \n",
        "            outputs = model(x) \n",
        "            score, pred = torch.max(outputs, 1)    \n",
        "            cur_train_loss = criterion(outputs, y)\n",
        "            cur_train_acc = (pred == y).sum().item() / batch_size\n",
        "            \n",
        "            cur_train_loss.backward()\n",
        "            optimizer.step()\n",
        "            optimizer.zero_grad()\n",
        "            \n",
        "            train_loss += cur_train_loss\n",
        "            train_acc += cur_train_acc\n",
        "\n",
        "      \n",
        "        model.eval()\n",
        "        with torch.no_grad():\n",
        "            for x, y in test_loader:\n",
        "                # move\n",
        "                x = x.to(device)\n",
        "                y = y.to(device)\n",
        "              \n",
        "                outputs = model(x) \n",
        "                score, pred = torch.max(outputs,1)                  \n",
        "                cur_test_loss = criterion(outputs, y)\n",
        "                cur_test_acc = (pred == y).sum().item() / batch_size \n",
        "                \n",
        "                test_loss += cur_test_loss\n",
        "                test_acc += cur_test_acc    \n",
        "\n",
        "        train_loss = (train_loss/len(train_loader)).item()\n",
        "        train_acc = train_acc/len(train_loader)\n",
        "        val_loss = (test_loss/len(test_loader)).item()\n",
        "        val_acc = test_acc/len(test_loader)\n",
        "        history['train_loss'].append(train_loss)\n",
        "        history['train_acc'].append(train_acc)\n",
        "        history['test_loss'].append(val_loss)\n",
        "        history['test_acc'].append(val_acc)\n",
        "        print(f\"Epoch:{epoch + 1} / {epochs}, train loss:{train_loss:.5f} train_acc:{train_acc:.5f}, Test loss:{val_loss:.5f} Test acc:{val_acc:.5f}\")\n",
        "  \n",
        "    return history\n",
        "\n",
        "    "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3erbf6hSvncT"
      },
      "source": [
        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DIMuDWsavncT",
        "outputId": "fd6d8e99-580e-42fe-edc2-9a208eaab75e"
      },
      "source": [
        "# Train Model A\n",
        "\n",
        "# dataset\n",
        "train_dataset = news_dataset(x_train, y_train)\n",
        "test_dataset = news_dataset(x_test, y_test)\n",
        "\n",
        "# load model\n",
        "modela = linear_model(h1 = 128)\n",
        "summary(modela, (24, 10000))\n",
        "hista = train_model(modela, train_dataset, test_dataset, device, epochs=50, l2 = 0)      "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "==========================================================================================\n",
              "Layer (type:depth-idx)                   Output Shape              Param #\n",
              "==========================================================================================\n",
              "linear_model                             --                        --\n",
              "├─Sequential: 1-1                        [24, 46]                  --\n",
              "│    └─Linear: 2-1                       [24, 128]                 1,280,128\n",
              "│    └─ReLU: 2-2                         [24, 128]                 --\n",
              "│    └─Linear: 2-3                       [24, 46]                  5,934\n",
              "==========================================================================================\n",
              "Total params: 1,286,062\n",
              "Trainable params: 1,286,062\n",
              "Non-trainable params: 0\n",
              "Total mult-adds (M): 30.87\n",
              "==========================================================================================\n",
              "Input size (MB): 0.96\n",
              "Forward/backward pass size (MB): 0.03\n",
              "Params size (MB): 5.14\n",
              "Estimated Total Size (MB): 6.14\n",
              "=========================================================================================="
            ]
          },
          "metadata": {},
          "execution_count": 12
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training Start\n",
            "Epoch:1 / 50, train loss:2.53346 train_acc:0.56141, Test loss:1.93352 Test acc:0.62934\n",
            "Epoch:2 / 50, train loss:1.61726 train_acc:0.66862, Test loss:1.57476 Test acc:0.65972\n",
            "Epoch:3 / 50, train loss:1.31948 train_acc:0.71039, Test loss:1.40157 Test acc:0.68533\n",
            "Epoch:4 / 50, train loss:1.14321 train_acc:0.74208, Test loss:1.29412 Test acc:0.70095\n",
            "Epoch:5 / 50, train loss:1.01899 train_acc:0.76595, Test loss:1.21689 Test acc:0.71571\n",
            "Epoch:6 / 50, train loss:0.92072 train_acc:0.78635, Test loss:1.15709 Test acc:0.72873\n",
            "Epoch:7 / 50, train loss:0.83885 train_acc:0.80414, Test loss:1.10843 Test acc:0.73915\n",
            "Epoch:8 / 50, train loss:0.76808 train_acc:0.82107, Test loss:1.06748 Test acc:0.74957\n",
            "Epoch:9 / 50, train loss:0.70543 train_acc:0.83398, Test loss:1.03235 Test acc:0.75738\n",
            "Epoch:10 / 50, train loss:0.64915 train_acc:0.84581, Test loss:1.00195 Test acc:0.76345\n",
            "Epoch:11 / 50, train loss:0.59814 train_acc:0.85699, Test loss:0.97542 Test acc:0.76736\n",
            "Epoch:12 / 50, train loss:0.55171 train_acc:0.86567, Test loss:0.95239 Test acc:0.77214\n",
            "Epoch:13 / 50, train loss:0.50934 train_acc:0.87283, Test loss:0.93250 Test acc:0.77214\n",
            "Epoch:14 / 50, train loss:0.47064 train_acc:0.88129, Test loss:0.91540 Test acc:0.77604\n",
            "Epoch:15 / 50, train loss:0.43528 train_acc:0.88585, Test loss:0.90091 Test acc:0.78299\n",
            "Epoch:16 / 50, train loss:0.40302 train_acc:0.89138, Test loss:0.88894 Test acc:0.78385\n",
            "Epoch:17 / 50, train loss:0.37358 train_acc:0.89724, Test loss:0.87924 Test acc:0.78429\n",
            "Epoch:18 / 50, train loss:0.34674 train_acc:0.90126, Test loss:0.87156 Test acc:0.78255\n",
            "Epoch:19 / 50, train loss:0.32231 train_acc:0.90560, Test loss:0.86576 Test acc:0.78385\n",
            "Epoch:20 / 50, train loss:0.30010 train_acc:0.90929, Test loss:0.86162 Test acc:0.78472\n",
            "Epoch:21 / 50, train loss:0.27992 train_acc:0.91276, Test loss:0.85900 Test acc:0.78602\n",
            "Epoch:22 / 50, train loss:0.26161 train_acc:0.91450, Test loss:0.85772 Test acc:0.78689\n",
            "Epoch:23 / 50, train loss:0.24501 train_acc:0.91699, Test loss:0.85766 Test acc:0.78559\n",
            "Epoch:24 / 50, train loss:0.22998 train_acc:0.91873, Test loss:0.85859 Test acc:0.78429\n",
            "Epoch:25 / 50, train loss:0.21636 train_acc:0.92068, Test loss:0.86039 Test acc:0.78385\n",
            "Epoch:26 / 50, train loss:0.20403 train_acc:0.92198, Test loss:0.86307 Test acc:0.78385\n",
            "Epoch:27 / 50, train loss:0.19286 train_acc:0.92307, Test loss:0.86628 Test acc:0.78385\n",
            "Epoch:28 / 50, train loss:0.18274 train_acc:0.92448, Test loss:0.87027 Test acc:0.78602\n",
            "Epoch:29 / 50, train loss:0.17356 train_acc:0.92535, Test loss:0.87458 Test acc:0.78776\n",
            "Epoch:30 / 50, train loss:0.16522 train_acc:0.92622, Test loss:0.87958 Test acc:0.78733\n",
            "Epoch:31 / 50, train loss:0.15766 train_acc:0.92687, Test loss:0.88457 Test acc:0.78646\n",
            "Epoch:32 / 50, train loss:0.15073 train_acc:0.92708, Test loss:0.89043 Test acc:0.78559\n",
            "Epoch:33 / 50, train loss:0.14445 train_acc:0.92697, Test loss:0.89613 Test acc:0.78516\n",
            "Epoch:34 / 50, train loss:0.13869 train_acc:0.92719, Test loss:0.90257 Test acc:0.78559\n",
            "Epoch:35 / 50, train loss:0.13344 train_acc:0.92708, Test loss:0.90875 Test acc:0.78646\n",
            "Epoch:36 / 50, train loss:0.12863 train_acc:0.92752, Test loss:0.91559 Test acc:0.78516\n",
            "Epoch:37 / 50, train loss:0.12423 train_acc:0.92730, Test loss:0.92205 Test acc:0.78429\n",
            "Epoch:38 / 50, train loss:0.12019 train_acc:0.92730, Test loss:0.92933 Test acc:0.78559\n",
            "Epoch:39 / 50, train loss:0.11650 train_acc:0.92719, Test loss:0.93603 Test acc:0.78559\n",
            "Epoch:40 / 50, train loss:0.11310 train_acc:0.92763, Test loss:0.94362 Test acc:0.78472\n",
            "Epoch:41 / 50, train loss:0.10998 train_acc:0.92752, Test loss:0.95051 Test acc:0.78342\n",
            "Epoch:42 / 50, train loss:0.10711 train_acc:0.92795, Test loss:0.95828 Test acc:0.78342\n",
            "Epoch:43 / 50, train loss:0.10447 train_acc:0.92784, Test loss:0.96534 Test acc:0.78299\n",
            "Epoch:44 / 50, train loss:0.10203 train_acc:0.92773, Test loss:0.97341 Test acc:0.78299\n",
            "Epoch:45 / 50, train loss:0.09979 train_acc:0.92795, Test loss:0.98042 Test acc:0.78212\n",
            "Epoch:46 / 50, train loss:0.09771 train_acc:0.92795, Test loss:0.98868 Test acc:0.78082\n",
            "Epoch:47 / 50, train loss:0.09579 train_acc:0.92806, Test loss:0.99575 Test acc:0.78082\n",
            "Epoch:48 / 50, train loss:0.09401 train_acc:0.92817, Test loss:1.00411 Test acc:0.78082\n",
            "Epoch:49 / 50, train loss:0.09236 train_acc:0.92806, Test loss:1.01119 Test acc:0.78038\n",
            "Epoch:50 / 50, train loss:0.09083 train_acc:0.92795, Test loss:1.01960 Test acc:0.78125\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "reM6NR4svncT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "38b53cef-9fe0-42dd-d9eb-7ca52ec5f523"
      },
      "source": [
        "# Train Model B\n",
        "\n",
        "# dataset\n",
        "train_dataset = news_dataset(x_train, y_train)\n",
        "test_dataset = news_dataset(x_test, y_test)\n",
        "\n",
        "modelb = linear_model(h1=128, h2=64)\n",
        "summary(modelb, (24, 10000))\n",
        "histb = train_model(modelb, train_dataset, test_dataset, device, epochs = 50, l2 = 0)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "==========================================================================================\n",
              "Layer (type:depth-idx)                   Output Shape              Param #\n",
              "==========================================================================================\n",
              "linear_model                             --                        --\n",
              "├─Sequential: 1-1                        [24, 46]                  --\n",
              "│    └─Linear: 2-1                       [24, 128]                 1,280,128\n",
              "│    └─ReLU: 2-2                         [24, 128]                 --\n",
              "│    └─Linear: 2-3                       [24, 64]                  8,256\n",
              "│    └─ReLU: 2-4                         [24, 64]                  --\n",
              "│    └─Linear: 2-5                       [24, 46]                  2,990\n",
              "==========================================================================================\n",
              "Total params: 1,291,374\n",
              "Trainable params: 1,291,374\n",
              "Non-trainable params: 0\n",
              "Total mult-adds (M): 30.99\n",
              "==========================================================================================\n",
              "Input size (MB): 0.96\n",
              "Forward/backward pass size (MB): 0.05\n",
              "Params size (MB): 5.17\n",
              "Estimated Total Size (MB): 6.17\n",
              "=========================================================================================="
            ]
          },
          "metadata": {},
          "execution_count": 13
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training Start\n",
            "Epoch:1 / 50, train loss:2.73816 train_acc:0.41612, Test loss:1.98960 Test acc:0.57465\n",
            "Epoch:2 / 50, train loss:1.68401 train_acc:0.62500, Test loss:1.60525 Test acc:0.63368\n",
            "Epoch:3 / 50, train loss:1.40015 train_acc:0.67350, Test loss:1.45258 Test acc:0.66363\n",
            "Epoch:4 / 50, train loss:1.24272 train_acc:0.70193, Test loss:1.36027 Test acc:0.67578\n",
            "Epoch:5 / 50, train loss:1.12872 train_acc:0.72721, Test loss:1.29260 Test acc:0.68750\n",
            "Epoch:6 / 50, train loss:1.03472 train_acc:0.74859, Test loss:1.23821 Test acc:0.69531\n",
            "Epoch:7 / 50, train loss:0.95395 train_acc:0.76671, Test loss:1.19353 Test acc:0.70356\n",
            "Epoch:8 / 50, train loss:0.88247 train_acc:0.78179, Test loss:1.15498 Test acc:0.71137\n",
            "Epoch:9 / 50, train loss:0.81660 train_acc:0.79720, Test loss:1.12144 Test acc:0.72569\n",
            "Epoch:10 / 50, train loss:0.75561 train_acc:0.80935, Test loss:1.09263 Test acc:0.73741\n",
            "Epoch:11 / 50, train loss:0.69965 train_acc:0.82096, Test loss:1.06830 Test acc:0.74002\n",
            "Epoch:12 / 50, train loss:0.64811 train_acc:0.83084, Test loss:1.04812 Test acc:0.74392\n",
            "Epoch:13 / 50, train loss:0.60058 train_acc:0.83941, Test loss:1.03176 Test acc:0.74826\n",
            "Epoch:14 / 50, train loss:0.55672 train_acc:0.84842, Test loss:1.01887 Test acc:0.75000\n",
            "Epoch:15 / 50, train loss:0.51581 train_acc:0.85720, Test loss:1.00847 Test acc:0.75521\n",
            "Epoch:16 / 50, train loss:0.47752 train_acc:0.86589, Test loss:1.00085 Test acc:0.75608\n",
            "Epoch:17 / 50, train loss:0.44246 train_acc:0.87478, Test loss:0.99617 Test acc:0.76302\n",
            "Epoch:18 / 50, train loss:0.41038 train_acc:0.88314, Test loss:0.99332 Test acc:0.76259\n",
            "Epoch:19 / 50, train loss:0.38103 train_acc:0.89008, Test loss:0.99246 Test acc:0.76649\n",
            "Epoch:20 / 50, train loss:0.35430 train_acc:0.89464, Test loss:0.99310 Test acc:0.76736\n",
            "Epoch:21 / 50, train loss:0.32984 train_acc:0.90007, Test loss:0.99549 Test acc:0.76736\n",
            "Epoch:22 / 50, train loss:0.30762 train_acc:0.90354, Test loss:0.99875 Test acc:0.76780\n",
            "Epoch:23 / 50, train loss:0.28728 train_acc:0.90668, Test loss:1.00343 Test acc:0.76780\n",
            "Epoch:24 / 50, train loss:0.26885 train_acc:0.91081, Test loss:1.00867 Test acc:0.76823\n",
            "Epoch:25 / 50, train loss:0.25190 train_acc:0.91363, Test loss:1.01503 Test acc:0.76866\n",
            "Epoch:26 / 50, train loss:0.23675 train_acc:0.91612, Test loss:1.02128 Test acc:0.77214\n",
            "Epoch:27 / 50, train loss:0.22296 train_acc:0.91753, Test loss:1.02901 Test acc:0.77214\n",
            "Epoch:28 / 50, train loss:0.21062 train_acc:0.91927, Test loss:1.03655 Test acc:0.77344\n",
            "Epoch:29 / 50, train loss:0.19928 train_acc:0.92177, Test loss:1.04524 Test acc:0.77214\n",
            "Epoch:30 / 50, train loss:0.18923 train_acc:0.92220, Test loss:1.05340 Test acc:0.77083\n",
            "Epoch:31 / 50, train loss:0.17987 train_acc:0.92274, Test loss:1.06298 Test acc:0.77083\n",
            "Epoch:32 / 50, train loss:0.17162 train_acc:0.92307, Test loss:1.07193 Test acc:0.77214\n",
            "Epoch:33 / 50, train loss:0.16387 train_acc:0.92437, Test loss:1.08224 Test acc:0.77300\n",
            "Epoch:34 / 50, train loss:0.15706 train_acc:0.92448, Test loss:1.09156 Test acc:0.77257\n",
            "Epoch:35 / 50, train loss:0.15057 train_acc:0.92556, Test loss:1.10223 Test acc:0.77300\n",
            "Epoch:36 / 50, train loss:0.14499 train_acc:0.92589, Test loss:1.11196 Test acc:0.77344\n",
            "Epoch:37 / 50, train loss:0.13955 train_acc:0.92665, Test loss:1.12285 Test acc:0.77257\n",
            "Epoch:38 / 50, train loss:0.13490 train_acc:0.92719, Test loss:1.13271 Test acc:0.77083\n",
            "Epoch:39 / 50, train loss:0.13034 train_acc:0.92730, Test loss:1.14376 Test acc:0.76953\n",
            "Epoch:40 / 50, train loss:0.12647 train_acc:0.92719, Test loss:1.15350 Test acc:0.76866\n",
            "Epoch:41 / 50, train loss:0.12259 train_acc:0.92752, Test loss:1.16486 Test acc:0.76780\n",
            "Epoch:42 / 50, train loss:0.11940 train_acc:0.92741, Test loss:1.17479 Test acc:0.76780\n",
            "Epoch:43 / 50, train loss:0.11608 train_acc:0.92773, Test loss:1.18588 Test acc:0.76866\n",
            "Epoch:44 / 50, train loss:0.11341 train_acc:0.92752, Test loss:1.19594 Test acc:0.76823\n",
            "Epoch:45 / 50, train loss:0.11057 train_acc:0.92773, Test loss:1.20722 Test acc:0.76780\n",
            "Epoch:46 / 50, train loss:0.10833 train_acc:0.92773, Test loss:1.21705 Test acc:0.76736\n",
            "Epoch:47 / 50, train loss:0.10589 train_acc:0.92860, Test loss:1.22809 Test acc:0.76693\n",
            "Epoch:48 / 50, train loss:0.10402 train_acc:0.92839, Test loss:1.23785 Test acc:0.76649\n",
            "Epoch:49 / 50, train loss:0.10187 train_acc:0.92871, Test loss:1.24900 Test acc:0.76649\n",
            "Epoch:50 / 50, train loss:0.10030 train_acc:0.92839, Test loss:1.25877 Test acc:0.76606\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UHIoVG9tvncT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7ac74575-19e8-444d-a945-3bd937d6a3ec"
      },
      "source": [
        "# Train Model C\n",
        "\n",
        "# dataset\n",
        "train_dataset = news_dataset(x_train, y_train)\n",
        "test_dataset = news_dataset(x_test, y_test)\n",
        "\n",
        "modelc = linear_model(h1=128, h2=16)\n",
        "summary(modelc, (24, 10000))\n",
        "histc = train_model(modelc, train_dataset, test_dataset, device, epochs = 50, l2 = 0)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "==========================================================================================\n",
              "Layer (type:depth-idx)                   Output Shape              Param #\n",
              "==========================================================================================\n",
              "linear_model                             --                        --\n",
              "├─Sequential: 1-1                        [24, 46]                  --\n",
              "│    └─Linear: 2-1                       [24, 128]                 1,280,128\n",
              "│    └─ReLU: 2-2                         [24, 128]                 --\n",
              "│    └─Linear: 2-3                       [24, 16]                  2,064\n",
              "│    └─ReLU: 2-4                         [24, 16]                  --\n",
              "│    └─Linear: 2-5                       [24, 46]                  782\n",
              "==========================================================================================\n",
              "Total params: 1,282,974\n",
              "Trainable params: 1,282,974\n",
              "Non-trainable params: 0\n",
              "Total mult-adds (M): 30.79\n",
              "==========================================================================================\n",
              "Input size (MB): 0.96\n",
              "Forward/backward pass size (MB): 0.04\n",
              "Params size (MB): 5.13\n",
              "Estimated Total Size (MB): 6.13\n",
              "=========================================================================================="
            ]
          },
          "metadata": {},
          "execution_count": 14
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training Start\n",
            "Epoch:1 / 50, train loss:3.21656 train_acc:0.30490, Test loss:2.64498 Test acc:0.52300\n",
            "Epoch:2 / 50, train loss:2.24610 train_acc:0.57487, Test loss:2.03986 Test acc:0.58854\n",
            "Epoch:3 / 50, train loss:1.80433 train_acc:0.61241, Test loss:1.78762 Test acc:0.60981\n",
            "Epoch:4 / 50, train loss:1.58792 train_acc:0.64214, Test loss:1.65083 Test acc:0.62934\n",
            "Epoch:5 / 50, train loss:1.45015 train_acc:0.66493, Test loss:1.56283 Test acc:0.64062\n",
            "Epoch:6 / 50, train loss:1.34920 train_acc:0.68088, Test loss:1.49835 Test acc:0.65234\n",
            "Epoch:7 / 50, train loss:1.26847 train_acc:0.69390, Test loss:1.44934 Test acc:0.66102\n",
            "Epoch:8 / 50, train loss:1.20002 train_acc:0.70421, Test loss:1.40866 Test acc:0.66623\n",
            "Epoch:9 / 50, train loss:1.13934 train_acc:0.71582, Test loss:1.37392 Test acc:0.67318\n",
            "Epoch:10 / 50, train loss:1.08390 train_acc:0.72754, Test loss:1.34355 Test acc:0.67491\n",
            "Epoch:11 / 50, train loss:1.03258 train_acc:0.74132, Test loss:1.31658 Test acc:0.68012\n",
            "Epoch:12 / 50, train loss:0.98438 train_acc:0.75174, Test loss:1.29240 Test acc:0.68663\n",
            "Epoch:13 / 50, train loss:0.93870 train_acc:0.76194, Test loss:1.27047 Test acc:0.69488\n",
            "Epoch:14 / 50, train loss:0.89521 train_acc:0.77279, Test loss:1.24988 Test acc:0.70312\n",
            "Epoch:15 / 50, train loss:0.85023 train_acc:0.78657, Test loss:1.22680 Test acc:0.70833\n",
            "Epoch:16 / 50, train loss:0.80471 train_acc:0.79546, Test loss:1.20695 Test acc:0.71484\n",
            "Epoch:17 / 50, train loss:0.76261 train_acc:0.80707, Test loss:1.18940 Test acc:0.72049\n",
            "Epoch:18 / 50, train loss:0.72283 train_acc:0.81456, Test loss:1.17435 Test acc:0.72743\n",
            "Epoch:19 / 50, train loss:0.68516 train_acc:0.82454, Test loss:1.16100 Test acc:0.73003\n",
            "Epoch:20 / 50, train loss:0.64948 train_acc:0.83116, Test loss:1.14952 Test acc:0.73047\n",
            "Epoch:21 / 50, train loss:0.61557 train_acc:0.83789, Test loss:1.13954 Test acc:0.73568\n",
            "Epoch:22 / 50, train loss:0.58342 train_acc:0.84570, Test loss:1.13168 Test acc:0.73741\n",
            "Epoch:23 / 50, train loss:0.55285 train_acc:0.85232, Test loss:1.12481 Test acc:0.73915\n",
            "Epoch:24 / 50, train loss:0.52388 train_acc:0.85851, Test loss:1.12006 Test acc:0.73958\n",
            "Epoch:25 / 50, train loss:0.49654 train_acc:0.86328, Test loss:1.11616 Test acc:0.74002\n",
            "Epoch:26 / 50, train loss:0.47072 train_acc:0.86903, Test loss:1.11390 Test acc:0.74262\n",
            "Epoch:27 / 50, train loss:0.44638 train_acc:0.87348, Test loss:1.11277 Test acc:0.74306\n",
            "Epoch:28 / 50, train loss:0.42338 train_acc:0.87728, Test loss:1.11274 Test acc:0.74436\n",
            "Epoch:29 / 50, train loss:0.40101 train_acc:0.88249, Test loss:1.11290 Test acc:0.74523\n",
            "Epoch:30 / 50, train loss:0.37976 train_acc:0.88770, Test loss:1.11418 Test acc:0.74566\n",
            "Epoch:31 / 50, train loss:0.36008 train_acc:0.89149, Test loss:1.11707 Test acc:0.74783\n",
            "Epoch:32 / 50, train loss:0.34176 train_acc:0.89410, Test loss:1.12024 Test acc:0.74913\n",
            "Epoch:33 / 50, train loss:0.32462 train_acc:0.89757, Test loss:1.12439 Test acc:0.75304\n",
            "Epoch:34 / 50, train loss:0.30859 train_acc:0.90007, Test loss:1.12905 Test acc:0.75174\n",
            "Epoch:35 / 50, train loss:0.29363 train_acc:0.90267, Test loss:1.13451 Test acc:0.75391\n",
            "Epoch:36 / 50, train loss:0.27971 train_acc:0.90592, Test loss:1.14003 Test acc:0.75477\n",
            "Epoch:37 / 50, train loss:0.26664 train_acc:0.90788, Test loss:1.14644 Test acc:0.75477\n",
            "Epoch:38 / 50, train loss:0.25456 train_acc:0.91048, Test loss:1.15330 Test acc:0.75477\n",
            "Epoch:39 / 50, train loss:0.24324 train_acc:0.91265, Test loss:1.16110 Test acc:0.75608\n",
            "Epoch:40 / 50, train loss:0.23274 train_acc:0.91417, Test loss:1.16842 Test acc:0.75477\n",
            "Epoch:41 / 50, train loss:0.22281 train_acc:0.91580, Test loss:1.17689 Test acc:0.75477\n",
            "Epoch:42 / 50, train loss:0.21373 train_acc:0.91732, Test loss:1.18488 Test acc:0.75521\n",
            "Epoch:43 / 50, train loss:0.20515 train_acc:0.91808, Test loss:1.19406 Test acc:0.75434\n",
            "Epoch:44 / 50, train loss:0.19721 train_acc:0.91938, Test loss:1.20291 Test acc:0.75521\n",
            "Epoch:45 / 50, train loss:0.18978 train_acc:0.92079, Test loss:1.21195 Test acc:0.75477\n",
            "Epoch:46 / 50, train loss:0.18287 train_acc:0.92090, Test loss:1.22132 Test acc:0.75521\n",
            "Epoch:47 / 50, train loss:0.17645 train_acc:0.92188, Test loss:1.23102 Test acc:0.75564\n",
            "Epoch:48 / 50, train loss:0.17044 train_acc:0.92285, Test loss:1.24041 Test acc:0.75477\n",
            "Epoch:49 / 50, train loss:0.16480 train_acc:0.92339, Test loss:1.25078 Test acc:0.75434\n",
            "Epoch:50 / 50, train loss:0.15961 train_acc:0.92470, Test loss:1.25983 Test acc:0.75434\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uOKcTEO9vncU",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 332
        },
        "outputId": "b6383b63-c02e-420f-ec82-9f9dc3f32faa"
      },
      "source": [
        "# Plot test loss curves\n",
        "epochs = range(1, len(hista['train_loss']) + 1)\n",
        "\n",
        "plt.plot(epochs, hista['test_loss'], 'r-', label='Model A')\n",
        "plt.plot(epochs, histb['test_loss'], 'b-', label='Model B')\n",
        "plt.plot(epochs, histc['test_loss'], 'g-', label='Model C')\n",
        "\n",
        "plt.title('Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1a870590>]"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1ac561d0>]"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1ac56610>]"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Test loss')"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 0, 'Epochs')"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Loss')"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f1f1ac51e50>"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5dn48e+dBQgJOyERAiRsBkJCgIiioKKVolg3bIti0aJVsbX6a7Xavm/dqq2+bxe1LrjvoFZAfAWt1A13Dcge1pCwkxBCQkIg2/3745khQ5hAtslkuT/Xda45c86ZOc+BybnPs4uqYowxxlQXEuwEGGOMaZ4sQBhjjPHLAoQxxhi/LEAYY4zxywKEMcYYvyxAGGOM8csChDFNSETiRURFJCzYaTHmRCxAmDZNRIp8lkoRKfF5P60e3/eJiFwXiLQa09TsKca0aaoa5V0XkSzgOlX9T/BSZEzzYTkIY/wQkRARuVNENotInoi8KSLdPfs6iMirnu37ReQ7EYkRkQeA8cBjnhzIY7U4T28ReUdE9onIJhH5hc++MSKSLiKFIrJHRP5+vPMH6t/CtF2WgzDGv5uBS4CzgFzgUeBx4ArgaqAL0Bc4DKQCJar6XyJyBvCqqj5by/O8DqwGegOJwGIR2ayqHwGPAI+o6isiEgUM93zG7/kbeL3GHMNyEMb4dyPwX6q6XVUPA/cAl3sql8uAHsAgVa1Q1aWqWljXE4hIX+AM4A5VPaSqy4FngemeQ8qAQSLSU1WLVPVrn+0NPr8xJ2IBwhj/+gPzPUU4+4EMoAKIAV4B/g28LiI7ReR/RCS8HufoDexT1QM+27KBPp71a4EhwDpPMdKFnu2NdX5jjssChDH+bQPOV9WuPksHVd2hqmWqeq+qDgNOBy6k6qm/LsMj7wS6i0gnn239gB0AqrpRVa8AegEPAW+JSOQJzm9Mo7EAYYx/s4AHRKQ/gIhEi8jFnvUJIpIsIqFAIa7Ip9LzuT3AgNqcQFW3AV8Cf/FUPKfgcg2ves5zlYhEq2olsN/zscoTnN+YRmMBwhj/HgHeAT4QkQPA18Cpnn2xwFu4m3MG8Cmu2Mf7uctFJF9EHq3Fea4A4nG5ifnA3T7NbCcBa0SkyPO9U1W15ATnN6bRiE0YZIwxxh/LQRhjjPHLAoQxxhi/LEAYY4zxywKEMcYYv1rVUBs9e/bU+Pj4YCfDGGNajKVLl+5V1Wh/+1pVgIiPjyc9PT3YyTDGmBZDRLJr2mdFTMYYY/yyAGGMMcYvCxDGGGP8alV1EMaYtqWsrIzt27dz6NChYCel2evQoQNxcXGEh9d+4F8LEMaYFmv79u106tSJ+Ph4RCTYyWm2VJW8vDy2b99OQkJCrT9nRUzGmBbr0KFD9OjRw4LDCYgIPXr0qHNOywKEMaZFs+BQO/X5d2rzAUJVuX/J/fx707+DnRRjjGlW2nyAEBH++uVfWbRxUbCTYoxpgUSEq6666sj78vJyoqOjufDCC4/zqWPFx8ezd+/eeh+zfPlyRIT333+/Tuc9njYfIABiomLYXbw72MkwxrRAkZGRrF69mpKSEgAWL15Mnz59TvCpxjdnzhzGjRvHnDlzGu07LUAAsVGx7C6yAGGMqZ8LLriAhQsXAu5GfcUVVxzZt2/fPi655BJSUlI47bTTWLlyJQB5eXlMnDiRpKQkrrvuOnwnb3v11VcZM2YMqamp3HDDDVRUVBz3/KrKv/71L1588UUWL17caM1+rZkrLkCs2L0i2MkwxjTErbfC8uWN+52pqfDwwyc8bOrUqdx3331ceOGFrFy5khkzZvDZZ58BcPfddzNy5EjefvttPvroI6ZPn87y5cu59957GTduHHfddRcLFy7kueeeAyAjI4M33niDL774gvDwcG666SZee+01pk+fXuP5v/zySxISEhg4cCBnn302CxcuZMqUKQ2+fAsQQGxkLP8uskpqY0z9pKSkkJWVxZw5c7jggguO2vf5558zd+5cAM455xzy8vIoLCxkyZIlzJs3D4DJkyfTrVs3AD788EOWLl3KKaecAkBJSQm9evU67vnnzJnD1KlTAResXn75ZQsQjSUmKoaCwwWUlJUQER4R7OQYY+qjFk/6gXTRRRdx22238cknn5CXl1fv71FVrr76av7yl7/U6viKigrmzp3LggULeOCBB450ijtw4ACdOnWqdzrA6iAAV8QEsKd4T5BTYoxpqWbMmMHdd99NcnLyUdvHjx/Pa6+9BsAnn3xCz5496dy5M2eeeSazZ88G4L333iM/Px+Ac889l7feeoucnBzA1WFkZ9c4IjcffvghKSkpbNu2jaysLLKzs5kyZQrz589v8DVZgMAnQBRZgDDG1E9cXBy//vWvj9l+zz33sHTpUlJSUrjzzjt56aWXAFc3sWTJEpKSkpg3bx79+vUDYNiwYdx///1MnDiRlJQUzjvvPHbt2lXjeefMmcOll1561LYpU6Y0Smsm8a05b+nS0tK0PhMGLdu1jNFPj+btn77NxYkXByBlxphAyMjIYOjQocFORovh799LRJaqapq/4y0HAcRExgBYU1djjPFhAQLoFelaCFiAMMaYKhYggPDQcHp27GkBwhhjfFiA8IiNirVWTMYY4yNgAUJE+orIxyKyVkTWiMgtfo45W0QKRGS5Z7nLZ98kEVkvIptE5M5ApdMrJjLGchDGGOMjkB3lyoHfquoyEekELBWRxaq6ttpxn6nqUcMeikgo8DhwHrAd+E5E3vHz2UYTGxXLl9u+DNTXG2NMixOwHISq7lLVZZ71A0AGUNshDscAm1Q1U1VLgdeBgLY/9Q7Y15qa/RpjAq85DPcdHx9PcnIyqampJCcns2DBgjqduyZNUgchIvHASOAbP7vHisgKEXlPRJI82/oA23yO2U4NwUVErheRdBFJz83NrXcaY6NiKSkvoai0qN7fYYxpe5rLcN8ff/wxy5cv56233vLbYa8+Ah4gRCQKmAvcqqqF1XYvA/qr6gjgn8Dbdf1+VX1aVdNUNS06Orre6fT2prZ6CGNMXQV7uG9fhYWFRwb+a6iADtYnIuG44PCaqs6rvt83YKjqIhF5QkR6AjuAvj6Hxnm2BYxvZ7nBPQYH8lTGmAAI4mjfQR/uG2DChAmoKpmZmbz55psNvnYIYIAQN0P2c0CGqv69hmNigT2qqiIyBpejyQP2A4NFJAEXGKYCVwYqrWA5CGNM/QV7uG9wRUw9e/Zk8+bNnHvuuZx99tlERUU16LoCmYM4A/gZsEpEvHH9D0A/AFWdBVwOzBSRcqAEmKoun1UuIr8C/g2EAs+r6poAptVGdDWmhQvyaN9BG+67uoEDBxITE8PatWsZM2ZMvdMBgW3F9LmqiqqmqGqqZ1mkqrM8wQFVfUxVk1R1hKqepqpf+nx+kaoOUdWBqvpAoNLp1aNjD0Il1HIQxph6CdZw39Xl5OSwZcsW+vfv3+BrsgmDPEIkhF6RvSxAGGPq5XjDfc+YMYOUlBQ6dux41HDfV1xxBUlJSZx++ul+h/uurKwkPDycxx9//IQ3/AkTJhAaGkpZWRkPPvggMTExDb4mG+7bx6inRtG7U2/evfLdRkyVMSZQbLjvurHhvhvAxmMyxpgqFiB8eHtTG2OMsQBxlJjIGPYU7aFSK4OdFGOMCToLED5io2IpqywjvyQ/2EkxxpigswDhwzrLGWNMFQsQPqyznDHGVLEA4SMmqmo8JmOMqY3mMNx3UVERN9xwAwMHDmT06NGcffbZfPONv8Gz68Y6yvmwIiZjTF35DvcdERERlOG+r7vuOhISEti4cSMhISFs2bKFtWsbPr+a5SB8dGnfhfah7S1AGGPqJJjDfW/evJlvvvmG+++/n5AQd0tPSEhg8uTJDb4uy0H4EBHrLGdMC3Xr+7eyfHfjjvedGpvKw5NOPApgMIf7XrNmDampqYSGhjbehXtYgKgmJirGchDGmDppDsN9B4IFiGpio2LJ2p8V7GQYY+qoNk/6gRSs4b6TkpJYsWIFFRUVjZ6LsDqIamIjbbgNY0zdBWu474EDB5KWlsbdd999pB4jKyvrSJ1IQ1iAqCY2Kpa9B/dSUVn7OWCNMeZ4w30vXbqUlJQU7rzzzqOG+16yZAlJSUnMmzfP73DfKSkpnHfeeezateu453722WfZs2cPgwYNYvjw4VxzzTWNUixlw31X8+R3T3LTopvY9dtdR5q9GmOaJxvuu25suO8Gss5yxhjjWICoxjrLGWOME7AAISJ9ReRjEVkrImtE5BY/x0wTkZUiskpEvhSRET77sjzbl4tIw8qN6uDIeExF1hfCmJagNRWTB1J9/p0C2cy1HPitqi4TkU7AUhFZrKq+/b+3AGepar6InA88DZzqs3+Cqh5/cJJGFhNpRUzGtBQdOnQgLy+PHj16ICLBTk6zpark5eXRoUOHOn0uYAFCVXcBuzzrB0QkA+gDrPU55kufj3wNxAUqPbUV2S6SqHZRFiCMaQHi4uLYvn07ubm5wU5Ks9ehQwfi4up2i22SjnIiEg+MBI43vOC1wHs+7xX4QEQUeEpVn67hu68HrgeONBNrqNioWHYXW4AwprkLDw8nISEh2MlotQIeIEQkCpgL3KqqhTUcMwEXIMb5bB6nqjtEpBewWETWqeqS6p/1BI6nwTVzbYw0x0bFWh2EMabNC2grJhEJxwWH11R1Xg3HpADPAher6pH+6aq6w/OaA8wHxgQyrb5io6w3tTHGBLIVkwDPARmq+vcajukHzAN+pqobfLZHeiq2EZFIYCKwOlBprS4m0gbsM8aYQBYxnQH8DFglIt4xeP8A9ANQ1VnAXUAP4AlPC4RyT4++GGC+Z1sYMFtV3w9gWo8SGxVL/qF8Dpcfpn1Y+6Y6rTHGNCuBbMX0OXDcdmeqeh1wnZ/tmcCIYz/RNHznpu7XpXEqvo0xpqWxntR+WGc5Y4yxAOGXdZYzxhgLEH7ZeEzGGGMBwq9ekW4cdQsQxpi2zAKEH+3D2tM9ojt7iq0OwhjTdlmAqIH1hTDGtHUWIGpgvamNMW2dBYgaWIAwxrR1FiBqEBsVa3UQxpg2zQJEDWKjYikqLaKotCjYSTHGmKCwAFEDb2c5601tjGmrLEDUwDrLGWPaOgsQNfAdsM8YY9oiCxA1sByEMaatswBRg54dexIiIRYgjDFtlgWIGoSGhBLdMdoChDGmzbIAcRzWWc4Y05ZZgDiO+K7xrM9bH+xkGGNMUAQsQIhIXxH5WETWisgaEbnFzzEiIo+KyCYRWSkio3z2XS0iGz3L1YFK5/GcFncaG/I2kHcwLxinN8aYoApkDqIc+K2qDgNOA34pIsOqHXM+MNizXA88CSAi3YG7gVOBMcDdItItgGn1a2zcWAC+3v51U5/aGGOCLmABQlV3qeoyz/oBIAPoU+2wi4GX1fka6CoiJwE/BBar6j5VzQcWA5MCldaapPVOI1RC+Wr7V019amOMCbomqYMQkXhgJPBNtV19gG0+77d7ttW0PSDKy6G4+Njtke0iGRE7wgKEMaZNCniAEJEoYC5wq6oWBuD7rxeRdBFJz83NrfPny8uhc2f4y1/87x8bN5Zvd3xLRWVFA1NqjDEtS0ADhIiE44LDa6o6z88hO4C+Pu/jPNtq2n4MVX1aVdNUNS06OrrOaQwLgz59YMMG//vHxo2lqLSI1Tmr6/zdxhjTkgWyFZMAzwEZqvr3Gg57B5juac10GlCgqruAfwMTRaSbp3J6omdbQAweDBs3+t83tq+rqLZiJmNMWxPIHMQZwM+Ac0RkuWe5QERuFJEbPccsAjKBTcAzwE0AqroP+BPwnWe5z7MtIIYMcQFC9dh9CV0T6BXZywKEMabNCQvUF6vq54Cc4BgFflnDvueB5wOQtGMMHuwqqXftgt69j94nIoyNG8tX2yxAGGPaFutJjQsQcJxiprixbNy3kb0H9zZdoowxJsgsQJSXM+SOS4HjVFT3tQ5zxpi2xwJEWBh985bTLqSsxhxEWu80wkLCrJjJGNOmWIAAQocOYVC7rTUGiI7hHRkRYx3mjDFtiwUIgKFDGVy2lg0b/DRj8vB2mCuvLG/ChBljTPBYgABITGRIRQabN0NFDR2mx/YdS3FZsXWYM8a0GRYgABITGcxGDh8Wtm3zf4h3ZFerhzDGtBUWIOBIgICam7rGd40nJjLG6iGMMW2GBQiAmBiGdHJTi9YUIESEsX3HWoAwxrQZFiAARDhpaFciQ0tq7AsBrphp075N5BbXfdRYY4xpaSxAeMjQRAaHbK4xBwE2w5wxpm2xAOGVmOiauq6red6HIx3mrJjJGNMGWIDw8lRUb8kOoazM/yER4RGkxqZagDDGtAkWILwSExnCBioqhKysmg+zDnPGmLaiVgFCRCJFJMSzPkRELvLMFtd6DBzI4NAtQM2D9oELEAfLDrJqz6omSpgxxgRHbXMQS4AOItIH+AA3EdCLgUpUUISHMyS+FKi5qSvYDHPGmLajtgFCVPUgcBnwhKr+GEgKXLKCo8fwk+gaUnDcHET/Lv2JjYq1AGGMafVqHSBEZCwwDVjo2RYamCQFjwxNZLBuYOOGypqP8cww9+W2L5swZcYY0/RqGyBuBX4PzFfVNSIyAPg4cMkKksREhuh6Nh6nqSvAeQPOIzM/k/Sd6U2UMGOMaXq1ChCq+qmqXqSqD3kqq/eq6q+P9xkReV5EckTE7/CnInK7iCz3LKtFpEJEunv2ZYnIKs++prsLe5q6bt0ZxqFDNR82LWUakeGRPPndk02WNGOMaWq1bcU0W0Q6i0gksBpYKyK3n+BjLwKTatqpqv+rqqmqmorLnXyqqvt8Dpng2Z9WmzQ2ipNPZggbUBU2b675sM7tOzMteRpzVs8hvyS/yZJnjDFNqbZFTMNUtRC4BHgPSMC1ZKqRqi4B9h3vGB9XAHNqeWzgdO3K4B7uhn+8imqAmafMpKS8hJdWvNQECTPGmKZX2wAR7un3cAnwjqqWATVPv1YHItIRl9OY67NZgQ9EZKmIXH+Cz18vIukikp6b2/BB9AYPDQOO39QVIDU2ldPiTmNW+ixUG+WfwhhjmpXaBoingCwgElgiIv2BwkZKw4+AL6oVL41T1VHA+cAvReTMmj6sqk+rapqqpkVHRzc4MV2G96WX5LBh/Ylv+jPTZrI+bz0fZ7W++npjjKltJfWjqtpHVS9QJxuY0EhpmEq14iVV3eF5zQHmA2Ma6Vwnluhp6ppRw4BMPn6S9BO6R3TnyXSrrDbGtD61raTuIiJ/9xbliMjfcLmJBhGRLsBZwAKfbZEi0sm7DkzEVYw3Dc+YTBs3nDgH0SGsAzNSZ/D2urfZeWBnEyTOGGOaTm2LmJ4HDgA/8SyFwAvH+4CIzAG+Ak4Wke0icq2I3CgiN/ocdinwgaoW+2yLAT4XkRXAt8BCVX2/lulsOE9T11157Tlw4MSH35B2A+WV5Ty77NnAp80YY5qQ1KaCVUSWe5qjHndbsKWlpWl6egO7TVRWMjdiGpeXzmHZMhg58sQf+eGrP2RNzhqybs0iLCSsYec3xpgmJCJLa+pOUNscRImIjPP5wjOAksZIXLMTEsLgBNeT+kRNXb1mps1kx4EdvLvh3QAmzBhjmlZtA8SNwOOeHs5ZwGPADQFLVZANSukInLipq9eFQy4krnOcVVYbY1qV2rZiWqGqI4AUIEVVRwLnBDRlQdRx+ADi2MbGdbWbFCgsJIxfjPoFH2z+gE37NgU4dcYY0zTqNKOcqhZ6elQD/CYA6WkePBXVG1YdrvVHrht1HaESylPpTwUwYcYY03QaMuWoNFoqmhtvU9ctta9w7t2pN5ckXsLzy5+npKx1Vs8YY9qWhgSI1ju+xODBDGYTeQfas6+2o0kBt5x6C/tK9vHnz/4cuLQZY0wTOW6AEJEDIlLoZzkA9G6iNDa9iAgG9yoAal9RDTC+/3imj5jOg188aHNWG2NavOMGCFXtpKqd/SydVLVVN/gfcrIrQatLgAD428S/0bVDV37xf7+govL4Ew8ZY0xz1pAiplZtwMguhFBRq0H7fPXs2JOHf/gw3+z4hie+eyJAqTPGmMCzAFGDdkmDiSeLjSsO1vmzVyZfyaRBk/j9h79na8HWAKTOGGMCzwJETRITGcEKPvsqlIo6lhSJCE9OfhJFmblwps0XYYxpkSxA1CQxkSuYw469Hfi4HtM9xHeN54FzHmDRxkW8seaNxk+fMcYEmAWImkRH86Oun9O13UFefrl+X3HzmJs5pfcp3PK+a/5qjDEtiQWImojQIWkgP+36b+bOhaKiun9FaEgoz/zoGfIO5nHbB7c1fhqNMSaALEAcz3nnMT3nrxw8CHPnnvhwf0bEjuB3Z/yOF5a/wMINCxs3fcYYE0AWII5nyhTG8iUDowvqXcwE8Mcz/0hqbCpT505l+e7ljZc+Y4wJIAsQx5OUhAwZwvSo+Xz8MWytZ4vViPAIFl65kK4dujJ59mS2FWxr3HQaY0wAWIA4HhGYMoWrsh9AFV57rf5f1btTbxZduYii0iImz55MwaGCxkunMcYEgAWIE5kyhQGVmxg/eDcvvwwN6dKQHJPM3J/MJWNvBj/+148pqyhrvHQaY0wjC1iAEJHnRSRHRFbXsP9sESkQkeWe5S6ffZNEZL2IbBKROwOVxloZNQri47m6wxusWwfffdewr/vBgB/wzI+eYXHmYm549wbrRGeMabYCmYN4EZh0gmM+U9VUz3IfgIiEAo8D5wPDgCtEZFgA03l8InDZZVye8Sc6dNAGVVZ7XZN6DXedeRcvLH+B+5fc3/AvNMaYAAhYgFDVJUB9eoeNATapaqaqlgKvAxc3auLqasoUupTncUlqNnPmQGlpw7/ynrPvYfqI6dz1yV08s/SZhn+hMcY0smDXQYwVkRUi8p6IJHm29QF8m/ls92zzS0SuF5F0EUnPzc0NTCpPOw1692a6vMK+fbBoUcO/UkR45kfPMGnQJK5/93ru+eQeK24yxjQrwQwQy4D+qjoC+Cfwdn2+RFWfVtU0VU2Ljo5u1AQeERICl17Ked//DzG9KhulmAmgXWg73pn6DtekXsO9n97Lzxf8nNKKRsieGGNMIwhagFDVQlUt8qwvAsJFpCewA+jrc2icZ1twTZlC2KEipo3ZxLvvQl5e43xteGg4z1/0PPeefS8vrXiJC167wJrAGmOahaAFCBGJFRHxrI/xpCUP+A4YLCIJItIOmAq8E6x0HjF+PPTsydXlz1FWBq+/3nhfLSLcddZdvHjxi3ya/SnjXhhnnemMMUEXyGauc4CvgJNFZLuIXCsiN4rIjZ5DLgdWi8gK4FFgqjrlwK+AfwMZwJuquiZQ6ay1sDC45BJSPn+CESmVvPhiw/pE+HN16tW8P+19thZs5dRnT+X7Xd837gmMMaYOpDVVjKalpWl6enrgTvD++3D++Tx54wpumpXCW2/BlCmNf5pVe1YxefZkcg/m8veJf+fGtBvxZLaMMaZRichSVU3zty/YrZhalnPOgS5d+EXxw4wYAbfcUr9hwE8kOSaZ737xHWf1P4ubFt3EZW9eRt7BRqr0MMaYWrIAURft2sFFFxH27ts88Wg5O3bAffcF5lQxUTEsmraIv038Gws3LGTErBF8kvVJYE5mjDF+WICoqylTID+f0w99xLXXwj/+Aav9DibScCESwm/G/oavrv2KjuEdOeelc/jjR3+kvLI8MCc0xhgfFiDqauJEiIyEuXN58EHo3BluuqnxK6x9je49mmU3LOPq1Ku5/7P7Gf/CeFbnBCgqGWOMhwWIuoqIgMmTYd48ekaW8OCD8Nln8MorgT1tVLsoXrj4BeZMmcOGvA2MfGokd/7nTg6WHQzsiY0xbZYFiPq46SbYuxeeeIJrr4VTT4Xbb4f8/MCfeurwqaz/1XquSrmKh754iKQnkmwqU2NMQFiAqI+zzoLzzoO//IWQokKefNLFi//+76Y5fc+OPXnh4hf45OpPiAiL4MI5F3L5m5ezozD4Hc6NMa2HBYj6euABN97Gww8zciT86lfw5JMQyG4Y1Z0VfxbLb1zOn8/5Mws3LiTx8UTuX3I/xaXFTZcIY0yrZR3lGuKyy+A//4EtWygI60FiIsTFwddfQ2ho0yUDIDM/k9s+uI356+YTGxXLPWfdw4yRMwgPDW/ahBhjWhTrKBcof/qT6yn30EN06eKavKanw51BmANvQLcBzPvpPL6Y8QUDuw3kxoU3MvzJ4czLmGfDiBtj6sUCREMkJcFVV8Fjj8HOnUydCr/8Jfz1r/Dii8FJ0ul9T+ezn3/GgqkLCJVQprw5hbHPjWXx5sUWKIwxdWIBoqHuuQfKylydBPDww/CDH8D118MXXwQnSSLCRSdfxMqZK3nuoufYXridia9O5NRnT2XBugVUamVwEmaMaVEsQDTUgAFw3XXw9NOQmUlYGLz5JsTHw6WXQnZ28JIWFhLGjJEz2PzrzTx14VPkleRxyRuXMGLWCGavmm09so0xx2UBojH893+74cDvvReAbt3gnXfc3NUXXRSYAf3qon1Ye64ffT3rf7WeVy59hUqtZNq8aSQ+lsis9FnW6skY45cFiMbQp49r5/rKK7B2LQCJiS4nsXq1q6aobAalOmEhYVyVchWrZq5i3k/m0S2iGzMXziTuH3Hc/sHtZO3PCnYSjTHNiAWIxnLHHRAVBXfddWTTxImuZdOCBU3Xia42QiSES4deyrfXfcvnP/+c8wacxz++/gcDHx3IZW9cxidZn1iFtjHGAkSj6dkTfvtbmDsXPvroyOabb3YV1n/5C8yaFcT0+SEinNHvDN788ZtsuWULd5xxB0uylzDhpQkkP5nMI18/YvNQGNOGWUe5xlRcDGlpUFAAK1e6oIGri7jsMli4EB55BH796+Al8URKykqYvWo2Ty19iu92fke70HZcNvQyrht5HRMSJhAi9kxhTGtyvI5yAQsQIvI8cCGQo6rD/eyfBtwBCHAAmKmqKzz7sjzbKoDymhJfXdADBMCKFTBmjCtfeucd8EwVWloKU6fC/Pnw0EPwu98FN5m1sWL3Cp77/jleXfkq+YfySeiawIyRM/hZys/o37V/sJNnjGkEwepJ/SIw6Tj7twBnqWoy8Jq4cLwAABweSURBVCfg6Wr7J6hqam2DQ7MxYoTrKffuu/DPfx7Z3K4dvPGGCxJ33OFmomvumbcRsSN49PxH2fnbncy+bDYJ3RL448d/JP6ReCa8NIEXvn+BwsOFwU6mMW3W/pICFq74mocXvRuQ7w9oEZOIxAPv+stBVDuuG7BaVft43mcBaaq6ty7naxY5CHB3/osugg8+gG++gdTUI7sqKly3iRdfhN//3vWv82QyWoSs/Vm8uvJVXl7xMhv3bSQiLIJLh17K9JTp/GDADwgNaeJBqIxpA/Yf2s9XmSv4cNUalm7NYNP+teRoBqXtdwEgh7pT8ee9SD1uJkEpYvKcOJ7aBYjbgERVvc7zfguQDyjwlKpWz1341WwCBLjxv1NS3JRzS5e6Weg8Kith5kzXt+7//T/4299aVpAAUFW+2fENL694mddXv07+oXziu8Zzw+gbmDFyBr0iewU7ica0OBWVFWzZv4Wvtqxk8crlLNuxgqxDKygO9+lxe7gTIfuG0r1yKAlRQ0ntPYwzhw3lykkDCQlpZQFCRCYATwDjVDXPs62Pqu4QkV7AYuBmVV1Sw+evB64H6Nev3+jsYHZdru6jj9y4Gz//OTz33FG7VOHWW+HRR+Gaa9xQ4R06BCeZDXW4/DAL1i9gVvosPs76mPCQcC4fdjkz02Yyrt+4ej3VGNOaVWolW/K3sHL3GpasW8PS7LVsLFhDrq6jIqTEc1AI5J1MROEIEjqkMjpuBBOShnP2qD707y+ENFIFQbMNECKSAswHzlfVDTUccw9QpKp/PdH5mlUOwuu//gv+/GeYM8dVQPhQdZ2v770XTjnFtZDt2zdI6WwkGbkZzEqfxUsrXqLgcAHDew3n56k/Z+rwqfTu1DvYyTOmSakqu4p2sTpnNcu2r+bzDatZtWcVO8vWUh7iM11wQRzkJtGtPImEqGGM6Z/CpFHDGZsWQa8AZ8abZYAQkX7AR8B0Vf3SZ3skEKKqBzzri4H7VPX9E52vWQaIsjI480zXw3rpUhg06JhD3n4bpk93OYh//ctNWNfSFZcW8/rq15m1dBbpO9MRhAkJE5iWPI0pQ6fQpUOXYCfRmEajquwu2s2a3DWszllDevZavt+xhi0H1lKCz1zERTGQM5wOhckMiEpiZJ/hjEscyqkjupCY6Ka8b2rBauY6Bzgb6AnsAe4GwgFUdZaIPAtMAbxlQuWqmiYiA3C5CoAwYLaqPlCbczbLAAGwZYvrH9G1K3z2GfQ+9kl63Tq45BLYtAn+/nfXwa61lMys37ue2atmM3v1bDbt20T70PZMHjKZnyb9lPMHnU+n9p2CnURjaqVSK8nen03G3gzW5q5lzZ4Mlm3LYNP+DA7q/qoDD3aH3CTIHUbnw0kM6zmcMwYP58zR0Ywa5UbnaS5/30HLQTS1ZhsgAL79Fs49F/r3h08/hR49jjmksNDlJBYsgJ/9DJ56KjhPFIGiqqTvTOe1Va/x+urX2VO8h3ah7fjBgB9wycmX8KOTf0RsVGywk2kMh8oPsSFvA+v2riMjN4N1eevIyFnHur3rOVxZUnVgcS/IHQq5Q2l/YBiDOicxMm4YY4bFkJIiDB/u90+9WbEA0Vx88glMmgTJyfDhh66FUzWVla7p6913w/Dh8OqrrjFUa1NRWcFX279ifsZ83l7/Npn5mQjC2L5j+dGQHzFx4ERSY1Ot57YJqKLSIjJyM47kCNbmriVjbwaZ+ZlV86aoEF4cT9muRBcM9iYSWTKMkXFDGZvandGjYfRoGDiw+eQK6sICRHPy7rtuoogzzoD33qsxi/Dee67x0759bmbT225r+nmum4qqsjpnNW+ve5v56+bz/e7vAegR0YNzB5zLDxJ+wHkDzyO+a3xwE2papOLSYjbu28imfZvYmOdeN+W79V1Fu44cF6LhRJQMoWzHMEp3DIXcYbA3kf6dhjBiWAQpKe5hLTXVBYPGakUUbBYgmps5c2DaNLjgAjf2Rni438P27nX9Jd56C04/HV56yW8dd6uz68AuPtzyIf/J/A+LMxez88BOwM27fWb/Mxnfbzzj+41nUPdB1oTWAFBaUcrWgq1szNvIhrwNrM9bz/q89WzI28D2wu1HHds5JIbIw4Op3DuIgi2DOLR1GOQOI/TAAJKHhTN6NIwa5QLB8OF+M/qtigWI5uipp+DGG13T11dfrTF7oOriyS9/6cZz+tvf4IYbWmZWtj5UlXV717E4czEfbfmIz7d+Tl6JG2E2NiqWcf3GMb7feE7tcyojYkfQIayFdiYxx6Wq5JXksWnfJjbv20xmfiaZ+zPZkr+FzPxMdhzYcdRUul3ad+Wk8JOJODiEwztPZu+6IeSsGwz7BkJpJzp1cjf/lBQXDEaNciW/7dsH8SKDxAJEc/U//+MGZrrqKteRrl27Gg/dvh1mzIDFi904gI8/3jZyE9VVaiXr9q7js+zP+GyrW7YWbAXchEjJvZJJ651GWu80Tul9Ckm9kmgXWvO/q2k+DpYdZGvBVrL2Z5G9P5vsgmw25292RUL7Nh0z7lfvTr1J6JpAr/ABtC9JoDxnAHmbBpL57clkZ/TEjQPq/k5GjXLBIDnZvfbv33Yesk7EAkRz9sADbjahCRNcT7lu3Wo8VNXNKXHHHXD4sBsR9ve/h44dmzC9zdC2gm2k70wnfWc63+38jvSd6eQfcm3Pw0LCSOyZSHKvZJJ7JZMSk0JyTDJ9O/e14qkmVF5Zzp6iPWwr3Ma2gm1sLdjq1gvdevb+bHIP5h71mfCQcOK7xjOw+0AGdRtE36iBsG8QBVkD2bkmnnWrI1i9+ugpfRMSXIty7zJqlGtdbmpmAaK5e+UVuPZa96izaBHExx/38F274Pbb4bXX3JPQP/7h+lDY/c5RVbbs38J3O75j5Z6VrMxZyao9q8guqBqGJTI8kkHdBx2zDOg2gD6d+tigg7V0uPwwu4t2s6toF7sO7Dr61Wc9pzjnqCIggKh2UfTt3Je+XfoS3yWe/l37079Lf+K7xtNZ+5OTeRIrvg9l2TJYtsz1FfLernr2dLmB4cPdkpwMSUmtv74gECxAtASffOJaN7Vr51o6nXLKCT+yZImbCnvVKlfs9M9/wpAhgU9qS1VwqIDVOatZuWcl6/PWHym6yMzPpKyy7MhxYSFhxHWOO3Kz6t+lP/269CMmKoZekb3oFdmL6I7RRLaLPM7ZWqaDZQfJO5hHXknekdfc4lxyD+ZWvR7MJac4h91Fu9lXsu+Y7wiREHpF9uKkqJM4qdNJ9I7q7V479T4SEPp16UeX9l0oLRXWrnW/4ZUr3euqVe4hyKtPH45UHHuX3r3tgaixWIBoKTIyXMumPXtczfTFF5/wI+Xl8MQT8Mc/QkkJ/OIX8Ic/uD8qUzsVlRVsK9x2pAI0u8CVf3vLwXcU7kA59u+kY3hHojtG06NjD7pHdHdLh+50i+hG94judGrXich2kXQM70jH8I5Ehrv1DmEdCA8NJzwknHah7Y6sh4WE+S328v6NKnrMenllOaUVpUeWssoySitKKS4t5kDpAQ4cPkDh4cIj6wdKD1BwqIDC0kIKDxe69cOF7D+0n7ySPA6VH6rx36l7RHeiO0a7ABkZTUxkzJEg4PsaHRlNWEjYMZ8vLnbzaS1bBt9/717XrHGj0YCrIB42zOUGvLmDkSMhJqY+/6umtixAtCR79ri5JL77Dv73f+E3v6nVo9KePa5z3XPPuQZRN90Ed95JwAf6agtKK0rZeWAnOcU5R5bcYvcUnXMwh30l+8gvyWdfyT63fiif8sryYCfbr4iwCLp06EKX9l3o3L4zXTp4Xtt3oUdED3p07HHMqzcI+rvp+1NeDhs3wurVblm1yr1u2lRVRBQdXZUbSE11FceDBkFY7U5hGpEFiJbm4EE31sa8eXDhhfDCC0fmtz6RzEzXse7ll93gfzff7Oormnt3/9ZEVSkqLeJA6QEOlh2kuLSYg2UH3XpZMYfKD1FWUXbkad+7frygIp4WOSJy1LpvLqRdaDu3HhJOx/COdGrfic7tO9OpXSc6te9EVLuoWt/ka6uw0OUKvv8eli93r2vXuibZ4DqTDRpUlSsYNcrlCprTWERtnQWIlkjVVSrcfrsLDq++6lo61dL69W4Y8ddfh6go13fiV79yldrG1NXhwy5XsGaNCwBr1riAsHlz1THR0e7mP2JEVRFRsEYoNbVnAaIl+/5715lu40Y3t8Tdd9cpH756tctRzJ3rYs4ll8Att8D48fYEZ46lCtnZLlewYoWrOF6zxv38KircMSEhbqiJlBQXEEaOdMVEJ51kv6mWyAJES1dUBL/+tStqOv10mD27zlmBbdtc57pnnnHjO6Wmuq+84oqWO5OdqT9VyMlx7SLWrXNBwBsQCgrcMSKueGj4cFd5nJTkXk8+2X4zrYkFiNZizpyqcTYefNCt13HEsIMHXf+JRx5xN4UuXeDHP3aducePbz0DkBlHFXbudDnJNWvcsm6dCwz5PvPYREW5HMGIEVVLcvJRU6mbVsoCRGuSmenasn70EZx2Gjz9tPtLriNV1/XixRddXXhRkZvu9MorXbAYXuMs4qY5Ki+HrCxXFLRxo6sn8AaF/T7z2PTqBUOHuiUxsWo9Ls6Kh9oqCxCtjaqrtP7Nb9xf/29/C3fdVe8xN4qL4Z133Ff++9+urDkpyTWgmjwZxo615ofNgaprzrx+vcsFrF8PGza4gJCZ6YKEV7duLsgnJVX1Nk5KqnVjONOGWIBorfLy3IBMzz/vBqF54gk3IVED5OS4lk/z58Pnn7ubTteu8MMfumAxaZJrrWICw1s3kJnpWgh5X71BwVs/AO55YPDgqmXIkKr16GjLEZjasQDR2n36qRs6fN06OP98Vz/RCNPQFRS40WMXLXLLnj1ue2Kiqys/4wy3DBliN6O6KC+HrVtdx7HNm49eMjNdjs5XXJyrGE5MrHpNTHR9CazOyDRU0AKEiDwPXAjkqOoxpdrixhV4BLgAOAhco6rLPPuuBv7bc+j9qvrSic7XZgMEuIbqjz4Kf/6zu7NPnw733Qf9+jXK11dWuha3ixfDF1/Al1+61lDgOuGNHVvVK3bECJehaYtBo7LS5QC2bXNDtG/f7n/dtziofXvXbHTAAPfqux4fby2GTDUFBa5sccOGqnLGsjLXlr0eghkgzgSKgJdrCBAXADfjAsSpwCOqeqqIdAfSgTRAgaXAaFXNr/4dvtp0gPDat8/lIB591L2/+WY3Jnj37o16mspK97v84gu3fPWVe1/pGbCzc+eqVjGJiVU3vv79W+akLKqu1/Du3ccu27ZVLTt2VPUi9mrXzuUC+vZ1r/37V/17DBzoBp6znIA5oqzMZTGzso5etmxx2U5vVh7cDychwVUyzZ9fr6eyoBYxiUg88G4NAeIp4BNVneN5vx4427uo6g3+jquJBQgfW7e6TnUvveTast56qwsWjRwofB086FrOrFjhetl629UfOFB1jIi7UQ4c6F6jo49eevVyOZJOndwSEdG4OZHycleEc+CAu+EXFBy95OdDbq5bcnKOfj3kZxy7sDBX1NO3r/8lLs7qA4wfFRWu/bE3J+C7bNlS1SsR3OBqffu67OSAAa6cccgQ9zpgQIOfuJpzgHgXeFBVP/e8/xC4AxcgOqjq/Z7tfwRKVPWvfr7jeuB6gH79+o3Ozs6ufkjbtmqVm5DonXdco/YbbnCtn5pouFdvyxvfClfvsnOnu/mWlNT8+ZCQqmARFeWm7w4Lc38zYWFV65WVbqmoOHopKXGBq7jYLdWf7v2JiKgKVr6vsbHHLt262dO/qUF+vrvhb9p0dC4gK8s9wJVVDTFPx47upu9dvOWL8fHuKSOAzQiPFyBafONFVX0aeBpcDiLIyWl+kpNhwQIXKB56yPWQe+wxV0fxu9+5Ji8BJFJ1Mz3jDP/HFBcf/aSel+ee8KsvxcUuB+C7VFS419BQV5QTEuLWvUuHDi4uVl+iolzGynfp3Nnd8K1zmKkV79OP9+afmXl0TiAv7+jjY2PdDf+UU1zv1Pj4qoDQTCe4CHaA2AH09Xkf59m2A5eL8N3+SZOlqjVKTnYdHf70JzeM+PPPu+Xii2HmTDj33KA9Cntv2ieYSM+Ypnf4sLvxb9zocgIbN1blArKzjy137NPH3fAvv7yq7fGgQe7H3QJHLQx2EdNk4FdUVVI/qqpjPJXUS4FRnkOX4Sqpj52+yofVQdTBnj0uN/HMM7B3r/sx33ADXHONjQ1u2o7KStfSoKYK4a1bq1pegMti+hb/VF9aYPYzmK2Y5uByAj2BPcDdQDiAqs7yNHN9DJiEa+b6c1VN93x2BvAHz1c9oKovnOh8FiDq4fBheOstePJJ1xypfXv46U/h+utdZ4dmmO01pk4OHz6644nv65Ytbr+vXr3czX7QILf49kYMYCOPYLGOcqZ2Vq6EWbPglVfc4EwJCW5wpiuvdMN4GtMclZa64p7MTHfDz86uKgLKznYTXPve5yIjXS5g0CDXCmjAgKocQP/+9R6ypqWyAGHq5sABN4Lf7Nnwn/+4LHZqKkyb5uamiIsLdgpNW+JtErp1a9VNf8uWquZw27YdXQwUHl7VLLR/f7d4cwQDB7pJri1nfIQFCFN/u3fDG2+4YPHtt25bWpqbN/uii1xvOPtjMw1RWeme8rdsqcoFeJfsbNf13LdfALh2x/66nyckuJmLQkODcy0tkAUI0zg2boR//Qv+7//gm29ctr1fv6pgMX68jQthjlVc7G70W7e6Zdu2o1+3bz+6HkDENftMSKjKAfTrV7Xet69rp2wahQUI0/h274aFC10HvMWLXY+09u1dZ4dzznFLWprL7pvWq7zc/Ra8g01t315VDORdqvcHCAmp6n7er597TUhwy4ABLXc8lhbKAoQJrIMH4eOP3SRGH33kxtkA95Q3fjyMGwdjxrgOQl26BDetpvYqK13PRe9AU9Wf/rdtc0VDvuX/4Cp5vU/7vnUA/fq55aSTbIKRZsQChGlae/e6Ici9AWPduqp9iYlw6qkuYIwe7VpHdeoUvLS2Rapu4KkdO1zlr/e1+rJr19HDzoIrQvQONNWvn2uwUH3p3t3qpVoQCxAmuPLzIT3d1Vt8+617zcmp2h8f73p6e6c+GzbMVTpa4Kg9VVfWv2ePW3JyqtZ93+/e7QLCwYPHfke3bq7op3fvquWkk44uCurRw27+rYwFCNO8qLqy6RUr3PCv3mXduqOfWL0tVXxbqfTpU3UT69y59d6sSktdYN23z5Xh5+W5nNnevUev+waCmkY97N7ddf6KiXHjAXn//Xz/LXv3bpFDQZiGswBhWobSUjfI2bp1R/d49bZ1r/5bjYysurlFR7unW+/Svbt77dLF1YX4LpGRjd8MUtW1xDl0yN2oDx1yT/RFRccuhYVHjzXuu+4NCEVFNZ+rfXs3uXSPHu6m7735V1+8w9C2a9e412palVY9mqtpRdq1qypmqs47XEJNZeZr1lQ9aVdvM1/TubxLeHjVeljY0bkS77qq+96ysqqhZL3rhw4dO1zDiYSEHD2MbJcu7mk+ObkqwPku3oDQs6erBG6tOSfTrFiAMC1D+/ZV4+Ecj3fqN2+wOHDg6Kd379jhhw65G3xpadVSVnb0GP3VcyzeCSi8k1J4lw4d3BIRUbXevv2xORfvpBbeXIzd5E0zZwHCtC4iVU/mAwYEOzXGtGg2F5Yxxhi/LEAYY4zxywKEMcYYvyxAGGOM8csChDHGGL8sQBhjjPHLAoQxxhi/LEAYY4zxq1WNxSQiuUD2CQ7rCextguQ0N3bdbYtdd9vSkOvur6rR/na0qgBRGyKSXtPAVK2ZXXfbYtfdtgTquq2IyRhjjF8WIIwxxvjVFgPE08FOQJDYdbctdt1tS0Cuu83VQRhjjKmdtpiDMMYYUwsWIIwxxvjVZgKEiEwSkfUisklE7gx2egJJRJ4XkRwRWe2zrbuILBaRjZ7XbsFMY2MTkb4i8rGIrBWRNSJyi2d7a7/uDiLyrYis8Fz3vZ7tCSLyjef3/oaItMqJqUUkVES+F5F3Pe/bynVnicgqEVkuIumebY3+W28TAUJEQoHHgfOBYcAVIjIsuKkKqBeBSdW23Ql8qKqDgQ8971uTcuC3qjoMOA34pef/uLVf92HgHFUdAaQCk0TkNOAh4B+qOgjIB64NYhoD6RYgw+d9W7lugAmqmurT/6HRf+ttIkAAY4BNqpqpqqXA68DFQU5TwKjqEmBftc0XAy951l8CLmnSRAWYqu5S1WWe9QO4m0YfWv91q6oWed6GexYFzgHe8mxvddcNICJxwGTgWc97oQ1c93E0+m+9rQSIPsA2n/fbPdvakhhV3eVZ3w3EBDMxgSQi8cBI4BvawHV7ilmWAznAYmAzsF9Vyz2HtNbf+8PA74BKz/setI3rBvcQ8IGILBWR6z3bGv23HtbQLzAtj6qqiLTK9s0iEgXMBW5V1UL3UOm01utW1QogVUS6AvOBxCAnKeBE5EIgR1WXisjZwU5PEIxT1R0i0gtYLCLrfHc21m+9reQgdgB9fd7Heba1JXtE5CQAz2tOkNPT6EQkHBccXlPVeZ7Nrf66vVR1P/AxMBboKiLeB8DW+Hs/A7hIRLJwRcbnAI/Q+q8bAFXd4XnNwT0UjCEAv/W2EiC+AwZ7Wji0A6YC7wQ5TU3tHeBqz/rVwIIgpqXRecqfnwMyVPXvPrta+3VHe3IOiEgEcB6u/uVj4HLPYa3uulX196oap6rxuL/nj1R1Gq38ugFEJFJEOnnXgYnAagLwW28zPalF5AJcmWUo8LyqPhDkJAWMiMwBzsYNAbwHuBt4G3gT6IcbEv0nqlq9IrvFEpFxwGfAKqrKpP+Aq4dozdedgquQDMU98L2pqveJyADck3V34HvgKlU9HLyUBo6niOk2Vb2wLVy35xrne96GAbNV9QER6UEj/9bbTIAwxhhTN22liMkYY0wdWYAwxhjjlwUIY4wxflmAMMYY45cFCGOMMX5ZgDDmBESkwjNqpndptAH/RCTed9RdY5oTG2rDmBMrUdXUYCfCmKZmOQhj6skzJv//eMbl/1ZEBnm2x4vIRyKyUkQ+FJF+nu0xIjLfM3fDChE53fNVoSLyjGc+hw88PaIRkV975rdYKSKvB+kyTRtmAcKYE4uoVsT0U599BaqaDDyG66kP8E/gJVVNAV4DHvVsfxT41DN3wyhgjWf7YOBxVU0C9gNTPNvvBEZ6vufGQF2cMTWxntTGnICIFKlqlJ/tWbjJejI9AwXuVtUeIrIXOElVyzzbd6lqTxHJBeJ8h37wDE2+2DPJCyJyBxCuqveLyPtAEW6YlLd95n0wpklYDsKYhtEa1uvCd6ygCqrqBifjZkIcBXznM0qpMU3CAoQxDfNTn9evPOtf4kYYBZiGG0QQ3DSQM+HIJD9davpSEQkB+qrqx8AdQBfgmFyMMYFkTyTGnFiEZ8Y2r/dV1dvUtZuIrMTlAq7wbLsZeEFEbgdygZ97tt8CPC0i1+JyCjOBXfgXCrzqCSICPOqZ78GYJmN1EMbUk6cOIk1V9wY7LcYEghUxGWOM8ctyEMYYY/yyHIQxxhi/LEAYY4zxywKEMcYYvyxAGGOM8csChDHGGL/+P8rm7j8i5UglAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xft_7LtUvncU",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 332
        },
        "outputId": "f9cf1cbf-213a-4c44-9287-d68f941307ca"
      },
      "source": [
        "# Plot test accuracy curves\n",
        "\n",
        "epochs = range(1, len(hista['test_acc']) + 1)\n",
        "\n",
        "plt.plot(epochs, hista['test_acc'], 'r-', label='Model A');\n",
        "plt.plot(epochs, histb['test_acc'], 'b-', label='Model B');\n",
        "plt.plot(epochs, histc['test_acc'], 'g-', label='Model C');\n",
        "\n",
        "plt.title('Test Accuracy');\n",
        "plt.xlabel('Epochs');\n",
        "plt.ylabel('Accuracy');\n",
        "plt.legend();\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1bd80ed0>]"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1af935d0>]"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1af93ad0>]"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Test Accuracy')"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 0, 'Epochs')"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Accuracy')"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f1f1bd9ae90>"
            ]
          },
          "metadata": {},
          "execution_count": 16
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUZfbA8e9JKAESek9oUgSRCArYFUEUEUXFAiuKBcGCbde267p2l9WfrmV1FbGLgIAilqWIIBakKVIDBAiS0EIPJf38/nhvyCRMICEzmSE5n+eZJzN37tz73hDumbedV1QVY4wxprCIUBfAGGNMeLIAYYwxxi8LEMYYY/yyAGGMMcYvCxDGGGP8sgBhjDHGLwsQxhhj/LIAYY57IrLP55ErIgd9Xl9/DMebLSJDi7FftHeO/x1byY0Jb5VCXQBjSktVo/Oei0gSMFRVvy2DUw8AMoDeItJYVbeUwTkBEJFKqppdVuczFZPVIEy5JSIRIvKIiKwVkR0i8qmI1PXeixKRj73tu0VkgYg0EpFngXOB/3i1g/8c4RRDgDeBJcDgQuc+R0R+9o69UURu8rZXE5EXRWSDiOwRkR+9bT1EJLnQMZJE5ELv+RMiMtEr817gJhHpLiJzvXNsFpH/iEgVn893FJEZIrJTRLaKyN9EpLGIHBCRej77nSoiqSJSuTS/b1P+WIAw5dndwBXA+UBTYBfwuvfeEKAW0AyoB9wOHFTVR4EfgBGqGq2qI/wdWERaAD2AMd7jxkLv/Q94DWgAdAYWe2//H3AacBZQF3gIyC3m9fQHJgK1vXPmAPcD9YEzgV7AnV4ZYoBvganetbcBZnq1nNnAtT7HvQEYp6pZxSyHqSAsQJjy7HbgUVVNVtUM4AngahGpBGThAkMbVc1R1UWqurcEx74BWKKqK4BxQEcR6eK99yfgW1Udq6pZqrpDVReLSARwC3CvqqZ45/3ZK1txzFXVyaqaq6oHvTL/oqrZqpoEvIULhgD9gC2q+qKqpqtqmqrO8977AK/GIyKRwCDgoxJcu6kgLECY8qwF8LnXBLMbWIn71t0Id0OcBowTkU0i8nwJm1huxH2LR1VTgO9xtRJwtZK1fj5TH4gq4r3i2Oj7QkTaichXIrLFa3Z6zjvHkcoA8AVwkoi0AnoDe1R1/jGWyZRjFiBMebYRuERVa/s8orxv71mq+qSqnoRr7ulHfjPREVMci8hZQFvgr97NeQtwOvAnr3ayEWjt56PbgfQi3tsPVPc5RySuecpX4XL9F0gA2qpqTeBvgPhc+wn+yq+q6cCnuFrEDVjtwRTBAoQpz94EnvX6BBCRBiLS33t+gYh08m7Ee3FNTnl9AVsp4ubqGQLMAE7C9S90Bk4GqgGX4GoWF4rItSJSSUTqiUhnVc0F3gVeEpGmIhIpImeKSFVgNRAlIpd6NZm/A1WPcn0xXtn3iUh74A6f974CmojIfSJSVURiROR0n/c/BG4CLscChCmCBQhTnr0CTAGmi0ga8Avumz5AY1yH715c09P35N8oX8H1VewSkVd9DygiUbgO3tdUdYvPY733+SGq+gfQF/gLsBPXQX2Kd4gHgKXAAu+9fwERqroH18E8GkjB1SgKjGry4wFcf0ca8DYwPu8NVU3DNR9dBmwB1gAX+Lz/Ey4g/qqqG45yHlNBiS0YZEzFJCLfAZ+o6uhQl8WEJwsQxlRAItIN10zWzKttGHMYa2IypoIRkQ9wcyTus+BgjiSoAUJE+ojIKhFJFJFH/LzfXERmichvIrJERPr6vPdX73OrROTiYJbTmIpEVYeoai1VfT/UZTHhLWhNTN7okNW4jrJkXKfcIG9iUd4+o4DfVPW/InIS8I2qtvSejwW642aBfgu0U9WcoBTWGGPMYYKZrK87kKiq6wBEZBwuVcAKn30UqOk9rwVs8p73x039zwDWi0iid7y5RZ2sfv362rJly4BegDHGlHeLFi3arqqF59wAwQ0QsRSc+ZlM/hDDPE/ghiDeDdQALvT57C+FPhtb+AQiMgwYBtC8eXMWLlwYkIIbY0xFISJFDnMOdSf1IOB9VY3DjRv/yMtXUyyqOkpVu6pq1wYN/AZAY4wxxyiYNYgUXD6YPHHeNl+3An0AVHWuNwmpfjE/a4wxJoiCWYNYALQVkVZejvqBuFmtvv7ApShGRDrgEpmlevsN9FIEtMLlvbFkYsYYU4aCVoNQ1WwRGYHLmBkJvKuqy0XkKWChqk7BpSJ4W0Tux3VY36RuWNVyEfkU16GdDdxlI5iMMaZslZuZ1F27dlXrpDbGmJIRkUWq2tXfe6HupDbGGBOmLEAYY4zxK5ijmIwxwbBlCyxYAEuXQqVKEBPjHjVr5v9s0ACaNIHKJVkkz5iCLECYiiUrC9auhaQk2LQJUlIKPgBOPBHat89/tGsH1aqFprwHDsC8eTB/vgsK8+fDxo1H/xyACDRsCE2bQmxs/s+8R97runXdviWVmwvbtxf8/W3e7AJUhw7ud9e8OURGlvzYJixYgDDlT2amu1ElJ8OaNZCQkP9YuxayswvuX79+/g1T1d2Qx493z8HdPFu0yA8YeTe/9u3dN/VjubkWJSvLBYKZM91j7lx3PQCtW8PZZ0O3btC9O3Tp4s69dy+kpeX/3LMHtm1zN+y8ILhxI/zyi7uhFxYVBY0aFayF5D2vVg327y94/LxzbN3qynskUVEuwLZv78rvG5iaNoXGjSEiwv1b+f47JSTAunWQU8TgxXr1Cga7vOOdcII7j9WcAsJGMZngO3Cg4M0q7/muXYff3PbudTemrl3dTbB7dzjlFKjqs/pmTg5s2JB/I1m92t1g8o67bVvB81euDG3b5t/cTzzR3UiaNnWPqn5W9jx4sGBwWbkSVq1yzw8ezN+vVi33DbzwzTUmpuQ3qT/+gDlzYN8+d+Pv3Bl69YILLoDTT3c3xdLKyHDBs/C/x9at+Td/33+PgwehRo3Dm7FiYtzNvfANunFjFzzyfld5v7uVK931FQ7OERFQpQqkpxf8nXbo4G70/v5tCtdcCv97V6rkPutbC4yN9R/IIyIgOrpg81yNGm57BXGkUUwWIEzwzJ4NN97ov0kkOtrd8PzdWLdvd00pW7e6fStXhvh411yRmOgCQkZG/rHq14dmzQo2oeQ9b9MGWrVyN41AyM1115N381uzxt0Qfb9d5z0v6ttvUerWdcGgZ0/3MxABIZzk5rqbeV5gyvu5f39+LaN9e9csVpJaWV6NMSXF1RB9ayFr1hy9luNPjRpFN43Vrev/b61VK1f+OnVKfr4QsgBhyt7bb8Odd7ob9I03Ht4cEBNz5M+rulpBXrv7ggXuBtC2bcEmnvbt3X9YY/zJzob16/O/bPh7f9++w4P7vn0uoBWm6r7A+Na+9u8vuE/DhgX/Pps1K3mNxF+tLTralc23zyevHI0awVNPlewcniMFCOuDMIGVnQ0PPggvvwx9+sC4ca7JoKRE3H+sZs3gqqsCX05TMVSq5L5UtG0bnOOr5t+0163Lb1JLSICJE2HnzuCc11eDBnD++UE5tAUIEzh79sDAgTB1Ktx3H7zwQuCadowJRyLuG37eyK1LLy34fl5toyRyc12/XeFaTVqaq0X4Nm01aeK/nyZA7H+vCYy1a+Gyy1yb71tvwbBhoS6RMaFXv757HKcqTle9CY70dHjzTTfKZutWmD7dgoMx5YQFCHNs9uyBkSOhZUu44w7Xxjtvnht9Y4wpF6yJyZTMli3wyivwxhuubfTii+GRR1wnWSAnjFVQu3cfPl9syxY3bcN3YEzbtoGf3O07vSBvcMz27VC9+uGZPKKjXeWx8DSWwlNafLdVquR/dGjLlu5hfz7hxwKEKWjdOnj0UVi82P/769e7ceVXX+0CQ5cuZVu+41RODvzvf641bu1a//vs2lVwNGbe/L7Gjd2E6nHjCk7ubtmyYNDIG/1bv37RN9u80cOFg1BioptKcCxTBgqrVKlgMImJcVM6srJg+XLXCpmWVvAzdeu6CeJ5k8S7dXPXbULLAoRx0tLguefgpZfc//BLLvE/Uejii+Guu4I3bLCc2b4d3nnHBYakJPet+eyz/d/Aa9QoeLMvPL/vwAH/k7tnzy44ubtuXf8ji1UhNbXgsP28ScvnnQdxcYena6pf3x27cM1g3z6XRcPfPMeoqKPXBtLS8mspiYn5013++c/8+YWxsdCx4+FBsFEjq22UFZsoV9Hl5sIHH8Df/ubaMm64wf0vjY0NdcmOG6puYrdvc0pqKnz8sUvplJEBPXq4uNq/f+DTBBWe3J2QcPjcrTx16xa82ZZ00nKwHTgAv/3mgsWiRf6vp2ZNlxqrcP5B3+cNGhxbtoy8rDDbtrnRo77Br0aN8PpdBYrNpDb+/fAD3H+/+5945pluclv37qEuVamouhv0pk1FN5mouvZzf+3lvhk8fGVnF0xVVLiN3d95oqPdJPI773TfhM2xUXU3bd8AuHFjfg1ky5b8prc8lSq5KQJ5gaOohLUZGe44ecfas6fochRO2+Qvt+GRnvtui44On3RPNpPa5FN1E9lGjnSJ4eLiYMwYGDTouPt6pArff+8qQBs2FJ35oDgiI91/4KpVi87p5vsfPC/56ZFuBt26uZ+mdETcn2lcHFx44eHvZ2e7IOHbue6bCWPlStf570/lyi6QtG/vUmDl1UAaNnQpnvx1vhd+vmVLwe2F8xEWJTra/99QTEzRaaCiovwHnKZN4ZxzinfekrAAUVFkZ8OECfCvf8Hvv7v/bS+/DLfd5oapHEdyc2HKFBfj5s3LbzY55RTo27dgNumiJpkWbj8vTru5CU+VKuUHkFDzbW480givop5v2OB+FpUGKm/kmG+fE7hpSL/8EvjrsQBR3uXkwOjR8PzzboRShw7w/vuuxlClSqhLdxhVV2R/GToyM+GTT1yMS0hwnbj//S8MGRK69XyM8SXivmxERbl+kGDJyzGYF1iC9eXGAkR5lpwMgwe7dpju3eHFF+Hyy8On8bOQrVvhmmtc14i/qvS6de6S4uNdoLjmGkv1ZCqmSpWgdm33COp5gnt4EzJTpsDNN7v67nvvua/ZYdyG8vvvLpXT9u3w0EOuil24+h0fD6NGuSSxYXwpxpQbFiDKm/R0d4d97TU3iW3cOLcYSxibPNlVdGrXhh9/hFNPDXWJjDFguZjKl1Wr4IwzXHC49143/TaMg4Oq62i+6io46SQ3WcqCgzHhw2oQ5cXkyXD99W5E0pdfQr9+oS7REaWnu6SvH30E113nWsGso9mY8GIBojz48EO45Rbo2hU++8yN7wwTecsCF14hMSHBDel78kl47DHrUzD+pWWksWrHKhK2Jxx6JO5MpH71+rSv377AIzYmFjnKH1JObg77MvexN2MvaZlpZOVk0Ti6MQ1qNCBCim5QUVV2HNzB5rTNREgEMVVjqFm1JjFVYoiMKDhpITs3m7SMtEPn2Juxl7SMNNIy0w793Juxl8ycTBrVaETTmKbE1owlNiaWRtGNqBQRPrfl8CmJOTb/+Q/cfTf06uVqEdHRoS4R4FJN/PnPLt1EnogIN8EsNhY6d3bTMK64InRlNIGlqiRsT2DNzjWk7E0hJS2FTWmbSElLIWVvCmmZaUc/iI+M7Ay27s/PXhgpkbSu25o2dduw48AOPlryEXsz9h56v0blGtSrXs/vsTJzMknLSGN/lv9ZlJUjKtMkpom7WcfE0qB6A3am7yxwHZk5mX4/W61SNWpWrYmipGWkcTD7oN/9iiNCImhUoxG1o2oXCEIxVWOIqRJDlUj/Q9Nb1GrBvWfce8znLUpQA4SI9AFeASKB0ao6stD7/wbyFhCoDjRU1dreeznAUu+9P1T18mCW9bij6pLr/f3vLsHPuHFubGgYFOu999yy1GlpLsVT//4uKDRqZMNSjydzN85lcsJk4mrGHfqWHlczrsC39A27NzBz/Uxmrp/Jd+u/Y8u+LYfei5AIGkc3JjYmlrb12lKraq2jfsP3VUkqcUKdEw6du3Xd1gVukKrK1v1bD9UsVqauZG/mXr/HqhxR+dCN1vemWymiElv2bSkQCJanLmfb/m3Uq1aP2JqxnN3s7EOBo0lME4ACtYK854A7tp8bu7/tlSIqsW3/NlL2Fgykm/dtZk/GnkPH3bJvy6Hn2bn+p2l3bdo1KAEiaLmYRCQSWA30BpKBBcAgVV1RxP53A11U9Rbv9T5VLfbX4QqVi0kVHn7Yrfl8ww3w7rthcedNSIDhw10Gj3POcSuPnnRSqEtlSkJVmZo4lZE/jWTOhjlESAS5mj+tt0blGpxY/0Ra1m7J4i2LWbdrHQANazSkV6te9GzVk1ManUJszVga1mgYVs0lxr9Q5WLqDiSq6jqvEOOA/oDfAAEMAh4PYnnKh5wcl/1t1CiXHvTVV0M68U3VrW/w4YduRFJ0tJu4ffPNYTsfz/iRnZvNhOUTGPnTSJZsXUJczThevvhlbj31VvZl7ivQB5CwPYGlW5fSsUFH7ul+D71O6EXHBh1LVDswx4dgBohYYKPP62TgdH87ikgLoBXwnc/mKBFZCGQDI1V1sp/PDQOGATRv3jxAxQ5jqvnB4W9/g2eeKfPe3c2b83P3L1jgHrt2ufeuv94tJ9GwYZkWyZRQ4aaZhO0JfLn6S9btWkeH+h14v//7DOo06FBzTnSVaBpHN6ZHyx6hLbgpc+FS/xsITFTVHJ9tLVQ1RUROAL4TkaWqWmAtLlUdBYwC18RUdsUNkVGj3OPhh+HZZ8v01D/95Lo7Zs92ryMj4eST3cJy3brBWWdZSutwtePADmYnzWbm+pn8uvlXErYnsCcjP691jco16Nq0Ky9e9CKXn3j5EUfzmIolmAEiBWjm8zrO2+bPQOAu3w2qmuL9XCcis4EuQBGLNVYAP/3kRitdckmZBofffnOB4ZtvXCfzc8+55ac7dz7uksBWCDm5OezJ2MP8lPnMXOc6jxdvWYyiRFeJplvTblzf6fpDHb8dGnQo1vBQUzEFM0AsANqKSCtcYBgI/KnwTiLSHqgDzPXZVgc4oKoZIlIfOBt4PohlDW+bNrmv6s2bu7UbikoWH0AJCfCPf7gM4XXquP6FESPcqlqm7OTk5hwa3eL7c1PaJjbv28zu9N0FRtMcyDpw6LNVIqtwVrOzeLLHk/Q6oRfdmnajcmSAl7Mz5VrQAoSqZovICGAabpjru6q6XESeAhaq6hRv14HAOC04nKoD8JaI5OLSgYwsavRTuZeRAQMGuDGjM2a4u3UQrVjh0ml//LGb2fz3v8Nf/hL8rJEVlaqyP2v/oeGMh/oFdrifq3esJj07vcBnqkRWoWlMU5rGNKVJdBPa1WuXP5TSG0LZqWEnzm5+NtUrWzXPHDtbcjTcDR/u+h0mTnSBIkh++cXVEr74wjUdDR8OjzxiHc6Bsm3/Nr5c9SVfrPqCNTvXHPrWvy9zX4FhpODmD7Sq3epQM1C7eu2IqxlHbEwssTVjqVetnjUJmYCxJUePV3md0n/9a1CCgypMm+YCw/ffu8rJ44+7pqT69QN+ugpn7c61TE6YzORVk/npj59QlJa1W9K1aVdqVql52CSqvPQRbeq2IapS6Cc9GmMBIlzNnevu1H36wNNPB+ywqrBkCXz+uauULF/uZjm/9JJbfTRMMnWElV0Hd5GwPeGw3Dp5r9My0tibWXDbzoM7SdqdBEDnxp15/PzHuaL9FcQ3irdv/+a4YQEiHG3e7GoMzZu7pdNK2Smdne0GQU2e7B5JSW76xNlnwzvvuLUYwnD10ZA4mHWQ37b8xoKUBSzYtID5KfNZs3NNkftHVYoq0PZfs2pNmkQ34cR6J3Lv6ffS/8T+tKrTqgyvwJjAsQARbjIzXXDYu9e1/5SyUzohweXx27QJqlaF3r1dx/Nll1n/QnZuNsu3LT8UCBZsWsDSrUvJ8abjNI1pSvfY7tzU+SbiG8VTt1rdAk1CMVVibFSQKdcsQISbvIV+Pv0UOnUq1aEyM+FPf3IDoSZMcK1VFb0JKT07nX/P/Tdfr/maXzf/eijzZu2o2nRr2o2Hz36Y7rHd6RbbjaYx4ZM23ZhQsAARTkaPhjffdDOlr7mm1Id77DE30W3yZJdRtaKbuW4md3x9B2t2ruGMuDMYdtowFwyadqNN3TbWN2BMIRYgwsUvv7jkexddFJCZ0rNmuWSvw4dbcEjdn8oDMx7gw98/pHWd1sy4YQYXnnBhqItlTNizABEOtmxx/Q5xcTB2bKk7pXfudFnA27aFF18MUBmPQ6rK+4vf54EZD5CWkcaj5z7Ko+c+SrXKtrapMcVhASLUMjNdGo3du13fQ926pTqcKtx+O2zd6iol5T01RuLORGatn8XOgzvzh6B6Q0/X7VrHb1t+4+xmZ/NWv7fo2NCyCRpTEhYgQu3++90Y1HHjID6+1If78EPXIT1yJJx2WgDKF2ZUlV83/3poAtqybcsOvRchEQVGGdWqWou3+r3F0FOHWoZSY46BBYhQ+uYbeOMNl+zouutKfbi1a93cuvPPhwceCED5wsj6Xet5ae5LTF41meS9yURKJOe1OI9X+rxC37Z9aRrTlGqVqllHszEBZLmYQmXfPreAQo0abqhR1aqlOlxWFpx7Lqxa5WZKN2t29M8cL75P+p6rPr2KA1kH6NOmD1eceAX92vUrcoF6Y0zxWS6mcPT3v8PGjfDjj6UODgD//jfMm+daqspTcHjn13e4/evbaVO3DV8O+pI2dduEukjGVBjWMBsK8+a5taTvvNMtxVZKmze7dE2XXRaQlqqwkJObw5+n/ZmhXw6lV6tezL11rgUHY8qY1SDKWmYmDB0KTZu65dkC4K9/dYd96aWAHC7k9mbsZdCkQXyz5hvu6X4PL178IpUi7E/VmLJm/+vK2gsvwLJlbuGFmjVLfbh58+CDD9zk6zbl4Av2ul3ruHzs5azasYo3L32T4V2Hh7pIxlRYFiDK0qpV8NRTcO21cPnlpT5cbq5L3dS4MTz6aADKF0Kqyru/vcv90+6nUkQlpg2eRs9WPUNdLGMqNAsQZSU3F4YNc8u1vfJKQA758ceuBvH++xATE5BDhsSmtE3c9uVtfLPmG3q07MF7/d+jZe2WoS6WMRWeBYiyMno0zJnjFmBo3LjUh0tLc81K3bu7tBrHI1Vl7LKxjPhmBOnZ6bzS5xVGdB9hk9qMCRMWIMrCli3w0ENwwQVw880BOeRzz7nDTp4MEcfh/TR1fyp3fH0Hk1ZO4oy4M/jgig9oV69dqItljPFhAaIsPPooHDgAb73llnIrpcREN2Lpxhvh9NMDUL4yNmfDHK6dcC270ncxstdIHjjrASIjSpeg0BgTeBYggm3xYnjvPfjzn1161QD4y1/cEqEjRwbkcGVGVXn5l5d5cMaDtK7bmuk3TCe+UenzTxljgsMCRDCpurt53bpu5nQATJ0KU6a44NCkSUAOWSb2Ze5j6JShjF8+nivaX8H7/d+nVlStUBfLGHMEFiCC6auv4Lvv4LXXoHbtUh/u++/dCNkTT4T77gtA+crI6h2ruWr8VazcvpKRvUby0NkPWVI9Y44DFiCCJTPTpVRt394t61ZKX3/tlo1o2RJmzAhI+qYyMTlhMjd+fiNVK1Vl+uDp9DqhV6iLZIwpJgsQwfLmm7B6tatFVK5cqkONHw+DB7vlIqZOhQYNAlTGIErdn8qfp/+Zj5d8TLem3Zh47USa12oe6mIZY0rgOBwgeRzYuROeeAIuvBD69i3Vod5+GwYNgjPPdK1V4R4c8mZEt3+9PeOXjeex8x7jh5t/sOBgzHHIahDB8PTTsGePG4tairb2//s/ePBBuOQSmDjRTcIOZytTV3L717czZ8Mczm1+Lm/1e4sODTqEuljGmGNkASLQVq+G//wHbr0VOnU6pkOowuOPuzhz3XVuGdEqVQJczgDKzMnkmTnPMPLHkURXiWb0ZaO5ucvNNiPamONcUP8Hi0gfEVklIoki8oif9/8tIou9x2oR2e3z3hARWeM9hgSznAH18MMQFeWS8h0DVTev7umnXYwZMya8g0NGdgZXf3o1T895mms7XkvCiARuPfVWCw7GlANBq0GISCTwOtAbSAYWiMgUVV2Rt4+q3u+z/91AF+95XeBxoCugwCLvs7uCVd6A+P57l/vi2WePKd+Sqlvb4V//cgOf3ngjvNNoZGRncPWEq/lq9Ve83vd17ux2Z6iLZIwJoGDefroDiaq6TlUzgXFA/yPsPwgY6z2/GJihqju9oDAD6BPEspaeqsu3FBcH999/9P2L+Pi//gV33BH+wSE9O52rPr2Kr1Z/xX8v/a8FB2PKoWD2QcQCG31eJwN+MweJSAugFfDdET4b6+dzw4BhAM2bh3iUzOefw/z5LltrtWol+qiqmzLx0ktw111uXl04zyNLz07nyvFXMjVxKm/1e4thpw0LdZGMMUEQLt9RBwITVTWnJB9S1VGq2lVVuzYI5fjP7GzXcdChg8ugVwKqrsLx0ktwzz3hHxwOZh2k/7j+TEucxtuXvW3BwZhyLJg1iBSgmc/rOG+bPwOBuwp9tkehz84OYNkC68MPISEBPvsMKhX/V6rqVoR77TX389//Pj6Cw7frvmX05aO5pcstoS6SMSaIglmDWAC0FZFWIlIFFwSmFN5JRNoDdYC5PpunAReJSB0RqQNc5G0LPwcPujGp3bvDFVcU+2M5OTB0qAsO998f/sHhQNYBLht7Gd+u+5Z3+79rwcGYCiBoNQhVzRaREbgbeyTwrqouF5GngIWqmhcsBgLjVFV9PrtTRJ7GBRmAp1R1Z7DKWipvvAHJya4WUcw7fGamS50xYQI89hg8+WR4B4f9mfu5bOxlzE6azftXvM+Np5SsGc0Yc3wSn/vyca1r1666cOHCsj3pnj1wwgnQrZtLklQMBw7AgAFu9//7P5cNPJztz9xPv7H9mLNhDh9c8QGD4weHukjGmAASkUWq2tXfezaTujReeMHlXXruuWLtvmcP9OsHP/3kciwNHRrk8pXSvsx9XPrJpfz4x498dOVH/KnTn0JdJGNMGbIAcay2bHEdB9ddB6eeetTdU1OhTx9YuhTGjXPrOoSztIw0+n7Sl583/syYq8Yw8OSBoS6SMaaMWc64/VcAACAASURBVIA4Vs884zoTnnnmqLumpsJ558GGDfDFFy75Xjjbm7GXS8ZcwrzkeYwdMJZrO4Z5NDPGBIUFiGOxbh289ZZrI2rT5oi7qrqcSuvWuYV+zjuvjMp4jDbs3sCATweweMtixl09jqtPujrURTLGhIgFiGPx3HNuEaDHHjvqrqNGwZdfutaocA8OM9bOYNCkQWTlZjF54GT6tesX6iIZY0IoXGZSHz8yM2HSJLjmGmja9Ii7JiS4OQ4XXeRmSYerXM3luR+e4+KPL6ZxdGMW3LbAgoMxxmoQJTZrFuze7RaIPoLMTLj+erfIz/vvh2/ivT3pexgyeQhfrPqCgScPZPRlo6lRpUaoi2WMCQNHDRAichnwtarmlkF5wt+kSRAdDb17H3G3f/wDfv3V5fBr0qSMylZCy7Yt48rxV5K0O4mXL36Ze06/BwnnGXvGmDJVnO+11wFrROR5Ly1GxZWd7e74/fq5RYGKMGsWPP88DBtWouwbZSYzJ5Nn5zxL11Fd2Ze5j+9u/I57z7jXgoMxpoCj1iBUdbCI1MSt1/C+iCjwHjBWVdOCXcCw8sMPsH27mwpdhF27XELXtm1dhtZw8+MfPzL8q+GsSF3B1Sddzat9XqVJTJhWcYwxIVWslnFV3QtMxC360wS4EvjVWwWu4pg0ya31UMREBlW3EtyWLfDJJ1AjjJrydx3cxfAvh3Pue+eyL3MfXw76kgnXTLDgYIwpUnH6IC4HbgbaAB8C3VV1m4hUB1YArwW3iGEiN9el877kkiLv/B995BLw/fOfcNppZVy+IuxJ38OUVVN4YMYD7Diwg7+c+Ree6PEE0VWiQ100Y0yYK84opgHAv1V1ju9GVT0gIrcGp1hh6JdfYPPmIpuX/vgD7r4bzj0XHnywjMvmSc9O5/ctv7Ng0wLmp8xnwaYFJGxPAKBb025MvX4qXZp0CU3hjDHHneIEiCeAzXkvRKQa0EhVk1R1ZrAKFnYmToQqVVwHdSG5uXDLLW6Nh/ffh8jIsi2aqvL8T8/zj9n/IDMnE4DG0Y3p1rQb13e6ntNjT6dnq55ERpRxwYwxx7XiBIgJwFk+r3O8bd2CUqJwpOqal3r3hpo1D3v7jTdg5kw3a/qEE8q2aOnZ6Qz7chgfLfmIK9tfyeD4wXSP7U5sTKyNSjLGlEpxAkQlVc3Me6Gqmd4KcRXHokUu097jjx/21urV8NBDrmuirNN3b923lSvHX8nc5Lk8fcHTPHruoxYUjDEBU5wAkSoil+etACci/YHtwS1WmJk0ya013b9/gc3Z2W5Ia1QUjB5dtqvCLdm6hMvGXkbq/lQmXDPBkuoZYwKuOAHidmCMiPwHEGAjUHHWnFR1/Q8XXAB16xZ46/nnYd48GDv2qGmZAmrKqin8adKfqB1Vmx9u/oHTmobJkCljTLlSnIlya4EzRCTae70v6KUKJ0uXQmIiPPBAgc2LF8MTT7j1ggaW0Vo66dnpPDvnWZ794VlOa3oaXwz8gqYxZRiZjDEVSrGS9YnIpUBHICqvjVtVnwpiucLHpEmu7cgnZ0ZGBtxwA9SrB6+/XjbFmLV+Frd/fTurd6xmyClDeOPSN6heuXrZnNwYUyEVZ6Lcm0B14AJgNHA1MD/I5Qofkya5yQ2NGh3aNHIkLFsGX3/tgkQwbT+wnQemP8AHv39A6zqtmT54Or1bHzlRoDHGBEJxUm2cpao3ArtU9UngTKBdcIsVJlatguXLD0vtPWaMW+Ohb9/gnVpVeX/x+7T/T3vGLB3D3875G0vvWGrBwRhTZorTxJTu/TwgIk2BHbh8TOXfpEnu51VXHdq0fj2sWQN33RW806Znp3PNhGv4avVXnNXsLEb1G0XHhh2Dd0JjjPGjOAHiSxGpDbwA/Aoo8HZQSxUuJk2CM86A2NhDm6ZNcz/79AnOKbNysrh2wrV8tforXr74Ze4+/W4iJExXGzLGlGtHDBAiEgHMVNXdwCQR+QqIUtU9ZVK6UNq61a348+yzBTZPnQotWkC7IDSy5eTmMPjzwXy5+kte7/s6d3a7M/AnMcaYYjriV1NvFbnXfV5nVIjgAPDtt+7nxRcf2pSVBd995zYFelJcruZy65Rb+XT5p7zQ+wULDsaYkCtO28VMERkgFS2Hw/TpbohSl/zsp3PnQlpagZgREKrKXV/fxQe/f8CTPZ7kgbMeOPqHjDEmyIoTIIbjkvNliMheEUkTkb1BLldoqboA0bs3ROT/iqZNc5lae/UK5KmUB2c8yJuL3uShsx7isfMeC9zBjTGmFI4aIFQ1RlUjVLWKqtb0Xh+e0tQPEekjIqtEJFFEHilin2tFZIWILBeRT3y254jIYu8xpfiXFADLlrll4S66qMDmadNcn3WtWoE71ZPfP8mLc19kRLcRjLxwpCXbM8aEjeJMlDvP3/bCCwj5+Vwkrv+iN5AMLBCRKaq6wmeftsBfgbNVdZeINPQ5xEFV7VyMawi86dPdz975cw5SU12f9ZNPBu40XyR8wZPfP8nNnW/mlUteseBgjAkrxRnm6rs+WhTQHVgE9DzK57oDiaq6DkBExgH9ccuU5rkNeF1VdwGo6rZilju4pk2Dk06CuLhDm2bMcC1Pgep/2Jy2mVun3EqXxl14s9+bNpTVGBN2itPEdJnPozdwMrCrGMeOxWV+zZPsbfPVDmgnIj+JyC8i4ju7IEpEFnrbr8APERnm7bMwNTW1GEUqhoMHYc4cv81LdesGZq3pXM3lpi9u4kDWAcZcNYYqkRVreQ1jzPGhWMn6CkkGOgTw/G2BHkAcMEdEOnnzLlqoaoqInAB8JyJLvcyyh6jqKGAUQNeuXTUgJfrhB5eNzydA5Oa6ANG7d2CWE31t3mtMXzud/176Xzo0CNSv0hhjAqs4fRCv4WZPg6txdMbNqD6aFKCZz+s4b5uvZGCeqmYB60VkNS5gLFDVFABVXScis4EuwFqCbfp0t/b0efldL0uWuHlzgZg9vWTrEh769iEua3cZw08bXvoDGmNMkBSn4Xshrs9hETAXeFhVBxfjcwuAtiLSyluidCBQeDTSZFztARGpj2tyWicidUSkqs/2synYdxE806fDOedAjRqHNuWl1yjU6lRiB7MOcv1n11Mnqg7vXP6OdUobY8JacZqYJgLpqpoDbnSSiFRX1QNH+pCqZovICGAaEAm8q6rLReQpYKG3hOk04CIRWQHkAA+q6g4ROQt4S0RycUFspO/op6DZvNktEDRyZIHN06ZBp06lXzXukW8fYdm2Zfzv+v/RoEaD0h3MGGOCrDgBYiZwIZC3klw1YDpw1tE+qKrfAN8U2vYPn+cK/Nl7+O7zM9CpGGULrBkz3E+foUr79sGPP8K995bu0FMTp/Lq/Fe5p/s99GkTpEx/xhgTQMVpYoryXWbUe14+lzKbPh0aNoT4+EObZs92OZhKM7x12/5t3DT5Jk5ueDL/6v2v0pfTGGPKQHECxH4ROTXvhYicBhwMXpFCJDfX1SD8pNeoVs11SxzTYTWXIZOHsDt9N2OuGkNUpagAFdgYY4KrOE1M9wETRGQTIEBj4LqglioUliyBbdv8zn/o0QOijvG+/uLPLzI1cSpv9H2D+EbxR/+AMcaEiaMGCFVdICLtgRO9Tau8YanlS95QJZ/0GqVdPW5e8jz+9t3fGNBhALd3vT0AhTTGmLJz1CYmEbkLqKGqy1R1GRAtIuVvsYLp091QpSb5q6nmxYxj6X/Ynb6bgZMGEhsTy+jLR9uQVmPMcac4fRC3eTObAfDyJt0WvCKFwP79bqhSoealqVOheXM48cQiPlcEVeW2L28jeW8y464eR+2o2gEsrDHGlI3iBIhI38WCvCyt5St50Jw5kJl5WICYOxd69iz56nGjFo1i4oqJPNvzWc6IOyOABTXGmLJTnE7qqcB4EXnLez0c+F/wihQC06dD1apw7rmHNu3d6/qs27cv2aGWbl3KfdPu4+LWF9vKcMaY41pxAsTDwDAgr5d1CW4kU/kxfbrLvVSt2qFNa72sT23aFP8w+zP3c+3Ea6kdVZsPr/zQUngbY45rxUn3nQvMA5Jwazz0BFYGt1hlKDkZVqw4rHkpMdH9LEmAeOHnF1i1fRUfX/kxDWs0PPoHjDEmjBVZgxCRdsAg77EdGA+gqheUTdHKSMOG8N130LZtgc15AaJ16+IdJjMnkzcXvsml7S6l1wkBXLTaGGNC5EhNTAnAD0A/VU0EEJH7y6RUZalKFbjg8Ji3Zg00bgzR0cU7zITlE9i6fyt3d787wAU0xpjQOFIT01XAZmCWiLwtIr1wM6krhMTEkjUvvTr/VU6sdyIXnnBh8ApljDFlqMgAoaqTVXUg0B6YhUu50VBE/isipVwZIfyVJEDMT5nP/JT5jOg+wjqmjTHlRnE6qfer6ieqehluVbjfcCObyq39+93SEMUNEK/Nf42YKjEMOWVIcAtmjDFlqERfd1V1l6qOUtVy3QubN8S1UL+1X1v3bWX8svHc1PkmYqrGBLdgxhhThqw9xI+SDHEdtWgUWblZjOg+IriFMsaYMmYBwo/iDnHNysnizUVvcnHri2lXr13wC2aMMWXIAoQfiYnQoAHUqnXk/T5b+Rmb0jbZ0FZjTLlkAcKP4o5gem3+a7Su05pL2l4S/EIZY0wZswDhR3ECxG+bf+OnjT9xV7e7bGirMaZcsjtbIQcPwsaNRw8Qr81/jeqVq3Nzl5vLpmDGGFPGLEAUsn69+3mkALH9wHY+WfoJN8bfaIsBGWPKLQsQhaxZ434eKUCM/nU0GTkZNrTVGFOuWYAo5GhzIHI1l1GLRtGjZQ86NuxYdgUzxpgyZgGikMREqFvXPfyZtX4W63evZ9ipw8q2YMYYU8YsQBRytBFMo38bTZ2oOlzZ4cqyK5QxxoSABYhCjhQgdhzYwWcrP+OG+BuIqhRVtgUzxpgyFtQAISJ9RGSViCSKyCNF7HOtiKwQkeUi8onP9iEissZ7lEma1IwM+OOPogPEx0s+JjMnk1tPvbUsimOMMSF1pBXlSkVEIoHXgd5AMrBARKao6gqffdoCfwXOVtVdItLQ214XeBzoCiiwyPvsrmCVFyApCXJz/QcIVeXtX9+me2x34hvFB7MYxhgTFoJZg+gOJKrqOlXNBMYB/Qvtcxvwet6NX1W3edsvBmao6k7vvRlAnyCWFTjyCKZ5KfNYnrqcoV2GBrsYxhgTFoIZIGKBjT6vk71tvtoB7UTkJxH5RUT6lOCziMgwEVkoIgtTU1NLXeAjBYjRv46mRuUaDDx5YKnPY4wxx4NQd1JXAtoCPYBBwNsiUuypyd7iRV1VtWuDBg1KXZjERKhZE+rXL7g9LSONccvGcV3H62xRIGNMhRHMAJECNPN5Hedt85UMTFHVLFVdD6zGBYzifDbg8kYwiRTcPn75ePZn7Wfoqda8ZIypOIIZIBYAbUWklYhUAQYCUwrtMxlXe0BE6uOanNYB04CLRKSOiNQBLvK2BVVRQ1zf/vVtOjboyBlxZwS7CMYYEzaCFiBUNRsYgbuxrwQ+VdXlIvKUiFzu7TYN2CEiK4BZwIOqukNVdwJP44LMAuApb1vQZGW5RH2F16FesnUJ81PmM/TUoUjhqoUxxpRjQRvmCqCq3wDfFNr2D5/nCvzZexT+7LvAu8Esn68NGyAn5/AaxDu/vkOVyCoMjh9cVkUxxpiwEOpO6rDhbwRTenY6Hy35iCvbX0n96vX9f9AYY8opCxAefwHi85Wfsyt9l3VOG2MqJAsQnsREqFEDGjXK3/bOb+/QsnZLerbqGbqCGWNMiFiA8BQe4qqqzE+Zz6VtL7U1p40xFZLd+TyFh7juSt9FWmYaJ9Q5IXSFMsaYELIAgRu9tG5dwQCxfpdbnLpV7VYhKpUxxoSWBQhg40Y3D8I3QCTtTgKgZe2WISmTMcaEmgUI/I9gWr/bq0HUsRqEMaZisgBBfoDwnUWdtDuJWlVrUTuq2LkDjTGmXAnqTOrjRWIiVKsGTZrkb1u/e73VHowJc1lZWSQnJ5Oenh7qooS9qKgo4uLiqFy5crE/YwECWLMGWreGCJ/6VNLuJNrVaxe6Qhljjio5OZmYmBhatmxpudKOQFXZsWMHycnJtGpV/C++1sTE4UNcVZWk3Uk2gsmYMJeenk69evUsOByFiFCvXr0S17QqfIDIzYW1awsGiNQDqRzIOmABwpjjgAWH4jmW31OFDxApKZCR4X8OhA1xNcZUZBU+QMTFwdatMNBnqem8ORDWSW2MORoRYfDg/OUAsrOzadCgAf369SvRcVq2bMn27duPeZ/FixcjIkydOrVE5z2SCh8gRKBhQ6hVK39b3hwIq0EYY46mRo0aLFu2jIMHDwIwY8YMYmNjy7wcY8eO5ZxzzmHs2LEBO6aNYvIjaXcS9avXJ7pKdKiLYowprvvug8WLA3vMzp3h5ZePulvfvn35+uuvufrqqxk7diyDBg3ihx9+AGDnzp3ccsstrFu3jurVqzNq1Cji4+PZsWMHgwYNIiUlhTPPPBO3fprz8ccf8+qrr5KZmcnpp5/OG2+8QWRkZJHnV1UmTJjAjBkzOPfcc0lPTycqKqrUl1/haxD+rN+93moPxphiGzhwIOPGjSM9PZ0lS5Zw+umnH3rv8ccfp0uXLixZsoTnnnuOG2+8EYAnn3ySc845h+XLl3PllVfyxx9/ALBy5UrGjx/PTz/9xOLFi4mMjGTMmDFHPP/PP/9Mq1ataN26NT169ODrr78OyHVZDcKPpN1JnNLolFAXwxhTEsX4ph8s8fHxJCUlMXbsWPr27VvgvR9//JFJkyYB0LNnT3bs2MHevXuZM2cOn332GQCXXnopderUAWDmzJksWrSIbt26AXDw4EEaNmx4xPOPHTuWgV5H6sCBA/nwww8ZMGBAqa/LAkQhuZpL0u4k+p/YP9RFMcYcRy6//HIeeOABZs+ezY4dO475OKrKkCFD+Oc//1ms/XNycpg0aRJffPEFzz777KFJcWlpacTExBxzOcCamA6zZd8WMnMybQ6EMaZEbrnlFh5//HE6depUYPu55557qIlo9uzZ1K9fn5o1a3LeeefxySefAPC///2PXbt2AdCrVy8mTpzItm3bANeHsWHDhiLPO3PmTOLj49m4cSNJSUls2LCBAQMG8Pnnn5f6mixAFGJzIIwxxyIuLo577rnnsO1PPPEEixYtIj4+nkceeYQPPvgAcH0Tc+bMoWPHjnz22Wc0b94cgJNOOolnnnmGiy66iPj4eHr37s3mzZuLPO/YsWO58sorC2wbMGBAQEYziW/P+fGsa9euunDhwlIfZ8ySMQz+fDAr71pJ+/rtA1AyY0ywrFy5kg4dOoS6GMcNf78vEVmkql397W81iELy5kC0qNUixCUxxpjQsgBRSNLuJBpHN6Za5WqhLooxxoSUBYhCbA6EMcY4FiAKsTTfxhjjWIDwkZObwx97/rAahDHGYAGigJS0FLJzs60GYYwxBDlAiEgfEVklIoki8oif928SkVQRWew9hvq8l+OzfUowy5nH5kAYY0oqHNJ9t2zZkk6dOtG5c2c6derEF198UaJzFyVoqTZEJBJ4HegNJAMLRGSKqq4otOt4VR3h5xAHVbVzsMrnj60DYYwpKd9039WqVQtZuu9Zs2ZRv359Vq1axUUXXUT//qVPFxTMXEzdgURVXQcgIuOA/kDhABE21u9ejyA0q9ks1EUxxpRQCLN9hzzdt6+9e/ceSvxXWsFsYooFNvq8Tva2FTZARJaIyEQR8b0zR4nIQhH5RUSu8HcCERnm7bMwNTW11AVO2p1EbM1YqlaqWupjGWMqjlCn+wa44IILOPnkkzn//PN55plnAnJdoc7m+iUwVlUzRGQ48AHQ03uvhaqmiMgJwHcislRV1/p+WFVHAaPApdoobWFsDoQxx68QZvsOebpvyG9iWrt2Lb169aJHjx5ER5du0bNgBogUwLdGEOdtO0RVfXPijgae93kvxfu5TkRmA12AAgEi0JJ2J3F+i/ODeQpjTDkVqnTfhbVu3ZpGjRqxYsUKunfvfszlgOA2MS0A2opIKxGpAgwECoxGEpEmPi8vB1Z62+uISFXveX3gbILcd5GVk0Xy3mSrQRhjjkmo0n0Xtm3bNtavX0+LFqXPJxe0GoSqZovICGAaEAm8q6rLReQpYKGqTgHuEZHLgWxgJ3CT9/EOwFsikosLYiP9jH4KqI17N5KruTYHwhhzTI6U7vuWW24hPj6e6tWrF0j3PWjQIDp27MhZZ53lN913bm4ulStX5vXXXz/qDf+CCy4gMjKSrKwsRo4cSaNGjUp9TZbu2zNz3Uwu/OhCvrvxOy5odUEAS2aMCRZL910ylu77GNkcCGOMKcgChGf97vVESiRxNeNCXRRjjAkLFiA8SbuTaFarGZUiQj3y1xhjwoMFCI/NgTDGmIIsQHhsHQhjjCnIAgSQnp3OprRNVoMwxhgfFiCADbvdJBSrQRhjSioc0n3v27eP4cOH07p1a0477TR69OjBvHnzSnR+f6xHlvwhrlaDMMaUVDik+x46dCitWrVizZo1REREsH79elasKP3cYgsQuA5qsDkQxhzP7pt6H4u3BDbfd+fGnXm5z9GzAIYy3ffatWuZN28eY8aMISLCNQq1atWKVq1Kfz+zJiZcDaJyRGWaRDc5+s7GGFNIKNN9L1++nM6dOxd7vYiSsBoErgbRonYLIiMC/ws2xpSN4nzTD5ZwSPcdDBYgcDUI638wxpRGqNJ9d+zYkd9//52cnJyA1yKsiQlYv2u9jWAyxpRKqNJ9t27dmq5du/L4448f6sdISkri66+/LvU1VfgAsT9zP6kHUq0GYYwplSOl+160aBHx8fE88sgjBdJ9z5kzh44dO/LZZ5/5TfcdHx9P79692bx58xHPPXr0aLZu3UqbNm04+eSTuemmmwLSLFXh032n7k/l3qn3cnPnm+nduncQSmaMCRZL910yJU33XeH7IBrUaMAnAz4JdTGMMSbsVPgmJmOMMf5ZgDDGHNfKSzN5sB3L78kChDHmuBUVFcWOHTssSByFqrJjxw6ioqJK9LkK3wdhjDl+xcXFkZycTGpqaqiLEvaioqKIiyvZipkWIIwxx63KlSsHJOeQ8c+amIwxxvhlAcIYY4xfFiCMMcb4VW5mUotIKlB0whKnPnDkJZvKr4p67XbdFYtdd8m1UNUG/t4oNwGiOERkYVFTysu7inrtdt0Vi113YFkTkzHGGL8sQBhjjPGrogWIUaEuQAhV1Gu3665Y7LoDqEL1QRhjjCm+ilaDMMYYU0wWIIwxxvhVYQKEiPQRkVUikigij4S6PMEiIu+KyDYRWeazra6IzBCRNd7POqEsYzCISDMRmSUiK0RkuYjc620v19cuIlEiMl9Efveu+0lveysRmef9vY8XkSqhLmswiEikiPwmIl95ryvKdSeJyFIRWSwiC71tAf9brxABQkQigdeBS4CTgEEiclJoSxU07wN9Cm17BJipqm2Bmd7r8iYb+IuqngScAdzl/RuX92vPAHqq6ilAZ6CPiJwB/Av4t6q2AXYBt4awjMF0L7DS53VFuW6AC1S1s8/8h4D/rVeIAAF0BxJVdZ2qZgLjgP4hLlNQqOocYGehzf2BD7znHwBXlGmhyoCqblbVX73nabibRizl/NrV2ee9rOw9FOgJTPS2l7vrBhCROOBSYLT3WqgA130EAf9brygBIhbY6PM62dtWUTRS1c3e8y1Ao1AWJthEpCXQBZhHBbh2r5llMbANmAGsBXarara3S3n9e38ZeAjI9V7Xo2JcN7gvAdNFZJGIDPO2Bfxv3daDqGBUVUWk3I5tFpFoYBJwn6rudV8qnfJ67aqaA3QWkdrA50D7EBcp6ESkH7BNVReJSI9QlycEzlHVFBFpCMwQkQTfNwP1t15RahApQDOf13Hetopiq4g0AfB+bgtxeYJCRCrjgsMYVf3M21whrh1AVXcDs4AzgdoikvcFsDz+vZ8NXC4iSbgm457AK5T/6wZAVVO8n9twXwq6E4S/9YoSIBYAbb0RDlWAgcCUEJepLE0BhnjPhwBfhLAsQeG1P78DrFTVl3zeKtfXLiINvJoDIlIN6I3rf5kFXO3tVu6uW1X/qqpxqtoS9//5O1W9nnJ+3QAiUkNEYvKeAxcBywjC33qFmUktIn1xbZaRwLuq+myIixQUIjIW6IFL/7sVeByYDHwKNMelRL9WVQt3ZB/XROQc4AdgKflt0n/D9UOU22sXkXhch2Qk7gvfp6r6lIicgPtmXRf4DRisqhmhK2nweE1MD6hqv4pw3d41fu69rAR8oqrPikg9Avy3XmEChDHGmJKpKE1MxhhjSsgChDHGGL8sQBhjjPHLAoQxxhi/LEAYY4zxywKEMUchIjle1sy8R8AS/olIS9/Mu8aEE0u1YczRHVTVzqEuhDFlzWoQxhwjLyf/815e/vki0sbb3lJEvhORJSIyU0Sae9sbicjn3toNv4vIWd6hIkXkbW89h+nejGhE5B5vfYslIjIuRJdpKjALEMYcXbVCTUzX+by3R1U7Af/BzdQHeA34QFXjgTHAq972V4HvvbUbTgWWe9vbAq+rakdgNzDA2/4I0MU7zu3BujhjimIzqY05ChHZp6rRfrYn4RbrWeclCtyiqvVEZDvQRFWzvO2bVbW+iKQCcb6pH7zU5DO8RV4QkYeByqr6jIhMBfbhUqVM9ln3wZgyYTUIY0pHi3heEr65gnLI7xu8FLcS4qnAAp8spcaUCQsQxpTOdT4/53rPf8ZlGAW4HpdEENwykHfAoUV+ahV1UBGJAJqp6izgYaAWcFgtxphgsm8kxhxdNW/FtjxTVTVvqGsdEVmCqwUM8rbdDbwnIg8CqcDN3vZ7gVEiciuupnAHsBn/IoGPvSAiwKveeg/GlBnrgzDmGHl9EF1VdXuoy2JMMFgTkzHGGL+sBmGMMcYvpr23qwAAACdJREFUq0EYY4zxywKEMcYYvyxAGGOM8csChDHGGL8sQBhjjPHr/wGU/2Vr58yUZgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_gGU6hGhvncU"
      },
      "source": [
        "### Fight overfitting \n",
        "- Take the `best model` you achieved from Q1 and add regularizers in order to reduce or at least delay overfitting without hurting its classification accuracy. You can consider the following strategies:\n",
        "    1. `dropout`: Add Dropout layers to input or hidden layers\n",
        "    2. `Batch Normalization`: Add Batch Normalization layers after hidden layers.\n",
        "    3. `L1/L2 Regularizer`: Add L2/L1 normal penality with appropriate coefficient.\n",
        "\n",
        "\n",
        "- Pick one or a combination of strategies and modify the model\n",
        "\n",
        "\n",
        "- Train the  model as in Q1 \n",
        "\n",
        "\n",
        "- Plot test loss and test accuracy vs. epoches from the training histories of all the models.\n",
        "\n",
        "\n",
        "- Write your analysis (as markdowns) on the following: \n",
        "   - Are you able to find a good strategy to address overfitting in this case? If so, why do you think the strategy you used can be effective?\n",
        "   - What other strategies have you tried? Are they effective?\n",
        "   - Overall, through Q1 and Q2, can you summarize what you have learned regarding modeling and training?\n",
        "   - Note, this is an open question. Feel free to share you learning experience. We'll summarize all students' analyses and share them in class.\n",
        "  \n",
        "\n",
        "A reference plot is provided. However, depending on the strategy you choose, you may get a different plot."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rvw68tK5vncV"
      },
      "source": [
        "# Define model with regularizer (dropout technique)\n",
        "class linear_model_with_regularizer(nn.Module):\n",
        "    \n",
        "   # add your code here\n",
        "\n",
        "    def __init__(self, h1 = 128 ,dropout_ratio=0.5):  # add more parameters if needsed\n",
        "        \n",
        "       # add your code\n",
        "        super(linear_model_with_regularizer, self).__init__()\n",
        "        self.classifier = nn.Sequential(\n",
        "            nn.Linear(in_features=10000, out_features=h1),\n",
        "            nn.ReLU(),\n",
        "            nn.Dropout(p=dropout_ratio),\n",
        "            nn.Linear(in_features=h1, out_features=46)\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        output = self.classifier(x)\n",
        "        return output\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "10pTtlYTvncV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c9acd590-8752-47c9-8050-15e73ab96b6a"
      },
      "source": [
        "# Train model with regularizer (dropout technique)\n",
        "\n",
        "# dataset\n",
        "train_dataset = news_dataset(x_train, y_train)\n",
        "test_dataset = news_dataset(x_test, y_test)\n",
        "\n",
        "modeld = linear_model_with_regularizer()\n",
        "\n",
        "summary(modeld, (24, 10000))\n",
        "\n",
        "histd = train_model(modeld, train_dataset, test_dataset, device, epochs = 50, l2=0)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "==========================================================================================\n",
              "Layer (type:depth-idx)                   Output Shape              Param #\n",
              "==========================================================================================\n",
              "linear_model_with_regularizer            --                        --\n",
              "├─Sequential: 1-1                        [24, 46]                  --\n",
              "│    └─Linear: 2-1                       [24, 128]                 1,280,128\n",
              "│    └─ReLU: 2-2                         [24, 128]                 --\n",
              "│    └─Dropout: 2-3                      [24, 128]                 --\n",
              "│    └─Linear: 2-4                       [24, 46]                  5,934\n",
              "==========================================================================================\n",
              "Total params: 1,286,062\n",
              "Trainable params: 1,286,062\n",
              "Non-trainable params: 0\n",
              "Total mult-adds (M): 30.87\n",
              "==========================================================================================\n",
              "Input size (MB): 0.96\n",
              "Forward/backward pass size (MB): 0.03\n",
              "Params size (MB): 5.14\n",
              "Estimated Total Size (MB): 6.14\n",
              "=========================================================================================="
            ]
          },
          "metadata": {},
          "execution_count": 18
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training Start\n",
            "Epoch:1 / 50, train loss:2.70317 train_acc:0.51270, Test loss:2.07249 Test acc:0.60590\n",
            "Epoch:2 / 50, train loss:1.83514 train_acc:0.62641, Test loss:1.68949 Test acc:0.64366\n",
            "Epoch:3 / 50, train loss:1.53009 train_acc:0.66558, Test loss:1.50506 Test acc:0.66536\n",
            "Epoch:4 / 50, train loss:1.35918 train_acc:0.68967, Test loss:1.39319 Test acc:0.67882\n",
            "Epoch:5 / 50, train loss:1.24086 train_acc:0.71235, Test loss:1.31552 Test acc:0.69401\n",
            "Epoch:6 / 50, train loss:1.15659 train_acc:0.72776, Test loss:1.25614 Test acc:0.70226\n",
            "Epoch:7 / 50, train loss:1.08316 train_acc:0.74533, Test loss:1.20819 Test acc:0.71658\n",
            "Epoch:8 / 50, train loss:1.01348 train_acc:0.76020, Test loss:1.16771 Test acc:0.72569\n",
            "Epoch:9 / 50, train loss:0.95323 train_acc:0.77289, Test loss:1.13264 Test acc:0.73264\n",
            "Epoch:10 / 50, train loss:0.90873 train_acc:0.78147, Test loss:1.10280 Test acc:0.74045\n",
            "Epoch:11 / 50, train loss:0.86111 train_acc:0.78863, Test loss:1.07503 Test acc:0.74740\n",
            "Epoch:12 / 50, train loss:0.82372 train_acc:0.80067, Test loss:1.05123 Test acc:0.75174\n",
            "Epoch:13 / 50, train loss:0.78166 train_acc:0.80914, Test loss:1.02860 Test acc:0.75781\n",
            "Epoch:14 / 50, train loss:0.74212 train_acc:0.81934, Test loss:1.00891 Test acc:0.76215\n",
            "Epoch:15 / 50, train loss:0.70839 train_acc:0.82737, Test loss:0.99058 Test acc:0.76562\n",
            "Epoch:16 / 50, train loss:0.66838 train_acc:0.83409, Test loss:0.97316 Test acc:0.76649\n",
            "Epoch:17 / 50, train loss:0.64733 train_acc:0.84104, Test loss:0.95734 Test acc:0.76823\n",
            "Epoch:18 / 50, train loss:0.61238 train_acc:0.84733, Test loss:0.94276 Test acc:0.77083\n",
            "Epoch:19 / 50, train loss:0.58611 train_acc:0.85254, Test loss:0.93020 Test acc:0.77257\n",
            "Epoch:20 / 50, train loss:0.55861 train_acc:0.85742, Test loss:0.91793 Test acc:0.77214\n",
            "Epoch:21 / 50, train loss:0.53540 train_acc:0.86317, Test loss:0.90693 Test acc:0.77778\n",
            "Epoch:22 / 50, train loss:0.51096 train_acc:0.87023, Test loss:0.89774 Test acc:0.77865\n",
            "Epoch:23 / 50, train loss:0.48977 train_acc:0.87500, Test loss:0.88978 Test acc:0.78212\n",
            "Epoch:24 / 50, train loss:0.46560 train_acc:0.87619, Test loss:0.88199 Test acc:0.78212\n",
            "Epoch:25 / 50, train loss:0.44456 train_acc:0.87956, Test loss:0.87576 Test acc:0.78299\n",
            "Epoch:26 / 50, train loss:0.42612 train_acc:0.88487, Test loss:0.87004 Test acc:0.78255\n",
            "Epoch:27 / 50, train loss:0.41112 train_acc:0.89062, Test loss:0.86321 Test acc:0.78429\n",
            "Epoch:28 / 50, train loss:0.38731 train_acc:0.89540, Test loss:0.86069 Test acc:0.78559\n",
            "Epoch:29 / 50, train loss:0.38009 train_acc:0.89562, Test loss:0.85734 Test acc:0.78472\n",
            "Epoch:30 / 50, train loss:0.35923 train_acc:0.89757, Test loss:0.85473 Test acc:0.78733\n",
            "Epoch:31 / 50, train loss:0.34884 train_acc:0.90017, Test loss:0.85240 Test acc:0.78950\n",
            "Epoch:32 / 50, train loss:0.32966 train_acc:0.90386, Test loss:0.85367 Test acc:0.78733\n",
            "Epoch:33 / 50, train loss:0.32154 train_acc:0.90549, Test loss:0.85203 Test acc:0.78863\n",
            "Epoch:34 / 50, train loss:0.30860 train_acc:0.90918, Test loss:0.85024 Test acc:0.78776\n",
            "Epoch:35 / 50, train loss:0.29902 train_acc:0.90875, Test loss:0.85211 Test acc:0.78733\n",
            "Epoch:36 / 50, train loss:0.28965 train_acc:0.91092, Test loss:0.85388 Test acc:0.78733\n",
            "Epoch:37 / 50, train loss:0.26954 train_acc:0.91515, Test loss:0.85491 Test acc:0.78733\n",
            "Epoch:38 / 50, train loss:0.26395 train_acc:0.91515, Test loss:0.85543 Test acc:0.78602\n",
            "Epoch:39 / 50, train loss:0.25342 train_acc:0.91829, Test loss:0.85725 Test acc:0.78776\n",
            "Epoch:40 / 50, train loss:0.24479 train_acc:0.91851, Test loss:0.85962 Test acc:0.78689\n",
            "Epoch:41 / 50, train loss:0.24356 train_acc:0.91938, Test loss:0.86045 Test acc:0.78472\n",
            "Epoch:42 / 50, train loss:0.23197 train_acc:0.92133, Test loss:0.86317 Test acc:0.78733\n",
            "Epoch:43 / 50, train loss:0.22223 train_acc:0.92274, Test loss:0.86820 Test acc:0.78776\n",
            "Epoch:44 / 50, train loss:0.21741 train_acc:0.92372, Test loss:0.87107 Test acc:0.78819\n",
            "Epoch:45 / 50, train loss:0.20765 train_acc:0.92578, Test loss:0.87149 Test acc:0.78906\n",
            "Epoch:46 / 50, train loss:0.20144 train_acc:0.92632, Test loss:0.87673 Test acc:0.78819\n",
            "Epoch:47 / 50, train loss:0.19373 train_acc:0.92958, Test loss:0.88298 Test acc:0.78472\n",
            "Epoch:48 / 50, train loss:0.19042 train_acc:0.92904, Test loss:0.88821 Test acc:0.78776\n",
            "Epoch:49 / 50, train loss:0.18400 train_acc:0.92817, Test loss:0.88865 Test acc:0.78776\n",
            "Epoch:50 / 50, train loss:0.18256 train_acc:0.92860, Test loss:0.89365 Test acc:0.78602\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XqAuJMxidATZ",
        "outputId": "d6ee9f7b-b83c-4795-dc8c-ea9bed067959"
      },
      "source": [
        "# Train model with regularizer (L2 technique)\n",
        "\n",
        "# dataset\n",
        "train_dataset = news_dataset(x_train, y_train)\n",
        "test_dataset = news_dataset(x_test, y_test)\n",
        "\n",
        "modele = linear_model(h1 = 128)\n",
        "\n",
        "summary(modele, (24, 10000))\n",
        "\n",
        "histe = train_model(modele, train_dataset, test_dataset, device, epochs = 50, l2 = 0.001)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "==========================================================================================\n",
              "Layer (type:depth-idx)                   Output Shape              Param #\n",
              "==========================================================================================\n",
              "linear_model                             --                        --\n",
              "├─Sequential: 1-1                        [24, 46]                  --\n",
              "│    └─Linear: 2-1                       [24, 128]                 1,280,128\n",
              "│    └─ReLU: 2-2                         [24, 128]                 --\n",
              "│    └─Linear: 2-3                       [24, 46]                  5,934\n",
              "==========================================================================================\n",
              "Total params: 1,286,062\n",
              "Trainable params: 1,286,062\n",
              "Non-trainable params: 0\n",
              "Total mult-adds (M): 30.87\n",
              "==========================================================================================\n",
              "Input size (MB): 0.96\n",
              "Forward/backward pass size (MB): 0.03\n",
              "Params size (MB): 5.14\n",
              "Estimated Total Size (MB): 6.14\n",
              "=========================================================================================="
            ]
          },
          "metadata": {},
          "execution_count": 19
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training Start\n",
            "Epoch:1 / 50, train loss:2.52734 train_acc:0.55968, Test loss:1.92834 Test acc:0.62977\n",
            "Epoch:2 / 50, train loss:1.62560 train_acc:0.66395, Test loss:1.57682 Test acc:0.66102\n",
            "Epoch:3 / 50, train loss:1.33720 train_acc:0.70378, Test loss:1.40999 Test acc:0.68012\n",
            "Epoch:4 / 50, train loss:1.17231 train_acc:0.73644, Test loss:1.30774 Test acc:0.69314\n",
            "Epoch:5 / 50, train loss:1.05833 train_acc:0.75716, Test loss:1.23541 Test acc:0.70964\n",
            "Epoch:6 / 50, train loss:0.97078 train_acc:0.77420, Test loss:1.18000 Test acc:0.72266\n",
            "Epoch:7 / 50, train loss:0.89898 train_acc:0.79036, Test loss:1.13488 Test acc:0.73090\n",
            "Epoch:8 / 50, train loss:0.83752 train_acc:0.80306, Test loss:1.09685 Test acc:0.74002\n",
            "Epoch:9 / 50, train loss:0.78354 train_acc:0.81456, Test loss:1.06401 Test acc:0.74740\n",
            "Epoch:10 / 50, train loss:0.73536 train_acc:0.82541, Test loss:1.03518 Test acc:0.75347\n",
            "Epoch:11 / 50, train loss:0.69190 train_acc:0.83626, Test loss:1.00973 Test acc:0.75955\n",
            "Epoch:12 / 50, train loss:0.65240 train_acc:0.84473, Test loss:0.98704 Test acc:0.76215\n",
            "Epoch:13 / 50, train loss:0.61630 train_acc:0.85145, Test loss:0.96681 Test acc:0.76736\n",
            "Epoch:14 / 50, train loss:0.58319 train_acc:0.85775, Test loss:0.94876 Test acc:0.76866\n",
            "Epoch:15 / 50, train loss:0.55277 train_acc:0.86393, Test loss:0.93269 Test acc:0.77214\n",
            "Epoch:16 / 50, train loss:0.52479 train_acc:0.86947, Test loss:0.91840 Test acc:0.77431\n",
            "Epoch:17 / 50, train loss:0.49900 train_acc:0.87391, Test loss:0.90568 Test acc:0.77474\n",
            "Epoch:18 / 50, train loss:0.47523 train_acc:0.87804, Test loss:0.89445 Test acc:0.77431\n",
            "Epoch:19 / 50, train loss:0.45330 train_acc:0.88281, Test loss:0.88445 Test acc:0.77474\n",
            "Epoch:20 / 50, train loss:0.43308 train_acc:0.88780, Test loss:0.87563 Test acc:0.77517\n",
            "Epoch:21 / 50, train loss:0.41439 train_acc:0.89214, Test loss:0.86789 Test acc:0.77778\n",
            "Epoch:22 / 50, train loss:0.39712 train_acc:0.89572, Test loss:0.86111 Test acc:0.77908\n",
            "Epoch:23 / 50, train loss:0.38116 train_acc:0.89822, Test loss:0.85518 Test acc:0.78038\n",
            "Epoch:24 / 50, train loss:0.36641 train_acc:0.89974, Test loss:0.85004 Test acc:0.78168\n",
            "Epoch:25 / 50, train loss:0.35276 train_acc:0.90267, Test loss:0.84551 Test acc:0.78255\n",
            "Epoch:26 / 50, train loss:0.34013 train_acc:0.90419, Test loss:0.84170 Test acc:0.78472\n",
            "Epoch:27 / 50, train loss:0.32842 train_acc:0.90603, Test loss:0.83837 Test acc:0.78602\n",
            "Epoch:28 / 50, train loss:0.31757 train_acc:0.90755, Test loss:0.83552 Test acc:0.78559\n",
            "Epoch:29 / 50, train loss:0.30749 train_acc:0.91005, Test loss:0.83312 Test acc:0.78472\n",
            "Epoch:30 / 50, train loss:0.29813 train_acc:0.91146, Test loss:0.83111 Test acc:0.78472\n",
            "Epoch:31 / 50, train loss:0.28942 train_acc:0.91233, Test loss:0.82945 Test acc:0.78385\n",
            "Epoch:32 / 50, train loss:0.28133 train_acc:0.91363, Test loss:0.82804 Test acc:0.78342\n",
            "Epoch:33 / 50, train loss:0.27376 train_acc:0.91461, Test loss:0.82692 Test acc:0.78385\n",
            "Epoch:34 / 50, train loss:0.26672 train_acc:0.91547, Test loss:0.82600 Test acc:0.78385\n",
            "Epoch:35 / 50, train loss:0.26012 train_acc:0.91699, Test loss:0.82539 Test acc:0.78385\n",
            "Epoch:36 / 50, train loss:0.25394 train_acc:0.91808, Test loss:0.82485 Test acc:0.78342\n",
            "Epoch:37 / 50, train loss:0.24816 train_acc:0.91938, Test loss:0.82456 Test acc:0.78342\n",
            "Epoch:38 / 50, train loss:0.24273 train_acc:0.92046, Test loss:0.82433 Test acc:0.78429\n",
            "Epoch:39 / 50, train loss:0.23763 train_acc:0.92155, Test loss:0.82434 Test acc:0.78385\n",
            "Epoch:40 / 50, train loss:0.23284 train_acc:0.92263, Test loss:0.82438 Test acc:0.78299\n",
            "Epoch:41 / 50, train loss:0.22833 train_acc:0.92296, Test loss:0.82459 Test acc:0.78299\n",
            "Epoch:42 / 50, train loss:0.22407 train_acc:0.92405, Test loss:0.82491 Test acc:0.78299\n",
            "Epoch:43 / 50, train loss:0.22007 train_acc:0.92491, Test loss:0.82522 Test acc:0.78255\n",
            "Epoch:44 / 50, train loss:0.21630 train_acc:0.92491, Test loss:0.82565 Test acc:0.78255\n",
            "Epoch:45 / 50, train loss:0.21271 train_acc:0.92502, Test loss:0.82618 Test acc:0.78255\n",
            "Epoch:46 / 50, train loss:0.20934 train_acc:0.92556, Test loss:0.82669 Test acc:0.78255\n",
            "Epoch:47 / 50, train loss:0.20615 train_acc:0.92578, Test loss:0.82733 Test acc:0.78342\n",
            "Epoch:48 / 50, train loss:0.20313 train_acc:0.92600, Test loss:0.82801 Test acc:0.78429\n",
            "Epoch:49 / 50, train loss:0.20028 train_acc:0.92708, Test loss:0.82856 Test acc:0.78429\n",
            "Epoch:50 / 50, train loss:0.19756 train_acc:0.92687, Test loss:0.82953 Test acc:0.78385\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I5FHsM8OgAuR",
        "outputId": "fa956390-02c5-4718-bb4e-01011af4cb6c"
      },
      "source": [
        "# Train model with regularizer (L2 + drop out)\n",
        "\n",
        "# dataset\n",
        "train_dataset = news_dataset(x_train, y_train)\n",
        "test_dataset = news_dataset(x_test, y_test)\n",
        "\n",
        "modelf = linear_model_with_regularizer()\n",
        "\n",
        "summary(modelf, (24, 10000))\n",
        "\n",
        "histf = train_model(modelf, train_dataset, test_dataset, device, epochs = 50, l2 = 0.001)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "==========================================================================================\n",
              "Layer (type:depth-idx)                   Output Shape              Param #\n",
              "==========================================================================================\n",
              "linear_model_with_regularizer            --                        --\n",
              "├─Sequential: 1-1                        [24, 46]                  --\n",
              "│    └─Linear: 2-1                       [24, 128]                 1,280,128\n",
              "│    └─ReLU: 2-2                         [24, 128]                 --\n",
              "│    └─Dropout: 2-3                      [24, 128]                 --\n",
              "│    └─Linear: 2-4                       [24, 46]                  5,934\n",
              "==========================================================================================\n",
              "Total params: 1,286,062\n",
              "Trainable params: 1,286,062\n",
              "Non-trainable params: 0\n",
              "Total mult-adds (M): 30.87\n",
              "==========================================================================================\n",
              "Input size (MB): 0.96\n",
              "Forward/backward pass size (MB): 0.03\n",
              "Params size (MB): 5.14\n",
              "Estimated Total Size (MB): 6.14\n",
              "=========================================================================================="
            ]
          },
          "metadata": {},
          "execution_count": 20
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training Start\n",
            "Epoch:1 / 50, train loss:2.64288 train_acc:0.52832, Test loss:2.02845 Test acc:0.59679\n",
            "Epoch:2 / 50, train loss:1.80513 train_acc:0.62424, Test loss:1.67009 Test acc:0.64106\n",
            "Epoch:3 / 50, train loss:1.52810 train_acc:0.66428, Test loss:1.50154 Test acc:0.66276\n",
            "Epoch:4 / 50, train loss:1.37158 train_acc:0.68522, Test loss:1.40138 Test acc:0.67708\n",
            "Epoch:5 / 50, train loss:1.25997 train_acc:0.70812, Test loss:1.32930 Test acc:0.69097\n",
            "Epoch:6 / 50, train loss:1.18990 train_acc:0.72168, Test loss:1.27490 Test acc:0.69922\n",
            "Epoch:7 / 50, train loss:1.11390 train_acc:0.73492, Test loss:1.23184 Test acc:0.70790\n",
            "Epoch:8 / 50, train loss:1.05729 train_acc:0.74740, Test loss:1.19603 Test acc:0.71832\n",
            "Epoch:9 / 50, train loss:1.01110 train_acc:0.75911, Test loss:1.16493 Test acc:0.72352\n",
            "Epoch:10 / 50, train loss:0.96753 train_acc:0.76747, Test loss:1.13679 Test acc:0.72873\n",
            "Epoch:11 / 50, train loss:0.92042 train_acc:0.77604, Test loss:1.11176 Test acc:0.73872\n",
            "Epoch:12 / 50, train loss:0.88964 train_acc:0.78461, Test loss:1.08879 Test acc:0.74002\n",
            "Epoch:13 / 50, train loss:0.85212 train_acc:0.79069, Test loss:1.06817 Test acc:0.74740\n",
            "Epoch:14 / 50, train loss:0.82487 train_acc:0.79590, Test loss:1.04979 Test acc:0.75043\n",
            "Epoch:15 / 50, train loss:0.79943 train_acc:0.80382, Test loss:1.03239 Test acc:0.75347\n",
            "Epoch:16 / 50, train loss:0.76492 train_acc:0.81120, Test loss:1.01617 Test acc:0.75608\n",
            "Epoch:17 / 50, train loss:0.74582 train_acc:0.81565, Test loss:1.00099 Test acc:0.75781\n",
            "Epoch:18 / 50, train loss:0.71761 train_acc:0.82183, Test loss:0.98699 Test acc:0.76042\n",
            "Epoch:19 / 50, train loss:0.69476 train_acc:0.82498, Test loss:0.97341 Test acc:0.76302\n",
            "Epoch:20 / 50, train loss:0.66882 train_acc:0.82747, Test loss:0.96173 Test acc:0.76780\n",
            "Epoch:21 / 50, train loss:0.64901 train_acc:0.83659, Test loss:0.95080 Test acc:0.76866\n",
            "Epoch:22 / 50, train loss:0.62694 train_acc:0.84418, Test loss:0.94048 Test acc:0.76997\n",
            "Epoch:23 / 50, train loss:0.61034 train_acc:0.84429, Test loss:0.93024 Test acc:0.77257\n",
            "Epoch:24 / 50, train loss:0.59022 train_acc:0.84885, Test loss:0.92113 Test acc:0.77387\n",
            "Epoch:25 / 50, train loss:0.58049 train_acc:0.85211, Test loss:0.91396 Test acc:0.77300\n",
            "Epoch:26 / 50, train loss:0.56265 train_acc:0.85601, Test loss:0.90644 Test acc:0.77474\n",
            "Epoch:27 / 50, train loss:0.54647 train_acc:0.85959, Test loss:0.89876 Test acc:0.77604\n",
            "Epoch:28 / 50, train loss:0.53377 train_acc:0.86274, Test loss:0.89315 Test acc:0.77648\n",
            "Epoch:29 / 50, train loss:0.51269 train_acc:0.86773, Test loss:0.88630 Test acc:0.77691\n",
            "Epoch:30 / 50, train loss:0.50197 train_acc:0.86795, Test loss:0.88008 Test acc:0.77865\n",
            "Epoch:31 / 50, train loss:0.49239 train_acc:0.87055, Test loss:0.87414 Test acc:0.77908\n",
            "Epoch:32 / 50, train loss:0.47580 train_acc:0.87554, Test loss:0.86968 Test acc:0.78255\n",
            "Epoch:33 / 50, train loss:0.46693 train_acc:0.87587, Test loss:0.86475 Test acc:0.78038\n",
            "Epoch:34 / 50, train loss:0.45278 train_acc:0.88075, Test loss:0.86147 Test acc:0.78212\n",
            "Epoch:35 / 50, train loss:0.44728 train_acc:0.88043, Test loss:0.85711 Test acc:0.78299\n",
            "Epoch:36 / 50, train loss:0.44002 train_acc:0.88227, Test loss:0.85498 Test acc:0.78168\n",
            "Epoch:37 / 50, train loss:0.42664 train_acc:0.88194, Test loss:0.85142 Test acc:0.77995\n",
            "Epoch:38 / 50, train loss:0.41713 train_acc:0.88759, Test loss:0.84604 Test acc:0.78082\n",
            "Epoch:39 / 50, train loss:0.40868 train_acc:0.88748, Test loss:0.84326 Test acc:0.78255\n",
            "Epoch:40 / 50, train loss:0.39290 train_acc:0.89301, Test loss:0.84167 Test acc:0.78385\n",
            "Epoch:41 / 50, train loss:0.38777 train_acc:0.89225, Test loss:0.84020 Test acc:0.78385\n",
            "Epoch:42 / 50, train loss:0.37853 train_acc:0.89355, Test loss:0.83837 Test acc:0.78429\n",
            "Epoch:43 / 50, train loss:0.37795 train_acc:0.89497, Test loss:0.83511 Test acc:0.78646\n",
            "Epoch:44 / 50, train loss:0.37164 train_acc:0.89855, Test loss:0.83465 Test acc:0.78429\n",
            "Epoch:45 / 50, train loss:0.36320 train_acc:0.89724, Test loss:0.83294 Test acc:0.78776\n",
            "Epoch:46 / 50, train loss:0.35657 train_acc:0.89996, Test loss:0.83003 Test acc:0.78906\n",
            "Epoch:47 / 50, train loss:0.34583 train_acc:0.90148, Test loss:0.83121 Test acc:0.78646\n",
            "Epoch:48 / 50, train loss:0.34342 train_acc:0.90256, Test loss:0.82960 Test acc:0.78689\n",
            "Epoch:49 / 50, train loss:0.33856 train_acc:0.90538, Test loss:0.83045 Test acc:0.78602\n",
            "Epoch:50 / 50, train loss:0.33430 train_acc:0.90104, Test loss:0.82686 Test acc:0.78733\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I9M0SE51vncV",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 332
        },
        "outputId": "c2f0517f-7234-4279-ac2a-75f7c322f32f"
      },
      "source": [
        "# Plot validation loss curves\n",
        "epochs = range(1, len(hista['train_loss']) + 1)\n",
        "\n",
        "plt.plot(epochs, hista['test_loss'], 'r-', label='Model A')\n",
        "plt.plot(epochs, histb['test_loss'], 'b-', label='Model B')\n",
        "plt.plot(epochs, histc['test_loss'], 'g-', label='Model C')\n",
        "plt.plot(epochs, histf['test_loss'], '-', color='purple', label='Model A with regularizer (L2 & DO)')\n",
        "\n",
        "plt.title('Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1b01df50>]"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1b030510>]"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1b030a10>]"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1b0309d0>]"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Test loss')"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 0, 'Epochs')"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Loss')"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f1f1b02df50>"
            ]
          },
          "metadata": {},
          "execution_count": 21
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVxVZf7A8c/DoqwubOICggjKIqCSlalplm1mi2Zq65RZVlP+mmayaSbbbJlpmpqyzMxsxXIpK5cyrbRcSlwR3AFFEdlk33l+f5wLXhCQ7XIBv+/X67zu5Z7nnPNclPu9z/Y9SmuNEEIIUZONtSsghBCibZIAIYQQolYSIIQQQtRKAoQQQohaSYAQQghRKwkQQgghaiUBQohWpJTyU0pppZSdtesixPlIgBAXNKVUntlWoZQqNPv59iac72el1HRL1FWI1ibfYsQFTWvtUvlcKZUITNda/2i9GgnRdkgLQohaKKVslFKzlVJHlFIZSqkvlVJupn0OSqlPTa+fUUr9oZTqoZSaC4wE3ja1QN5uwHV6KaW+UUplKqUOK6XuN9s3TCm1XSmVo5RKVUq9Xt/1LfW7EBcuaUEIUbs/AzcBlwNpwP+AecBU4G6gK+ADFAORQKHW+mml1GXAp1rrhQ28zhIgFugFDATWKaWOaK03AG8Cb2qtP1FKuQBhpmNqvX4z368Q55AWhBC1exB4WmudrLUuBp4FJpkGl0sBd6C/1rpcax2jtc5p7AWUUj7AZcCTWusirfUuYCFwl6lIKdBfKeWhtc7TWm81e73Z1xfifCRACFG7vsBXpi6cM0A8UA70AD4BvgeWKKVOKqX+pZSyb8I1egGZWutcs9eSgN6m5/cBQcB+UzfSeNPrLXV9IeolAUKI2h0HrtVadzPbHLTWJ7TWpVrr57TWIcBwYDxnv/U3Jj3yScBNKeVq9povcAJAa31Iaz0V8AJeBZYppZzPc30hWowECCFqNx+Yq5TqC6CU8lRK3Wh6PkYpNUgpZQvkYHT5VJiOSwX6NeQCWuvjwGbgZdPAczhGq+FT03XuUEp5aq0rgDOmwyrOc30hWowECCFq9ybwDfCDUioX2ApcbNrnDSzD+HCOB37B6PapPG6SUipLKfW/BlxnKuCH0Zr4CphjNs32GmCfUirPdN4pWuvC81xfiBaj5IZBQgghaiMtCCGEELWSACGEEKJWEiCEEELUSgKEEEKIWnWoVBseHh7az8/P2tUQQoh2IyYmJl1r7Vnbvg4VIPz8/Ni+fbu1qyGEEO2GUiqprn3SxSSEEKJWEiCEEELUSgKEEEKIWnWoMQjRcZWWlpKcnExRUZG1qyJEu+Tg4ECfPn2wt2944l8JEKJdSE5OxtXVFT8/P5RS1q6OEO2K1pqMjAySk5Px9/dv8HHSxSTahaKiItzd3SU4CNEESinc3d0b3QKXACHaDQkOQjRdU/5+LvgAobXmxY0v8v3h761dFSGEaFMu+AChlOK1za+x6tAqa1dFtHFKKe64446qn8vKyvD09GT8+PH1HHUuPz8/0tPTm1xm165dKKVYu3Zto64rRGNd8AECwNvFm1N5p6xdDdHGOTs7ExsbS2FhIQDr1q2jd+/e5zmq5UVHRzNixAiio6Nb/driwiIBAgkQouGuu+46Vq0yWpvR0dFMnTq1al9mZiY33XQT4eHhXHLJJezZsweAjIwMxo0bR2hoKNOnT8f8Jl2ffvopw4YNIzIykgceeIDy8vJ6r6+1ZunSpSxevJh169bJtF9hUTLNFSNA7EjZYe1qiIaaNQt27WrZc0ZGwhtvnLfYlClTeP755xk/fjx79uzh3nvvZdOmTQDMmTOHwYMH8/XXX7Nhwwbuuusudu3axXPPPceIESN45plnWLVqFR988AEA8fHxfPHFF/z222/Y29vz0EMP8dlnn3HXXXfVef3Nmzfj7+9PQEAAo0ePZtWqVUycOLFlfgdC1CABAujp0lNaEKJBwsPDSUxMJDo6muuuu67avl9//ZXly5cDcMUVV5CRkUFOTg4bN25kxYoVAFx//fV0794dgPXr1xMTE8NFF10EQGFhIV5eXvVePzo6milTpgBGsPr4448lQAiLkQCB0YLILcklvyQf507O1q6OOJ8GfNO3pAkTJvDEE0/w888/k5GR0eTzaK25++67efnllxtUvry8nOXLl7Ny5Urmzp1btfgpNzcXV1fXJtdDiLrIGARGgACkFSEa5N5772XOnDkMGjSo2usjR47ks88+A+Dnn3/Gw8ODLl26MGrUKD7//HMA1qxZQ1ZWFgBjx45l2bJlnD59GjDGMJKS6sy8zPr16wkPD+f48eMkJiaSlJTExIkT+eqrryzxNoWQAAESIETj9OnTh0cfffSc15999lliYmIIDw9n9uzZfPTRR4AxNrFx40ZCQ0NZsWIFvr6+AISEhPDiiy8ybtw4wsPDueqqq0hJSanzutHR0dx8883VXps4caLMZhIWo8xnVLR3UVFRuik3DNp9ajeR70Wy7NZlTAyR/ty2KD4+nuDgYGtXQ4h2rba/I6VUjNY6qrby0oJAWhBCCFEbCRCAh5MHtspWAoQQQpiRAAHY2tji5ewlAUIIIcxIgDDxdvEmJa/uAUIhhLjQWCxAKKV8lFI/KaXilFL7lFKP1VJmtFIqWym1y7Q9Y7bvGqXUAaXUYaXUbEvVs5Kk2xBCiOosuVCuDPiL1nqHUsoViFFKrdNax9Uot0lrXS0dplLKFpgHXAUkA38opb6p5dgW4+3izZ7UPZY6vRBCtDsWa0ForVO01jtMz3OBeKChqS+HAYe11ke11iXAEuBGy9TU4O3iTWp+KhW6wpKXEe1YW0j37efnx6BBg4iMjGTQoEGsXLmyUdcWojFaZQxCKeUHDAa21bL7UqXUbqXUGqVUqOm13sBxszLJ1BFclFIzlFLblVLb09LSmlzHni49KasoI7Mws8nnEB1bW0n3/dNPP7Fr1y6WLVtW64I9IVqKxQOEUsoFWA7M0lrn1Ni9A+irtY4A3gK+buz5tdYLtNZRWusoT0/PJtdT1kKIhrB2um9zOTk5VYn/hLAEiybrU0rZYwSHz7TWK2ruNw8YWuvVSql3lFIewAnAx6xoH9NrFlMZIFJyUwjzCrPkpUQzWTHbt9XTfQOMGTMGrTVHjx7lyy+/bPZ7F6IuFgsQyrhD9gdAvNb69TrKeAOpWmutlBqG0aLJAM4AgUopf4zAMAWYZqm6grQgRMNYO903GF1MHh4eHDlyhLFjxzJ69GhcXFxa8m0KAVi2BXEZcCewVylV+X3v74AvgNZ6PjAJmKmUKgMKgSnaaH+XKaUeAb4HbIFFWut9FqyrBIh2xMrZvq2W7rumgIAAevToQVxcHMOGDWtyPYSoiyVnMf2qtVZa63CtdaRpW621nm8KDmit39Zah2qtI7TWl2itN5sdv1prHaS1DtBaz7VUPSu5dHLByd5JAoQ4L2ul+67p9OnTJCQk0Ldv35Z4W0KcQ24YZKKUMu4sly8BQtSvvnTf9957L+Hh4Tg5OVVL9z116lRCQ0MZPnx4rem+KyoqsLe3Z968eef9wB8zZgy2traUlpbyyiuv0KNHj5Z/k0Ig6b6rGbFoBJ3tOrP+rvUtWCvREiTdtxDNJ+m+m8HbxZuUXMnHJIQQIAGiGsnHJIQQZ0mAMOPt4k1WURbFZcXWrooQQlidBAgzlVNdU/NTrVwTIYSwPgkQZnq69ARkLYQQQoAEiGpksZwQQpwlAcKMeT4mIWpqC+m+8/LyeOCBBwgICGDo0KGMHj2abdtqS5IsRPPJQjkzXs5GHhxpQYjamKf7dnR0tEq67+nTp+Pv78+hQ4ewsbEhISGBuDiL3UdLXOCkBWHG3tYeDycPCRCiTtZM933kyBG2bdvGiy++iI2N8afr7+/P9ddfb4m3KoS0IGrydvGWdBtt3Ky1s9h1qmXzfUd6R/LGNefPAmjNdN/79u0jMjISW1vblnvjQtRDAkQNPV16SgtC1KktpPsWorVIgKjB28WbQ8cOWbsaoh4N+aZvSdZK9x0aGsru3bspLy+XVoRoFTIGUUNlPqaOlMRQtCxrpfsOCAggKiqKOXPmVP3/TExMrBoTEaKlSYCowdvFm+LyYrKLs61dFdFG1ZfuOyYmhvDwcGbPnl0t3ffGjRsJDQ1lxYoVtab7Dg8P56qrriIlpf4p1gsXLiQ1NZX+/fsTFhbGPffcI91SwmIk3XcNn+/9nNtX3E78w/EM9BjYQjUTzSXpvoVoPkn33UyymloIIQwWCxBKKR+l1E9KqTil1D6l1GO1lLldKbVHKbVXKbVZKRVhti/R9PoupVTzmgWNIPmYhBDCYMlZTGXAX7TWO5RSrkCMUmqd1tp82WcCcLnWOkspdS2wALjYbP8YrXX9OQlamKTbEEIIg8UChNY6BUgxPc9VSsUDvYE4szKbzQ7ZCvSxVH0aqptDNzrZdpIWhBDigtcqYxBKKT9gMFBfVrH7gDVmP2vgB6VUjFJqRj3nnqGU2q6U2p6WltYSdZXV1EIIQSsslFNKuQDLgVla65w6yozBCBAjzF4eobU+oZTyAtYppfZrrTfWPFZrvQCja4qoqKgWmZIltx4VQggLtyCUUvYYweEzrfWKOsqEAwuBG7XWVctStdYnTI+nga+AYZasqzkJEKI2bSHdN8CuXbtQSrF27dpGXdfc9OnTq7LAvvTSS1WvJyYmEhYW1uTztoTRo0fT2OnqzzzzDD/++GOL12Xnzp3cd999ACxevJhHHnmk2v6CggKuv/56Bg4cSGhoKLNnz67zXN9++y0hISGEhYXx9NNP11lu8eLFeHp6MnjwYAIDA7n66qvZvPlsb7zWmhdffJHAwECCgoIYM2YM+/btq9p/5ZVXVi3GbC5LzmJSwAdAvNb69TrK+AIrgDu11gfNXnc2DWyjlHIGxgGxlqprTZKPSdTGPN03YJV032BkkR0xYgTR0dFNPsfChQsJCQkBqgeIpigrK2vW8c1VXl7O888/z5VXXtms89T2Pl566aVaF0Wae+KJJ9i/fz87d+7kt99+Y82aNbWWmzVrFqtWrSI2Npbp06fXe87bbruNnTt3cujQIWbPns0tt9xCfHw8APPmzWPz5s3s3r2bgwcP8tRTTzFhwgSKiooAuPPOO3nnnXca8pbPy5ItiMuAO4ErTFNVdymlrlNKPaiUetBU5hnAHXinxnTWHsCvSqndwO/AKq11078uNZK3izdp+WmUVVj3P75oe6yZ7huMb49Lly5l8eLFrFu3rupDwdzSpUt5/PHHAXjzzTfp168fAEePHuWyyy4Dzn5Lnz17NoWFhURGRnL77bcDxgfu/fffT2hoKOPGjasKiObuueceHnzwQS6++GL+9re/ceTIEa655hqGDh3KyJEj2b9/P2CkKL/kkksYNGgQ//jHP3BxcQGMVCTmLa9HHnmExYsXn3OdmTNnEhUVRWhoKHPmzKl63c/PjyeffJIhQ4awdOlS7rnnHpYtW8b27duJjIwkMjKSQYMGYXxPpc761Xwf5nJzc9mzZw8RERHUxcnJiTFjxgDQqVMnhgwZQnJycq1lO3XqVLXP39+/znPWNGbMGGbMmMGCBQsAePXVV3n77bdxcnICYNy4cQwfPrwqzcuECROa9eXBnCVnMf0KqPOUmQ6cE0q11keBuv9VLMzbxRuN5nT+aXq59rJWNUQd1s5ay6ldLdvC84705po3rjlvOWum+wbYvHkz/v7+BAQEMHr0aFatWsXEiROrlRk5ciT/+te/ANi0aRPu7u6cOHGCTZs2MWrUqGplX3nlFd5++2127TLSpycmJnLo0CGio6N5//33mTx5MsuXL6/WtVYpOTmZzZs3Y2try9ixY5k/fz6BgYFs27aNhx56iA0bNvDYY4/x2GOPMXXqVObPn3/e329Nc+fOxc3NjfLycsaOHcuePXsIDw8HwN3dnR07dgBUdbdFRUVVvZe//vWvXHON8W86Y8aMWutX832Y2759e6O6286cOcO3337LY4+ds+SLiooKQkJCuPfee1m3bh1+fn6N+j0MGTKE9957j5ycHPLz86uCfqWoqKiqbqbu3btTXFxMRkYG7u7ujbpOTZLNtRbmq6klQAhz1k73HR0dzZQpUwAjWH388cfnBAhvb2/y8vLIzc3l+PHjTJs2jY0bN7Jp0yZuueWW875Hf39/IiMjARg6dCiJiYm1lrv11luxtbUlLy+PzZs3c+utt1btKy4uBmDLli18/fXXAEybNo0nnnjivNc39+WXX7JgwQLKyspISUkhLi6uKkDcdtttdR73xRdfsGPHDn744Yd662f+PmpKSUnB09OzQfUsKytj6tSpPProo+d8eAO89dZbREREMHPmTG644QY2bNhAYmIir776KsuWLTvv+RubEsnLy4uTJ09KgLAESbfRtjXkm74lWSvdd3l5OcuXL2flypXMnTsXrTUZGRnk5ubi6uparezw4cP58MMPGTBgACNHjmTRokVs2bKF//znP+e9TufOnaue29ra1trFBMaYDBjfjrt161b1zb0h7OzsqKioqPq5tq6yhIQEXnvtNf744w+6d+/OPffcU61c5fVrio2N5dlnn2Xjxo3Y2tqet351ncfR0bHWetVmxowZBAYGMmvWrFr3f//99/ztb39j9OjR/POf/+T6669n2LBhVcH+fHbu3ElwcDBdunTB2dmZo0ePVgtEMTExXH755VU/FxUV4ejo2KBz10dyMdVCAoSoj7XSfa9fv57w8HCOHz9OYmIiSUlJTJw4ka+++uqcsiNHjuS1115j1KhRDB48mJ9++onOnTvTtWvXc8ra29tTWlratF8G0KVLF/z9/Vm6dClgBL7du3cDcMkll1S1qpYsWVJ1TN++fYmLi6O4uJgzZ86wfv36c86bk5ODs7MzXbt2JTU1tc7BX3Nnzpxh6tSpfPzxx1Xf/uurX32Cg4M5fPjwecv94x//IDs7mzfeqPs+JYMHD+bTTz+loqKCyZMnExgYyOeff96g28X+8ssvLFiwgPvvvx8wus4effTRqsD9448/8uuvvzJt2rSq93fq1KlGd2PVRgJELSRAiPpYK913dHQ0N998c7XXJk6cWOuA5MiRIzl+/DijRo3C1tYWHx8fRowYcU45ML79hoeHVw1SN8Vnn33GBx98QEREBKGhoaxcuRKAN954g9dff53w8HAOHz5cFaB8fHyYPHkyYWFhTJ48mcGDB59zzoiICAYPHszAgQOZNm1a1QB7fVauXElSUhL3339/1WB1ffWrz8CBA8nOziY3N7fqtcWLF9OnT5+qLTk5mblz5xIXF8eQIUOIjIxk4cKF55zr6aefRmtNWFgYQ4cOpUePHjzwwANMmzatWkuq0hdffEFkZCRBQUG89NJLLF++vCoL65///GcuuugiBg0axIABA3jhhRdYuXJlVYshJiaGSy65BDu75ncQSbrvOnR/tTt3DLqDt657q0XOJ5pH0n23TwUFBTg6OqKUYsmSJURHRzfow7mt+O9//4urq+t5p6W2JY899hgTJkxg7Nix5+xrbLpvGYOog6TbEKL5YmJieOSRR9Ba061bNxYtWmTtKjXKzJkzq7qm2ouwsLBag0NTSICog6ymFqL5Ro4c2aD+/rbKwcGBO++809rVaJTKsYqWIGMQdZAA0fZ0pO5QIVpbU/5+JEDUwdtZAkRb4uDgQEZGhgQJIZqgckq0g4NDo46TLqY69HTtSV5JHnklebh0crF2dS54lTNGWiKluxAXIgcHB/r0adwtdyRA1MF8qmt/t/5Wro2wt7dvVP4aIUTzSRdTHWQthBDiQicBog4SIIQQFzoJEHWQACGEuNBJgKiDh5MHtspWAoQQ4oIlAaIONsqGHi49JEAIIS5YEiDq4e3iTUpe3cnThBCiI7vgA0RFWQVb/ruFhA0J5+zr79affaf31XKUEEJ0fBYLEEopH6XUT0qpOKXUPqXUOffhU4b/KaUOK6X2KKWGmO27Wyl1yLTdbbF62io2Pr+RuGVx5+wb3mc4SdlJnMg5YanLCyFEm2XJFkQZ8BetdQhwCfCwUiqkRplrgUDTNgN4F0Ap5QbMAS4GhgFzlFLdLVFJpRQewR6kx6efs2+4z3AAtiRvscSlhRCiTbNYgNBap2itd5ie5wLxQO8axW4EPtaGrUA3pVRP4GpgndY6U2udBawDLHafSY+BHqTFn5vCIdI7Ekc7RzYf32ypSwshRJvVKmMQSik/YDCwrcau3sBxs5+TTa/V9Xpt556hlNqulNrelDw9FRVQ4ORBfmo+hVnV771rb2vPRb0vkgAhhLggWTxAKKVcgOXALK11TkufX2u9QGsdpbWOqrwHbWO99L5xXPr+WrqZ+gxnR8oOCktrv3G7EEJ0VBYNEEope4zg8JnWekUtRU4APmY/9zG9VtfrLc7GBlz9PQDqHIcorSglJiXGEpcXQog2y5KzmBTwARCvtX69jmLfAHeZZjNdAmRrrVOA74FxSqnupsHpcabXLKJ3WDfKlW2t4xCX+lwKIN1MQogLjiXTfV8G3AnsVUrtMr32d8AXQGs9H1gNXAccBgqAP5n2ZSqlXgD+MB33vNY601IVDRpgQ7p2Jy3u3BaEh5MHQe5BEiCEEBcciwUIrfWvgDpPGQ08XMe+RUCr3OE8KAi+w5NTe0/Wun+4z3BWHVyF1hqjYSSEEB3fBb+SGq0Jck0hHQ/ykrMoKyo7p8jwPsNJK0jjSNYRK1RQCCGsQwJEeTlBU4aQhgdoyDiYcU6RygVz0s0khLiQSICws8M9yJ0SO0eAWgeqgz2D6dq5qwQIIcQFRQIEQHAwnrZZaGqf6mqjbLjU51IJEEKIC4oECIDgYPoXx5Fr273WAAHGOETs6Viyi7JbuXJCCGEdEiAAgoMJ4gCnyj1I3Vd7uo7hPsPRaLadqJktRAghOiYJEGAKEAdJw4PMQxlUlFecU2RY72HYKBvpZhJCXDAkQAAEBRHEIdLxpKKknDOJZ84p4trZlUFegyRACCEuGBIgAJyc6O9bYkx1pfaBajC6mbYmb6W8orw1ayeEEFYhAcLEOdSPTnZG11JtU13BCBC5JbnsS5PbkAohOj4JEJWCg/GrOEqxvXO9LQiQBXNCiAuDBIhKAwcSVLGf0xUetd4XAsC/mz89nHtIgBBCXBAkQFQyzWQ6Ve5JWlw6Rh7B6pRSDPcZLgFCCHFBkABRyWyqa3F2Efmp+bUWG+4znCNZR0jNS23lCgohROuSAFHJ3Z2g7umkY9x+tL6BaoAtyVtarWpCCGENDQoQSilnpZSN6XmQUmqC6XaiHYpfqDNZdAPqnuo6pOcQOtl2km4mIUSH19AWxEbAQSnVG/gB405xiy1VKWuxDwnEU2VQbtepzhaEg50DQ3sOlQAhhOjwGhoglNa6ALgFeEdrfSsQarlqWUlwMEF6P9l27mTsP/e+EJVG+o7k9xO/k15QeytDCCE6ggYHCKXUpcDtwCrTa7bnOWCRUuq0Uiq2jv1/VUrtMm2xSqlypZSbaV+iUmqvad/2hr6ZZgsOZgAHOFHiUWcLAuCuiLsorSjlgx0ftFrVhBCitTU0QMwCngK+0lrvU0r1A346zzGLgWvq2qm1/rfWOlJrHWk69y9a60yzImNM+6MaWMfmM81kSq3wIvdELsU5xbUWC/UKZbTfaN7d/q6k3RBCdFgNChBa61+01hO01q+aBqvTtdaPnueYjUBmfWXMTAWiG1jWcvr0Icjh+NmcTHUsmAN4KOohkrKTWH1odWvVTgghWlVDZzF9rpTqopRyBmKBOKXUX1uiAkopJ4yWxnKzlzXwg1IqRik1oyWu0yA2NgT1rzjvVFeAmwbeRC/XXryz/Z3Wqp0QQrSqhnYxhWitc4CbgDWAP8ZMppZwA/Bbje6lEVrrIcC1wMNKqVF1HayUmqGU2q6U2p6WVvcHekP1DPekmE5oG5s6p7oC2NvaM2PIDNYeXsvhzMPNvq4QQrQ1DQ0Q9qZ1DzcB32itSzG+5beEKdToXtJanzA9nga+AobVdbDWeoHWOkprHeXp6dnsyqiQYAI5RKGjW71dTAAzhs7AzsaOd/94t9nXFUKItqahAeI9IBFwBjYqpfoCOc29uFKqK3A5sNLsNWellGvlc2AcRrdW66hMuVHhVm8LAqCna09uCb6FRbsWUVBa0EoVFEKI1tHQQer/aa17a62v04YkYEx9xyilooEtwAClVLJS6j6l1INKqQfNit0M/KC1Nk981AP4VSm1G/gdWKW1Xtuod9UcAwcSxEGSCr3IPJJJeUn9s5QevuhhzhSdIXqv9cfYhRCiJdk1pJDpm/4coHIs4BfgeSC7rmO01lPPd16t9WJqrMjWWh8FIhpSL4vo358gdZiv9c3ock3GoQy8Qr3qLD7SdyRhXmHM+2Me9w6+F6VUK1ZWCCEsp6FdTIuAXGCyacsBPrRUpayqUyeCfArPe/vRSkopHr7oYXae2snW5K2tUUMhhGgVDQ0QAVrrOVrro6btOaCfJStmTYFhncnAHah/LUSlO8LvwLWTK/P+mGfpqgkhRKtpaIAoVEqNqPxBKXUZUGiZKllf9whfunGGMpeu521BALh0cuHuiLtZGreU0/mnW6GGQghheQ0NEA8C80w5khKBt4EHLFYrazMNVGfZeZK0KQldcf4ZvQ9d9BAl5SUs3LGwFSoohBCW19BZTLu11hFAOBCutR4MXGHRmlmTaarrnpIB5BzPIXlb8vkP8QzmCv8rmL99PmUVZa1QSSGEsKxG3VFOa51jWlEN8LgF6tM2mFoQvxcMwrazLfu+2Negwx4d9ijHc47zzh+SfkMI0f4155ajHXc+p6srQe4ZFNMZj0sDiVsa16BupgkDJnBt/2t5av1TJJ5JtHw9hRDCgpoTIFoq1UabFBRoPKqwEHJP5nLs12PnPUYpxfzx87FRNtz/7f1o3aF/RUKIDq7eAKGUylVK5dSy5QK9WqmOVhEwuAuKCk65BmHnaMe+LxvWzeTb1ZdXr3yVH4/+yOJdiy1bSSGEsKB6A4TW2lVr3aWWzVVr3aBV2O2V46D++HKMg/HlBF0fRNyyOCrKKxp07INRDzLSdySP//A4KbkpFjudD88AACAASURBVK6pEEJYRnO6mDq24GDC2cOvmxXBt4aSn5pP0i9JDTrURtmwcMJCCksLeWTNIxauqBBCWIYEiLoEBzOVaI6ddiTFJRB7Z3tiv2h4Utkg9yCeG/0cK+JXsCxumQUrKoQQliEBoi5eXtzU9We6dCrkkyX2DJgwgPjl8VSUNaybCeAvw//CkJ5DeHj1w2QWNvTuq0II0TZIgKiLUjiG9mNKt+9Zvhz63RBKYUYhCRsSGnwKOxs7Fk1YRGZhJo9/33GXjQghOiYJEPW5+mruOf0vCgpgZ25/Onfp3KhuJoAI7wievOxJPtr9EV/Ff2WhigohRMuTAFGfW2/lErYQ5JXFR5/ZMeDGAexfsf+8NxGq6R+j/sFFvS7i9hW3S0pwIUS7IQGiPsHBqJAQ7nFeysaN4DYqlKIzRRz98WijTuNg58B3076jl2svboi+gUMZhyxUYSGEaDkSIM7n1lu5M+EFlNJsSAzAoZtDg3MzmfNy9mLtHcadU6/57BpJCy6EaPMkQJzPpEn0IZkrByTz0ae2DLhpIPu/3k9ZUeMztvZ36893U78jJTeF8Z+PJ78k//wHCSGElVgsQCilFimlTiulah3VVUqNVkplK6V2mbZnzPZdo5Q6oJQ6rJSabak6NkhoKAwcyD02H5OUBOXBoRTnFHP4+8NNOt3FfS7mi0lfEJMSw+RlkyU1uBCizbJkC2IxcM15ymzSWkeatucBlFK2wDzgWiAEmKqUCrFgPeunFEyaxE3xL9PFtYJvY/1xdHck7su4Jp/yhgE38M5177D60GoeWvWQJPUTQrRJFgsQWuuNQFNWhw0DDpvufV0CLAFubNHKNdakSTjpfG6LOMCyFbYE3hjC/q/3k3cqr8mnfCDqAZ4e+TTv73ifv637GxW64QvwhBCiNVh7DOJSpdRupdQapVSo6bXewHGzMsmm12qllJqhlNqulNqelpZmmVqGh0NgIPcUzSc/H9KDhlNeWs76p9c367QvjHmBmVEzeW3La0xdPpXC0g57m28hRDtkzQCxA+hrupXpW8DXTTmJ1nqB1jpKax3l6enZohWsYupmunTHPAL7lfPZGjcufuxidn24i5QdTc/WqpRi3nXz+PdV/+bLfV9yxcdXyOwmIUSbYbUAYbp9aZ7p+WrAXinlAZwAfMyK9jG9Zl233oqqKOee8B388gv43jEKJw8nvv+/75s1hqCU4onhT7Ds1mXsOrWLSxZeQnxafAtWXAghmsZqAUIp5a2UUqbnw0x1yQD+AAKVUv5KqU7AFOAba9WzSmQk9OvHnZlvohQs+cqBMS+MIWljEvErmv+BPjFkIr/c8wsFpQVc+sGlbEjY0AKVFkKIprPkNNdoYAswQCmVrJS6Tyn1oFLqQVORSUCsUmo38D9gijaUAY8A3wPxwJda68avTGtpSsGtt+Kz+QvGjipl8WIIu3MIXoO8WPfXdU1aF1HTsN7D2DZ9G3269OHqT6/m/Zj3ZYaTEMJqVEf6AIqKitLbt2+33AW2b4eLLmLt/33Ptf8dx9y5MOXio3xy5SeMfWUsI54c0SKXyS7KZvKyyfxw5Acmh05m/vXz6e7YvUXOLYQQ5pRSMVrrqNr2WXsWU/sydCj4+XHN/jeYOBFeeAG0Xz8GTBjAprmbmjXt1VxXh66snraal8e+zIr4FYTPD+eXxF9a5NxCCNFQEiAawzSbiR9/5M3nzmBnB488Alf++yrKisrY8M+WGzewtbFl9ojZbLlvC452joz5aAx/X/93SstLW+waQghRHwkQjTVpEpSW0jvmG158EdauhQ273Bn252Hs/GAnp3adatHLRfWKYscDO7hv8H28/OvLDF80XLLBCiFahQSIxho2DHx8YOlSHn4YhgyBWbMg8tHLcXRzZO2stS0+sOzSyYX3J7zP8snLOZp1lIj5Eby86WVKykta9DpCCGFOAkRjKQWTJ8PatdgdT2D+fDh1Cub+xzTt9Zck/pj3h0UufUvwLex5cA/XBl7L3zf8nYj5ETIdVghhMRIgmuL//g/s7GDOHC66CB5+GN5+GxgaRdANQXz/f9+TtCnJIpfu3aU3yycvZ/W01ZSUlzD247FMWz6NlNymr+gWQojaSIBoit694dFH4dNPYe9eXnwRevSAB2cqJiy+mW7+3Vh661JyTuRYrArXBl5L7MxY5lw+hxXxKxjw9gDe3PqmDGILIVqMBIimevJJ6NIFnn6arl3hjTdgxw5Y9JkDt311GyV5JSydtJSyYsvd78HR3pFnRz9L7EOxDPcZzqzvZxHyTghfxH4h2WGFEM0mAaKp3NyMIPHtt/Dbb0yeDFdfDU8/DbkOXty0+CaStyaz9rG1Fq9Kf7f+rLl9Dd9O/RZHO0emLJ9C1IIovj/cvDxRQogLmwSI5nj0UfD2htmzUWjefRfs7eGGG6DXlSFc9uRlxLwXw44Pdli8KkopxgeNZ+cDO/nk5k/IKsrims+u4YqPr2Br8laLX18I0fFIgGgOZ2d45hn49VdYswZ/f1ixAg4fNiY6jXruCvpd1Y/VD63mxO+tk5DW1saWO8Lv4MAjB3jr2reIS4vj0g8u5brPrmNT0qZWqYMQomOQXEzNVVoKwcFGsNi5E2xs+PBDuPdemDkT/v18AQsvep+Ksgru/+N+XLxdWrV6eSV5vLXtLf679b+kFaRxmc9lzB4xm+sDr8eUTFcIcQGTXEyWZG8PL74Ie/ZAdDQAf/oT/O1v8O678MHnTkxeMZnCzEIWj15MTrLlZjbVxqWTC0+NfIrEWYm8de1bHM85zg3RNxAxP4LP935OWYXlBtGFEO2btCBaQkWFkcgvOxv274dOnaiogIkT4ZtvjHHsUNckPr/+c5zcnbhr/V1072ed7Kyl5aUsiV3CK7+9QlxaHL5dfZkZNZPpQ6bj4eRhlToJIaxHWhCWZmMDL78MCQnw/vtVL336KUREwJQpkNOtL3dvuJvinGI+HPkhafEWun/2edjb2nNnxJ3snbmXlVNWEugWyFPrn6LP633408o/EXMyxir1EkK0PdKCaClaw5gxEB8Phw4ZaySAEyeM9E329rB1K6i0VD656hN0uebOdXfiHeltnfqaiUuLY97v8/ho90fkl+ZzSZ9LeCjqISaFTMLR3tHa1RNCWJC0IFqDUvCvf0F6Ojz4oBEwMBZdf/MNpKXB5ZdDSbce/Gnjn7BztOOjMR+RvDXZyhWHEM8Q5l0/jxOPn+DNa94kszCTu76+i57/6cnM72byx4k/ZD2FEBcgaUG0tBdfhH/+ExYuhPvuq3r5t9/guuuge3dYvx7c7c7w8diPyTuVx20rbiNgXIAVK11dha5gY9JGFu1cxLK4ZRSWFRLmFca9kfdyR/gdeDp7WruKQogWUl8LwmIBQim1CBgPnNZah9Wy/3bgSUABucBMrfVu075E02vlQFldla+pTQSI8nJjSfXmzfDHHxAaWrUrJsbY1akT/Pgj+HTP5dNxn3J632kun3M5o/4xChvbttWoyy7K5ot9X/DBzg/4/cTv2NnYcW3/a7kz/E7GB42XLigh2gCtdZOnrVsrQIwC8oCP6wgQw4F4rXWWUupa4Fmt9cWmfYlAlNY6vTHXbBMBAoz83xER4OkJv/8OTk5Vu/btgyuvhLIy+P57CAsqYdVDq9jzyR76XdmPmz+9GZcerbtWoqFiT8fy0a6P+Dz2c07mnqRL5y5MCp7EnRF3MqrvKGxU2wpuQnQ0haWFxKYc5Ke9+9l2NJ74tP2cKI6nnHLyXo1t0jmtEiBMF/YDvqstQNQo1x2I1Vr3Nv2cSHsOEAA//GA0F6ZPr5rZVOnwYRg71pgVu3o1XHqpZueinax5ZA0O3RyYGD0Rv9F+1ql3A5RXlPNz4s98sucTlscvJ68kD58uPtw7+F6mD5lOny59rF1FIdq1Cl1B4plEfj20hx9jd7PjxB6SinaTZ38UlOkzWys4449LYTA+DqHE/vcVbGwa34poDwHiCWCg1nq66ecEIAvQwHta6wX1HDsDmAHg6+s7NCnJMvdhaJK//92Y/vr55zB1arVdx44ZLYmTJ430HOPGQeqeVJbeupTMw5mMfn40I58aiWrCP3hrKigt4JsD37B412J+OPJDVU6oB4Y+wNUBV2NrY2vtKgrRZmmtSc5JZu+pODYdiOOPxDgOZO3jVHksZba5pkIKMgJxygvHp3MoIZ7BXBwQzNjBgYQHO9KpU/Pq0KYDhFJqDPAOMEJrnWF6rbfW+oRSygtYB/xZa73xfNdrUy0IMPqRLr/cWGW9cyf0719t96lTRiMjNhbmzjWSw5bkFfPdA98RGx2L/xX+jF8wHrcANyu9gcZJyErg/R3v88HODzidf5q+Xfsyfch07o64G5+uPtaunhBWo7XmZO5JYk/Hsi1hH78diiU+fR+nyuMptck9WzDfE9KC6VYSTmCXcC7xi+DqIaFcOtQZNwt9DLTZAKGUCge+Aq7VWh+so8yzQJ7W+rXzXa/NBQgwmgqRkeDvbwxcd+5cbXdeHtx/PyxZAjfdBIsXQ5cumh0Ld/DDX36gorSCUc+MYvhfhmPbqX18Gy8pL2Hl/pW8F/Me6xPWAzDSdyRTw6YyKWSSzIISHZbWmhO5J4hLi2Pf6Ti2HY1j98k4EvJjKVbZZwvmeUFaKM4FofR1DGWQdwiXDQhmeIQnwcHVhi0trk0GCKWUL7ABuEtrvdnsdWfARmuda3q+Dnhea33eGyu0yQABsHKl8el/223w2WdgW/2DXmv43//giSegXz+jyyk0FHJO5LD2sbXEL4/HM9ST8e+Nx/cyXyu9iaY5nHmY6L3RRMdGE58ej62y5aqAq5gaNpUbB9xIV4eu1q6iEI1WVlHG0ayj7E/fT3xaPLuS97PrRDwJeXEUY9YiKHCDtFA4HYa3bShhXmGMHBDKyKEeREYa096tzVqzmKKB0YAHkArMAewBtNbzlVILgYlA5aBBmdY6SinVD6NVAWAHfK61ntuQa7bZAAHw2mvw178amfwWLjRycdSwaZORJjwnBz74wEjRAXDg2wOsfng1OcdzGPrAUMa+PBbH7u1reqnWmj2pe1gSu4To2GiSspOwt7FnjP8YJgRNYMKACdINJdqcnOIcDqQfYH/6fg5kGI+xp/Zz5MwhynTJ2YK53pAeDGkhuBaFENgthKG+IVwU4smgQYrw8NZtFTSG1VoQra1NBwiAZ5+F556Dhx+Gt94yVl/XkJJiBIlff4VHHoFXXzX+Y5XklfDTnJ/Y9sY2nDycuPzZyxkyfQi29u2j28mc1pqtyVv5av9XrDywkoMZRu/iYO/B3DjgRsYHjWdwz8EybVa0Cq01KXkpxKfFE58eb/a4n1P5KVXllLbFJrsf5akDIS0YMgbS1ymYYf0GcklkNyIjYdAgY3Z7eyIBoq3Q2sgD/tprxuMrr9QaJEpLjQHr//4XgoKMcYlLLzX2pexIYe2stRzbdAz3IHfGvjyWgTcPbNf3dtifvp9vDnzDygMr2XJ8CxqNu6M7Y/zHMNZ/LFf2u5KA7gHt+j0K69Jak1mYyeHMwxzOPMyRrCMczjzMocxDxKfFk118dnzAvrwLtlnBFCUPhHRj65w3gNCeAUSEdSIy0kjeHBEBLm1zyVKjSIBoS7Q2WhDvvgvPP2+k5ajDhg3GjYeOHzfGJ557DhwcjP/sB787yI9P/kh6fDp9Lu3DVf+6Ct8R7Wt8ojapeamsO7qOH4/+yPqE9STnGLmqfLv6MtZ/LCN9RzLCdwT93fpLwBDnyC3O5VDmIQ5mHKy2Hco8xJmiM1XlFApX7UOn3P4UJQ8kLzHYaBWkB+Pr1pMhgxUREUaLIDzcGBu0bX+N9QaRANHWVFQYYxEffwz/+Q88/nidRXNyjKGLBQsgJAQ++giiTP+UFWUV7Fq8i5+e+Ym8lDwG3DiAMc+PoUd4j1Z6I5alteZQ5qGqYPFTwk9kFWUB4OXsxQjfEYzwGcEI3xFEeEfQybaZE8JFm1daXkpSdhIJWQkknEkg8UwiCWcSqn4+nX+6qqxC4dXZly6lQZAZSN6x/qTt70/Z6f5wxh+bCgeCgmDwYGMbMsR4tNR00rZKAkRbVFYG06bB0qVGX9KsWfUW//57I/ffqVMwezY8/TQ4msapS/JL2PrGVjb/azPFOcUMvGkgo/45ip5DerbCG2k9FbqC/en7+fXYr1VbwpkEAOxt7AnvEc7QnkMZ2msoUb2iCPMKk6DRzmitSS9I51j2MZKykziSeYQjWaYt8wjHso9RrsurytvZ2OHb1ZdeTv64lvqjs/qRmzCAYzuDOL47AMqMP5LevY2WQFiY0SoYNAgGDjRa5Bc6CRBtVUmJMVXpq6/gz3+G118HO7s6i585Y8SRjz6Cvn2N4jfffHYYozCrkG3/28a2N7ZRdKaIoPFBjPrnKHoP691Kb6j1ncw9yebjm9l+cjvbT24nJiWmqiuhk20nQjxDGOQ1iDCvMAZ5DWJQj0H0du0t3VNWUlBaQHJOMsezj3M853jV86TsJJKykziWfYyC0oJqx7g5uhHQPYAAtwD6OAVgmxNAUUo/Mg77k7inN/tibcnKOlve19cYI6jchgwBL69WfqPtiASItqy83Biwfv11Ix/4kiXg6lrvIb/8YsSTvXuNnE5vvlktaSxF2UX8/vbvbH19K4WZhfS/pj+XPXkZfS/v2+E/GLXWHM06SkxKDNtPbmdP6h5iT8dyIvdEVZluDt0I9ggmwC3A+ODpHkB/t/4EuAXg6eTZ4X9HLU1rTW5JLqfyTpGSm2I85qUYz/ON11LyUjiZe5LMwsxzjvd08qRvt7707doX366+9O3al77d+uJa4Ut2Yj8O7unGzp2wY4eRx6xS165GiyA01HisbB14yJ1zG0UCRHswf74xrzU01LiJtW/9A85lZfDee8YYd06OETDmzIFu3c6WKc4t5o93/mDLa1soSC/AO9Kbi2ddTNiUMOw6191S6YgyCzOJPR1L7OlY9qbu5UDGAY5kHeF49nE0Z/8GnO2da/2w8uniQw+XHng6edLNoVuHDyIFpQVkFmaSUZBBWkEaaflp1R8L0kjNSyUlzwgINb/1g9Ht5+3iTU/XnvR0MTafrj74dPGpeuzdpTelhQ7s22eknDHfUlPPnsvfv/o4QUQE9OpV6yRA0UgSINqLH36AW281Fj58++3Z0eh6pKfDP/5hDGJ7eMBTTxk3tHM0W0dXWljK3s/2svWNraTtS8PZy5moh6KIejCqzaYWby3FZcUknEmo6us+mnXU6O44Y3R51PaN197GHg8nDzydPfF08sTdyZ3uDt1xc3Q7++jYHddOrjh3csbJ3qna5mDngL2NPXY2di0WaLTWlFWUUVJeQmlFKfkl+eSW5JJbnHvOY05xDtnF2eQU51Rt2cXZZBRkkFGYQWZhJkVlRbVeR6Fwd3LH08mTHi496OnS0wgClY+uZ392c3Q75/2lpBhpySq3XbvgyJGz+52cqrcIIiONgNAWVhx3VBIg2pN9+2D8eOPr08cfw6RJDTps505jKuyGDeDtbQxkz5hRPVBorUlYn8DWN7ZyaNUhbDvZEnpbKEMfGIrPcJ8O/624KfJK8jiWfYzj2cc5nX+62rfoyp+zCrPILMwkszCz2gBqQ9jb2GNva19vwKj8G9Xoc56bB4XGsFW2dHXoSpfOXejSuQtdO3fF3ckdNwc349HRDXdH49HT2dMIiE6euDm6NShDb24uxMWdbQ3s22fkrDRvFQQEGAEgMvLsALKfX61JBoQFSYBob06fhhtvhK1bjebAf/7T4HX6GzcaXU0//ww9exotivvvP3e2RvqBdH5/63d2f7ybktwSvMK8GDJjCBF3RuDQTaZ2NIXWmrySPDILM8kqyiKvJI+C0gLyS/IpKC2o2orKiiitKKW0vLTqw720vJSyirI6z10ZOBSq2nN7W3s62XbC3sZ47GTbCXtbe5zsnXDt5IprZ1dcO7ni0sml6nlXh6442jm2yBeCigpISDBaApXb3r1gnnXf0dGYoh0WdnZKaUSEMYYgrE8CRHtUUmL0Hf373xAcbNxTIjKywYf//LMRKDZuNKb4PfaYMU225hzvkrwSYpfEErMghpN/nMTO0Y6w28IYcv8Q+lzaR1oVAjDWd6amQny80TKIi4Pdu41WQa4pN52trTF1tHIaaWVXkZ9fx11k1hFIgGjPfvwR7roLMjKM1ByPPdbgNrjWRpfTCy8YM58cHeH2240B7fDwc8un7EghZkEMez/bS0leCd0DuhN+RziDbh+Ee6B7C78x0VZlZRmtgD17jK0yIJhPJe3Sxfg/VNlFFBFhBATH9pVDUiABov1LTzduXbpypXGHocWLjYGGRti9G95+28g2XlgIo0YZgeKmm85delGcW0z8inj2frqXo+uPgobeF/cm/I5wQm8LxdnTueXem7CavDw4cAD276/eIjh+/GwZNzfjgz8kpPrWs6fMIOooJEB0BFob81off9wYj3jtNbj77kb/lWZmwqJFMG8eJCYaC4huu81Y1H3xxeeeLudEDrHRsez5ZA+pe1JRtgq/0X4E3xLMwJsH4tqz/jUbwvrS041B4n37jC6i/fuNLTn5bBk7O6N7KDzc2CIijEcJBB2fBIiOJD7eaE1s3mw0A955p/oquQYqL4fVq42JUt9+C8XFRkKyadOMbqiBA889JnVPKrFLYolfEU/GgQxQ4HOpDwNvGUjwLcF095e5iNZSXm7cvPDQIWPbv/9sUDh9Nj0Rrq7Gv23NrX9/mn1vY9E+SYDoaCoq4MMPjRXYOTnG/NZ//rPJdyTJzjayfXz+Oaxfb5x+0CC4/nq49loj1bi9/dnyWmvS49OJWx7H/hX7ObXrFAA9wnsQNCGIARMG0GtoL5SNfPVsSVob6wgOHDC2gwfPBoSjR4008ZVcXIzvDTW33r2lRSCqkwDRUaWlGUFi8WIjOdNbbxlrKJrxCZCSAl98AV9/Db/9ZqzY7tIFrrrKyARy9dXGh4y5rKNZxH8Vz8FvD3Js0zF0hcalpwtBNxjBwv8Kf+wd7Wu/oKimogJOnDA+8BMSjMcjR84GhFyzu1k6OkJgoPHtv+ajrDIWDSUBoqPbuBFmzjRGGq+4Al5+GYYNa/Zps7ONFsXq1bBmDZw8abzu5wfDh5/dBg06O9BdkFHA4TWHOfDNAQ6vOUxJXgl2Dnb4jfYj4JoAAq8NxC3w3BW2F5L8fOPD/8gRY6sMAkePGuNCJWZ3srSxMbKuBAXBgAHVt969ZVGZaD6rBQil1CJgPHBaax1Wy34FvAlcBxQA92itd5j23Q38w1T0Ra31R+e73gUbIMD4VHn3XZg712hZ3HKL8by2wYQm0NqY4fLjj7BlizEEkmK6G6OzsxGPKhdAhYcbSzdsKSPplyQOrTnE4TWHjXELoHu/7gRcE0D/q/vjN9qPzl06t0gd24KCAmPw9/hxoyWQnHzulpZW/ZiuXY1Vxf36nX309zcefX2rd++JC1xlE/PgwbN9jAcPGk39tWubdEprBohRQB7wcR0B4jrgzxgB4mLgTa31xUopN2A7EAVoIAYYqrXOqnkOcxd0gKiUm2tkhn3tNePT6k9/MlbM+fi06GW0NgZFN282tq1bjZQKRaYUPpWzYiIijG+7/fqBt0MW6vBhUn87TML6BEoLSlG2ij4X98H/Sn/6XdmPPhf3wbZT21tVlZ9vLBQ7dercx8qAkJxszBKryd0d+vQ5u/n4GIGgcrvQblAj6qG1seYpIcFoTiYmGsvSK58fPWrMU6/k6Gg0L0ND4dNPm9SvaNUuJqWUH/BdHQHiPeBnrXW06ecDwOjKTWv9QG3l6iIBwkxaGrz0kjHLSSlj5tNf/mJ8NbWQsjLjC03lAqva5tWD0eLo71fGAOdkehYcxTX9KHapJ0FrbBzs6R7Rlx4X98V3ZF/6Du9FNzdbOnduep+61kYDq6DA+KDPzzfWAGRnG2P85o9ZWcavruZm/jdZSamzH/4+PtUDQOVj796yeEzUYB4EKmcZmLcGsrOrl+/e3Rhj9PMzvmkFBRlbYKAx2NTMfsa2HCC+A17RWv9q+nk98CRGgHDQWr9oev2fQKHW+rVazjEDmAHg6+s7NMk8CYwwvn288IIxn7WiAiZPNga2G5G2o7mKis4OuFZuR44YYxqVH8C6sAg/EunHEfxJxJN0AEqw5zg+HLfx5bSjHxkOvVH2dtjaGq0UOzsjjUNFhbGVl1ffioqMgFDewBx6Tk7g6Vn71qOHsT6x8tHTs977O4kLWUGBcfOKQ4eM/+w1WwIFZunRlTo70BQYaGz9+hkBoW9fiyetqi9AtPv/3lrrBcACMFoQVq5O29O3LyxcCM89B2+8Ydx3Ijoaxo2DJ5+EMWMsPt3FwcEYkwgOrrtMfr4DaWkDSUsbSGYmnDmRT/qOJHL3JtH/YBL9Tv2MygddaEuJZy+K3H0p8OhLgbsPJTYO2NgYgaLysXJzcDBaLE5OxmPlcxcX4++ucuvSxdhkLYBokIoKo4+x8kM/Kcn45lPZEjhxonr57t2ND/ygIONvr/LDPzDQ6Gdso/c+tXaAOAGYd473Mb12AqMVYf76z61Wq46od28j8d/TTxuD2W++adyOLiLCyBh7++3nvZOdJVV+ePv5Vb0C94YAIQAUZhZy7NdjxrbpGCe3b6Fr2W+goMegHvS+uDe9onrR66JeeIV5YWvf9sYxRDtTXGw0fStbAocPG1tCgjEAV1xcvby7u/GBf8UVZ1sClfOO22nqWmt3MV0PPMLZQer/aa2HmQapY4AhpqI7MAapaxkCPEvGIBqhqMjodnrnHWOwwNnZCBIPPGDctquNKy0oJXlbMsc2GUHj5PaTFGUZI+S2nW3xjvCm10W96DmkJ96DvfEK9WqTg9/CikpLjQ96864f8+fHjxvjBZW6djU+7M27fyof+/a16hes5rDmLKZojJaAB5AKzAHs6w6hXwAADwFJREFUAbTW803TXN8GrsGY5vonrfV207H3An83nWqu1vrD811PAkQTaA2//27keVqyxBiNjYoybiIxaVK7mWKjtSbraBYnt580tj9OkhKTQkmesajAxt4Gr1AvvAd7G1ukNz0G9ZB7X3RkWhvTyio/9CsXnlRux45VH5yysTFa2pUf+v37V9/c3Drk6kNZKCca5swZY6rce+8Zc1bt7Y2l01OnwoQJRsd9O6IrNJmHM0nZmcKpXac4tdPY8k/nV5Xp2rcrPcJ7nN0ieuDW3w0bW1mB1i5kZVWf/ZCQYASEypZBfn718u7utS868fMzpp5dgItOJECIxtHauIdpdLTRqkhONuZqTphgpH4dN87okmqHtNbkpeRxavcpUvekkro7ldQ9qaTvT0eXG38Ldg52eIV54TXIq1rwcPJoWq4r0URaG1Pcjh2rvlW2CI4eNb7UmHN3P9vlU3MLCGi3YwGWJAFCNF1FhZGUKToavvzSmL/t4GAMxN1wg5HRr4UX4VlDWVEZafFpRsDYm8rpPadJ3ZtKfurZb6BOHk54BHvgEeyBZ7Cn8XygB136dJEWR1NUVJwNAImJZxeHmS8Sq1x5WcnJyZgSWrnU3Hzz///27jY2sqs84Pj/mRfPi+fVdnbXya69TkhEUqALClFa+EAiWqUFlUqtyqZUQhESKqI0rfpC2g99QUVq+VBoWj40bSmRSkkRbWiIECVKNi0CGrJbstmEJS14d7PZeLO212OPPZ73px/OGc+MPbtmNzOe9czzk47Ovcfj8T3ZyTz3nHvvc2Z27XWAfrIAYbqjUnFL0z3+uMsRPjvr2g8dckkC3/1uuPNOiAxO6ozV11a5cMIFi4WTCyycXGD+5Dzri80n5wLhAOmpNJmDmWaZcXX2xiyJfYnhzD21uurO9htTPi+/7C78Nsorr7QnngJ3O+jMTHPaZ3raBYSpKbedzQ7kdYB+sgBhuk/VLTrw1a+6gPGtb7kzwmjUZfC76y5X3v72gXy4YG1+zQWMHyywdGqJ5dPL5E7nWDq11DbqAAjFQhvBIjPj6rGbxsjemCV7Y5ZwfBfOe6+tNZNLbU46dfasCwpLmzLjBIPuIvCBA1tLIyDYFNCOswBhei+Xc1lljxxx5fhx1x6PuyBxxx3NcuDAQJ8FVgoVcmdyLmDMLrE0u0TuVHO7nG8/a07sS5C9KUt2JktiMkFiMkFyMrlRJ69PMpLYoSCr6v4t5+bco+6vvtp+1t8om+f+wd3l08g30pj3b5z5T0255emCdqvxtcYChNl5i4tuOurpp+GZZ+C555rTCXv2NNO/vulNLtHYLbcMxR0kqsr64vpGsLj4o4sugMy6gJKfy1Mrbc0LEs1ESU+nyUxnSE2lyExnSE+nSR9Ik9qfIrEvQSB0mesgqi7h1Pnz7kv/3LlmAGgtc3Nb5/0BJiaaCaYapTUD4fXXX/WCVaa/LECY/iuV4MQJ98xFo7z0kpuWAhccbrnFBYxbb22/FXHPnoEecbRSVYpLRfJzeVbnVsnP5cm/mmf55WWWz7iSO5PbMgqRACTSQVJJJRUtkwwWSNaXSZXmSa6dJ5k7S7KySIRNc/7JpPtyn5x00z+Tk26/0TY56QKAZRwcWBYgzLWpWGxfPPmFF1x96lT7E6yjo827VG64of0LrLE9NrZ7py/K5WY62YsXO5fFRVcWFmBxkeJ8ntyKsEKKFVLkfe1KmrykKOnWaalwRIhnRohPxInvS7kyESc+EWd07yije9rLSGJkOC+wD5GBTtZndrFo1N0BtTmzbLHoLnJuXm7t1Cl3MXxxsfP7ZTLuPvixMVePj7sMfImEK6Ojze1YzF08Hxlxo5fGdijUebSi6p66rVZdqVSa26WSewK9WGwvhYK7k2dzyeebOcZXVjpP6bRKJFxfJiY2HvSKjo+zb3ycfXv3uhFWa51KgQjl1TIr51bIv5onfy7PyrkVVs+vsr64TmGhQGGhwOLsWQoLhS0jkoZgJEg0EyWajhJJR1ydihBJR4iNxYhPxImNuzo+7gJNJB0hHAsTjocJRoIWYHYxG0GY3adYbM6lN8riYvuZdmM7n3dfytt9CfdCINAMSJtLI4VsayrZdNoFt9aSyezIXWDVUpXCfIG1C2tbSjFXpLRSorRcorhc3KjXL65TXa9e/o0FwrEwoViIkcTIRrCJZlyJZCLEsjFG9466C/P7EiT2JRjdO2rrmO8QG0GYwRKNulsim6lft1etNs/o83kXMCoVN73TKJWKK5fSWIBic4lGt5ZIxNW75Ow5FAmR2p8itT91Rb9XKVQoLBbaRiWlfIlKoUJ1vUqlUHFlvUI5X3bBJVckdyZH8XjRBZ/lUsf3jqQiRDNRRhIjriRdHUlGCMVCBCNBQpHOdXDEb48EN9pD0RChmK99CcfCRFIRQlH7KuzE/quY4RAKNc/UTdeE42HScXc31dWqV+usza+xen6V1blVV/tSzpcp5UuUV8uU82UK8y4AVderVEtVaqUa1VJ1I03K1QqOBDemzhrTaIFwAAnIRgkE3X4gHHDBJ9oSeHzwiSQjjCRH2upQLES9WqdeqVMr16hVatQrderV+sZ7haLtAQ6Fes29prWEIqHm6Csd6fkT/BYgjDF9FQgF3PMek0l469W9R71W3wgWtVKNWtlvl2uuvegCSrVYdcGl6EqlUKGUb06flVfKbhptpeQCT11dqbm68aXd+P2N9y661+60xigrPZXmvm/e1/X3twBhjNn1AsEAgXigr0+lV0vV5oinpa4WqwTCAYLh4EYdHAkiQaFeqTdHQsXmqAhxgTMQCri+hQJIUKiVahRzxS0lEO7NSMIChDHGdEEoEiIUCQ1U1l9LQWmMMaYjCxDGGGM6sgBhjDGmo54GCBG5R0ReEpEfisgDHX7+aRF5zpf/FZFcy89qLT97rJfHaYwxZqueXaQWkSDwWeBngFeAZ0XkMVX9fuM1qvrbLa//GO03ua2r6qYcDMYYY3ZKL0cQdwA/VNVZVS0DjwDvu8zr7wW+2MPjMcYYcwV6GSBuAM627L/i27YQkWlgBniqpTkqIkdF5L9F5Bcv9UdE5MP+dUfn5+e7cdzGGGO4di5SHwa+rKqtK6VM+wRSvwp8RkRu6vSLqvqQqt6uqrdfd911O3GsxhgzFHr5oNw54EDL/n7f1slh4KOtDap6ztezIvI07vrEjy73B48dO7YgIme2Oa4JYGGb1wwi6/dwsX4Pl9fT7+lL/aCXAeJZ4GYRmcEFhsO40UAbEXkjkAW+09KWBQqqWhKRCeAdwKe2+4Oquu0QQkSOXiq17SCzfg8X6/dw6VW/exYgVLUqIr8B/AcQBD6nqi+KyCeAo6rauHX1MPCIti9McSvwtyJSx02D/Xnr3U/GGGN6r6e5mFT1a8DXNrX90ab9P+nwe98G3tzLYzPGGHN518pF6p30UL8PoE+s38PF+j1cetLvgVpy1BhjTPcM4wjCGGPMj8EChDHGmI6GJkBslzhwkIjI50Tkgoi80NI2JiJPiMj/+Trbz2PsNhE5ICJHROT7IvKiiNzv2we931ER+a6IHPf9/lPfPiMiz/jP+7+IyEi/j7UXRCQoIt8Tkcf9/rD0+7SInPDJTI/6tq5/1ociQLQkDvw54DbgXhG5rb9H1VOfB+7Z1PYA8KSq3gw86fcHSRX4HVW9DbgT+Kj/Nx70fpeAu1X1J4FDwD0icifwF8CnVfUNwBLwoT4eYy/dD5xs2R+WfgPcpaqHWp5/6PpnfSgCBFeeOHBXU9X/Ai5uan4f8LDffhi4ZH6r3UhV51T1f/x2HvelcQOD329V1VW/G/ZFgbuBL/v2ges3gIjsB94D/L3fF4ag35fR9c/6sASIHztx4ADbq6pzfvs8sLefB9NLInIQl5rlGYag336a5TngAvAELiVNTlWr/iWD+nn/DPD7QN3vjzMc/QZ3EvANETkmIh/2bV3/rPf0QTlzbVJVFZGBvL9ZRBLAvwK/paor7qTSGdR++ySXh0QkAzwKvLHPh9RzIvJe4IKqHhORd/X7ePrgnap6TkT2AE+IyA9af9itz/qwjCCuJHHgoHpNRCYBfH2hz8fTdSISxgWHL6jqv/nmge93g6rmgCPATwEZEWmcAA7i5/0dwC+IyGnclPHdwF8x+P0G2pKZXsCdFNxBDz7rwxIgNhIH+rsaDgPDtozpY8AH/fYHgX/v47F0nZ9//gfgpKr+ZcuPBr3f1/mRAyISw63geBIXKH7Zv2zg+q2qf6Cq+1X1IO7/56dU9QMMeL8BRGRURJKNbeBngRfowWd9aJ6kFpGfx81ZNhIHfrLPh9QzIvJF4F24FMCvAX8MfAX4EjAFnAF+RVU3X8jetUTkncA3gRM056T/EHcdYpD7/RbcBckg7oTvS6r6CRG5EXdmPQZ8D/g1VS3170h7x08x/a6qvncY+u37+KjfDQH/rKqfFJFxuvxZH5oAYYwx5soMyxSTMcaYK2QBwhhjTEcWIIwxxnRkAcIYY0xHFiCMMcZ0ZAHCmG2ISM1nzWyUriX8E5GDrVl3jbmWWKoNY7a3rqqH+n0Qxuw0G0EYc5V8Tv5P+bz83xWRN/j2gyLylIg8LyJPisiUb98rIo/6tRuOi8hP+7cKisjf+fUcvuGfiEZEftOvb/G8iDzSp26aIWYBwpjtxTZNMb2/5WfLqvpm4G9wT+oD/DXwsKq+BfgC8KBvfxD4T792w9uAF337zcBnVfUngBzwS779AeCt/n1+vVedM+ZS7ElqY7YhIquqmujQfhq3WM+sTxR4XlXHRWQBmFTVim+fU9UJEZkH9remfvCpyZ/wi7wgIh8Hwqr6ZyLydWAVlyblKy3rPhizI2wEYczro5fYvhKtuYJqNK8Nvge3EuLbgGdbspQasyMsQBjz+ry/pf6O3/42LsMowAdwSQTBLQP5EdhY5Cd9qTcVkQBwQFWPAB8H0sCWUYwxvWRnJMZsL+ZXbGv4uqo2bnXNisjzuFHAvb7tY8A/isjvAfPAfb79fuAhEfkQbqTwEWCOzoLAP/kgIsCDfr0HY3aMXYMw5ir5axC3q+pCv4/FmF6wKSZjjDEd2QjCGGNMRzaCMMYY05EFCGOMMR1ZgDDGGNORBQhjjDEdWYAwxhjT0f8D02pTQaPZRTQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wrqxFW-SvncV",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 332
        },
        "outputId": "73edd6e1-d05c-4063-c277-02c3b2246511"
      },
      "source": [
        "# Plot validation accuracy curves\n",
        "epochs = range(1, len(hista['test_acc']) + 1)\n",
        "\n",
        "plt.plot(epochs, hista['test_acc'], 'r-', label='Model A');\n",
        "plt.plot(epochs, histb['test_acc'], 'b-', label='Model B');\n",
        "plt.plot(epochs, histc['test_acc'], 'g-', label='Model C');\n",
        "plt.plot(epochs, histf['test_acc'], '-', color='purple', label='Model A with regularizer (L2 & DO)');\n",
        "\n",
        "plt.title('Test Accuracy');\n",
        "plt.xlabel('Epochs');\n",
        "plt.ylabel('Accuracy');\n",
        "plt.legend();\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1b308a90>]"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1b2ff0d0>]"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1b2ff5d0>]"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1b2ffc90>]"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Test Accuracy')"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 0, 'Epochs')"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Accuracy')"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f1f1b2818d0>"
            ]
          },
          "metadata": {},
          "execution_count": 22
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hUZdr48e+dnkACIYWQ0EIN0hFwRV2kI6sCwgrorrhYf1jXte7rurpre33Xta3dVUERVNxFlCpVRYSEIgokEEgoKaQR0vvz++NMkklIIAmZTMr9ua65MnPmzDnPBHLu87T7EWMMSimlVHUuzi6AUkqp5kkDhFJKqRppgFBKKVUjDRBKKaVqpAFCKaVUjTRAKKWUqpEGCKWUUjXSAKFaPBHJsXuUiUi+3esbG3C8LSJyax32a287x5qGlVyp5s3N2QVQ6kIZY9qXPxeReOBWY8yGJjj1LKAQmCQiIcaY5CY4JwAi4maMKWmq86m2SWsQqtUSERcReVREjohIuoh8JiKdbO95icjHtu2ZIhIpIp1F5BngCuBfttrBv85xivnAW8A+4HfVzn25iPxgO/YJEbnZtt1bRF4UkWMickZEvrdtu1JETlY7RryITLQ9f1JEltvKnAXcLCKjRWS77RxJIvIvEfGw+/xAEflGRDJE5JSI/FlEQkQkT0QC7PYbISKpIuJ+Ib9v1fpogFCt2T3ADGAsEAqcBl63vTcf6AB0AwKAO4F8Y8z/AN8Bdxtj2htj7q7pwCLSA7gSWGJ73FTtvTXAa0AQMAzYa3v7H8DFwBigE/AwUFbH7zMdWA50tJ2zFPgjEAhcCkwAFtrK4AtsANbavnsfYKOtlrMFuN7uuL8HlhljiutYDtVGaIBQrdmdwP8YY04aYwqBJ4HZIuIGFGMFhj7GmFJjzC5jTFY9jv17YJ8x5gCwDBgoIsNt790AbDDGLDXGFBtj0o0xe0XEBVgA3GeMSbCd9wdb2epiuzFmhTGmzBiTbyvzj8aYEmNMPPA2VjAEuBpINsa8aIwpMMZkG2N22N5bhK3GIyKuwDzgo3p8d9VGaIBQrVkP4L+2JphM4CDWXXdnrAviOmCZiCSKyAv1bGK5CesuHmNMArAVq1YCVq3kSA2fCQS8anmvLk7YvxCRfiLytYgk25qdnrWd41xlAPgSuEhEwoFJwBljzM4Glkm1YhogVGt2ArjKGNPR7uFlu3svNsY8ZYy5CKu552oqm4nOmeJYRMYAfYHHbBfnZOAS4AZb7eQE0LuGj6YBBbW8lwv42J3DFat5yl71cr0JRAN9jTF+wJ8BsfvuvWoqvzGmAPgMqxbxe7T2oGqhAUK1Zm8Bz9j6BBCRIBGZbns+TkQG2y7EWVhNTuV9Aaeo5eJqMx/4BrgIq39hGDAI8AauwqpZTBSR60XETUQCRGSYMaYMeB/4p4iEioiriFwqIp7AIcBLRH5jq8k8Dnie5/v52sqeIyIRwP+ze+9roIuI3C8iniLiKyKX2L2/GLgZuBYNEKoWGiBUa/YKsBJYLyLZwI9Yd/oAIVgdvllYTU9bqbxQvoLVV3FaRF61P6CIeGF18L5mjEm2e8TZPj/fGHMcmAb8CcjA6qAeajvEg8DPQKTtvf8FXIwxZ7A6mN8DErBqFFVGNdXgQaz+jmzgXeDT8jeMMdlYzUfXAMnAYWCc3fvbsALibmPMsfOcR7VRogsGKdU2icgm4BNjzHvOLotqnjRAKNUGicgorGaybrbahlJn0SYmpdoYEVmENUfifg0O6ly0BqGUUqpGDq1BiMhUEYkRkVgRebSG97uLyGYR2SMi+0Rkmt17j9k+FyMiUxxZTqWUUmdzWA3CNnzwENZIipNYozbm2Waelu/zDrDHGPOmiFwErDbG9LQ9XwqMxkoTsAHoZ4wpre18gYGBpmfPng75Lkop1Vrt2rUrzRhTfc4N4NhsrqOBWGPMUQARWYaVS+aA3T4G8LM97wAk2p5Px8oNUwjEiUis7XjbaztZz549iYqKatxvoJRSrZyI1DrM2ZFNTGFUTQ1w0rbN3pPA72xZLFdjJVer62cRkdtFJEpEolJTUxur3EoppXD+KKZ5wIfGmK5YE4s+siU0qxNjzDvGmJHGmJFBQTXWkJRSSjWQI5uYErAShpXrattm7xZgKoAxZrttlmpgHT+rlFLKgRxZg4gE+opIuG0Rk7lYaQ/sHcfKYY+IDMDKdJlq22+uLYdMOFZiNM02qZRSTchhNQhjTImI3I2VUtkVeN8Ys19E/gZEGWNWYuWqeVdE/ojVYX2zsYZV7ReRz7A6tEuAu841gkkppVTjazUT5UaOHGl0FJNSStWPiOwyxoys6T1nd1IrpZRqphzZSa2UUm2OMYac5BzSotNIi06jrKSMvtP60ql3J2cXrd40QCjVkhgDx45BZCTs2wdubuDraz38/Cp/BgVBWBi0b+/sErd6hdmF/LL0F45/f7wiKBRlF1XZZ+29awkeFEz/Gf2JmBFBlxFdEJFajlh3ZSVlnI47TXFuMSHDQi74eNVpH4RqW3JyICYGjh+HhATrkZhY+dwY6N8fIiKqPvz9nVPezEzYtg127rSCQmQkpKVZ77m4QFnZuT/v62sFirAwCA2t+WdICLjXZzlum8JCSEqq+jtMTLQCVPnvrU8f8PKq/7GdJC89j/gt8STtSiJ4UDA9x/XEt4tvjfumHkgl8s1Iflr0E0XZRfiG+hJ0URABEQEERgRaj/6BlBaVErMyhugV0Rz/7jimzODX1Y/wCeGYMkNhViFF2UUUZhdSmFVIcV4xPgE++Ib64htme4Raj/yMfNKi00iPTrd+Hk6nrLiMsEvCuPXHWxv0nc/VB6EBQrUuxsCZM9bF6uRJOHwYoqOtoBAdbW2z5+YGXbpUXkTLyqx9Dx+G4uLK/YKDqwaM8iDSowe4ujZe+fPyrICwcSNs2gS7dlllcnGBiy6CUaMqH0OGWOfOyYGsLMjOth5nzkBKytnBLyHBuqCXlFQ9pwgEBFSthZQ/9/aG3FzruNXPcfr02eX38IAiu7tnFxcID7d+V7171xygPDzgyBHr38f+ERcHpbUMXgwIqPlY4eHWv41vzRf16opyijj27THiNsURtzGO5J+SrfGUQsUK4EEXBdFzfE/Cx4fT/bLuxG+NJ+qNKOK3xOPq4crAOQMZtXAUYZeEnbdWkJeWx6FVh4hZEcPJH0/i5uWGh68Hnr6eePp54uHrgbu3O3npeWQnZpOdkE1uSm6VY7i4udCpTycC+lcGouBBwYSODK3Td65OA4RynrIySE2teodZ/vP06aoXnfLnvr4wciSMHm1dCEeOhA4dKo+Znw+HDlVeSA4fti785cfNy6taBj+/qrWC/v2hVy/rohIUZF3Eqispgfj4ynMcPGgFjpiYyjt4sO68O3Y8++Lq61v/u/ITJ2D7dusC6+YGv/oVjB8P48ZZv4PGaC4qK7PKX/3f4tSpyn8H+3+L/Hxo1+7sZixfX6vmYX9xDguzalp5eVX/fcp/f/Hx1jGrE7ECe7mwsMqA4uFR63cwCYlkHc8kLbmEtOIOpBFAUfky3t4+0MEP/DpABz9K3b0pLCijMK+UooIyCvPLKMovJT+nFFMGrh4udBseSPivuxE+uS9dLgsn5UBaReA4/t1xivMqbxg69OjAyP83kuELhtMuqN2F/7ucQ2lRKTnJOWQnZuPl74V/L39c3RvvpkQDhHKOr7+G+fMhI6PqdhHo3Lnmu1ZfX+sCFhkJsbGVn+nfH7p1s7YdO1Z5QRGx7uK7dat6oSp/3qePdSFrhPbeCmlplTWSw4etu+maAl1td7+16dQJrrwSJkyAyy9vkv6DgswC9i7aS+qBVLpf1p3wCeH4hfnVur8xhsz4TJJ2J3Hm+BmyE6y73OzEbLISsshOzMbTz7OyicXu0aF7ByQ3xwpI9sEpNxf69q0M3jXc/RfnFZO0J4nEyEQSoxJJPZBKekx6lYu2p687Pj4CxUVQVGzVAIuKwJThQhmeFOJBEZ4UVjz3IZceHKMbJ3CnhpqVTalxIYEwjtOdYN98+vQowqVr2Nm1l4gI6Nq15puOZkoDhGpaxsCLL8LDD8OwYbBgQdULd0iIdYd8PhkZEBVV2faemFh5IbFv4/b2dvx3amWS9yaz8/Wd/PLJLxTnFePR3oOiHKtpKKBfAOETwgkfH07I8BDSDqaRsDOh4uKcl1ZZQ3PzcqtoI/cL86N9l/YUZBZUdNYWnC6o2NfD14PQi0MJHWU9wkaHWUHDdiE2xpCfkV8RcDLjM0mMSiQxMpGU/SmYUuta5RvmS+fBnau29UcE0i643dlNPMZYtaPk5Jp/EaWlZzfRZWdb22rq3zHGukGwb7o7dapqDcjbu2qNtSEBo6ZaW/v2Vtmq1/4SEqy/rX/8o37nsNEAoZpOYSHccQcsWgS//S18+CH4+Di7VG1eWUkZhVmFHF59mMg3Ijm5/SRu3m4MvmEwoxaOImRYCKf2nbKaVDbFcWzrsYqAASAuQtDAIOvCPiqM0JGh+Pf2x6ujV63t7sYY8lLzSItJI+1gGsk/JZMYmcipn05RWmTVrnyCfPDv5U9uSi7ZidmUFlatdXl38q4MKKPCCB0VWmunsdOUlFgB6OjRs/tR4uOrBo/G5u1tBYfLL4cPPmjQITRAqKaRkgIzZ8IPP8Bf/wpPPNGiqtotWXZiNgmR1l1+0q4kcpJzKMy2jY7JKqSkoLL5pFPfToxaOIqh84fi7V9z7au0uJTEqERSfk4hcEAgXUZ0waNdDf0BDVBSWELKzylWeXcmcub4Gdp3aV85asdWG/Hr6odfN79GGQ7qNPn51t9FfZSVWf041Zssy/vn7Ju1OnS44OZTDRDK8fbtg2uusf4YFi2C6693dolatLLSsoqhj/ZDIO2f52fkc2rvKRJ2JpCdaHX+iqsQPDCYDt07VIyKsR8lEzw4mPBx4YhLC77oqkZ1rgChE+XUhcnIgNdfh//9X+tu5rvvrBE3qlamzJB1Mquinb78kXUyqyII2He+nkunvp3oOa5nRRNMyLAQ3H0aMKdBqRpogFANc/IkvPQSvP22NQrlN7+Bd96xqr5tRH5GPpnxmXh38qZ9l/a4eZ7951RWWkZ6TDoJOxMqmoBS96dWHX3TwZOgAUGEXhyKh591t+/h64Gnn2flc7tx8vbv13ROpRqL/u9S9RMdDS+8AB9/bLWVzp1rjVYaMsTZJXOYgjMFnNh24qw7/rzUqvMtfAJ9KtrQ23duz+m40yTtSqro7C0fxTPithEEDjh79E1JSWU/595oiPnFep6cXDmC0n4qR9fAxvuOxljTUqoPkElLs8YY1DSgpqDg7Cbycz13c6t5MnfPnjB0qA5Ga460D0JVFRVlXfD37q35/cxMK3XCLbfAn/5k/XW3QlkJWcR8GUPMlzHEbY6jrNga8ugT5FNlaGXHnh0pyCyomANQMS8gKZsO3TpUjMDpNCiMdbsCeedd4ciRms+Zk1N18nZIiBUIQkKsScXR0daFtly7djVnBenbt/bsFtnZlVM47B9HjlgX/Op8fa1+1uqTr2siYgWOmqa2+PlZ381+dKj9+VxdYdCgyrmRo0bBwIENywCi6kc7qdX5JSbCn/9sdTAHBVlDVGtKIdGlC9x6q7VPK5KXlkdadBrHvj1G9IpoEiMTAauNP2JGBH2n9SV4cDA+AfUbsnvoELz5pjUC8cwZGDwYxo6teeBJu3ZVawgdO1Z93xirNmF/YS+/2B87VrmfSM0BwpizL8p9+lROJ+nateqdfZcu1nGMsUYv29cKcnKs9+yDQLt2dR+0Zox1r5GYaM19LJ/qEhlZmcHDywv69Ts7APbrZ51LNQ4NEKp2+fnwz3/Cc89Zt3j3328FCvvUFq1AWUlZRbqCrIQsTh85XaW5KD89v2LfsNFhFVk3AyMCzzHO/+wLZ/nz1FRYsgQ2bLDugmfPhoUL4bLLGndSd7ny7BblASM3t+b9OnWqvND26lVzJgtnMsaqzURGWmmo7NMy2c9bCwysPfdg+c/asqjUJje3soaTklIZAKvXhtq1c8y/obNogFBnKy2Fzz6DRx+1Mpted53Vt9C7t7NL1mDGWBfnhASI3Z7K4fe/ozAhjdLMbMqyciqSr1Vo346yToEU+gaS5x3AGfdAzniHUOx19kSskpKaUxWdq+mla1e4806rNS6k8TMxtykFBVZNwz7non1/SfXJzFCZhzE01AqMNV3UCwsrM3+cOVO3sri41NyUVj2QnOt9+76cxsz12BA6zFVVKiiAxYvh//7P+osbOtSa7TxunLNLVm/FxbBsmfV1jh2zZevOy+NKtjKKSIrw4ATdyCaELHzJxpds/MjGl9N0pCDHG7eCqn+0nkUgWWefy8WlMj9dXS4Cfn5WX0BdMoqo8/PysvooBg2q+f3yyczVM1CUP09Nrflz7u4wYICV/sq+9hEcbKVxqq3TvabXKSmV27Ky6tZvA9YggOrBpbagUVOtxs/PSkc2a1bdzlcf+t+3rThzBt56C15+2fpLGjkSPv/cmvns7FuYesrNhX//20r3dPy41V4/fGgp14VF4b1jCxQUEjb9Yi55dBwdutTcZ1D+h+bl1bqaC9oqNzerxta1q7NLYqmt+bGuwcY+Y7r9Mc+cse7ryvcvb0q89FINEKohcnPhmWesyWxZWTBpkjVEdfz4ZnllLCy0mg/K747sL+Dp6dbXePVV6/nll8Prrxv6cphvHlxPekw6vSb1Yso/pxA8KNi5X0S1aeUDBby8HDueo7TU+hOvKaA0Bg0QrdnevdY8hUOHrFFJDz8MF1/s7FLV6uBBuPbaqlm+XV0rq9NpaVZn7LW/KeWWifG4Ho4m5s4YdiVkE9AvgHlfzaPvb/q27Nw9StVD+d+Ho2iAaI2Mgddeg4cestZc2LDBqjE0Y2vXwpw51h3Xm29ad0b21e+c9EL802MZ5BpNyveH2bOqEHcfd3pP6U3EzAgGzRmEq0fLaipTqrnTANHapKbCH/4Aq1bB1VdbA/ADG3HKbSMzBl55xZpzN3gwrFwJ3btb72UnZROzMoaYFTHEbYqjtKiU9CAfBswaQMSMCHpN7IW7t86kUspRNEC0Jhs3wu9/bzXQv/oq3H13s+xnKFdUBHfdBe+9Z/WVf/DvUvLiUvh+6ZGKNXsB/Hv7M/qe0fSf3p9uY7rh4qopxJVqChogWouXXrJuw/v3hzVrrOGrzVjKKcNNv0knaVcCfxmZSPekRF7rklSxYEzoyFDGPT2OiBkRBF0UpP0KSjmBBoiWzhhrYZ6nn7bGuS1a1GzyEJSVWctSHzpkjUdPOl5M4YEjtD8ZQ2jOIS7FSnbnftAd1xFdGH33aEJHhdL9su74dXVgz5tqcsYYCksL6/WZwpJCYjNiiU6LJjotmpj0GKLToonNiCXAJ4CIwAgiAiKsn4ER9A/sT6BPzc2pRaVFZBdmk1WYRXZRNtmF2WQXZVNUWkRI+xDCfMMI9Q3F2/3sjIGFJYUkZieSmJ1IUk4SLuKCn6cfvh6++Hr6Vjw3mIrjZhVmnfd5YWkhndt1rjh3mJ/1M9Q3FF8P32ZxU6QBoiUrK4P77oN//cuarvv2281mTsNPP1krj+7bkUc/DjHQNYa+ZbG4mRJK3b0wA/oy7PpwLp0dRuCAQG02agUy8jPYGr+VwxmHScxOJCE7gYSshIqLa3FZ3da4qImLuNDLvxcRgRFMCJ9Aen460WnRLPppEdlF2Y32Hfy9/An1DSW4XTDp+ekkZCWQnp/eaMf3cPWoCCwerh4k5ySTVXj2zEwXcTkrAJV/pib9A/rzzyn/bLRylnNogBCRqcArgCvwnjHm+WrvvwSUT+H1AYKNMR1t75UCP9veO26MudaRZW1xiothwQJrTsOf/mTNjG4Gdxy5ufDk/xSz7tUYLnbfx1UusVBm8OviR/8Zw4mYEUGPX/fA1b15BDJVs8KSQj7e9zErYlbQ1bdrxV16RGAE3Tp0w0VcyCnK4btj37EpbhMb4zayN3kvxpbPpL1He8J8wwjzC+OKHlcQ5htGB88O9bordnNxqwgKvf174+nmedY+xhiScpIqahk1XWwB3F3c8fX0xdfDdsG1PXdzcSM5J7kioJX/TMlNoUeHHozpOqbK3X2X9l0AqtREyp8DFRdz+3PYX+Rr+g45RTkVQbQ8oJ4pPFO11mH7WVJW8/Ts4HaOmffjsFxMIuIKHAImASeBSGCeMeZALfvfAww3xiywvc4xxrSv6/naVC6mggJrTOjKlVbT0p//7PTgYMoMy56P5+tn99E99yBeFNI+1I+hvx/MRb+9iC4jujSLKrM6t+zCbN7e9TYv/fgSidmJ9OzYk8yCTDILMiv28XbzpkfHHsRmxFJSVoKHqweXdr2U8eHjGR8+nqGdh+LreXY+K9U8OSsX02gg1hhz1FaIZcB0oMYAAcwD/urA8rQO2dkwfTps3mw1Ld11l1OL88vmVL7710+cXPMzHvlZhLt40P2qi5j04BB6jO2hTUctREpuCq/ueJXXI18nsyCT8eHj+WD6B0zqNanifft+gKOnjzK9/3QmhE/gsu6X4eNevzToqmVwZIAIA07YvT4JXFLTjiLSAwgHNtlt9hKRKKAEeN4Ys6KGz90O3A7QvXzwfGtWUmKNB/32W6tp6cYbm/T0BQXWqXdsyuXYqp/xit5HUEkSZQgnXHrTb85EHnwrgnYddW5Cc1VmykjISqhokolOiyY6PZofTvxAYUkhMwfM5JHLHmF02Ogqn+vcvjOd23dmbM+xTiq5cobm0kk9F1hujCm129bDGJMgIr2ATSLyszGmylpcxph3gHfAamJquuI6ySOPWHMd3n+/SYNDUZF1yrf+ksDAtK30IZZuGAo6dcF77BRG3zKIR8e31yUjm6HCkkJ+PPkjG+M2siluE3uS95BXXLlUagfPDkQERnDL8Fu4Z/Q99A/s78TSqubGkQEiAehm97qrbVtN5gJV2kqMMQm2n0dFZAswHKhlscY2YMkSa2Gfu++2Zko3gdJSq6Ly1FPgERfN9S5f4OnvxbD5Yxh921CCLmpdq8q1RMYY8kvyq3SWZhZksjNhJ5viNvH98e/JL8nHRVwYGTqS20bcxoDAAfQP7E9EYASd23XWviFVK0cGiEigr4iEYwWGucAN1XcSkQjAH9hut80fyDPGFIpIIHAZ8IIDy9q87d5tLfP5619bQcLBysrgiy+s6RXR0XBdt50McVlL6MhQbvhqHu2Cm8c8i9ausKSQI6ePcDj9cOWQ0ZzKkS5JOUmcKThDaZWKd6XBwYO5/eLbGR8+nrE9xtLBq3WtEqgcz2EBwhhTIiJ3A+uwhrm+b4zZLyJ/A6KMMSttu84Flpmqw6kGAG+LSBnggtUHUVvnduuWmmr1OwQGWus3OHAV95IS+PRT+N//hZ9/hgERhn/N2Ejaim30u6Yfs5bOwqNdM1ujsoUxxlBQUlBlslb55KnknOSKPoHyjuAyU7nOpqu40sW3C6G+ofQL6MfYHmPx9/avcbz8wKCBdG7f2YnfVLUGuuRoc1ZcDFOmwA8/wPffW4v8OEBenpXT7x//gPh4a4WtRx8swWfDSvYv/ZmL77yYaa9Nw8VNRyQ1RFFpEZvjNvNlzJd8GfMlidmJte7r6epJv4B+VeYd9O3Ul65+XQluF4yri84fUY1LlxxtqR56yBrOumiRQ4JDRga88YaVTTUtzVqV6pVXYOLlBXw++1P2b45nwnMTuOyRy7Sduh5Ky0o5XXCaDUc38GXMl6w+vJqswix83H2Y2mcqI7uMrDKRqvx5oE8gPTr00CCgmg0NEM3VRx9ZV+v77oObbmq0wyYnw1dfwYoVsPWbIvyLTzFtcBpXTk7HOzeN4w+n8eKR0yAw86OZDPndkEY7d0uUkZ9BVGIUB1MPVp09W1SZU8e+gzirMKvKKKEgnyB+e9FvmRExgwnhE2rM9aNUc6VNTM1RVJS1nuall8L69Rfc7xAbC//5jxUUfvzRyu83pGs6M9LfxyXfupi5erjSqW8nAiMCCYwIpP+1/QkbHdYY36bFyCvOY3fSbiITIolMtB6xGbFV9vFy8zorlcJZz20/R4eN5lddf6U1AtWsaRNTS5KSYnVKd+4Mn312wcFh/XqYNs0asjpihDVkddr4fLYvWEp+AVy95Ho6D+5Mx54d21QfQ3FpMT+n/FwRDHYm7GR/6v6KTuGufl0ZFTqKBcMWMCpsFEM6D8Hfyx93V50EqNoODRDNSXGxtXZ0Whps23bBq52npsL8+RARAatXWyu1lRaXsuSqzzkdd5qbNt5Ejyt6NFLhW4bD6Yd5YP0DfHPkm4r00wHeAYwKG8WMiBmMCh3FqLBRhLQPcXJJlXI+DRDNyQMPVKbRGDHigg5ljJUB/PRpWLfOCg7GGNbcu4a4jXFM/2B6mwoOhSWFvLDtBZ757hk83TxZOGohl4RdwqiwUYR3DNdOeKVqoAGiufjgAyv53gMPNEoajbfesjqjX34Zhtj6mSNfj2TXW7sY8/AYht087ILP0VJsjd/KnavuJDotmusHXs/LU16mi28XZxdLqWZPA0RzsHMn3HknTJhgzVK7QAcPWnFmyhS45x5rW+y6WNbet5b+1/ZnwrMTLvgcLUF6XjoPffMQH+z9gPCO4ay5cQ1T+0x1drGUajE0QDhbcjJcdx2EhlrTmN0u7J+ksBDmzYP27eHDD8HFBVIPprL8+uUEDw7muiXXtZoU3Cm5KXwV8xVbjm0hIz/jrCUlMwsyMRgevexR/jL2L5qSWql60gDhTCUlVqd0RgZs3w4BARd8yD//2Vru86uvICQETv18ik9nfIqblxvzVs7Do33LTpURmxHLiugVfBnzJduOb8Ng6NLeSj/h6+lL9w7dK4aadvDqwI2Db2Rw58HOLrZSLZIGCGd65RUrhcZHH8HQoRd8uPXrrVx+CxfClPHFbHh0K9tf3I5nB09u+PoGOnRvucnavoz+kv/Z9D/sT90PwLCQYTwx9glmRMxgaOeh2smslANogHCWo0fhL3+Ba65plE7ptDRrSOuAAbBw0mHeGLiazPhMhv1hGJNemIRPYMtsXjHG8Ox3z/L45scZHDyYl6e8zPSI6fTs2NPZRSd1xdcAACAASURBVFOq1dMA4QzGwB13WP0Nb7zRKOtJP/wwFKZl88ch61g+cz+BEYHM3zKfnmN7Xnh5nSS/OJ9bVt7C0l+WcuPgG3nv2vfwcvNydrGUajM0QDjDokWwYYMVHLp2veDD7dhexi8f7OJej42c2lrCuL+PY8xDY3DzbLn/vEnZScz4dAY7E3by7PhnefTyR7UZSakm1nKvIC3VqVPWGNTLLrNqERcocXcyyyZ9zW9IoNul4Ux/9zcE9L3wzm5n2pW4i+nLppNZkMmKOSuYHjHd2UVSqk3SANHU7r0XcnPhvfesMagNVJRTxJYnt7D9pR9xK/Mh4PaZ/OGtwS36LtsYwyc/f8JtX91GULsgti3YxtCQC++8V0o1jAaIprRypZWA7+9/txIkNVDMVzGsuXsNZ46f4YD3CBIGTOR/3/RujK4Mp0nNTWXh6oUsP7Ccy7pdxhfXf6EroinlZBogmkpWljX+dNAgq0e5gdY/uJ7tL24naGAQBTf8gc8+6c6Pb1xQZcTp/nPwP9z59Z2cKTzDcxOe48ExD+Lmov81lXI2/StsKo8+ComJ1sIMHg2brLb3w71sf3E7F995MX3uvoohw12ZPx8uuaSRy9pEMvIzuGfNPXzy8ycMDxnOppmbGBQ8yNnFUkrZaIBoCj/+CG++CfffD6NHN+gQCZEJfH3n14RPCGfaa9O4doYLXl7w3HONXNYmsvrwam5deSupeak8deVTPHb5Y7rWglLNjAYIRysrswJDSIjV99AAOady+HTmp/h28WX2stmsXe/CqlXwwgvQpYUlJS0pK+Evm/7C89ueZ3DwYFbdsIrhXYY7u1hKqRpogHC0Zctgxw54/30rg149lRaV8vnsz8nPyOeW7bfg5ufDH/8I/fpZy1W3JCm5Kcz7Yh6b4jZx+4jbeeWqV3Tim1LNmAYIR8rLs/oehg+38mA0wNo/ruX498eZtXQWIUNDePxxOHQIVq1qcFeGU+w4uYPZn88mLS+N9699nz8M/4Ozi6SUOg8NEI70z3/CiRNWMr4GDDPa/e/dRL0RxZiHxjBwziAeewyefx5uuslaZ7olMMbwVtRb3Lf2Prr6deWHBT9ok5JSLYQGCEdJTLSu5jNnwtix9f74yR9PsnrhanpN6sW4ZyawcKG1Stwdd8DrrzugvA6QWZDJvWvu5aN9HzGt7zQ+nvkx/t7+zi6WUqqONEA4yuOPQ1GR1ZNcT2nRaSybvgzfMF+mfzSbm+a7sHSp1Vr17LONktvPoYwxfLb/M+5fdz8puSk8deVTPP7rx3GRFjxZQ6k2SAOEI+zebS3n9sAD0KdPvT6acSSDxRMWg8Cs/97IDbd4s2qVNZz10UcdU9zGdPT0Ue5afRdrY9cyossIvp73NReHXuzsYimlGkADRGMzxgoMAQFWLaIezhw/w+IJiykpLGH21zdz472BfPedNYXizjsdVN5GUlxazIvbX+SprU/h5uLGy1Ne5q7Rd+mMaKVaMIf+9YrIVOAVwBV4zxjzfLX3XwLG2V76AMHGmI629+YD5VfYp40xixxZ1kazYgVs3Wp1FHTsWOePZSdms3jCYgoyC5i5Yj5z7glm715YssRaY7o5i0mLYfbns/kl5RdmRszk1atepavfhacxV0o5lxhjHHNgEVfgEDAJOAlEAvOMMQdq2f8eYLgxZoGIdAKigJGAAXYBFxtjTtd2vpEjR5qoqKhG/hb1VFgIAweCp6e1MLRb3eJvbmouH479kKwTWVy7/Pf8/rGu7N8Py5dbC841ZwdSDzB+0XgMhveueY9r+jfzAiulqhCRXcaYkTW958gaxGgg1hhz1FaIZcB0oMYAAcwD/mp7PgX4xhiTYfvsN8BUYKkDy3vh3ngDjhyBNWvqHBzyM/L5aNJHZMZncvXSG7nxka5ER1sVkauucnB5L9AvKb8wftF4XF1c2TJ/CxGBDc9Qq5Rqfhw5rCQMOGH3+qRt21lEpAcQDmyqz2dF5HYRiRKRqNTU1EYpdIOdOQPPPAMTJ8LUqXX6SEFmAR9P/Zi0g2lM/XAuv/9LT2JirKzgzT047Du1j3GLxuHu6q7BQalWqrmMO5wLLDfGlNbnQ8aYd4wxI40xI4OCghxUtDr6xz8gPd2a+1AH+aetmkPy3mQmvfdb5v+tN7Gx8NVXMHmyg8t6gfYm72XconF4unqyZf4W+gf2d3aRlFIO4MgAkQB0s3vd1batJnOp2nxUn886X3KyNWv6+uvh4vMP6cxLz2PxhMWc2neKyf+ewx+e609cnJU+Y+LEJijvBdidtJvxi8bTzr0dW2/eSt+Avs4uklLKQRwZICKBviISLiIeWEFgZfWdRCQC8Ae2221eB0wWEX8R8Qcm27Y1T3//uzUp7umnz7trXpoVHFIPpDL+3TnMf6Yfx4/D6tUwbtx5P+5UkQmRTFg8AT9PP7bevJXenXo7u0hKKQdyWCe1MaZERO7GurC7Au8bY/aLyN+AKGNMebCYCywzdsOpjDEZIvJ3rCAD8LfyDutmJzYW3nkHbr0V+p77bjo3NZfFExaTcTiDK96Yy7y/9CEzE9auhcsvb6LyNtCOkzuY/PFkOnl3YvP8zfTs2NPZRVJKOZjDhrk2NacNc503z+pVjo095+IMOadyWDxhMaePnmb0S/P43RO9KCuDdetgxIgmLG8DbD+xnSkfTyGoXRCb52+me4fuzi6SUqqRnGuYa3PppG6Zdu+21nu4//5zBof80/ksGreI00dPM+T5G/jtI73w8IDvvmv+weH7498z+ePJdG7fma03b9XgoFQbogHiQjz2GHTqBA8/fM7dNj62kfRD6fR98gbmPBpOcDB8/z1ENPORod8e+5apH08l1DeUrTdv1dnRSrUxGiAaatMmWL8e/vxn6NCh1t0SIhPY9c4u/KeO5nePh9Ovn1Vz6NGjCcvaAFvit3DVkqvo1qEbW+ZvIdQ31NlFUko1MQ0QDWGMlVq1Wze4665adysrLWP1wtW4+7fnkdVXMno0bNkCnTs3XVEbYsPRDUxbMo2eHXuyZf4Wuvi2sIWvlVKNQgNEQ3zxBURGwlNPgVftayrv+fceEqMSWVs6ifABXqxbV6/8fU3OGMM/t/+TqR9PpU+nPmyev5nO7Zt5NFNKOYzmYm6IV16Bfv2stT9rkZeWx8bHNpLdqQc7sgez8xNo164Jy1hP2YXZ3LLyFj4/8DkzI2by4YwP8fP0c3axlFJOpAGivpKSYNs2ePJJcHWtdbeNf95IfmYBH5VN4/l/CkOHNl0R6+tg6kGu++w6DqUf4oWJL/DgmAeR5r5snVLK4c4bIETkGmCVMaasCcrT/P33v1YfxKxZte5ycsdJdr+3m50uv2LY5GDuu68Jy1dPn+//nAUrF+Dj7sOG329gXHgzn86tlGoydemDmAMcFpEXbGkx2rYvvrDGp150UY1vl5WWsWrhavJd27Onw5V8+CG4NMOenrziPO5fez/XL7+ewcGD2X37bg0OSqkqznvpMsb8DhgOHAE+FJHttjTbvg4vXXOTmmoNQ5o9G2ppgtn97m6SdyexqmQyb33gea75c06z5vAaBr4xkFd2vMI9o+9hy81bCPOrMRO7UqoNq9O9rTEmC1gOLAO6ADOB3bZV4NqOFSugrKzW5qXc1FzWPbyROHpyxZ2DuPbaJi7feSRlJzFn+RymfTINLzcvtszfwqtXvYqHq4ezi6aUaobq0gdxLfAHoA+wGBhtjEkRER+s1eFec2wRm5EvvoDevamtx3n1AxspzC4iutdVbHqx+XTylpaV8vaut3ls42MUlhTy93F/56ExD+Hp5unsoimlmrG6jGKaBbxkjPnWfqMxJk9EbnFMsZqhjAzYuBEeeKDG5qWEqET2f7yHKJdf8dYXwfj4OKGMdlJzU4lMjCQyIZKVh1ayO2k3E3tN5I1pb+gaDkqpOqlLgHgSSCp/ISLeQGdjTLwxZqOjCtbsrFwJJSVW/0M1xhg+umEtubRj7BNjGTas6Yt3Ov807+95nx0JO4hMjCQ+Mx4AQRgYPJCPZ37MDYNv0OGrSqk6q0uA+BwYY/e61LZtlENK1Fx98QV07w4jz86Ku/W1nyk8fIJjfa7h+cdrn1ntKNFp0Vyz9BpiM2Lp2bEno0JHcdeouxgVOooRXUbg69n2xhMopS5cXQKEmzGmqPyFMabItkJc25GVZSXmu+uus5qXCrKKWP/wBjJcQnl29fBzzZ1ziHWx65izfA6ebp58/4fvuaz7ZU1bAKVUq1WXUUypto5qAERkOpDmuCI1Q19/bS0pWkPz0iuzv8OzMJtBD06lT9+ma74xxvDKj68w7ZNp9OjYg5237tTgoJRqVHWpQdwJLBGRfwECnABqT0LUGi1fDqGh8KtfVdkctT6DnG+2kxk2hCee79ZkxSkqLeLu1Xfz7u53mRExg49mfkR7j/ZNdn6lVNtw3gBhjDkC/EpE2tte5zi8VM1JTg6sWWOtOW03JbqkBP49Zz2dcOG+rybWNm+u0aXlpTH7s9lsPbaVxy5/jKfHP42LNMOp2kqpFq9OyfpE5DfAQMCrfBSMMeZvDixX87FmDRQUnNW89OwtRwjJjCH4hgn0Ge74TmBjDEt+XsID6x4gqzCLj2d+zI1DbnT4eZVSbVddJsq9BfgA44D3gNnATgeXq/lYvhyCg+Hyyys2Re4oJWXxWjq078Rt7//qHB9uHIfSD7Fw1UI2xm3kkrBLePvqtxka0ozTwyqlWoW6tE2MMcbcBJw2xjwFXAr0c2yxmon8fFi1CmbOrJLa+60FkQSRxqx3p+Dm6biM6YUlhfxt698Y8uYQohKjeGPaG2xbsE2Dg1KqSdTl6lZg+5knIqFAOlY+ptZv3TrIza3SvJSfZ/A7uIPi0B4Mn+O4Gcnbjm/jlpW3EJMew5yBc3hpyku69KdSqknVJUB8JSIdgf8DdgMGeNehpWouli+HTp1g7NiKTV//K46OJpPwG8c7bFbyhqMbuPqTqwn1DWXNjWuY2meqQ86jlFLncs4AISIuwEZjTCbwhYh8DXgZY840SemcqaTEal6aMQPc3Ss2//TBHkrwYvojAxxy2u+Ofce1S6+lX0A/Ns/fTIBPgEPOo5RS53POPgjbKnKv270ubBPBASAyEjIz4aqrKjbln87HJeYgp7sOpkNA4/c97Di5o2Li24abNmhwUEo5VV06qTeKyCxpa1ne1q+30mpMmFCx6bvXf8bVlNJr1vBGP92epD1MXTKV4HbBbPj9BoLbBTf6OZRSqj7qEiDuwErOVygiWSKSLSJZDi6X861bB6NGQUDlXXzUu3tIIoRptzRuZ/H+lP1M+mgSvh6+bLppk67uppRqFuqy5KivMcbFGONhjPGzvfary8FFZKqIxIhIrIg8Wss+14vIARHZLyKf2G0vFZG9tsfKun+lRpCZCTt2wOTJFZuS9iRRfDyZox2GM2hQ453qUPohJiyegIerB5vmb6JHxx6Nd3CllLoAdZko9+uatldfQKiGz7li9V9MAk4CkSKy0hhzwG6fvsBjwGXGmNMiYt+ukm+MccLKClgLA5WVwZQpFZt2v7eHElzpNm1wo6XVSMtLY+LiiZSZMjbP30yfTn0a58BKKdUI6tLT+pDdcy9gNLALGH+ez40GYo0xRwFEZBkwHWuZ0nK3Aa8bY04DGGNS6lhux1q/Hnx94ZJLACgpKGHvRz9zkAH8drp3o5zCGMOtK28lOSeZH275gQFBjhkVpZRSDVWXJqZr7B6TgEHA6TocOwwr82u5k7Zt9voB/URkm4j8KCL2A/69RCTKtn1GHc7XOIyx+h8mTKgY3hq9IpqS7AL2MJyJExvnNO/ufpcvY77kuQnPMTL07EWIlFLK2RoyVvMk0Fi3u25AX+BKoCvwrYgMts276GGMSRCRXsAmEfnZllm2gojcDtwO0L1798Yp0eHDcOwYPPJIxaY9/95DnkcHgoaF2/dZN1h0WjT3r72fib0m8sdL/3jhB1RKKQeoSx/Ea1izp8GqcQzDmlF9PgmA/SIJXW3b7J0EdhhjioE4ETmEFTAijTEJAMaYoyKyBRgOVAkQxph3gHcARo4caWgM69dbP239D5nxmRzdcJSdMpYpUy+886GotIgbvrgBH3cfFs1YpKm6lVLNVl2uTlFYfQ67gO3AI8aY39Xhc5FAXxEJty1ROheoPhppBVbtAREJxGpyOioi/iLiabf9Mqr2XTjOunXQuzf06gXA3g/3gsAeM4ypjZDx4vFNj7MneQ//vvbfhPqGXvgBlVLKQerSxLQcKDDGlII1OklEfIwxeef6kDGmRETuBtYBrsD7xpj9IvI3IMoYs9L23mQROQCUAg8ZY9JFZAzwtoiUYQWx5+1HPzlMURFs3gw3WQvmlZWWsfeDveR36YXkdWTUqAs7/MajG/m/H/6POy6+g+kR0xuhwEop5Th1CRAbgYlA+Upy3sB6YMz5PmiMWQ2srrbtCbvnBnjA9rDf5wdgcB3K1ri2b7eyt9qal+I2xXHm+Bm2+U9k0iRwu4DsGul56dy04ib6B/TnxckvNlKBlVLKcerSxORlv8yo7bmP44rkROvWWVFg3DjA6px29/Pix9MR9lMi6s0Yw21f3UZqbipLZy2lnUe7RiqwUko5Tl0CRK6IjCh/ISIXA/mOK5ITrV8Pl14Kfn4YY4hdG0tZxABKcLugAPHOrnf4b/R/eWb8Mwzv0vh5nJRSyhHq0mhyP/C5iCQCAoQAcxxaKmdITYXdu+Fv1lLbmfGZFJ4p5HBWGAMHQteuDTvsvlP7uG/tfUzuPZk/jflTIxZYKaUc67wBwhgTKSIRQH/bphjbsNTWZcMGa5KcraqQvDcZgO9jQ5h9X8MOmVOUw5zlc/D39uejmR/pkFalVIty3iuWiNwFtDPG/GKM+QVoLyILHV+0JrZunbV63AirNS15TzK4CAklwQ1uXrp79d3EpMWw5Lolmr5bKdXi1OWW9jbbzGYAbHmTbnNckZzAGKv/YdIkcHUFrBpEScdA3L3dueKK+h9y8U+LWfTTIv7y678wPvx8aauUUqr5qUuAcLVfLMiWpdXDcUVygv37ISmpSnrv5L3JJJSEMHYseHnV73DRadEsXLWQsT3G8sTYJ87/AaWUaobq0km9FvhURN62vb4DWOO4IjnBunXWT1uAyEvPI+tEFoddQriinoOO8ovzmbN8Dt7u3iy5bgmuLq6NXFillGoadQkQj2AlxLvT9nof1kim1mP9erjoooqhSuUd1IllIfTtW79DPbDuAfad2seqG1bpynBKqRatLum+y4AdQDzWGg/jgYOOLVYTys+Hb789q3kJIJkQ+tRjDZ+vD33NW7ve4qExDzGt77TGLqlSSjWpWmsQItIPmGd7pAGfAhhjxjVN0ZpIZibMmgXTK3MjJe9JxqWjH3mZPvUKEM9//zy9/HvxzPhnHFBQpZRqWudqYooGvgOuNsbEAohI61u8oEsX+PjjKpuS9yZT4B9Cu2IIqWNj2q7EXWw7sY2XpryEu6u7AwqqlFJN61xNTNcBScBmEXlXRCZgzaRu1Yrzi0mLTiPV1Wpequv606/tfI127u34w7A/OLaASinVRGoNEMaYFcaYuUAEsBkr5UawiLwpIpNr+1xLl/JLCqbUcDSv7v0PKbkpLP1lKfOHzqeDVwfHFlAppZpIXTqpc40xnxhjrsFaFW4P1simVqm8g3pfSt0DxLu73qWotIi7R9/twJIppVTTqldyIGPMaWPMO8aYCY4qkLMl70nGvb0nqSUd6xQgikuLeTPqTSb1msSAoMZaqlsppZxPs8dVk7w3Ge/wEEDqNAfiv9H/JSE7gXsvudfhZVNKqaakAcJOWWkZp/adojigM0CdahCv7XyN3v69dd6DUqrV0QBhJyM2g+LcYtI9uuDtbY2APZfdSbv5/vj33DXqLk3lrZRqdfSqZqe8gzq+wOqgdjnPb6diaOtwHdqqlGp9NEDYSd6bjIu7C/tTgs7bvJSam8rSn5dy09Cb6OjVsWkKqJRSTUgDhJ3kPckEDQzm8FHX8waId3e/S2FpoQ5tVUq1Whog7CTvTca3TwhFRefuoC4f2jqx10QuCrqo6QqolFJNSAOETU5yDrmncinrbCVfOtcQ1xXRKziZdZJ7R+vQVqVU66UBwiZpTxIAmV5WgDhXDeKNqDcI7xiuQ1uVUq2aBgib8hFMxwo74+UFYbWs9XM4/TBb4rdw24jbdLU4pVSrpgHC5tTeU/j38if2hBe9e9c+xPW93e/hKq7cPOzmJi2fUko1NQ0QNkl7kggZFkJsbO3NS8WlxXz404dc3e9quvieZxadUkq1cBoggMLsQjJiMwge2pkjR2oPEF8d+oqU3BRuG3Fb0xZQKaWcwKEBQkSmikiMiMSKyKO17HO9iBwQkf0i8ond9vkictj2mO/Icqb8nAIG3Lt1oaCg9gDx3u73CPMNY0qfKY4sjlJKNQvnWnL0goiIK/A6MAk4CUSKyEpjzAG7ffoCjwGXGWNOi0iwbXsn4K/ASMAAu2yfPe2IspaPYMpuV/sQ1+NnjrM2di2P//px3Fwc9mtTSqlmw5FXutFArDHmKICILAOmAwfs9rkNeL38wm+MSbFtnwJ8Y4zJsH32G2AqsNQRBU3em4x3gDfHM32BmmsQ7+95H4AFwxc4ogjqPIqLizl58iQFBQXOLopSLZKXlxddu3bF3d29zp9xZIAIA07YvT4JXFJtn34AIrINcAWeNMasreWzZw08FZHbgdsBunfv3uCCJu9JpsvwLvxyRPDwgK5dq75fWlbK+3veZ1LvSfTs2LPB51ENd/LkSXx9fenZsydS14XClVIAGGNIT0/n5MmThIeH1/lzzu6kdgP6AlcC84B3RaTOme9sq9uNNMaMDAoKalABSotLSfklhc7DOnP4MPTqBa7VpjesP7KeE1kntHPaiQoKCggICNDgoFQDiAgBAQH1roE7MkAkAN3sXne1bbN3ElhpjCk2xsQBh7ACRl0+2yhyU3Lp1KcToReHEhtbc//Du7vfJcgniGv7X+uIIqg60uCgVMM15O/HkQEiEugrIuEi4gHMBVZW22cFVu0BEQnEanI6CqwDJouIv4j4A5Nt2xqdX5gfC39ZyMA5g2qcA5Gck8xXh75i/tD5eLh6OKIISinVLDksQBhjSoC7sS7sB4HPjDH7ReRvIlJ+K74OSBeRA8Bm4CFjTLqtc/rvWEEmEvhbeYe1oyQlQX7+2QFi0d5FlJSVcOuIWx15etUCiAi/+93vKl6XlJQQFBTE1VdfXa/j9OzZk7S0tAbvs3fvXkSEtWvX1uu8StWXQ8drGmNWA6urbXvC7rkBHrA9qn/2feB9R5bP3uHD1k/7JiZjDO/teY8rul9B/8D+TVUU1Uy1a9eOX375hfz8fLy9vfnmm28Iqy1plwMtXbqUyy+/nKVLlzJ16tQmP79qO3RAv01srPXTvgax9dhWYjNieeLXT9T8IeUc998Pe/c27jGHDYOXXz7vbtOmTWPVqlXMnj2bpUuXMm/ePL777jsAMjIyWLBgAUePHsXHx4d33nmHIUOGkJ6ezrx580hISODSSy/Fui+yfPzxx7z66qsUFRVxySWX8MYbb+BafZSEHWMMn3/+Od988w1XXHEFBQUFeHl5Xfj3V6oGzh7F1GzExoK7O3Sz6xp/d/e7dPDswOyLZjuvYKpZmTt3LsuWLaOgoIB9+/ZxySWVI7f/+te/Mnz4cPbt28ezzz7LTTfdBMBTTz3F5Zdfzv79+5k5cybHjx8H4ODBg3z66ads27aNvXv34urqypIlS855/h9++IHw8HB69+7NlVdeyapVqxz3ZVWbpzUIm9hYCA8HN7vfyDdHvmFGxAy83b2dVzB1tjrc6TvKkCFDiI+PZ+nSpUybVnU9kO+//54vvvgCgPHjx5Oenk5WVhbffvst//nPfwD4zW9+g7+/PwAbN25k165djBo1CoD8/HyCg4PPef6lS5cyd+5cwApWixcvZtasWY36HZUqpwHC5vDhqv0POUU5pOal0j9A+x5UVddeey0PPvggW7ZsIT09vcHHMcYwf/58nnvuuTrtX1payhdffMGXX37JM888UzH5KTs7G19f3waXQ6naaBMTYAxnDXE9lnkMQGdOq7MsWLCAv/71rwwePLjK9iuuuKKiiWjLli0EBgbi5+fHr3/9az75xMpDuWbNGk6ftlKKTZgwgeXLl5OSYmWYycjI4NixY7Wed+PGjQwZMoQTJ04QHx/PsWPHmDVrFv/9738d8TWV0gABcOoU5OZWDRBxmXEAhPvXfVq6ahu6du3KvfeevR75k08+ya5duxgyZAiPPvooixYtAqy+iW+//ZaBAwfyn//8pyItzEUXXcTTTz/N5MmTGTJkCJMmTSIpKanW8y5dupSZM2dW2TZr1iyWLnVIijKlEPsRFS3ZyJEjTVRUVIM++/33cMUVsGYNlI8afG3Ha9y79l6S/5RM5/adG7GkqiEOHjzIgAEDnF0MpVq0mv6ORGSXMWZkTftrDYKa50DEZ8bj7eZNcLtzdxoqpVRrpQECq//BzQ169KjcFpcZR8+OmjlUKdV2aYDAChA9e1Yd4hqXGaf9D0qpNk0DBNSYxTU+M56eHXo6pTxKKdUctPkAYYzVB2E/gimzIJPMgkytQSil2rQ2HyBSUyE7u9oQ19O2Ia4dNUAopdquNh8gfH1h3Tq41m4toPjMeEAnyamqmkO67549ezJ48GCGDRvG4MGD+fLLL+t1bqXqo82n2vD2hsmTq27TSXKqJs0l3ffmzZsJDAwkJiaGyZMnM3369CYvg2ob2nyAqEnc6Tj8PP3w9/J3dlFUDZyY7dvp6b7tZWVlVST+U8oR2nwTU03iz8TrHAhVI2en+wYYN24cgwYNYuzYsTz99NOO+aJKoTWIGsWdjqNPpz7n31E5hROzfTs93TdUNjEdOXKECRMmcOWVJjwGDgAAEsdJREFUV9K+ffvG/JpKARogzmKMIS4zjkm9Jjm7KKqZcla67+p69+5N586dOXDgAKNHj25wOZSqjTYxVZOWl0ZecZ6OYFK1cla67+pSUlKIi4ujh32OGKUakdYgqtERTOp8zpXue8GCBQwZMgQfH58q6b7nzZvHwIEDGTNmTI3pvsvKynB3d+f1118/7wV/3LhxuLq6UlxczPPPP0/nzpptWDmGpvuu5tNfPmXuF3PZd+c+BncefP4PqCah6b6VunCa7vsC6SQ5pZSyaICoJi4zjgDvAHw9dY1fpVTbpgGiGk3zrZRSFg0Q1cRnxmvzklJKoQGiijJTRnxmvGZxVUopNEBUkZSdRFFpkQYIpZTCwQFCRKaKSIyIxIrIozW8f7OIpIrIXtvjVrv3Su22r3RkOcvpCCZ1Ls0h3XdOTg533HEHvXv35uKLL+bKK69kx44d9Tq/UnXlsIlyIuIKvA5MAk4CkSKy0hhzoNqunxpj7q7hEPnGmGGOKl9NdJKcOpfmkO771ltvJTw8nMOHD+Pi4kJcXBwHDlT/k1KqcThyJvVoINYYcxRARJYB04Fm+7+5fCW5Hh00dUFzdv/a+9mb3Lj5voeFDOPlqefPAujMdN9Hjhxhx44dLFmyBBcXq/IfHh5OeLje0CjHcGQTUxhwwu71Sdu26maJyD4RWS4i3ey2e4lIlIj8KCIzajqBiNxu2ycqNTX1ggscnxlPSPsQvN29L/hYqnVyZrrv/fv3M2zYsDqvF6HUhXJ2LqavgKXGmEIRuQNYBIy3vdfDGJMgIr2ATSLyszHmiP2HjTHvAO+AlWrjQgsTlxmnHdQtQF3u9B2lOaT7VqqpODJAJAD2NYKutm0VjDH2uZLfA16wey/B9vOoiGwBhgNVAkRji8uMY0y3MY48hWoFnJXue+DAgfz000+UlpZqLUI1CUc2MUUCfUUkXEQ8gLlAldFIItLF7uW1wEHbdn8R8fz/7d19VFVlvsDx7y9EQUbFQbOZsDRDEYg3qeVVcbC0vJeim+YbXtdYKl3SxLUyh669F668EWreWuuSmblSNDUjc0WpmZq4ZhRBQvPmGw4YJuMMCr4wAc/942xOBzy8KXiU8/usxWLv52z2eR7YnN9+nr3377GWewBDaeNrF1U1VRSdK9IehGqSq9J99+vXj6ioKF5++WX7dYzCwkI2b97c6m1UCtqwB2GMqRKRWcBXgAew3BhzUEReA/YZYz4HZotIHFAF/B2Yav34QOB/RaQGWxB708ndT62q+Hwx1aZab3FVTXJluu9ly5bx7LPPcvfdd+Pt7U2PHj1466232qahyu1pum/L9hPbuX/l/WydspUH7nqgFWumWoOm+1bq2mm676tU+5CcPgOhlFI2GiAsJ8pOcIvcQu+uvZveWCml3IAGCMuJshP4d/XH08PT1VVRSqkbggYIi2ZxVUqpujRAWE7844TewaSUUg40QACVVZX8VP6T9iCUUsqBBgjgr+f+isHoHUyqUTdCum+AvLw8RISsrKwWva+j6dOn27PALliwwF5eWFhISEjIVe+3NcTExNDSW9Zfeukltm7d2up1yc3NZdq0aQCsWLGCWbPqJp6+ePEisbGxBAYGEhwcTHLyFbMa2G3atImgoCBCQkKYP39+g9utWLGCnj17EhERQUBAAA899BDZ2dn2140xvPHGGwQEBNC/f39GjBjBwYMH7a+PHDnS/jDmtdIAwa9pvnWISTXGMd034JJ03wAZGRkMGzaMjIyMq97HsmXLCAoKAuoGiKtRVVV1TT9/raqrq3nttdcYOXLkNe3HWTsWLFjg9KFIR3PnzuXw4cPk5uaye/duvvzyS6fbzZkzh82bN1NQUMD06dOdblNrwoQJ5ObmcuTIEZKTkxkzZgw//PADAO+++y7Z2dkcOHCAH3/8keeff564uDguX74MwJQpU3jvvfea0+QmuTpZ3w2hNs23DjHdHLLmZHE673Sr7vO28NsYvXh0k9u5Mt032M4e161bx5YtW4iOjuby5ct4eXnV2WbdunXs2bOHtLQ0lixZwpIlSzh+/DjHjx9nypQp7N69m5iYGFJTU1m/fj2XLl0iPDyc4OBgUlJSqK6uZsaMGWRnZ3P77beTmZmJt3fdDMdTp07Fy8uL3Nxchg4dysyZM5k5cyalpaV07tyZ999/n8DAQI4dO8bkyZO5cOECjz76KIsXL6aiooJvv/2W1NRUvvjiCwBmzZpFVFQUU6dOrfM+iYmJ7N27l0uXLvH444/z6quvArYe1oQJE9iyZQvz5s0jKyuLhx9+mD59+tg/fKurqykoKMAYw7Fjx5zWr3470tLS7O9dXl5Ofn4+YWFhDf49OnfuzIgRIwDo2LEjkZGRFBcXO922Y8eOFBcXtzhF+4gRI0hISCA9PZ1FixaxcOFCduzYQefOnQF48MEHGTJkCKtWrWLatGnExcURHR3daC+lubQHge0OJs9bPPl9l9+7uirqBufKdN8A2dnZ9O3bl379+hETE+M0D1N0dLQ9aO3atQs/Pz9OnTrFrl27GD58eJ1t33zzTby9vcnLy7O/95EjR5g5cyYHDx7E19fXnqG2vuLiYrKzs0lLSyMhIYGlS5eSk5NDamoqTz/9NABJSUkkJSXx/fff4+/v35xfcR0pKSns27eP/Px8duzYQX5+vv01Pz8/9u/fz8SJE+1lUVFR5OXlkZeXx+jRo5k7dy5Ag/Wr3w5H+/bta9FwW1lZGZs2beKBB67MxFBTU0NQUBBPPvkkhYWFzd5nrcjISA4fPsz58+e5cOECd911V53Xo6Ki7MNM3bt3p7Ky8poSSdbSHgS2IaY7ut2Bxy2aIfNm0Jwz/bbi6nTfGRkZ9g/EiRMnsnLlSsaOHVtnm9tuu42KigrKy8spKioiPj6enTt3smvXLsaMGdNkG/v27Ut4uG0yx0GDBjX4gTZu3Dg8PDyoqKggOzubcePG2V+rrKwEYM+ePXz22WcAxMfH2z+wm+uTTz4hPT2dqqoqSkpKOHToEKGhoYBtGKYha9euZf/+/Xz99deN1s+xHfWVlJTQs2fPZtWzqqqKSZMmMXv27Cs+vAGWLl1KWFgYiYmJPPLII3zzzTcUFhaycOFC1q9f3+T+W5oS6dZbb+Wnn37Cz8+vRT9XnwYIrHkg9AK1aiZXpfuurq5mw4YNZGZmkpKSgjGGs2fPUl5eTpcuXepsO2TIED788EMGDBhAdHQ0y5cvZ8+ePbz99ttNvk+nTp3syx4eHvZrLvX5+PgAtrNjX19f8vKaP8tfhw4dqKmpsa/Xjp87OnHiBKmpqezdu5fu3bszderUOtvVvn99BQUFvPLKK+zcuRMPD48m69fQfry9vZ3Wy5mEhAQCAgKYM2eO09e/+uor5s2bR0xMDC+++CKxsbHcd999dXo/jcnNzWXgwIF07doVHx8fjh8/XicQ5eTk8Ic//MG+fvny5SuGBa+GDjGhD8mplnFVuu9t27YRGhpKUVERhYWFnDx5krFjx7Jx48Yrto2OjiY1NZXhw4cTERHB9u3b6dSpE926dbtiW09PT3755Zer+2UAXbt2pW/fvqxbtw6wBb4DBw4AMHjwYHuvas2aNfafufPOOzl06BCVlZWUlZWxbdu2K/Z7/vx5fHx86NatGz///HODF38dlZWVMWnSJFauXGk/+2+sfo0ZOHAgR48ebXK7F154gXPnzrF4ccMTWUVERPDxxx9TU1PD+PHjCQgIYPXq1cTGxja5/x07dpCens6MGTMAeO6555g9e7Y9cG/dupXvvvuO+Ph4e/tOnz5Nnz59mtx3U9w+QFz45wXOXDijdzCpZmss3XdOTg6hoaEkJyfXSfe9c+dOgoOD+fTTT52m+w4NDWXUqFGUlJQ0+L4ZGRk89thjdcrGjh3r9G6m6OhoioqKGD58OB4eHvTu3Zthw4Y53W9CQgKhoaFMnjy52b+D+latWsUHH3xAWFgYwcHBZGZmArB48WLS0tIIDQ3l6NGj9gDVu3dvxo8fT0hICOPHjyciIuKKfYaFhREREUFgYCDx8fEMHTq0yXpkZmZy8uRJZsyYQXh4uH2orKH6NSYwMJBz585RXl5uL1uxYgX+/v72r+LiYlJSUjh06BCRkZGEh4ezbNmyK/Y1f/58jDGEhIQwaNAgevXqxVNPPUV8fHydnlSttWvXEh4eTv/+/VmwYAEbNmywZ2F95plnuPfee7nnnnsYMGAAr7/+ep0bCXJychg8eDAdOlz7AJHbp/suvVBKUlYST4Q/wah+o9qgZqo1aLrvm9PFixfx9vZGRFizZg0ZGRnN+nC+USxatIguXbo0eVvqjSQpKYm4uDinF8tbmu7b7a9B9PTpyeqxq11dDaXapZycHGbNmoUxBl9fX5YvX+7qKrVIYmKifWjqZhESEuI0OFwNt+9BqJuD9iCUunY6YZBqt9rLyYxSrnA1/z8aINRNwcvLi7Nnz2qQUOoq1N4SXf+p+6a4/TUIdXOovWOktLTU1VVR6qbk5eXV4qfZNUCom4Knp2eL8tcopa6dDjEppZRySgOEUkoppzRAKKWUcqrdPAchIqVAw4lsbHoAjU/l1X65a9u13e5F291ydxpjnKatbTcBojlEZF9DD4S0d+7adm23e9F2ty4dYlJKKeWUBgillFJOuVuASHd1BVzIXduu7XYv2u5W5FbXIJRSSjWfu/UglFJKNZMGCKWUUk65TYAQkdEi8n8iclREkl1dn7YiIstF5IyIFDiU/VZEtojIEet7d1fWsS2ISG8R2S4ih0TkoIgkWeXtuu0i4iUifxGRA1a7X7XK+4rIn63jfa2IdHR1XduCiHiISK6IfGGtu0u7C0XkexHJE5F9VlmrH+tuESBExAN4F/hXIAiYJCJBrq1Vm1kBjK5XlgxsM8YEANus9famCnjWGBMEDAZmWn/j9t72SuB+Y0wYEA6MFpHBwEJgkTHmbuAfwDQX1rEtJQE/OKy7S7sBRhhjwh2ef2j1Y90tAgRwH3DUGHPcGPNPYA3wqIvr1CaMMTuBv9crfhT4yFr+CPj361qp68AYU2KM2W8tl2P70Liddt52Y1NhrXpaXwa4H1hvlbe7dgOIiD8QCyyz1gU3aHcjWv1Yd5cAcTtQ5LBebJW5i17GmBJr+TTQy5WVaWsi0geIAP6MG7TdGmbJA84AW4BjQJkxpsrapL0e74uBeUCNte6He7QbbCcBX4tIjogkWGWtfqzrfBBuxhhjRKTd3tssIr8BNgBzjDHnbSeVNu217caYaiBcRHyBjUCgi6vU5kTkYeCMMSZHRGJcXR8XGGaMOSUitwJbROSw44utday7Sw/iFNDbYd3fKnMXP4vI7wCs72dcXJ82ISKe2ILDKmPMp1axW7QdwBhTBmwH/gXwFZHaE8D2eLwPBeJEpBDbkPH9wBLaf7sBMMacsr6fwXZScB9tcKy7S4DYCwRYdzh0BCYCn7u4TtfT58AfreU/ApkurEubsMafPwB+MMakObzUrtsuIj2tngMi4g2Mwnb9ZTvwuLVZu2u3MeZ5Y4y/MaYPtv/nb4wxk2nn7QYQER8R6VK7DDwIFNAGx7rbPEktIv+GbczSA1hujElxcZXahIhkADHY0v/+DLwMfAZ8AtyBLSX6eGNM/QvZNzURGQbsAr7n1zHp/8J2HaLdtl1EQrFdkPTAdsL3iTHmNRG5C9uZ9W+BXOA/jDGVrqtp27GGmOYaYx52h3ZbbdxorXYAVhtjUkTEj1Y+1t0mQCillGoZdxliUkop1UIaIJRSSjmlAUIppZRTGiCUUko5pQFCKaWUUxoglGqCiFRbWTNrv1ot4Z+I9HHMvKvUjURTbSjVtEvGmHBXV0Kp6017EEpdJSsn/39befn/IiJ3W+V9ROQbEckXkW0icodV3ktENlpzNxwQkSHWrjxE5H1rPoevrSeiEZHZ1vwW+SKyxkXNVG5MA4RSTfOuN8Q0weG1c8aYe4D/wfakPsBS4CNjTCiwCnjHKn8H2GHN3RAJHLTKA4B3jTHBQBkw1ipPBiKs/fxnWzVOqYbok9RKNUFEKowxv3FSXohtsp7jVqLA08YYPxH5G/A7Y8wvVnmJMaaHiJQC/o6pH6zU5FusSV4QkT8BnsaYN0QkC6jAlirlM4d5H5S6LrQHodS1MQ0st4RjrqBqfr02GIttJsRIYK9DllKlrgsNEEpdmwkO3/dYy9nYMowCTMaWRBBs00Amgn2Sn24N7VREbgF6G2O2A38CugFX9GKUakt6RqJU07ytGdtqZRljam917S4i+dh6AZOssmeAD0XkOaAUeMIqTwLSRWQatp5CIlCCcx7Ax1YQEeAda74Hpa4bvQah1FWyrkFEGWP+5uq6KNUWdIhJKaWUU9qDUEop5ZT2IJRSSjmlAUIppZRTGiCUUko5pQFCKaWUUxoglFJKOfX/Ocp+4U6ZWJEAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 332
        },
        "id": "Ix8vMMH2fEKs",
        "outputId": "0293d1a5-e6e7-40b6-8361-46f066cfc2fe"
      },
      "source": [
        "# Plot validation loss curves\n",
        "epochs = range(1, len(hista['train_loss']) + 1)\n",
        "\n",
        "\n",
        "plt.plot(epochs, histd['test_loss'], '-', color='purple', label='Dropout technique')\n",
        "plt.plot(epochs,histe['test_loss'], '-', color='red', label='L2 technique')\n",
        "plt.plot(epochs,histf['test_loss'], '-', color='green', label='L2 + dropout technique')\n",
        "\n",
        "plt.title('Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1a9be510>]"
            ]
          },
          "metadata": {},
          "execution_count": 23
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1a9bead0>]"
            ]
          },
          "metadata": {},
          "execution_count": 23
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1a9c5090>]"
            ]
          },
          "metadata": {},
          "execution_count": 23
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Test loss')"
            ]
          },
          "metadata": {},
          "execution_count": 23
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 0, 'Epochs')"
            ]
          },
          "metadata": {},
          "execution_count": 23
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Loss')"
            ]
          },
          "metadata": {},
          "execution_count": 23
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f1f1a9b84d0>"
            ]
          },
          "metadata": {},
          "execution_count": 23
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xUZdr4/8+d3kmnJaH3hCQQmoA0pQiCIoiCIMKq7K6sro+u7ld3xWf1t7qrrvqga0MRcRUrsoqoKMXQQy+hE5KQQDrpbeb+/XGSIYH0ZDLAXO/X637N5NTrjDjXnHM3pbVGCCGE/XKwdQBCCCFsSxKBEELYOUkEQghh5yQRCCGEnZNEIIQQdk4SgRBC2DlJBEJYgVKqs1JKK6WcbB2LEPWRRCDsglIqv0oxK6WKqvw9pwnH26iU+o01YhWitcmvFWEXtNZele+VUgnAb7TW620XkRBXD7kjEHZNKeWglHpSKXVKKZWplPpMKeVfsc5NKbWyYnmOUmqXUqqtUup5YCSwtOKOYmkDztNBKbVGKZWllDqplLq/yrrBSqk4pVSuUuqCUuqVus5vrc9C2C+5IxD2bjFwGzAKSAdeB94A7gbuBdoAoUAJEAUUaa2fUkoNB1Zqrd9r4Hk+BQ4BHYDewE9KqVNa61+A14DXtNYfKaW8gPCKfWo8fzOvV4gryB2BsHeLgKe01sla6xJgCTCjopK3DAgAumutTVrr3Vrr3MaeQCkVCgwHntBaF2ut9wHvAfMqNikDuiulArXW+Vrr7VWWN/v8QtRHEoGwd52AryseveQA8YAJaAt8BPwAfKqUSlFK/UMp5dyEc3QAsrTWeVWWnQU6VrxfCPQEjlY8/plSsbylzi9EnSQRCHuXBEzSWvtWKW5a63Na6zKt9bNa677ADcAULv2Kb8ywvSmAv1LKu8qyMOAcgNb6hNb6biAYeBH4QinlWc/5hWgxkgiEvXsLeF4p1QlAKRWklJpW8X6MUipCKeUI5GI8qjFX7HcB6NqQE2itk4CtwN8rKoD7Y9wFrKw4zz1KqSCttRnIqdjNXM/5hWgxkgiEvXsNWAP8qJTKA7YDQyrWtQO+wPgSjgc2YTyuqdxvhlIqWyn1egPOczfQGePu4GvgmSrNVycCh5VS+RXHvUtrXVTP+YVoMUomphFCCPsmdwRCCGHnJBEIIYSdk0QghBB2ThKBEELYuWtuiInAwEDduXNnW4chhBDXlN27d2dorYNqWnfNJYLOnTsTFxdn6zCEEOKaopQ6W9s6eTQkhBB2ThKBEELYOUkEQghh5665OgIhrmVlZWUkJydTXFxs61DEdcrNzY2QkBCcnRs+UK0kAiFaUXJyMt7e3nTu3BmllK3DEdcZrTWZmZkkJyfTpUuXBu8nj4aEaEXFxcUEBARIEhBWoZQiICCg0XeckgiEaGWSBIQ1NeXfl90kggsHL/DzUz9TlC1TvgohRFV2kwiyT2UT+//FknMmp/6NhbiOOTo6EhUVRb9+/YiMjOTll1/GbLbdfDevvvoqhYWFjV5Xn+XLl/PQQw81ap+33nqLFStWNOl81zK7SQTeHYxZAvNS8urZUojrm7u7O/v27ePw4cP89NNPfP/99zz77LNXbFdeXt4q8VgrETTFokWLmDfP/mYDlUQghB0LDg7mnXfeYenSpWitWb58OVOnTmXs2LGMGzeOrKwsbrvtNvr378/QoUM5cOAAAEuWLGHu3LkMGzaMHj168O677wJGq5XHH3+c8PBwIiIiWLVqFQAbN25kypQplvM+9NBDLF++nNdff52UlBTGjBnDmDFjqsVW07off/yRYcOGMWDAAGbOnEl+fj4Au3bt4oYbbiAyMpLBgweTl2f8f56SksLEiRPp0aMHf/rTnyzH9vLy4qmnniIyMpKhQ4dy4cIFy3W99NJLAOzevZvIyEgiIyMt1wRX3mlMmTKFjRs31hnf1c5umo96tvUEBbnncm0dihAArHtkHef3nW/RY7aLasfEVyc2ap+uXbtiMplIS0sDYM+ePRw4cAB/f38WL15MdHQ0q1ev5pdffmHevHns27cPgAMHDrB9+3YKCgqIjo5m8uTJbNu2jX379rF//34yMjIYNGgQN954Y63n/sMf/sArr7zChg0bCAwMrHNdRkYGzz33HOvXr8fT05MXX3yRV155hSeffJJZs2axatUqBg0aRG5uLu7u7gDs27ePvXv34urqSq9evVi8eDGhoaEUFBQwdOhQnn/+ef70pz/x7rvv8vTTT1c7/3333cfSpUu58cYbefzxx+v9HGuL769//Wuj/nvYgt0kAkdnR7zaeskdgRD1uPnmm/H39wcgNjaWL7/8EoCxY8eSmZlJbq7xY2ratGm4u7vj7u7OmDFj2LlzJ7Gxsdx99904OjrStm1bRo0axa5du/Dx8Wl2XNu3b+fIkSMMHz4cgNLSUoYNG8axY8do3749gwYNAqh2rnHjxtGmTRsA+vbty9mzZwkNDcXFxcVyhzJw4EB++umnaufKyckhJyfHksTmzp3L999/36T4rgV2kwjAeDyUn3Jt3KqJ619jf7lby+nTp3F0dCQ4OBgAT0/PBu13eTPFupotOjk5VauQbkrPaq01N998M5988km15QcPHqx1H1dXV8t7R0dHS72Hs7OzJd6qyxuitmupLb5rgd3UEew/v581g9dwPr1lb8WFuJalp6ezaNEiHnrooRq/yEeOHMnHH38MGM/5AwMDLb+4v/nmG4qLi8nMzGTjxo0MGjSIkSNHsmrVKkwmE+np6WzevJnBgwfTqVMnjhw5QklJCTk5Ofz888+Wc3h7e1ue6V+u6rqhQ4eyZcsWTp48CUBBQQHHjx+nV69epKamsmvXLgDy8vKaXdHt6+uLr68vsbGxAJbPAIyh8Pft24fZbCYpKYmdO3fWGd+1wG7uCBJyEviu3Xd0K+xm61CEsKmioiKioqIoKyvDycmJuXPn8uijj9a47ZIlS1iwYAH9+/fHw8ODDz/80LKuf//+jBkzhoyMDP7yl7/QoUMHbr/9drZt20ZkZCRKKf7xj3/Qrl07AO68807Cw8Pp0qUL0dHRluM88MADTJw4kQ4dOrBhw4Zq57983fLly7n77rspKSkB4LnnnqNnz56sWrWKxYsXU1RUhLu7O+vXr2/25/TBBx+wYMEClFKMHz/esnz48OF06dKFvn370qdPHwYMGABAUFBQrfFd7ZTW2tYxNEpMTIxuysQ0u1N2E/NuDLM+ncXH+z/G0cXRCtEJUbf4+Hj69Olj6zCabcmSJXh5efHYY4/ZOpRWkZCQwJQpUzh06JCtQ2mQmv6dKaV2a61jatreao+GlFKhSqkNSqkjSqnDSqmHa9hGKaVeV0qdVEodUEoNsFY8IT4hAOT65JJ/XuoJhBCikjUfDZUD/6O13qOU8gZ2K6V+0lofqbLNJKBHRRkC/LvitcUFeQbhrJzJ9cklLyWPNmFtrHEaIezCkiVLbB1Cq+rcufM1czfQFFa7I9Bap2qt91S8zwPigY6XbTYNWKEN2wFfpVR7a8TjoBxo597OkgiEEEIYWqXVkFKqMxAN7LhsVUcgqcrfyVyZLFBKPaCUilNKxaWnpzc5jhCfEHJ9cqVTmRBCVGH1RKCU8gK+BB7RWjfpG1hr/Y7WOkZrHRMUFNTkWDoFdCLPJ0/uCIQQogqrJgKllDNGEvhYa/1VDZucA0Kr/B1SscwqKu8IJBEIIcQl1mw1pIBlQLzW+pVaNlsDzKtoPTQUuKi1TrVWTCE+IZQ7lZOaZrVTCHHV8/LyumLZK6+8Qt++fenfvz/jxo3j7NmzV2yTk5PDm2++2eTzzp8/ny+++KJR+/zmN7/hyJEj9W8omsWadwTDgbnAWKXUvopyi1JqkVJqUcU2a4HTwEngXeB3VozH0oQ0KTepni2FsC/R0dHExcVx4MABZsyYUW2kzkrNTQRN8d5779G3b99WPac9smaroVittdJa99daR1WUtVrrt7TWb1Vso7XWv9dad9NaR2itG99TrBEqE0FKYYo1TyPENWfMmDF4eHgAxlAJycnJV2zz5JNPcurUKaKioiyjcf7zn/9k0KBB9O/fn2eeecay7YoVK+jfvz+RkZHMnTvXsnzz5s3ccMMNdO3a1XJ3sHHjRkaPHs2MGTPo3bs3c+bMobKj6+jRo6nsQPrBBx/Qs2dPBg8ezP33328ZCvryO42qdzy1xSeqs5shJuBSIsh0yqS0oBQXTxcbRyTs2iOPQMWQzi0mKgpefbVZh1i2bBmTJk26YvkLL7zAoUOHLMNQ//jjj5w4cYKdO3eitWbq1Kls3ryZgIAAnnvuObZu3UpgYCBZWVmWY6SmphIbG8vRo0eZOnUqM2bMAGDv3r0cPnyYDh06MHz4cLZs2cKIESOq7ffMM8+we/du2rRpw5gxY6oNU1GT2uKra1hse2VXiaCdVzsccTR6F6fm49/d39YhCXFVWblyJXFxcWzatKnebX/88Ud+/PFHyxdyfn4+J06cYP/+/cycOdMyv0DlkNYAt912Gw4ODvTt29cyGQzA4MGDCQkxfqhFRUWRkJBQLRHs2LGD0aNHU9lqcNasWfUO6FZbfJIIrmRXicDRwZEglyBLyyFJBMKmmvnLvaWtX7+e559/nk2bNlUbvrk2Wmv+/Oc/8+CDD1Zb/n//93+17lP1uFXHOattuOiGqDostNlsprS0tM74xJXsZhjqSh29OkqnMiEus3fvXh588EHWrFljmZfgcpcPFz1hwgTef/99y3SM586dIy0tjbFjx/L555+TmZkJUO3RUFMNGTKETZs2kZmZSVlZGZ9//rllXefOndm9ezcAa9asoaysrM74xJXs544gNhb+8Q/CbgsmwSdB+hIIu1VYWGh5DAPw6KOPsnbtWvLz85k5cyYAYWFhrFmzptp+AQEBDB8+nPDwcCZNmsQ///lP4uPjLbNweXl5sXLlSvr168dTTz3FqFGjcHR0JDo6muXLlzcr5vbt27NkyRKGDRuGr68vUVFRlnX3338/06ZNIzIykokTJ1om1hk/fnyN8dWW6OyZ3QxDzXffwZQpPPL+nfz75Gq+Kf6GiS9fHTNECftxvQxDbWvLly8nLi6OpUuX2jqUq9JVMwz1VSfU6MAcWuJKqUspaeflFlEIIcCeHg1V3AqHXDTugBKzE20ZjRCiGebPn8/8+fNtHcZ1w37uCPz8wMODkAyjRcG5fKsNaSSEENcU+0kESkFoKCHnjEri86XnudbqR4QQwhrsJxEAhITQPiEThSLbPZuSiyW2jkgIIWzOvhJBaCguiecIcAyQ4aiFEKKCfSWCkBBITaWDe3vpVCbsVlOHobbGeVvL8uXLSUmpebDJutbVZ+PGjUyZMqVR+6xZs4YXXnihSeezFvtKBKGhYDYT6hYkdwRCVNGQYair2rhxY4u02mnMUBLNYa1E0BRTp07lySefbLXzNYR9JYKKJqSdnNtIIhCiioYMQ91YZ86cYdiwYURERPD0009blm/cuJGRI0cydepU+vbtS3FxMffddx8RERFER0ezYcMGwPiCnjZtGqNHj6ZHjx48++yzlmO88sorhIeHEx4ezqsVYzYlJCQQHh5u2eall15iyZIlfPHFF8TFxTFnzhyioqIoKiqybFPTut27dzNq1CgGDhzIhAkTSE01JrI6efIkN910E5GRkQwYMIBTp04BxmB2NQ2h3blzZ5555hkGDBhAREQER48etVxX5RDal39GlXdNl99pPPTQQ5be2bXF1xz2048ALJ3KwsrdKXEr4ULqhXp2EMJ6Hln3CPvOt+ww1FHtonh1onWGoW6shx9+mN/+9rfMmzePN954o9q6PXv2cOjQIbp06cLLL7+MUoqDBw9y9OhRxo8fbxlZdOfOnRw6dAgPDw8GDRrE5MmTUUrxwQcfsGPHDrTWDBkyhFGjRuHn51djHDNmzGDp0qW89NJLxMTE1LmurKyMxYsX88033xAUFMSqVat46qmneP/995kzZw5PPvkkt99+O8XFxZjNZpKSkuocQjswMJA9e/bw5ptv8tJLL/Hee+81+DOqSV3xNYd9JYLKTmUVVQNJGTJTmRBV1TcM9ZAhQygpKSE/P5+srCzLmD8vvvgiEyZMqLbtli1b+PLLLwGYO3cuTzzxhGXd4MGD6dKlCwCxsbEsXrwYgN69e9OpUydLIrj55psJCAgAYPr06cTGxqKU4vbbb7eMKTR9+nR+/fVXpk6d2uzrP3bsGIcOHeLmm28GwGQy0b59e/Ly8jh37hy33347AG5ubtWupbYhtKdPnw7AwIED+eqrK6dtr+szakx8zWVficDXFzw9jU5l3pB0URKBsJ3m/nJvaQ0ZhnrHjh2A8ehi+fLl9Q4mZ0xdfqXKL/H6XL5/bceD6sNRAxQXFzfoHFVprenXrx/btm2rtrzqqKuXq2sI7cp1dQ2tXdM11XYttcXXXPZVR1DZqayibiC1WCaxFwIaNgx1Yw0fPpxPP/0UgI8//rjW7UaOHGlZf/z4cRITE+nVqxcAP/30E1lZWRQVFbF69WqGDx/OyJEjWb16NYWFhRQUFPD1118zcuRI2rZtS1paGpmZmZSUlPDtt99aznH5ENpVVV3Xq1cv0tPTLV+0ZWVlHD58GG9vb0JCQli9ejUAJSUlFBYWNvMTqv0z6tSpE0eOHKGkpIScnBx+/vnnOuNrLvtKBAAhIXQ8Y4yTnmZOQ5uld7GwL5XDUFeWV155hccff9wyDHVUVFSLPGZ57bXXeOONN4iIiODcudqHdPnd736H2WwmIiKCWbNmsXz5cssv6cGDB3PHHXfQv39/7rjjDmJiYhgwYADz589n8ODBDBkyhN/85jdER0fj7OzMX//6VwYPHszNN99M7969LeeYP38+ixYtuqKy+PJ1JpOJL774gieeeILIyEiioqLYunUrAB999BGvv/46/fv354YbbuD8+fNW+4xCQ0O58847CQ8P584777TMsubi4lJrfM1hP8NQV1qwAH74Af8Hi+i6qyublm3CM7hht6lCNJcMQ91w9jjUtJeXl2UineaQYajrU9GprJ1LW2lCKoQQWDERKKXeV0qlKaUO1bK+jVLqv0qp/Uqpw0qp+6wVSzWhoaA1Ia6B0rtYiKvY/Pnz7epuAGiRu4GmsOYdwXKgrinAfg8c0VpHAqOBl5VSLlaMx1DZqczJV+4IhE1ca49jxbWlKf++rJYItNabgbpmrdaAtzLaTnlVbGv9/uYVnco640GRRxEZKRlWP6UQldzc3MjMzJRkIKxCa01mZma1fg4NYct+BEuBNUAK4A3M0lqba9pQKfUA8AAYk2o3S8UdQWi+0XY34UJC844nRCOEhISQnJxMenq6rUMR1yk3NzdLB7eGsmUimADsA8YC3YCflFK/aq2veGivtX4HeAeMVkPNOmubNuDlRUhmmdGpLFs6lYnW4+zsbOlRK8TVwpathu4DvtKGk8AZoHc9+zTfZTOVpRS03qiDQghxNbJlIkgExgEopdoCvYDTrXLmkBA6Jhidys6XNb9TiBBCXMus9mhIKfUJRmugQKVUMvAM4AygtX4L+BuwXCl1EFDAE1rr1qm5DQ3F8/tDeGtvMhwyMJWZcHR2bJVTCyHE1cZqiUBrfXc961OA8dY6f51CQuD8edo6dSXXJ5eCCwX4hPjYJBQhhLA1++tZDJZOZR2dA6RTmRDC7tlnIqhoWhXmKDOVCSGEfSaCyk5lypMCrwIyz2XaOCAhhLAd+0wElcNMlBhVJAmpCbaLRQghbMw+E0GbNuDtTWhmGQCJGYk2DkgIIWzHPhMBVMxUZoz0l5yXbONghBDCduw3EYSEEJJgjIknU1YKIeyZ/SaC0FB8ElJxN7uTZk6zdTRCCGEz9psIQkLgwgWCHYLIds2mrKjM1hEJIYRN2G8iqOxU5hAgfQmEEHbNfhNB5bwEDj6SCIQQds1+E0FFp7IwPMnzziMrua7J1IQQ4vplv4mg4o6gq9kFFBw9eNTGAQkhhG3YbyLw8QEfH8JyTADEx8fbOCAhhLAN+00EUK1T2cnkk2izTCguhLA/9p0IQkLofjoHJ5xIapNExrHWmRdHCCGuJvadCEJD8TibQoRfBEmhSZzbcc7WEQkhRKuz70RQ0alsVLcRnOt4joSdCbaOSAghWp19J4KKJqQjPHtT7lzO9uPbbRyQEEK0PvtOBBVNSIeXtQdgb8leGWpCCGF37DsRVNwRtEsvIswljMSOiZzfd97GQQkhROuy70RQcUdAUhIjOo8gMSyR5O0yN4EQwr7YdyLw9jZmK0tKYkyvMRR6FrJr3y5bRyWEEK3KaolAKfW+UipNKXWojm1GK6X2KaUOK6U2WSuWOoWGQnIyw0OHA7D9vFQYCyHsizXvCJYDE2tbqZTyBd4Epmqt+wEzrRhL7UJCICmJ3oG9aUMb4t3jKcwotEkoQghhC1ZLBFrrzUBdQ3rOBr7SWidWbG+bacIq7giUUsQExBgdy3ZKxzIhhP2wZR1BT8BPKbVRKbVbKTWvtg2VUg8opeKUUnHp6ektG0VICKSlQUkJY/uOJTMwk0M7a32aJYQQ1x1bJgInYCAwGZgA/EUp1bOmDbXW72itY7TWMUFBQS0bRUUTUs6dY1T3UQBsPr65Zc8hhBBXMVsmgmTgB611gdY6A9gMRLZ6FFWakMZ0iMFZO7O7YDday0ikQgj7YMtE8A0wQinlpJTyAIYArT8pQOUdwdmzuDq50s+lH2eCzpB9KrvVQxFCCFuwZvPRT4BtQC+lVLJSaqFSapFSahGA1joeWAccAHYC72mtW//hfPfu4OkJO3cCMKLzCFI6pHBy+8lWD0UIIWzByVoH1lrf3YBt/gn801oxNIiTEwwbBrGxANwcfTNLTyxl476NDL5nsE1DE0KI1mDfPYsrjRgBBw7AxYuM6DwCgO2p0rFMCGEfJBGAkQi0hm3b8Hf3J8wUxiHHQ5hKTbaOTAghrE4SAcCQIeDoaHk8NDhgMIkdE0ndn2rjwIQQwvokEQB4ecGAAfDrrwCMCx9HiVsJv2771caBCSGE9UkiqDRihNFyqKSEm6JuAmDTcduMgyeEEK1JEkGlESOguBj27KGbfzd8y3zZU7DH1lEJIYTVSSKoNNwYhprYWJRSRLpEctz3OMU5xbaNSwghrEwSQaW2baFHD0uF8YhOI7joe5HdW3bbODAhhLAuSQRVjRgBW7aA2cykoZMA+GnPTzYOSgghrEsSQVUjR0JmJhw9yuDug3Etd2XD2Q22jkoIIayqQYlAKeWplHKoeN9TKTVVKeVs3dBsYITRq5jYWJwdnRnlMoodQTs4d0ImqhFCXL8aekewGXBTSnUEfgTmYkxFeX3p3h2Cgy31BH+4+Q+UuJXw71X/tnFgQghhPQ1NBEprXQhMB97UWs8E+lkvLBtRyrgrqEgEtwy5heDCYFadX2XjwIQQwnoanAiUUsOAOcB3FcscrROSjY0YAWfOwLlzKKWYGTiTk0En2bJ1i60jE0IIq2hoIngE+DPwtdb6sFKqK3B91qKOHGm8bjG++B+d+SgOJgde//51GwYlhBDW06BEoLXepLWeqrV+saLSOENr/Qcrx2YbUVHGRDUV4w517dyVARcHsLZsLSXlJTYOTgghWl5DWw39Rynlo5TyBA4BR5RSj1s3NBtxcoKhQy31BAD39rmXfPd8PvzvhzYMTAghrKOhj4b6aq1zgduA74EuGC2Hrk9VJqoBWHDXAnxyfXgn7h0bByaEEC2voYnAuaLfwG3AGq11GaCtF5aNjRgBZjNsN2Yp8/D1YHzxePY47+FMxhkbByeEEC2roYngbSAB8AQ2K6U6AbnWCsrmhg6tNlENwKJRi9BoXl39qg0DE0KIltfQyuLXtdYdtda3aMNZYIyVY7MdLy+IjrZUGAOMmjaKHmd78EnCJ5jMMoWlEOL60dDK4jZKqVeUUnEV5WWMu4Pr14gRsGMHlJYC4OTqxG2et5HunM738d/bODghhGg5DX009D6QB9xZUXKBD6wV1FWhykQ1lRZMW4BHgQev/yB9CoQQ14+GJoJuWutntNanK8qzQNe6dlBKva+USlNKHapnu0FKqXKl1IyGBt0qKieqqfJ4qOfYngw6MYhfcn8hrSDNRoEJIUTLamgiKFJKjaj8Qyk1HCiqZ5/lwMS6NlBKOQIvYgxkd3Vp1w769YPVqy2LHBwdmN1lNiZlYtn2ZTYMTgghWk5DE8Ei4A2lVIJSKgFYCjxY1w5a681AVj3HXQx8CVydP6/nz4etWyE+3rJo8l2TCU0MZem2pdLTWAhxXWhoq6H9WutIoD/QX2sdDYxtzokrhrS+Hah3jGel1AOVFdXp6enNOW3jzJ1rNCP94FJ1SIdBHZhyfAopphTeinur9WIRQggradQMZVrr3IoexgCPNvPcrwJPaK3NDTjvO1rrGK11TFBQUDNP2wht28KUKbBiBZSVAaCU4p7b7qHrqa48+8uzXCy+2HrxCCGEFTRnqkrVzHPHAJ9WPGqaAbyplLqtmcdseQsXwoUL8P2lJqODfjuIW/fcSnZZNi9uedGGwQkhRPM1JxE0a4gJrXUXrXVnrXVn4Avgd1rr1fXs1vomTTIqjt9/37LI2cOZWQtnEXEggn9t/RfncmUqSyHEtavORKCUylNK5dZQ8oAO9ez7CbAN6KWUSlZKLVRKLVJKLWrB+K3PyQnmzYNvv4Xz5y2LYxbFMOXgFMrLy3lm4zM2DFAIIZqnzkSgtfbWWvvUULy11k717Hu31rq91tpZax2itV6mtX5La31FDavWer7W+ovmXozV3HcfmEzw0UeWRc7uzkx7aBoxO2L4YO8HHE47bMMAhRCi6ZrzaMh+9O4NN9xgPB7Sl56IDXxgILccvwXXMleeXP+kDQMUQoimk0TQUAsXwtGjlqGpAZzcnJjw6ASGbxzOtye+ZfPZzTYMUAghmkYSQUPNnGlMYVml0hhgwG8GcFPSTfgV+/H4j4+j9fU7TYMQ4vokiaChvL3hzjvh008hP9+y2MnNibFPjuXGdTeyM2UnXxy5eqs6hBCiJpIIGmPBAiMJfFH9yz56QTQjs0bSMa8jf/75zxSWFdooQCGEaDxJBI0xfOHMMbgAACAASURBVDj06HHF4yEnVydG/b9RjPtqHKeyT/HYj4/ZKEAhhGg8SQSNoZRxV/Drr3D8eLVV0fdFE22OZvyp8fw77t+sPnr19Y0TQoiaSCJorHvvNQaiW7682mJHF0dGPzuawf8ZTG+n3ixcs5Dk3GTbxCiEEI0giaCx2reHyZPh7bfhYvUB5yLnRdJzTE9uefsWSspKmPf1PJnfWAhx1ZNE0BRLlkBWFrxYfcA5pRS3vncrgdmB3B1/NxsSNvCPLf+wTYxCCNFAkgiaIjoa5syBV1+Fc9UHnPPt5MtN/7iJjh93ZLz7eP6y4S/sSN5ho0CFEKJ+kgia6rnnjPGHnrlywLmYB2PoMroLQ18aSkfPjsz+aja5Jbk1HEQIIWxPEkFTde4Mv/udMXvZkSPVVikHxdRlU3EtcGX+jvkk5CTw+7W/t02cQghRD0kEzfHUU+DlBX/+8xWr/Lr6Me7v43D43IEHfB5g5YGVvLr9VRsEKYQQdZNE0ByBgfDkk7BmDcTGXrF68EODCRsRRtj/hjGtyzT++MMf+fjAxzYIVAghaieJoLkefhg6doTHH682RDVUPCJ6fyrmIjO3f3k7ozuNZv4381l3cp2NghVCiCtJImguDw949lljeOqvv75idUCPAMY+P5aENQn8JfcvhAeHc8dnd0hLIiHEVUMSQUu4917o08eoKygru2L10EeG0vPWnsQ+Gsvbnd6mnVc7Jv9nMvHp8TYIVgghqpNE0BKcnOCFF4zxhy4bkA6MR0TTV04nsFcgG+ds5PNRn+Pk4MSElRNIuphkg4CFEOISSQQt5dZbYcQIo19BVtYVq119XLnrm7vQWrPjnh2suX0NF0suMmHlBDILM20QsBBCGCQRtBSl4PXXjSTw4INXVBwD+Hf3Z+ZnM0mPT+f0/5xm9Z2rOZ19mlHLR8kAdUIIm5FE0JKio+FvfzMmrvnwwxo36XpTV8a/PJ5j3xyD5bB2zloSLyYy/P3hHMs41rrxCiEEVkwESqn3lVJpSqlDtayfo5Q6oJQ6qJTaqpSKtFYsreqxx2DUKFi8GE6dqnGTIX8YQtR9UWz+22baxrVl4/yNFJUVMeKDEew6t6uVAxZC2Dtr3hEsBybWsf4MMEprHQH8DXjHirG0HkdH+OgjowL5nnugvPyKTZRSTP73ZEKGhfDN/G9ol9KOLQu24OXixZgPx/DTqZ9sELgQwl5ZLRForTcDV9aaXlq/VWudXfHndiDEWrG0utBQeOsto2/Bc8/VuImTqxOzvpqFR6AHKyesxCfFh60LttLNvxuT/zOZVYdWtXLQQgh7dbXUESwEvq9tpVLqAaVUnFIqLj09vRXDaoZZs2DuXKPOYNu2GjfxaufFvF/m4eTqxEc3fYTzOWc2zd/E0JCh3P3l3fxr27/QNVQ6CyFES7J5IlBKjcFIBE/Uto3W+h2tdYzWOiYoKKj1gmuupUshLMyYuyC35mGo/bv5M++XeaDgw7EfYk4288M9P3B7n9t59MdHmbd6HoVlha0cuBDCntg0ESil+gPvAdO01tdfY3ofH1i5Es6ehT/8odbNAnsFMm/9PEylJlaMW0FJSgmfz/ycv435Gx8f+Jjh7w/nTPaZVgxcCGFPbJYIlFJhwFfAXK31cVvFYXXDh8P/+39Gc9J33611s+DwYOb+NJeS3BJWjF1B/rl8nr7xab6b/R0JOQnEvBsjlchCCKuwZvPRT4BtQC+lVLJSaqFSapFSalHFJn8FAoA3lVL7lFJx1orF5p55BiZOhN/+FtbVPvJo++j23PPjPRSkF7Bi3AryUvOY1GMSu+7fRUfvjkz8eCIvxr4o9QZCiBalrrUvlZiYGB0Xdw3mjLw8GDnS6FsQGwuRtXebSNySyMoJK/Fu7809P9yDX1c/CkoLWLhmIasOr2J6n+m8d+t7+Ln7teIFCCGuZUqp3VrrmJrW2byy2G54e8N334GvL0yeDMm1DykRNjyMuT/NpSiriGXDlpGyOwVPF08+ueMTXh7/MmuOrSHyrUg2n93cihcghLheSSJoTR07GskgN9dIBrW0JAIIHRbKgi0LcHJ34sPRH3Lqx1MopXh02KNsW7gNNyc3Ri8fzdO/PE2Z6cqhr4UQoqEkEbS2/v2NsYgOH4Y776xx/oJKgb0DWbh1IX7d/PjP5P+w/6P9AMR0iGHPg3u4L+o+nv/1eUZ+MJJTWTUPZyGEEPWRRGAL48fD22/DDz/A735X40illbw7eDN/03w63diJ1fNWE/tiLFprvFy8WDZtGZ/N+IxjmceIejuKFftXSEWyEKLRJBHYysKF8NRT8N57RvPSOr7A3dq4MXvtbMLvCufnJ39m7UNrMZWaAJjZbyb7F+0nul00966+lymfTCHxYmJrXYUQ4jogicCW/vY3eOABY3azxx6rMxk4uTox/ePpDHtsGHFvxrF81HIuJl0EIKxNGBvu3cCrE15lU8Im+r7Rl9d3vI7JbGqtKxFCXMMkEdiSUsbgdIsXwyuvwEMPgdlc++YOivH/HM+Mz2aQdjiNt6Pf5sT3JwBwdHDk4aEPc/h3h7mx0408vO5hhr8/nIMXDrbW1QghrlGSCGxNKXjtNeOO4M03jdnNTHX/ku83sx8PxD2AT0cf/nPLf/jl6V8wm4wE0sm3E9/N/o6Pp3/MqexTDHhnAE//8jRFZUWtcTVCiGuQJIKrgVLwj3/A008bdQbz59c4j0FVAT0DWLh9IdELo/n1+V/56OaPyD+fX3E4xeyI2cT/Pp7ZEbN5/tfn6fNGH7488qVUJgshriCJ4GqhlFFn8Le/GQPVzZ5dZ9NSAGd3Z6a+N5Vpy6eRvD2Zt6Pf5uQPJy3rAz0C+fC2D9lw7wZ8XH2Y8fkMbvroJg6nHbb21QghriGSCK42Tz8N//wnfP45TJtWZ6ezSlH3RnH/zvtxD3Dn44kfs+6RdZQXX7qjGN15NHse3MPSSUvZm7qXyLciefj7h8kuyq7jqEIIeyFjDV2t3nnH6GPQuzf897/QpUu9u5QVlbH+yfXsfH0nweHBTP/PdNpGtK22TWZhJn/Z8Bfe3v02/u7+LBm1hPsH3o+Lo4u1rkQIcRWQsYauRQ88YHQ4O3cOBg82Bqqrh7O7M5Nem8Sc7+dQkF7AuzHvsu1f29DmS8k+wCOANye/ye4HdtMvqB8Pff8QvZb2YsX+FdLcVAg7JYngajZuHOzYAX5+MHYsLF/eoN26T+zObw/+lu4Tu/Pjoz+ycuJKcs7mVNsmql0UG+7dwLo56/B39+fe1fcS+VYkq4+ulgplIeyMJIKrXc+eRjK48Ua47z544ok6+xpU8gzyZNbqWUx+azJJW5J4o88b/Pr3Xy09ksFoXTSh+wR23b+Lz2Z8Rrm5nNtX3c7QZUP54eQPkhCEsBNSR3CtKCszprt86y2YMsWY8czfv0G7Xky8yA9//IH4r+IJ6BXALW/cQtdxXa/Yrtxczor9K1iycQlJuUlEt4vmyRFPckefO3B0cGzpKxJCtCKpI7geODsbHc6WLjXqDqKiGlRvANAmrA13fnkns9fOxlxu5qObPuLLu78kLyWv2nZODk4siF7AicUnWDZ1GQVlBcz6Yha9lvbi7bi3KS4vtsaVCSFsTO4IrkVxcXDXXXDmDDz7LPz5z+DYsF/s5cXlxL4YS+zfY3F0cWTUM6MYsngIji5X7m8ym/jm2Df8PfbvxKXE0dazLY8MfYQHBz4os6MJcY2p645AEsG1KjcXFi2CTz6BMWOMTmgdOjR496yTWax7eB0n1p7Av4c/E16ZQI/JPVBKXbGt1poNCRv4e+zfWX96PR7OHtwXdR8PD3mYHgE9WvKqhBBWIongeqU1fPCBMVidl5dRbzBpUqMOcWLtCX549Acyj2XSbUI3JvxrAkF9gmrdfv/5/fxr+7/4z8H/UG4u59Zet/Lo0Ee5sdONNSYRIcTVQRLB9S4+HmbNgoMH4Te/MXom+/o2eHdTmYldb+xi45KNlOaXMuj3gxj9zGjc/d1r3Sc1L5U3d73Jv+P+TWZRJtHtonlo8EPcFX4XHs4eLXFVQogWJInAHhQVwZIl8NJL0LYt/PvfxhAVjVCQXsCGv25gzzt7cPF2YegjQxn6yFDcfN1qP21ZER8d+IjXd7zO4fTD+Lr5cl/UfSyKWUTPgJ7NvCghREuRRGBP4uKM2c8OHDDmRH79dSMxNMKFgxfYtGQT8V/F49rGlaF/rEgIbWpPCFprfk38lTd3vcmX8V9Sbi7npq438buY3zGl5xScHZ2be2VCiGawSSJQSr0PTAHStNbhNaxXwGvALUAhMF9rvae+40oiaICyMmNY6//9X6Pu4LXXYM4cY4TTRji/7zyb/ncTR78+ipuvG0P/OJQhDw+pMyEAnM8/z7I9y3h799sk5SbRzqsd90bey4LoBXKXIISN2CoR3AjkAytqSQS3AIsxEsEQ4DWt9ZD6jiuJoBHi4427g23bYNQoePVVo/9BI6XuTWXTs5s49s0xXNu4Muj3gxjyhyF4tfWqc79yczlrT6xl2d5lfHf8O0zaxMiwkSyMXsiMvjPwdPFs6pUJIRrJZo+GlFKdgW9rSQRvAxu11p9U/H0MGK21Tq3rmJIIGslkMia7eeopyM42KpOfew6Cam8ZVJvUPanE/j2WI18ewdHFkegF0dzw2A34da2/T0FqXior9q9g2d5lnMg6gbeLNzP7zuSe/vcwqvMoHJT0bRTCmq7WRPAt8ILWOrbi75+BJ7TWV3zLK6UeAB4ACAsLG3j27FmrxXzdys42HhUtXQqenvDMM/D734NL44efzjyeydaXtrL/w/2Yy830u7Mfw58YTruodvXuq7UmNjGWZXuX8WX8l+SX5tPRuyOzI2YzJ2IO/dv2l2aoQljBNZ8IqpI7gmaKj4c//tEYpqJXL3jhBaN1URO+fPNS8tj+6nbi3oqjNK+ULmO7MPTRofSY1APlUP/xCssK+e+x/7Ly4ErWnVxHubmc8OBwZofP5q7wu+jiV/8cDEKIhrlaE4E8GrIVrWHtWnj0UTh+HGJijMdF48c3KSEU5xQT93YcO/9vJ3nn8gjoFcDQPw4lcm4kzh4Nay2UUZjB54c/Z+XBlWxN2grA0JCh3NXvLu7sdyftvds3Oi4hxCVXayKYDDzEpcri17XWg+s7piSCFlReDh99ZIxXdPYsjBhhJIRRo5p0OFOZiSOfH2HbK9tI3Z2Ke4A7MYtiiFkUg0+IT4OPk5CTwKpDq/j08KfsO78PB+XA6M6jubPvndza61Y6eDd8KA0hhMFWrYY+AUYDgcAF4BnAGUBr/VZF89GlwESM5qP31fdYCCQRWEVpKSxbZiSBlBS46Sajc9rw4U06nNaaxNhEtr+ynaPfHEUpRY/JPRj44EC6T+yOg2PDK4bj0+NZdXgVnxz6hOOZxwEY0H4At/a8lSk9pzCg/QCpaBaiAaRDmWiYoiKjR/Lf/w4ZGUYi+NOfjPkPHJr2ZZt9Jps97+5h7/t7KbhQQJuwNgy4fwDRC6Lx7uDd4ONorTmSfoT/Hv8v3x7/lm3J2zBrM+282jG5x2Qmdp/ITV1vwtet4UNrCGFPJBGIxikogPffh5dfNh4Z9e4Njz0G99wDrq5NOqSp1MTRb46y++3dnPn5DMpR0XNyTyLvjaTnlJ41DoNdl4zCDL4/8T3fnviWdSfXkVuSi4NyYGjIUCZ0m8CEbhOI6RAjE+oIUUESgWia8nL4/HOjl/K+fdC+PSxebHRSCw5u8mGzTmax+93dHPjoAPmp+bj7uxM+O5yoe6NoP7B9o5uPlpvL2ZG8g3Un1/HDqR+IS4lDo/F392dsl7Hc1OUmxnUdRze/btI0VdgtSQSiebSG9euNhLB+vdH34I474Le/NSqYm/jlai43c3r9afYt38fR1UcxlZgI6htExD0R9Jneh8BegU06bkZhButPr+eHUz+w/vR6knOTAQhrE2ZJCuO6jKOtV+PGYBLiWiaJQLScI0eMeZM//NCYHCc83JggZ+5c8Gl4y6DLFecUc/izw+z/cD9JW5MACOobRO/pvekzvQ/toto16de81poTWSf4+fTPrD+zng1nNpBdnA1Av6B+jOsyjnFdxzGq0yjauLVpcvxCXO0kEYiWV1AAn35qVC7v3g0eHjB9upEQxo1r8NSZNclNzuXo6qPEfxXP2U1n0WaNb2dfet/em9639SZ0eGijWh5VZTKb2Ht+Lz+f/plfEn7h17O/UlRehINyIKZDDGM6j2FE2AhuCL0Bf3f/Jl+DEFcbSQTCunbtMsYz+uwzyMkx6hJmz4Z586B//2YduiC9gOP/PU78V/Gc/uk0plITHoEe9Jzak9639abrTV1xdm/6ENcl5SVsT97Oz2d+5uczP7Pr3C7KzGUA9A3qy/DQ4YwIG8GwkGF08+8mTVXFNUsSgWgdxcXw7bdGJ7W1a43K5v79jXkR7rjDaH3UDCV5JZxcd5Jjq49x/NvjlOSW4OzpTPcJ3el+S3e6T+jeqI5rNSkqK2JXyi5iE2PZkrSFLYlbuFhyEQAvFy8igiOIbBtJ/7b9iWwXSURwBN6uDW8GK4StSCIQrS8jA1atgo8/NobBBujXz0gId9wBERFNrmQGozlqwsYEjq4+yrE1x8g7lwdAUL8guk/sTveJ3QkbEYaTm1OzLsOszRxJP8KO5B3sv7DfKOf3W5KDQtHdvzvR7aOJbldR2kcT7Nn0VlVCWIMkAmFbycnw9dfw5Zfw669gNkP37sZgd5MmGS2Pmtg/AYwK4fQj6Zxcd5JT605xdvNZTKUmnNydCB0WSuiIUMJGhBEyNARX76afp+r5Ei8mWpLC3vN72Xt+Lwk5CZZtOnh3YED7AQxoN4CBHQYyoP0AOnp3lOarwmYkEYirR1oarF5tJIWNG43hLTw9jWEtJk0ySlhYs05RWlBKwsYETv14isRfE7mw/wLarFGOinaR7QgdEUrI0BA6DuqIXze/Fvtyzi7KZt/5fZbEsCd1D0czjmLWZgCCPIIY0H4A4cHh9PDvQXf/7vQI6EGIT4jUPQirk0Qgrk4FBfDLL/D990ZJSDCW9+4NY8YYZfToJk2iU1VJbgnJ25NJjE0kMTaR5O3JlBeVA+DaxpUOAzvQPqY9HWI60GFgB3y7+LZYcigoLeDAhQPsSd3D7tTd7Endw7HMYxSXF1u2cXV0pZt/N3oF9CI8OJyI4Agi2kbQ3b87Tg7Ne7Qlrh1aa7RZYyo1kZuUS/aZbHIScsg5U1EScoiYE8GQP9Q7kWONJBGIq5/WcPSokRDWrzceIeXnG+siIi4lhWHDoF39E+DUxVRmIv1IOilxKaTEpZAal8r5/ecxlxm/3N383Gg/oD3tB7Y3ksTA9vh1bbk7B7M2cy73HCeyTnAy6yQnMk9wMvsk8enxnMg6YbmDcHV0pU9QH8KDw+kT2McoQX3o5tcNZ8emt5QSraessIysU1lkn86+VE4Zr3kpeZjLzJhNZrTJSAI1cXB2wLeTL75dfOl/T38i50U2KRZJBOLaU1Zm9E/YsMG4a9iyxRgUD6BrVyMh3HCDUcLDwal5v5zLS8pJO5hGyu4UUnenkro7lQsHL1iSg6uPK4G9AwnsY5SgPkEE9g7Er6sfDk4t91inqKyIoxlHOZh2kIMXDnIw7SCH0g5xLu+cZRsnBye6+3end2BvegX0Mkqg8RrgEdBisYiG0VpTmF5I5vFM0uPTyYjPMMrRDHLO5kCVr1hXH1f8uvnh19UPnxAfHF0cUY4KB0cHy6uDswM+HX3w7eKLXxc/vNp7NbnfTFWSCMS1r6TESAzbtsHWrUY5f95Y5+kJUVEwcCAMGGC89u7d7ORgKjWRdshIDhf2X7D8z52XkmfZxsHZAf9u/vj3MEpAjwACegbg38MfnxCfFruLyCvJ41jmMeLT4zmacZT4DOP1ZNZJS78HAH93f3oF9KKLXxdCfUKN0sZ4DWsThr+7v1RY18JUaqIoq4jCzEKKc4oxl5nRZm35xW42mTGXm8k7l1ftl3326WxK80stx3FydyKwV8WPht6BBPQKwL+bP35d/XDzc7PZ5y+JQFx/tDZGRt26FbZvhz17YO9eKCw01ru7Q2Sk0Y8hPNwoEREQ2LTxi6oqvlhM5rFLv/6yTmSReSKTrJNZlroHABdvF4LDgwmOCCY4PJi2EW0JjgjGI8Cj2TFUKjeXk5CTwLGMYxzLPMaxjGMczzrO2ZyzJOcmV0sSAJ7OnnT27XxFCfUJJdgzmCDPIDydPa+7ZFGaX2o8c6941l75Pjcpl8LMQoqyiijNK63/QBWc3Jzw62r8sq/8he/f3Z/APoH4dvJt0FStrU0SgbAPJpMx9eaePcbdw549cPAgZGVd2qZtWyMp9OljzNlcWUJCmjznQiVt1uSeyyXrRBYZxzJIP5xO2qE00g6mUZRVZNnO2cMZNz833HyN4u7nbrz3d8Mz2PPKEuSJi7dLo7+czdpMWkEaiRcTSbqYRFJuEokXE0nISeBMzhkSchLIKc65Yj93J3eCPIMI8giirVdbQrxDCPExSmibUEJ8Qujo3REvFy+rJQyzyUxpfiklF0sovlhsvOYUW96Xl5RjKjVhLjNjKjVhKjNhKjVRmldKUVYRRVlFFGcXW95X/cUOxn8D3y6+tAlrg0egB+4B7rj7u+MRYLx383XD0cXReGTjoKo9vvHu4I1XW6+r8su+LpIIhP3S2niEdOiQkRQqX48evVQZDcYdRI8eRunaFbp0ufTaqVOz+znkp+aTdiiNCwcvkJ+aT1F2ESU5xpdbUXYRxTnFFGYU1vqrVDmqS0nD79Krm68brm1ccWtT/VWbNeXF5dVLUTnKQeHs4YyzpzMuni4UuRaR5pBGukrnorpIDjlkm7LJKMsgsziT8/nnOZd3jrSCtCticlJOeDp44qk88cADD7MH7iZ3vE3e+Jb74lPmg0+JUbyLvPEr8sOjxMPyiKXycYup1ERpfmm1UvXOqiEcXRxxcHbA1dsVd/+Kz8jf3VI8gz0tz9x9u/jiEehx3d311EcSgRCXq0wQx44ZdxHHjhnl5EmjGWtJyaVtlYKOHSE01OjjEBZ26X1oqDG2UnBwswbaq1ReXE5BegEFaQUUphdSkGa8L8o2fuEWZxdb3hdlF1l+MZtKTM0+9+UcnBxwdHWkrLCMMscy8rzzyPXJtZRit2JKXEuuKIWehRR4FlDudOWXuUepB0GFQQQVBRFcHEzbkrYElwXj5+KHn6sf/u7+eHt64+rliouny6VE5+uGW5uKxOfjipO7E47OjpbKVnv7Um8KSQRCNIbZDKmpcOYMnD5tlIQESEqCxETjtWqiAOOxUnCwkRTatTNe27Y1lgUHG30hKt8HBBhzOrSg8pJyy+OTktwSlIPCyc0JJ3cn47WioI0Od2UFZcZrYdml9wVl1X+ZFxi/zJ09nXH1dsXFy6XG4uzpfOm9uzPKQaG1JrcklwsFF7iQf4Hz+edJyk3iZNZJTmWf4lTWKRJyEjDpKxOYi6ML/u7++Ln54eXihYezBx7OHni6eBrvnYy/3Z3djVcnd8vfXi5eNZY2rm1wdWp+r/JrmSQCIVqS1pCefikppKbWXNLTjXqLmnh7g7+/kRQCAoz3/v7g62uUNm2qv/f2rl6a2SLqalBuLifxYiKJFxPJKsqqsRSWFVJQVkBhWaHxvrSAgrICisqKKCwrRNPw7y9vF2+CPIOMSnEPow7Ez90PJwcnHJUjjg6OllcnBycj+Th74uniiZeLl+W9n5sfAR4B+Lr5XlM9wiURCGELZrMxLHdampEU0tKMkpFhVGBnZhql8n1WFly8aIzaWh83NyMheHpeKh4e1d+7uxul8r2Hh7Gfq+ulUvVvF5cri7OzUZycriw2fhyjtabUVEpReZElURSWFZJfmn9FyS7KJr0wnfTCdNIK0kgvMN5nF2VTbi7HrM013p3UxQEH/Fx8CHBug7+zD87KkRJTCcWmEopNpZSYSyk2l6LReCgXPJUrHjjjqVzwwBk37UiZuZwyXU6pLqdMmyjV5Zi0CX+zK21N7rQrd6NtqTPtSl1oW+xErxunE3rv4iZ9XnUlgmv/Z4UQVysHh0u/9Bs6BLfWRse5nByjXLxovOblVS+5ucZrYaExVEdlSUkxXouKqhezueWvTykjITg6XkoOjo5GcXC4VJS68n3V19oSSm0/UrUGrVFa42o246o1vmazsdxsblwxmS69B8wKTArKHaDQGQpcoMAZ8l2M9/kukO0GmR6Q6W4m0yOHTPccMj2M/fzLwa0cXE0Vr+WgMI5V6Gwcq9AZLrhAiSM4m8HZBC4m472HCRw0ZHoojnrDBQ9NSZXWxn86By/StERQF0kEQlxNlDJ+uXt4QIcOLXNMrY3B/YqKjDkjSkouvVZ9X1ZmbFe1lJQYX5bl5dVLWZmxvOq6yvd1ffFWfIlT9Ytb69qTQV3La0oolyehqgmotnWVyx0dcagozo6OuDs6ElA10VUmuZqWOTrWfOfk7Fx9XdXXyuWXJ9LKeDHuei6WXORC/gUuFFygnVfzhlepjVUTgVJqIvAa4Ai8p7V+4bL1YcCHgG/FNk9qrddaMyYh7I5Slx7/iGuKUgpfN1983XzpFdjLauexWk2HUsoReAOYBPQF7lZK9b1ss6eBz7TW0cBdwJvWikcIIUTNrFnlPRg4qbU+rbUuBT4Fpl22jQYq5xZsA6RYMR4hhBA1sGYi6AgkVfk7uWJZVUuAe5RSycBaqLkWRCn1gFIqTikVl56ebo1YhRDCbtm6EezdwHKtdQhwC/CRUlc2zNVav6O1jtFaxwQ1c5ISIYQQ1VkzEZwDQqv8HVKxrKqFwGcAWuttgBvQ/OEhhRBCNJg1E8EuoIdSqotSygWjMnjNZdskAuMAlFJ9MBKBPPsRQohWZLVEoLUuBx4CfgDiMVoHHVZK/a9SamrFZv8D3K+U2g98AszX11pXZyGE7c2ukwAABX9JREFUuMZZtR9BRZ+AtZct+2uV90eA4daMQQghRN2uubGGlFLpwNl6NgsEMlohnKuNXLf9sddrl+tuvE5a6xpb21xziaAhlFJxtQ2udD2T67Y/9nrtct0ty9bNR4UQQtiYJAIhhLBz12sieMfWAdiIXLf9sddrl+tuQddlHYEQQoiGu17vCIQQQjSQJAIhhLBz110iUEpNVEodU0qdVEo9aet4rEUp9b5SKk0pdajKMn+l1E9KqRMVr362jNEalFKhSqkNSqkjSqnDSqmHK5Zf19eulHJTSu1USu2vuO5nK5Z3UUrtqPj3vqpiOJfrjvr/27u3EKvqKI7j31+TD5KhZSWSyRAKUWSjRFj5YEIhKRUUSRhIBJFEGXTRegkiH+qhi9VL9yArotKiB1FUKii6mKZFvRS+yOgoZSWElP16+K9pDpODI7jn1H+vDwxn7/8ZDv/F/GfWvsxeS+qRtF3SB7FffdySdkvaJWmHpC9jrJF1XlUiGGUznFq8AiwcNrYK2Gx7JrA59mvzJ3CP7fOBucAd8TOuPfbDwALbFwF9wEJJc4FHgSdszwB+phRyrNEKSqmaQW2J+wrbfR3PDjSyzqtKBIyuGU4VbH8E/DRs+FpK60/i9boxndQYsN1v+6vY/o3yx+FsKo/dxaHYHRdfBhYAb8d4dXEDSJoGLAJeiH3RgrhH0Mg6ry0RjKYZTs2m2O6P7b3AlG5OpmmSeoHZwGe0IPa4PLIDGAA2AT8AB6PAI9S73p8E7gf+iv3JtCNuAxslbZN0W4w1ss4bLTqXuse2JVX7v8GSJgDvAHfb/rUcJBa1xm77CNAnaRKwDjivy1NqnKTFwIDtbZLmd3s+Y2ye7T2SzgI2Sfq+880Tuc5rOyMYTTOcmu2TNBUgXge6PJ9GSBpHSQJrbb8bw62IHcD2QWArcCkwSdLgAV2N6/1y4BpJuymXehcAT1F/3NjeE68DlMR/CQ2t89oSwWia4dTsfWBZbC8D3uviXBoR14dfBL6z/XjHW1XHLunMOBNA0njgSsr9ka3ADfFt1cVt+wHb02z3Un6ft9heSuVxSzpF0qmD28BVwDc0tM6re7JY0tWUa4o9wEu2V3d5So2Q9AYwn1KWdh/wELCe0vpzOqVU9422h99Q/l+TNA/4GNjF0DXjByn3CaqNXdIsys3BHsoB3Fu2H5Z0LuVI+XRgO3Cz7cPdm2lz4tLQvbYX1x53xLcudk8GXre9WtJkGljn1SWClFJKx6e2S0MppZSOUyaClFJquUwEKaXUcpkIUkqp5TIRpJRSy2UiSClIOhKVHge/TljhOkm9nZViU/ovyRITKQ353XZftyeR0ljLM4KUjiHqwj8WteE/lzQjxnslbZG0U9JmSdNjfIqkddE74GtJl8VH9Uh6PvoJbIwnhJF0V/RX2CnpzS6FmVosE0FKQ8YPuzS0pOO9X2xfCDxDeXId4GngVduzgLXAmhhfA3wYvQPmAN/G+EzgWdsXAAeB62N8FTA7Puf2poJLaST5ZHFKQdIh2xOOMr6b0hTmxyh4t9f2ZEkHgKm2/4jxfttnSNoPTOsseRAlszdFQxEkrQTG2X5E0gbgEKVEyPqOvgMpjYk8I0hpdDzC9vHorIVzhKF7dIsonfXmAF90VNVMaUxkIkhpdJZ0vH4a259QKmICLKUUw4PSQnA5/NNMZuJIHyrpJOAc21uBlcBE4F9nJSk1KY88UhoyPjqADdpge/BfSE+TtJNyVH9TjN0JvCzpPmA/cEuMrwCek3Qr5ch/OdDP0fUAr0WyELAm+g2kNGbyHkFKxxD3CC62faDbc0mpCXlpKKWUWi7PCFJKqeXyjCCllFouE0FKKbVcJoKUUmq5TAQppdRymQhSSqnl/gaIseWH8b0b0QAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 332
        },
        "id": "nbEAkCjehkBo",
        "outputId": "e8e8d8f1-159f-4575-f366-8b92bb63af11"
      },
      "source": [
        "# Plot validation accuracy curves\n",
        "epochs = range(1, len(hista['test_acc']) + 1)\n",
        "\n",
        "\n",
        "plt.plot(epochs, histd['test_acc'], '-', color='purple', label='Dropout technique');\n",
        "plt.plot(epochs,histe['test_acc'], '-', color='red', label='l2 technique')\n",
        "plt.plot(epochs,histf['test_acc'], '-', color='green', label='l2 + dropout technique')\n",
        "\n",
        "plt.title('Test Accuracy');\n",
        "plt.xlabel('Epochs');\n",
        "plt.ylabel('Accuracy');\n",
        "plt.legend();\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1aa1eb50>]"
            ]
          },
          "metadata": {},
          "execution_count": 24
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1a9f6190>]"
            ]
          },
          "metadata": {},
          "execution_count": 24
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f1f1a9f65d0>]"
            ]
          },
          "metadata": {},
          "execution_count": 24
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 1.0, 'Test Accuracy')"
            ]
          },
          "metadata": {},
          "execution_count": 24
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0.5, 0, 'Epochs')"
            ]
          },
          "metadata": {},
          "execution_count": 24
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Accuracy')"
            ]
          },
          "metadata": {},
          "execution_count": 24
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f1f1aa17550>"
            ]
          },
          "metadata": {},
          "execution_count": 24
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hUVf748fcnPRAghVBDT6jSQ4AVFRCUKIqsqGDBrrgi6q6uZX1+ol/dXd21obguNkAUUBFhwUgHBSEktISeEEoaEFIgpJB2fn/cIUxCygQZQsjn9TzzzMy55945N+h85nQxxqCUUko5yqW2C6CUUqpu0cChlFKqRjRwKKWUqhENHEoppWpEA4dSSqka0cChlFKqRjRwKKWUqhENHOqKJSKn7R4lIpJn9/6eC7jeWhF5xIF8PrbPiLiwkit1eXOr7QIo5SzGGJ+zr0XkEPCIMWblJfjo24EzwEgRaWGMOXoJPhMAEXEzxhRdqs9T9ZPWOFS9IyIuIvKiiBwQkXQR+VZE/G3HvERkji09S0SiRKS5iLwJXAN8ZKtNfFTFR9wPfALEAPeW++whIvKb7dqJIvKALd1bRN4RkcMiclJE1tvShopIUrlrHBKREbbXU0Xke1uZTwEPiEiYiGy0fUaqiHwkIh525/cQkRUikiEix0TkZRFpISK5IhJgl6+fiKSJiPvv+XurK48GDlUfPQXcBlwHtAIygem2Y/cDTYA2QAAwCcgzxvwN+BWYbIzxMcZMrujCItIOGAp8bXtMLHcsAvgQCAT6ANtth/8N9Af+APgDfwVKHLyfMcD3gK/tM4uBZ4GmwGDgeuBPtjI0AlYCP9vuPRhYZasVrQXutLvufcA8Y0yhg+VQ9YQGDlUfTQL+ZoxJMsacAaYC40TEDSjEChjBxphiY8wWY8ypGlz7PiDGGLMbmAf0EJG+tmN3AyuNMXONMYXGmHRjzHYRcQEeAp42xiTbPvc3W9kcsdEY86MxpsQYk2cr8yZjTJEx5hDwX6wgCTAaOGqMeccYk2+MyTbGRNqOzcJWQxIRV2AC8FUN7l3VExo4VH3UDlhoa8rJAvZg/UpvjvVFuQyYJyIpIvJ2DZtqJmL96scYkwysw6rFgFWLOVDBOU0Br0qOOSLR/o2IdBaRJSJy1NZ89XfbZ1RVBoBFQHcR6QCMBE4aYzZfYJnUFUwDh6qPEoFwY4yv3cPL9mu/0BjzmjGmO1az0WjONTdVuZS0iPwBCAFesn1pHwUGAnfbajOJQKcKTj0B5FdyLAdoYPcZrljNXPbKl+s/wF4gxBjTGHgZELt771hR+Y0x+cC3WLWO+9DahqqEBg5VH30CvGnrc0BEAkVkjO31MBHpafuCPoXVdHW2r+EYlXzp2twPrAC6Y/Vf9AGuAryBcKyayAgRuVNE3EQkQET6GGNKgC+Ad0WklYi4ishgEfEE9gNeInKzrebzCuBZzf01spX9tIh0BZ6wO7YEaCkiz4iIp4g0EpGBdsdnAw8At6KBQ1VCA4eqjz4AFgPLRSQb2IRVMwBogdXRfAqrCWsd575AP8DqC8kUkWn2FxQRL6yO5Q+NMUftHgdt599vjDkC3AT8BcjA6hjvbbvEc0AsEGU79hbgYow5idWx/RmQjFUDKTPKqgLPYfWnZAOfAvPPHjDGZGM1Q90CHAXigGF2xzdgBcqtxpjD1XyOqqdEN3JSStkTkdXAN8aYz2q7LOrypIFDKVVKRAZgNbe1sdVOlDqPNlUppQAQkVlYczye0aChqqI1DqWUUjWiNQ6llFI1Ui8WOWzatKlp3759bRdDKaXqlC1btpwwxpSfN1Q/Akf79u2Jjo6u7WIopVSdIiIVDsnWpiqllFI1ooFDKaVUjWjgUEopVSMaOJRSStWIBg6llFI1ooFDKaVUjWjgUEopVSMaOJSqx0yJ4cj6I2x6fxMn9p6o7eJccYwxFOZVvGV7UUnRJS7NxVMvJgAqpcpK25NGzJwYYr+O5eThkwAse3YZLfu3pNe9vbhq/FX4tPApc44xhswDmSRFJpESnYKblxv+wf4EhATgH+KPTwsfRKSij7tiGWM4c+oMeRl5ZCdnkx6XTkZcBhnxGaXPBacL8GzpSU6fHI4FH+Nw08Psc9/HkYIjjOg4gqcHPk14SDguUnd+x9eLRQ5DQ0ONzhxX9V3O8Rxivo4hdk4sqVtTEReh0w2d6HlvT9oMbsO+xfuImRND6hbrWMcRHen6x66cPnqa5Mhkkjcnk5eeB4CbtxslhSWUFJWUXt+9oTv+wf40adME7wBvvP29zz37e+PZ2LNGgUVchKZdm9K4TeMqzys4XcDB1QeJi4gjNy0Xv05+pcHMP9ifRq0a1ehzjTEkb04m61AWeRl55KXnlX3OyCM3Pbf0tSku+x3q4uaCbwdffIN9ie0Ry08+P7Hf7KdQrJqHT7YPQUlB+Gb5srffXrI8s+jk14mnwp7igT4P0MSryXllOpV8io3vbOT4zuMVltnb35tON3QieFQwjVo1AiDpVBKbkzczpssYXF1cHb5/eyKyxRgTel66Bg6lrmwpW1LYPG0zO+ftpLigmFYDWtHr3l70uKsHPs19zst/Yu+J0gCTdSgLBAK7B9J6YGuCBgYRNCiIwO7W8kUnj5y0fmWf/YUdl0F2Snbpl2thTsXNNDXh08KH1gNb0zqstfU8oDWnkk8RHxFP3E9xHPn1CMUFxXj4eNCoVSMyD2ZSUmgX0Bq40+yqZnS/szs9J/Qs/WItrzC3kJg5MUROiyRtV1qZYx4+HngEeHCk8xEa+TSip1tPGgY0xNvfmwYBDfD296Zh84YEhATQpF0Too9F8/TPTxOZHMlVza5iVKdRDAwayMDWA2nm2ozMA5kcXneY1W+sZmvzrey8ZSd7Pffi4+HD/b3vp4NvBzLyMjiWfoz92/eTnJJMnlceDbwa0Cy3Gc1zm9Mst5n1Oqc52YnZ7Gc/SUFJpPdI50jLI6RLOgA7Ju2gV/NeF/S318ChgUPVFampEBl57hEbCw0agL8/+PtT3MSP3CJPTua4UtQiCI8+3fAZ0pdGA7ohbtYvy+LCYvb8sIfN0zaT+FsiHj4e9H6gNwP+NIDAbuetWVchYwwn9p6gcevGeDaubpvzihWdKSI/M5/c9FwKsgtqdG5xYTHHYo5ZtZ3IZNL3p5+XJ7BHIMHhwYTcFELbq9vi6uFKSXEJJ4+cJCMuozSoJW5IJCUqBQQ6DO9Ar3t70e2P3fBs7EnW4SyiPo5i66dbyc/Mp0WfFoRNCSNoYBBe/l7E5sUyd89c5u+aT1quFVDa+7bn7qvu5t5e99ItsFtpeZJPJfPiqheZEzOHFj4t+Mf1/2Bi74mVNkPlZebxy//9wuYPN3O0/VES7k9gNaspKC7AxbjgneuNd543AY0CaNepHQVuBcRnxJN0quzuwYJgsL7LA/MCaXGgBa0TW9PxZEde+OYF2vZtW6O/fel1NXBo4FCXqdOnYeFC+N//YNMmSEy00t3cML17k9++K7kpJylMPoY5kY5b7im8ycObPFw598u6CDdOeTYl1y+IrFMuFOQW4tnIk8AegfiHBODm4WoFoOBgCAmxntu3B3f32rnvGsrLyCN50xHSV8fg7edF29v64Ns9CBxshkrfn15ak8pMyMTNy42W/VuStDEJBLr9sRv9J/fHp58PqadT+XHvj8yJnUN8Rjyerp7c2uVW7u55N6cLTjMnZg4rElZQYkro17If9/S8h9MFp3lrw1sUlxTz58F/5qUhL9HIs+LaTXkn9p1g+V+WE7c0Ds+Onpw+cRq3HDf6PdCPa/52DX7tfSElBU5a/VG5RXkcyEkkLvsw8aePkEchoe0GExYyjMDGLcjPyidhZQIHlh8gfFo4bl4X1p2tgUMDh7qcFBXBihUwZw78+CPk5kJQEFx9NYW9+pFkWrNzvztxK5PITrE24/Py8yJoYJDVXDOwNa36taD40BFy1m/nzNZdsD8Ot6RDNMhMwoszuPt44OblRpmv1VOnINtucz9XV+jQATp2hKZNISDAqtmcfW7SBHJyID0dMjLOPWdkQGENm6Hc3EprTaXX9/cHX1+rHOUVF8PhwxAfD3Fx1iMhAc6cOZfH3f3cdfz9oWVLCA7GBAdzIKgBkd4ZRObsJS4jHvvvuvysfLJTs8nNzKU4sJiCxgVkFWZx8szJ0jyCMLT9UO7tdS+3d7v9vL6Ho6ePMm/nPObEzGFL6hYAbu92O/8a+S86+HWo2d8GwBgOzv2NvW/9jxaNTtPtKje80hKt+46Ph7w8x67j61v27/vJJ9YPhAuggUMDh7rECnMLyThga/vfk8qZbXswe/fT7HgswVlb8C48RYGnD8e7DyXzDzeT1bI7B5YnkPhbIiVFJXg28Szt8Gx7TVv8g/1//6glYyAt7dwX8dkv5YMHzwWFrCwAClxhRUfoeRzanv0+9fGxvpT8/MCz4uarMy4lJHifIa5hPvEN8olrmE9cg3xyXIrof8yVgQcLGRSfT3AGOHQ3Xl5W7ci+puTmdi6ApadTmHGCDUUJ/OKSaAWL1ob0BtbpDQqhW7YXbub8T3Mx4Fvoin+BKwEF1rN/gSsBZ1y5Jr8ZbRq2Oj/YeXufd529hamcMUX09mjjyB1ZCgvh0KGy/xa5ueeOu7tbAT0k5Nx9BwRUXMM6cwYyM88P7unp8MMP1o+SC6CBQwOHuohMiWHHVzv45f9+IffEuf/ZXUwRzYtTaVmciF9+Kv5kEEA6TThZ+iVZ7OLOEb9e7Pbsx96CDuRkFpaOzGneuzkhN4UQHB5Mm8FtcHG79EM0TWEhS7Z/y19+fYW47EMAXNdyMPf0uY9xPcfj5+1XJn9hcSEbEjcQERdBRHwEO4/vLG1vB/D39ifYPxgvNy+2pGwhpzDHSvf0I8yvB4MbdmWk/wDCmnTDVexqHiLWF17r1uBy/t8h+VQyEfHWZ65MWMmpM6cQhO6B3RnYuBsDS1oxMMObHgdzcEs5agVNh/8Ixqqd2X8BZzthG3Y3t7LB4WyACAmBtm0rroldQho4NHCoi+TIhiMse2YZKdHJdOnpQbfWp2hydC++R/fSOO0ALsXWxK5C70YUtmoPIcF49O6OW89u1hdCt27Q6FzbtzGGguwCSopK8PY//9fspbTr+C6eXfYsKxJW0CWgC68NfY396fuZEzuH/en78XD1YHTn0Uy4agJZ+VllvrTdXdwZ0nYI17S9hs4BnQn2DyYkIAR/b//S6xeXFLM7bTeRyZFEJkUSmRzJrrRdlJgS/L39ubHTjYQHh3Nj8I00a9is9JwjJ48QlxFHfEY8+9P3s+bQGmKOxQDQpnEbwoPDCQ8JZ3iH4TT2bOycP05hoRVE7JvKfg8XF2jRwgoelykNHBo41O90MvYwsU99RuG69bTzPEZbj6O4ZWdaBxs0gP79YeDAc48gxztua1t6bjqvrn2VT6I/oZFnI6ZeN5U/DfgT7q5Wx7kxhi2pW5gTM4e5O+dyPMeaTxDUOIjw4HBuCrmJ6ztc73BnsL2MvAxWHFhRWns4nnMcQejdojd5hXkkZCZQWHKuP6WBewPCWodxU/BNhIeE0yOwR72beHipaODQwKFqorAQs2MHZ5b/QtEvG5DoKBqmW6OdDILp2hWXwYPOBYmrrrqsfzkWlxQTlRLF9qPbycjLICMvg/S89NLXscdiyS7IZlL/Sbw27DWaNmha6bWKSorYcGQD/t7+XNXsqov6pV1iStiWuo2I+AjWHlpLE68mhPiHEOIfUlqDaenTUgPFJaKBQwOHqooxnPphOVlvfUqDhFh8Mw7iZqxfuadpSBJBFPXsR/sX78Jn9DBo7KTmkIvoRO4JlsUvIyI+gp/jfyY979w8iAbuDfD39ifAOwB/b3/aNGnD8394nquaXVWLJVaXm8oCh1N/IonIKOADwBX4zBjzz3LH3wOG2d42AJoZY3xFZBjwnl3WrsB4Y8yPIjITuA44O87jAWPMdifehrqSnTkD335L4T/fofHuHXjjRpp3W+Laj+BMtz7IoEH4hHWnZfdAmrQ5fymI2lBiSjiZf7LCmkNGXgbpuelEJkeyOXkzBkNgg0BuCrmJ8OBwhrQdQmDDQLzcvGr7NlQd5rTAISKuwHRgJJAERInIYmPM7rN5jDHP2uV/CuhrS18D9LGl+wPxwHK7yz9vjPneWWVX9UBqqjW+/ZNP4PhxTrkEst33dnqt+BetQjvQqpaLl56bzubkzUQmR7Lt6DaO5xwnPdcKEJn5mZSYkkrPbeLZhK5NuzJ16FTCg8Pp36p/nVpAT13+nFnjCAPijTEJACIyDxgD7K4k/wTg1QrSxwERxpjcCo6pK1VBwbmhkGeHQ54+bU1Isx9T7+dn9S2UlJwbPll+HHv51+npsHUrFBWREzaURSdvJj2oD/etmohvO99LfqvGGPac2MOqhFXWaKPkSOIz4gFwERe6Nu1K60atadekXWnT0tlHQIOAMk1Oft5+uLlcvn0t6srgzP/CWgOJdu+TgIEVZRSRdkAHYHUFh8cD75ZLe1NE/h+wCnjRGHPe+DgReQx4DKBt2wtbp0VdQqdPWzOov/4a1q+33jvKx8eaVVtcXHmexo3LBpynniKu/Qjm/SWawO6BPLjs3goX/KupwuJCYo/HEpUchcGUduq2adKmzK/+0wWnWX1wNT/F/UREfARHTh4BoKVPSwYFDeKRvo8wMGgg/Vv2v6CRSko50+Xy02Q88L0xpsz/+SLSEugJLLNLfgk4CngAM4AXgNfLX9AYM8N2nNDQ0Ct/BEBdVFQEK1day24sXGjNmm3XDu6/3xrfXn5pCh8fq1ZRvhaRlQUNG5bNGxBAvnhT1LAJXh1b4uZTdn7E1s+2suTxJQQNDuLuJXfj5Xthbf7Jp5LZkLihdE7CltQt5Bfln5fP09WTTv6dCPYPJqcgh1+P/EpBcQE+Hj6M6DiCv13zN27sdCNtm7TVEUPqsufMwJEM2M+/D7KlVWQ88GQF6XcCC40xpYO4jTGptpdnRORL4LmLUFZ1KWVnw1tvwaefwvHjVnPTfffBvffCH/5Q4SxhR5QUl5ASnWJbbnsXKdEpnJ3A7N7AvXR/CA8fDxI3JBI8Kpg7F9yJe4OaLfJnjGFlwkqmbZ7G0v1LMRi83Lzo17IfT4Q+wcDWAxkYNBBXcSU+I564jDji0uOIz4wnLj0OF3FhStgUwkOszmoPV48Lul+laoszA0cUECIiHbACxnjg7vKZRKQr4AdsrOAaE7BqGPb5WxpjUsX6WXYbsPNiF1w5SUkJzJoFL78MR4/C2LFW7WLUqErXPapOUX4Re3/cy/4l+4n/OZ689DzERWg9sDVDXxtKw2YNK9yIZ8CTA7jx3Rtx9XB8SYfTBaf5asdXfLj5Q/ac2EOzhs145dpXGNNlDL2a9yqdLGevTZM2DOswrIKrKVV3OS1wGGOKRGQyVjOTK/CFMWaXiLwORBtjFtuyjgfmmXITSkSkPVaNZV25S38tIoFY66NtByY56x7URfTrr/DMM1an9ODBsGgRhIVd8OWyU7KJ+k8UW/67hdy0XBoENiDkphBCbgqh48iONAhocFGKnZGXwebkzSyLX8aX27/k5JmT9GvZj1m3zeKuHnfh6XZhAU+pukwnACrnSkiAl16Cb7+1luB46y2YMOGCluIwxpC0KYnN0zaz+/vdlBSX0OXWLoQ9FUaHYR0Ql9/XN5BbmMuu47tKRzZFJkUSlxEHgKu4Mq77OKYMnMLgoMHaD6HqhVqZAKjqmdxcq0Zhv3vdkSPWMtRTp8Lzz1trOtWAMYa0XWnERcSx+9vdpESn4NnEk7ApYYQ9GYZfR7/qL1JOem466w6vs/odzvZBZMSRkp1SmqeFTwsGth7Ig30eZGDQQEJbhTpv8Tyl6hgNHOr327kTHn0UoqLODYlt1w4GDYKnn4Y77oA2ju9TcCb7DAdXHSQuIo74iHhOJZ4CrCXHb/r4Jnrf1xsPnwvrUP7l8C/c8d0dpYv0NWvYjGD/YEZ2HEmIfwhdmnYhrHUYbRq30VqFUpXQwKF+n+XLrcDQsCG8+KK14F9YGDRvXuVpxhhOp54u3RM6I872iM8gbU8aJYUleDTyoOOIjlz3/64jeFQwjYMu/Be/MYYPIj/gueXP0cm/E9/d8R29m/c+b1c3pVT1NHCoC/fpp/DEE9CjByxZ4nCtIn5ZPMv/spy0XWmlaS7uLvh19CMgJIDg8GA63diJtle3rdGop8rkFOTw2JLH+Cb2G8Z0GcOs22ZpwFDqd9DAoWqupMQaUvvWWxAeDvPnl9mYqDIn9p1g+V+WE7c0Dr9Oftz4/o0EdgvEP8SfJm2b4OJ68ddTOpBxgLHzx7Lz+E7eHP4mLw55UddtUup30sChaiYvDyZOhO+/t2ob06ZVuw9FXmYe615fR9RHUbh5uzHi7REMnDIQN0/n/ednjGHh3oU8vPhhBCHinghuDL7RaZ+nVH2igUM5bu9eePBBa7TUO+/As89WOazWGMPWT7ey6uVV5GXk0e+Rfgz7v2EXZU2oyuQU5PBVzFdMi5zGnhN76N28NwvvWkgHvw5O+0yl6hsNHKpqJSUQEWHVLJYvt4bTLlhgzfquQs7xHBY9uIi4n+Jod107Rr0/ihZ9WjitmAczDzI9ajqfb/ucrPwsnaSnlBNp4FAVO3kSZs6Ejz6C+Hho3RrefNMadhsYWOWp8T/H8+MDP5KflU/4h+EMeHKAU4a2GmNYc2gN0yKnsXjfYlzERSfpKXUJaOBQ55s+3Rpae/q0tejgm29aNQz3qhcDLMovYuVLK4l8P5JmVzXjvhX30bxn1cNyL0RuYS5zYuYwLXIau9J20bRBU16+5mUmhU4iqHHQRf88pVRZGjhUWe+9B3/+s7Xw4P/9H4Set9pAhdJ2p7FgwgKOxRxjwOQBjHx7JO7eNVt1tjqHsw4zPWo6n239jMz8TPq06MOXY75k/FXjdStUpS4hDRzqnPfft4LGuHHwzTfV1jDOOrD8APNum4eHjwcTlkyg882dL0pxSkwJ21K3lW52tClpEy7iwthuY5kSNoUhbYdoc5RStUADh7J88IE1Sur222sWNFYcYN6YeQR0CeCeiHto1PL37VZ3uuA0S/cvJSI+goj4CI7nHEcQQluF8up1r/Jg3wdp20R3dFSqNmngUNaIqWeesYLG3LkOB42ElQnMu3UeAZ0DmLhyIg2aXvhS5gcyDvDR5o/4YvsXnDpzCj8vP0YFjyI8OJwbg2+kWcNmF3xtpdTFpYGjvvvwQ2shwj/+scZBY+4tc62gserCgoYxhlUHVzEtchpL9i/B1cWVO7rfwaTQSVzd5mpcXX7/ciNKqYtPA0d99uGHMGWKNWJq3jzHg8YqK2j4h/hfUNAoKC5g9o7ZvLvxXfac2ENgg0BeufYVJoVOolWjVhdyJ0qpS0gDR301fboVNG67reZBY/SFBY3C4kJm7ZjFm7++yaGsQzpJT6k6yqmBQ0RGAR9gbR37mTHmn+WOvwec3ZC5AdDMGONrO1YMxNqOHTHG3GpL7wDMAwKALcB9xpgCZ97HFWf6dJg8GcaMsRYo9HBsb4vd3+9m4cSF+AdbQaNhYEOHzissLmT2jtm88esbHMo6xIBWA/j4po8ZFTxKR0UpVQc5LXCIiCswHRgJJAFRIrLYGLP7bB5jzLN2+Z8C+tpdIs8Y06eCS78FvGeMmScinwAPA/9xxj1ckeyDxrffOhQ0Ck4X8PMzP7Pt8220DmvNhP9NcChoFBYX8lXMV7zxyxsczDpIaKtQpt80nfDgcA0YStVhzqxxhAHxxpgEABGZB4wBdleSfwLwalUXFOvbZjhwty1pFjAVDRyO+fhjK2jceqvDQSNlSwoLJiwgIz6DIS8PYejUobi6V91pXT5g9G/Znw/DP+SmkJs0YCh1BXBm4GgNJNq9TwIGVpRRRNoBHYDVdsleIhINFAH/NMb8iNU8lWWMKbK7ZutKrvkY8BhA27Y67p///AeefBJuuQW++67aoGFKDL/9+zdWv7Ian+Y+3L/6ftoPbV/lOUUlRXy14yve+PUNEjIT6N+yP9PCp3FzyM0aMJS6glwunePjge+NMcV2ae2MMcki0hFYLSKxwElHL2iMmQHMAAgNDTUXtbR1zSefwJ/+BKNHOxQ0slOzWXjfQg6uOki327txy4xb8Pb3rjS/MYY5MXN4bd1rHMg8QL+W/Vg8fjGjO4/WgKHUFciZgSMZsN9LNMiWVpHxwJP2CcaYZNtzgoisxer/WAD4ioibrdZR1TUVwOLFVtC4+WZr8yXPqkcv5WXmMfv62Zw8fJJbPruFvg/1rfbL/92N7/Lciuc0YChVTzgzcEQBIbZRUMlYweHu8plEpCvgB2y0S/MDco0xZ0SkKXA18LYxxojIGmAc1siq+4FFTryHum3XLrjnHujf36ppVBM0is4UMX/sfDLiM7hvxX20v659tR8RERfBX1f+lXHdxzF/3HzdllWpesBp/5fbagSTgWXAHuBbY8wuEXldRG61yzoemGeMsW9O6gZEi8gOYA1WH8fZTvUXgD+LSDxWn8fnzrqHOi093eoE9/GBH38E78qbmsBqblr80GIOrzvMbTNvcyho7D2xl/ELxtOreS9mjpmpQUOpesKpfRzGmJ+An8ql/b9y76dWcN5vQM9KrpmANWJLVaawEO64A5KTYd06axOmaqx+ZTWx38Qy/M3h9Ly7wj99GZl5mYyZNwZPV08WjV9EQw/H5nQopeq+y6VzXF1Mf/4zrFkDs2bBwAoHspWx5dMtrP/7evo+0pchLw2pNn9RSRETFkzgYOZBVt+/WlerVaqe0cBxpZkxw9ru9bnnYOLEarPH/xzP0ieWEjwqmJs/dmzY7AsrXmDZgWV8dstnDGlbfaBRSl1ZtFH6SvLLL9ZcjVGj4J//rDb70e1H+e6O72jesznjvh1X7cQ+gJnbZ/LupneZEjaFh/s9fDFKrZSqY7TGcaVISrL20+jUyVoe3bXqIJCTlsPcW+bi5evF3UvvxrNRxSOuSkwJ+9P3E5kUSWRyJJ9v+5wRHUfwzo3vOOMulFJ1gAaOK4Ex8OijkJsL69eDr2+V2UuKS/jh7h/IScvh4d8eplGrsrv2JZ5M5HotznYAACAASURBVNOtn7IpaRNRKVFk5WcB0MijEaOCR/HlmC9xc9H/dJSqr/T//ivBzJnw88/WTn5dulSbfe2ra0lYmcAtn91Cy34tyxwrKinilrm3EHs8lp7NenJn9zsZGDSQga0H0rVpV91cSSmlgaPOS0qy9gq/5hqrf6Ma+5fs59c3f6Xvw33p93C/845/HPUxO47t4Ls7vmNc93HOKLFSqo7TzvG6zBh4/HEoKIAvvgCXqv85MxMyWXjfQlr0bUH4h+HnHU/NTuWV1a9wQ6cbuL3b7c4qtVKqjtMaR102ezb89BO8/z4EB1eZtTCvkG9v/xaAOxfcibv3+Tv+PbfiOc4Un+Gj8I90rSmlVKW0xlFXJSfD009bTVRPPVVt9p8m/8TR7UcZO2csfh38zju+5uAavon9hheufoGQgBBnlFgpdYXQwFEX2TdRff55tU1UWz/byvYvtnPNK9fQ+ebO5x0vKC7gyZ+epINvB14a8pKzSq2UukJoU1VdNGcOLF0K770HIVXXDg7/epifnvyJjiM7MnTq0ArzvLfxPfac2MP/JvwPb/eqF0NUSimtcdQ1KSkwZQpcfXW1TVQn9p1g3ph5+Lb35fa5t+Piev4/95GTR3j9l9cZ02UMozuPdlaplVJXEA0cdUlJCdx/P+TnW6OoqpgdnnM8h6/Dv8bFzYV7Iu6hQUCDCvM9u+xZjDF8MOoDZ5VaKXWF0aaquuTtt2HlSmshw87n91WcVZhbyNxb5nL66GkeWPsAfh3P7wwHaxOmH/b8wJvD36SdbztnlVopdYXRGkddsXEjvPIK3HknPPJIpdlKiktYcPcCkqOSuX3u7bQOq3gvjsikSO7/8X46B3TmL4P/4qxSK6WuQE4NHCIySkT2iUi8iLxYwfH3RGS77bFfRLJs6X1EZKOI7BKRGBG5y+6cmSJy0O68Ps68h8tCZiZMmABt2li1jUrmWBhjWPbsMvYt2seoD0bRdUzXCvMt2L2AobOG4uPhw+Lxi/F0q3pLWaWUsue0pioRcQWmAyOBJCBKRBbbbQGLMeZZu/xPAX1tb3OBicaYOBFpBWwRkWXGmCzb8eeNMd87q+yXlbMLGCYnWwsYNmlSadZN729i84ebGfTsIAY+df4GTsYY/v3bv3lh5QsMChrEovGLCGwY6MzSK6WuQM6scYQB8caYBGNMATAPGFNF/gnAXABjzH5jTJztdQpwHKif33AzZsCCBfD3v1e5m9/B1QdZ/pfldPtjN2749w3nHS8qKeKJpU/w15V/5Y4ed7Bq4ioNGkqpC+LMwNEaSLR7n2RLO4+ItAM6AKsrOBYGeAAH7JLftDVhvSciFbaziMhjIhItItFpaWkXeg+1KzYWnnkGbrwR/lJ5P8SZ7DMsemgR/sH+jP1qLOJStinr1JlTjP5mNP/d8l9eGvISc2+fq/M1lFIX7HLpHB8PfG+MKbZPFJGWwFfAg8aYElvyS0BXYADgD7xQ0QWNMTOMMaHGmNDAwDr4yzonB+66y9pbY/bsKmeHr3xxJSePnGTMl2Nwb1B2DaojJ48w5IshrDq4is9u+Yy/X/93XORy+WdXStVFzhyOmwy0sXsfZEuryHigzJrgItIYWAr8zRiz6Wy6MSbV9vKMiHwJPHfRSnw5+dvfYO9eWL4cmjWrNNvBNQeJ/jiaQc8Oou3Vbcsc25KyhdFzR5NbmEvEPRGM6DjC2aVWStUDzvzpGQWEiEgHEfHACg6Ly2cSka6AH7DRLs0DWAjMLt8JbquFINbyrbcBO512B7Xl2DH473/hwQdhROVf9gWnC1j88GL8g/0Z/sbwMscW7V3EtTOvxdPVk98e+k2DhlLqonFa4DDGFAGTgWXAHuBbY8wuEXldRG61yzoemGeMMXZpdwLXAg9UMOz2axGJBWKBpsAbzrqHWjNtGpw5Ay9U2ApXauVLK8k6lMWtX9xa2kRljOGDTR8wdv5YegT2IPKRSHo063EpSq2Uqiek7Pf1lSk0NNRER0fXdjEck50NbdvC8OHWaKpKHFp3iFlDZzHw6YGMen8UYI2cevbnZ/ko6iPGdh3LnD/OoYF7xUuNKKVUdURkizEmtHy6LjlyuZkxA7KyqqxtFOQUsPihxfh18mP4m1YTVX5RPuO+HcfSuKU8N/g53hr5lnaCK6WcQgPH5aSgwFoqfehQCAurNNuql1eRmZDJA+sewKOhBwD/2vAvlsYt5eObPuaJAU9cogIrpeojDRyXk6+/tmaIf/ZZpVkSf0tk87TNhD0VRrtrrYUJE08m8o/1/2Bc93EaNJRSTqdtGZeLkhJr9dveva0Jf5VY/4/1NGzWkOv/cX1p2gsrX8Bg+NfIf12Kkiql6jkNHJeL//3PmrfxwguVLmKYmZDJ/qX76f94/9ImqvVH1jN351ye/8PztPdtfwkLrJSqrzRwXA6Mgbfegg4d4I47Ks0W9XEU4iL0f7w/AMUlxUyJmEJQ4yBeuLrqobtKKXWxaB/H5WD9emu/jY8+AreK/0kKcwvZ9vk2uv2xG41bNwbgi21fsO3oNubePpeGHg0vZYmVUvVYtTUOEblFRMd1OtVbb0HTptZM8UrEfhNLflY+YU9Zo62y8rN4efXLXNP2Gu7qcVel5yml1MXmSEC4C4gTkbdty4Ooi2nnTli6FKZMgQYVT9YzxrD5o80079WctkOs9aheW/sa6bnpfDDqA6SSPhGllHKGagOHMeZerA2WDgAzbTvzPSYijZxeuvrg7behYUN48slKsyRuSOTYjmMMmDwAEWFP2h4+ivqIR/o9Qt+WfSs9TymlnMGhJihjzCnge6zNmFoCY4Gttl371IXKzIT5860mKn//SrNt/mgzXr5e9Ly7J8YYnl32LA3dG/Lm8DcvYWGVUsriSB/HrSKyEFgLuANhxphwoDdQ+e5CqnrffmvNFq+ibyM7JZs9C/bQ56E+eDT0YNG+RSw7sIypQ6fqDn5KqVrhyKiq24H3jDG/2CcaY3JF5GHnFKuemD0bevSAvpU3N0X/N5qS4hIG/GkAx3OO8/iSx+nVvBdPDqi8aUsppZzJkcAxFTi7eRIi4g00N8YcMsasclbBrnjx8fDbb9aIqko6t4sLitny3y2E3BSCX0c/bpt/GyfzT7Jq4ircXd0rPEcppZzNkT6O74ASu/fFtjT1e3z1lRUw7rmn0iy7F+wm51gOYZPD+GLbFyzet5h/XP8Prmp21SUsqFJKleVI4HAzxhScfWN77eG8ItUDJSVWM9WIEdC6daXZoj6Kwj/EH0Lh6Z+fZniH4Tw96OlLWFCllDqfI4EjzX7HPhEZA5xwXpHqgQ0b4NAhmDix0iypW1NJ/C2Rvn/qy8RFE3FzcWPmmJm6x4ZSqtY58i00CXhZRI6ISCLwAvC4IxcXkVEisk9E4kXkxQqOv2e3Nex+EcmyO3a/iMTZHvfbpfcXkVjbNadJXZz9Nnu2NXdj7NgKDxtj+OWNX3Bv6M6q7qvYmLSRj2/+mDZN2lzigiql1Pmq7Rw3xhwABomIj+39aUcuLCKuwHRgJJAERInIYmPMbrtrP2uX/ymsiYaIiD/wKhAKGGCL7dxM4D/Ao0Ak8BMwCohwpEyXhbw8axjuuHFW8KjA1s+2snfhXlr/vTVPRD7B+KvGc3fPuy9xQZVSqmIOLXIoIjcDPQCvsz/wjTGvV3NaGBBvjEmwXWMeMAbYXUn+CVjBAuBGYIUxJsN27gpglIisBRobYzbZ0mcDt1GXAsfixXDqVKXNVMd3HefnKT/T+sbWvNPoHZoXNOfjmz6+xIVUSqnKOTIB8BOs9aqeAgS4A2jnwLVbA4l275NsaRV9RjugA7C6mnNb2147cs3HRCRaRKLT0tIcKO4lMns2tGljbQ9bTmFuId/f9T2eTTyJeiSKvel7mXXbLPy8/S59OZVSqhKO9HH8wRgzEcg0xrwGDAY6X+RyjAe+N8YUX6wLGmNmGGNCjTGhgYGXyQzro0dh2TK4915wOf9P//OzP5O2Kw2f932YsWsGfx70Z67veH0FF1JKqdrjSODItz3nikgroBBrvarqJAP2vblBtrSKjAfmOnBusu21I9e8/MydC8XFcN995x3a9e0uts7YSsiLIbx8+GX6t+zPP0b8oxYKqZRSVXMkcPxPRHyBfwFbgUPANw6cFwWEiEgHEfHACg6Ly2eyLdXuB2y0S14G3CAifiLiB9wALDPGpAKnRGSQbTTVRGCRA2W5PMyeDQMGQLduZZIzD2byv0f/R8vBLZkePJ2ikiLmjZuHh6tOl1FKXX6q7By3beC0yhiTBSwQkSWAlzHmZHUXNsYUichkrCDgCnxhjNklIq8D0caYs0FkPDDPGGPszs0Qkf/DCj4Ar5/tKAf+BMwEvLE6xetGx3hMDGzfDh9+WCa5uLCYBeMXgEDc83H8FvMb3/zxG4L9g2upoEopVTWx+76uOIPINmNMnd70ITQ01ERHR9duIZ5/Ht5/H1JTrd3+bFa+uJINb22gzZdteOTwIzzY50E+H/N5LRZUKaUsIrLFGBNaPt2RpqpVInJ7nZxod7koKoKvv4abby4TNHJP5LLpvU20e7Adf0v/G12admFa+LRaLKhSSlXPkXkcjwN/BopEJB9rSK4xxjR2asmuJN99Z9U0HnqoTPKO2TsoLCzkq7CvyDyRybJ7l9HQo+JJgUopdblwZOa4bhH7exhjLZ3erRuMHm2XbNgyYwt77trD6mOr+fimj+nVvFctFlQppRxTbeAQkWsrSi+/sZOqxPLlsGMHfPFFmbkbR349QtLhJJbcvYRbO9/KpNBJtVhIpZRynCNNVc/bvfbCWkpkCzDcKSW60rz1lrV0erl9N7bM2ELs4FjyTB6vDX0N7UJSStUVjjRV3WL/XkTaAO87rURXks2bYc0a+Pe/wePcnIzc9FxiF8QS/Xw017W7jj4t+tRiIZVSqmYcWuSwnCSgW7W5lFXb8PWFxx4rkxzzVQy7O+zmuOtx/jvov7VUOKWUujCO9HF8iLW0OVjDd/tgzSBXVdm3DxYuhJdegkbnxhec7RTfNnIbHXw7cEvnW6q4iFJKXX4cqXHYz5wrAuYaYzY4qTxXjrPNU1OmlElO3JBITFYMcf5xvBv2Lq4urrVUQKWUujCOBI7vgfyzK9eKiKuINDDG5Dq3aHVYaqq1LtVDD0Hz5mUObZmxhaghUfi4+/BQ34cquYBSSl2+HJo5jrUu1FnewErnFOcK8f771mzx554rk5yXkcempZuI7R7Lg30fpIlXk1oqoFJKXThHAoeX/XaxttcNnFekOu7kSfjkE2tr2E6dyhyKmRNDZK9IiqWYp8KeqqUCKqXU7+NI4MgRkX5n34hIfyDPeUWq4z75xNoa9oUXyiQbY9j02Sa2Dt7KzZ1vJiQgpJYKqJRSv48jfRzPAN+JSArWOlUtsLaSVeXl51vNVCNGQL9+ZQ4lbUxijesasj2zeWbgM7VUQKWU+v0cmQAYZdtsqYstaZ8xptC5xaqjfvzR2h521qzzDkXPiGbzHzbTo2kPhnfQSfdKqbqr2qYqEXkSaGiM2WmM2Qn4iMifnF+0OuiHH6BZM7i+7D7heRl5LN24lNRmqTwz+BldXkQpVac50sfxqG0HQACMMZnAo84rUh2Vnw8REXDbbeBadm5G9H+j2dB3A/4e/tzT855KLqCUUnWDI4HD1X4TJxFxBRzaDFtERonIPhGJF5EXK8lzp4jsFpFdIvKNLW2YiGy3e+SLyG22YzNF5KDdsctjoaeVK+H0aRg7tkxy0ZkiIr6MYF/XfUwKm4S3u3clF1BKqbrBkc7xn4H5InJ2UaXHcWCfb1uAmQ6MxFrfKkpEFhtjdtvlCQFeAq42xmSKSDMAY8warKVNEBF/IB5Ybnf5540x3ztQ9ktn4UJo3BiGl+2/2Dl3J+var8NFXHhiwBO1VDillLp4HKlxvACsBibZHrGUnRBYmTAg3hiTYIwpAOYBY8rleRSYbmv+whhzvILrjAMiLuuZ6kVFsHixtTWs3Sq4xhjWfbCO7QO2M7bbWIIaB9ViIZVS6uKoNnAYY0qASOAQVjAYDuxx4NqtgUS790m2NHudgc4iskFENonIqAquMx6YWy7tTRGJEZH3RMSzog8XkcdEJFpEotPS0hwo7u+wYQOcOHFeM1XCigRWy2pyPXKZHDbZuWVQSqlLpNLAISKdReRVEdkLfAgcATDGDDPGfHSRPt8NCAGGAhOAT0XE164MLYGewDK7c14CugIDAH+sGtF5jDEzjDGhxpjQwMDAi1TcSvzwA3h6Qnh4meQN/95A9NXR9AzsybXtKtxIUSml6pyqahx7sWoXo40xQ4wxHwLFNbh2MtDG7n2QLc1eErDYGFNojDkI7McKJGfdCSy0nzdijEk1ljPAl1i1oNpjjDV/44YbwMenNPlYzDHW7V1HStMUJg+crENwlVJXjKoCxx+BVGCNiHwqItdjzRx3VBQQIiIdRMQDq8lpcbk8P2LVNhCRplhNVwl2xydQrpnKVgvBNtLrNmBnDcp08W3dCkeOnNdMtfHdjUT/IZomHk10CK5S6opSaeAwxvxojBmP1Sy0BmvpkWYi8h8RuaG6CxtjioDJWM1Me4BvjTG7ROR1EbnVlm0ZkC4iu22f8bwxJh1ARNpj1VjWlbv01yISi9VJ3xR4w9GbdYqFC8HFBW45tyFTdko2GxZvYFfXXTzU7yEaejSsxQIqpdTFJcaY6nOdzSziB9wB3GWMub66/JeL0NBQEx0dXX3GC9GjhzVbfM2a0qSVL63kzcg3WXfdOvY/tZ9g/2DnfLZSSjmRiGwxxoSWT3dkOG4pY0ymrdO5zgQNp9q3D3bvLtNMVXC6gMgZkWz/w3bCQ8I1aCilrjg1ChyqnIULrefbbitN2vblNra12kaWexaTB+gQXKXUlUcDx++xcCGEhkLbtgCUFJew6b1N7Lh+B8H+wdwYfGMtF1AppS4+DRwXKjkZNm8u00x1YPkBdufvJt4vnicHPImL6J9XKXXl0W+2C/Xjj9azXeCI+ymOLYO30MC9AQ/0eaB2yqWUUk6mgeNCLVwIXbpAt26lSTGrY4i5Kob7et2Hr5dvFScrpVTdpYHjQqSnw9q1ZWobGQcy2Oi5kUKXQh7v/3jtlU0ppZxMA8eFWLoUiovL9m8sO0B8cDwtvVvSp8XlsUWIUko5gwaOC7F+Pfj7WyOqbPYv28/B4IOM6jJK16VSSl3RNHBciK1boW9fa6kRoLigmF/3/kqeZx6jgitaGV4ppa4cGjhqqrAQYmOhX7/SpMTfEtnbai8uuDCi44haLJxSSjmfBo6a2r0bCgrKBI74ZfHEh8QzoOUA/L39a7FwSinlfBo4amrrVuu5b9/SpB2rdpDSKoVRnbWZSil15dPAUVPbtlkbNoVY+02dPnqaTbmbMGK0f0MpVS9o4KiprVuhd+/SjvEDy61huL7uvgxoNaCWC6eUUs6ngaMmioth+/bz+jcSOidwQ8gNuLq41mLhlFLq0nBq4BCRUSKyT0TiReTFSvLcKSK7RWSXiHxjl14sItttj8V26R1EJNJ2zfm2bWkvjfh4yMkpDRymxLBhywZONTylK+EqpeoNpwUOEXEFpgPhQHdggoh0L5cnBHgJuNoY0wNre9qz8owxfWyPW+3S3wLeM8YEA5nAw866h/OU6xhP3ZrKzgBry/MbOlW7m65SSl0RnFnjCAPijTEJxpgCYB4wplyeR4HpxphMAGPM8aouKNaU7OHA97akWcBtlZ9xkW3bBh4e0N2Kf/HL4okPjqe7f3eCGgddsmIopVRtcmbgaA0k2r1PsqXZ6wx0FpENIrJJROyHJXmJSLQt/WxwCACyjDFFVVwTABF5zHZ+dFpa2u+/G7BqHD17grs7ALuX7yaxXSLhXcIvzvWVUqoOqO3OcTcgBBgKTAA+FZGz65G3s22Sfjfwvoh0qsmFbXujhxpjQgMDA39/SY2xAoetfyP/ZD7rj62nyLVIh+EqpeoVZwaOZKCN3fsgW5q9JGCxMabQGHMQ2I8VSDDGJNueE4C1QF8gHfAVEbcqrukcR45AZmZp4Di46iBxHePwcvFiSNshl6QISil1OXBm4IgCQmyjoDyA8cDicnl+xKptICJNsZquEkTET0Q87dKvBnYbYwywBhhnO/9+YJET7+Gcch3j8cviORBygGEdhuHl5nVJiqCUUpcDpwUOWz/EZGAZsAf41hizS0ReF5Gzo6SWAekishsrIDxvjEkHugHRIrLDlv5PY8xu2zkvAH8WkXisPo/PnXUPZWzdCq6u0KsXxhgiN0SS7p/OqBBtplJK1S9u1We5cMaYn4CfyqX9P7vXBviz7WGf5zegZyXXTMAasXVpbdtmbRPr7U363hNs994OwI2ddP6GUqp+qe3O8brj7B4cwIEV1jIjbX3a0jmgcy0XTCmlLi0NHI44ehRSU0s7xhO3J3Ko4yHCu4Trbn9KqXpHA4cjtm2znm01js1HN3PG44wOw1VK1UsaOBxxdkRVnz4YY9hWvA0xwvAOw2u3XEopVQs0cDhi2zYIDoYmTcg5lsOxhsdo7tqcxp6Na7tkSil1yWngcIRdx3janjQy/TLp0KhDLRdKKaVqh1OH414RMjPh4EF47DEA0nankeGfwfUtrq/lgqn6prCwkKSkJPLz82u7KOoK4+XlRVBQEO62dfiqo4GjOtut+RpnaxxH9h4ht2kuXYO61mKhVH2UlJREo0aNaN++vY7mUxeNMYb09HSSkpLo0MGxlhRtqqpOuaVG9hzeA0Cwf3BtlUjVU/n5+QQEBGjQUBeViBAQEFCjmqwGjups2wZBQdCsGQDx6fEAdPKr0WK9Sl0UGjSUM9T0vysNHNWx6xjPP5lPiqQA0MlfA4dSqn7SwFGVnBzYu7d0xviJPSfI8M/Az81Ph+KqesnV1ZU+ffrQo0cPevfuzTvvvENJSUmtlef9998nNze3xseqM3PmTCZPnlyjcz755BNmz559QZ9X12jgqEpMjLWBky1wnB2K27FJx1oumFK1w9vbm+3bt7Nr1y5WrFhBREQEr7322nn5ioqKKjj74nNW4LgQkyZNYuLEiZfs82qTjqqqSrmO8RN7TpDpn8mgFoNqsVBKwc/P/MzR7Ucv6jVb9GnBqPcdX0anWbNmzJgxgwEDBjB16lRmzZrFDz/8wOnTpykuLmbhwoU89NBDJCQk0KBBA2bMmEGvXr2YOnUqBw4cID4+nhMnTvDXv/6VRx99FGMMf/3rX4mIiEBEeOWVV7jrrrtYu3Yt//73v1myZAkAkydPJjQ0lFOnTpGSksKwYcNo2rQpa9asKS3btGnTzju2fPlyXn31Vc6cOUOnTp348ssv8fHxISoqiqeffpqcnBw8PT1ZtWoVACkpKYwaNYoDBw4wduxY3n77bQB8fHx4+umnWbJkCd7e3ixatIjmzZszdepUfHx8eO6559iyZQsPPfQQADfccAMRERHs3LmTmTNnEh0dzUcffQTA6NGjee655xg6dGil5bscaY2jKtu2QdOmVuc4cHTvUU42OUlwgI6oUgqgY8eOFBcXc/z4cQC2bt3K999/z7p163j11Vfp27cvMTEx/P3vfy/zazwmJobVq1ezceNGXn/9dVJSUvjhhx/Yvn07O3bsYOXKlTz//POkpqZW+tlTpkyhVatWrFmzpkzQqOjYiRMneOONN1i5ciVbt24lNDSUd999l4KCAu666y4++OCD0s/19vYGYPv27cyfP5/Y2Fjmz59PYmIiADk5OQwaNIgdO3Zw7bXX8umnn55XtgcffJAPP/yQHTt2OPR3rKx8lyutcVTF3R2GDQPbiIN9ifsw/Y2OqFK1riY1g0tp5MiR+Pv7A7B+/XoWLFgAwPDhw0lPT+fUqVMAjBkzBm9vb7y9vRk2bBibN29m/fr1TJgwAVdXV5o3b851111HVFQUjRv//v7ETZs2sXv3bq6++moACgoKGDx4MPv27aNly5YMGDAAoMxnXX/99TRp0gSA7t27c/jwYdq0aYOHhwejR48GoH///qxYsaLMZ2VlZZGVlcW1114LwH333UdERMQFle9ypYGjKv/5T+nLovwiDuceBnRElVJnJSQk4OrqSjPbcPWGDRs6dF754Z9VDQd1c3Mr0wF/ITPnjTGMHDmSuXPnlkmPjY2t9BxPT8/S166urqX9Nu7u7qXltU93RGX3Uln5LldObaoSkVEisk9E4kXkxUry3Ckiu0Vkl4h8Y0vrIyIbbWkxInKXXf6ZInJQRLbbHn2ceQ9npe9PJ903HdA5HEoBpKWlMWnSJCZPnlzhF/8111zD119/DcDatWtp2rRp6S/6RYsWkZ+fT3p6OmvXrmXAgAFcc801zJ8/n+LiYtLS0vjll18ICwujXbt27N69mzNnzpCVlVXaBwHQqFEjsrOzKyyf/bFBgwaxYcMG4uOteVg5OTns37+fLl26kJqaSlRUFADZ2dm/u2Pf19cXX19f1q9fD1D6NwBo374927dvp6SkhMTERDZv3lxl+S5XTqtxiIgrMB0YCSQBUSKy2G7vcEQkBHgJuNoYkykizWyHcoGJxvz/9u49OsryTuD490fCHYpchRBwKAaVSxhLlpXLKnCWyxYK4lIojVxKCyccN8JhwUg5YOAYFQ+7RViOp2wXE1QEjULTrsvFAmKBFgKMGBKh3Eq5CAn3UMQYfvvHvBmHkNskGSZkfp9z5mTe573M8wwv85vnfZ/5PfoXEYkC9onIJlW94qyfo6rpwap7SYpGVDWMaEjbJm3v5UsbU2PcvHkTt9tNQUEBkZGRTJgwgVmzZpW4bXJyMlOmTCE2NpZGjRqRlpbmWxcbG8vAgQPJy8tj/vz5REVFMXr0aHbv3k3Pnj0REV5//XXatvX+Xxs7dizdu3enU6dOPO4MVgGYNm0aw4YN893P8Fd8XWpqKuPHj+fWrVsAvPzyy3Tp0oV169aRmJjIbPJZRAAAErRJREFUzZs3adiwIZ988kmV36e33nqLKVOmICIMGTLEV96vXz86depE165deeyxx/iBM2KzdevWpdavJhLvtN9BOLBIHyBZVYc6y3MBVPVVv21eB46o6m/KOdbnwBgnkKQCvw8kcMTFxWlmZmYlWvGd7cnbSTiSQGS/SLKey6rSsYypjJycHB577LFQV6PK/EcfhYOTJ08yYsQIsrJq9udGSeeXiOxT1bji2wbzUlV74G9+y6edMn9dgC4islNE/iQid93xE5HeQD3gmF9xinMJ61ciUr/4Ps5+00QkU0Qyc3Nzq9YSvENxrz5oI6qMMSbUN8cjgRhgABAN7BCRHkWXpESkHfA2MElVi+4ozQW+whtMVgJJwKLiB1bVlc564uLiqtytOp99nouPXrT7G8ZUUXJycqircE+5XK4a39sIVDB7HGeADn7L0U6Zv9NAhqoWqOoJ4AjeQIKIfA/4X2Ceqv6paAdVPadet4C3gN5BbAMAt7+9zV/P/pWCOgU2osoYE/aCGTj2AjEi0klE6gE/ATKKbbMBb28DEWmF99LVcWf79cDq4vcynF4I4h3G8TQQ9FB++cRlcpt6L3dZj8MYE+6CdqlKVb8VkX8DNgERwCpVPSQii4BMVc1w1g0RkWygEO9oqYsi8izwJNBSRCY7h5ysqh7gXRFpDQjgARKC1YYieTl5XG5+GbDfcBhjTFDvcajqx8DHxcoW+D1XYJbz8N/mHeCdUo45qPprWrbcHO90sRESwUPNHrrXL2+MMTWK5aqqgLycPK5HXadjs47UjajYnLzG1EZFSfc8Hg99+vShW7duxMbGsm7duhK3T01N5ezZs5V6re3bt/tSe1RURkYGr732WqVez1RcqEdV3RfycvK42v+qXaYyxtGoUSNWr15NTEwMZ8+epVevXgwdOpQHHnjgju1SU1Pp3r07UVFR96ReI0eOZOTIkffktcKZBY5yqCq5ObnkDctjSPMh5e9gzL0wcyZ4PNV7TLcbli6t0Kb+v2iOioqiTZs25Obm3hE40tPTyczMJD4+noYNG7J7926ys7OZNWsW+fn5tGrVitTUVNq1a8fRo0dJSEggNzeXiIgIPvjgAwDy8/MZM2YMWVlZ9OrVi3feeQcRweVyMWnSJH73u99RUFDABx98wKOPPnpH2vITJ07w05/+lPz8fEaNGsXSpUvJz88vNU375MmT2bdvX4n1M3eyS1XluH72OlcLrnK9znUbUWVMCfbs2cM333xD5853/v8YM2YMcXFxvPvuu3g8HiIjI0lMTCQ9Pd03X8W8efMAiI+P57nnnuPzzz9n165dvg/rAwcOsHTpUrKzszl+/Dg7d+70Hb9Vq1bs37+f6dOns2TJkrvqNWPGDKZPn84XX3xRoQ//goKCUutn7mQ9jnLYiCpTI1WwZxBs586dY8KECaSlpVGnTtnfQw8fPkxWVhaDBw8GoLCwkHbt2nH9+nXOnDnD6NGjAWjQoIFvn969exPtzIfjdrs5efIk/fv3B+CZZ54BvKnNP/roo7teb+fOnb607hMmTCApKalS9TN3s8BRjqIRVWC/4TDG37Vr1xg+fDgpKSk88UT5s2KqKt26dWP37t13lJeW3RZKT23uv66s1OYlZe0tK7V5SfUzd7NLVeUoGlEF8P3mNte4MeCdaGj06NFMnDiRMWPGlLqdf2rzRx55hNzcXN8Hc0FBAYcOHaJp06ZER0ezYcMGAG7dulUtc4X369ePtWvXAnemNi8tTXtp9TN3s8BRjrycPG48dIM2jdvQtH7TUFfHmBrh/fffZ8eOHaSmpuJ2u3G73XhKuFk/efJkEhIScLvdFBYWkp6eTlJSEj179sTtdrNr1y4A3n77bZYtW0ZsbCx9+/blq6+qPp/6G2+8wYoVK+jRowdnznyX7ahDhw6+NO1jx471pWmvV69eqfUzdwpaWvWapCpp1Ze0XULaz9Jo+nBTdv3cTiITOrUlrXqoNGnShPz8/FBXo8aqKWnV73s3L93kxvkb5DbMtRvjxhjjsMBRhtycXL6N+JYLesFujBtzn7PeRvWxwFGGoqG4ivJwC5vAyRhjwAJHmXJzcrn24DXAhuIaY0wRCxxlyMvJ4+su3jHedo/DGGO8LHCUYcDCATQa1ogm9ZrQulHrUFfHGGNqBAscZWj/D+3Ja5JH5+adS/wFqjHhJtC06lUxefJk0tPTy98wCDweDx9//HHA6yrC5XKRl5cX0D59+/at9OsFQ1ADh4gME5HDInJURF4sZZuxIpItIodEZI1f+SQR+YvzmORX3ktEvnCOuUyC/Il+7PIxu0xlTDFFadUPHTrExo0bmTlzJleuXClzH5fLVS2vXVhYWC3HKUswA0dl1LQfIgYtV5WIRAArgMHAaWCviGSoarbfNjHAXKCfql4WkTZOeQvgJSAOUGCfs+9l4E1gKvBnvLMLDgP+LxhtuK23OXH5BD/q8qNgHN6YSpu5cSaer6o3rbq7rZulw6ovrXqgVJXExES2bNlChw4dqFevnm+dy+Vi3LhxbNmyhRdeeAFV5ZVXXkFVGT58OIsXLwa8PaKpU6eyefNm2rZty9q1a2ndujUej4eEhAT+/ve/07lzZ1atWkXz5s0ZMGAAS5YsIS4ujry8POLi4jhy5AgLFizg5s2b/PGPf2Tu3LmMGzcO8KZaKb5uxIgRJCYmkpWVRUFBAcnJyYwaNYrCwkKSkpLYuHEjderUYerUqSQmJgKwfPnyu1LCJycnc+rUKY4fP86pU6eYOXMmzz//vK9d+fn5Jb5HU6ZMYcyYMbhcLjIzM2nVqhWZmZnMnj2b7du3c+PGjRLrVxXB7HH0Bo6q6nFV/QZYCxSv7VRghRMQUNULTvlQYIuqXnLWbQGGiUg74Huq+idn2tnVwNPBasCZa2e4VXjLRlQZU4bS0qoHav369Rw+fJjs7GxWr15917fsli1bsn//fp588kmSkpLYunUrHo+HvXv3+vJc3bhxg7i4OA4dOsRTTz3FwoULAZg4cSKLFy/m4MGD9OjRw1deknr16rFo0SLGjRuHx+PxBY3S1qWkpDBo0CD27NnDtm3bmDNnDjdu3GDlypWcPHkSj8fDwYMHiY+P9x2ntJTwX375JZs2bWLPnj0sXLiQgoKCgN6jkpRWv6oIZnbc9sDf/JZPA/9YbJsuACKyE4gAklV1Yyn7tncep0soD4pjl48BNqLK1DwV7RkEW3lp1VNSUnyTMp09exa32w14ExCuWLHijm137NjB+PHjiYiIICoqikGDBt2xvugDfO/evQwYMIDWrb0DVuLj49mxYwdPP/00derU8W337LPP8swzz3D16lWuXLnCU089BcCkSZP48Y9/XG3vwebNm8nIyPAFgK+//ppTp07xySefkJCQQGSk92O2RYsWvn1KSwk/fPhw6tevT/369WnTpg3nz5/3pZWvyHsUSP2qkr4m1GnVI4EYYAAQDewQkR7VcWARmQZMA+jYsWOljnHskhM4rMdhzF0qklZ93rx5vsmQXC5XiYkQK6px48YB71PeLVD/FOtF6dUDpap8+OGHPPLIIxXep7SU8GWlkS9PaW2pTP3KE8xLVWeADn7L0U6Zv9NAhqoWqOoJ4AjeQFLavmec52UdEwBVXamqcaoaV/TNJFDHLh8jsk4kHZp1KH9jY8JIRdOqB+LJJ59k3bp1FBYWcu7cObZt21bidr179+bTTz8lLy+PwsJC3nvvPV9v4vbt276RWGvWrKF///40a9aM5s2b89lnnwHeTLxF27tcLvbt2wdwxwgu/3TwxRVfN3ToUJYvX05RwtgDBw4AMHjwYH7961/7PvwvXbpUuTfGT1nvkX9biiawKqt+VRHMwLEXiBGRTiJSD/gJkFFsmw14exuISCu8l66OA5uAISLSXESaA0OATap6DrgmIk84o6kmAr8NVgOOXT6G6wEXkXVC3TEzpmapaFr1QIwePZqYmBi6du3KxIkT6dOnT4nbtWvXjtdee42BAwfSs2dPevXq5bvZ27hxY/bs2UP37t3ZunUrCxYsACAtLY05c+YQGxuLx+Pxlc+ePZs333yTxx9//I4hsgMHDiQ7Oxu3233XUOPi6+bPn09BQQGxsbF069aN+fPnA/CLX/yCjh07EhsbS8+ePVmzZg1VVdZ79NJLLzFjxgzi4uKIiIjwlZdWv6oIalp1EfkhsBTv/YtVqpoiIouATFXNcD78/wPvyKhCIEVV1zr7TgF+6RwqRVXfcsrjgFSgId7RVIlaTiMqm1b91c9e5dqta7z6z68GvK8x1c3Sqpcv3FKnT548mREjRlRLry+QtOpB/Sqtqh/jHTLrX7bA77kCs5xH8X1XAatKKM8Euld7ZUsw95/m3ouXMcaY+4pdgzHG1Brh1NsASE1NDcnrWsoRY+4j4TBjp7n3Aj2vLHAYc59o0KABFy9etOBhqpWqcvHiRRo0aFDhfexSlTH3iejoaE6fPk1ubm6oq2JqmQYNGtzxQ8PyWOAw5j5Rt25dOnXqFOpqGGOXqowxxgTGAocxxpiAWOAwxhgTkKD+crymEJFc4K/lbNYKCGxartrB2h1erN3hpartfkhV70r2FxaBoyJEJLOkn9bXdtbu8GLtDi/BarddqjLGGBMQCxzGGGMCYoHjOytDXYEQsXaHF2t3eAlKu+0ehzHGmIBYj8MYY0xALHAYY4wJSNgHDhEZJiKHReSoiLwY6voEk4isEpELIpLlV9ZCRLaIyF+cv81DWcfqJiIdRGSbiGSLyCERmeGU1+p2A4hIAxHZIyKfO21f6JR3EpE/O+f8Omdq51pFRCJE5ICI/N5ZrvVtBhCRkyLyhYh4RCTTKav2cz2sA4eIRAArgH8BugLjRaRraGsVVKl4p+n19yLwB1WNAf7gLNcm3wL/rqpdgSeA55x/49reboBbwCBV7Qm4gWEi8gSwGPiVqj4MXAZ+HsI6BssMIMdvORzaXGSgqrr9fr9R7ed6WAcOoDdwVFWPq+o3wFpgVIjrFDSqugO4VKx4FJDmPE8Dnr6nlQoyVT2nqvud59fxfpi0p5a3G7xTM6tq0ZR4dZ2HAoOAdKe81rVdRKKB4cBvnGWhlre5HNV+rod74GgP/M1v+bRTFk4eVNVzzvOvgAdDWZlgEhEX8DjwZ8Kk3c4lGw9wAdgCHAOuqOq3zia18ZxfCrwA3HaWW1L721xEgc0isk9Epjll1X6u23wcxkdVVURq5fhsEWkCfAjMVNVr3i+hXrW53apaCLhF5AFgPfBoiKsUVCIyArigqvtEZECo6xMC/VX1jIi0AbaIyJf+K6vrXA/3HscZoIPfcrRTFk7Oi0g7AOfvhRDXp9qJSF28QeNdVf3IKa717fanqleAbUAf4AERKfrSWNvO+X7ASBE5iffS8yDgDWp3m31U9Yzz9wLeLwq9CcK5Hu6BYy8Q44y4qAf8BMgIcZ3utQxgkvN8EvDbENal2jnXt/8HyFHV//RbVavbDSAirZ2eBiLSEBiM9x7PNmCMs1mtaruqzlXVaFV14f3/vFVV46nFbS4iIo1FpGnRc2AIkEUQzvWw/+W4iPwQ7zXRCGCVqqaEuEpBIyLvAQPwplo+D7wEbADeBzriTT0/VlWL30C/b4lIf+Az4Au+u+b9S7z3OWptuwFEJBbvzdAIvF8S31fVRSLyfbzfxlsAB4BnVfVW6GoaHM6lqtmqOiIc2uy0cb2zGAmsUdUUEWlJNZ/rYR84jDHGBCbcL1UZY4wJkAUOY4wxAbHAYYwxJiAWOIwxxgTEAocxxpiAWOAwppJEpNDJQlr0qLZEiSLi8s9ibExNYilHjKm8m6rqDnUljLnXrMdhTDVz5kR43ZkXYY+IPOyUu0Rkq4gcFJE/iEhHp/xBEVnvzJvxuYj0dQ4VISL/7cylsdn59Tci8rwzv8hBEVkbomaaMGaBw5jKa1jsUtU4v3VXVbUH8F94MxMALAfSVDUWeBdY5pQvAz515s34AXDIKY8BVqhqN+AK8K9O+YvA485xEoLVOGNKY78cN6aSRCRfVZuUUH4S7wRKx50Ei1+paksRyQPaqWqBU35OVVuJSC4Q7Z8Cw0kBv8WZfAcRSQLqqurLIrIRyMebLmaD35wbxtwT1uMwJji0lOeB8M+lVMh39ySH45258gfAXr+sr8bcExY4jAmOcX5/dzvPd+HN2AoQjzf5Inin85wOvomXmpV2UBGpA3RQ1W1AEtAMuKvXY0ww2TcVYyqvoTO7XpGNqlo0JLe5iBzE22sY75QlAm+JyBwgF/iZUz4DWCkiP8fbs5gOnKNkEcA7TnARYJkz14Yx94zd4zCmmjn3OOJUNS/UdTEmGOxSlTHGmIBYj8MYY0xArMdhjDEmIBY4jDHGBMQChzHGmIBY4DDGGBMQCxzGGGMC8v9cyzfFN71vBAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EygYQ9Hf-tlM"
      },
      "source": [
        "**Analysis on the following:**\n",
        "\n",
        "\n",
        "**Are you able to find a good strategy to address overfitting in this case? If so, why do you think the strategy you used can be effective?**\n",
        "\n",
        "The best strategy to address overfitting in this specific case would be to either to use a L1, L2 regulizer or a combination of the L1, L2 regulizers along with the drop out technique. As you can see in the graphs that I've plotted above for different techniques, you can see that using a stand alone L2 regulizer or a combination of both the L2 and the drop technique works the best for fixing overfitting. Either one of these techniques can be used in order to fix overfitting. \n",
        "\n",
        "**Reasons:**\n",
        "\n",
        "**Dropout Technique**\n",
        "\n",
        "The reasons to why this strategy is effective is because, while using the dropout technique few of the nodes/units are dropped out while training, this forces the neighbouring nodes in a given hidden layer to perform better. Adittionaly, these dropped out nodes/units are back when testing the model by scaling down the output values by a factor equal to the dropout rate. \n",
        "\n",
        "**L1/L2 regularization** \n",
        "\n",
        "This works on the principal that a simpler a model is, the better it performs on the test set. That is, increasing the model complexity can lead to overfitting. One of the ways to make the model less complex is to put constrains on the complexity by forcing the weights to take smaller values. Thus is done by L1/L2 regulizers where the the loss function of the network is given a penality or a cost for having large weights. We've specifically targeted on L2 regularization for this example where the cost added is proportional to the square of the value of the weights coefficients. \n",
        "\n",
        "Either using L2 alone or combining the above two techniques give you the best chances to reduce overfitting in this specific scenario by reducing it without altering the accuracy. \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yQJJw3U0GKwN"
      },
      "source": [
        "**What other strategies have you tried? Are they effective?**\n",
        "\n",
        "The other strategies that I tried were using dropout techinque by itself and the batch normalization. \n",
        "\n",
        "Out of the two, the dropout technique seemed to be more effective compared to batch normalization. But, comparing the dropout technqiue with the L2 Regulizer or combining both L2 & Dropout, it seemed that using a L2 regulizer or combining L2 & Dropout technique worked better and a visual representation if given in the graph above. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xuf26wOMHdSQ"
      },
      "source": [
        "**Overall, through Q1 and Q2, can you summarize what you have learned regarding modeling and training?**\n",
        "\n",
        "\n",
        "\n",
        "*   Increasing the model complexity leads to overfitting\n",
        "*   Increasing the number of hidden layers does not result in a better performing model\n",
        "*   Increasing the number of units in the hidden layer does not result in a better performing model.\n",
        "*   All the three regularization techniques 'L1 & L2 Regularization', 'Dropout technique', 'Batch Normalization' can prevent overfitting but not all three work the best in every situation, one must implement a regularization technique or a combination of them based on the given problem to avoid overfitting.  \n",
        "*  Tuning of hyperparameters is very important in order to get a good performing model. \n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RpgUR_RIvncV"
      },
      "source": [
        "### Imbalanced Dataset\n",
        "\n",
        "Here we have a very imbalance dataset. Many classes only have a few training samples. As a result, when you display the classification report (see below), you'll see precision/recall/f1 is very low for some classes. \n",
        "\n",
        "Assuming that it is very difficult to collect extra samples, could you think of one method to improve the peformance of some underpresented classes without hurting the overall performance?\n",
        "\n",
        "Implement your method and demonstrate the improvement you can achieve. \n",
        "\n",
        "Please describe your method as markdowns and explain why this method can address dataset skewness."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vtgfI7PSvncW"
      },
      "source": [
        "# Classification report of Model in Q2\n",
        "\n",
        "#print(classification_report(Y, P))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IEu0Pr-i4n_7"
      },
      "source": [
        "**Weighting Samples**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sq8z5irFof5n"
      },
      "source": [
        "#Because there’s a class imbalance, we use stratified split to create our train and test sets.\n",
        "target_list = []\n",
        "i = 0\n",
        "for i in train_labels:\n",
        "    target_list.append(i)                                           \n",
        "    \n",
        "target_list = torch.tensor(target_list)\n",
        "target_list = target_list[torch.randperm(len(target_list))]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nD4p6Ik0om80"
      },
      "source": [
        "#Obtaining the count of all classes in our training set.\n",
        "Count_vals = pd.Series(train_labels).value_counts()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X1VA4qiTos26",
        "outputId": "5052b362-4f4a-4d85-9378-a04b9f4fbd22"
      },
      "source": [
        "#We use the reciprocal of each count to obtain it’s weight.\n",
        "class_count = [i for i in Count_vals]\n",
        "class_weights = 1./torch.tensor(class_count, dtype=torch.float) \n",
        "\n",
        "print(class_weights)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([0.0003, 0.0005, 0.0018, 0.0023, 0.0023, 0.0026, 0.0037, 0.0058, 0.0072,\n",
            "        0.0081, 0.0099, 0.0100, 0.0109, 0.0135, 0.0152, 0.0161, 0.0182, 0.0200,\n",
            "        0.0204, 0.0204, 0.0208, 0.0208, 0.0222, 0.0244, 0.0256, 0.0256, 0.0278,\n",
            "        0.0312, 0.0333, 0.0385, 0.0417, 0.0417, 0.0476, 0.0500, 0.0526, 0.0526,\n",
            "        0.0526, 0.0556, 0.0588, 0.0625, 0.0667, 0.0667, 0.0769, 0.0833, 0.0909,\n",
            "        0.1000])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jiMyamxfoxNZ"
      },
      "source": [
        "#Weighting each sample\n",
        "class_weights_all = class_weights[target_list]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7SbYpwAwozTr"
      },
      "source": [
        "#Initializing WeightedRandomSampler\n",
        "weighted_sampler = WeightedRandomSampler(\n",
        "    weights=class_weights_all,\n",
        "    num_samples=len(class_weights_all),\n",
        "    replacement=True\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M_IqwbROtUoI"
      },
      "source": [
        "preds = []"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pY6Oj-eYo04I"
      },
      "source": [
        "# Define a train function\n",
        "\n",
        "\n",
        "\n",
        "def new_train_model(model, train_dataset, test_dataset, device, lr=0.0002, epochs=30, batch_size=256, l2= 0):  \n",
        "    \n",
        "    history = {'train_loss': [], \n",
        "               'train_acc': [],\n",
        "               'test_loss': [],\n",
        "               'test_acc': []}\n",
        "    \n",
        "    # add your code\n",
        "\n",
        "    train_loader = DataLoader(train_dataset, batch_size=batch_size, sampler=weighted_sampler)    #Adding Sampler\n",
        "    test_loader = DataLoader(test_dataset, batch_size=batch_size)\n",
        "\n",
        "    \n",
        "    model = model.to(device)   \n",
        "\n",
        "  \n",
        "    criterion = nn.CrossEntropyLoss()\n",
        "    optimizer = torch.optim.RMSprop(model.parameters(), lr=lr, weight_decay= l2) \n",
        "\n",
        "    \n",
        "    print('Training Start')\n",
        "    for epoch in range(epochs):\n",
        "        model.train()\n",
        "        train_loss = 0\n",
        "        train_acc = 0\n",
        "        test_loss = 0\n",
        "        test_acc = 0\n",
        "        for x, y in train_loader:\n",
        "          \n",
        "            x = x.to(device)\n",
        "            y = y.to(device)\n",
        "            \n",
        "            outputs = model(x) \n",
        "            score, pred = torch.max(outputs, 1)    \n",
        "            cur_train_loss = criterion(outputs, y)\n",
        "            cur_train_acc = (pred == y).sum().item() / batch_size\n",
        "            \n",
        "            cur_train_loss.backward()\n",
        "            optimizer.step()\n",
        "            optimizer.zero_grad()\n",
        "            \n",
        "            train_loss += cur_train_loss\n",
        "            train_acc += cur_train_acc\n",
        "\n",
        "\n",
        "        model.eval()\n",
        "        with torch.no_grad():\n",
        "            for x, y in test_loader:\n",
        "                \n",
        "                x = x.to(device)\n",
        "                y = y.to(device)\n",
        "                \n",
        "                outputs = model(x) \n",
        "                score, pred = torch.max(outputs,1)                  \n",
        "                cur_test_loss = criterion(outputs, y)\n",
        "                cur_test_acc = (pred == y).sum().item() / batch_size             \n",
        "                \n",
        "                test_loss += cur_test_loss\n",
        "                test_acc += cur_test_acc    \n",
        "               # y_pred_list.append(pred.cpu().detach().numpy()[0])\n",
        "\n",
        "        train_loss = (train_loss/len(train_loader)).item()\n",
        "        train_acc = train_acc/len(train_loader)\n",
        "        val_loss = (test_loss/len(test_loader)).item()\n",
        "        val_acc = test_acc/len(test_loader)\n",
        "        history['train_loss'].append(train_loss)\n",
        "        history['train_acc'].append(train_acc)\n",
        "        history['test_loss'].append(val_loss)\n",
        "        history['test_acc'].append(val_acc)\n",
        "        print(f\"Epoch:{epoch + 1} / {epochs}, train loss:{train_loss:.5f} train_acc:{train_acc:.5f}, Test loss:{val_loss:.5f} Test acc:{val_acc:.5f}\")\n",
        "        \n",
        "  \n",
        "    return history\n",
        "\n",
        "    "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kK6ipL8SpEyJ"
      },
      "source": [
        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3IBKwPHjpJLc",
        "outputId": "9b76516f-a654-478a-af36-ab85f1e7626e"
      },
      "source": [
        "# Train Model \n",
        "\n",
        "# dataset\n",
        "train_dataset = news_dataset(x_train, y_train)\n",
        "test_dataset = news_dataset(x_test, y_test)\n",
        "\n",
        "# load model\n",
        "modelz = linear_model(h1 = 128)\n",
        "summary(modelz, (24, 10000))\n",
        "histx = new_train_model(modelz, train_dataset, test_dataset, device, epochs=50, l2 = 0)    "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "==========================================================================================\n",
              "Layer (type:depth-idx)                   Output Shape              Param #\n",
              "==========================================================================================\n",
              "linear_model                             --                        --\n",
              "├─Sequential: 1-1                        [24, 46]                  --\n",
              "│    └─Linear: 2-1                       [24, 128]                 1,280,128\n",
              "│    └─ReLU: 2-2                         [24, 128]                 --\n",
              "│    └─Linear: 2-3                       [24, 46]                  5,934\n",
              "==========================================================================================\n",
              "Total params: 1,286,062\n",
              "Trainable params: 1,286,062\n",
              "Non-trainable params: 0\n",
              "Total mult-adds (M): 30.87\n",
              "==========================================================================================\n",
              "Input size (MB): 0.96\n",
              "Forward/backward pass size (MB): 0.03\n",
              "Params size (MB): 5.14\n",
              "Estimated Total Size (MB): 6.14\n",
              "=========================================================================================="
            ]
          },
          "metadata": {},
          "execution_count": 41
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training Start\n",
            "Epoch:1 / 50, train loss:2.42863 train_acc:0.60373, Test loss:1.95887 Test acc:0.61285\n",
            "Epoch:2 / 50, train loss:1.44464 train_acc:0.72179, Test loss:1.58999 Test acc:0.65365\n",
            "Epoch:3 / 50, train loss:1.15611 train_acc:0.75684, Test loss:1.41776 Test acc:0.67882\n",
            "Epoch:4 / 50, train loss:0.96362 train_acc:0.79319, Test loss:1.31928 Test acc:0.68707\n",
            "Epoch:5 / 50, train loss:0.83819 train_acc:0.81554, Test loss:1.24336 Test acc:0.70877\n",
            "Epoch:6 / 50, train loss:0.71201 train_acc:0.83789, Test loss:1.19241 Test acc:0.71918\n",
            "Epoch:7 / 50, train loss:0.64278 train_acc:0.85276, Test loss:1.14767 Test acc:0.73351\n",
            "Epoch:8 / 50, train loss:0.54156 train_acc:0.87956, Test loss:1.11036 Test acc:0.74045\n",
            "Epoch:9 / 50, train loss:0.50688 train_acc:0.88661, Test loss:1.07988 Test acc:0.74436\n",
            "Epoch:10 / 50, train loss:0.47664 train_acc:0.88780, Test loss:1.05533 Test acc:0.74696\n",
            "Epoch:11 / 50, train loss:0.42010 train_acc:0.90191, Test loss:1.03084 Test acc:0.75347\n",
            "Epoch:12 / 50, train loss:0.39882 train_acc:0.90137, Test loss:1.01212 Test acc:0.76128\n",
            "Epoch:13 / 50, train loss:0.35721 train_acc:0.91536, Test loss:0.99386 Test acc:0.75868\n",
            "Epoch:14 / 50, train loss:0.32099 train_acc:0.91840, Test loss:0.97725 Test acc:0.76519\n",
            "Epoch:15 / 50, train loss:0.29444 train_acc:0.92426, Test loss:0.96274 Test acc:0.76476\n",
            "Epoch:16 / 50, train loss:0.26464 train_acc:0.92893, Test loss:0.96128 Test acc:0.76649\n",
            "Epoch:17 / 50, train loss:0.25357 train_acc:0.92828, Test loss:0.94098 Test acc:0.77257\n",
            "Epoch:18 / 50, train loss:0.23234 train_acc:0.93294, Test loss:0.93074 Test acc:0.77387\n",
            "Epoch:19 / 50, train loss:0.21999 train_acc:0.93490, Test loss:0.92821 Test acc:0.77474\n",
            "Epoch:20 / 50, train loss:0.20635 train_acc:0.93316, Test loss:0.93485 Test acc:0.76866\n",
            "Epoch:21 / 50, train loss:0.18540 train_acc:0.93783, Test loss:0.91729 Test acc:0.77561\n",
            "Epoch:22 / 50, train loss:0.18275 train_acc:0.94141, Test loss:0.92114 Test acc:0.77170\n",
            "Epoch:23 / 50, train loss:0.16640 train_acc:0.93989, Test loss:0.91523 Test acc:0.77387\n",
            "Epoch:24 / 50, train loss:0.14621 train_acc:0.94520, Test loss:0.90989 Test acc:0.77691\n",
            "Epoch:25 / 50, train loss:0.15652 train_acc:0.94162, Test loss:0.90639 Test acc:0.78082\n",
            "Epoch:26 / 50, train loss:0.14969 train_acc:0.94260, Test loss:0.90883 Test acc:0.78299\n",
            "Epoch:27 / 50, train loss:0.13711 train_acc:0.94531, Test loss:0.91289 Test acc:0.77778\n",
            "Epoch:28 / 50, train loss:0.14488 train_acc:0.94434, Test loss:0.93172 Test acc:0.77214\n",
            "Epoch:29 / 50, train loss:0.13059 train_acc:0.94553, Test loss:0.90396 Test acc:0.78125\n",
            "Epoch:30 / 50, train loss:0.12833 train_acc:0.94596, Test loss:0.91083 Test acc:0.78038\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C19bxewJbNIj",
        "outputId": "7a4e3ae0-091e-4b9f-e12c-34ba36377a61"
      },
      "source": [
        "test_loader = DataLoader(test_dataset)\n",
        "model = modelz.to(device)\n",
        "model.eval()\n",
        "y_pred_list = []\n",
        "with torch.no_grad():\n",
        "    for x,y in test_loader:\n",
        "        x = x.to(device)\n",
        "        outputs = modelz(x)\n",
        "        score,pred = torch.max(outputs, 1)\n",
        "        y_pred_list.append(pred.cpu().detach().numpy()[0])\n",
        "\n",
        "y_true = y_test\n",
        "y_pred = np.array(y_pred_list)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "linear_model(\n",
              "  (classifier): Sequential(\n",
              "    (0): Linear(in_features=10000, out_features=128, bias=True)\n",
              "    (1): ReLU()\n",
              "    (2): Linear(in_features=128, out_features=46, bias=True)\n",
              "  )\n",
              ")"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5llhDEMFpLyq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9faec66c-1c96-46bf-adc1-18ee8e7f4a61"
      },
      "source": [
        "from sklearn.metrics import classification_report\n",
        "classification_report(y_true, y_pred, output_dict=True)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:1272: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
            "  _warn_prf(average, modifier, msg_start, len(result))\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'0': {'f1-score': 0.761904761904762,\n",
              "  'precision': 0.8888888888888888,\n",
              "  'recall': 0.6666666666666666,\n",
              "  'support': 12},\n",
              " '1': {'f1-score': 0.7359307359307359,\n",
              "  'precision': 0.6746031746031746,\n",
              "  'recall': 0.8095238095238095,\n",
              "  'support': 105},\n",
              " '10': {'f1-score': 0.8813559322033899,\n",
              "  'precision': 0.896551724137931,\n",
              "  'recall': 0.8666666666666667,\n",
              "  'support': 30},\n",
              " '11': {'f1-score': 0.6594594594594596,\n",
              "  'precision': 0.5980392156862745,\n",
              "  'recall': 0.7349397590361446,\n",
              "  'support': 83},\n",
              " '12': {'f1-score': 0.3,\n",
              "  'precision': 0.42857142857142855,\n",
              "  'recall': 0.23076923076923078,\n",
              "  'support': 13},\n",
              " '13': {'f1-score': 0.6027397260273972,\n",
              "  'precision': 0.6111111111111112,\n",
              "  'recall': 0.5945945945945946,\n",
              "  'support': 37},\n",
              " '14': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 2},\n",
              " '15': {'f1-score': 0.1818181818181818,\n",
              "  'precision': 0.5,\n",
              "  'recall': 0.1111111111111111,\n",
              "  'support': 9},\n",
              " '16': {'f1-score': 0.7109004739336493,\n",
              "  'precision': 0.6696428571428571,\n",
              "  'recall': 0.7575757575757576,\n",
              "  'support': 99},\n",
              " '17': {'f1-score': 0.375, 'precision': 0.75, 'recall': 0.25, 'support': 12},\n",
              " '18': {'f1-score': 0.6153846153846154,\n",
              "  'precision': 0.631578947368421,\n",
              "  'recall': 0.6,\n",
              "  'support': 20},\n",
              " '19': {'f1-score': 0.6666666666666667,\n",
              "  'precision': 0.664179104477612,\n",
              "  'recall': 0.6691729323308271,\n",
              "  'support': 133},\n",
              " '2': {'f1-score': 0.7027027027027027,\n",
              "  'precision': 0.7647058823529411,\n",
              "  'recall': 0.65,\n",
              "  'support': 20},\n",
              " '20': {'f1-score': 0.5481481481481482,\n",
              "  'precision': 0.5692307692307692,\n",
              "  'recall': 0.5285714285714286,\n",
              "  'support': 70},\n",
              " '21': {'f1-score': 0.7142857142857143,\n",
              "  'precision': 0.6896551724137931,\n",
              "  'recall': 0.7407407407407407,\n",
              "  'support': 27},\n",
              " '22': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 7},\n",
              " '23': {'f1-score': 0.6086956521739131,\n",
              "  'precision': 0.6363636363636364,\n",
              "  'recall': 0.5833333333333334,\n",
              "  'support': 12},\n",
              " '24': {'f1-score': 0.3448275862068966,\n",
              "  'precision': 0.5,\n",
              "  'recall': 0.2631578947368421,\n",
              "  'support': 19},\n",
              " '25': {'f1-score': 0.7037037037037037,\n",
              "  'precision': 0.8260869565217391,\n",
              "  'recall': 0.6129032258064516,\n",
              "  'support': 31},\n",
              " '26': {'f1-score': 0.9333333333333333,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.875,\n",
              "  'support': 8},\n",
              " '27': {'f1-score': 0.4, 'precision': 1.0, 'recall': 0.25, 'support': 4},\n",
              " '28': {'f1-score': 0.3, 'precision': 0.3, 'recall': 0.3, 'support': 10},\n",
              " '29': {'f1-score': 0.4444444444444445,\n",
              "  'precision': 0.4,\n",
              "  'recall': 0.5,\n",
              "  'support': 4},\n",
              " '3': {'f1-score': 0.925211097708082,\n",
              "  'precision': 0.9076923076923077,\n",
              "  'recall': 0.9434194341943419,\n",
              "  'support': 813},\n",
              " '30': {'f1-score': 0.4444444444444444,\n",
              "  'precision': 0.6666666666666666,\n",
              "  'recall': 0.3333333333333333,\n",
              "  'support': 12},\n",
              " '31': {'f1-score': 0.7000000000000001,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.5384615384615384,\n",
              "  'support': 13},\n",
              " '32': {'f1-score': 0.8421052631578948,\n",
              "  'precision': 0.8888888888888888,\n",
              "  'recall': 0.8,\n",
              "  'support': 10},\n",
              " '33': {'f1-score': 0.8000000000000002,\n",
              "  'precision': 0.8,\n",
              "  'recall': 0.8,\n",
              "  'support': 5},\n",
              " '34': {'f1-score': 0.4,\n",
              "  'precision': 0.6666666666666666,\n",
              "  'recall': 0.2857142857142857,\n",
              "  'support': 7},\n",
              " '35': {'f1-score': 0.6666666666666666,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.5,\n",
              "  'support': 6},\n",
              " '36': {'f1-score': 0.13333333333333333,\n",
              "  'precision': 0.25,\n",
              "  'recall': 0.09090909090909091,\n",
              "  'support': 11},\n",
              " '37': {'f1-score': 0.5, 'precision': 0.5, 'recall': 0.5, 'support': 2},\n",
              " '38': {'f1-score': 0.5,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.3333333333333333,\n",
              "  'support': 3},\n",
              " '39': {'f1-score': 0.33333333333333337,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.2,\n",
              "  'support': 5},\n",
              " '4': {'f1-score': 0.8498985801217038,\n",
              "  'precision': 0.818359375,\n",
              "  'recall': 0.8839662447257384,\n",
              "  'support': 474},\n",
              " '40': {'f1-score': 0.18181818181818182,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.1,\n",
              "  'support': 10},\n",
              " '41': {'f1-score': 0.3333333333333333,\n",
              "  'precision': 0.5,\n",
              "  'recall': 0.25,\n",
              "  'support': 8},\n",
              " '42': {'f1-score': 0.5,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.3333333333333333,\n",
              "  'support': 3},\n",
              " '43': {'f1-score': 0.923076923076923,\n",
              "  'precision': 0.8571428571428571,\n",
              "  'recall': 1.0,\n",
              "  'support': 6},\n",
              " '44': {'f1-score': 0.888888888888889,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.8,\n",
              "  'support': 5},\n",
              " '45': {'f1-score': 1.0, 'precision': 1.0, 'recall': 1.0, 'support': 1},\n",
              " '5': {'f1-score': 0.0, 'precision': 0.0, 'recall': 0.0, 'support': 5},\n",
              " '6': {'f1-score': 0.8275862068965518,\n",
              "  'precision': 0.8,\n",
              "  'recall': 0.8571428571428571,\n",
              "  'support': 14},\n",
              " '7': {'f1-score': 0.5,\n",
              "  'precision': 1.0,\n",
              "  'recall': 0.3333333333333333,\n",
              "  'support': 3},\n",
              " '8': {'f1-score': 0.6944444444444445,\n",
              "  'precision': 0.7352941176470589,\n",
              "  'recall': 0.6578947368421053,\n",
              "  'support': 38},\n",
              " '9': {'f1-score': 0.8085106382978724,\n",
              "  'precision': 0.8636363636363636,\n",
              "  'recall': 0.76,\n",
              "  'support': 25},\n",
              " 'accuracy': 0.796527159394479,\n",
              " 'macro avg': {'f1-score': 0.5639989820402037,\n",
              "  'precision': 0.7011642633089431,\n",
              "  'recall': 0.5194688841910194,\n",
              "  'support': 2246},\n",
              " 'weighted avg': {'f1-score': 0.7851736383644926,\n",
              "  'precision': 0.7899025367806548,\n",
              "  'recall': 0.796527159394479,\n",
              "  'support': 2246}}"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6p-qDmz8f-rm",
        "outputId": "337dc17c-1f3e-4a42-e907-2f80b5207e47"
      },
      "source": [
        "report = classification_report(y_true, y_pred, output_dict=True)\n",
        "df = pd.DataFrame(report).transpose()\n",
        "print(df)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "              precision    recall  f1-score      support\n",
            "0              0.888889  0.666667  0.761905    12.000000\n",
            "1              0.674603  0.809524  0.735931   105.000000\n",
            "2              0.764706  0.650000  0.702703    20.000000\n",
            "3              0.907692  0.943419  0.925211   813.000000\n",
            "4              0.818359  0.883966  0.849899   474.000000\n",
            "5              0.000000  0.000000  0.000000     5.000000\n",
            "6              0.800000  0.857143  0.827586    14.000000\n",
            "7              1.000000  0.333333  0.500000     3.000000\n",
            "8              0.735294  0.657895  0.694444    38.000000\n",
            "9              0.863636  0.760000  0.808511    25.000000\n",
            "10             0.896552  0.866667  0.881356    30.000000\n",
            "11             0.598039  0.734940  0.659459    83.000000\n",
            "12             0.428571  0.230769  0.300000    13.000000\n",
            "13             0.611111  0.594595  0.602740    37.000000\n",
            "14             0.000000  0.000000  0.000000     2.000000\n",
            "15             0.500000  0.111111  0.181818     9.000000\n",
            "16             0.669643  0.757576  0.710900    99.000000\n",
            "17             0.750000  0.250000  0.375000    12.000000\n",
            "18             0.631579  0.600000  0.615385    20.000000\n",
            "19             0.664179  0.669173  0.666667   133.000000\n",
            "20             0.569231  0.528571  0.548148    70.000000\n",
            "21             0.689655  0.740741  0.714286    27.000000\n",
            "22             0.000000  0.000000  0.000000     7.000000\n",
            "23             0.636364  0.583333  0.608696    12.000000\n",
            "24             0.500000  0.263158  0.344828    19.000000\n",
            "25             0.826087  0.612903  0.703704    31.000000\n",
            "26             1.000000  0.875000  0.933333     8.000000\n",
            "27             1.000000  0.250000  0.400000     4.000000\n",
            "28             0.300000  0.300000  0.300000    10.000000\n",
            "29             0.400000  0.500000  0.444444     4.000000\n",
            "30             0.666667  0.333333  0.444444    12.000000\n",
            "31             1.000000  0.538462  0.700000    13.000000\n",
            "32             0.888889  0.800000  0.842105    10.000000\n",
            "33             0.800000  0.800000  0.800000     5.000000\n",
            "34             0.666667  0.285714  0.400000     7.000000\n",
            "35             1.000000  0.500000  0.666667     6.000000\n",
            "36             0.250000  0.090909  0.133333    11.000000\n",
            "37             0.500000  0.500000  0.500000     2.000000\n",
            "38             1.000000  0.333333  0.500000     3.000000\n",
            "39             1.000000  0.200000  0.333333     5.000000\n",
            "40             1.000000  0.100000  0.181818    10.000000\n",
            "41             0.500000  0.250000  0.333333     8.000000\n",
            "42             1.000000  0.333333  0.500000     3.000000\n",
            "43             0.857143  1.000000  0.923077     6.000000\n",
            "44             1.000000  0.800000  0.888889     5.000000\n",
            "45             1.000000  1.000000  1.000000     1.000000\n",
            "accuracy       0.796527  0.796527  0.796527     0.796527\n",
            "macro avg      0.701164  0.519469  0.563999  2246.000000\n",
            "weighted avg   0.789903  0.796527  0.785174  2246.000000\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py:1272: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
            "  _warn_prf(average, modifier, msg_start, len(result))\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NA88MXnCxdB0"
      },
      "source": [
        "**Explanation**\n",
        "\n",
        "Because there’s a class imbalance, we use stratified split to create our train, validation, and test sets.\n",
        "While it helps, it still does not ensure that each mini-batch of our model see’s all our classes. We need to over-sample the classes with less number of values. To do that, we use the WeightedRandomSampler.\n",
        "First, we obtain a list called target_list which contains all our outputs. This list is then converted to a tensor and shuffled.\n",
        "\n",
        "Then, we obtain the count of all classes in our training set. We use the reciprocal of each count to obtain it’s weight. Now that we’ve calculated the weights for each class, we can proceed.\n",
        "\n",
        "Finally, let’s initialize our WeightedRandomSampler. We’ll call this in our dataloader below."
      ]
    }
  ]
}