notebooks/11_Convolutional_Neural_Networks.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "11_Convolutional_Neural_Networks_cn",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "metadata": {
        "id": "bOChJSNXtC9g",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 卷积神经网络 Convolutional Neural Networks"
      ]
    },
    {
      "metadata": {
        "id": "OLIxEDq6VhvZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/logo.png\" width=150>\n",
        "\n",
        "本课程中，我们将学习基本的卷积神经网络（CNN），并应用到自然语言（NLP）处理中。CNN 常用于图像处理，并且有很多[例子](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html)。但我们专注于将它用于文本数据，也有惊人的效果。\n",
        "\n",
        "In this lesson we will learn the basics of Convolutional Neural Networks (CNNs) applied to text for natural language processing (NLP) tasks. CNNs are traditionally used on images and there are plenty of [tutorials](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html) that cover this. But we're going to focus on using CNN on text data which yields amazing results. \n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "VoMq0eFRvugb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 总览 Overview\n",
        "\n",
        "[论文](https://arxiv.org/abs/1510.03820)中图表展示了句子中的词如何进行一维卷积计算。\n",
        "\n",
        "The diagram below from this [paper](https://arxiv.org/abs/1510.03820) shows how 1D convolution is applied to the words in a sentence. "
      ]
    },
    {
      "metadata": {
        "id": "ziGJNhiQeiGN",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/cnn_text.png\" width=500>"
      ]
    },
    {
      "metadata": {
        "id": "qWro5T5qTJJL",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* **目的：**  检测输入数据的空间结构。Detect spatial substructure from input data.\n",
        "* **优点：** \n",
        "  * 权重数量小（共享）Small number of weights (shared)\n",
        "  * 可并行的 Parallelizable\n",
        "  * 检测空间结构（特征提取）Detects spatial substrcutures (feature extractors)\n",
        "  * 通过滤波器可翻译的 Interpretable via filters\n",
        "  * 可用于图像、文本、时间序列等 Used for in images/text/time-series etc.\n",
        "* **缺点：**\n",
        "  * 很多超参数（核大小，步幅等）Many hyperparameters (kernel size, strides, etc.)\n",
        "  * 输入宽度必须相同（图像维度，文本长度等）Inputs have to be of same width (image dimensions, text length, etc.)\n",
        "* **其他：** \n",
        "  * 很多深度 CNN 框架持续更新 SOTA 性能  Lot's of deep CNN architectures constantly updated for SOTA performance"
      ]
    },
    {
      "metadata": {
        "id": "8nCsZGyWhI9f",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 滤波器 Filters"
      ]
    },
    {
      "metadata": {
        "id": "lxpgRzIjiVHv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "CNN 的核心是滤波器（权重，核等），它通过对输入进行卷积计算，提取相关特征。虽然滤波器是随机初始化的，但可以学习从输入选取有意义的特征，有助于优化目标。我们将通过不寻常的方式介绍 CNN，聚焦应用于二维文本数据。每个输入有多个词组成，我们使用独热编码（one-hot encoder）表示每个词，得到一个二维输入。每个滤波器代表一个特征，我们将应用滤波器到其他输入，捕获相同特征。这也称为参数共享。\n",
        "\n",
        "At the core of CNNs are filters (weights, kernels, etc.) which convolve (slide) across our input to extract relevant features. The filters are initialized randomly but learn to pick up meaningful features from the input that aid in optimizing for the objective. We're going to teach CNNs in an unorthodox method where we entirely focus on applying it to 2D text data. Each input is composed of words and we will be representing each word as one-hot encoded vector which gives us our 2D input. The intuition here is that each filter represents a feature and we will use this filter on other inputs to capture the same feature. This is known as parameter sharing.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/conv.gif\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "1kTABJyYj91S",
        "colab_type": "code",
        "outputId": "5f7c9497-63bf-46e3-e01b-74b5902ae569",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Loading PyTorch library\n",
        "!pip3 install torch"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (1.0.0)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "kz9D2rrdmSl9",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1q1FiiIHXjI_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "我们的输入是一组二维文本数据。设定输入有64个样本，其中每个样本有8个词，每个词用一个长度为10的数组（词汇量为10的独热编码）表示。这样输入的体积为（64，8，10）。[Python CNN 模块](https://pytorch.org/docs/stable/nn.html#convolution-functions)倾向于输入的通道维度（本例中是一个独热编码向量）在第2个位置，所以我们的输入维度是（64，10，8）。\n",
        "\n",
        "Our inputs are a batch of 2D text data. Let's make an input with 64 samples, where each sample has 8 words and each word is represented by a array of 10 values (one hot encoded with vocab size of 10). This gives our inputs the size (64, 8, 10). The [PyTorch CNN modules](https://pytorch.org/docs/stable/nn.html#convolution-functions) prefer inputs to have the channel dim (one hot vector dim in our case) to be in the second position, so our inputs are of shape (64, 10, 8)."
      ]
    },
    {
      "metadata": {
        "id": "tFfYwCcjZj79",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/cnn_text1.png\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "b6G2nBvOxR-e",
        "colab_type": "code",
        "outputId": "00f935dc-b4be-472c-8a9c-a3dc608955e9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Assume all our inputs have the same # of words\n",
        "batch_size = 64\n",
        "sequence_size = 8 # words per input\n",
        "one_hot_size = 10 # vocab size (num_input_channels)\n",
        "x = torch.randn(batch_size, one_hot_size, sequence_size)\n",
        "print(\"Size: {}\".format(x.shape))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([64, 10, 8])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "GJmtay_UZohM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "使用滤波器对输入进行卷积运算。为了简单一点，我们使用5个（1，2）的滤波器，深度和通道数相同（独热编码大小）。这样，滤波器维度是（5，2，10）。但是如上所述， PyTorch 更倾向于通道维度在第2个位置，所以滤波器维度是（5，10，2）。\n",
        "\n",
        "We want to convolve on this input using filters. For simplicity we will use just 5 filters that is of size (1, 2) and has the same depth as the number of channels (one_hot_size). This gives our filter a shape of (5, 2, 10) but recall that PyTorch CNN modules prefer to have the channel dim (one hot vector dim in our case) to be in the second position so the filter is of shape (5, 10, 2)."
      ]
    },
    {
      "metadata": {
        "id": "ZJF0l88qb-21",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/cnn_text2.png\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "WMK2TzgDxR8B",
        "colab_type": "code",
        "outputId": "32609363-1d3d-4437-c709-57bbac524064",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Create filters for a conv layer\n",
        "out_channels = 5 # of filters\n",
        "kernel_size = 2 # filters size 2\n",
        "conv1 = nn.Conv1d(in_channels=one_hot_size, out_channels=out_channels, kernel_size=kernel_size)\n",
        "print(\"Size: {}\".format(conv1.weight.shape))\n",
        "print(\"Filter size: {}\".format(conv1.kernel_size[0]))\n",
        "print(\"Padding: {}\".format(conv1.padding[0]))\n",
        "print(\"Stride: {}\".format(conv1.stride[0]))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([5, 10, 2])\n",
            "Filter size: 2\n",
            "Padding: 0\n",
            "Stride: 1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "lAcYxhDIbeWE",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "我们在输入上使用这个滤波器，得到（64，5，7）的输出。64是样本大小，5是通道维度（因为使用了5个滤波器），7是卷积输出，因为：\n",
        "\n",
        "$\\frac{W - F + 2P}{S} + 1 = \\frac{8 - 2 + 2(0)}{1} + 1 = 7$\n",
        "\n",
        "其中：\n",
        "  * W：输入宽度\n",
        "  * F：滤波器大小\n",
        "  * P：填充\n",
        "  * S：步幅\n",
        "  \n",
        "When we apply this filter on our inputs, we receive an output of shape (64, 5, 7). We get 64 for the batch size, 5 for the channel dim because we used 5 filters and 7 for the conv outputs because:\n",
        "\n",
        "$\\frac{W - F + 2P}{S} + 1 = \\frac{8 - 2 + 2(0)}{1} + 1 = 7$\n",
        "\n",
        "where:\n",
        "  * W: width of each input\n",
        "  * F: filter size\n",
        "  * P: padding\n",
        "  * S: stride"
      ]
    },
    {
      "metadata": {
        "id": "2c_KKtP4hrJx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/cnn_text3.png\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "yjxtrM89xR5a",
        "colab_type": "code",
        "outputId": "d7787d5f-8450-4cdc-b234-03b68452c739",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Convolve using filters\n",
        "conv_output = conv1(x)\n",
        "print(\"Size: {}\".format(conv_output.shape))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([64, 5, 7])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "vwTtF7bBuZvF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 池化 Pooling"
      ]
    },
    {
      "metadata": {
        "id": "VXBbKPs1ua9G",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "对输入进行卷积计算的结果是一个特征映射。因为卷积和重叠，特征映射有很多冗余信息。池化是一种降维的方式。池化可以是一定接收域的最大值，平局值等。\n",
        "\n",
        "The result of convolving filters on an input is a feature map. Due to the nature of convolution and overlaps, our feature map will have lots of redundant information. Pooling is a way to summarize a high-dimensional feature map into a lower dimensional one for simplified downstream computation. The pooling operation can be the max value, average, etc. in a certain receptive field.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/pool.jpeg\" width=450>"
      ]
    },
    {
      "metadata": {
        "id": "VCag6lk2mSwU",
        "colab_type": "code",
        "outputId": "d60fdc17-1791-41ba-f7ae-86e78553e20a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Max pooling\n",
        "kernel_size = 2\n",
        "pool1 = nn.MaxPool1d(kernel_size=kernel_size, stride=2, padding=0)\n",
        "pool_output = pool1(conv_output)\n",
        "print(\"Size: {}\".format(pool_output.shape))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([64, 5, 3])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "c_e4QRFwvTt8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "$\\frac{W-F}{S} + 1 = \\frac{7-2}{2} + 1 =  \\text{floor }(2.5) + 1 = 3$"
      ]
    },
    {
      "metadata": {
        "id": "l9rL1EWIfi-y",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# CNN 文本处理 CNNs on text"
      ]
    },
    {
      "metadata": {
        "id": "aWtHDOJgHZvk",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "我们正在用卷积神经网络处理文本数据，进行词级别的卷积计算，获取有用的n-grams（一种语言模型）。\n",
        "\n",
        "你可以把这个配置用于[时间序列](https://arxiv.org/abs/1807.10707) 数据或者与其他神经网络相结合。对于文本数据，我们会建立不同大小的滤波器，（1，2），（1，3）和（1，4），他们用于不同 n-gram 的特征选取器。把输出拼接起来，输入到一个全连接网络。本例中，我们使用字符级的一维卷积。在 [embeddings notebook](https://colab.research.google.com/github/GokuMohandas/practicalAI/blob/master/notebooks/12_Embeddings.ipynb)，我们在词级中应用了一维卷积。\n",
        "\n",
        "**词嵌入**：捕获相邻词汇的临时联系，使得相似单词有相似意义。\n",
        "\n",
        "**字符嵌入**：创建一个字符级别映射。例如\"toy\"和“toys”彼此接近。\n",
        "\n",
        "We're going use convolutional neural networks on text data which typically involves convolving on the character level representation of the text to capture meaningful n-grams. \n",
        "\n",
        "You can easily use this set up for [time series](https://arxiv.org/abs/1807.10707) data or [combine it](https://arxiv.org/abs/1808.04928) with other networks. For text data, we will create filters of varying kernel sizes (1,2), (1,3), and (1,4) which act as feature selectors of varying n-gram sizes. The outputs are concated and fed into a fully-connected layer for class predictions. In our example, we will be applying 1D convolutions on letter in a word. In the [embeddings notebook](https://colab.research.google.com/github/GokuMohandas/practicalAI/blob/master/notebooks/12_Embeddings.ipynb), we will apply 1D convolutions on words in a sentence.\n",
        "\n",
        "**Word embeddings**: capture the temporal correlations among\n",
        "adjacent tokens so that similar words have similar representations. Ex. \"New Jersey\" is close to \"NJ\" is close to \"Garden State\", etc.\n",
        "\n",
        "**Char embeddings**: create representations that map words at a character level. Ex. \"toy\" and \"toys\" will be close to each other."
      ]
    },
    {
      "metadata": {
        "id": "bVBZxbaAtS9u",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 配置 Set up"
      ]
    },
    {
      "metadata": {
        "id": "y8QSdEcDtXUs",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import os\n",
        "from argparse import Namespace\n",
        "import collections\n",
        "import copy\n",
        "import json\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import re\n",
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "VADCXjMwtXYN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Set Numpy and PyTorch seeds\n",
        "def set_seeds(seed, cuda):\n",
        "    np.random.seed(seed)\n",
        "    torch.manual_seed(seed)\n",
        "    if cuda:\n",
        "        torch.cuda.manual_seed_all(seed)\n",
        "        \n",
        "# Creating directories\n",
        "def create_dirs(dirpath):\n",
        "    if not os.path.exists(dirpath):\n",
        "        os.makedirs(dirpath)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "mpiCYECstXbT",
        "colab_type": "code",
        "outputId": "0ef00d64-963e-4e23-cbea-d2e8187f5553",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Arguments\n",
        "args = Namespace(\n",
        "    seed=1234,\n",
        "    cuda=False,\n",
        "    shuffle=True,\n",
        "    data_file=\"names.csv\",\n",
        "    vectorizer_file=\"vectorizer.json\",\n",
        "    model_state_file=\"model.pth\",\n",
        "    save_dir=\"names\",\n",
        "    train_size=0.7,\n",
        "    val_size=0.15,\n",
        "    test_size=0.15,\n",
        "    num_epochs=20,\n",
        "    early_stopping_criteria=5,\n",
        "    learning_rate=1e-3,\n",
        "    batch_size=64,\n",
        "    num_filters=100,\n",
        "    dropout_p=0.1,\n",
        ")\n",
        "\n",
        "# Set seeds\n",
        "set_seeds(seed=args.seed, cuda=args.cuda)\n",
        "\n",
        "# Create save dir\n",
        "create_dirs(args.save_dir)\n",
        "\n",
        "# Expand filepaths\n",
        "args.vectorizer_file = os.path.join(args.save_dir, args.vectorizer_file)\n",
        "args.model_state_file = os.path.join(args.save_dir, args.model_state_file)\n",
        "\n",
        "# Check CUDA\n",
        "if not torch.cuda.is_available():\n",
        "    args.cuda = False\n",
        "args.device = torch.device(\"cuda\" if args.cuda else \"cpu\")\n",
        "print(\"Using CUDA: {}\".format(args.cuda))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using CUDA: False\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "ptb4hJ4Bw8YU",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 数据 Data"
      ]
    },
    {
      "metadata": {
        "id": "bNxZQUqfmS0B",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import re\n",
        "import urllib"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "MBdQpUTQtMgu",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Upload data from GitHub to notebook's local drive\n",
        "url = \"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/data/surnames.csv\"\n",
        "response = urllib.request.urlopen(url)\n",
        "html = response.read()\n",
        "with open(args.data_file, 'wb') as fp:\n",
        "    fp.write(html)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "6PYCeGrStMj7",
        "colab_type": "code",
        "outputId": "849f86d5-1bdc-404f-bb45-44b3532a5069",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Raw data\n",
        "df = pd.read_csv(args.data_file, header=0)\n",
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>surname</th>\n",
              "      <th>nationality</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Woodford</td>\n",
              "      <td>English</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Coté</td>\n",
              "      <td>French</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Kore</td>\n",
              "      <td>English</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Koury</td>\n",
              "      <td>Arabic</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Lebzak</td>\n",
              "      <td>Russian</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    surname nationality\n",
              "0  Woodford     English\n",
              "1      Coté      French\n",
              "2      Kore     English\n",
              "3     Koury      Arabic\n",
              "4    Lebzak     Russian"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "metadata": {
        "id": "pbfVM-YatMnD",
        "colab_type": "code",
        "outputId": "58746131-868b-4ab6-cef3-147f137e04e6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 323
        }
      },
      "cell_type": "code",
      "source": [
        "# Split by nationality\n",
        "by_nationality = collections.defaultdict(list)\n",
        "for _, row in df.iterrows():\n",
        "    by_nationality[row.nationality].append(row.to_dict())\n",
        "for nationality in by_nationality:\n",
        "    print (\"{0}: {1}\".format(nationality, len(by_nationality[nationality])))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "English: 2972\n",
            "French: 229\n",
            "Arabic: 1603\n",
            "Russian: 2373\n",
            "Japanese: 775\n",
            "Chinese: 220\n",
            "Italian: 600\n",
            "Czech: 414\n",
            "Irish: 183\n",
            "German: 576\n",
            "Greek: 156\n",
            "Spanish: 258\n",
            "Polish: 120\n",
            "Dutch: 236\n",
            "Vietnamese: 58\n",
            "Korean: 77\n",
            "Portuguese: 55\n",
            "Scottish: 75\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "KdGOoKFjtMpz",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Create split data\n",
        "final_list = []\n",
        "for _, item_list in sorted(by_nationality.items()):\n",
        "    if args.shuffle:\n",
        "        np.random.shuffle(item_list)\n",
        "    n = len(item_list)\n",
        "    n_train = int(args.train_size*n)\n",
        "    n_val = int(args.val_size*n)\n",
        "    n_test = int(args.test_size*n)\n",
        "\n",
        "  # Give data point a split attribute\n",
        "    for item in item_list[:n_train]:\n",
        "        item['split'] = 'train'\n",
        "    for item in item_list[n_train:n_train+n_val]:\n",
        "        item['split'] = 'val'\n",
        "    for item in item_list[n_train+n_val:]:\n",
        "        item['split'] = 'test'  \n",
        "\n",
        "    # Add to final list\n",
        "    final_list.extend(item_list)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "DyDwlzzKtMsz",
        "colab_type": "code",
        "outputId": "ba9fc7a2-819f-4889-941b-05b2868cb342",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# df with split datasets\n",
        "split_df = pd.DataFrame(final_list)\n",
        "split_df[\"split\"].value_counts()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "train    7680\n",
              "test     1660\n",
              "val      1640\n",
              "Name: split, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "metadata": {
        "id": "17aHMQOwtMvh",
        "colab_type": "code",
        "outputId": "eba52fd0-e276-459c-de8a-633f6ed9e05a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Preprocessing\n",
        "def preprocess_text(text):\n",
        "    text = ' '.join(word.lower() for word in text.split(\" \"))\n",
        "    text = re.sub(r\"([.,!?])\", r\" \\1 \", text)\n",
        "    text = re.sub(r\"[^a-zA-Z.,!?]+\", r\" \", text)\n",
        "    return text\n",
        "    \n",
        "split_df.surname = split_df.surname.apply(preprocess_text)\n",
        "split_df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>nationality</th>\n",
              "      <th>split</th>\n",
              "      <th>surname</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>bishara</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>nahas</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>ghanem</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>tannous</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>mikhail</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  nationality  split  surname\n",
              "0      Arabic  train  bishara\n",
              "1      Arabic  train    nahas\n",
              "2      Arabic  train   ghanem\n",
              "3      Arabic  train  tannous\n",
              "4      Arabic  train  mikhail"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "metadata": {
        "id": "6nZBgfQTuAA8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 词汇表 Vocabulary"
      ]
    },
    {
      "metadata": {
        "id": "TeRVQlRZuBgA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Vocabulary(object):\n",
        "    def __init__(self, token_to_idx=None, add_unk=True, unk_token=\"<UNK>\"):\n",
        "\n",
        "        # Token to index\n",
        "        if token_to_idx is None:\n",
        "            token_to_idx = {}\n",
        "        self.token_to_idx = token_to_idx\n",
        "\n",
        "        # Index to token\n",
        "        self.idx_to_token = {idx: token \\\n",
        "                             for token, idx in self.token_to_idx.items()}\n",
        "        \n",
        "        # Add unknown token\n",
        "        self.add_unk = add_unk\n",
        "        self.unk_token = unk_token\n",
        "        if self.add_unk:\n",
        "            self.unk_index = self.add_token(self.unk_token)\n",
        "\n",
        "    def to_serializable(self):\n",
        "        return {'token_to_idx': self.token_to_idx,\n",
        "                'add_unk': self.add_unk, 'unk_token': self.unk_token}\n",
        "\n",
        "    @classmethod\n",
        "    def from_serializable(cls, contents):\n",
        "        return cls(**contents)\n",
        "\n",
        "    def add_token(self, token):\n",
        "        if token in self.token_to_idx:\n",
        "            index = self.token_to_idx[token]\n",
        "        else:\n",
        "            index = len(self.token_to_idx)\n",
        "            self.token_to_idx[token] = index\n",
        "            self.idx_to_token[index] = token\n",
        "        return index\n",
        "\n",
        "    def add_tokens(self, tokens):\n",
        "        return [self.add_token[token] for token in tokens]\n",
        "\n",
        "    def lookup_token(self, token):\n",
        "        if self.add_unk:\n",
        "            index = self.token_to_idx.get(token, self.unk_index)\n",
        "        else:\n",
        "            index =  self.token_to_idx[token]\n",
        "        return index\n",
        "\n",
        "    def lookup_index(self, index):\n",
        "        if index not in self.idx_to_token:\n",
        "            raise KeyError(\"the index (%d) is not in the Vocabulary\" % index)\n",
        "        return self.idx_to_token[index]\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Vocabulary(size=%d)>\" % len(self)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.token_to_idx)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "bH8LMH9wuBi9",
        "colab_type": "code",
        "outputId": "8293745c-751e-40d9-9993-78e84093e203",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "# Vocabulary instance\n",
        "nationality_vocab = Vocabulary(add_unk=False)\n",
        "for index, row in df.iterrows():\n",
        "    nationality_vocab.add_token(row.nationality)\n",
        "print (nationality_vocab) # __str__\n",
        "index = nationality_vocab.lookup_token(\"English\")\n",
        "print (index)\n",
        "print (nationality_vocab.lookup_index(index))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Vocabulary(size=18)>\n",
            "0\n",
            "English\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "57a1lzHPuHHm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 向量化 Vectorizer"
      ]
    },
    {
      "metadata": {
        "id": "MwS5BEV-uBlt",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class SurnameVectorizer(object):\n",
        "    def __init__(self, surname_vocab, nationality_vocab):\n",
        "        self.surname_vocab = surname_vocab\n",
        "        self.nationality_vocab = nationality_vocab\n",
        "\n",
        "    def vectorize(self, surname):\n",
        "        one_hot_matrix_size = (len(surname), len(self.surname_vocab))\n",
        "        one_hot_matrix = np.zeros(one_hot_matrix_size, dtype=np.float32)\n",
        "                               \n",
        "        for position_index, character in enumerate(surname):\n",
        "            character_index = self.surname_vocab.lookup_token(character)\n",
        "            one_hot_matrix[position_index][character_index] = 1\n",
        "        \n",
        "        return one_hot_matrix\n",
        "    \n",
        "    def unvectorize(self, one_hot_matrix):\n",
        "        len_name = len(one_hot_matrix)\n",
        "        indices = np.zeros(len_name)\n",
        "        for i in range(len_name):\n",
        "            indices[i] = np.where(one_hot_matrix[i]==1)[0][0]\n",
        "        surname = [self.surname_vocab.lookup_index(index) for index in indices]\n",
        "        return surname\n",
        "\n",
        "    @classmethod\n",
        "    def from_dataframe(cls, df):\n",
        "        surname_vocab = Vocabulary(add_unk=True)\n",
        "        nationality_vocab = Vocabulary(add_unk=False)\n",
        "\n",
        "        # Create vocabularies\n",
        "        for index, row in df.iterrows():\n",
        "            for letter in row.surname: # char-level tokenization\n",
        "                surname_vocab.add_token(letter)\n",
        "            nationality_vocab.add_token(row.nationality)\n",
        "        return cls(surname_vocab, nationality_vocab)\n",
        "\n",
        "    @classmethod\n",
        "    def from_serializable(cls, contents):\n",
        "        surname_vocab = Vocabulary.from_serializable(contents['surname_vocab'])\n",
        "        nationality_vocab =  Vocabulary.from_serializable(contents['nationality_vocab'])\n",
        "        return cls(surname_vocab, nationality_vocab)\n",
        "\n",
        "    def to_serializable(self):\n",
        "        return {'surname_vocab': self.surname_vocab.to_serializable(),\n",
        "                'nationality_vocab': self.nationality_vocab.to_serializable()}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "zq7RoFAXuBo9",
        "colab_type": "code",
        "outputId": "3555b683-841a-4e2a-a246-c8ae4cb74636",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        }
      },
      "cell_type": "code",
      "source": [
        "# Vectorizer instance\n",
        "vectorizer = SurnameVectorizer.from_dataframe(split_df)\n",
        "print (vectorizer.surname_vocab)\n",
        "print (vectorizer.nationality_vocab)\n",
        "vectorized_surname = vectorizer.vectorize(preprocess_text(\"goku\"))\n",
        "print (np.shape(vectorized_surname))\n",
        "print (vectorized_surname)\n",
        "print (vectorizer.unvectorize(vectorized_surname))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Vocabulary(size=28)>\n",
            "<Vocabulary(size=18)>\n",
            "(4, 28)\n",
            "[[0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            "  0. 0. 0. 0.]\n",
            " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            "  0. 0. 0. 0.]\n",
            " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            "  0. 0. 0. 0.]\n",
            " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            "  0. 0. 0. 0.]]\n",
            "['g', 'o', 'k', 'u']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "wwQ8MNp5ZfeG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Note**: Unlike the bagged ont-hot encoding method in the MLP notebook, we are able to preserve the semantic structure of the surnames. We are able to use one-hot encoding here because we are using characters but when we process text with large vocabularies, this method simply can't scale. We'll explore embedding based methods in subsequent notebooks. "
      ]
    },
    {
      "metadata": {
        "id": "Mnf7gXgKuOgp",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 数据集 Dataset"
      ]
    },
    {
      "metadata": {
        "id": "YYqzM53fuBrf",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from torch.utils.data import Dataset, DataLoader"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "gjolk855uPrA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class SurnameDataset(Dataset):\n",
        "    def __init__(self, df, vectorizer):\n",
        "        self.df = df\n",
        "        self.vectorizer = vectorizer\n",
        "\n",
        "        # Data splits\n",
        "        self.train_df = self.df[self.df.split=='train']\n",
        "        self.train_size = len(self.train_df)\n",
        "        self.val_df = self.df[self.df.split=='val']\n",
        "        self.val_size = len(self.val_df)\n",
        "        self.test_df = self.df[self.df.split=='test']\n",
        "        self.test_size = len(self.test_df)\n",
        "        self.lookup_dict = {'train': (self.train_df, self.train_size), \n",
        "                            'val': (self.val_df, self.val_size),\n",
        "                            'test': (self.test_df, self.test_size)}\n",
        "        self.set_split('train')\n",
        "\n",
        "        # Class weights (for imbalances)\n",
        "        class_counts = df.nationality.value_counts().to_dict()\n",
        "        def sort_key(item):\n",
        "            return self.vectorizer.nationality_vocab.lookup_token(item[0])\n",
        "        sorted_counts = sorted(class_counts.items(), key=sort_key)\n",
        "        frequencies = [count for _, count in sorted_counts]\n",
        "        self.class_weights = 1.0 / torch.tensor(frequencies, dtype=torch.float32)\n",
        "\n",
        "    @classmethod\n",
        "    def load_dataset_and_make_vectorizer(cls, df):\n",
        "        train_df = df[df.split=='train']\n",
        "        return cls(df, SurnameVectorizer.from_dataframe(train_df))\n",
        "\n",
        "    @classmethod\n",
        "    def load_dataset_and_load_vectorizer(cls, df, vectorizer_filepath):\n",
        "        vectorizer = cls.load_vectorizer_only(vectorizer_filepath)\n",
        "        return cls(df, vectorizer)\n",
        "\n",
        "    def load_vectorizer_only(vectorizer_filepath):\n",
        "        with open(vectorizer_filepath) as fp:\n",
        "            return SurnameVectorizer.from_serializable(json.load(fp))\n",
        "\n",
        "    def save_vectorizer(self, vectorizer_filepath):\n",
        "        with open(vectorizer_filepath, \"w\") as fp:\n",
        "            json.dump(self.vectorizer.to_serializable(), fp)\n",
        "\n",
        "    def set_split(self, split=\"train\"):\n",
        "        self.target_split = split\n",
        "        self.target_df, self.target_size = self.lookup_dict[split]\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Dataset(split={0}, size={1})\".format(\n",
        "            self.target_split, self.target_size)\n",
        "\n",
        "    def __len__(self):\n",
        "        return self.target_size\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        row = self.target_df.iloc[index]\n",
        "        surname_vector = self.vectorizer.vectorize(row.surname)\n",
        "        nationality_index = self.vectorizer.nationality_vocab.lookup_token(row.nationality)\n",
        "        return {'surname': surname_vector, 'nationality': nationality_index}\n",
        "\n",
        "    def get_num_batches(self, batch_size):\n",
        "        return len(self) // batch_size\n",
        "\n",
        "    def generate_batches(self, batch_size, collate_fn, shuffle=True, \n",
        "                         drop_last=True, device=\"cpu\"):\n",
        "        dataloader = DataLoader(dataset=self, batch_size=batch_size,\n",
        "                                collate_fn=collate_fn, shuffle=shuffle, \n",
        "                                drop_last=drop_last)\n",
        "        for data_dict in dataloader:\n",
        "            out_data_dict = {}\n",
        "            for name, tensor in data_dict.items():\n",
        "                out_data_dict[name] = data_dict[name].to(device)\n",
        "            yield out_data_dict"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "hvy-CJVSuPuS",
        "colab_type": "code",
        "outputId": "878b8447-2345-4918-d621-182583e88b7e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Dataset instance\n",
        "dataset = SurnameDataset.load_dataset_and_make_vectorizer(split_df)\n",
        "print (dataset) # __str__\n",
        "print (np.shape(dataset[5]['surname'])) # __getitem__\n",
        "print (dataset.class_weights)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Dataset(split=train, size=7680)\n",
            "(6, 28)\n",
            "tensor([0.0006, 0.0045, 0.0024, 0.0042, 0.0003, 0.0044, 0.0017, 0.0064, 0.0055,\n",
            "        0.0017, 0.0013, 0.0130, 0.0083, 0.0182, 0.0004, 0.0133, 0.0039, 0.0172])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "XY0CqM2Rd3Im",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 模型 Model"
      ]
    },
    {
      "metadata": {
        "id": "pWGpAzKPd32f",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "d7Q0_nkjd30L",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class SurnameModel(nn.Module):\n",
        "    def __init__(self, num_input_channels, num_output_channels, num_classes, dropout_p):\n",
        "        super(SurnameModel, self).__init__()\n",
        "        \n",
        "        # Conv weights\n",
        "        self.conv = nn.ModuleList([nn.Conv1d(num_input_channels, num_output_channels, \n",
        "                                             kernel_size=f) for f in [2,3,4]])\n",
        "        self.dropout = nn.Dropout(dropout_p)\n",
        "       \n",
        "        # FC weights\n",
        "        self.fc1 = nn.Linear(num_output_channels*3, num_classes)\n",
        "\n",
        "    def forward(self, x, channel_first=False, apply_softmax=False):\n",
        "        \n",
        "        # Rearrange input so num_input_channels is in dim 1 (N, C, L)\n",
        "        if not channel_first:\n",
        "            x = x.transpose(1, 2)\n",
        "            \n",
        "        # Conv outputs\n",
        "        z = [conv(x) for conv in self.conv]\n",
        "        z = [F.max_pool1d(zz, zz.size(2)).squeeze(2) for zz in z]\n",
        "        z = [F.relu(zz) for zz in z]\n",
        "        \n",
        "        # Concat conv outputs\n",
        "        z = torch.cat(z, 1)\n",
        "        z = self.dropout(z)\n",
        "\n",
        "        # FC layer\n",
        "        y_pred = self.fc1(z)\n",
        "        \n",
        "        if apply_softmax:\n",
        "            y_pred = F.softmax(y_pred, dim=1)\n",
        "        return y_pred"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "7XlJwSKQkL_C",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 训练 Training"
      ]
    },
    {
      "metadata": {
        "id": "rh_1heUNSUYN",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**填充：** 输入必须有相同的维度。我们的向量器把输入向量化，但是特殊的批次中，我们可能碰到不同大小的输入。解决方法是找到最长的输入，把其他的填充到这个长度。通常，数量最小的输入批次，用0向量填充。\n",
        "\n",
        "我们使用 Trainer 类中的 pad_seq 函数，由 collate_fn 触发，传递到 Dataset 类中的 generate_batches 函数。基本上，批次生成器产生样本，我们使用 collate_fn 确定最大的输入，填充批次中的其他输入，得到一个统一的输入维度。\n",
        "\n",
        "**Padding:** the inputs in a particular batch must all have the same shape. Our vectorizer converts the tokens into a vectorizer form but in a particular batch, we can have inputs of various sizes. The solution is to determine the longest input in a particular batch and pad all the other inputs to match that length. Usually, the smaller inputs in the batch are padded with zero vectors. \n",
        "\n",
        "We do this using the pad_seq function in the Trainer class which is invoked by the collate_fn which is passed to generate_batches function in the Dataset class. Essentially, the batch generater generates samples into a batch and we use the collate_fn to determine the largest input and pad all the other inputs in the batch to get a uniform input shape."
      ]
    },
    {
      "metadata": {
        "id": "wLLmIuKRkNYW",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.optim as optim"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "sV-Dc_5ykNgS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Trainer(object):\n",
        "    def __init__(self, dataset, model, model_state_file, save_dir, device, shuffle, \n",
        "               num_epochs, batch_size, learning_rate, early_stopping_criteria):\n",
        "        self.dataset = dataset\n",
        "        self.class_weights = dataset.class_weights.to(device)\n",
        "        self.model = model.to(device)\n",
        "        self.save_dir = save_dir\n",
        "        self.device = device\n",
        "        self.shuffle = shuffle\n",
        "        self.num_epochs = num_epochs\n",
        "        self.batch_size = batch_size\n",
        "        self.loss_func = nn.CrossEntropyLoss(self.class_weights)\n",
        "        self.optimizer = optim.Adam(self.model.parameters(), lr=learning_rate)\n",
        "        self.scheduler = optim.lr_scheduler.ReduceLROnPlateau(\n",
        "            optimizer=self.optimizer, mode='min', factor=0.5, patience=1)\n",
        "        self.train_state = {\n",
        "            'done_training': False,\n",
        "            'stop_early': False, \n",
        "            'early_stopping_step': 0,\n",
        "            'early_stopping_best_val': 1e8,\n",
        "            'early_stopping_criteria': early_stopping_criteria,\n",
        "            'learning_rate': learning_rate,\n",
        "            'epoch_index': 0,\n",
        "            'train_loss': [],\n",
        "            'train_acc': [],\n",
        "            'val_loss': [],\n",
        "            'val_acc': [],\n",
        "            'test_loss': -1,\n",
        "            'test_acc': -1,\n",
        "            'model_filename': model_state_file}\n",
        "    \n",
        "    def update_train_state(self):\n",
        "\n",
        "        # Verbose\n",
        "        print (\"[EPOCH]: {0} | [LR]: {1} | [TRAIN LOSS]: {2:.2f} | [TRAIN ACC]: {3:.1f}% | [VAL LOSS]: {4:.2f} | [VAL ACC]: {5:.1f}%\".format(\n",
        "          self.train_state['epoch_index'], self.train_state['learning_rate'], \n",
        "            self.train_state['train_loss'][-1], self.train_state['train_acc'][-1], \n",
        "            self.train_state['val_loss'][-1], self.train_state['val_acc'][-1]))\n",
        "\n",
        "        # Save one model at least\n",
        "        if self.train_state['epoch_index'] == 0:\n",
        "            torch.save(self.model.state_dict(), self.train_state['model_filename'])\n",
        "            self.train_state['stop_early'] = False\n",
        "\n",
        "        # Save model if performance improved\n",
        "        elif self.train_state['epoch_index'] >= 1:\n",
        "            loss_tm1, loss_t = self.train_state['val_loss'][-2:]\n",
        "\n",
        "            # If loss worsened\n",
        "            if loss_t >= self.train_state['early_stopping_best_val']:\n",
        "                # Update step\n",
        "                self.train_state['early_stopping_step'] += 1\n",
        "\n",
        "            # Loss decreased\n",
        "            else:\n",
        "                # Save the best model\n",
        "                if loss_t < self.train_state['early_stopping_best_val']:\n",
        "                    torch.save(self.model.state_dict(), self.train_state['model_filename'])\n",
        "\n",
        "                # Reset early stopping step\n",
        "                self.train_state['early_stopping_step'] = 0\n",
        "\n",
        "            # Stop early ?\n",
        "            self.train_state['stop_early'] = self.train_state['early_stopping_step'] \\\n",
        "              >= self.train_state['early_stopping_criteria']\n",
        "        return self.train_state\n",
        "  \n",
        "    def compute_accuracy(self, y_pred, y_target):\n",
        "        _, y_pred_indices = y_pred.max(dim=1)\n",
        "        n_correct = torch.eq(y_pred_indices, y_target).sum().item()\n",
        "        return n_correct / len(y_pred_indices) * 100\n",
        "    \n",
        "    def pad_seq(self, seq, length):\n",
        "        vector = np.zeros((length, len(self.dataset.vectorizer.surname_vocab)),\n",
        "                          dtype=np.int64)\n",
        "        for i in range(len(seq)):\n",
        "            vector[i] = seq[i]\n",
        "        return vector\n",
        "    \n",
        "    def collate_fn(self, batch):\n",
        "        \n",
        "        # Make a deep copy\n",
        "        batch_copy = copy.deepcopy(batch)\n",
        "        processed_batch = {\"surname\": [], \"nationality\": []}\n",
        "        \n",
        "        # Get max sequence length\n",
        "        max_seq_len = max([len(sample[\"surname\"]) for sample in batch_copy])\n",
        "        \n",
        "        # Pad\n",
        "        for i, sample in enumerate(batch_copy):\n",
        "            seq = sample[\"surname\"]\n",
        "            nationality = sample[\"nationality\"]\n",
        "            padded_seq = self.pad_seq(seq, max_seq_len)\n",
        "            processed_batch[\"surname\"].append(padded_seq)\n",
        "            processed_batch[\"nationality\"].append(nationality)\n",
        "            \n",
        "        # Convert to appropriate tensor types\n",
        "        processed_batch[\"surname\"] = torch.FloatTensor(\n",
        "            processed_batch[\"surname\"]) # need float for conv operations\n",
        "        processed_batch[\"nationality\"] = torch.LongTensor(\n",
        "            processed_batch[\"nationality\"])\n",
        "        \n",
        "        return processed_batch    \n",
        "  \n",
        "    def run_train_loop(self):\n",
        "        for epoch_index in range(self.num_epochs):\n",
        "            self.train_state['epoch_index'] = epoch_index\n",
        "      \n",
        "            # Iterate over train dataset\n",
        "\n",
        "            # initialize batch generator, set loss and acc to 0, set train mode on\n",
        "            self.dataset.set_split('train')\n",
        "            batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, collate_fn=self.collate_fn,\n",
        "                shuffle=self.shuffle, device=self.device)\n",
        "            running_loss = 0.0\n",
        "            running_acc = 0.0\n",
        "            self.model.train()\n",
        "\n",
        "            for batch_index, batch_dict in enumerate(batch_generator):\n",
        "                # zero the gradients\n",
        "                self.optimizer.zero_grad()\n",
        "\n",
        "                # compute the output\n",
        "                y_pred = self.model(batch_dict['surname'])\n",
        "\n",
        "                # compute the loss\n",
        "                loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "                loss_t = loss.item()\n",
        "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "                # compute gradients using loss\n",
        "                loss.backward()\n",
        "\n",
        "                # use optimizer to take a gradient step\n",
        "                self.optimizer.step()\n",
        "                \n",
        "                # compute the accuracy\n",
        "                acc_t = self.compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "            self.train_state['train_loss'].append(running_loss)\n",
        "            self.train_state['train_acc'].append(running_acc)\n",
        "\n",
        "            # Iterate over val dataset\n",
        "\n",
        "            # initialize batch generator, set loss and acc to 0; set eval mode on\n",
        "            self.dataset.set_split('val')\n",
        "            batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, collate_fn=self.collate_fn, \n",
        "                shuffle=self.shuffle, device=self.device)\n",
        "            running_loss = 0.\n",
        "            running_acc = 0.\n",
        "            self.model.eval()\n",
        "\n",
        "            for batch_index, batch_dict in enumerate(batch_generator):\n",
        "\n",
        "                # compute the output\n",
        "                y_pred =  self.model(batch_dict['surname'])\n",
        "\n",
        "                # compute the loss\n",
        "                loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "                loss_t = loss.to(\"cpu\").item()\n",
        "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "                # compute the accuracy\n",
        "                acc_t = self.compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "            self.train_state['val_loss'].append(running_loss)\n",
        "            self.train_state['val_acc'].append(running_acc)\n",
        "\n",
        "            self.train_state = self.update_train_state()\n",
        "            self.scheduler.step(self.train_state['val_loss'][-1])\n",
        "            if self.train_state['stop_early']:\n",
        "                break\n",
        "          \n",
        "    def run_test_loop(self):\n",
        "        # initialize batch generator, set loss and acc to 0; set eval mode on\n",
        "        self.dataset.set_split('test')\n",
        "        batch_generator = self.dataset.generate_batches(\n",
        "            batch_size=self.batch_size, collate_fn=self.collate_fn,\n",
        "            shuffle=self.shuffle, device=self.device)\n",
        "        running_loss = 0.0\n",
        "        running_acc = 0.0\n",
        "        self.model.eval()\n",
        "\n",
        "        for batch_index, batch_dict in enumerate(batch_generator):\n",
        "            # compute the output\n",
        "            y_pred =  self.model(batch_dict['surname'])\n",
        "\n",
        "            # compute the loss\n",
        "            loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "            loss_t = loss.item()\n",
        "            running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "            # compute the accuracy\n",
        "            acc_t = self.compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "            running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "        self.train_state['test_loss'] = running_loss\n",
        "        self.train_state['test_acc'] = running_acc\n",
        "    \n",
        "    def plot_performance(self):\n",
        "        # Figure size\n",
        "        plt.figure(figsize=(15,5))\n",
        "\n",
        "        # Plot Loss\n",
        "        plt.subplot(1, 2, 1)\n",
        "        plt.title(\"Loss\")\n",
        "        plt.plot(trainer.train_state[\"train_loss\"], label=\"train\")\n",
        "        plt.plot(trainer.train_state[\"val_loss\"], label=\"val\")\n",
        "        plt.legend(loc='upper right')\n",
        "\n",
        "        # Plot Accuracy\n",
        "        plt.subplot(1, 2, 2)\n",
        "        plt.title(\"Accuracy\")\n",
        "        plt.plot(trainer.train_state[\"train_acc\"], label=\"train\")\n",
        "        plt.plot(trainer.train_state[\"val_acc\"], label=\"val\")\n",
        "        plt.legend(loc='lower right')\n",
        "\n",
        "        # Save figure\n",
        "        plt.savefig(os.path.join(self.save_dir, \"performance.png\"))\n",
        "\n",
        "        # Show plots\n",
        "        plt.show()\n",
        "    \n",
        "    def save_train_state(self):\n",
        "        self.train_state[\"done_training\"] = True\n",
        "        with open(os.path.join(self.save_dir, \"train_state.json\"), \"w\") as fp:\n",
        "            json.dump(self.train_state, fp)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "OkeOQRwckNd1",
        "colab_type": "code",
        "outputId": "77032240-5a41-4461-8767-89827357f6f7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "cell_type": "code",
      "source": [
        "# Initialization\n",
        "dataset = SurnameDataset.load_dataset_and_make_vectorizer(split_df)\n",
        "dataset.save_vectorizer(args.vectorizer_file)\n",
        "vectorizer = dataset.vectorizer\n",
        "model = SurnameModel(num_input_channels=len(vectorizer.surname_vocab),\n",
        "                     num_output_channels=args.num_filters,\n",
        "                     num_classes=len(vectorizer.nationality_vocab),\n",
        "                     dropout_p=args.dropout_p)\n",
        "print (model.named_modules)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<bound method Module.named_modules of SurnameModel(\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(28, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(28, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(28, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=18, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "3JJdOO4ZkNb3",
        "colab_type": "code",
        "outputId": "c80d4f6a-a05e-4f58-fb4f-91ccd95f68a3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "cell_type": "code",
      "source": [
        "# Train\n",
        "trainer = Trainer(dataset=dataset, model=model, \n",
        "                  model_state_file=args.model_state_file, \n",
        "                  save_dir=args.save_dir, device=args.device,\n",
        "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
        "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
        "                  early_stopping_criteria=args.early_stopping_criteria)\n",
        "trainer.run_train_loop()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 2.82 | [TRAIN ACC]: 20.2% | [VAL LOSS]: 2.71 | [VAL ACC]: 37.1%\n",
            "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 2.54 | [TRAIN ACC]: 43.9% | [VAL LOSS]: 2.40 | [VAL ACC]: 49.8%\n",
            "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 2.18 | [TRAIN ACC]: 48.7% | [VAL LOSS]: 2.12 | [VAL ACC]: 48.2%\n",
            "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 1.88 | [TRAIN ACC]: 51.1% | [VAL LOSS]: 1.90 | [VAL ACC]: 50.6%\n",
            "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 1.67 | [TRAIN ACC]: 54.8% | [VAL LOSS]: 1.74 | [VAL ACC]: 50.6%\n",
            "[EPOCH]: 05 | [LR]: 0.001 | [TRAIN LOSS]: 1.52 | [TRAIN ACC]: 57.4% | [VAL LOSS]: 1.65 | [VAL ACC]: 55.8%\n",
            "[EPOCH]: 06 | [LR]: 0.001 | [TRAIN LOSS]: 1.37 | [TRAIN ACC]: 60.0% | [VAL LOSS]: 1.60 | [VAL ACC]: 60.4%\n",
            "[EPOCH]: 07 | [LR]: 0.001 | [TRAIN LOSS]: 1.28 | [TRAIN ACC]: 62.2% | [VAL LOSS]: 1.53 | [VAL ACC]: 59.8%\n",
            "[EPOCH]: 08 | [LR]: 0.001 | [TRAIN LOSS]: 1.20 | [TRAIN ACC]: 63.0% | [VAL LOSS]: 1.49 | [VAL ACC]: 58.5%\n",
            "[EPOCH]: 09 | [LR]: 0.001 | [TRAIN LOSS]: 1.12 | [TRAIN ACC]: 64.4% | [VAL LOSS]: 1.43 | [VAL ACC]: 61.9%\n",
            "[EPOCH]: 10 | [LR]: 0.001 | [TRAIN LOSS]: 1.05 | [TRAIN ACC]: 65.8% | [VAL LOSS]: 1.40 | [VAL ACC]: 62.7%\n",
            "[EPOCH]: 11 | [LR]: 0.001 | [TRAIN LOSS]: 1.00 | [TRAIN ACC]: 66.3% | [VAL LOSS]: 1.38 | [VAL ACC]: 63.4%\n",
            "[EPOCH]: 12 | [LR]: 0.001 | [TRAIN LOSS]: 0.95 | [TRAIN ACC]: 67.2% | [VAL LOSS]: 1.34 | [VAL ACC]: 64.7%\n",
            "[EPOCH]: 13 | [LR]: 0.001 | [TRAIN LOSS]: 0.90 | [TRAIN ACC]: 69.1% | [VAL LOSS]: 1.35 | [VAL ACC]: 63.9%\n",
            "[EPOCH]: 14 | [LR]: 0.001 | [TRAIN LOSS]: 0.86 | [TRAIN ACC]: 69.0% | [VAL LOSS]: 1.31 | [VAL ACC]: 67.8%\n",
            "[EPOCH]: 15 | [LR]: 0.001 | [TRAIN LOSS]: 0.82 | [TRAIN ACC]: 70.8% | [VAL LOSS]: 1.32 | [VAL ACC]: 65.1%\n",
            "[EPOCH]: 16 | [LR]: 0.001 | [TRAIN LOSS]: 0.78 | [TRAIN ACC]: 71.0% | [VAL LOSS]: 1.31 | [VAL ACC]: 66.1%\n",
            "[EPOCH]: 17 | [LR]: 0.001 | [TRAIN LOSS]: 0.74 | [TRAIN ACC]: 72.2% | [VAL LOSS]: 1.28 | [VAL ACC]: 69.2%\n",
            "[EPOCH]: 18 | [LR]: 0.001 | [TRAIN LOSS]: 0.72 | [TRAIN ACC]: 72.6% | [VAL LOSS]: 1.30 | [VAL ACC]: 68.1%\n",
            "[EPOCH]: 19 | [LR]: 0.001 | [TRAIN LOSS]: 0.71 | [TRAIN ACC]: 72.7% | [VAL LOSS]: 1.30 | [VAL ACC]: 69.8%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "0QLZfEyznVpT",
        "colab_type": "code",
        "outputId": "67d28ed8-2252-4947-ffc5-71c5c4f4fedd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 335
        }
      },
      "cell_type": "code",
      "source": [
        "# Plot performance\n",
        "trainer.plot_performance()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2gAAAE+CAYAAAD4XjP+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd4HOW99vHvNvUurXqvlmzJRe69\nNwzG2GAggIMDCTkhhBMOIcmbE0LgpBBIQgIJCb1jkAsG4yYjV2zJ3bIlWb23Ve9ld+f9Q7ZAAXdJ\nu2v/Ptfla7XzzM7eq0uW9t6ZeUalKIqCEEIIIYQQQgiLU1s6gBBCCCGEEEKIPlLQhBBCCCGEEMJK\nSEETQgghhBBCCCshBU0IIYQQQgghrIQUNCGEEEIIIYSwElLQhBBCCCGEEMJKSEET4irFxcVRXV1t\n6RhCCCHEsLjzzju55ZZbLB1DiOueFDQhhBBCCHFRubm5uLq6EhgYyPHjxy0dR4jrmhQ0IQZZd3c3\nv/71r1m0aBFLlizhD3/4AyaTCYB3332XJUuWsHjxYlatWkVeXt5FlwshhBDWYOPGjSxevJhly5ax\nadOm/uWbNm1i0aJFLFq0iMcff5yenp4LLk9PT2fBggX9j/36/b///e/86le/YtWqVbz55puYzWae\neuopFi1axNy5c3n88cfp7e0FoKGhgYceeoh58+Zx8803s3//fnbv3s2yZcsGZL7ttttITU0d6m+N\nEINOa+kAQlxv3nrrLaqrq9myZQtGo5F77rmHzz77jHnz5vHCCy+QlpaGi4sLW7duZffu3QQEBHzr\n8piYGEu/FCGEEAKTycTOnTv50Y9+hEaj4fnnn6enp4fa2lr++Mc/smnTJnx9ffnxj3/M22+/zeLF\ni791eWJi4kWfZ8+ePXzyySd4eXmxfft2jhw5wmeffYbZbGbFihV8/vnnLF++nOeff56oqChefvll\nsrKyuP/++9m3bx8Gg4GcnBxGjBhBZWUlpaWlzJw5c5i+S0IMHiloQgyy3bt3s3btWrRaLVqtlptv\nvpkDBw6wdOlSVCoVKSkpLFu2jCVLlgDQ29v7rcuFEEIIa7B//34SExNxcXEBYOLEiaSlpdHU1MTY\nsWPx8/MD4Pnnn0ej0bB+/fpvXX706NGLPs/o0aPx8vICYNGiRcyZMwedTgdAYmIiZWVlQF+Re+WV\nVwBISEhg165d2NnZsWjRIrZs2cKIESNITU1l3rx52NnZDf43RIghJoc4CjHIGhoacHd377/v7u5O\nfX09Op2ON998k2PHjrFo0SLuvvtuzp49e8HlQgghhDXYsGEDu3fvZvz48YwfP54dO3awceNGGhsb\ncXNz61/P3t4erVZ7weWX8vW/nQ0NDTzxxBMsWrSIxYsXs2vXLhRFAaCpqQlXV9f+dc8Xx5tuuokt\nW7YAkJqaytKlS6/thQthIVLQhBhkPj4+NDU19d9vamrCx8cH6Puk729/+xsHDx5k+vTpPPnkkxdd\nLoQQQlhSc3MzGRkZpKenc+TIEY4cOcLhw4fJzMxErVbT2NjYv25bWxt1dXV4enp+63KNRtN/TjZA\nS0vLBZ/3L3/5C1qtlk8//ZRt27Yxa9as/jEPD48B2y8vL6e3t5cJEyZgNBpJS0sjLy+PqVOnDta3\nQYhhJQVNiEE2e/ZsUlJSMJlMdHR08MknnzBr1izOnj3LI488Qk9PD3Z2dowaNQqVSnXB5UIIIYSl\nbdmyhcmTJw84VFCr1TJ9+nR6eno4duwY5eXlKIrCk08+SUpKCrNmzfrW5Xq9HoPBQH19PSaTiU8/\n/fSCz1tfX09sbCx2dnbk5ORw/PhxOjo6AJg7dy4bN24EID8/n9tuuw2TyYRarWbp0qU8/fTTzJ07\nt//wSCFsjZyDJsQ1uPfee9FoNP33n3nmGe69917Kysq46aabUKlULF68uP+8suDgYJYtW4ZOp8PZ\n2Zlf//rXxMbGfutyIYQQwtI2bdrEmjVrvrF8wYIF/OMf/+C3v/0ta9asQaPRkJiYyP3334+9vf0F\nl69cuZJbb72VwMBAli9fTnZ29rc+79q1a3niiSfYsGED48eP54knnuD//b//R1JSEo8//jhPPPEE\nc+fOxdnZmeeeew4HBweg7zDHN954Qw5vFDZNpZw/oFcIIYQQQggbVldXx4oVK9i9e/eAD1CFsCVy\niKMQQgghhLgu/O1vf+Ouu+6SciZsmhQ0IYQQQghh0+rq6pg3bx51dXWsXbvW0nGEuCZyiKMQQggh\nhBBCWAnZgyaEEEIIIYQQVkIKmhBCCCGEEEJYiWGfZt9gaL3mbXh6OtHY2DEIaYaX5B4+tpgZbDO3\nLWYG28xti5n1eldLR7Ap8jfStnLbYmawzdy2mBlsM7ctZgbby32xv482uQdNq7XNmXkk9/Cxxcxg\nm7ltMTPYZm5bzCyGn63+nNhiblvMDLaZ2xYzg23mtsXMYLu5v41NFjQhhBBCCCGEuB5JQRNCCCGE\nEEIIKyEFTQghhBBCCCGshBQ0IYQQQgghhLASUtCEEEIIIYQQwkpIQRNCCCGEEEIIKyEFTQghhBBC\nCCGshBQ0IYS4gezeveuy1nvhheeprKwY4jRCCCGE+E9S0IQQ4gZRVVVJaur2y1r3Jz95jMDAoCFO\nJIQQQoj/pLV0gCvV3WPi032FjIn0xMHO5uILIYTF/PnPfyQ7+wwzZkxg4cIlVFVV8te//oPf//63\nGAy1dHZ2snbt95k2bQYPP/x9fvrTn5GWtov29jZKS0uoqCjnkUceY8qUaZZ+KUIIIcSgURQFo8lM\nV4+Jrh4T3T0munrP3faY6O41DliWFOVDZKDbkOWxuYZztqyRf2/KZF5yMN9ZEGvpOEIIYTPuuute\nNmz4iIiIKEpLi/nHP16lsbGBiRMns2TJMioqyvnf//0506bNGPC42toannvubxw69CWffLJeCpoQ\nQgirpCgKHd1Gmtp6aG7rprmth6b2vtvm9h46uwcWre7erwqZWVEu+3kq6tr50YrEIXsdNlfQEsK9\nCPBxZvfxCuYnB+Pn5WTpSEIIccU++iKfwzm1V/QYjUaFyXThPyATRvhyx9zoy9pWfPxIAFxd3cjO\nPsPmzRtQqdS0tDR/Y92kpDEA+Pr60tbWdkWZhRBCiGtlVhRaO3ppbuvuL19N7QNLWGunkcaWLnqN\n5ktuz06nxkGnwcFOi4u7Dns7DQ46zYDbr77W4mCnwf78cp2GiADXIX29NlfQtBo1a5Ym8Ie3D5Oy\np2BI26sQQlyvdDodADt3bqOlpYWXXnqVlpYWHnjg3m+sq9Fo+r9WruATRiGEEOJqdHT1crqogZP5\n9Zwta6Spteeie7jUKhWebvYE+Tjj4WKPu4sd7s52uLvY43H+1sUOR3st9joNarVqGF/NlbO5ggYw\nNSmAqCA3jp41kF/eTHSwu6UjCSHEFbljbvRl7+06T693xWBovernVKvVmEymAcuampoICAhErVaz\nZ88X9Pb2XvX2hRBCiKuhKArVDR2czK/nVEEdeeXNmMx9hczNSUdkoBvuLnZ4OJ8rXy52fUXMue/W\nxUmHn6/bNf2NtCY2WdBUKhWr58Twu3ePsi4tj1/ek4xKZd1NWAghLC0sLIKzZ3MICAjEw8MDgNmz\n5/Lzn/+UrKzT3HTTLfj6+vLGG69YOKkQQojrndFkJresiZP59ZwsqKO2sbN/LCLAldFRPoyO9iHU\nz+WGe59vkwUNIDrYneRYPUdzDRw9a2D8CF9LRxJCCKvm6enJhg1bBiwLCAjkrbc+7L+/cOESAO6/\n/0EAIiO/2ssXGRnNiy/+exiSCiGEuB61tPdwqqCvkJ0paqCrp++oDns7DcmxepKivUmK9Mbdxd7C\nSS3LZgsawKrZUZzIryNldwFjYnzQauSybkIIIYQQQlgDRVEorWnjVEEdJwvqKaps4fyZZHoPB6Yn\nBTA6yofYEA90Wnkff55NFzQ/Lydmjw1i19Fy0o5XsGB8iKUjCSGEEEIIcd1TFIWeXjPtXb20dxnp\nOHfb3tl3W93QQWZhPY2t3UDfRB6xIR6MjvZhdLQ3/l5ON9yhi5fLpgsawC3TwvnydBWfHihm2ih/\nnBx0lo4khBBCCCGEzWls7aawpo2q2tb+stXRZaS9+9xt58Aydn4ijwtxdtAyZaQfo6N9GBnhhbO8\nT78sNl/QXJ3sWDo5jPV7CtlyqITbZ1/ZrGhCCCHEtfr444/ZvHlz//3Tp0/zwQcf8Jvf/AaAuLg4\nnnrqKQulE0KIiyurbWNbegkZ2bUXLV0atQonBy1ODjr0Ho44OehwdtDi5KDFuf/rvlsPV3vC/Fyt\nfkp7a2TzBQ1gwfgQ0o5XsPNwOXPHBuPt7mDpSEIIIW4gt99+O7fffjsAGRkZbN26lf/7v//jl7/8\nJUlJSTz22GPs2bOHWbNmWTipEEL0URSF7JJGtqWXcrqoAYBAH2fmTwxFoygDypeLow4nh75riMlh\niUPvuihodjoNK2ZE8tqWbDbsLeTBmxMsHUkIIcQN6qWXXuL3v/8999xzD0lJSQDMmTOHgwcPSkET\nQlicyWzmSI6BbemllNT0XTcsLsSDxZNCSYzyvq6uJ2arbK6gVbZV8+zRF7gzdiWhrsH9y6eM8mfn\n4TIOnqlm4YQQwvxdLZhSCCFs16pVN/P22+twcnKydBSbc+rUKQICAtBoNLi5ufUv9/b2xmAwWDCZ\nEOJG19VjZN+pKnYeLqOuuQuVCsbH6Vk8KYzIQLdLb0AMG5sraEbFSElzBZ/kb+XHYx/sX65Wqbh9\nbjTPf3iCj9Ly+Z87x8guWCGEEMMqJSWFFStWfGO5olz8RPrzPD2d0Go115xDr7fNDyltMbctZgbb\nzG2LmcHyuRtbu/hsfxGfHyiirbMXO52GpVPDuXVWNAE+zt/6GEtnvlpDmVtRFLINeaQWHiBBH8P8\nqOlD9lw2V9BCXYNJ9BtBZk0Ohc0lRLqH9Y+NDPdiVKQXpwsbyCysJynKx4JJhRDCuqxd+x1+97vn\n8ff3p7q6il/84jH0el86Ozvp6uriv//7cRISRlk6pk1LT0/nV7/6FSqViqampv7lNTU1+Pr6XvLx\njY0d15xBr3e1ycOTbDG3LWYG28xti5nBsrmrGzrYnlHKgcxqjCYzLo46lk+PYO64IFyd7EAxf2s2\n+V4P1Gs2crTmBGll+ylvqwTAzmzPaLdre66LlUmbK2gAq0YuJbMmh63Fqfxo9PcGjN0xJ5ozRRl8\nnFbAyAgvNGq56J0QQgDMnDmHAwf2snLlHezbt4eZM+cQFRXDzJmzOXr0MO+99xb/939/snRMm1VT\nU4OzszN2dnYAREZGcuTIEcaPH8+OHTu49957LZxQCHEjyC9vZmt6CSfy6lDouyD0oomhTEsMwF53\n7XvobxTN3a3srzjIvopDtPa2oULFWH0ic0JmDNhBNBRssqDF62OI8Ygkq/4sJS1lhLl9dYHqYL0L\n0xMD2HeqigOZ1cwcHWjBpEII8e025H/G8drMK3qMRq266PTHY30TuS162QXHZ86cw4sv/pWVK+9g\n//49PPzwf/Phh+/wwQfv0Nvbi4ODzIB7LQwGA15eXv33f/nLX/LrX/8as9nM6NGjmTp1qgXTCSGs\nTXeviaNna+nuMaHRqNFp1Gi1arQaVd/X5+73fa3qu9+/TNX/tVqlwqwonMirY1t6KfkVzQBEBLix\nZFIo42L1NjvV/em6bHaV7kXv5EOCVyyxntE46RyH9DlLW8vZXXaAozUnMComHLWOzA+dxcygqXg7\neg7pc59nkwUNYGnEfF44/m+2FqfyUNL9A8ZunRFJenYNG/cVMineD3s7+bRACCEiI6OorzdQU1NN\na2sr+/btxsfHl//936fJycnixRf/aumINm3UqFG8+uqr/fejo6N5//33LZhICGGNunqMpB2rYHtG\nKS0dvde8PY1ahUqlwmgyAzA6ypvFk0KJDfGw2fkYFEVhe0kanxVuR0Eht6mAA5XpqFAR7hZKvHcs\n8V6xhLkGo1Ff+/t8s2LmlOEMaeX7yW8qAsDPSc/s4OlMCkjGXmN3zc9xJWy2oMV4RBHpHk5mXTZl\nrRWEuAb1j3m62rNoQiifflnM9oxSbpkeYcGkQgjxTbdFL7vo3q5vMxjH10+ZMp1///sfzJgxi6am\nRqKiYgDYsycNo9F4TdsWQghxYZ3dRnYdLWfH4TLaOntxtNewbGoYwXoXeo1mjCYzRpPyta/77htN\nZnpNZoznlvealP6v+8dMCqG+LiycGErQBSb+sBVdxm7ezf6I44ZMPOzdeWDUvSgoZDfkkl2fS3FL\nKUUtJXxetBNHrSNxntHEe8UwzWkcKq6sSHX0dvJlVQZ7y7+kvqsRgHivWOaETCfeKxa1yjKnStls\nQVOpVCwNn8+LJ19lW/EuHky8b8D44kmh7DlRwdb0UmaNCcTdxd5CSYUQwnrMmjWHhx5ay5tvfkBX\nVyfPPPMkaWmprFx5B6mpO9iyZbOlIwohxHWlvauX1CPl7DxcRke3EWcHLbfOiGB+cjBODjpLx7Mq\ndZ0N/OvUm1S2VxPlHs4DiffiZtc3mUakexg3RSygo7eT3Mb8vsLWkMsJQyYnDJl8cHYDvk4+xHv1\n7V2L8YjCQfvt7/9rOwzsLj/Awaoj9Jh60Kl1TA+azJzgafg7+w3nS/5WNlvQAEZ4xRDuFsoJw2kq\n2qoIcgnoH3O017J8RiTvbD/Lpv1FrFk8woJJhRDCOsTHj2TPnvT++++9l9L/9fTpfRdRvummW4Y9\nlxBCXG/aOnvZvjWbzfsK6Ow24eKoY+WsSOaOC8bR3qbfgg+JnIY8Xj/9Hu3GDmYETWFVzM1o1d/8\nPjnpHBnjm8gY30QURcHQWUd2Qx6FbYVk1uSwp/xL9pR/iUalIdI9jBFesSR4xRLsGkhuYwFpZfs4\nXZ8DgIe9O0vC5zEtcBLOOuu59qdN/3SoVCqWhM/jn6feYFvxLr436p4B4zNHB5B6pIy9JyuZPz7E\n5nf5CiGEEEII69bS0cP2jFK+OFZBd48JNycdy+aEM2dsEA52Nv3We0goikJa+X425m9BhYq741Yy\nLWjSZT1WpVLh66TH10nPKv0iqmoaKWouJachl6yGXPKbishrKuTTwm3o1Fp6zX2H8ke4hTEnZDpj\n9KMG5Ry2wWbzPyUjvUcQ6hrE8dpMqttrBuyW1KjVrJodxd/XZ5KSls9Pbh9twaRCCCGEEOJ61dzW\nzbaMUtKOV9DTa8bd2Y57l8STHO095NPbl7aU46h1RO/kPaTPM9h6Tb18cHYD6dVHcbVz4cFR9xHl\nEX7V29OqtcR4RhLjGcnNUYtp62nnbGNe3x625mJCXIOYEzKdcLfQwXsRQ8DmC5pKpWJx+Hz+nfkW\n24q/4Lsj7xowPibah9gQD04W1JNT0siIsOGZHlMIIYQQQlz/Glu72XqohD0nK+k1mvF0tef22WHM\nSAogKNBjyC/6nFF9jLez1qFRa1getYTZwdMsNrnFlWjsauKVzHcoaS0jzDWE7yfdh4e9+6A+h4ud\nM8l+Y0j2GzOo2x1qNl/QAJJ8EghyCeBIzQmWRMzHz0nfP6ZSqVg9N5qn3zrCurR8/nfNeNQ2OuWo\nEEIIIYSwDvXNXXyeXsK+k5UYTQrebg7cNCWMaYkB6LTDU5DOlzMHrT0alYb1eZ+SWZfNvfG34+Vg\nvTslCpqKeeX027T2tDHJP5m74m5Dp5EJU867Lgpa37lo83n19DtsL/6C+xJWDxiPCHBjUoIf6Vk1\nZGTVMHmkv4WSCiGEEEIIW9XW2UtBRTPH8wwcyKzGZFbQeziwbEo4U0b5o9UM356rw9XH+8vZj8c8\niJeDJ+/lpJBZl8XvMv7CHbG3MsFvrNVdC21/xSE+yv0EBYVVMbcwO3ia1WW0tOuioAGM1o8kwNmP\nwzXHWRoxHx/HgcfgrpwZydGztazfU0BynB6d1vpOCBRCCCGEENbBrChU1bVTUNlCfnkz+RXNVDd0\n9I/7eTqybGo4k0f6oVEP7yGFh6uP81bWh/3lLMwtBIAfJK7hYNVhUvI281bWh2TWZXFn3G1WMUOh\n0Wzk47zN7K84hLPOiQdG3UOsZ7SlY1ml66agqVVqFofP440z77O9OI3vxK8aMO7j4ci85GC2Z5SR\nerScJZPCLJRUCCGEEEJYm85uI4VVLRRU9JWxwooWOrqN/eP2dhoSwj2JDnInOtidhDAv1Orh3/Nz\noXIGfUeVTQ2cSKxnFG9lreNY7SkKmoq4J/4OErzjhj3reS09rbya+Q4FzcUEuQTwg8Q1eDt6WSyP\ntbtuChrAON8kPi/ayaHqIywOn4e348Bjb5dNDWf/qSo++7KEGUmBuDjKsa5CCCGEEDcaRVEwNHWS\nX9FMQUUL+RXNlBvaUJSv1vH1dGR0tA/Rwe5EB7kT5ONskUL2dRcrZ1/n4+jNf497iNSSPXxWtIOX\nTr7GzKCprIheip3Gblgzl7SU8e/Mt2nqbibZdzTfib8d+2HOYGuuq4J2fi/aW1kfsqM0jbvibhsw\n7uygY9nUcNZ9kc+nB4q5a36MhZIKIYQQQojhYDKbqWvuoqahkwpD27lS1kxLR2//Ojqtmpggd6KC\n3YkOdCcqyB03Z+sqEUcus5ydp1apWRg+h3jvWN7M+pC9FV+S05jLdxPuuuRjB0tG9THez0nBaDax\nPGoJC0Jny/lml+G6KmgAyb6j2VK0k0OVh1kcNhdPB48B43PHBbPraDlfHCtnXnIQvp6WPyZXCCGE\nEEJcPbNZob6li5rGDmoaOqlp7KC2sZOahg7qmrswmZUB63u62jNhhC/RQX1lLNTPZVgn+LhSR6qP\n8+YVlLOvC3EN4ufjH2Fz4Ta+KNvHc0dfYnH4PBaHzR2yizSbzCY2FXzOF2X7cNQ68GDifYz0HjEk\nz3U9uu4KmkatYXHYXN7N+ZidpXu4I3b5gHGdVs3KWVH8a/MZ1u8p5Ie3jrJQUiGEEEIIcbnMikJT\nazfVDR3UnCtftY19ZczQ1InRpHzjMS6OOsL9XfH1dMLPy5EAb2eiAt3wcnOwwCu4Ol8vZw+PeeCq\n9n7pNDpWxtzMKO943sn+iM+LdnKmPoc1CXcOuDzV1eo19VLcUkZ+UxH5TYUUtpTQY+rB38mX7yet\nGZTnuJFcdwUNYKL/OLYWp3KgMp1FYXNwt3cbOB7vy47DpRzOqWVhRTNRQYN7UTwhhBBCCHFtzGaF\nnNJGMrJrKKlpo7KunV6j+RvrOdlrCfF1wc/TCV9PR/y8nPA7V8icHWx7voEjNScGlLNwt9Br2l6c\nVzS/nPjffJT7CYdrjvH7jL9yW/QyZgRNvqJDD7uMXRQ1l5LfVEheUxElLaUYFVP/eICzH/FesSyN\nWICj1nbKsLW4LguaRq1hYdgcPji7gdTSPayMuXnAuEql4o450fzx/eN8+mUxj94+2kJJhRBCCCHE\neYqiUFDZQkZWDYdzamlu7wHA0V5LoLczfl6OfXvD+ouYIy6OuuvyvKYjNSd488wH2GsGp5yd56Rz\n5Lsj7yRJn8CHORtYl7uRzLosvhO/Cg/7b99p0d7bQUFT0bk9ZEWUtVVgVvrKsgoVIa6BRHtEEu0R\nQZR7BC52zoOS9UZ1WQXt2Wef5ejRoxiNRn7wgx+wcOHC/rG5c+fi7++PRtN3DOtzzz2Hn5/f0KS9\nApMCxrOt+Av2VRxiQdhs3OxcB4zHhXoSFehGZkE9NY0d+Mm5aEIIIYQQw05RFMpq28jIriUju4a6\n5i6g7/DE2WMCmZTgx9SxIdTXt1k46fD5ejn78djBK2dfN843iUj3MN7N/pishrP8Lv0v3DniNhbp\np9Hc3dJfxvKbCqlsr+5/nEalIdwtpL+QRbqHy16yQXbJgnbo0CHy8vJYt24djY2NrFixYkBBA3jl\nlVdwdraupqxTa1kYNpt1uZvYVbqXFdE3fWOd+eND+NfmM+w6Ws7d82MtkFIIIYQQ4sZU09BBenYN\n6Vk1VNX3XQDa3k7DlJH+TErwIyHcs3/iDktPbz+cjg5DOTvPw96dH43+HvsqDrIhfwuvnX6XTQVb\nqO9s7F9Hp9YR6xlNtEcEMR4RhLuFDvtU/TeaSxa0CRMmkJSUBICbmxudnZ2YTKb+PWbWbErABLYV\nf8HeioMsCJ39jd2tyXF6PFzs2H+qihUzInG0vy6P+BRCCCGEsAoNLV1kZNeSnl1DSXUrAFqNmuRY\nPZMS/EiK8sZOZ/3vMYfK0ZoTvDFM5ew8lUrFzOCpxHlG815OCtUdNYz0HkG0RwTRHpGEugahVct7\n5OF0ye+2RqPByanv8L+UlBRmzpz5jXL25JNPUlFRQXJyMo899pjVHAes0+hYEDablLzNfFG2j1ui\nFg8Y12rUzBkbxMZ9RRzIrGL++OG5JoQQQgghxI2ipaOHozm1pGfXklfWhAKoVSpGRXoxKd6PcbF6\nm/mQvLWnjZOG0xQ2l6B39CbYNZAQ1yDc7dyu+f3v18vZYJ5zdrn8nH35afJ/ode7YjC0Dutzi4Eu\n+39DamoqKSkpvP766wOWP/LII8yYMQN3d3d+9KMfsX37dhYvXnyBrYCnpxNa7bV/MqLXu156JWC5\n5zx2lu1mb8WXrB67FBf7gXvRVs6P47ODJew+UcnqRfFDvgv9cnNbG1vMbYuZwTZz22JmsM3ctphZ\nCHHj6e4xcTS3lkNZNWQVNWJW+qbAjw3xYFKCH8lxetycbOMwucauJk4YTnPCkElBUzEK35zO39XO\nhRCXIEJcz/8LxNvB67Kf4+i52RrPl7MI9+EtZ8K6XFZB27dvHy+//DKvvvoqrq4D3xzceuut/V/P\nnDmT3Nzcixa0xsaOq4z6lStt9nODZ7Axfwsfn9jGssiF3xifGO/Lgcxq0jKKSYryueZ8F2Krn0jY\nYm5bzAy2mdsWM4Nt5rbVzEKIG4OiKBRUtLA/s5KM7Fq6evqmXQ/zd2VSvB8T431t5vpjho56Thgy\nOW7IpKSlDOibrTDCPYyx+lHEecXQ0NVIeWslZa0VlLVVktVwlqyGs/3bcNQ6EukVgp+9X39x83PS\no1YNvCD2+XJmp7aTciaAyyhora2tPPvss7z55pt4eHh8Y+zRRx/ln//8J3Z2dhw+fJhFixYNWdir\nNSNoCjtLdrO7fD/zQmfgqHVtgfU+AAAgAElEQVQcMD4/OYQDmdWkHikf0oImhBBCCHG9aWzt5uCZ\navafqqK6oe+DeC83e+aPD2HaKH/8vKx/pmxFUahqr+GEIZMThtNUtFUBoFapifOMZox+FKP1owZc\nWzfIJYBEn4T++2297V8VttYKytsqyarN4wy5/evYqXUEuQQScu7QSKPZyMd5m8+Vs+9JORPAZRS0\nzz//nMbGRh599NH+ZZMmTSIuLo4FCxYwc+ZMVq9ejb29PQkJCRfde2Yp9ho75oXM5JPCrewu+5Il\nEfMGjIf5uxIb7M7pogaq6tsJ8LauGSmFEEIIIayJ0WTmRF4d+zOryCysR1H6zu2fGO/LjKRA4sM8\nrX7mRUVRKGut4Lghk5OG09R0GADQqjSM8h7BaH0iST4Jl31NLxedMyO8YhjhFfPVMg8dJ4pzv7an\nrYKS1jKKWkr613HQOJwrZ2GD+wKFzbpkQVu9ejWrV6++4PiaNWtYs2bNoIYaCjODp5BauocvyvYy\nJ2QaDv9xvYb540PILW8m9Wg59y6Ms1BKIYQQQgjrVVrTyv7MKg6dqaGtsxeAiABXpicGMDHBD2cH\nnYUTXpxZMVPUXMqJc6WsvqtvOnmdWscYfSJj9KMY5TPiG0dbXS1HncO52RAj+pf1mnqpbK+mvLUS\nQ2c9yX5jCHENHJTnE9cH25gyZxA4aB2YEzKDz4q2s7f8IAvD5wwYHxvrg5ebPV9mVrNyZiROVv4L\nRgghhBBiOLR19pKeVcP+U1WU1PSdB+vqpGPhhBCmJwUQrHexcMJvZ1bMNHQ1UtFWTWVbFRXt1RQ2\nFdHc0/caHDQOjPcbw1h9IgneccN2bS+dRkeYWwhhbjJ7uPh2N0xBA5gdMpVdZXvYVbaXWSHTsP/a\nf0SNWs3cccGk7C5g/6kqFk6UY4CFEEIIcWMymxXOFDew/1QVx/MMGE0KapWKMdE+TE8KICnKu/8i\n0tago7eDirZqKtqrqDxXyCrbq+k29QxYz0XnzJSACYw5N9GHTq7vJazQDfVT6ah1ZHbwdLYWp7Kv\n4iDzQ2cNGJ85OpDN+4tIPVrO/PEhVn/stBBCCCHEYFAUhYaWbkprWsmvaOZQVg2Nrd0ABPo4Mz0x\ngCmj/HF3tuzU+EazkZoOAxVtfUXsfCFr6m4esJ5apcbfyZdAF3+CnAP6bl0C8LB3t5rr9QpxITdU\nQQOYEzKdtLJ9pJbuYWbQlAG7s10cdUwe6c/ek5WcLKhjbIzegkmFEEIIIQaf2axQ09hBSXUrJTWt\nlFa3UlLT1n9OGYCjvZbZY4OYnhhARICrRUpNl7Gbsta+STUM+bUU1pdR3VGLWTEPWM/D3p0ErziC\nXL4qYn5OerSyd0zYqBvuJ9dZ58Ss4GlsL/mCA5UZzAmZPmB8fnIwe09WknqkXAqaEEIIIWya2axQ\n1dBxroS1UlLdSpmhjY4u44D19B4OjAj1IMzflVA/V+JCPLDTaYYtp8lsorK9muKWMkrO/atqrxlw\nUWg7jR2hrsEEufgT6BzQd+sSgLPO+qfxF+JK3HAFDWBuyAzSyvezsySN6YGT0Gm+mhAk2NeFEaEe\nZJc0Um5os9oTX4UQQgghvs5oMlNhaO8rYjWtlNa0UlbTRo/xqz1OKiDI14WkSGdC/VzPFTKXYZ19\nUVEUDJ11lLSUU9JSRnFLGeVtFfSavyqNdmodke7hhJ+bTGNMWCyqTvtvXORZiOvRDVnQXOycmRnU\nN+3+gaoMZgdPGzC+YHwIOaVN7DpazprFIyyUUgghhBDi0tq7evn0QDFpxyvo/VoZ06hVBPo4E3au\niIX5uRLs60xIkCcGQ+uw5WvubqWkpZSS1vL+vWMdxs7+cbVKTZCzP6FuIf2FzN/JF436qz14eldX\nDF3Dl1kIS7ohCxrA/NBZ7Ks4yNaiVCb5jxtwvYvR0T74uDtw8HQ1K2dF4eIoU+4LIYQQwroYTWZ2\nH6/gk/1FtHcZ8XZzIDHSi9DzZUzvjE47fIcpfl2PqYcN+Vs4XZdNY3fTgDG9ozcJ3nGEu4US5hZM\nsEvgsE1xL4QtuGELmqudCwvD5vJp4Ta2F6dxa/TS/jG1WsW85GDWfZHPvpOVLJksV3YXQgghhHVQ\nFIWTBfV89EU+1Q0dONpruGNONPOSg9FpLX8IYFtPOy+feoOillJcdS4k+sQT5hpKuFsIoW7Bcs6Y\nEJdwwxY06DsXbX/FIdLK9jE9aBI+jt79YzOSAti4r5AvjpWzcGIIGrXlf+EJIYQQ4sZWVtvGui/y\nyCpuRKWCOWODWD4jAjcn69gDVdfZwEsnX6W2o44JfmO5J/52mU1RiCt0Q7cOO42OW6OWYFRMbCrY\nOmDMyUHHtFEB1Ld0czy3zkIJhRBCCCGgub2HN7fm8Js3MsgqbmRUhBe/XTuRexfFWU05K20t57mj\nL1LbUceC0Nncl7BaypkQV+GG/1+T7DeGtPIDHK89RX5TEdEeEf1j85KDSTteQerRcsaP8LVgSiGE\nEELciHqNJnYcLuOzgyV095gI9HFm9dxoEiO9L/3gYZRVf5ZXT79Dj6mX22OXf2MCNiHE5buh96AB\nqFQqVsbcDMCGvM8GXPww0MeZkRFe5JY1UVojMwcJIYQQYngoikJGdg2//Hc66/cUotOouWdhLE+t\nnWB15exg1RH+eeoNzIqZB0bdI+VMiGt0w+9BA4h0DyPZdzRHa09ypOYEE/3H9Y/NTw7mTFEDqUfK\nWXtTvAVTCiGEsGabN2/m1VdfRavV8sgjjxAXF8fPfvYzTCYTer2eP/3pT9jZWcehaMK6FVa28OGu\nPPIrmtGoVSyeGMqyqWE4DeO1yi6HoihsL/mCTwu346R15KGk+4nyCLd0LCFsnhS0c5ZHLeFk3Rk+\nKdjKGP2o/uleE6O88fV05FBWDavmRFnNcd5CCCGsR2NjIy+99BLr16+no6ODv//972zfvp27776b\nJUuW8Oc//5mUlBTuvvtuS0cVVqalp5X8piL0jt5oja58uq+MQ1k1ACTH6bl9dhS+ntY366HJbOKj\n3E3sr0zHy8GTH41ei7+zn6VjCXFduOEPcTzP29GLuSEzaOpuZlfp3v7lalXflPtGk5k9JyotmFAI\nIYS1OnjwIFOmTMHFxQVfX1+efvpp0tPTmTdvHgBz5szh4MGDFk4prE1HbycvHPsXr51+lz8cfoGn\njz3Dcd1HuI86wdSFDYyd1Emb2kBHb+elNzaMekw9vHL6HfZXphPsEsj/JP9IypkQg0j2oH3NwrA5\nHKw8zI6SNKYETsDD3h2A6YkBbNxbSNqxcpZMCkWrkV4rhBDiK+Xl5XR1dfHQQw/R0tLCj3/8Yzo7\nO/sPafT29sZgMFg4pbAmJrOJ106/S3VHLermQHq6tOhcOrB37aDLXM3xpmqON2X0r+9m54q/sx/+\nTr74O/ueu/XDzc4FlUo1bLm/fo2zEZ4xPJB4L45ah2F7fiFuBFLQvsZR68CyyIV8cHYDnxZu5974\nO/qW22uZnhhA6tFyjuUamBgvnxIJIYQYqKmpiRdffJHKykruu+8+FEXpH/v61xfj6emEVqu55ix6\nves1b8MSbDH31WRWFIV/HHqXnMY8TI16TMVjWDUnhttmR+Ngr6Wtp52KlmrKm6uoaKmmorXv69zG\nfHIb8wdsy1nnSJBbAMFu/oS4BzI2YCSBbv5DkrumzcBfM/5JVVstM8Im8sMJ96LVDN9bSVv8+QDb\nzG2LmcF2c/8nKWj/YWrgRPaUf0l61VFmB08jxDUI6Jtyf9fRcnYeKZOCJoQQYgBvb2/Gjh2LVqsl\nNDQUZ2dnNBoNXV1dODg4UFNTg6/vpS/X0tjYcc1Z9HpXDAbbm3nYFnNfbeZ3jm7lUPOXmNtdCema\nwYNrk/D1dKK1pZPzW/PCFy83X5LcRvc/rtvUQ01HLdXt5/511FLdXkN+QzG59YUAvHUiBX9nP8bq\nRzFGn0iQS8A39rBdTe7SlnL+cfJ1WnvbWBg2h1siF9PYMHyHXtrizwfYZm5bzAy2l/tiZVIK2n9Q\nq9TcFrOMF0+8yvq8T/nJ2B+gUqnw83IiMcqbUwX1FFW1EBHgZumoQgghrMT06dP5+c9/zoMPPkhz\nczMdHR1Mnz6d7du3s3z5cnbs2MGMGTMsHVNYWEdXL//84gsK7NOg156FPrex/KYE1OrLO0TRXmNH\nqGswoa7BA5YbzUYMnfUUt5Rx0nCa7IZcthbvYmvxLnwcvRlzrqyFuQWjVl35aRpnzl3jrNfUy+rY\nW5kZPPWKtyGEuHxS0L5FvFcso7xHcLo+h1N1WYzWjwRg/vhgThXUk3qknAdvTrBwSiGEENbCz8+P\nRYsWcccdfYfG/+pXvyIxMZEnnniCdevWERgYyK233mrhlMKSThfV89qudLrD9qFSNNyfcB/jQ2MG\nZdtatZYAZz8CnP2YEjCeLmM3Z+pzOGHI5HR9Dqmle0gt3YOHvTuj9aOYo0zEG7/LKmsHKw/z/tn1\naFRqHki8lzH6UYOSWQhxYVLQLmBF9E1kNeSyMf8zRnrHoVVrGRnuRYC3ExnZNdwxJwp3F3tLxxRC\nCGEl7rzzTu68884By9544w0LpRHWoqvHyEdpBew+XYDDyEOoNCbWJtxDsv/glLNv46C1J9lvNMl+\no+kx9ZLTkMsJw2lO1WWxp/wAe8oP4KJzZrR+JGP0icR6RqFVD3xLqCgK24p38VnRDpy1Tjw0+rtE\nuocPWWYhxFekoF2Av7MfM4Ims6f8S/ZWHGRuyAxUKhXzk4N5Z0cuu09Usnx6hKVjCiGEEMJKnS1t\n5PXPszG0tOOSeBKTXRfLo5aQ7J80bBnsNDqS9CNJ0o/EaDaS11hITmsO6WUnOFCZwYHKDBy1jiT6\nxDNGn0i8VywalZp1uRs5UJlx7hpn38Pf+dLnUAohBocUtItYGr6AjOpjfF6UykT/cbjonJkyyp+U\nPYWkHa9g6eQwdFqZcl8IIYQQX+npNbFhbyE7D5eBSiF0Qj4GGpkSMIEFobMtlkur1hLvHcvMEcks\nD1tGQVMxJwyZnDCcJqP6GBnVx7DT2OHj4EVlezXBLoH81+i1uNvLefdCDCdpFxfhYufMkvD5dBo7\n2VqUCoCDnZYZSQG0tPdwJKfWwgmFEEIIYU2Kqlp46s3D7Dhchq+nIzMWtWKgiBiPSO6MWzGs1yy7\nGLVKTYxnJLfHLufpqb/g8fEPsyB0Nm52rlS2VzPCM4ZHxz0k5UwIC5A9aJcwM3gqeysOsrfiIDOD\npuDn7Mu85GB2Hi5j55EyJo/0s5pftkIIIYSwDKPJzOYDxXx+sASzojA/OZiQEY18mPclvo4+PJh4\n3zfO87IWapWacLdQwt1CWR61hPquRrwcPK5qxkchxLWT/3mXoFNrWRF9E2bFzMaCLQDoPRwZE+ND\ncXUrBZUtFk4ohBBCCEsqr23jmbeO8NmXxXi62vP4nWOYMF7Dx/mbcNI68sPR9+Osc7J0zMuiUqnw\ncfSSciaEBcn/vssw2mckMR6RZNZlk9OQB8D85L5rkKQeKbNkNCGEEEJYiMlkZsvBYp568zCltW3M\nHB3Ab783ES9fE//OfBuABxPvw9dJb9mgQgibIgXtMqhUKm6LWYYKFRvyP8OsmBkR5kmQ3pmjZw00\ntnZbOqIQQgghhlF1QwdPvLSf9XsKcXHS8ejtSXx3STxmdQ8vn3yDDmMnd8XdRqxnlKWjCiFsjBS0\nyxTqGsxE/3FUtFVxsOpw/5T7JrNC2vFyS8cTQgghxDAwKwo7Dpfx5OsZnC1pZHKCH09/bxJJUT4Y\nzUZeyXyb2s46FobNYUrgBEvHFULYICloV+CWqMXYqXV8WridLmMXk0f64+ygZffxSnqNJkvHE0II\nIcQQqmns4Nn3jvHhrjzsdRp+ft8Evn/LSFwcdSiKwgdnN5DXVMgYfSI3Ry6ydFwhhI2SgnYFPOzd\nmR82m9aeNnaU7MZep2HmmEDaOntJz5Ip94UQQojrkVlR2HW0nCdfzyC3vJnkWD3PPDCJaaMD+9fZ\nWbqbQ1VHCHUNZk3CaplkQwhx1eS3xxWaHzoLD3t3dpXtpb6zkbljg1GrVGzPKMVsViwdTwghhBCD\nqK6pk+c+OM57O3PRadT84JaR/NeKUbg52/Wvc6I2k08KtuJh785DSd/FTmN3kS0KIcTFSUG7QvYa\nO26JXIzRbGRz4Va83R2YMsqPirp20rNqLB1PCCGEEINAURR2H6/gf1/PIKe0iTHRPjzzwCQmJQy8\n/mlJSxlvZn2IncaOHybdLxd2FkJcMyloV2GC/1hCXYM4UnOCouYSlk+LQKNWsWl/IUaT2dLxhBBC\nCHEN6pu7+PO6E7y9/SwalYoHlsXz45WJuLvYD1yvo5F/nXoTo9nI2pF3E+waeIEtCiHE5ZOCdhXU\nKjUrY24BYH3eZ3i7OzB7bBCGpi72naqycDohhBBCXA1FUdh3spJfv57OmeJGEiO9efqBSUwdFTBg\nrxlAl7GbP+z7B809rdwWs4xEnwQLpRZCXG+koF2laI8IxugTKWop4VjtSZZNCcNOp2bzgSJ6emVG\nRyGEEMKWNLZ289ePT/HG1hwUBe5fMoJHb0/C09X+G+sqisLbWR9S0lTO9KDJzAmeboHEQojrlRS0\na3Br1FK0Kg2bCrbi5KhmwfgQmtt62HVMrosmhBBC2AJFUTiQWcWvXk0ns7CekeGePP29ScwYHfiN\nvWbn7S4/wMm6M4z0jeWOmOUXXE8IIa6GFLRroHfyZlbINBq6GtlVto/Fk0Jxstfy+cESOrqMlo4n\nhBBCiItobuvm7+szeW1LNmazwn2L4vjp6jF4uztc8DGlLeVszN+Cq86Fn0xei0atGcbEQogbgRS0\na7Q4bB4uOme2FqfSbKxnyeRQ2ruM7DhcauloQgghhPgWiqJwKKuaX72azon8OkaEevDb701k9tig\ni+4N6zJ28fqZ9zApJtYk3ImHo/swphZC3CikoF0jJ50j3xmxCqPZyBtn3mfWWH/cnHRsP1xGS0eP\npeMJIYQQ4mta2nv4x8bT/HtzFr0mM99ZEMv/3DUWvYfjRR+nKAofnN2AobOeBaGzifeOHabEQogb\njRS0QZCkH8n0oMlUtlezrWwHy6aG091j4vODJZaOJoQQQohzymrbePL1DI7mGogJdueptROZlxyM\n+jLOITtUdYQjNSeIcAvj5shFw5BWCHGjkoI2SFZGL8PPyZe0sv34hrbj7ebAF8cqaGjpsnQ0IYQQ\n4oZXUNHMH987RnN7D6tmR/HE3ePw83S6rMdWt9fwUe4mHLWO3D/yLjnvTAgxpKSgDRI7jV3fL22V\nhg9yP2bRVD+MJjObDxRbOpoQQghxQ8subuC5D0/Q1WPigWXxLJ0chlp9eTMv9ph6ee30e/SYe7ln\nxCq8Hb2GOK0Q4kYnBW0QhbgGcUvUYlp6WslT7cXf25H9p6qoaeiwdDQhhBDihnQ8z8BfPj6FyWzm\nh7eOYuqogCt6/Pr8T6lsr2Zm0BTG+CYOUUohhPiKFLRBNjdkBrGe0ZyuzyZhXCtmRWHjvkJLxxJC\nCCFuOIfOVPPShtOo1fCTVaNJjtNf0eOP1Z5if8UhglwCuC162RClFEKIgaSgDTK1Ss2ahNU4aR05\n0rqHoGCFjOxaSmtaLR1NCCGEuGHsPl7BK59mYW+n4bHVYxgZcWWHJtZ1NvBedgp2ah1rR34HnUY3\nREmFEGIgKWhDwMPenbtHrKLX3Is6/DiozGzcK3vRhBBCiOGw9VAJb28/i4uTjp/dNZaYYI8rerzJ\nbOL1M+/RZepiddwK/J19hyipEEJ8kxS0ITLWN5GpAROo66nFN76UkwX1ZBc1WDqWEEIIcd1SFIX1\newr4eHcBnq72/Pw74wjzd73i7Wwu3EZJSxkT/ccxOWD8ECQVQogLk4I2hFbG3IKvow+tLjmo3ep5\ne2sWiqJYOpYQQghx3TErCu/vzGPLwRJ8PRz5xT3jCPB2vuLtnKnPIbV0D76OPqyOvXUIkgohxMVd\nVkF79tlnWb16NStXrmTHjh0Dxr788ktWrVrF6tWreemll4YkpK1y0Nrz3ZF3oVapcYo9zemSKs4U\ny140IYQQYjCZzGbe2JLNrmPlBOmd+fk94/Bxd7zi7TR1N/N21jq0Kg1rR92Dg9ZhCNIKIcTFXbKg\nHTp0iLy8PNatW8err77K7373uwHjzzzzDH//+9/54IMPOHDgAPn5+UMW1haFuYWwLGIhJnUnduFn\nSNlTIHvRhBBCiEHSazTz8qYzHDhdTUSAG0/cPQ4PF/sr3o5ZMfPWmQ9p621nRcwyQlwDhyCtEEJc\n2iUL2oQJE3jhhRcAcHNzo7OzE5PJBEBZWRnu7u4EBASgVquZNWsWBw8eHNrENmhB2GyiPSLQeNVQ\nYczhWK7B0pGEEEIIm9fdY+Jv609xNNfAiFAP/ufOMbg4Xt1si9uLvyC3qYDRPiOZFTR1kJMKIcTl\n015qBY1Gg5OTEwApKSnMnDkTjUYDgMFgwMvrq2lrvby8KCsru+j2PD2d0Go115IZAL3+yk/6taSf\nTn+An257GiUsm40ZQSyYuhyNWmXpWJfN1r7fYJuZwTZz22JmsM3ctphZiKHQ0WXkryknyS9vJinK\nm/+6dRR2uqt7f5HXWMiWop142nvwnfjbUals5++zEOL6c8mCdl5qaiopKSm8/vrr1/SEjY0d1/R4\n6HuDYjDY2nXFdDw04Tv89eBr1Lsf5JO0OGYkBVs61GWxxe+3LWYG28xti5nBNnPbamYhBltLRw9/\nXneC0po2Jsb78sCyBLSaq5v3rK23nTezPkClUnH/yLtx1jkNclohhLgyl/XbbN++fbz88su88sor\nuLp+9cfW19eXurq6/vs1NTX4+sq1Qi5kauh4xniPQe3SQkrONnqNZktHEkIIIWxKQ0sXf3zvGKU1\nbcwcHcj3bx551eVMURTeyfqIpu5mbopYSJRH+OCGFUKIq3DJ32itra08++yz/Otf/8LDY+CFHoOD\ng2lra6O8vByj0UhaWhrTpk0bsrDXg3tG3oaD4kqvdy7rj2ZYOo4QQghhM2obO/j9u8eoqu9g0cQQ\n1iyOQ30Npwukle/ndH02IzxjWBg2e/CCCiHENbjkIY6ff/45jY2NPProo/3LJk2aRFxcHAsWLOA3\nv/kNjz32GABLly4lIiJi6NJeBxy1Dnx31F388/Qr7GvcyqKOUXg6uVg6lhBCCGHVyg1tPL/uBM1t\nPdw6I4Kbp4Zf07lipS3lbMr/HFedC/cl3IlaJZeGFUJYh0sWtNWrV7N69eoLjk+YMIF169YNaqjr\nXaJfNFHZyRSqjvDS4ff4fzO/LyckCyGEEN9CURQO59TyzvaztHcZuWteDAsmhFzTNjuNXbx25j1M\niok1CXfibi/nSgohrId8XGQh35+0HNo9qTIVsK/ssKXjCCGEEFanvrmLF1JO8fInZ+gxmlm7NP6a\ny5miKHx4dgN1nfUsDJtDvHfsIKUVQojBcdmzOIrB5epoz1yfZexqe5+U/E+I94lC7+Rt6VhCCCGE\nxZnNCruOlrNhbyHdvSbiwp2JG9NKk10mW4vOoFapUavUaNSavttz99UqDZr++/85pkaj0lDcUsqR\nmhNEuIWxLGKhpV+qEEJ8gxQ0C1o2PoE9HyZhCjrOa6ff5/Hx/4VGfe3XiBNCCDG80tPT+clPfkJM\nTAwAsbGxPPDAA/zsZz/DZDKh1+v505/+hJ2dnYWTWr+y2jbe3JpDUVULTg4aFszUcqp7BzsrB+8S\nE45aR+4febf8zRVCWCUpaBZkr9OwYuQMPsyvpowythbvYlmkfJonhBC2aOLEifztb3/rv/+LX/yC\nu+++myVLlvDnP/+ZlJQU7r77bgsmtG7dvSbW7ylgW3opJrPC2JEuqIJOs78pB61ay00RC4jxiMSk\nmDErZkyK6dxt332zYsZk/s9lJkz9903nlimM9U3C29HT0i9ZCCG+lRQ0C5sxOpDPD4+l3bWJbcW7\niPeKleuwCCHEdSA9PZ2nnnoKgDlz5vD6669LQbuA7OIG3k1Np6quHS83e8ZOaeNIy2Z6mnqI8Yjk\nrhEr8XPSWzqmEEIMCyloFqbVqLlt2ghe212Hffxh3jjzPj9N/iFeDvLJnhBC2JL8/Hweeughmpub\nefjhh+ns7Ow/pNHb2xuDwXDJbXh6OqHVXvthd3r9tc1KmFWbx5elR5gaOp54ffSQzTTc2tHD65vP\nkHq4FLUK5kx3o8bpEF82luJi58wDyXcyK3yyVc90fK3fa0uxxdy2mBlsM7ctZgbbzf2fpKBZgUkJ\nfnx+KISa8iYaQ3J54di/eHTcQ3g6eFz6wUIIISwuPDychx9+mCVLllBWVsZ9992HyWTqH1cU5bK2\n09jYcc1Z9HpXDIZrO1/r7WPrKWwuYUfBXgKd/ZkZPJUJfmNx0Npfcz7o+35kZNfyQWouLR29BPs5\nMGJyHQerd2BuMjPRfxy3RS/D1c6Furq2QXnOoTAY32tLsMXctpgZbDO3LWYG28t9sTIp0+xbAbVa\nxYqZkRirIvHpSqKuq4EXjv9/9u48PMry3v/4e9Ykk33fE5IQICGEfTdsCgpuWOuGHGu11dat7bFF\nPadWezynpyr1p9W2KlXr1kpLrXKsCi6gguxrICELS/Z939f5/RGIUHZIMjPJ53VdXJnM88zMN+M4\nz3zmvu/v8xK1bXWOLk1ERM5BaGgoixYtwmAwEBMTQ1BQEHV1dbS2tgJQVlZGSEiIg6s8Nw3tjRyu\nyyfCM4yJIWMpbS7nnax3+fnX/8Oq7NWUNZ99JPBMKutaeG7VXl5avZ+W9i5mp5mxj1jPxpINBLj5\ncd+47/Gd5Jvxtnr1zR8kIuJiNILmJMYnBpEQ4cPBvXYmz3VnX9NWntv1Ej8e/wN83XwcXZ6IiJzB\n6tWrqaio4M4776SiooKqqiq+9a1vsWbNGq699lrWrl1LWlqao8s8J+mVmdixMzV8IpfFzKa2rY6N\nxVvZULSZdYUbWFe4gac8grkAACAASURBVKSAEcyOmsHowFEYDef2XW93t51PdxTyj6Ot8xPj3PFN\nzGFrbU/b/MVJlzM7JA2rSZ0uRWRoU0BzEgaDgTuvSuaXf9rG3o1BzJw/g00VX/PbXS/zowl342Md\nHHNqRUQGo3nz5vHTn/6Uzz77jI6ODh5//HGSkpJ46KGHWLlyJRERESxevNjRZZ6T9MoMAFKDkgHw\nc/Plyrj5XB47lz0V+/iicBOZ1dlkVmcT6O5PWuR0pkdMxsviedr7zC9r4PWPD3C4pAGbu4nZaR2k\nt35BYW0LcT4x3DLqesbFjXCp6UkiIv1FAc2JhAXY+O7CUbz4/n6ytoYx+5JL+KJoQ09IG3+3pnuI\niDgpLy8vXnzxxZOuf+211xxQzYVr7+ogszqbUFsIIf/SNdFsNDMxdBwTQ8dR1FjCF4Vfs610J+8d\n/JB/Hl7LxJBxzIqaTqxPdO9tOjq7eX/DYT7ekk+33c64FDfaQneztSEPd5MbN41YzCWR0855FE5E\nZChQQHMyU5JCySqoZd3OIoYdHM6c4XbWF27sDWle1tN/QykiInIxsmpy6Oju6B09O51Ir3CWjLqe\nxQmL2Fy6nS8Lv2Zz6XY2l25nmE8Ms6NmMC54DK99mM2WjDIC/SwkTa5id/0Wuhq6GBecwg0jrsXP\nzXeA/jIREdehgOaEbp6XyKGiejbuLeWOqMnMiuzmy6JNPL97BQ+MvwtPi83RJYqIyCDUO70x+MwB\n7RibxYN50WnMiZrJgeocvij8mv1VB3g94x3+wvs01YcTkRCGOSqbHXWV+Ln5ctOIxaQGj+7PP0NE\nxKVpToETspiN/PC6FDzczLy1NpsZ/pdxScRUChuLeX73Cpo7Lr4Ns4iIyPG67d2kV2biZfFkmE/M\ned3WaDCSHDiSH479Lo9Pf4ixPlNo6+zEEnGImsCvqWypYk7UTB6d+qDCmYjIWSigOakQPw/uWJRE\ne2c3L76/n2vjrmZG+GQKGop4YfcrtHS2OLpEEREZRPIbCqlvbyAlKOmi1oR1NLux56tgOvfO4/Lw\nq5gZMZWfTrqXG0Zci7vZvQ8rFhEZnDTF0YlNHBnMgsnRrN1WwFtrc7jzym/RbbezuXQ7v9v9CveO\n+x4eOtiJiEgfSK84sXvjhWhu7eC3f0+npa2L7181mulJYX1VnojIkKERNCf37TkJxEf4sHl/GRv2\nlnJr0reZHDqBw/X5/H7PK7R2tjq6RBERGQT2VmZgNpoZFTDigm7f3W3npdUZlFU3c8WUGKanKJyJ\niFwIBTQnZzYZ+eG1KXi6m3n7kxwKy5u4LflGJoWO41BdHr/f8xptXe2OLlNERFxYZUs1xU2ljPIf\njtsFnij6718eJP1QFSnxAXx7TkIfVygiMnQooLmAQF93vndVMp1d3fz+vX20tnVzW9JNjA9J5WDd\nYf6w51XaFdJEROQCHeveOOYCpzdu3l/KR5vzCfX34O5rRmM0GvqyPBGRIUUBzUWMHR7EommxlNe0\n8KePMjEajHw3+RbGBaeQU3uIF/f+ifauDkeXKSIiLuhYQEsJSjrv2x4pree1jw7gbjVx//WpeLpb\n+ro8EZEhRQHNhVw3K44RUb5sz6rg851FmIwmvjt6CalBo8mqyeXl9NfpUEgTEZHz0NzRQk7tIWK9\no8/7xNF1jW08//d0Oju7ueua0UQEefZTlSIiQ4cCmgsxGY3cfW0K3jYL73yWw+GSesxGM3em3EpK\nYBKZ1dm8vO8NOro7HV2qiIi4iIzqLLrt3ec9vbGjs5vf/WMfNQ1tfGt2POOGB/VThSIiQ4sCmovx\n93bjrqtH091t5/f/2EdTawdmo5nvjfk3kgNHklGVxR/T36RTIU1ERM7B3or9AKQGn3tAs9vtvP1J\nFrlFdUxJCmHRtNj+Kk9EZMhRQHNBo+MCuHrmMKrqW3nlg0zsdjsWo5m7Um5jlH8i+6oy+eO+t3Qy\naxEROaPO7k4yqrMIcPcnwvPc2+J/vrOIL/eUEBPqxXcXJWEwqCmIiEhfUUBzUdfMjCMp1p/duZWs\n2VoAgMVk4e7U2xnpP5z0ygye2Lyc7WW7sdvtDq5WREScUW7tYVo6W0kNSj7nkJWZV8NfPs3B22bh\n/m+l4mYx9XOVIiJDiwKaizIaDdx1zWh8Pa2sWn+QnMJaAKwmCz8cewdXxV1Oc2cLr+3/M8/vXkFZ\nU7mDKxYREWdzvu31K2pb+MN7+zAY4N7rxhDo696f5YmIDEkKaC7M19PKD64djR07L76/n4bmnnOh\nWYxmFsZdys+nPkhy4EiyanL51db/x/8dWqNW/CIiAvSsI0uvzMDD7E6iX/xZ929t7+T5v++lsaWD\nW+ePYES03wBUKSIy9CigubiRMf5clxZPTUMbKz7IoPu46YxBHoHck3oH3x9zG15WLz4+8hn/veU3\n7KvMdGDFIiLiDIqbSqlqrSE5YCQm45mnKXbb7bzyz0wKK5qYOz6SOeMjB6hKEZGhRwFtEFg0PZaU\n+AD2Harmw015J2wzGAyMC07h0ak/5bKY2dS01fKHva/x8t7XqW6tcVDFIiLiaMemN6aew/TGD74+\nwo6sCkZE+3HLZYn9XZqIyJCmgDYIGA0Gvn9VMv7ebvzjq0McyDs5eLmb3bhu+JU8MvnHJPjGsady\nP09sXs4neevp6u5yQNUiIuJIeyszMBqMJAeOPON+O7MreO+rwwT6uHHPdSmYTfroICLSn/QuO0h4\n26z88NoUjAYDL63eT11T+yn3i/AK4ycTfsC/Jd2I1WTlvYMf8qttz5JTc2iAKxYREUepa6snr76A\n4X7x2Cy20+5XWNHIig8ysJqN3H99Kj426wBWKSIyNCmgDSLDo3y5fnYCdU3tvLx6P51d3afcz2Aw\nMC18Er+Y9jMuiZxGWVM5z+56kTcyVtLQ3jjAVYuIyEA7thb5TNMbG1s6eP7ve2lr7+KOK5OICfUe\nqPJERIY0BbRB5vIp0YxPDCIzr4aX3j99SAPwtNi4ZeS3eHDivUR7RbCldAe/3Pw0XxZuott++tuJ\niIhr29vbXj/plNu7urv5w3v7qKht5aoZsUxJCh3I8kREhjQFtEHGYDBw19WjGRXjx47sCn7/j310\ndJ45bMX5xrBs8gPcMOJa7HY7K7P/wfLtvyO/vnCAqhYRkYHS1tVOVk0OEZ5hBHkEnnKfjzbnk5lX\nw7jhQSxOO3sLfhER6TsKaIOQm9XEj24YS/Iwf3bnVvLCu+l0dJ65EYjRYGRO1Ex+Me1nTAodR15D\nAU9tf56VWf+gqL50gCoXEZH+dqA6h47uzjOenDrjSDUAd16VhNFgGKjSREQEBbRBy81i4kffTmVM\nfCDph6r47aq9tHWcvVujr5s33x29hAfG3UWILYgvizbxk49+yS83P8W7uR+QW3tY0x9FRFxYeu/0\nxtMHtOKqZoJ83fF0twxUWSIicpTZ0QVI/7GYTdz3rTH84b197M6t5Lm/7eGBb6fibj37f/aRAcN5\nZMpP2Fm2hwMNWewuyeCz/C/5LP9LvCyepAQlkRo0mqSARKwmdfUSEXEF3fZu0isz8LF6E+sTdcp9\nGls6qG9qJzXh1NMfRUSkfymgDXIWs5F7rkvhpff3syO7gv/31z38+IaxeLid/T+9xWhmavhErkqd\nQ3FpNVk1ueyt3E96ZSabS7azuWQ7FqOZUQGJpAaNZkxQMt5WrwH4q0RE5EIcqc+nsaOJmRFTMBpO\nPYmmuLIJgIhAz4EsTUREjlJAGwLMJiN3XzuaP36QwdbMcp5ZuZuf3DgOm/u5/+e3mCykBCWREpRE\nt72bvPpC9lbuZ29lBumVmaRXZmLAQJxvDKlBo0kNSibUM6Qf/yoRETlfeyvOPr2xpKonoIUHnf78\naCIi0n8U0IYIs8nI969OxmQ0sGl/Gcvf2cWDN4+7oPUFRoORON8Y4nxjuDZhIeXNlUdH1jI4WHuE\nQ3V5vHfwQ0JsQUfD2mjifGNO+22tiIgMjPTKDCxGCyP9E0+7T3FlMwARQRpBExFxBAW0IcRkNHLn\nlcmYjEY2pJfw9J97Qpq37eLWkIXYgrgsZjaXxcymsb2JfVWZ7K3MILMqi0/zv+DT/C/wtngxNXwi\n86LT8HXz6aO/SEREzlV5cwWlzeWkBo3Gajr9l3PFx0bQAhTQREQcQQFtiDEaDdy+aBQmk4Evdhfz\n9F928dObx+Pj2TeNPrysnkwLn8S08Em0d3WQfXTd2p6K/Xya/wXrCzcyLXwS82Nmn/b8OyIi0vfS\nKzOBM09vhJ41aP7ebuc1DV5ERPqO3n2HIKPBwG2Xj8RsNPLZzkKe+ssufnbzOHy93Pr0cazHrVu7\nIfFaNpfu4NO89Wwo2szGoi1MDB3Lgti5RHqF9+njiojIydIrMzBgICVo1Gn3aWnrpKahjdHD/Aew\nMhEROZ4C2hBlMBhYMj8Rk8nA2m0F/PrPu1h2y3j8vfs2pB1jMVlIi5zGjPDJ7Crfy5q8dWwv2832\nst2MCUpiQew84n1j++WxRUSGusaOJg7WHWGYTww+Vu/T7ldS1bP+LFzrz0REHEYBbQgzGAzcNG84\nJqOBj7bk8+TbO/nZLeMJ9HXvt8c0GU1MChvPxNBx7KvKZG3eut4ukIl+8VweO49RAYkYDIZ+q0FE\nZKjJqMqi295N6jlMbwQ1CBERcSQFtCHOYDDw7TkJmExGPvj6CE/+eSfLbhlPkJ9Hvz/umKBkUgKT\nyK09zJq8z8mszian9hDR3pEsiJ3LuOAUdX4UEekDeyuPttcPPktAq9I50EREHE0BTTAYDHxrVjxm\nk4H3vjrMr4+GtBD//j8HjsFgINE/nkT/ePIbClmbt57d5em8su8tQm3BzI+Zw+Sw8ZiNeqmKiFyI\nju5OMquyCPIIJMx25vNTagRNRMTxNDwhva6ZGcf1s+Oprm/j12/v7D1Z6UCJ8Y7ieylLeXTqg0wP\nn0xlSzVvHfgbj296inUFG2jvah/QekREBoPcmkO0drWRGpR81unjJVVN+NgseHmc/zkyRUSkbyig\nyQmunD6Mm+YNp7axnaf+vIuiyoENaQChniEsTbqBX05/iLnRl9DU0cSqnNU8+vX/8tHhz6hrqx/w\nmkREXNWx6Y1nW3/W1tFFZW2rRs9ERBzsnOaNZWdnc88993D77bezdOnSE7bNmzePsLAwTCYTAMuX\nLyc0NLTvK5UBc/mUGMwmI29/ks1Tf97J//xwJl6Wgc/y/u5+fDvxGq6IvZT1hRtZX7iRDw6v4YPD\na/Bz8yXWO4oYnyhivHv+eVn1oUJEHKu1tZWrrrqKe+65h+nTp7Ns2TK6uroIDg7m6aefxmrtm3NO\nniu73U56ZQY2swfxvsPOuG9pVTN2IFzrz0REHOqsAa25uZknnniC6dOnn3afFStW4OmpN/TB5NKJ\nUZiMBt5Yk8Ujv9/Ij29IJSHC1yG1eFk9uSp+AZfFzOLrkm1k1+SSV1/Insr97Knc37tfoLt/T1jz\niSK1ewQ+XQHYLP3b7ERE5Hh/+MMf8PXtea/87W9/y5IlS1i4cCHPPPMMq1atYsmSJQNaT2FjMTVt\ntUwOnYDJaDrjvr0NQjSCJiLiUGcNaFarlRUrVrBixYqBqEecyJzxkVgtRl798ADL/7KbB76dSlKs\n405e6m52Z150GvOi0wCobasjv76Q/IZC8hoKya8vZFdFOrsq0nn/4EcAhHgEnTDKFu0dgbu5/04j\nICJD18GDB8nNzWXOnDkAbNmyhV/+8pcAzJ07l1dffXXAA1rv9MazdG+E4xqEBPZ/gygRETm9swY0\ns9mM2Xzm3R577DGKioqYOHEiDz74oM5hNYjMSAknJMiLp97czv/76x7uuS6FccODHF0WAH5uvvgF\n+5IaPBromcpT3VpLfkMhlV3lHCg9RF5DYe8JsQEMGAi1BRPjE8VwvzgmhIzFQ4FNRPrAk08+yaOP\nPsp7770HQEtLS++UxsDAQCoqKs56H/7+NszmM490nYvg4J6TUWfuOoDJaCJtxISzziiobuxpxDRm\nZCj+Po55XzxWtytxxZrBNet2xZrBNet2xZrBdev+Vxfdu/yBBx4gLS0NX19f7r33XtasWcMVV1xx\n2v37+uDjalyx7uBgbx69cxr/89pWfvduOg8umUja+EhHl3VKIfgwipieX1J7QltZUyWHqvM4WJ3H\noZp8DlXns7V0J1tLd/L3nP9jesxELou/hMTAOKf4csFVXyOuyBXrdsWah4L33nuPcePGER0dfcrt\ndrv9nO6npqb5omsJDvamoqKBmtZaDtcUkBQwgqbaTppoOOPtDhfV4elupqO1nYq2jouu43wdq9uV\nuGLN4Jp1u2LN4Jp1u2LN4Hp1n+l4ftEBbfHixb2XZ82aRXZ29hkDWl8efFyNK9cdHeDBv984ludW\n7eHpt7ZTXtXIrLERji7ttI5/rk24k+gxksTIkRAJ3fZuypsr2V2RztfFW1l/eBPrD28iwjOMGRFT\nmBo2AZvFMVN8XPE14oo1g2vW7ao1DwXr16+noKCA9evXU1paitVqxWaz0drairu7O2VlZYSEnPkc\nZH0tvTITgDFn6d4I0NHZTXlNC/ERPk7xRZWIyFB2UQGtoaGBH//4x/zhD3/AarWybds2Lr/88r6q\nTZzMiGg/lt0ygd+s3M2fPjpAa1snC6bEOLqs82Y0GAnzDOEKz0tZEDuXrJpcNhZvZW/FflblrOb9\ngx8yLjiVmRFTGO7nHKNqIuLcnn322d7Lzz//PJGRkezatYs1a9Zw7bXXsnbtWtLS0ga0pvSj68/G\nBCWddd+ymma67XYigrT+TETE0c4a0Pbt28eTTz5JUVERZrOZNWvWMG/ePKKiopg/fz6zZs3ipptu\nws3NjeTk5DOOnonriw3z5uFbJ7D8nV2883kuLe1dXDNzmMuGGKPBSFLACJICRtDQ3sjmku18XbyV\nbWU72Va2k1Bb8NFRtYl4W70cXa6IuJD777+fhx56iJUrVxIREXHCjJP+1trZSnZNLlFeEQS4n725\nU0lVz+yWCLXYFxFxuLMGtJSUFN58883Tbv/Od77Dd77znT4tSpxbRJAnjyydyNN/2cX7Gw7T0tbJ\nTfOGu2xIO8bb6sX82DlcFjObnNpDbCzewu6Kffwj95+sPvgx44JTmBExhRH+CRgNOse7iJza/fff\n33v5tddec0gNmdU5dNq7zml6IxzXwVEt9kVEHO6i16DJ0BTs58EjSyey/J1drN1WQGt7J7ddPgqj\n0bVDGoDBYGCEfwIj/BNo7GhiW+kuNhZvYUf5HnaU7yHII5CZ4VOYGj4JX7ehsb5GRFzLsemNqQpo\nIiIuRwFNLpi/txsP3zqBZ1bu4cs9JbS2d/G9q5IxmwbP6JKXxZO50ZcwJ2omh+vz2Fi0lR3le3j/\n0Ef83+E1jAlKZnr4JBJ8hzmssYiIyPG6urvYV5WJn5sv0d7n1nG3uKoJN6sJf2+3fq5ORETORgFN\nLoq3zcrPbhnPc6v2sDWznNb2Lu5ZnILVcvGnUnAmBoOBeN9hxPsO49sjrmZb6W42Fm9hT8U+9lTs\nAyDEFkSsdwyxPlEM84kmyisCi8ni4MpFZKjJrjpEU0czaZHTz2nqeVd3N6VVzcSEern8VHURkcFA\nAU0ums3dzL/fNI4X3k1n78Eqnv3bHu6/PhUPt8H58vIwezArajppkdPIbyhkT8V+8uoLyGso6G0u\nAj0NSCK9won1iWaYdzSxPtGEeYZo/ZqI9KvtRXuBc2uvD1BR20pXt10NQkREnMTg/AQtA87NYuKB\n61N5efV+dmRXsPyd3fzkxrF4eQzeESSDwUCsT0/wgp7zq1W0VJFXX8CR+gLy6gsobCymoKGIDWwG\nwM1kJcY7qvd2sd7RBLj76VtrEekz24v34mayMsI/4Zz21/ozERHnooAmfcZiNvKDxaN57cMDfL2v\nlKf+vJMHbxqHr9fQWNNgNBgJtQUTagtmStgEADq7OyluLO0NbHkNBeTWHian9lDv7bwtXsT6RBPj\nE0VcYwTGdit+br74ufngbnZ31J8jIi6otKmckoZyxgWPwWI8t0P8sYAWrhE0ERGnoIAmfcpkNHLH\nlUm4W018vrOIX7+9k5/ePJ5A36EZNMxGMzE+UcT4RAHTgZ7zE+U3FJ0w0ravKpN9VZlw+MTbu5vc\n8D0a1vzcfPE9+vP4332s3po2KSLA+XdvhJ4GIYBOUi0i4iQU0KTPGQ0Gbp0/Ag83M//clMf/vr2D\nn948nrAAHfwB3M3uvW38j6lra6CwsYhOSysFleXUtdVR21ZPbVsdde31lDWXn/b+jAYjPlbv48Kb\nL7HeUYwKSMTXzWcg/iQRcRKZ1dkYDAZGB44659uUVDZjMRsJ8vXox8pERORcKaBJvzAYDFw/OwEP\nNzOr1h/k12/t4P5vp5IQ4evo0pySr5s3vm6jCA72psKn4aTtHV0d1LXX94a22rY66nov11PXVkdR\nQzF59QUn3C7SK5xRAYkkB4wkwXeYukqKDHLjgscwKXoMXtZzm67YbbdTUtVEeIBtUJzHUkRkMFBA\nk361aFos7lYTb63N5ldv7uDSCVFcNyt+0HZ47C8Wk4Ugj0CCPAJPu4/dbqexo4nq1hpyag+RWZVN\nbt1hihpL+Cz/SyxGC4l+8SQFJJIUOJIwW4iak4gMMrOipvd80VNx8hc9p1JV10p7Z7cahIiIOBF9\nSpZ+N29CFOGBnryxJotPdxSyI7uCJZeNYOLIYEeXNqgYDAa8rV54W3uajlwWM5v2rg5yaw+RWZ3N\ngeocMqqzyKjOgtwP8HPzPTq6NoKRAYl4WfQBTWSo+aZBiKagi4g4CwU0GRBJsf781x2T+eemPD7c\nnMfv/pHOuOFB3Dp/xJBtIDIQrCYLyYEjSQ4cCUBtWx2Z1TlkVmVxoCaHzSXb2VyyHQMGor0jSQ4Y\nwaiAEcT7xmIynvlk453dnbR0ttLR0ExhXSUtnS20dLbQ3NlCS0crzccud7ZgMVpI8B3GcL84QmzB\nGrkTcRLfNAjRFzQiIs5CAU0GjMVsYnFaPFOTQ3nj4yx251aSmVfDdWlxXDopCpNRnQj7m5+bL9PD\nJzE9fBLd9m4KGorIrM7hQHU2B+uOkN9QyMd5n+NuciPRP4EAdz+aO44Fr9aj4avn9/bujvN67C2l\nO4Ce0woM94tjuF88if7xhHuGqguliIOUVDYDCmgiIs5EAU0GXHigJ8uWjGdjeikrP8/hnc9z2bS/\njNuuGElcuLoODhSjwdh7wuwrhs2jtbOVnNpDZFRlc6A6u7dd9zEGDNjMHnhYPPB188bD7IHN7IG/\ntw+GThM2sw2b2b13n2PbPcweNHU0cbDucM854GoOsasinV0V6QB4mD0Y7jeM4X7xDPeLI9or8qyj\ndyLSN4qrmjAZDQT7qYOjiIizUEAThzAYDFySGk7q8ED+9nkuG/eV8t9vbFcTEQdyN7szJiiZMUfP\nn1TVUkNrVyseZnc8zB64m9xOOTXxXBoS+Lp5E+EVRlrkdOx2O1Wt1eTUHia39hC5tYdJr8wkvTIT\nAKvJSrxPbG9gG+YTre6TIv3AbrdTXNlEWIANs0mj2CIizkKfgsWhfGxW7rwqmRkpYSc0Ebl1/ggm\njFATEUcK9PDvl/s1GAy9HSmnh08CetbG5dYcIufoKNuBmhwO1OQAYDaYiPWJIdE/nmjvSOx2Ox3d\nHXR2d9J+9GdHVwcd3Z10dB/3s+votn+9vruDzu4u3MwWDHYjFqMFs9GMxWjGYjJjMVpOvO6Ey2bM\nRss3l00WAt39CbOFKESKy6lpaKO1vUsNQkREnIwCmjiFpGEB/NedU/jnpjz+uSmPF95NZ3xiTxOR\nAB81ERns/Nx8mRQ2nklh4wFoaG/kYN2R3hG2Q3VHOFh3+KIe4/jAZTKYaO1so62zvTe4XQyjwUiI\nRxARXmFEeoUT4dnzM8Dd3+UaonTbu+no7sSIQaFzkCup0vozERFnpIAmTuP4JiKvf5zFrpxKMo7U\ncN2seC6dGKkmIkOIt9WLccEpjAtOAaCls4VDdXmUNJVhNhwbyTJjMVl6Q5fl2MjW0evMRjNWowWz\n0YLZaDqpEcnxUzPtdjtd9q5vRtq6OunsHXU7/eX2rnbKWyopbiyluLGE0uZydpbv7X0Md5Mb4Z5h\nRHqFEdEb3MKwWfpmxKKju5PmjhaaO5tp6mimuaOZls5W2rs76Ohqp727g7audjq6Omjvbj/ucgft\nXe09/7o7aD+6vb2rJ7ACWI0WHpu+DD83nVx+sDrWYl8BTUTEuSigidMJD/TkoSXj2ZBewl8/z+Wd\nz3LYtK+U7ywcybAwNREZijzMHowOHMXowFH9cv8GgwGzoSfUXWirBLvdTnVrLcVNJRQdDWxFTaXk\nNRRwuD7vhH393Hx7Rts8w3tH3dxN7t8Erc4WmjqaaelooamzJ3g1dbbQQRt1LY29Yex8O2me8Ddj\nwGqyYDVasZos+Fi9ey9bTBYC3PzwNGvq22DW22I/UAFNRMSZKKCJUzIYDKSlRjB2eBB//TyXr/eV\n8sTr27l0YhTXpamJiDgfg8FAoIc/gR7+vY1WoGeUq6ypnKLGEoqbSiluLKWosYSMqiwyqrLO+3E8\nzO7YzDbCPEN6OmdaPLBZbHgevexhdsfNaMVish4XwI5dtvRcPrquztWmX0rfKq5swmCA0AAFcRER\nZ6JPueLUfGxWvndVMjOPNRHZXsjm/WXMmxDJpROj8LZZHV2iyBlZjGaivCOI8o444frGjiZKGkt7\nRtuaSmjv6sTLcjRwHQ1bnhYbNrMNz6MhLCYsmOqj64ZELsaxDo4hfh5YzJo+LiLiTBTQxCUcayLy\n8ZZ81m4rYPXGI3y8JZ+0sRFcPjmaIJ3DR1yMl8WTRP8EEv0Tzvk2Oj+c9JWG5g6aWjsZEe3n6FJE\nRORfKKCJy7CYTVw9M475k6P5ak8Ja7bl89mOQtbtLGJKUggLp8USHeLl6DJFRJyeGoSIiDgvBTRx\nOe5WM/MnRzN3lifXrAAAH8FJREFUQiRbM8v4aEs+mzPK2JxRRkp8AIumxhIUpKAmInI6ahAiIuK8\nFNDEZZlNRmakhDN9dBh7D1bx0eY89h2qZt+hat7beJj5E6MYPyIYoxohiIic4NgIWniQGoSIiDgb\nBTRxeQaDgbHDgxg7PIjcojo+2pzHrpxKsvNrCQ2wsXBqDNNHh2khvIjIUcdOUh0eoBE0ERFno4Am\ng8rwSF/uvz6V1m7488eZbNpXyp8+OsA/vjrEgknRzB4Xic1dL3sRGdqKK5sI8nXHzarGMyIizkZD\nCjIoRYd6c8eiJJ764QyumBJDa3sXf1t/kJ/9YSN/W59LbWObo0sUEXGIxpYO6pra1SBERMRJaShB\nBjV/bzdunDecq2bEsm5XEZ9sL+Sjzfl8sq2AS8aEc80lcfh5uTm6TBGRAVOiBiEiIk5NAU2GBJu7\nhSunD2PB5Gg2ppfy8ZZ81u8u5uv9pVwxJYYrpsbgbtX/DiIy+PU2CAlUgxAREWekT6QypFjMJuaM\njyRtbDhf7S3hva8Os3rjEdbvLmZxWhxpqeGYjJr5KyKD17EGIZriKCLinPRJVIYkk9HInHGR/Pru\naVwzcxit7Z288XEWv3hlK7tzK7Hb7Y4uUUSkX3wzgqaAJiLijDSCJkOau9XM4rR45oyP5L2vDvPV\n3mJ+u2ovo2L8uGHucOLCfRxdoohInyquasLf200dbUVEnJRG0EQAPy83bl84iv+6YwqpCYEcyK/l\nide389Lq/VTWtji6PBGRPtHS1kl1fZvWn4mIODF9fSZynMhgL358w1gy82r46+e5bMkoY0dWOZdO\njOKqGcPwdLc4ukQRkQtWWn10/ZmmN4qIOC2NoImcQlKsP4/ePonvX52Mr6eVNVsLePjFTazZmk9H\nZ7ejyxMRuSDH1p+pQYiIiPNSQBM5DaPBwPTRYfzqrmncMDeBbjus/DyX/1yxmS0ZZWokIiIuRwFN\nRMT5KaCJnIXFbGLh1Fie/MF05k+KpqahjZdW7+e/39hOVn6No8sTETlnCmgiIs5Pa9BEzpGXh4Vb\nLkvk0klRvPvFQbZmlvPkn3cxbngQl0+JZkS0HwaDwdFliogDtLS08PDDD1NVVUVbWxv33HMPo0aN\nYtmyZXR1dREcHMzTTz+N1Wp1aJ3FVU142yx4eWg9rYiIs1JAEzlPIX4e/ODaFOZPruNvn+eyO7eS\n3bmVRAR5Mnd8JDNSwvBw0/9aIkPJunXrSElJ4fvf/z5FRUXccccdTJgwgSVLlrBw4UKeeeYZVq1a\nxZIlSxxWY3tHF5W1rYyI9nNYDSIicnaa4ihygRIifHno1gk8snQCU5NDKatu5u1Psvn3Fzbyxpos\nCssbHV2iiAyQRYsW8f3vfx+AkpISQkND2bJlC5deeikAc+fOZdOmTY4skdLqZuxoeqOIiLPT1/wi\nF8FgMJAY5UdilB83X5rIV3uKWb+7iPW7ev4lRvkyd0Ikk0aGYDbp+xCRwe7mm2+mtLSUF198ke9+\n97u9UxoDAwOpqKg46+39/W2YzaaLriM42Puk6/YX1AEwYljAKbc7A2et60xcsWZwzbpdsWZwzbpd\nsWZw3br/lQKaSB/x9bRy1YxhLJoWy56DlazbWcS+w9XkFNbxji2HtLERzBkXSaCvu6NLFZF+8s47\n75CZmcnPfvazEzq9nmvX15qa5ouuITjYm4qKhpOuzzpcCYC3m+mU2x3tdHU7M1esGVyzblesGVyz\nblesGVyv7jOFSQU0kT5mNBoYnxjM+MRgymqaWb+riA17S/jnpjw+3JzH2IQg5k2IJDkuAKOaiogM\nCvv27SMwMJDw8HCSkpLo6urC09OT1tZW3N3dKSsrIyQkxKE1llQePUm1pjiKiDg1zbkS6Ueh/jZu\nmpfIb+6dyR2LkhgW5s3u3Eqe+ese/uPlzazZmk9jS4ejyxSRi7R9+3ZeffVVACorK2lubmbGjBms\nWbMGgLVr15KWlubIEimuasLmZsbX07GdJEVE5Mw0giYyAKwWE5ekhnNJajiHS+pZt7OILZllrPw8\nl3e/PMTUpFDmTogkKMjL0aWKyAW4+eab+c///E+WLFlCa2srv/jFL0hJSeGhhx5i5cqVREREsHjx\nYofV19nVTVl1C/ERPjodiIiIk1NAExlgceE+xF3pw43zhrMxvYR1u4rYkF7ChvQSokO9mJgYzNTk\nUEIDbI4uVUTOkbu7O7/5zW9Ouv61115zQDUnK6tupttuJyJI7ysiIs5OAU3EQbw8LFw+JYb5k6PJ\nOFLNF7uL2XuwivfKDvPehsPEhnkzNSmUKUkhBPiosYiIXLjiqp71Z+GBWn8mIuLsFNBEHMxoMJAS\nF0hKXCCe3u58sukwWzLK2X+4mrzSBv66LpcRUb5MTQ5l4qgQfGxaPyIi56eksglQgxAREVdwTgEt\nOzube+65h9tvv52lS5eesO3rr7/mmWeewWQyMWvWLO69995+KVRkKLC5W5iREs6MlHAamtvZkVXB\nlowysgtqyS6s4+1Pckge5s/U5FDGJwZjc9d3LCJydsVVRwOaRtBERJzeWT/dNTc388QTTzB9+vRT\nbv/v//5vXnnlFUJDQ1m6dCmXX345w4cP7/NCRYYab5uVOeMjmTM+kpqGNrZllrEls4x9h6vZd7ga\nsymL1IRApiaHMjYhEKvl4k9uKyKDU3FlE25WEwE+bo4uRUREzuKsAc1qtbJixQpWrFhx0raCggJ8\nfX0JDw8HYPbs2WzatEkBTaSP+Xu7sWBKDAumxFBW08zWjDK2ZJazM7uCndkVuFlNjE8MYmpSKKPj\nAjCbdAYNEenR1d1NaXUz0SFe6uAoIuICzhrQzGYzZvOpd6uoqCAgIKD394CAAAoKCvquOhE5Sai/\njatnxnH1zDgKyxvZklnGlowyNu/v+efpbmbiyGAmJ4UyKsYPk1FhTWQoq6htpbPLrgYhIiIuYsAX\nsPj72zCbL34qVnCwdx9UM/BU98BxxZrh/OoODvZm/Ohw7rbbyc6v4ctdRWzYU8SXe0r4ck8JPp5W\nZqRGcMnYCFISgjAZ++fb86HwXDsLV6xZHEsNQkREXMtFBbSQkBAqKyt7fy8rKyMkJOSMt6mpab6Y\nhwR6PqBUVDRc9P0MNNU9cFyxZri4ugNsFhbPHMY102PJKaxl64Fydhwo5+NNR/h40xF8bBYmjgph\nyqgQEqP8MPZRWBuKz7WjuGrN4lhqECIi4louKqBFRUXR2NhIYWEhYWFhrFu3juXLl/dVbSJyAYxG\nAyNj/BkZ48+tl40gq6CWbQfK2ZFVzrqdRazbWYSvl5VJI0OYPCqE4VG+GLUuRWTQKu4dQdNJqkVE\nXMFZA9q+fft48sknKSoqwmw2s2bNGubNm0dUVBTz58/n8ccf58EHHwRg0aJFxMXF9XvRInJujEYD\nSbH+JMX6c+v8RA7k17LtaHORz3YU8tmOQvy8rEwaFcKUUaHER/oorIkMMsWVzZhNRoJ8PRxdioiI\nnIOzBrSUlBTefPPN026fPHkyK1eu7NOiRKTvmYxGRg8LYPSwAJYuGMGB/Bq2ZpazK7uCT7cX8un2\nQgJ83HpG1pJCiA/3Ucc3ERfXbbdTUt1EeKCtz6Y1i4hI/9JZbkWGILPJSEpcIClxgXRePpKMIzVs\nO1DGzuxK1m4rYO22AgJ93Jk4MpjUhEBGRPupdb+IC6qua6W9o1sNQkREXIgCmsgQZzYZSU0IJDUh\nkNsu7ybjSDVbM8vZnVvRG9bcrSZGxwUc3S8IX0+ro8sWkXPwTYMQrT8TEXEVCmgi0stiNjJ2eBBj\nhwfR0dlNdkEte3Ir2Xuwih1ZFezIqgAgLtyb1IQgUhMCCQz0cnDVInI6xZU9nZM1giYi4joU0ETk\nlCxmI6PjAhgdF8Atl9kprW5mT24Vew9WklNYx+GSBt7fcBh/bzdGxwUwNiGQ5GEBeLjpbUXEWRzr\n4KiTVIuIuA59khKRszIYDIQHehIe6MkVU2Nobu0k40g1e3Ir2Xekmg17S9iwtwST0cDIGD9SE4IY\nmxBIaICmVYk4UklVEyajgRB/dXAUEXEVCmgict5s7mYmjQph0qgQAgO92JZe3DsVMuNIDRlHanjn\nsxxC/T0YO7xnKmRilB8WsxqNiAwUu91OcVUToQE2NfkREXEhCmgiclGMRgPxET7ER/hw3ax4ahra\nSD9UxZ7cSjKO1PQ2GnGzmEge5s+Y+EDGxAcS6Ovu6NJFBrXaxnZa2roYPUwj2SIirkQBTUT6lL+3\nG7PGRjBrbERvo5G9B6vYe6iKXTmV7MqpBCAyyJMxCT1hLTHKV9/wi/SxY+vP1CBERMS1KKCJSL85\nodEIiZTXNJN+qJr0Q1Vk5tXw8ZZ8Pt6Sj7vVRPKwAMbEBzAmPpAAH42uiVysYy321SBEROTcrF//\nGXPmXHrW/Z577jfccMPNRERE9ksdCmgiMmBC/G1cOtHGpROjaO/oIuvo6Fr6oSp2ZlewM7unjX9U\ncM/oWmp8IAmRGl0TuRAlGkETETlnJSXFfPrpmnMKaD/60YP9WosCmog4hNVi6l2PBlBW3czeQz1h\n7UBeLYUV+Xy0OR8Pt2Ojaz37+nu7ObhyEddQXNmEwQBhAergKCJyNs888ySZmftJS5vMggULKSkp\n5tlnf8///u9/UVFRTktLC3fccRczZ6Zx33138e//vox16z6jqamR/Pw8iooKeeCBB5k+feZF16KA\nJiJOITTAxvwAG/MnRdPW0UVWfk3P2rV/OUl2iJ8HiVG+JEb7kRjlS1iADYPB4ODqRZyL3W6nqLKJ\nED8PLGaTo8sRETkvf/08l20Hys/rNiaTga4u+2m3Tx4Vwo3zhp92+y23/BvvvvtX4uISyM8/wu9/\n/0dqaqqZMmUaCxdeRVFRIY8++jAzZ6adcLvy8jKWL/8tmzd/zfvv/10BTUQGJzeLidSEIFITgrDb\ne06SnX6omowj1eQU1rFxXykb95UC4OVhITHKlxHRfiRG+RET6qUpkTLkNTR30NTaSWKUn6NLERFx\nOUlJowHw9vYhM3M/q1e/i8FgpL6+7qR9U1PHARASEkJjY2OfPL4Cmog4teNPkr1gcjTddjvFFU3k\nFNaSU1hHdmHtCd0hrWYj8RE+JEb5MSLaj/gIHzzc9FYnQ0tJldafiYjrunHe8DOOdp1KcLA3FRUN\nffL4FosFgE8++Zj6+np+97s/Ul9fz/e+928n7WsyfTNLwW4//Qje+dCnFhFxKUaDgagQL6JCvJg7\nIQqAqrrW3sCWU1hLVn4tB/JrATAYICbEu3eUbXiUL35eWscmg9s3LfZ1DjQRkXNhNBrp6uo64bra\n2lrCwyMwGo188cXndHR0DEgtCmgi4vICfd0J9A1j2ugwAJpaO8gtrOsNbIdL6skra+DTHYVAzzq2\nsSOCGRbixcgYP7X1l0GnuLIZ0AiaiMi5io2NIyvrAOHhEfj59UwPnzNnHg8//O9kZOzjyiuvISQk\nhNdeW9HvtSigicig4+luYezwIMYODwKgo7OLwyUNx42y1fHJ1vze/UP8PRgV48eoGH9GxvirU6S4\nvN5zoAUooImInAt/f3/effefJ1wXHh7B66+/0/v7ggULAfjud78PQHz8N9Mw4+OH88ILL/dJLQpo\nIjLoWcwmRkT3rEkD6O6209jRzaY9RWTl15JVUMuXe0r4ck8J0NNRclSMHyOPhjZNiRRXU1zVRKCP\nO25WdXAUEXE1CmgiMuQYjQYSovzwcTNx+ZQYurvt5Jc3cCCvlgP5NWQX1PLF7mK+2F0MQNjRwDYq\n1p+R0X74KrCJE2tsbqeusb33HIMiIuJaFNBEZMgzGg0MC/NhWJgPV0yNoau7m/yyRg7k13Agr5bs\nwlrW7y5m/dHAFh5oY1SMP6Ni/RkR7Yevp9XBf4HINwrKeto8q0GIiIhrUkATEfkXJqORuHAf4sJ9\nWDg1lq7ubvJKjwa2/BpyCupYt6uIdbuKgJ7ANjLajxExfoyM1ho2caz8sp420xGBWn8mIuKKFNBE\nRM7CZOw5t1p8hA+LpsXS2dVNXmkDB/JryMqvJaeo7oQRthB/D0ZG96xhGxntT6CvukTKwCk4FtDU\nwVFExCUpoImInCezyUhCpC8Jkb5cOZ3eEbasgqOBrbCWr/aW8NXenqYjQb7u34ywxfgT7OuOwWBw\n8F8hg1VBeU9AC9cImoiIS1JAExG5SMePsC2cGkt3t52C8kay8mvIKqglu6CWjftK2bivFAB/b7ej\no2s9gS3U30OBTfpMQVkDfl5WbO46xIuI9LVvf/tq3nhjJTZb/63z1bu3iEgfMxoNxIZ5ExvmzYIp\nMXTb7RRVNJ0Q2DbvL2Pz/jIAfD2txIZ5E+jjTqCvO0G+7r2XfTytGBXe5By1tHVSUdNC8jB/R5ci\nIiIXSAFNRKSfGQ0GokO8iA7x4rJJ0djtdoqrmsk+GtiyCmrZe7DqlLc1mwwE+PQEtqhQbzzdTAT6\nfBPi/H3cMBmNA/wXibMqrW4G1CBEROR83XHHrfzqV78hLCyM0tISHnnkQYKDQ2hpaaG1tZWf/ORn\nJCenDEgtCmgiIgPMYDAQGeRJZJAncydEAdDc2klVfSuVdS1U1bVSVd96ws/MvBoy82pOcV89UyaP\njbgFHg1zPaHOjQAfdzzc9FY/VBRXNgEQrgYhIuLC3s39gF3l6ed1G5PRQFe3/bTbx4eM4VvDrzrt\n9lmz5rJx45dcf/2NfPXVF8yaNZeEhERmzZrDjh3bePvt1/mf/3n6vGq6UDpqi4g4AZu7GZt7zyjb\nqbR1dGE3mcg9UkXlv4S3qvpWcovqyCmsO/V9u5kJ8HEnwMftaHhzOy7EuePnbdUoXB946qmn2LFj\nB52dndx9992MGTOGZcuW0dXVRXBwME8//TRWa/+eM6+4qiegRQTqHGgiIudj1qy5vPDCs1x//Y1s\n2PAF9933E955503+8pc36ejowN194DoyK6CJiLgAN4uJ4GBv3E+Tozq7uqlpaKOqrpXqhlaq6tuo\nqe/5WV3fSkVdC4UVjae8rcEAfl4nhreoEC+mJodq/ds52rx5Mzk5OaxcuZKamhquu+46pk+fzpIl\nS1i4cCHPPPMMq1atYsmSJf1aR0nl0SmOGkETERf2reFXnXG061SCg72pqGi44MeMj0+gqqqCsrJS\nGhoa+Oqr9QQFhfDoo09w4EAGL7zw7AXf9/lSQBMRGQTMJiPBfh4E+3mccrvdbqelrZPq+jaq6lup\nPi689fzexqHienKLvpkekhDpS8hp7k9ONHnyZFJTUwHw8fGhpaWFLVu28Mtf/hKAuXPn8uqrr/Z7\nQGtq7SDY3wNvW/+O1ImIDEbTp1/Cyy//nrS02dTW1pCQkAjAF1+so7Ozc8DqUEATERkCDAYDNncL\nNncLUaeZRtndbae2sY3q+jbs2BXOzoPJZOptubxq1SpmzZrFhg0beqc0BgYGUlFRcdb78fe3YTab\nLriO//juVDq7ugl20SYhwcHeji7hvLlizeCadbtizeCadbtizXDxdV977ZXcfPPNrF69mubmZh56\n6CE2blzPrbfeyrp1n/Dll2sxmYwEBXnh6dl/77MKaCIiAvScHiDg6No0uTCffvopq1at4tVXX2XB\nggW919vtp1+4fryamuaLriHsIqf5OMrFTk9yBFesGVyzblesGVyzblesGfqm7vDwOL74YgsAvr7w\nxht/7d32zjvvATBr1gKam7tpbr64xzpTmNSqcBERkT7w1Vdf8eKLL7JixQq8vb2x2Wy0trYCUFZW\nRkhIiIMrFBERV6CAJiIicpEaGhp46qmneOmll/Dz8wNgxowZrFmzBoC1a9eSlpbmyBJFRMRFaIqj\niIjIRfrwww+pqanhxz/+ce91v/71r/n5z3/OypUriYiIYPHixQ6sUEREXIUCmoiIyEW66aabuOmm\nm066/rXXXnNANSIi4so0xVFERERERMRJKKCJiIiIiIg4CQU0ERERERERJ6GAJiIiIiIi4iQU0ERE\nRERERJyEApqIiIiIiIiTMNjtdrujixARERERERGNoImIiIiIiDgNBTQREREREREnoYAmIiIiIiLi\nJBTQREREREREnIQCmoiIiIiIiJNQQBMREREREXESZkcXcDa/+tWv2LNnDwaDgf/4j/8gNTW1d9vX\nX3/NM888g8lkYtasWdx7770OrPRETz31FDt27KCzs5O7776bBQsW9G6bN28eYWFhmEwmAJYvX05o\naKijSgVgy5Yt/OhHPyIxMRGAESNG8Oijj/Zud9bn+m9/+xurV6/u/X3fvn3s2rWr9/fRo0czYcKE\n3t//9Kc/9T7vAy07O5t77rmH22+/naVLl1JSUsKyZcvo6uoiODiYp59+GqvVesJtzvT6d2Tdjzzy\nCJ2dnZjNZp5++mmCg4N79z/ba8kRNT/88MPs378fPz8/AO68807mzJlzwm2c8bl+4IEHqKmpAaC2\ntpZx48bxxBNP9O7/7rvv8txzzxETEwPAjBkz+OEPfzjgdYtj6Pg4MHR8HBg6RjquZh0jnZDdiW3Z\nssV+11132e12uz03N9d+4403nrB94cKF9uLiYntXV5f9lltusefk5DiizJNs2rTJ/r3vfc9ut9vt\n1dXV9tmzZ5+wfe7cufbGxkYHVHZ6mzdvtt9///2n3e6sz/XxtmzZYn/88cdPuG7KlCkOquZETU1N\n9qVLl9p//vOf299880273W63P/zww/YPP/zQbrfb7b/5zW/sb7/99gm3OdvrfyCcqu5ly5bZ//nP\nf9rtdrv9rbfesj/55JMn3OZsr6X+dqqaH3roIfvnn39+2ts463N9vIcffti+Z8+eE677+9//bv/1\nr389UCWKE9HxceDo+Nj/dIwcODpGugannuK4adMmLrvsMgASEhKoq6ujsbERgIKCAnx9fQkPD8do\nNDJ79mw2bdrkyHJ7TZ48meeeew4AHx8fWlpa6OrqcnBVF86Zn+vj/e53v+Oee+5xdBmnZLVaWbFi\nBSEhIb3XbdmyhUsvvRSAuXPnnvScnun1P1BOVfdjjz3G5ZdfDoC/vz+1tbUDWtPZnKrms3HW5/qY\nQ4cO0dDQ4JBvLMU56fjoHJz5uT6eMx8fQcfIgaRjpGtw6oBWWVmJv79/7+8BAQFUVFQAUFFRQUBA\nwCm3OZrJZMJmswGwatUqZs2addK0gccee4xbbrmF5cuXY7fbHVHmSXJzc/nBD37ALbfcwsaNG3uv\nd+bn+pi9e/cSHh5+wjQCgPb2dh588EFuvvlmXnvtNQdVB2azGXd39xOua2lp6Z2uERgYeNJzeqbX\n/0A5Vd02mw2TyURXVxd//vOfufrqq0+63eleSwPhVDUDvPXWW9x222385Cc/obq6+oRtzvpcH/PG\nG2+wdOnSU27bunUrd955J9/5znfIyMjozxLFiej4OLB0fOxfOkYOHB0jXYPTr0E7nrO8UZ+rTz/9\nlFWrVvHqq6+ecP0DDzxAWloavr6+3HvvvaxZs4YrrrjCQVX2GDZsGPfddx8LFy6koKCA2267jbVr\n154039tZrVq1iuuuu+6k65ctW8Y111yDwWBg6dKlTJo0iTFjxjigwjM7l9e2M73+u7q6WLZsGdOm\nTWP69OknbHPG19K1116Ln58fSUlJvPzyy7zwwgv84he/OO3+zvRct7e3s2PHDh5//PGTto0dO5aA\ngADmzJnDrl27eOihh/i///u/gS9SHM6ZXrPnQsfHgePqx0fQMbK/6RjpfJx6BC0kJITKysre38vL\ny3u/AfrXbWVlZec1XNvfvvrqK1588UVWrFiBt7f3CdsWL15MYGAgZrOZWbNmkZ2d7aAqvxEaGsqi\nRYswGAzExMQQFBREWVkZ4PzPNfRMhRg/fvxJ199yyy14enpis9mYNm2aUzzXx9hsNlpbW4FTP6dn\nev072iOPPEJsbCz33XffSdvO9FpylOnTp5OUlAT0NCH419eBMz/X27ZtO+20jYSEhN6F3OPHj6e6\nutqlp4vJudPxceDo+OgYOkYOHB0jnY9TB7SZM2eyZs0aAPbv309ISAheXl4AREVF0djYSGFhIZ2d\nnaxbt46ZM2c6stxeDQ0NPPXUU7z00ku9HXGO33bnnXfS3t4O9LywjnXycaTVq1fzyiuvAD1TNqqq\nqno7Zznzcw09b9yenp4nfft06NAhHnzwQex2O52dnezcudMpnutjZsyY0fv6Xrt2LWlpaSdsP9Pr\n35FWr16NxWLhgQceOO32072WHOX++++noKAA6Pmw8q+vA2d9rgHS09MZNWrUKbetWLGCDz74AOjp\nbhUQEODQLmwycHR8HDg6PjqGjpEDR8dI52OwO9M45SksX76c7du3YzAYeOyxx8jIyMDb25v58+ez\nbds2li9fDsCCBQu48847HVxtj5UrV/L8888TFxfXe93UqVMZOXIk8+fP5/XXX+e9997Dzc2N5ORk\nHn30UQwGgwMrhsbGRn76059SX19PR0cH9913H1VVVU7/XENP6+Bnn32WP/7xj/z/du4XVbkgDgPw\ni1zxTzUIVlcgYhaMrsENGA2C0XYQbScIZ0UuwC0IFhcgGL5muXxVB+7z5Akvh2F+54VhkqRpmsxm\ns0wmkxyPx1wul7RarSwWi689r3q9XnM4HHK73fLz85PhcJjT6ZTdbpfn85nRaJSqqtJut7PZbFJV\nVbrd7q/9/79D6JO5H49HOp3O+3Aej8fZ7/fv3K/X69dems/nX828Wq3SNE16vV76/X6qqspgMCj+\nW9d1nbquM51Os1wu32vX63XO53Pu93u22+37J+tbTx/zHebjZ5iPn8lpRn4vsxlZnuILGgAAwF9R\n9BVHAACAv0RBAwAAKISCBgAAUAgFDQAAoBAKGgAAQCEUNAAAgEIoaAAAAIVQ0AAAAArxDwNoFZVy\n0cUJAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f1e1eaabeb8>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "BWGzMSaBnYMb",
        "colab_type": "code",
        "outputId": "9fb5ed9a-8da0-4fc4-df7b-87a0ce67796a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Test performance\n",
        "trainer.run_test_loop()\n",
        "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
        "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test loss: 1.30\n",
            "Test Accuracy: 68.2%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "5672VEginYnY",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Save all results\n",
        "trainer.save_train_state()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "HN1g2vP3nad_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 预测 Inference"
      ]
    },
    {
      "metadata": {
        "id": "Myr8QQjKnZ7k",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Inference(object):\n",
        "    def __init__(self, model, vectorizer, device=\"cpu\"):\n",
        "        self.model = model.to(device)\n",
        "        self.vectorizer = vectorizer\n",
        "        self.device = device\n",
        "  \n",
        "    def predict_nationality(self, dataset):\n",
        "        # Batch generator\n",
        "        batch_generator = dataset.generate_batches(\n",
        "            batch_size=len(dataset), shuffle=False, device=self.device)\n",
        "        self.model.eval()\n",
        "        \n",
        "        # Predict\n",
        "        for batch_index, batch_dict in enumerate(batch_generator):\n",
        "            # compute the output\n",
        "            y_pred =  self.model(batch_dict['surname'], apply_softmax=True)\n",
        "\n",
        "            # Top k nationalities\n",
        "            y_prob, indices = torch.topk(y_pred, k=len(self.vectorizer.nationality_vocab))\n",
        "            probabilities = y_prob.detach().to('cpu').numpy()[0]\n",
        "            indices = indices.detach().to('cpu').numpy()[0]\n",
        "\n",
        "            results = []\n",
        "            for probability, index in zip(probabilities, indices):\n",
        "                nationality = self.vectorizer.nationality_vocab.lookup_index(index)\n",
        "                results.append({'nationality': nationality, 'probability': probability})\n",
        "\n",
        "        return results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "-VVn_zxkRcbf",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Load vectorizer\n",
        "with open(args.vectorizer_file) as fp:\n",
        "    vectorizer = SurnameVectorizer.from_serializable(json.load(fp))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Wx46FK2YRchi",
        "colab_type": "code",
        "outputId": "26a09df1-1548-469e-df7f-6613a644512c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "cell_type": "code",
      "source": [
        "# Load the model\n",
        "model = SurnameModel(num_input_channels=len(vectorizer.surname_vocab),\n",
        "                     num_output_channels=args.num_filters,\n",
        "                     num_classes=len(vectorizer.nationality_vocab),\n",
        "                     dropout_p=args.dropout_p)\n",
        "model.load_state_dict(torch.load(args.model_state_file))\n",
        "print (model.named_modules)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<bound method Module.named_modules of SurnameModel(\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(28, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(28, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(28, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=18, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "LZE2Ov4xRcfq",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Initialize\n",
        "inference = Inference(model=model, vectorizer=vectorizer, device=args.device)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kpPDszLpRfww",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class InferenceDataset(Dataset):\n",
        "    def __init__(self, df, vectorizer):\n",
        "        self.df = df\n",
        "        self.vectorizer = vectorizer\n",
        "        self.target_size = len(self.df)\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Dataset(size={1})>\".format(self.target_size)\n",
        "\n",
        "    def __len__(self):\n",
        "        return self.target_size\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        row = self.df.iloc[index]\n",
        "        surname_vector = self.vectorizer.vectorize(row.surname)\n",
        "        return {'surname': surname_vector}\n",
        "\n",
        "    def get_num_batches(self, batch_size):\n",
        "        return len(self) // batch_size\n",
        "\n",
        "    def generate_batches(self, batch_size, shuffle=True, drop_last=False, device=\"cpu\"):\n",
        "        dataloader = DataLoader(dataset=self, batch_size=batch_size, \n",
        "                                shuffle=shuffle, drop_last=drop_last)\n",
        "        for data_dict in dataloader:\n",
        "            out_data_dict = {}\n",
        "            for name, tensor in data_dict.items():\n",
        "                out_data_dict[name] = data_dict[name].to(device)\n",
        "            yield out_data_dict"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "LDpg2LPKRf0c",
        "colab_type": "code",
        "outputId": "8f7a26c1-ab27-4b9b-cac2-28937d5d910b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 340
        }
      },
      "cell_type": "code",
      "source": [
        "# Inference\n",
        "surname = input(\"Enter a surname to classify: \")\n",
        "infer_df = pd.DataFrame([surname], columns=['surname'])\n",
        "infer_df.surname = infer_df.surname.apply(preprocess_text)\n",
        "infer_dataset = InferenceDataset(infer_df, vectorizer)\n",
        "results = inference.predict_nationality(dataset=infer_dataset)\n",
        "results"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Enter a surname to classify: Goku\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[{'nationality': 'Japanese', 'probability': 0.91458},\n",
              " {'nationality': 'Russian', 'probability': 0.025362423},\n",
              " {'nationality': 'Czech', 'probability': 0.021216955},\n",
              " {'nationality': 'Greek', 'probability': 0.011900755},\n",
              " {'nationality': 'Arabic', 'probability': 0.010394401},\n",
              " {'nationality': 'Korean', 'probability': 0.006069167},\n",
              " {'nationality': 'Polish', 'probability': 0.004974973},\n",
              " {'nationality': 'Vietnamese', 'probability': 0.0015066856},\n",
              " {'nationality': 'English', 'probability': 0.00087393716},\n",
              " {'nationality': 'Chinese', 'probability': 0.0006946102},\n",
              " {'nationality': 'Irish', 'probability': 0.00063219015},\n",
              " {'nationality': 'Dutch', 'probability': 0.0006308032},\n",
              " {'nationality': 'French', 'probability': 0.00042536092},\n",
              " {'nationality': 'German', 'probability': 0.00037614783},\n",
              " {'nationality': 'Scottish', 'probability': 0.0001570268},\n",
              " {'nationality': 'Portuguese', 'probability': 0.0001270036},\n",
              " {'nationality': 'Italian', 'probability': 4.4490163e-05},\n",
              " {'nationality': 'Spanish', 'probability': 3.3140543e-05}]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "metadata": {
        "id": "HQSsKNRSxjRB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 批量正则化 Batch normalization"
      ]
    },
    {
      "metadata": {
        "id": "r3EamVazx2hx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "尽管我们对输入做了标准化，有零平均值和单元方差，帮助收敛。在训练过程中，输入随着穿过不同层和非线性化，也在变化。这也称为内变量移位，它使训练变慢，并且需要更小的学习率。解决方法是[批量正则化](https://arxiv.org/abs/1502.03167)(batchnorm) ，把正则化作为模型架构的一部分。这使得我们可以使用更大的学习率，或的更好的性能，速度更快。\n",
        "\n",
        "$ BN = \\frac{a - \\mu_{x}}{\\sqrt{\\sigma^2_{x} + \\epsilon}}  * \\gamma + \\beta $\n",
        "\n",
        "其中：\n",
        "* $a$ = 激活函数 | $\\in \\mathbb{R}^{NXH}$ （$N$ 是样本数， $H$ 是隐含层维度)\n",
        "* $ \\mu_{x}$ = 每个隐含层均值 | $\\in \\mathbb{R}^{1XH}$\n",
        "* $\\sigma^2_{x}$ = 每个隐含层方差 | $\\in \\mathbb{R}^{1XH}$\n",
        "* $\\epsilon$ = 噪声\n",
        "* $\\gamma$ = 比例参数（已学习的参数）\n",
        "* $\\beta$ = 移位参数 （已学习的参数）\n",
        "\n",
        "\n",
        "Even though we standardized our inputs to have zero mean and unit variance to aid with convergence, our inputs change during training as they go through the different layers and nonlinearities. This is known as internal covariate shift and it slows down training and requires us to use smaller learning rates. The solution is [batch normalization](https://arxiv.org/abs/1502.03167) (batchnorm) which makes normalization a part of the model's architecture. This allows us to use much higher learning rates and get better performance, faster.\n",
        "\n",
        "$ BN = \\frac{a - \\mu_{x}}{\\sqrt{\\sigma^2_{x} + \\epsilon}}  * \\gamma + \\beta $\n",
        "\n",
        "where:\n",
        "* $a$ = activation | $\\in \\mathbb{R}^{NXH}$ ($N$ is the number of samples, $H$ is the hidden dim)\n",
        "* $ \\mu_{x}$ = mean of each hidden | $\\in \\mathbb{R}^{1XH}$\n",
        "* $\\sigma^2_{x}$ = variance of each hidden | $\\in \\mathbb{R}^{1XH}$\n",
        "* $\\epsilon$ = noise\n",
        "* $\\gamma$ = scale parameter (learned parameter)\n",
        "* $\\beta$ = shift parameter (learned parameter)\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "9koMITOdzfZB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "但是在非线性操作之前，激活函数进行零平均值和单元方差的意义是什么呢？它不是整个激活矩阵都有的属性，只是替代了应用于隐含层维度（本例中是 num_output_channels）的批量正则化。所以计算了每个批次所有样本在每个隐含层的均值和方差。同样的，batchnorm 使用计算的训练中激活函数均值和方差。然而，测试时，因为模型使用训练中保存的均值和方差，可能使模型不准确。PyTorch 的 [BatchNorm](https://pytorch.org/docs/stable/nn.html#torch.nn.BatchNorm1d) 会全部自动处理。\n",
        "\n",
        "But what does it mean for our activations to have zero mean and unit variance before the nonlinearity operation. It doesn't mean that the entire activation matrix has this property but instead batchnorm is applied on the hidden (num_output_channels in our case) dimension. So each hidden's mean and variance is calculated using all samples across the batch. Also, batchnorm uses the calcualted mean and variance of the activations in the batch during training. However, during test, the sample size could be skewed so the model uses the saved population mean and variance from training. PyTorch's [BatchNorm](https://pytorch.org/docs/stable/nn.html#torch.nn.BatchNorm1d) class takes care of all of this for us automatically.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/batchnorm.png\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "RsWdAKVEHvyV",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Model with batch normalization\n",
        "class SurnameModel_BN(nn.Module):\n",
        "    def __init__(self, num_input_channels, num_output_channels, num_classes, dropout_p):\n",
        "        super(SurnameModel_BN, self).__init__()\n",
        "        \n",
        "        # Conv weights\n",
        "        self.conv = nn.ModuleList([nn.Conv1d(num_input_channels, num_output_channels, \n",
        "                                             kernel_size=f) for f in [2,3,4]])\n",
        "        self.conv_bn = nn.ModuleList([nn.BatchNorm1d(num_output_channels) # define batchnorms\n",
        "                                      for i in range(3)])\n",
        "        self.dropout = nn.Dropout(dropout_p)\n",
        "       \n",
        "        # FC weights\n",
        "        self.fc1 = nn.Linear(num_output_channels*3, num_classes)\n",
        "\n",
        "    def forward(self, x, channel_first=False, apply_softmax=False):\n",
        "        \n",
        "        # Rearrange input so num_input_channels is in dim 1 (N, C, L)\n",
        "        if not channel_first:\n",
        "            x = x.transpose(1, 2)\n",
        "            \n",
        "        # Conv outputs\n",
        "        z = [F.relu(conv_bn(conv(x))) for conv, conv_bn in zip(self.conv, self.conv_bn)]\n",
        "        z = [F.max_pool1d(zz, zz.size(2)).squeeze(2) for zz in z]\n",
        "        \n",
        "        # Concat conv outputs\n",
        "        z = torch.cat(z, 1)\n",
        "        z = self.dropout(z)\n",
        "\n",
        "        # FC layer\n",
        "        y_pred = self.fc1(z)\n",
        "        \n",
        "        if apply_softmax:\n",
        "            y_pred = F.softmax(y_pred, dim=1)\n",
        "        return y_pred"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "s_QcGx4vN3bQ",
        "colab_type": "code",
        "outputId": "0371a89e-fd9a-4e31-d5bc-033f1f99f9a9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        }
      },
      "cell_type": "code",
      "source": [
        "# Initialization\n",
        "dataset = SurnameDataset.load_dataset_and_make_vectorizer(split_df)\n",
        "dataset.save_vectorizer(args.vectorizer_file)\n",
        "vectorizer = dataset.vectorizer\n",
        "model = SurnameModel_BN(num_input_channels=len(vectorizer.surname_vocab),\n",
        "                        num_output_channels=args.num_filters,\n",
        "                        num_classes=len(vectorizer.nationality_vocab),\n",
        "                        dropout_p=args.dropout_p)\n",
        "print (model.named_modules)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<bound method Module.named_modules of SurnameModel_BN(\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(28, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(28, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(28, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (conv_bn): ModuleList(\n",
            "    (0): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    (1): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    (2): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=18, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "tBXzxtiaxmXi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "You can train this model with batch normalization and you'll notice that the validation results improve by ~2-5%."
      ]
    },
    {
      "metadata": {
        "id": "ERMGiPgAPssx",
        "colab_type": "code",
        "outputId": "7fe57758-e8d8-4d90-b2b3-f4b83ae832ee",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "cell_type": "code",
      "source": [
        "# Train\n",
        "trainer = Trainer(dataset=dataset, model=model, \n",
        "                  model_state_file=args.model_state_file, \n",
        "                  save_dir=args.save_dir, device=args.device,\n",
        "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
        "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
        "                  early_stopping_criteria=args.early_stopping_criteria)\n",
        "trainer.run_train_loop()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 2.63 | [TRAIN ACC]: 24.7% | [VAL LOSS]: 2.31 | [VAL ACC]: 42.2%\n",
            "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 2.01 | [TRAIN ACC]: 43.1% | [VAL LOSS]: 1.95 | [VAL ACC]: 51.7%\n",
            "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 1.63 | [TRAIN ACC]: 51.1% | [VAL LOSS]: 1.70 | [VAL ACC]: 58.9%\n",
            "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 1.37 | [TRAIN ACC]: 58.1% | [VAL LOSS]: 1.56 | [VAL ACC]: 55.4%\n",
            "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 1.22 | [TRAIN ACC]: 59.4% | [VAL LOSS]: 1.43 | [VAL ACC]: 57.8%\n",
            "[EPOCH]: 05 | [LR]: 0.001 | [TRAIN LOSS]: 1.08 | [TRAIN ACC]: 63.2% | [VAL LOSS]: 1.47 | [VAL ACC]: 61.3%\n",
            "[EPOCH]: 06 | [LR]: 0.001 | [TRAIN LOSS]: 0.97 | [TRAIN ACC]: 65.8% | [VAL LOSS]: 1.32 | [VAL ACC]: 61.5%\n",
            "[EPOCH]: 07 | [LR]: 0.001 | [TRAIN LOSS]: 0.89 | [TRAIN ACC]: 67.9% | [VAL LOSS]: 1.31 | [VAL ACC]: 67.4%\n",
            "[EPOCH]: 08 | [LR]: 0.001 | [TRAIN LOSS]: 0.81 | [TRAIN ACC]: 69.6% | [VAL LOSS]: 1.38 | [VAL ACC]: 66.8%\n",
            "[EPOCH]: 09 | [LR]: 0.001 | [TRAIN LOSS]: 0.77 | [TRAIN ACC]: 70.8% | [VAL LOSS]: 1.32 | [VAL ACC]: 65.3%\n",
            "[EPOCH]: 10 | [LR]: 0.001 | [TRAIN LOSS]: 0.66 | [TRAIN ACC]: 72.5% | [VAL LOSS]: 1.29 | [VAL ACC]: 66.8%\n",
            "[EPOCH]: 11 | [LR]: 0.001 | [TRAIN LOSS]: 0.63 | [TRAIN ACC]: 74.0% | [VAL LOSS]: 1.28 | [VAL ACC]: 65.2%\n",
            "[EPOCH]: 12 | [LR]: 0.001 | [TRAIN LOSS]: 0.60 | [TRAIN ACC]: 74.4% | [VAL LOSS]: 1.26 | [VAL ACC]: 68.4%\n",
            "[EPOCH]: 13 | [LR]: 0.001 | [TRAIN LOSS]: 0.61 | [TRAIN ACC]: 74.4% | [VAL LOSS]: 1.33 | [VAL ACC]: 70.4%\n",
            "[EPOCH]: 14 | [LR]: 0.001 | [TRAIN LOSS]: 0.58 | [TRAIN ACC]: 75.8% | [VAL LOSS]: 1.27 | [VAL ACC]: 69.2%\n",
            "[EPOCH]: 15 | [LR]: 0.001 | [TRAIN LOSS]: 0.53 | [TRAIN ACC]: 76.8% | [VAL LOSS]: 1.27 | [VAL ACC]: 69.9%\n",
            "[EPOCH]: 16 | [LR]: 0.001 | [TRAIN LOSS]: 0.53 | [TRAIN ACC]: 77.4% | [VAL LOSS]: 1.30 | [VAL ACC]: 71.2%\n",
            "[EPOCH]: 17 | [LR]: 0.001 | [TRAIN LOSS]: 0.53 | [TRAIN ACC]: 77.6% | [VAL LOSS]: 1.26 | [VAL ACC]: 69.5%\n",
            "[EPOCH]: 18 | [LR]: 0.001 | [TRAIN LOSS]: 0.50 | [TRAIN ACC]: 77.2% | [VAL LOSS]: 1.23 | [VAL ACC]: 70.5%\n",
            "[EPOCH]: 19 | [LR]: 0.001 | [TRAIN LOSS]: 0.51 | [TRAIN ACC]: 77.8% | [VAL LOSS]: 1.27 | [VAL ACC]: 70.5%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "iiAW6AL0QAJ8",
        "colab_type": "code",
        "outputId": "b287547c-4193-4044-c673-0b1b5ea0d43d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 335
        }
      },
      "cell_type": "code",
      "source": [
        "# Plot performance\n",
        "trainer.plot_performance()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2gAAAE+CAYAAAD4XjP+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd8lWWe///XqaknyUly0gukB0ho\ngVBClyaIYkMdGUdndsddp7jLb8b9urPq7Mw2V2fXGXXdGcdh3BnLiAIqFkQSDBhKINR0Sno5Sc5J\nPcmpvz+CUZQSSDk5yef5eCDJfd+5zztHIOd97uu+LoXL5XIhhBBCCCGEEMLtlO4OIIQQQgghhBCi\nnxQ0IYQQQgghhBgjpKAJIYQQQgghxBghBU0IIYQQQgghxggpaEIIIYQQQggxRkhBE0IIIYQQQogx\nQgqaEDcoNTWVxsZGd8cQQgghRsU999zDhg0b3B1DiHFPCpoQQgghhLiq8vJydDodUVFRFBUVuTuO\nEOOaFDQhhllfXx9PPPEEq1evZu3atfz7v/87DocDgD/96U+sXbuWNWvWcOedd1JRUXHV7UIIIcRY\nsH37dtasWcP69evZsWPHwPYdO3awevVqVq9ezU9+8hOsVusVtx86dIiVK1cOfO1XP//Nb37Dz372\nM+688062bt2K0+nk5z//OatXr2b58uX85Cc/wWazAdDW1sbDDz/MihUruOWWW9i/fz95eXmsX7/+\nksy33347e/bsGemnRohhp3Z3ACHGmz/+8Y80Njaya9cu7HY7999/P++//z4rVqzgueeeIzc3F39/\nfz788EPy8vKIjIy87Pbk5GR3fytCCCEEDoeDTz75hEceeQSVSsWzzz6L1WqlubmZ//iP/2DHjh2E\nhYXxwx/+kFdffZU1a9ZcdntGRsZVH2ffvn3s3LmT4OBgPv74YwoLC3n//fdxOp1s3LiRDz74gFtv\nvZVnn32WxMREXnrpJYqLi3nwwQfJz8/HaDRSWlpKWloa9fX1VFdXs3jx4lF6loQYPlLQhBhmeXl5\nPPTQQ6jVatRqNbfccgsHDhzg5ptvRqFQsG3bNtavX8/atWsBsNlsl90uhBBCjAX79+8nIyMDf39/\nAObOnUtubi5ms5mZM2cSHh4OwLPPPotKpeLtt9++7PajR49e9XGmT59OcHAwAKtXr2bZsmVoNBoA\nMjIyqKmpAfqL3O9+9zsApkyZwqeffopWq2X16tXs2rWLtLQ09uzZw4oVK9BqtcP/hAgxwmSIoxDD\nrK2tjcDAwIHPAwMDaW1tRaPRsHXrVo4dO8bq1au57777KCsru+J2IYQQYix45513yMvLIysri6ys\nLHbv3s327dsxmUwEBAQMHOfl5YVarb7i9mv56s/OtrY2HnvsMVavXs2aNWv49NNPcblcAJjNZnQ6\n3cCxXxTHdevWsWvXLgD27NnDzTffPLRvXAg3kYImxDALDQ3FbDYPfG42mwkNDQX63+n79a9/TUFB\nATk5OTz55JNX3S6EEEK4U3t7O4cPH+bQoUMUFhZSWFjIkSNHOHXqFEqlEpPJNHBsV1cXLS0t6PX6\ny25XqVQD92QDdHR0XPFx/+u//gu1Ws17773HRx99xJIlSwb2BQUFXXL+2tpabDYbc+bMwW63k5ub\nS0VFBQsWLBiup0GIUSUFTYhhtnTpUrZt24bD4aCnp4edO3eyZMkSysrK+NGPfoTVakWr1TJt2jQU\nCsUVtwshhBDutmvXLubNm3fJUEG1Wk1OTg5Wq5Vjx45RW1uLy+XiySefZNu2bSxZsuSy2w0GA0aj\nkdbWVhwOB++9994VH7e1tZWUlBS0Wi2lpaUUFRXR09MDwPLly9m+fTsAlZWV3H777TgcDpRKJTff\nfDO/+MUvWL58+cDwSCE8jdyDJsQQbN68GZVKNfD5L3/5SzZv3kxNTQ3r1q1DoVCwZs2agfvKYmJi\nWL9+PRqNBj8/P5544glSUlIuu10IIYRwtx07dvDAAw98Y/vKlSt58cUX+ed//mceeOABVCoVGRkZ\nPPjgg3h5eV1x+x133MFtt91GVFQUt956KyUlJZd93IceeojHHnuMd955h6ysLB577DH+8R//kczM\nTH7yk5/w2GOPsXz5cvz8/HjmmWfw9vYG+oc5/uEPf5DhjcKjKVxfDOgVQgghhBDCg7W0tLBx40by\n8vIueQNVCE8iQxyFEEIIIcS48Otf/5p7771XypnwaFLQhBBCCCGER2tpaWHFihW0tLTw0EMPuTuO\nEEMiQxyFEEIIIYQQYoyQSUKEEEKIIeru7uaxxx6jvb0dm83GI488gsFg4KmnngIgNTWVn//85+4N\nKYQQwiNIQRNCCCGGaPv27UyePJktW7bQ1NTEAw88gMFg4PHHHyczM5MtW7awb9++S9ZyEkIIIS5n\n1Aua0dg55HPo9b6YTD3DkGZ0Se7R44mZwTNze2Jm8MzcnpjZYNC5O8Ko0Ov1lJWVAf2L7wYFBVFX\nV0dmZiYAy5Yto6Cg4JoFTX5GelZuT8wMnpnbEzODZ+b2xMzgebmv9vPRIycJUas9c2YeyT16PDEz\neGZuT8wMnpnbEzNPFOvWraO+vp6VK1dy//3389Of/pSAgICB/SEhIRiNxlHJ4ql/TjwxtydmBs/M\n7YmZwTNze2Jm8NzclyNDHIUQQogh2rlzJ1FRUfz+97+ntLSURx55BJ3uy3dHBzsfl17vOywvMjz1\nyqUn5vbEzOCZuT0xM3hmbk/MDJ6b++ukoAkhhBBDdOzYMXJycgBIS0ujr68Pu90+sL+pqYmwsLBr\nnmc4hucYDLphGSo52jwxtydmBs/M7YmZwTNze2Jm8Lzc426IoxBCCDGWxMfHc+LECQDq6urw8/Mj\nMTGRwsJCAHbv3s2iRYvcGVEIIYSHkCtoQgghxBBt2rSJxx9/nPvvvx+73c5TTz2FwWDgiSeewOl0\nMn36dBYsWODumEIIITyAFDQhhBBiiPz8/Hjuuee+sf21115zQxohhBCeTIY4CiGEEEIIIcQYIQVN\nCCGEEEIIIcYIKWhCCDGB5OV9OqjjnnvuWerr60Y4jRBCCCG+TgqaEEJMEA0N9ezZ8/Ggjv3xj7cQ\nFRU9womEEEII8XUeN0mIpc/Oe/nnmJkYjJdm/KwYLoQQI+1Xv/oPSkrOsGjRHFatWktDQz3//d8v\n8m//9s8Yjc1YLBYeeuivWbhwET/4wV/z93//U3JzP6W7u4vq6irq6mr50Y+2MH/+Qnd/K0IIIcSo\ncTpdGNst1Ld009jWQ2qsnoSogBF7PI8raKVVJn674xT33pTMyqxYd8cRQgiPce+9m3nnnb8weXIi\n1dUXePHFlzGZ2pg7dx5r166nrq6Wf/qnf2DhwkvX62pubuKZZ37NwYOfs3Pn21LQhBBCDBun00Wv\n1U6v1YGlz47F6uj/vK//c6vdic5Xg17nRZB//y+NemQGAdodTppMFhpauqlv7aa+pZv6lh4a23qw\nO5wDx81ONfDIxowRyQAeWNBiw/2B/qImBU0I4an+sreSI6XN1/U1KpUCh8N1xf1z0sK4e3nSoM6V\nnj4VAJ0ugJKSM7z77jsoFEo6Otq/cWxm5gwAwsLC6Orquq7MQgghJgaXy0VHjw2jyUKzuQdTZx+W\nPgcWq53evi8LWK/V0V/CLn7cZ3Nc92P5+2j6y5pOS5C/F3p/L2IjA1ADQToten8vdL5alErFZb/e\nanPQ2NZzsYT10HCxjDWbLDicl/6c1WqURBv8iArxIyrUl6hQP9Li9DfyFA2axxW00EAfwoN9Kas2\n43S6rvjECyGEuDKNRgPAJ598REdHBy+88DIdHR1873ubv3GsSvXlcHKX68oFUQghxPjmcDppbe+l\n2Wy5WMQsGM29NJssGM2WQZUtrUaJj1aNt1ZFkM4LH60Kb60aH6/+3729VAP7fbzUaNRKOrqtmLus\nmDr7MHf1/2rtsFBrvPKbhkqFgkB/7cWrbloC/bSYu6zUt3RjNFv4+k8zHy81kyJ1RIX4ERniR1Ro\nfyELDvBGqRjdvuFxBQ0gMymUTw5XU93cyaSIkRv/KYQQI+Xu5UmDvtr1BYNBh9HYecOPqVQqcTgu\n/eFpNpuJjIxCqVSyb99ebDbbDZ9fCCGE57P02alp7hooXf1lrIdms4XW9j6cl3mjzkurIkzvgyHI\nh7AgHwx6H0ICvPH1vli0LhYwL60KlXL4hif2Wu2Yu6yYO/twKJVU15sxd1oxXSxx5s4+apo7Od/w\nZWadr4bk2KD+AhbSf0UsMsSPIH8tilEuYlfi0QWttMosBU0IIQYpPn4yZWWlREZGERQUBMDSpcv5\nh3/4e4qLT7Nu3QbCwsL4wx9+5+akQgghRkt3r43SKhPFF0wUX2ijyWS57HEBfloSogL6S5j+yyIW\nFuSDzlfjlnLjrVUTEawmIti3/03M2MBvHONyuejutdPe1YfOT0uAr3bUc14vjyxoGUmhAJRUmViT\nHefmNEII4Rn0ej3vvLPrkm2RkVH88Y9vDHy+atVaAB588K8ASEj48ipfQkISzz//21FIKoQQYqTY\n7A4qa9spruovZBcaOgeG+3lrVcxINhDkr+0vYBfLmCHIG2+tR9YGFAoF/j4a/H007o4yaB75TIcE\n+hAR7Et5rRm7w4laJcu5CSGEEEII8XVOl4uapi6KL7RRfKGN8tp2bPb+GQlVSgXJMYFMmRTMlEnB\nTIrUERkROKTh9GLoPLKgAaTF68krqqOqsZPE6G9ezhRCCCGEEGIiMpotFwuZiZIqE12WL+8vjjH4\nXSxkelJigzz2yth45rH/R9IvFrSSKpMUNCGEEEIIMWF1WWyUXByyWHyhDaO5d2CfXudFTkYkUybp\nSY/XE+jv5cakYjA8tqClxvXf4F5abWL9gknuDSOEEEIIIcQosvTZKaowcqi4meILbQPrd/l4qZmZ\nHDpwlSwi2HfMzE4oBsdjC1qAr5YYgx8VF8fRjtSK4kIIIYQQQowFVpuDk2dbOVTSxMmzrQP3ksWH\n65iVEsqUycFMitAN61T2YvQNqqA9/fTTHD16FLvdzve//31WrVo1sG/58uVEREQMLGT6zDPPEB4e\nPjJpvyYtTk+tsZtz9e2kjvCK3kIIIYQQQow2u8NJ8YU2DhU3U1RhpNfav55lZIgv2enhzJ0STkSw\nr5tTiuF0zYJ28OBBKioqePPNNzGZTGzcuPGSggbwu9/9Dj8/vxELeSVp8Xr2HK2ltNosBU0IIYbJ\nnXfewquvvomvr/zAF0IId3A6XZTXmDlc0kRhmXFgko+QAG+WzYomOz2c2DB/Gbo4Tl2zoM2ZM4fM\nzEwAAgICsFgsOByOgStm7pQaF4SC/vXQbs2Z7O44QgghhBBC3BCXy8X5hk4OFTdxpLQJc5cV6F8g\n+qbZMcydEk5iVICUsgngmgVNpVINvIu6bds2Fi9e/I1y9uSTT1JXV8fs2bPZsmXLqP3B8fPWEBeu\n41x9O302B14a95dGIYQYqx566Fv8678+S0REBI2NDfy//7cFgyEMi8VCb28vf/d3P2HKlGnujimE\nEBNKrbGLQ8VNHC5pGph90c9bzeLpkcxNDyctTo9SKaVsIhn0JCF79uxh27ZtvPLKK5ds/9GPfsSi\nRYsIDAzkkUce4eOPP2bNmjVXPI9e74taPfQiZTDoAJiVHk5VXictXVZmpIQN+bwj7YvcnsYTc3ti\nZvDM3J6YGTwz91Ayr1mzmpMnD5OR8S0+/HA7a9asJi0tjZtuuomCggJee+01fvOb36BSKQkN9XfL\n0HUhhBgr+mwOzJ19tHX2Ye7sw9TVh6mz/1dHjxWn04XLBfDF73DxU1z9/0GlVmKzOb/Y85X9/VfM\nrDYnrR39pcxLo2LelHDmpoczLSEYtUom+pioBlXQ8vPzeemll3j55ZfR6S59cXDbbbcNfLx48WLK\ny8uvWtBMpp4bjPolg0E3sMJ5vKH/6t7Bk/VE632GfO6R9NXcnsQTc3tiZvDM3J6YGdyf+53K9ylq\nPnVdX6NSKgamUb6cmWEZ3J60/or7s7IW8vzz/82qVRv46KPd/OAHf8cbb/wfL730W2w2G97e3hiN\nnTgcTlpauujpcV5XvsvxxBIshBjfXC4XnRZbf+n66q+uvku29fTZr3gOhaL/32RQoFDAwPUtBSj6\n/4MCUCoVuFxf7v9ikNkXo82USgUzk0PJnhLO9MRQvLQyGkwMoqB1dnby9NNPs3XrVoKCgr6x79FH\nH+V//ud/0Gq1HDlyhNWrV49Y2MtJjglCqVBQWm0a1ccVQghPk5CQSGurkaamRjo7O8nPzyM0NIx/\n+qdfUFpazPPP/7e7IwohxLBzuVycq+8g/2QDxRfaMHf1YXdc+c0uHy81ep0Xk6MC0Pt7EaTzQv/F\nL//+3/19NSgHcUuPu98MFJ7pmgXtgw8+wGQy8eijjw5sy87OJjU1lZUrV7J48WI2bdqEl5cXU6ZM\nuerVs5Hg46VmcqSO8/WdWPrs+Hh57NJuQogJ5Pak9Ve92nU5w/GDfv78HH772xdZtGgJZrOJxMRk\nAPbty8Vuv/K7xUII4Wnau60UnG4k/2Q9Da39I7j8fTTEhvkT5P+V0nWxeH1RxLy18lpSuNc1/wRu\n2rSJTZs2XXH/Aw88wAMPPDCsoa5XWryes/UdVNS2k5kY4tYsQggxli1ZsoyHH36IrVtfp7fXwi9/\n+SS5uXu444672bNnN7t2vevuiEIIccPsDienzraSf7KBk2dbcbpcqFUK5qaHkZMZyZT4YJlwQ4x5\n4+ItgrR4PbsKqiitMklBE0KIq0hPn8q+fYcGPv/zn7cNfJyTswSAdes2jHouIYQYirqWbg6cbODz\n0w109PSvGRYfriMnM5LsKeH4+2jcnFCIwRsXBS0pOhCVUkGJ3IcmhBBCCDEhWPrsHC5pYv/JBs7W\ndwD909PfNDuGnMxI4sJlkiLhmcZFQfPSqEiMCqCitp3uXht+3vIuiRBCCCHEeON0uSivNpN/soGj\nZc1Y7U4UwLSEYBZlRjEjKRSNWqanF55tXBQ06B/mWF7bTnm1mZkpBnfHEUIIIYQQw8RosvDegfPs\nP9UwsJizIcibnMwoFk6LIDjA280JhRg+46agpcfreffABUqqTVLQhBBCCCE8nMvloviCiU8Kazh1\nrhWXC7QaJQunRZCTGUlybNCgproXwtOMm4KWEBWIRq2ktEruQxNCCCGE8FR9NgcFZxrZU1hLfUs3\nAKlxeuZPDWdOWpgsqSTGvXHzJ1yjVpIUHUhJlYmOHisBvlp3RxJCCCGEEIPU1tHL3mN17DteR3ev\nHZVSwbyp4azMimVuZrQs+CwmjHFT0KD/PrSSKhNl1WbmpIW5O44QQgghhLiGs3XtfFJYQ2GpEafL\nhb+PhvULJrFsZjR6nZe74wkx6sZVQUuP17MdKK0ySUETQgghhBij7A4nR8uMfFJYw7mLU+THGPxY\nmRVL9pRwtBqVmxMK4T7jqqBNitDhpVFRKuuhCSGEEEKMOV0WG/uO17H3WB2mzj4UwIykUFZmxZAW\nr0chk34IMb4KmlqlJDk2kNPn2jB19sllcSGEEEKIMaDO2MUnhbUUnGnEZnfirVVxU1YMK2bHEK73\ndXc8MUxcLhf13Y0cN56mzWIiK2IGafpkKd7XaVwVNOgf5nj6XBtl1SbmTY1wdxwhhBBCiAnJ6XJx\n6mwrnxTWUHyhf3STIcibm2bHkpMZKbMxjhNOl5MLHTWcMJ7muPE0LZbWgX0HGwuJ8A1jaexC5kbM\nxkslk/gNxrj7m5EWpwegpEoKmhBCCCHEaHI6XVTWtXOs3MixciMt7f2LSqfFBbEyK5bpSaEolXI1\nxdM5nA4qzOc4YTzNCeNp2q39M2x6qbTMCstkhmEagV6B7K87yLHmk7xRtp2dZz9iQdQclkQvIMQn\n2M3fwdg27gpafLgOHy+13IcmhBBi1Lz11lu8++67A5+fPn2a119/naeeegqA1NRUfv7zn7spnRAj\ny2pzUHzBxLEKI8crWuiy2ADw1qrIyYxkZVYssWH+bk4phsrqsFHaVs5x42lOtRTTY7cA4Kf2ZV5k\nFjMM00jTJ6NRaQa+JiloMhuT1pFfd5D9dQf5tPoz9lbnk2mYytKYhSQHJYz48Ee7005VRy1n289j\ndViH5ZwZoVOID4gdlnNdzrgraEqlgtTYII5XttDSbiE00MfdkYQQQoxzd911F3fddRcAhw8f5sMP\nP+Rf/uVfePzxx8nMzGTLli3s27ePJUuWuDmpEMOjy2Lj5NkWispbOHW+FavNCUCgn5alM6KYmWIg\nLU6PRq10c9Kxyeqwkl93kAP1h9F7BTIlJJWpIWmE+xrG1P1aFnsvZ1pKOG48zZm2soGCE+QVyJyI\nmcwwTCMxcDIq5ZVn3Qz0CmB9wipWT1rOsaYT5NXuH7jyFu0fydKYhWSFz0T7lWI3FA6ng5quOspN\nZyk3neWs+TxWp21Yzv2Fhu5m/ipj87Ce86vGXUGD/vXQjle2UFplJidTCpoQQojR88ILL/Bv//Zv\n3H///WRmZgKwbNkyCgoKpKAJj9ba3ktRhZGiihbKqs04XS4AwoN9mZUSyqxkA5OjAlCOoYIx1vQ5\nrHxW+zmfVn9Gp60LtVJNU08zpaYK3ql8nxBvPVND0pgakkaKPhGtG+7Zau/t4ED9YY4bT1PeVond\n5QDA4BPCDEMG0w3TiA+IQam4vvKtUarJjpzN3IhZnO+oIrdmP8eNp/lz6TZ2VH7AwuhsFkfPR+8d\ndF3ndbqc1HU1cKitlmM1xVSaz9Pr6B3YH+EXTkpQIsn6BHSa4bmSG6uLGpbzXInHFTSLvZd3Sw8y\nM3AG3mrvyx6THt9/H1pptYmczMjRjCeEEGICO3nyJJGRkahUKgICAga2h4SEYDQa3ZhMiOvncrmo\nM3ZzrMJIUXkLVU2dA/sSogKYmRzKrBQDkSF+bkzpGXrtfXxW11/MumzdeKu8WTNpBctjF2F32ilu\nLeNMaymlpgo+qyvgs7oC1Eo1yUEJFwtbKmG+hmHN5HK5aLd2UNfVSH1XA/XdjdR3NVLX3YDrYvmO\n9o9khmEaMwwZRPqFD8vVPYVCQULgJBICJ2HqNfcPf6w/yO6qXPZU72O6YRrLYnJICIy/7OO5XC4a\nupv6r5CZz1JpOke3vWdgf5hPKLP100nRJ5IclEigl27ImUebxxW0s+bz/OnkOxjjzWxIXHPZY6IN\nfvj7aCipMuFyucbUpWIhhBDj17Zt29i4ceM3tn/xYuda9Hpf1OqhL9BrMHjeCxLwzNyemBmunNvl\nclFyoY2CUw0cPN1AY2v/C1+1SsGs1DDmTYtg7tQIQtxwC4knPtc9Ngv5xv3sKvuUTms3vhof7py6\njptTluGv/bLYJsVEs4Hl2J0OylvOcbzxDEX1pylpK6ekrZxtFRDub2Bm5FRmRk5lqiEFrXrwV9d6\nbBZq2uupNtdT3V5HdXv/793WnkuO06g0pIQkMDd6BnNjphPuP7yl8OsM6EiJjWWz/Vb2VxfyYfle\nippPUtR8ksn6WNYmL2NBXBYtPW2caSrnTHMZZ5rLae/78s0Cg28wc2KnMy0slalhKYT46kc082jw\nuIKWok9C5+XP/vqDrJm0/LKXfpUKBWlxQRSWGWk2W2R9DSGEEKPi0KFD/OxnP0OhUGA2mwe2NzU1\nERYWds2vN5l6rnnMtRgMOozGzmsfOMZ4Ym5PzAyXz221OThY3MSewlpqjV1A/yQfc9PDmJlsICMh\nBF/v/peNTqt91L9vT3uuLXYLeTWfk1uXT7e1Bx+1D+smr2RpTA6+Gh8s7U4sXP77MSgiWBkZwcrI\nFZh6zRS3lXGmtYyytgo+qsjjo4o8NEo1KfokpoSkMi0kjVCfEKD//qumHiP1XQ3UXbwiVt/dSFvv\npZPnKVBg8Akh2ZBAlF8EUf6RRPlHYPAJITwssP+5toDRMnrPeYYug2mzplFpPkde7QFOGM/w4uFX\n+e2RPw8MswQI1AYwJ3wmKfpEUvRJhF6cEfKLPyPGbs/4c3K1Nxw8rqBpVRpWJi7ineIPOdx4jJzo\neZc9Li1eT2GZkdIqkxQ0IYQQI66pqQk/Pz+02v43DhMSEigsLCQrK4vdu3ezefPI3VAuxI1q6+gl\nt6iOfcfr6bLYUCkVzE0PY2FGpEzycQN6bBZya/eTW7Mfi92Cv9aPWxJWsyRmIT5XuDXnavTeQSyM\nymZhVDZ2p51z7Rc4c3E45Be/3mInYb6hqBVqmnqMOL5SZgB0Wn/S9MlE+UdcLGMRRPqFu+X+tmtR\nKBQk6xNJ1ifSamnjs7oCTreWEuUX3l/IghIJG2MTqYwEjytoAKuSFrOzZDe5NftZGJV92f9JX10P\nbcmM6NGOKIQQYoIxGo0EB3+5ts/jjz/OE088gdPpZPr06SxYsMCN6YT4ksvlorK2nU8KazhaZsTp\ncuHvo2Hd/HiWz4pBr/Nyd0SP02PrYW/NfvJq92Ox9+Kn8eXWhLXcPmMVXebhmUFQffGqWYo+iY1J\n62jrNXGmtYzi1jJKTRXgchGjiyL6iytiF8uYTuuZSxyE+ASzMWkdG5PWuTvKqPPIghbsE8SssOkc\naTpGaVsF6SEp3zgmMsSXQD8tpdVmuQ9NCCHEiJs2bRovv/zywOdJSUm89tprbkwkxKVsdidHSpvI\nO9FAZU3/ENwYgz8rs2LInhKOVjP0+x8nmi5bN7nV+eTVHqDX0Ye/xo/bEm9mUfR8vNVe+Gi86WJ4\np3j/QrC3nkXR81gUPQ+nq3+Zg+udWVGMTR5Z0ACWx+ZwpOkYe2vzL1vQFAoFafF6DhU3Ud/aQ3So\nzDAkhBBCiImnvdtKXlEduUV1dHRbUSpgZnIoK7NiSY0Lkjexb0CXtZtPaz5jX+0B+hxWdBp/1k6+\niUXR8/Fyw9BBKWbji8cWtLiAGBIDJ1PcWkZjdxMRfuHfOCb9YkErrTJJQRNCCCHEhHKhsYNPjtRy\npLQJu8OFj5ea1XNjufOmVFROp7vjeaxDDUd5o3w7VoeVAK2O9ZNXkRM9b0ze0yU8k8cWNOi/ina2\n/Ty5tQe4N/X2b+xPi+tf6K60ysSK2TGjHU8IIYQQYlQ5nE6OlhnZc7SWytp2oP+2j5tmxzB/WgTe\nWjWGED+PmhFxrHC6nOw8+yG01rieAAAgAElEQVR7qvfho/bhzuQNLIzKRqvSuDuaGGc8uqBlGqYS\n4q3nUMNRNiSswU9z6WyNhiAfggO8KK024XS5ZGV7IYQQQoxLPb12cotq2XusDlNnHwAZCSGszIph\nyuRgeQ00RBZ7L1vPvM7p1hLCfEN5OPNBwod54WghvuDRBU2pULI0ZiFvV77PgbpDrJq07JL9CoWC\n9Dg9B043UtvcRVy45y1wKIQQQghxJTa7g0+P1rGr4ALdvXa8NCpWzIph+exoIkPk9o7h0GJp5aWT\nW2nobiJNn8x3p30LX40s4SRGjkcXNID5UXN4//xu9tV9zoq4xaiUl85AlBbfX9BKq0xS0IQQQggx\nLjicTj4/1ciO/ecxdfbh46Xm9sUJLJ8VM7CgtBi6CtM5fnf6VbptPSyJWcgdSeu/8VpTiOHm8X+D\nfdQ+zI+cQ17tAYqMp8gKn3HJ/i/WQyutNrNqbpw7IgohhBBCDAuXy8WxciPvfHaOhtYeNGola7Pj\niE/pYdvZrZw4EUC4XxgRvmFEXPw9zNcg90ndgM/rD/NG2XZcuLgn9XYWRc9zdyQxQXh8QQNYGpPD\nvtrPya3Z/42CFhLoTViQD2U1JhxOJyqlTEMqhBBCCM9TUmViW95Zzjd0oFQoWDw9ig0LJ6HxtvPL\nQ8/S6+jD3ttGfXfjJV+nQEGIt36guCV3xuHnDCDCN0yG6l2Gw+lgx9kP2FuTj5/al+9l3E+KPsnd\nscQEMi4KmsE3hIzQKZxsOcO59ioSAuMv2Z8WH8RnJxqobupicmSAm1IKIYQQQly/qsZOtu07y5nz\nbQBkpYWxcdHkgXvMXjn9Z7ps3dyetJ7lsYsw97XT2NNMY3czjT3NNHX3f3ymtZQzraV8WvPluXVa\n/4tX28K/vOrmF0agNmBMrI/mcrmo7aqnTaFB7zKMeCaL3cIrp1+juK2MCN8wvp/5HcJ8Q0f0MYX4\nunFR0ACWxeZwsuUMuTX5lyloej470UBplUkKmhBCCCE8QlNbD9vzz3G4pBmAKZP03LEk8ZLXMkXN\npzjafIKEwHiWxeagUCjQeweh9w4iPTjlkvN123po7G6mW9lORVP1QHmrNJ+nwnzukmOj/CLIjpxN\nVvgMgrwCR/6b/ZouazeHm45RUH9k4IpguG8Yi6Pnkx05Cx+1z7A/ZnNPCy+d3EpTTzNTQlJ5aOp9\nI/I4QlzLuCloyUEJRPtHctx4mrZeE8He+oF9X9yHVlJlYu28+CudQgghhBDC7Uydfbx34DyfnWjA\n6XIxKULHHUsTmTop+JLjuqzdvFH2DhqlmvvT7kKpuPptHH4aXxKDJmEw6MgM+HIdNKvDSlOPsf9K\nW08zNZ11lLRVsL1yFzsqPyAtOJnsiNlMN0wd0cWYnS4nJW3lFNQf4WRLMQ6XA5VCxQxDBv6+Phys\nPspbFTvZefYD5kTMYnH0fGJ0UcPy2OWmSl4+9Se67T0sj13ExqR113w+hRgp46agKRQKlsUu4k8l\nf2Ff7edsTFo3sC/I34vIEF8qatuxO5yoVfIXTgghhBBjS3evjQ8PVrOnsAar3Ul4sC93LE5gdurl\nh/b9pXwHXbZuNiatI9wv7IYfV6vSEquLJlYXPbCty9bNsaYTHGo8RklbOSVt5XiptMwMyyQ7YjZJ\nQZOHrcA097RwsKGQQ41HMff1L64d5RfB/Kg5zAmfiU7rj8GgY33sGgrqj5Bff5AD9Yc4UH+IhMBJ\nLI6ez4ywDDTKG3tZm19XwF/Kd6JAwbfS7mRB1Nxh+b6EuFHjpqABZIXPYGflBxyoP8zaSTfhrfYa\n2JcWryf3WB3nGzpIjglyY0ohhBBCiC/12Rx8erSWDwqq6Omzo9d5cV/OZBZmRFxxcrMvhjZODohn\neeyiYc/kr/FjccwCFscsoKm7mcONxzjUeIyDDYUcbChE7xVEdsQs5kbMuqFy2OewUtR8koKGI1Sa\nzwPgo/ZmUfR85kdmEaeL+UYp1Wn9WTVpGTfFL+FMaymf1RZQ3FbGufYL6CreY0HUXHKisy8ZRXU1\nDqeDtyvfY1/t5/hr/PjetM0k6xOu+3sRYriNq4KmUapZFDOfD85/wuHGoyyOWTCwLz2uv6CVVpmk\noAkhhBDC7VwuFweLm3grtxJzlxU/bzV3LUtkxawYtJorr7XVZe3mzbLtaJRqNqdfe2jjUIX7hXFL\n4hrWJayi0nyeQ41HKWo+yUdVe/moai+TAuLIjpjFrPDp+GuuvDi2y+XifEc1BfVHONZ8gl5HHwAp\n+iTmR2YxwzBtUEMolQolGaFTyAidQnNPC/vrDlLQcISPq/ayuyqXaaHpLIleQGpw0hWfmx5bD78/\n/WdKTRVE+oXzcOaDhPoEX/ZYIUbbuCpoAIui57H7wl5ya/aTEz1v4C9malx/KSutNnPLQncmFEII\nIcRE19bRy6sfl3HybCtatZJ18+NZmx2Hr/e11yv7S/kOOm1dQx7aeL2UCiUp+kRS9IlsSrmNE8Yz\nHGo8SmlbBRc6qtlW8R7TQtPJjpjN1JBU1BeHHHZYOznc2D/hR2NP/4Qneq8glsUuYl5k1pCKUZhv\nKLcnr2d9wmqONp/gs9rPOdVSzKmWYgw+IQNX5L66nEBTdzMvndxKs6WFaSHpfGfqvfiovYf25Agx\njMZdQQvQ6siKmMnBhkKKW8uYFpoOgM5XS4zBn4radmx2Bxq1rAIvhBBCiNHlcrnIP9nAm3srsPQ5\nSI/X8521aRiCBjdb4PERHto4WFqVljkRM5kTMRNzXzuFTcc51HCUE8bTnDCexk/jy6yw6bT3dXC6\ntQSny4laoWJ22HTmR80hVX/lq1s3lkfD/Mgs5kdmUdVRw2e1BRQ2H+edyvd579zHZIXPYHH0fLrt\n/VfOLHYLK+OWsiFxjUwGIsaccVfQAJbF5HCwoZDcmv0DBQ0gPV5PrbGLs3UdpMUPbnyyEEIIIcRw\nMJotbP2wlJIqEz5eKr6zNo1FmZGDXturf9bG7ahHaWjjYAV5BXJT3BJWxC6mtquBw41HOdJYRH5d\nAQCx/lHMuzjhh98oLIwdHxDL5imxbExex8GGQvJrCyhoOEJBwxEA1AoVm9PvZl5k1ohnEeJGjMuC\nFqOLIiUokVJTBfVdjUT5RwD9C1Z/UlhDSZVJCpoQQgghRoXT5WLv0Vq27TuL1eZkemIIm1enEhxw\nfcPq3DW0cbAUCgWxuihidVHclngzlebz+Gp8iR2mqfCvl7/Gj5vilrA8dhElbeV8VltAs8XI5vS7\nSQic5JZMQgzGuCxo0L9wdbn5LLk1+Xwr/S4AUmODUCigtNrk5nRCCCGEmAga23p45YMSKmvb8fNW\n88CaNOZNCR/0VbMvjJWhjYOlUqpIDU5ydwyg/965qSFpTA1Jc3cUIQZl3Ba0aaHphPqEcLipiA2J\na9Fp/fH11hAfruNcfQd9VgdeWrkPTQghhBDDz+F0svtwDdvzz2N3OMlKNfCtVakE+l3/Qs9jdWij\nEGJkjNu/4UqFkmUxOdiddvbXHRrYnhavx+F0UVFndmM6IYQQQoxXtc1d/MurR3kr7yy+3mr+9rZp\n/O3GjBsqZwBvVeyk09bFLQmrx+TQRiHE8Bq3BQ1gXuRsvFXefFb3OTanHeifKASgtEoKmhBCCCGG\nj93h5N395/n51iNcaOxk/tQIfvm9bLLSbrxUHTeeprDpOJMD4jxiaKMQYujG7RBHAG+1Nwui5rC3\nJp9jTSfIjpxNckwgKqWCkiq5D00IIYQQw+NCYwev7Cqh1tiNXufFA2tSyUwMHdI5u6zdvFH6Dmql\nmvvT75ahjUJMEOO6oAEsjVlIbs1+cmvymRsxC2+tmkmROs7Xd2Lps+PjNe6fAiGEEEKMEJvdwc79\nF/joUDVOl4slM6K4a2kSvt5Df33xxdDGjUnriJChjUJMGOO+nYT4BDPdMI3jxlOcbb9AUtBk0uP1\nnK3roLzGzPSkob27JYQQQoiJqeR8G7967SiNbT2EBnrznbVpTJkUPCznlqGNQkxcE+Ja+bLYHAD2\n1uQDkBbXfx+aDHMUQgghxPVyuVy8ve8sj72QT1NbDzdlxfCL72YPWznrssnQRiEmsnF/BQ0gMXAS\ncboYThrP0GJpJSk6CLVKIeuhCSGEEOK6OF0u/ry7nNyiOiJD/fjOmlSSY4KG9THeKu8f2nhb4s0y\ntFGICWhQb8k8/fTTbNq0iTvuuIPdu3dfsu/zzz/nzjvvZNOmTbzwwgsjEnKoFAoFy2JzcOFiX+3n\naDUqEqMCqWnqostic3c8IYQQQngAp8vFqx+VkltUR4zBn//84aJhL2dfHdq4Im7xsJ5bCOEZrlnQ\nDh48SEVFBW+++SYvv/wy//qv/3rJ/l/+8pf85je/4fXXX+fAgQNUVlaOWNihmBWWSaBWx+f1h7HY\ne0mL1+MCyqplun0hhBBCXJ3T6eKVXSV8dqKB+HAdP71vJoH+XsP6GF22bt4ok6GNQkx01/ybP2fO\nHJ577jkAAgICsFgsOBwOAGpqaggMDCQyMhKlUsmSJUsoKCgY2cQ3SK1UszhmAb2OPg42FA6sh3bq\nXIubkwkhhBBiLHM4nfz2vTN8frqRhKgAfnLvDPx9NMP+OG+V76TT2sX6yatkaKMQE9g1C5pKpcLX\n1xeAbdu2sXjxYlQqFQBGo5Hg4C9viA0ODsZoNI5Q1KHLiZqHRqkmr2Y/CVE6QgK8OFTSjKXP7u5o\nQgghhBiD7A4nL+08w+GSZpJiAtmyaQa+3sNfzk5cHNo4SYY2CjHhDXqSkD179rBt2zZeeeWVIT2g\nXu+LWq0a0jkADAbd9X8NOhZPmsen5/ZTY7/A2gWT+dNHpZypNrN2weQhZxpUhhvIPRZ4Ym5PzAye\nmdsTM4Nn5vbEzEJ4Kpvdyf/sOM3xyhbS4oL40Z2ZeGuHf361Lls3r18c2rg5/S4Z2ijEBDeof2Xy\n8/N56aWXePnll9HpvnxxEBYWRkvLl0MEm5qaCAu7+iV5k6nnBqN+yWDQYTR23tDXzgudy6fn9rPz\nzCd8J+VBXlMoeC//HLOTQlAoFEPOdjVDye1OnpjbEzODZ+b2xMzgmbk9NbMQnshqc/DC9tOcOtfK\nlEl6fnhHJl6aob/BfDlfDG3sn7UxfEQeQwjhOa5Z0Do7O3n66afZunUrQUGXzlQUExNDV1cXtbW1\nREREkJubyzPPPDNiYYdDlH8EafpkSk0VdLpamJkSytEyI+caOkiMCnR3PCGEEEK4WZ/NwW/ePknx\nBRMZCSH84PZpaAYx+sflcmF12uhz9NFnt/b/7rDS6+gb+Lh/35cfd1q7ONp8QoY2CiEGXLOgffDB\nB5hMJh599NGBbdnZ2aSmprJy5UqeeuoptmzZAsDNN9/M5MmjM1RwKJbHLaLUVEFuzX6WzljJ0TIj\neUV1UtCEEEKICa7Xaue5t05SVmNmZnIoD986DY360iGHTpeTXec/oexYOd19lq+ULysuXNf9mH5q\nXxnaKIQYcM2CtmnTJjZt2nTF/XPmzOHNN98c1lAjLT04hXBfA0ebjrNh/lrCgnw4XNLMPSuS8RuB\nG3+FEEIIMfZZ+uz811snqKxtJyvVwF9vmIpadWlpsjnt/LH4DYqaT6JRafBReeOl0qLT6vBSafFS\ne+Gl8ur/WOWF91c+vtL+AK/+rxVCCLiOSULGE6VCydKYHN4s305ubT5LZkzjrbyzfH66kZVZse6O\nJ4QQQohR1tNr49k3T3C+oYPsKeF8b306KuWl5cxi7+W3p16l3FRJUtBk/nHZD+hpd7gpsRBivJqw\n19LnRc5G7xXE3pp8kpKUqJQK8orqcLmuf2iCEEII8e6777JhwwZuv/128vLyaGhoYPPmzdx33338\n+Mc/xmq1ujuiuIIui43/fP045xs6WDgtgr9aP+Ub5azD2slzRf9LuamS6aFTeWT69/DT+ropsRBi\nPJuwBU2r0nJP6kacLifvVr3L7DQDDa09VNS2uzuaEEIID2MymXjhhRd47bXXeOmll/j000/59a9/\nzX333cdrr71GfHw827Ztc3dMcRkdPVaefq2IqqZOFk+P5MF16SiVl87q3GJp41dHX6Sms44FkXP4\n7rT70arklgghxMiYsAUNYFpoOjPDMjnXXkXo5GYA8orq3JxKCCGEpykoKGD+/Pn4+/sTFhbGL37x\nCw4dOsSKFSsAWLZsGQUFBW5OKb6uvauPp18rotbYxbJZ0Xx7TRrKry25U9tZz7NHX8BoaWVN/HLu\nS7sTlXJkptsXQgiY4AUN4M7kW/BWeXPQlEd4mJLCsmY6e2QYihBCiMGrra2lt7eXhx9+mPvuu4+C\nggIsFgtabf/EDyEhIRiNRjenFF9l6uzj318ror6lm5VZsdy/MuUb5azCdJb/OvYSHdZO7kq+lVsS\n14z4mqlCCDEhJwn5qiCvQG5NXMub5dsJTqqgqTmRA6caWZMd5+5oQgghPIjZbOb555+nvr6eb3/7\n25fc0zzY+5v1el/Ug1hv61o8dYHw0crdbOrhmTeO09TWwx3Lknhg3ZRvFK/Dtcd54cTvceLiR/Me\nIid+zmXPJc/16PHEzOCZuT0xM3hu7q+b8AUNICc6m8ONRznfUYE2OJC84z6smhv7jXfShBBCiMsJ\nCQlh5syZqNVq4uLi8PPzQ6VS0dvbi7e3N01NTYSFhV3zPCZTz5CzGAw6jMbOIZ9ntI1WbqPZwn++\nXkRLey8bFk7i5rmxtLR0XXLMgfpDvF76DhqVhr+Z9m1SfVMum02e69HjiZnBM3N7YmbwvNxXK5MT\nfogj9E+7f2/aHSgVSnwSSmhu76SkyuTuWEIIITxETk4OBw8exOl0YjKZ6OnpYcGCBXz88ccA7N69\nm0WLFrk55cTldLkoqzbx6sdl/PPWI7S097Jx0WRuW5RwyZUzl8vFRxc+5bXSt/HT+PLjmX9NekiK\nG5MLISYiuYJ2UbR/JDfFLWF3VS7q6LPsK4pk6qRgd8cSQgjhAcLDw1m9ejV33303AD/72c/IyMjg\nscce48033yQqKorbbrvNzSknFpfLRXVTF4eKmzhU0oSpsw+AAD8t31qZworZMZcc73Q52VbxHvtq\nDxDsrecH079LuN+1r3oKIcRwk4L2FWsnreBY0wlaIi5wvDgKc1cKQf5e7o4lhBDCA9xzzz3cc889\nl2z7wx/+4KY0Y0NTj5HdVbmsjFtChF/4qDxmY1tPfykrbqKxrX/IqI+XmpyMSLKnhpMWF/SNNc7s\nTjuvFr/J0eYTRPlF8MiM7xLkFTgqeYUQ4uukoH1F/9pot/P8iZdRxZ/msxMZbFiY4O5YQgghhMdx\nuVy8Xvo2FeZznDIW8zfTH2Jy4MhMwNXW0cvhkmYOlTRR1dh/D4pGrSQrLYzs9HAyE4PRXGHylV57\nL7879X+UmipICJzE32R+B1+NLEAthHAfKWhfkx6SwszQ6RRxgr1Vn7N+/uRvLFgphBBCiKsrbiun\nwnyOMN9QjD2t/Lrof/lexreZGpI6LOfvstgoLGvm0JkmymvMuAClQkFGQgjZU8KYmWzAx+vqL3M6\nrV28eOIVqjtryQhN56Gp30Kr0g5LPiGEuFFS0C5jU9qtnMwvwWo4w6GKC8xPnezuSEIIIYTHcLqc\n7Dz7AQoUfG/aZlotbbxy5s+8dPIPbE6/m7kRs27ovL1WO8crWjhY3MSZ8204nP3LFyTHBDJvSjiz\n08II8B1cwWq1tPH8iZdp7mlhXmQW96XeIQtQCyHGBClol6HT+nNT1Eo+btjFjvPvMz/1h+6OJIQQ\nQniMI41F1HU1kB0xm2j/SKL9I/nBjL/ipZNb+WPxG3RZu1get3jQ5+uzOvjTJ2UcKW3GanMCEBfm\nT/bUcOamhRMS6H1d+eq6Gnjh+O9pt3awMm4ptyaulQWohRBjhhS0K1iftoi95w/R5V3DgaoiFsbP\ndHckIYQQYsyzOWy8d+5j1Eo16xNWDWxPCprM3816mBeO/563K9+nw9o16GK062AVB041EhbkQ/aU\ncLKnhBMV6ndD+SrN53np5FYsdgt3JK2/rqIohBCjQdZBuwKlQslN4Tfjcip4++y79Np73R1JCCGE\nGPPy6wow9ZlZEr2AYG/9Jfui/SPZMvtvCfMN5ZPqPP5U+hYOp+Oq52tt7+Xjw9UE+Wv5+UNz2bg4\n4YbKWa+9jx2VH/Bc0f/S5+jjgSn3SDkTQoxJUtCuYmVGOormJPro5t2zH7s7jhBCCDGmWewWPrqw\nFx+1N6smLbvsMSE+wfz9rL8lThfDwYZCfnf6VawO6xXP+fZnZ7HZndyxJBEv7fXfI+ZyuShqPsUv\nDj3DJ9V56L0CeWT6d2/4PjghhBhpUtCuwkurIjt0AU6LL5/VfU5VR427IwkhhBBj1idV++i297Aq\nbhn+mitf5dJp/fnxzO+Tpk/mVEsJvzn+Ml3W7m8cd66+g4NnmogP1zF/WsR152nuaeHFE6/w8un/\no8vaxdpJK/hZ9v9HWnDydZ9LCCFGixS0a1g+Ix7bham46F/P5VpDMYQQQoiJyNzXzt6afIK8Alka\nm3PN473VXvzN9AeZHTadc+0XeHLvrzD3tQ/sd7lcvLG3AoB7ViShvI5JPKwOG++f282/HP4VxW1l\npAen8I/Zf8/6hNVoVZrr/+aEEGIUySQh1xAT5k9CYAJVxnpqqCOv9gArZMy6EEIIcYkPzn+CzWlj\n3eSVgy5BaqWa70y9F3+tP/tqD/Ds0Rf5wfTvEu4XRmGZkcradmalGEiN01/7ZBedbinhrfKdtPS2\nEeQVyB3JtzDTkCGzNAohPIZcQRuEpTOisNWkosGb9899TKvF5O5IQgghxJjR2N3M5/VHiPANIzti\n9nV9rVKh5K7kDdyTsYG2XhO/OvY/nDVd4K3cSlRKBXctSxzUeVotJv735B/5n5N/oK3PzIq4xfxT\n9hZmhWVKORNCeBQpaIOQlRqGn9oXZ20aVqeNv5TvwOVyuTuWEEIIMSa8e+4jXLjYkLj2hhZ7VigU\n3D5lLfel3UG3rYfnin5Lm6uWFbNjCNf7XvVr7U47H1/Yyy8OPcPJljMkBk7m/815lNuT1uOtvr71\n0YQQYiyQIY6DoNWoWJgRye4jNiYnxHG6tYQi4ylmhWW6O5oQQgjhVufaqzhhPE1CYDyZoVOGdK6F\nUdkoHV78qewNvFKOEpOScNXjS9sq+Ev5Dpp6jOg0/tybejtzI2bJFTMhhEeTgjZIS2ZEsftIDcq6\nDNTh9Wwr30l6cDI+ah93RxNCCCHcwuVysaNyFwC3Ja4blmJUWexDX2UW/unHeb3iL9joZdnXJh0x\n97XzTsX7HG0+gQIFi6MXcEvCanw18jNZCOH5ZIjjIEWG+JEWF8TZ8w4Whi2i3drJu2c/cncsIYQQ\nwm1Ot5Zwtv0CmaFTSQyaNOTz1Rm7yDteR5gmhr+b/TcEaHVsq3iX985+hMvlwuF0sLcmn18cfIaj\nzSeID4jlp3N+yKbU26ScCSHGDbmCdh2WzoymtNqMvWEyEX5nyK87yJyIWSQExrs7mhBCCDGqnC4n\nO85+iAIFGxLXDMs538ytxOWCu5cnER8YypbZj/D88d/xUdVejJZWGnuaqetqwFftw72pt7Mgai5K\nhbzXLIQYX+RfteswK8WAzldDwalm7k7eKGujCSGEmLAONRylsbuJ+ZFZRPqFD/l8R0ubOH2ujSmT\n9ExPDAEg1CeYLbMfIVYXzdHmE9R1NbAgcg5PzvspOdHzpJwJIcYl+ZftOqhVShZlRtHda6etwY+F\nUXOp727k05rP3B1NCCGEGDVWh433z+9Go1Rz8+SVQz6fw+nk9++eQQFsWp58yb1sOq0/j878Pusm\nr2TL7Ef4Vvpd+Gv9hvyYQggxVklBu06LZ0QBkHu8jtsSb0an8eeD83tosbS6OZkQQggxOvbVHsDc\n187SmBz03kFDPt9nJxqoaepk0fRIYsP8v7HfW+3NzZNXyi0FQogJQQradQoL8mHa5GAqa9tpMzu5\nM/kWbE4br5x5jS5bt7vjCSGEECOq29bDx1W5+Kp9WBW/bMjn6+m1syP/HD5eKjYuuvq0+kIIMRFI\nQbsBS2dGA7CvqJ7Z4TPIjphNVUcNzx59Qa6kCSGEGNd2V+VisVtYPWn5sMycuKvgAp09Nu5cnkKg\nv9fQAwohhIeTgnYDpieFEOSv5fMzDVhtTu5Pv4tV8cto7mnhmcIXqOqocXdEIYQQYtiZes3k1R5A\n7xXEkugFQz6f0Wzhk8IaQgK8uHVJ4jAkFEIIzycF7QaolEoWT4/C0ufgUEkTSoWSWxPXsillI122\nbv772Eucail2d0whhBBiWL1/fjd2p531CavQqDRDPt9beWexO1zcsTQRL41qGBIKIYTnk4J2gxZP\nj0KhgH3H677cFjOfv874Ni7gf0/+kfy6AvcFFEIIIYZRfVcjhxqOEuUXwdyIWUM+X0WtmcLSZhKi\nAshOH/o0/UIIMV5IQbtBwQHeTE8M5XxDJxcaOwa2Zxqm8uis7+On8eWNsu3sPPshTpfTjUmFEEKI\nodt59kNcuLg1ce2Q1x9zuly88WklAPesuHRafSGEmOikoA3B0pn9U+7n/f/s3Xd8VFX+B/zPnZ6Z\nTPpMKumh99BCTOgI2LCsSBZs7FoRHxdl0WddfH7uTxfF/WF3xbIqFpR1WVZdwUIRCKGHTggJ6WXS\nM2nT7vNHkoFICal3Jvm8XxszM/fOyWev49z5zjn3nMOFbR6P9ArHk/FLYfQIwNacbfj45AbYHDYp\nIhIREXXZ2cosHC8/hTifaAzzH9zl9tJOliC7qAYThhgRG+rdDQmJiPoOFmhdMDzKH/5eaqSdLEFD\nU9sCzKD1x/L4RxHlFYH9JYfx5pH3UWeplygpERFR54iiiH+f+w4AcEvMvC73djVZ7di4/RwUchnu\n4MQgRESXYIHWBTKZgOTRoWiy2rH3RPEl2z1VOiwb8wBGG4Yjo+oc/vzTGlQ2VkmQlIiIqHPSy04g\nuyYXow0jEOUd3uX2tn4sOmUAACAASURBVO7LRWVtE2aPH4AAn65P009E1NewQOuipJHBkMsEbNmf\nB6vt0mvNVHIllgxfhGlh1yGvpggvH3gD+bWFl2mJiIjItdgddmw+91/IBBlujr6+y+1VmZvw3d5c\n6LVK3JAQ0Q0JiYj6HhZoXeTjqca0MaEorWzA9/tyL7uPTJDhjoE34+7Rd6DaUoP/O/Q2TlVk9HJS\nIiKijkkt2o+SehMmB49HoM7Y5fb+tTMLTVY7bk2Khoda0Q0JiYj6HhZo3WB+UjS8dCp8u+c8yqoa\nrrjfjYNmYMnwRbCJdryV/gH2Fh3oxZRERETXrsluwXfZP0AlU2Je1Kwut5dbUotdR4sQatAhaVRw\nNyQkIuqbWKB1A61GgQXTYmGxOfD5T2evuu9Y40g8Nvr30MjV+OTUl/hv9o8QRbGXkhIREV2bbXm7\nUG2pxfTwZHirvbrUliiK2PBzJkQAC6bHQi7jxw8ioivhO2Q3mTQsEAMH+ODw2TKkZ5Zddd9Ynygs\nj38U/hpffJO9FZ+d3gi7w95LSYmIiK7ObKnDDznboVNqMTN8SpfbS88sx6mcSoyM8cfwKP9uSEhE\n1HexQOsmgiBg0eyBkAkCPvsxAxbr1QuuIJ0Ry+OXIlwfij1F+/HO0X+g0dbYS2mJiIiu7Ifc7Wi0\nN2Ju5Ex4KDRdastmd2DDtkzIBAG/mRbbTQmJiPouFmjdKMzgiZnjwmCqasR/0y4/YcjFvNV6PD7m\nIQzzH4yTFWew9tA7qG6q6YWkREREl2exW7C7cB/0Sk9cFzqpy+1tO1yAkop6TBkTgtAAXTckJCLq\n21igdbNbrouCj6cK36bmoLSy/YWpNQo1HhxxDxJDJiDPXIiXD7yBQvOla6oRERH1hoMl6WiwNSAx\nZAKUsq7NtGhusGLzrmx4qBW45bqobkpIRNS3sUDrZh5qBRZMj4PN7sBnP569pglA5DI5Fg66HTdF\nX4/Kpir87dBbOFOR2QtpiYiI2tpZkAoBAhJDJ3a5re/TclHXaMONkyPgpVV1Qzoior6PBVoPmDDE\niCERvjh6rhxH2pkwpJUgCJgTOQP3Dl0Iq92KN9Lf4zT8RETUq3Jq8pBbm4/hAUPgp/HtUlvmBit+\nOpQPb50KM8aGdVNCIqK+jwVaDxAEAb+dNRBymYDPfjiLpnYmDLnY+KAxWHrRNPzfZG3lNPxERC4u\nLS0NkyZNwuLFi7F48WI8//zzKCoqwuLFi5GSkoLHH38cFotF6pjt2lmQCgBICk3ocls/HcxHk8WO\n6yeEQ6WUd7k9IqL+4poKtIyMDMycORPr16+/ZNv06dORkpLiPCmVlJR0e0h3FBKgw+wJA1Be04hv\nU3M69Nw432g8Gf8oAjR++O/5H/HxqQ2wOmw9lJSIiLrDhAkT8Mknn+CTTz7Bs88+i9deew0pKSn4\n7LPPEBERgY0bN0od8arqrfU4WHIEARo/DPGL61JbDU02/LA/D54eSkwdE9JNCYmI+od2C7T6+no8\n//zzSEi48rdp69atc56UAgMDuzWgO7tpciR89Wp8n5aDkor2Jwy5WKDOiCfHLUWkVzj2FR/Cm0fe\nQ721Y20QEZF00tLSMGPGDADAtGnTkJqaKnGiq9tbdABWhw3XhU6CTOjaAJufD+WjvsmG2eMHQKPq\n2kQjRET9TbvvwCqVCuvWrYPRaOyNPH2KRqXAwhlxsNlFrP8ho8NDFfUqTzw+5kGMNozA2aosrDn4\nFsoaKnooLRERdUVmZiYeeughLFy4ELt370ZDQwNUquaJMfz9/WEymSROeGUO0YFfCvZCIVMgIXh8\nl9pqstixZV8ePNQKTOe1Z0REHdbu11oKhQIKxdV3W7VqFQoKChAfH4/ly5dDEIRuC+ju4gcZMCzK\nDyeyK7DnWBEGBus79HyVXIklw3+LTee+w0+5O7HmwBt4aNS9iPQK76HERETUUZGRkVi6dCnmzp2L\nvLw83H333bDbL1x/fK1f0Pn6aqFQdP16LYOhY+eao8WnUNpQhuSIiYgKDerS39604xzMDVYsmDUQ\nEQM6NtFIR3O7AnfMDLhnbnfMDLhnbnfMDLhv7l/r8riDZcuWISkpCd7e3nj00UexZcsWzJkz54r7\nS3XykdJjC8Zg6cvb8N6mY3j7jzOgUXf8sD9oXIjIgBB8cHgDXj38dyybdD8mhI3ugbSX507Hu5U7\nZgbcM7c7ZgbcM7c7Zu4PAgMDMW/ePABAeHg4AgICcOzYMTQ2NkKj0aCkpOSaRqJUXsP6me0xGPQw\nmWo79JxvTv4MAJgQMK7Dz72Y1WbHxp8zoFbJkTg0sENtdSa31NwxM+Ceud0xM+Ceud0xM+B+ua92\nPu9ygTZ//nzn7eTkZGRkZFy1QJPq5CMlFYA5Ewfgmz05+HDzcdwxNaZT7Yz1GQvVCA+8f+JTvLL7\nXdwWewOmDUjq8R5LdzvegHtmBtwztztmBtwzt7tm7g82b94Mk8mEJUuWwGQyoby8HLfddhu2bNmC\nW265BVu3bkVSUpLUMS+rqqkaR8tOIswzpMujM3YdLUK12YK5E8Ph6aHspoRERP1Ll64Crq2txZIl\nS5xTB+/fvx9xcV2b+amvuiEhEkZfD2zZl4ui8rpOtzM8YAieGPsQvFSe+GfmN/jq7L/hEB3dmJSI\niDpq+vTp2L9/P1JSUvDII4/gueeewxNPPIFNmzYhJSUFVVVVbb7QdCW7C9LgEB1IDk3o0hd+NrsD\n3+3NgUohw+wJHIZPRNRZ7fagHT9+HKtXr0ZBQQEUCgW2bNmC6dOnIywsDLNmzUJycjIWLFgAtVqN\noUOHXrX3rD9TK+X4/fwR+N8P92H91gw8edfoTp8Iw/VheGrcY3gr/QPsyN+DisZK3Dfst1DLVV3O\n6RAdKDQXI6PqHDIqM1FkLsE98XcgWh3b5baJiPoqT09PvPPOO5c8/uGHH0qQ5trZHXbsLtwHjVyD\ncUFjutRW6vFilNc0Yea4MHjrun4+IiLqr9ot0IYPH45PPvnkitvvuece3HPPPd0aqq+aOCwII2P8\ncfRcOfafLsWEIZ1fksBX44M/xD+M946tx7GyU1h76G08NPI+eKu9OtSOKIooqTchozITGZXnkFF1\nDnUXTecvQMD/7XkPi4fciQlBYzudl4iIXM/RspOottRgSlhil77kszsc+HZvDhRyAXPYe0ZE1CVc\nnKQXCYKAlJlxOHm+El/8dBYjov3h0YkJQ1p5KDzwyKj78cWZr7GnaD9ePvAGHhl1P0I8rzwDlyiK\nKG+sQEblOZypzMTZynOotly4psVX7YPhQUMw0DcGA31jUN1Ui7ePvo+PT26AzWHD5JAJnc5LRESu\n5ZeC5rXZkkIndamd/adKUVrZgKmjQ+DnpemOaERE/RYLtF5m9NVi3qRwbN59Hpt3Z2PB9K5dsyeX\nyZEy+A74e/jjP1nf45WDb+H3IxZjsN+Fdisbq5p7x1p6yCoaK53b9CpPxBtHYZBvLOJ8Y2Dw8G8z\n9NJP44tV057A/2x7FZ+e3giLw4qpYYldykxERNIrqSvFmcpMxPlEI1jX+REdDlHEN6k5kAkC5k2K\n6MaERET9Ews0CcybFIE9x4vxw/58JI4IRpjBs0vtCYKAOZHTEaDxxSenvsSb6e9jTsR01FhqkVF5\nDqUNZc59dQotRhuGI843BoN8YxGkNbZ7LVyk7wA8PuZBvH5kHb7K+DesditmRUztUmYiIpLWL4V7\nAQBJoQldaufQGRMKy+qQOCIIAT4e3RGNiKhfY4EmAZVSjt/OGohXNx7Fp1szsCJlTLdMlT8uaAx8\nND549+hH+O78jwAAjVyN4f6tQxZjEeoZBJnQ8ck7QzyD8MTYh/Da4XXYdO47WOwWzIuaxUXJiYjc\nkMVuwd6ig/BS6THKMKzT7YiiiG/2nIeA5tmKiYio61igSWRUbABGxwbgSGYZ9p4sQcKwK1831hGx\nPlFYMX4ZTlVkIMwzBOH6UMhlXV8YHACMWgOeGPsQXj38Lr47/yOsDhtuiZnLIo2IyM0cKElHg60B\nUyJnQCHr/EeB9HPlyC01Y8IQI4L8tN2YkIio/+rSOmjUNSkz46BUyPDlz5mob7R1W7sBHn5ICp2E\nKO/wbivOWvl7+OEP8Q8jUGvAD7nbuQ4bEZEb+qVgDwQIuC5kYqfbaO09A4AbJ0d2TzAiImKBJqUA\nHw/cmBCB6joLNu3KkjrONfNRe+P/GfsQQnRB2JG/B5+f/ppFGhGRm8ipyUNubQFGBAyFr8an0+2c\nzKlEVmENxg40dPlaaiIiuoAFmsTmTAyH0dcDPx3MR25JbftPcBFeKj0eH/sgBuhDsadoHz4+uQF2\nh13qWERE1I6d+c1T6yd3cXKQ/+w+DwC4cTJnbiQi6k4s0CSmVDRPGCKKwPofMuAQRakjXTNPpQ7L\nRj+AKK8I7C85jA9OfAabo/uGavYEh+jAwZIj+Ou+tfh/d/8v/nPue1Q2Vkkdi4ioV9RZ63Gw9AgC\nPPwxyC+20+2cya1ERl4VRsb4IzLIqxsTEhERJwlxASOi/RE/0ICDGSakHi9G4ohgqSNdM63SA0tH\n/w7vHP0QR0zHsO6YFb8bvhhKuVLqaG3YHDbsKz6MH3K2obShDAIEqOVqfJ/zM7bkbMPIgKFIDpuM\nQb6xnPSEiPqsvUUHYHXYkBQ6qVMz+rb6JjUHAK89IyLqCSzQXMRdM+JwLLscX/x0FiEBOkQFu883\nkhqFGo+Muh/vHvsYx8tP452j/8ADI++BWq6SOhosdiv2FO7Dj7k7UNlUBbkgx+TgCZgVMRU+ai8c\nLEnHzoI9SC87gfSyEwjUGpAUmoBJwfHwUHA9HyLqOxyiA7sK9kIhU2BS8LhOt5NVWIMT2RUYEuGL\n2FDvbkxIREQACzSX4e+twd3XD8L7357CS58fxmO3jcDQSD+pY10zlVyFB0feiw+Of4qjZSfw5pH3\n8fCo++Ch0EiSp97agK052/Bz7i+otZqhlCkxLew6zAhPbnNRfELIeEwKHofzNXnYWbAHh0rSsfHs\nZmw+91+MDxqLKWGTEerpPj2aRERXklF5DqUNZZgYFA9Ppa7T7bTO3HgTe8+IiHoECzQXMnl4MNRK\nOf6++QTWfpWOB24ahnGDjVLHumZKmQK/G74I/zj5OQ6VHsXrR9Zh6agl0Cp7b20cs7UO2/N2YWfB\nHtRZG6CRazA7YhqmD0iCXnX5WcYEQUCUdziivMNxW+yNSC3cj18K92J3YRp2F6YhxjsSyWGTMdow\nvEvrBRERSWlnQfPkIEldmBwkt6QWRzLLEBvmjUHhnZ8BkoiIroyfNl1M/CAjnviNAq99fQxv//s4\n7m4chCmjQ6WOdc3kMjnuG5YCpUyJtOKDePXwu1g6+ndXLI66S3VTDX7K3YlfCvfCYrdAr9Lhpujr\nkRw6GVrltQ9V1Ks8MTtyGmZGTMGJ8tPYkb8HpyoycK76PPQqTySGTMR1IRO7NDU1EVFvq2yswrGy\nkxjgGYJIrwGdbqf12rObJkfyel0ioh7CAs0FDYn0w4qFY/B/X6bjo+/PwNxgxbxJEW5zMpQJMiwa\n8hso5UrsKtiLtYf/jmWjfw9vdfdfV1feUIEfcncgtWg/bA4bvFVeuCn6etwycgZqKy2dblcmyDAi\nYChGBAxFab0JvxTsRWrRAXx//idsbZ1UJHQyBvrGuM2/Fyk4RAdOlp9BrcWMUYbhHSqWiaj77C7c\nB4foQFJYQqffswrL6nDwdCkig/QYHuU+Q/CJiNwNCzQXFRXshacXjcUrG47gnzuyUFtvxZ3TYyFz\nk2JAJshw18BboZQpsC1vF/7v0NuYGzkTWqUHPBQe8FBoWn48oJarOjybWHFdKbbmbMP+ksNwiA4E\naPwwO2IaJgTHQylTQKNQoxadL9AuZtQacHvcTbgp+nrsLzmMnfmpOGI6jiOm4wjUGjE5ZDzC9WEI\n1BrhpfJkwYbmqbxTi/ZjZ34qyhsrAAAbMv6FMcaRSAyZiBhvfvtO1FvsDjv2FKbBQ6HBuMAxnW7n\n29TzEMHeMyKinsYCzYUF++vwzKJ4vLLhCLbuz0NdgxX3zhsMucw9lq8TBAG3x94ElUyFLTk/4+NT\nGy6/HwRoFBpoFZqW35cWcReKOSUOlR5Duuk4RIgI0gXi+ohpiDeOglwm79H/Pyq5CokhEzE5eAKy\na3KxM38PDpcexb8yv3Xu46HwQJDWgECdEUFaIwK1BgTpjPDX+PV4PleQX1uIHfl7sL/kMKwOK5Qy\nJRJDJsBf44e9RQewr/gQ9hUfQqDWgMkhEzAxKL7Hh78S9XfpZSdQbanF1LDETs+uW1pZj70nSxBm\n0GFUXEA3JyQioouxQHNxfl4arPztWKz9Kh27jxejrtGGh24ZBpXSPT7sC4KAm2PmYLBfHEz1ZWiw\nN6LB2oB6WyMabI1osDW0/DTfL2+oQIG9qd12w/VhmBM5HSMChnZpLZ/OEAQB0d4RiPaOwO1xN+Fk\n+RkU15eipK4UxfUm5NTmI7smt81zFIIcBm0AArXGNgWcUWuARqHu1fzdzeaw41DpUWzP241z1dkA\nAH+NH5LDEjA5eLxzkpjZEdNwtioLuwvTcMR0HP/K/Babz32PkYZhSAyZgEG+sb3+75KoP/ilYC8A\nICl0Uqfb+DY1B6LYvO6Zu4zkICJyVyzQ3IBeq8KTd43Bm/86hiOZZfjbl+lYdvtIaDXu869voG8M\nBvrGXNO+DtGBRlvjFYu4YF2gyyworVd5YmJwfJvHbA4byhoqnEVbSb0JxXWlKKkvRVFdySVt+Kp9\nENhStIWWGSBYlPBSeULf+qP0dLmFvwGgxlKL3QX7sCc1DRUNVQCAIX4DMSVsMob5D76k2BIEwfk6\nqLPWY1/xIewp3IfDpUdxuPQo/DW+SAiegISQcfBRu+baSrUWMwrMRc6foroSeKn0iPIOR6RXOCK8\nBki2tATR5RTXlSKjMhMDfWIQpAvsVBvl1Y3Yc7wYQX5ajBvkPjMLExG5K/f5hN/PeagVePyOUVj3\nnxM4cMaElz47hCcWjIa3TvrFoLubTJBBq9T26vT83UkhUyBIZ0SQzggYLjwuiiKqLTUtxZoJJfWl\nztunK8/idOVZIP/ybWrkGnipPOGp8rzwW9laxOnbFHMeCk2PFq/na3KxPW8PDpemwyba4aHQYEpY\nIqaEJiBQd20f3nRKLaYNuA5TwxJxviYXuwv34WDJEXyTvQXfZm/F8IDBSAyZiKF+gyQZGmp32FFS\nb2pTjBWYC1FtqW2zn0KQI1fMx/HyUwCah+sG6YyI8gpHpHc4orwiEKQzsmeQJLOrtfcsrPNT6/83\nLQd2h4gbEiIgk0n/xRgRUV/HAs2NKBUyPHTLcKzfegbbjxTixfUHsXzBaBh8ODOeOxAEAT5qb/io\nvTHYL67NtgZbI0z1ZRA87Mg3laLWakat5aIfqxk1llqYqsshQrzq31HIFPBRecFX4wM/jW/LT/Nt\nX40P/NQ+He6RszpsOFSSjh35e5BTmwcACNQaMSVsMm4YPgXmKmvHDkaL5jXoIhDVMlz0QMkR7ClM\nw7GyUzhWdgreKi8kBI9DQsgEBHj0zKxxZmsdCs1FyDcXoaC2uRArqi+FzWFrs5+v2gfD/YcgzDMY\nofoQhHoGw+Dhj1pLHc7X5OJ8TS6yq3OQU5uPoroS7CnaDwDQyNWI8BrgLNoivcJd/ro7URRhE+1Q\nct0/t9Zkt2Bv8QF4qfQYFTCsU21UmZuwM70IAd4aTBrWuR44IiLqGJ593YxMJmDx9YPgqVXimz05\neKGlSAszuPYHPro6D4UG4V5hMBj0GKCsveJ+DtGBOmt9m8LtQiFX21LImVHVWIWzVVlXbEev8oSf\nurlwu1DIXSjotAoPCIKAysYq7CrYi12FaTBb6yBAwMiAYZgSNtk5zNRDqYEZnSvQfn0MkkInISl0\nEvJqC7CncB/2FR/G9zk/Y0vONgzyjUV84Gio5Uo4RBEO0QEHRIiiCFF0wAEHRFFs3ua83fIbLfuI\nDogQYXFYUXGqAtkVeahqqm6TQylTIEQXhFDPYIR6BjcXZJ7BV+zR9VbrMcowDKMMzR+A7Q47iupK\nkF2Ti/PVuciuycWZykycqcx0PidA4+fsYYvyDkeoZ7Cki6A7RAfyzYXIrMrGuapsZFZlo9HehOcm\nreCaf27sYMkRNNgaMTUysdM90d+n5cJmd+CGhAi3maCKiMjdsUBzQ4Ig4LbkGHhqlPji50z8df0h\n/D93jkJsqGtet0PdRybInMMZ22N12FDVWI2KxkpUNFU1/26sRGVj8+0Cc6GzN+zXVHIVfNXeMDWU\nwyE6oFV4YGb4FCSHJsC/h3qyLjZAH4oFg27FrbE34FDpUewu3HdhGGg38lF7Y5j/4DbFmMEjoEvD\nKuUyOcL0IQjThzgnZai31iOnJh/ZNTnOwu1AyREcKDkCoLnXM1QXjBDPIIToAhHsGYQQXXCPLdtg\nddiQW5OPzKosZFZlI6s6B432Rud2H7U3JhqGQafUdfvfpt4hiiJ2FqRCJsiQGDKxU23U1Fuw/UgB\nfPVqTB4e3M0JiYjoSligubHZE8Kh81Diw+9OY80Xh/HorSMwItpf6ljkIpQyBQxafxi0l39NOEQH\nai11qGyqREVjawFX5SziKhurEOoZjOTQBIwLHA1VJ6fn7gqVXIVJweMwKXgciutKkFHZ3CsoEwQI\nggAZZJAJspbbAoTW24Ks5X7zbQEtjwkCBAiQyxQYOiASjTVXHy7aXbRKLYb4D8QQ/4EAmj88lzaU\nOXvYzlfnIP8yBbNOqUWILqilcAvCUCEaHjY9PBQdG9bcZLcguzrHWZCdr8mF9aIhnEZtAMZ6j0Cs\nTzRifaLgp/F1iUl4qPNyavOQV1uAUQHDOt0L+sP+PFisDvxmagSUCvaeERH1FhZobi5xRDB0GiXe\n/vdxvLbxKJbcOASThgZJHYvcgEyQwVuth7daj0ivcKnjtCtIF9jpWeguR6/2RCOuPJy0JwmC0Dxz\np9bgnAXU7rCjtKEMheZiFNYVo8hcjIK6YmRWZV8Yrnqm+Zev2sdZtLX+DtQZndeM1Vvrca76PM62\nFGR5tQVwiI7mvw0BIZ5BiPWJQqxPNGK8o+Ct1vf6MaCe9Ut+1yYHqWu04qeD+fDSqZA0kr1nRES9\niQVaHzA6LgDLF4zGqxvTsW7zSdQ12DAjPkzqWETUAXKZHMG6QATrAhGPUc7Hm+wWFNeVoNBcjEpH\nBc6ZclFYV4wT5adxovy0cz+ZIGseninIUFRX4pxMRibIEKEPQ4xPFGJ9ohDjHem2M6TStTE31eFg\n6REYPPwxyDe2U238fKgAjRY7bk6Mcpt1N4mI+goWaH3EwAE++GPKWPzty3R8+kMGzA1W3JwYyWFK\nRG5OLVchwmsAIrwGwGDQw2Rq7vUzW+pQVNfcy1bU0utWaC6BXbQhzifaWZBFeUdALcHwVJLO9vOp\nsDpsSApN6PQSD8ezyiEIwJTRId2cjoiI2sMCrQ8JD9TjmUVjseaLI/j3rmyY661YOCsOMhZpRH2O\np0qHOFUM4i5aAF4URYgQue5aP+YQHdiauRNKmQKTgsd1qg2b3YGc4lqEGTzhoebHBCKi3sazeB9j\n9NXimcXxCDXo8NOhfLz3zUnY7A6pYxFRL2idFIX6rzOVmSg2mxBvHA1dJ4eyFpbVwWJzICrYq5vT\nERHRteCZvA/y8VRj5W/HIibUC3tPlOCNr4+hyWqXOhYREfWw3QVpAICksEmdbiOrsAYAEB3CAo2I\nSAos0PoonUaJJxeMwfAoPxw9V46/bTiC+sauLyZMRESuy1fjg8TwcYjQD+h0G1lFLQUae9CIiCTB\nAq0PU6vkWHbHSEwYYsTZ/Gqs/uwwqussUsciIqIecnvcTXg8YUmXJojKLqyBWilHSAAXKicikgIL\ntD5OIZfhgZuGYeroEOSVmvHi+oMoq2qQOhYREbmghiYbCsvqEBmkh0zGCaaIiKTAAq0fkMkELL5+\nEG6cHIHSyga8sP4gCkxmqWMREZGLOV9cCxG8/oyISEos0PoJQRBwW3IMFkyPRZXZgr9+egjnCqul\njkVERC4kq+W8wBkciYikwwKtn7l+QjjunzcE9U02rPn8CE5kV0gdiYiIXER2UfNC6OxBIyKSDgu0\nfui6kcF49NYRsDtErP0qHQdOl0odiYioT2hsbMTMmTPx9ddfo6ioCIsXL0ZKSgoef/xxWCyuP0lT\nVmE1vD1V8NWrpY5CRNRvsUDrp8YONOCJO0dBqZDh7X8fx44jBVJHIiJye2+//Ta8vb0BAK+99hpS\nUlLw2WefISIiAhs3bpQ43dVV1jahymxBdLBXl2aBJCKirmGB1o8NifDFipQx0GmU+Oj7M/hub47U\nkYiI3Na5c+eQmZmJqVOnAgDS0tIwY8YMAMC0adOQmpoqYbr2tV5/xuGNRETSYoHWz0UGeeHpRWPh\n56XGxu3n8OW2TIiiKHUsIiK3s3r1aqxcudJ5v6GhASqVCgDg7+8Pk8kkVbRrwgWqiYhcg0LqACS9\nYH8dnv5tPF7ZcATfp+WirsGKe+YM5ho4RETXaNOmTRg9ejQGDBhw2e3X+sWXr68WCoW8y3kMBn2H\nn5NvqocgAONGhECrUXY5Q2d0JrfU3DEz4J653TEz4J653TEz4L65f40FGgEA/L01WLloLP7vy3T8\ncrQI9Y02PHDzMCgV7GQlImrP9u3bkZeXh+3bt6O4uBgqlQparRaNjY3QaDQoKSmB0Whst53Kyvou\nZzEY9DCZajv0HIdDREZeJYL9dairbURdbWOXc3RUZ3JLzR0zA+6Z2x0zA+6Z2x0zA+6X+2rFJAs0\ncvLSqrBi4Ri8/s+jOJhhwtqv0rH0thFSxyIicnlr16513n799dcRGhqKw4cPY8uWLbjllluwdetW\nJCUlSZjw6grL69BksXN4IxGRC2D3CLXhoVbgiTtHYUxcAE7lVGLNF0dQWdP736QSEbm7xx57DJs2\nbUJKSgqqqqowjEfmKgAAIABJREFUf/58qSNdUXZh8/VnUZwghIhIcuxBo0soFXI8cutw/OO/p7H7\nWDEefXkbFs8eiHGD2x+eQ0TU3z322GPO2x9++KGESa4dJwghInId7EGjy5LLZLh/3hCkzIxDk9WO\ntzYdx7v/OYG6RqvU0YiIqJtlF9ZAqZAh1KCTOgoRUb/HAo2uSBAEzBw3AK/+YQqigr2w90QJ/vz+\nPpzIrpA6GhERdZMmqx35pjpEBOmhkPNjARGR1PhOTO0KM+rxzOKxuDUpCjV1Fryy4QjWbz2DJotd\n6mhERNRFOcW1cIgihzcSEbkIFmh0TeQyGW5KjMKf7h6H0AAdfj5UgFUf7kNmQbXU0YiIqAuyWiYI\nieYEIURELoEFGnVIRJAef753HOZMCIepsgEvrj+If+44B5vdIXU0IiLqhNYJQqLYg0ZE5BKuqUDL\nyMjAzJkzsX79+ku27dmzB3fccQcWLFiAN998s9sDkutRKuS4c3osVqSMgb+XBt+m5uAvHx1AfqlZ\n6mhERNRB2YU10GuVCPDWSB2FiIhwDQVafX09nn/+eSQkJFx2+1/+8he8/vrr+Pzzz7F7925kZmZ2\ne0hyTYPCffH/3T8ByaOCkVtqxv98tB//TcuBwyFKHY2IiK5BdZ0F5TWNiAr2giAIUschIiJcQ4Gm\nUqmwbt06GI2XroGVl5cHb29vBAcHQyaTYcqUKUhNTe2RoOSaPNQK3Dt3CJbdMRJajRJfbTuH1Z8d\nQmlVg9TRiIioHdm8/oyIyOW0u1C1QqGAQnH53UwmE/z8/Jz3/fz8kJeXd9X2fH21UCjkHYx5KYNB\n3+U2pNBXc88y6DFhRAje/udR7D5aiOc+2IclNw/H9ZMiJPtWtq8ea1fkjpkB98ztjpnJdWUVNU/0\nxBkciYhcR7sFWnerrKzvchsGgx4mU203pOld/SH3/XMHYViED9ZvzcCbG9Ox81A+7ps3GD6e6h5O\n2VZ/ONauwh0zA+6Z210zk+tq7UGLYg8aEZHL6NIsjkajEWVlZc77JSUllx0KSf2HIAiYNCwI/7Nk\nAoZF+uJYVjmefS8N+06VSB2NiIgu4hBFZBXVItBPC51GKXUcIiJq0aUCLSwsDGazGfn5+bDZbNi2\nbRsSExO7Kxu5MT8vDf6wYDQWzR4Iq82Bd/59Amu/SkdOsXt9+09E1FeVVNSjocmG6GD2chIRuZJ2\nhzgeP34cq1evRkFBARQKBbZs2YLp06cjLCwMs2bNwnPPPYfly5cDAObNm4eoqKgeD03uQRAETB8b\nhmGRfvjo+9M4eq4cR8+VY9wgA25JikZogE7qiERE/daFBaq9JU5CREQXa7dAGz58OD755JMrbh8/\nfjw2bNjQraGobwn00+KphWNwMqcSX+/IwoEzJhzMMGHS0CDckhQFo4+H1BGJiPqdbC5QTUTkknp9\nkhDqnwRBwLBIPwyN8MWRzDL8a2c2Uk8UY9+pEiSNDMaNkyPh58VFUomIektWYQ0UcgEDjJ5SRyEi\noouwQKNeJQgCxsQZMCo2AAdOl+Jfv2Rj+5FC7DpWjGljQnFDQgS8dCqpYxIR9WlWmx15pWaEB+qh\nVHTpcnQiIupmLNBIEjJBwIQhgYgfZMCe48XYvOs8fjiQh53phZg5LgxzJoZzVjEioh6SW2KG3SFy\ngWoiIhfEAo0kJZfJkDQyBJOGBmFneiG+2XMe36bm4OdDBZgzYQBmjhsADzVfpkRE3ck5QQivPyMi\ncjn85EsuQamQYUZ8GK4bGYxthwrw3d4c/OuXbPxwIB83JERg2phQqJRyqWMSEfUJrROEsAeNiMj1\ncOA5uRS1Uo45E8Ox+qEEzL8uCnaHAxt+zsTKv6di2+EC2OwOqSMSEbm9rMIa6DQKGH05iy4Rkath\ngUYuyUOtwM3XRWH1Q5Mxb1IE6pts+GTLGTzz7l7sOlrEQo2IqJPMDVaUVjUgKtgLgiBIHYeIiH6F\nBRq5NE8PJe6YGoPVDyZgZnwYqsxN+OC7U3jm3b3YfqQAVhsLNSKijmi9/ozrnxERuSYWaOQWvD3V\nSJk1EH99MAEzxoahymzBx9+fwcq/p+Kng/mwWO1SRyQicgu8/oyIyLWxQCO34uelwW9nD8RLDydg\n9vgBqGu04tMfMvDHd1KxZV8umiws1IiIrsbZg8YCjYjIJbFAI7fk46nGXTPi8NLDzdeoNVrt2PBz\nJp56ew++TT2P+kar1BGJiFyOKIrILqpBgLcGXlqV1HGIiOgyWKCRW/PSqnDH1Bi8/PBk3JwYCbtD\nxD93ZGHJX37A5l3ZLNSIiC5iqmqAucHK4Y1ERC6M66BRn+DpocT8pGjMHh+Onw7l48cDedi0Kxtb\n9udiRnwYZo8Ph6eHUuqYRESSyiriAtVERK6OPWjUp2g1Ctw0ORLv/2k2fjMtBkq5DN/sycFTb+3B\nl9syUV1nkToiEZFkWq8/iw7xljgJERFdCXvQqE/yUCswd2IEpo8Nw84jhfhvWg6+T8vFzwfzkTw6\nBHMnRsBXr5Y6JhFRr8ouqoFcJiA80FPqKEREdAUs0KhPUyvlmDV+AKaOCcGuo0X4dm8OfjyQj+2H\nCzBpaBBGxwVgSIQvPNT8T4GI+jab3YGcYjPCDJ5QKeVSxyEioivgp1LqF5QKOaaNDUPSqBDsOV6M\nb1PPY9exIuw6VgS5TEBcmDdGxPhjRLQ/QgN0EARB6shERN0q32SGze7g9PpERC6OBRr1Kwq5DMmj\nQnDdiGBkFdXg2LlyHMsqx+ncKpzOrcJX287Bz0uN4VHNxdrQSPauEVHf4Lz+jBOEEBG5NH7ypH5J\nJhMQG+qN2FBv3JocjZo6C45nl+NYVgWOZ5VjZ3ohdqYXXuhdi27pXTOwd42I3BMXqCYicg8s0IgA\neOlUmDw8GJOHB8PhaF7I9VhWc+/amdbete3n4KtXY0S0X0vvmh9714jIbWQX1UCjkiPYXyt1FCIi\nugp+uiT6FZlMQEyoN2JCvTE/qbl37UR2hbNg25lehJ3pzdeuxYZ6Y3RcAKaNCeVF90TksuobrSgq\nr8eQCF/IOAqAiMilsUAjaoeXToWE4UFIGB50Se9aRl4VzuRVYceRQtw/bwhiw7i2EBG5nuziWgBA\nNIc3EhG5PBZoRB1wSe9avQXf7DmPnw7k48X1BzFr/ADcmhwNNXvTiMiFOK8/4wQhREQuTyZ1ACJ3\n5qVVIWXmQPzxt2Nh9PXA1v15WPXBPmTkVUkdjYjIKbt1Bkf2oBERuTz2oBF1g4EDfPDc/ROw6Zcs\nbN2Xh9WfHsL0+DDcMSUGahV704j6uoaGBqxcuRLl5eVoamrCI488gsGDB2PFihWw2+0wGAx4+eWX\noVKpej2bKIrIKqqBr14NH091r/99IiLqGPagEXUTtVKOBdPj8PTieAT5a/HTwXz8+YM0nM6plDoa\nEfWwbdu2Yfjw4Vi/fj3Wrl2Lv/71r3jttdeQkpKCzz77DBEREdi4caMk2SpqmlBTZ2HvGRGRm2CB\nRtTNYkO98dx94zF3UjjKqhvx0ueH8cnWM2i02KSORkQ9ZN68efj9738PACgqKkJgYCDS0tIwY8YM\nAMC0adOQmpoqSbasIi5QTUTkTjjEkagHKBVy/GZqLOIHGvHBd6ew7VABjmaW4755gzE00k/qeETU\nQ+666y4UFxfjnXfewX333ecc0ujv7w+TySRJJl5/RkTkXligEfWg6BAvrLp3PDbvzsZ/9+ZizRdH\nMGV0CO6cFstFron6oC+++AKnTp3CU089BVEUnY9ffPtqfH21UCi6ft2qwaB33s4rq4NMAOKHh7j8\n+87Fud2FO2YG3DO3O2YG3DO3O2YG3Df3r7n2OzVRH6BUyHD7lBjEDzLgg29PYceRQhzLKse9cwZj\neLS/1PGIqBscP34c/v7+CA4OxpAhQ2C326HT6dDY2AiNRoOSkhIYjcZ226msrO9yFoNBD5Oped0z\nu8OBs3mVCAnQwVzTAHOXW+85F+d2F+6YGXDP3O6YGXDP3O6YGXC/3FcrJnkNGlEviQzywp/vHY+b\nEyNRbbbgb1+m44PvTqG+0Sp1NCLqogMHDuCDDz4AAJSVlaG+vh6TJ0/Gli1bAABbt25FUlJSr+cq\nMNXBYnVweCMRkRthgUbUixRyGeYnRePZe8Yh3OiJXUeL8Oz7+5CeWSZ1NCLqgrvuugsVFRVISUnB\nAw88gD//+c947LHHsGnTJqSkpKCqqgrz58/v9VytE4RwgWoiovZt3/7TNe336quvoLCwoMdycIgj\nkQTCA/X40z3j8N3eHPxn93m8uvEoEoYF4YbrouGlkcPTQyl1RCLqAI1Gg1deeeWSxz/88EMJ0lxw\nYYIQb0lzEBG5uqKiQvz44xZMnTqj3X0ff3x5j2ZhgUYkEYVchpsTozA2zoD3vzuF1BPFSD1RDADw\n99IgIkiPiEBPhAfqERGk5wKzRNRhWUU1UCllCAnQSh2FiMil/e1vq3Hq1AkkJY3H7NlzUVRUiLVr\n38KLL/4PTKZSNDQ04P77H0BiYhKWLn0Af/jDCmzb9hPq6szIzc1BQUE+li1bjoSExC5nYYFGJLEw\noyf+dHc80jPLUVLdiFPZ5cgtrsWhDBMOZVyYlttbp0JEkB7hgZ6ICNQjIlAPf28NBEGQMD0RuaqG\nJhsKTXWIG+ADuYxXNBCR+/jy50zsP13aoefI5QLs9ivPmDt+sBF3To+94vaFCxfj66+/RFRUDHJz\nz+Ott95DZWUFJkyYhLlzb0RBQT6efXYlEhPbXk9cWlqCNWtew969e/Dvf/+TBRpRXyGXyTB2oME5\nA5EoiqgyW5BTUovc4trm3yW1OHquHEfPlTufp9MomnvYAlsKtyA9An21kMlYtBH1dznFtRDBBaqJ\niDpqyJBhAAC93gunTp3A5s1fQxBkqKmpvmTfkSNHAwCMRiPM5u6ZK5cFGpELEgQBvno1fPVqjI4N\ncD5eW29BbokZuSXNRVtOcS1O5VTiVE6lcx+1Uo4wow4DjHoMMHpigNETYQYdNCr+507Un2QXcYFq\nInJPd06PvWpv1+V05zT7SmXzXAA//PA9ampq8Oab76Gmpga/+93iS/aVyy+sXXmta162h5/YiNyI\nXqvCsCg/DIvycz7W0GRDXqkZOS09bTkltThfVItzBTXOfQQARl8PZ8HWWrz5eak5RJKoj8oq5AyO\nRETXSiaTwW63t3msqqoKwcEhkMlk2LHjZ1itvbM0Egs0IjfnoVZg4AAfDBzg43zManOgsKwOeaXm\nlp9a5JWaceCMCQfOXLiuTadRNPewtRRu4UY9QgK0UCrkl/tTVyWKIuwOERarHU1WB6w2OyxWB5ps\ndijlMgT7d65dIuqcrKIaeOtU8PPiBENERO2JiIjCmTOnERwcAh+f5s9UU6dOx8qVf8DJk8dxww03\nw2g04sMP1/V4FhZoRH2QUiFrngUy6MIq9aIoorK2CbnOos2MvJJanMmtwuncKud+MkFAsL8WAwI9\n4alRwtJaaFntsNoczQVYy2+rzQGbXUSDxQar1QHHVbr2ZYKAQD8PhBqah1yGtfwO8PGAjL14RN2q\nsrYJlbVNGB0bwF5yIqJr4Ovri6+//rbNY8HBIfjooy+c92fPngsAuO++3wMAoqMvDMOMjo7FG2+8\n2y1ZWKAR9ROCIMDPSwM/L02b69qaLHbkmy4q2lp+CsrqrtiWSimDSiGHWimDp1YJL50SKqUcaoUM\nKqUcypbfaoUcKqUMDS1/o8BkRlF5PQ6cvtCWWilHSIAOoRcVbWEGT3jpVD15OIj6NF5/RkTkvlig\nEfVzapUcMaHeiAm9sJCtQxRhqmpAk8UOlVIOVWvBpZRBIZe1+Ua+IxfliqKIipom5JvMzQVbWR3y\nS+uQW1Lr/EDZykurbOltay7aQg2eCA3QQa3iMEmi9jivP2OBRkTkdligEdElZIKAQN/uX9hWEAT4\ne2vg763BqIt68Wx2B0oq6pFvqmvpaWv+/esZKuUyAcOi/DB+sBFj4gzQavgWRnQ5rV94RAWxQCMi\ncjf8dENEklPIZc09ZAZPTESg8/GGJltzL5vJjILSOpzNr3KuBaeQn8bwKH+MH2zE6LgAeKj5dkYE\nAHaHiOyiGgT7a/klBhGRG+I7NxG5LA+1ArGh3oi9aPhlSUU99p8uxf7TpTiSWYYjmWVQyGUYEd3c\nszYqlsUa9W8FpbVotNi5QDURkZvipxgiciuBflrcODkSN06ORFF5HQ60FGuHz5bh8NnmYm1kjH9L\nsebPBbqp38nIbR4WzOvPiIjcEz+5EJHbCvbX4abEKNyUGIXCsgvF2qEMEw5lmKBUyDAy2h/jhxgx\nKiaAE4xQv3CmZdkMzuBIRNT97rjjJnz88QZotd1/rX4rFmhE1CeEBOhw83VRuPm6KBSYzM5hkAcz\nTDiYYYJK0dKzNiQQI6P9pY5L1GMyciuhkMsQZvCUOgoREXUCCzQi6nNaJxy55booFJTVYf+p5mLt\nwBkTDpwxQSFvXr9NLjRPUKJQNC8foJTLoJALUChab7c8rhAuut36uABly5IDIgCIIlqX6b54vW4R\nIlr+17Ltwkbxosc9PZQIDdAhJEAHTw9lzx8k6pOarHacL6pBVLAeCrlM6jhERG7j/vt/ixdeeAVB\nQUEoLi7C008vh8FgRENDAxobG/HEE09h6NDhvZLlmgq0F154Aenp6RAEAc888wxGjhzp3DZ9+nQE\nBQVBLm8eOrRmzRoEBgZeqSkiol4jCELLOmqemJ8UhXxTHfafLsXxrHJY7Q40NtnRaLHD1mCF1e6A\nzSbCcXF1JRFvnap58e4AHUIMOmfhZuilv+8QRTgcIj/gu6Hcklo4HCKig73b35mIyEV9nfkNDpce\n69Bz5DIBdseVz+FjjCNwW+yNV9yenDwNu3fvxO2334lfftmB5ORpiImJQ3LyVBw8uB+ffvoR/vd/\nX+5Qps5qt0Dbt28fcnJysGHDBpw7dw7PPPMMNmzY0GafdevWQafT9VhIIqKuEgQBA4yeGGD0xG3J\n0VdcYNvhEJuLNbsDNpuj5bZ40e3Wx0XY7A5nb5kgAALQ8g9AgHDR7ebtrQ8IFzY5b1SbLSgoq0Nh\nWR0KTHWXrAEHAH5eagT5aZ3FW2iAJ0ICdNc0lbrd4UBtvRU1dRbU1FlQXWdx3q+us6Cm3uLcVltv\nhVIhwwsPTIKvXt2Bo0xSu7BAtV7iJERE7iU5eRreeGMtbr/9TuzatQNLlz6BL774BJ9//gmsVis0\nGk2vZWn3rJ6amoqZM2cCAGJiYlBdXQ2z2QxPT45tJ6K+RyYToJbJoVZKO6FIQ5MNReX1KCgzNxdt\nZXUormjAyfOVOHm+beHmq1cjJECHEH8dvHRK1NRZ2xRc1XUW1DVY0V7foEYlh5dOBYOvB4L9tNBx\nDS2307pAdXQIe9CIyH3dFnvjVXu7LudKX7xeq+joGJSXm1BSUoza2lr88st2BAQY8eyzz+P06ZN4\n4421nW67o9o9+5aVlWHYsGHO+35+fjCZTG0KtFWrVqGgoADx8fFYvnw5BEG4XFMAAF9fLRSKrn/w\nMRjc89tB5u497pgZcM/c7pgZcP3c4WG+lzxW32hFbkktcotrkdfyO7e4BieyK3Aiu+KS/XUeSvh4\nqhER7AUfvRq+nmr46Ft+nLc18PZUcUmCPqCqtgkB3hoYvHvvm14ior4iIeE6vPvuW0hKmoKqqkrE\nxMQBAHbs2AabzdZrOTp8NhZ/dX3GsmXLkJSUBG9vbzz66KPYsmUL5syZc8XnV1bWdzzlr3S1QpYK\nc/ced8wMuGdud8wMuGdug0GPutpG+GuV8I/2w5hoP+e2+kYbCsvrUN9ohV6rgrdOBb1WBaXiGq4j\ns9tRW92Anjgarl4E9zUP3DwM3j5aCA6H1FGIiNzOlCnT8NBD9+Mf//gcjY0N+MtfVmHbth9x++13\n4scft+Lbbzf3So52CzSj0YiysjLn/dLSUhgMFy5Vnz9/vvN2cnIyMjIyrlqgERFR99NqFIgN5bC2\n/s7PSwODv87tvnwgInIFQ4YMw44dac77n3660Xn7uuumAABuuOHmHs/R7leriYmJ2LJlCwDgxIkT\nMBqNzuGNtbW1WLJkCSwWCwBg//79iIuL68G4REREREREfVe7PWhjx47FsGHDcNddd0EQBKxatQpf\nf/019Ho9Zs2aheTkZCxYsABqtRpDhw5l7xkREREREVEnXdM1aE8++WSb+4MHD3bevueee3DPPfd0\nbyoiIiIiIqJ+iKuQEhERERERuQgWaERERERERC6CBRoREREREZGLYIFGRERERETkIligERERERER\nuQgWaERERERERC6CBRoREREREZGLEERRFKUOQUREREREROxBIyIiIiIichks0IiIiIiIiFwECzQi\nIiIiIiIXwQKNiIiIiIjIRbBAIyIiIiIichEs0IiIiIiIiFyEQuoA7XnhhReQnp4OQRDwzDPPYOTI\nkc5te/bswd/+9jfI5XIkJyfj0UcflTBpWy+99BIOHjwIm82GBx98ELNnz3Zumz59OoKCgiCXywEA\na9asQWBgoFRRAQBpaWl4/PHHERcXBwAYOHAgnn32Wed2Vz3WX331FTZv3uy8f/z4cRw+fNh5f9iw\nYRg7dqzz/j/+8Q/nce9tGRkZeOSRR3Dvvfdi0aJFKCoqwooVK2C322EwGPDyyy9DpVK1ec7VXv9S\n5n766adhs9mgUCjw8ssvw2AwOPdv77UkReaVK1fixIkT8PHxAQAsWbIEU6dObfMcVzzWy5YtQ2Vl\nJQCgqqoKo0ePxvPPP+/c/+uvv8arr76K8PBwAMDkyZPx8MMP93pukgbPj72D58fewXOkdJl5jnRB\nogtLS0sTH3jgAVEURTEzM1O8884722yfO3euWFhYKNrtdnHhwoXi2bNnpYh5idTUVPF3v/udKIqi\nWFFRIU6ZMqXN9mnTpolms1mCZFe2d+9e8bHHHrvidlc91hdLS0sTn3vuuTaPTZgwQaI0bdXV1YmL\nFi0S//SnP4mffPKJKIqiuHLlSvG7774TRVEUX3nlFfHTTz9t85z2Xv+94XK5V6xYIX777beiKIri\n+vXrxdWrV7d5TnuvpZ52ucx//OMfxZ9//vmKz3HVY32xlStXiunp6W0e++c//yn+9a9/7a2I5EJ4\nfuw9PD/2PJ4jew/Pke7BpYc4pqamYubMmQCAmJgYVFdXw2w2AwDy8vLg7e2N4OBgyGQyTJkyBamp\nqVLGdRo/fjxeffVVAICXlxcaGhpgt9slTtV5rnysL/bmm2/ikUcekTrGZalUKqxbtw5Go9H5WFpa\nGmbMmAEAmDZt2iXH9Gqv/95yudyrVq3C9ddfDwDw9fVFVVVVr2Zqz+Uyt8dVj3WrrKws1NbWSvKN\nJbkmnh9dgysf64u58vkR4DmyN/Ec6R5cukArKyuDr6+v876fnx9MJhMAwGQywc/P77LbpCaXy6HV\nagEAGzduRHJy8iXDBlatWoWFCxdizZo1EEVRipiXyMzMxEMPPYSFCxdi9+7dzsdd+Vi3Onr0KIKD\ng9sMIwAAi8WC5cuX46677sKHH34oUTpAoVBAo9G0eayhocE5XMPf3/+SY3q1139vuVxurVYLuVwO\nu92Ozz77DDfddNMlz7vSa6k3XC4zAKxfvx533303nnjiCVRUVLTZ5qrHutXHH3+MRYsWXXbbvn37\nsGTJEtxzzz04efJkT0YkF8LzY+/i+bFn8RzZe3iOdA8ufw3axVzljfpa/fjjj9i4cSM++OCDNo8v\nW7YMSUlJ8Pb2xqOPPootW7Zgzpw5EqVsFhkZiaVLl2Lu3LnIy8vD3Xffja1bt14y3ttVbdy4Ebfe\neuslj69YsQI333wzBEHAokWLMG7cOIwYMUKChFd3La9tV3r92+12rFixApMmTUJCQkKbba74Wrrl\nllvg4+ODIUOG4N1338Ubb7yBP//5z1fc35WOtcViwcGDB/Hcc89dsm3UqFHw8/PD1KlTcfjwYfzx\nj3/Ef/7zn94PSZJzpdfsteD5sfe4+/kR4Dmyp/Ec6XpcugfNaDSirKzMeb+0tNT5DdCvt5WUlHSo\nu7an/fLLL3jnnXewbt066PX6Ntvmz58Pf39/KBQKJCcnIyMjQ6KUFwQGBmLevHkQBAHh4eEICAhA\nSUkJANc/1kDzUIgxY8Zc8vjChQuh0+mg1WoxadIklzjWrbRaLRobGwFc/phe7fUvtaeffhoRERFY\nunTpJduu9lqSSkJCAoYMGQKgeRKCX78OXPlY79+//4rDNmJiYpwXco8ZMwYVFRVuPVyMrh3Pj72H\n50dp8BzZe3iOdD0uXaAlJiZiy5YtAIATJ07AaDTC09MTABAWFgaz2Yz8/HzYbDZs27YNiYmJUsZ1\nqq2txUsvvYS///3vzhlxLt62ZMkSWCwWAM0vrNaZfKS0efNmvP/++wCah2yUl5c7Z85y5WMNNL9x\n63S6S759ysrKwvLlyyGKImw2Gw4dOuQSx7rV5MmTna/vrVu3Iikpqc32q73+pbR582YolUosW7bs\nituv9FqSymOPPYa8vDwAzR9Wfv06cNVjDQDHjh3D4MGDL7tt3bp1+OabbwA0z27l5+cn6Sxs1Ht4\nfuw9PD9Kg+fI3sNzpOsRRFfqp7yMNWvW4MCBAxAEAatWrcLJkyeh1+sxa9Ys7N+/H2vWrAEAzJ49\nG0uWLJE4bbMNGzbg9ddfR1RUlPOxiRMnYtCgQZg1axY++ugjbNq0CWq1GkOHDsWzzz4LQRAkTAyY\nzWY8+eSTqKmpgdVqxdKlS1FeXu7yxxponjp47dq1eO+99wAA7777LsaPH48xY8bg5Zdfxt69eyGT\nyTB9+nTJplc9fvw4Vq9ejYKCAigUCgQGBmLNmjVYuXIlmpqaEBISghdffBFKpRJPPPEEXnzxRWg0\nmkte/1d6E+rN3OXl5VCr1c4355iYGDz33HPO3Dab7ZLX0pQpUyTNvGjRIrz77rvw8PCAVqvFiy++\nCH9/f5fcY4hAAAAAiElEQVQ/1q+//jpef/11xMfHY968ec59H374Ybz99tsoLi7GU0895fyQJdXU\nxyQNnh///3bt4AQAAARi2P5bu4UWTCbwdxTcYR937rSRdzfbyJ58oAEAAHyRfnEEAAD4RKABAABE\nCDQAAIAIgQYAABAh0AAAACIEGgAAQIRAAwAAiBBoAAAAEQMc8cPDWyTNBwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f1e1d485b00>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "GPQH0NVwQAO3",
        "colab_type": "code",
        "outputId": "2c969a71-a95a-4793-b4cf-7ccec9866643",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Test performance\n",
        "trainer.run_test_loop()\n",
        "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
        "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test loss: 1.24\n",
            "Test Accuracy: 70.8%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "9JhAPHwSBZPY",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "w6WRq-O3d1ba",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 待完善 TODO"
      ]
    },
    {
      "metadata": {
        "id": "oEcbaRswd1d0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* 图像分类例子\n",
        "* 切片\n",
        "* 深度 CNN 架构\n",
        "* 小的 3X3 滤波器\n",
        "* 填充和步幅的细节（控制接收域，使每个像素作为滤波器中心）\n",
        "* 嵌套网络（1x1 卷积）\n",
        "* 残差连接/残差块\n",
        "* 可解释性（n-grams 火）\n",
        "* image classification example\n",
        "* segmentation\n",
        "* deep CNN architectures\n",
        "* small 3X3 filters\n",
        "* details on padding and stride (control receptive field, make every pixel the center of the filter, etc.)\n",
        "* network-in-network (1x1 conv)\n",
        "* residual connections / residual block\n",
        "* interpretability (which n-grams fire)"
      ]
    }
  ]
}