4_cifar10_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Training a Classifier\n",
    "=====================\n",
    "\n",
    "This is it. You have seen how to define neural networks, compute loss and make\n",
    "updates to the weights of the network.\n",
    "\n",
    "Now you might be thinking,\n",
    "\n",
    "What about data?\n",
    "----------------\n",
    "\n",
    "Generally, when you have to deal with image, text, audio or video data,\n",
    "you can use standard python packages that load data into a numpy array.\n",
    "Then you can convert this array into a ``torch.*Tensor``.\n",
    "\n",
    "-  For images, packages such as Pillow, OpenCV are useful\n",
    "-  For audio, packages such as scipy and librosa\n",
    "-  For text, either raw Python or Cython based loading, or NLTK and\n",
    "   SpaCy are useful\n",
    "\n",
    "Specifically for vision, we have created a package called\n",
    "``torchvision``, that has data loaders for common datasets such as\n",
    "Imagenet, CIFAR10, MNIST, etc. and data transformers for images, viz.,\n",
    "``torchvision.datasets`` and ``torch.utils.data.DataLoader``.\n",
    "\n",
    "This provides a huge convenience and avoids writing boilerplate code.\n",
    "\n",
    "For this tutorial, we will use the CIFAR10 dataset.\n",
    "It has the classes: ‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’,\n",
    "‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’. The images in CIFAR-10 are of\n",
    "size 3x32x32, i.e. 3-channel color images of 32x32 pixels in size.\n",
    "\n",
    "   <img src=\"cifar10.png\">\n",
    "\n",
    "   cifar10\n",
    "\n",
    "\n",
    "Training an image classifier\n",
    "----------------------------\n",
    "\n",
    "We will do the following steps in order:\n",
    "\n",
    "1. Load and normalizing the CIFAR10 training and test datasets using\n",
    "   ``torchvision``\n",
    "2. Define a Convolutional Neural Network\n",
    "3. Define a loss function\n",
    "4. Train the network on the training data\n",
    "5. Test the network on the test data\n",
    "\n",
    "1. Loading and normalizing CIFAR10\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "\n",
    "Using ``torchvision``, it’s extremely easy to load CIFAR10.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output of torchvision datasets are PILImage images of range [0, 1].\n",
    "We transform them to Tensors of normalized range [-1, 1].\n",
    "<div class=\"alert alert-info\"><h4>Note</h4><p>If running on Windows and you get a BrokenPipeError, try setting\n",
    "    the num_worker of torch.utils.data.DataLoader() to 0.</p></div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100.0%"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./data/cifar-10-python.tar.gz to ./data\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n",
    "                                        download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n",
    "                                          shuffle=True, num_workers=2)\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
    "                                       download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n",
    "                                         shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us show some of the training images, for fun.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB5CAYAAAAgYXpDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO19aYwl13Xed9++9N493bOvpEgOKYqkaVGKIoehLEi2DClAZEdeFUQIEUBB7MBALMcILAX5YSOBnQSwHchLTCWGZVm2tcXWalKyJIqbRHGZITnD2Wd6pqf37re/qpsf55w6p7ure3oWT8+z7gcMuuZWvVu3bt2qOud8Z3HeewQEBAQE9B4yWz2AgICAgIBrQ3iBBwQEBPQowgs8ICAgoEcRXuABAQEBPYrwAg8ICAjoUYQXeEBAQECP4rpe4M65dzvnXnXOHXfOfeRGDSogICAg4Mpw1+oH7pzLAngNwDsBnAPwDICf9t4fuXHDCwgICAhYD7nr+O2bARz33p8AAOfcJwG8D8C6L/BKpeKHhoau45QBAQEBP3iYnJyc9t5vW91+PS/wXQDOmv+fA/DQRj8YGhrCo48+eh2nDAgICPjBw8c+9rHTae1/7ySmc+5R59yzzrln6/X63/fpAgICAn5gcD0v8PMA9pj/7+a2FfDef9x7/6D3/sFKpXIdpwsICAgIsLieF/gzAG53zh1wzhUAfADA527MsAICAgICroRrtoF777vOuX8L4EsAsgD+yHv/8tX28+ef+TQAIFssaGPG0eCcfl+6zRYAIOrGdH6zL5uhbeectmWz1FVGj/O8O8fnanc6yb5Wi/pvNBprxlg0Y5Nz5Ut52tenWoXn83dbbe13qQYAWJ5f1mtpRzSOfJ6vqav7eDvD4wcAH9HxUbuVtH34wx9eMcaPfvSja/qw82G3rwbWS2mjPuI43lQfsh3xNQHA/Pw8AODpp58GANxxxx3Jvl27dgFYeR8LhcKaNkGtRvP9xBNPJG3PPfccAODSpUtJ29ve9jYAwOuvv76mj3d95D8BANpdvd4opnO1dNhYiOlaOinXnsyVcfKKYmqLY22U7djzurZOYRlaA5FpjKVbe29zvO5Ag8tndF+ZpygfaR/VLJ1rR0WPGyrQq0BWc8GMo8r9lcw4Mnx+uyT+s1mDAPDCt76UbMsKr7d0rRdL9OMop5NaqpT5+miMswtL2kck86wnvWPvAQDAvh37krZzF+g+f//oUQBAs6nPjTy3za4++1GHxlTJF/W4Io2jxH/7S/ZZ4vve0XHv37cTAHDvnbclbYND/QCAM+cvAgAuLei1LxX2AwCGcvquqM3TcTvHtmOzuB4SE977vwbw19fTR0BAQEDAteG6XuA3AvVmEwBQyqk0RS7mK6U6kdi6/LXMZIyEypKYlfRESnPma9311EecpbZsXi8/btE4coW8GQcdZ6Udz8PM5Oi3+WIp2dfq0Jd+ebmWtNUWF6nNShIshQwODtI4jLRdr9f42tdKO4iN+LcBMikayY2QwDfav9n+0/rrGE0IUG0IAIpFkopyOb1XaeeS9bHI822l+FdffRUAsLCwkLTNzMysO8YyS7TFrK5JEWDLZvhFFoc7Xb5/ZliZtQI4Ovyf2Js12WUJPFp7PFjLtBJ4xNJf1z4bPB8ya4Ws9l/i7bKRIMd5iR/SZYci73a87nJmjnM8qhysJkXnz2D9+97u6nqN+TgzNBSKNIBarFLowjLdP5el5zcyVt4YdM5WXY+fmaXjz55+Omk7f3EaANBo0boaZUkYAEqsTcfmWSr29QEAykbTHt62AwAwUKHflrK6Rqdl7UT6nBdEejfX12UJvbNEx3cWdf6Ku6t0nrH9SZtzw7w1h80ihNIHBAQE9CjCCzwgICCgR7HlJpRLl6cAAHsGDiRton6KeQUAsqx9ZHKkduUyOnRRy7uGDBQ1vOm1j5qYJ1gVrFSq5pxCMKlqKqaIjFGlmSdCxxPZuWAIy2TgRnVs1+m4uKNjE3VfVPrhoeFk38jIKABgaUlNLnk2H+Qy126m2My+NNOEPV7mxrZlUsxXq/tbbSIBgHxeTVViHuljVbZara453mIjwnR8fBwAcPq0xj3cfffdAICHH344aXv++ecBKIFqkfN0ryJYsw1dpzUB5MVMIssj5fbYWZGVYofvc7LuaF2vmMcUwjLixmZHO2lzh1k2zWS97stxd1mva7jI2xVjkXPJ0qUftLzu9GJq8WYN8znFVJmKvO4bLLOpMattOTahuJaas+b5Ga03abYsUZhYTSOdj9lpMjcsLWuMSZ7NGS1+zoplJSf7ykRK5s048nnquM+4OY9vo3UU8fn7S7peF2bonFln3hVZWiuLdV3r9QY9390mzWnRrPl6RO8ne7srsu6bwYQSEBAQ8A8eWy6Bjw5TbpRCThkVx5KmM6SCExeiNpMn5sslLoj5kn5pRWpuGxei1uwsAJWKBgcHkn3FKv3WuhZ2WcppNrWtw2Rnk90NrdRf4a97mf8CQJndkKwk0eURFHi8EXTfYB+Nqa+q0oCcCxtInhYucfHamMTcTCIz+7s0t72E6OW+0lwGbZtI27YvkagnJycBqBR9NWNbfc3WHfTgwYMAVt4r68a4GgkxZyQsJcPNfHg59/pjtHcsITazG8y7X7vtrQIoZKqZP5HsC125F0YC55MW82au+Kf6ZADsN4CINYLIEq0sjRc63hzP0rNbXwbMmnW9c5A154I+o7kM3Y8FZ9zrFuhqRsr0XnBlfS90mcBdiPXe9veTI8Do8ETSVs7we4Of35GBvmTfiVlaY/m8Oh9sH6VzbRsbTNrETfkcE5aDJdWSc1m6hoy5FllOs/NKbLouvSsi1iaWjNbUYHfX2enJpC3TmudzGXb5CggSeEBAQECPIrzAAwICAnoUW25CaTD5kMkrmdQ/RKqMVbO7bVKzJCLTxUYl7LISmVfVI8uEQc74WAtJtsxJtSJjh/HsCJstKtEQN0jlya3wRaXfZJi89B0lMdsxqUyFnCFImEAZMH7MXY4Cy3K/sVHn52ZJZfORqluyv2hUtmuFNWeIGSHNbzwNm/H1TjNrbEh0QU0bly9fBrCSwN3IzGPJzNVj27t3b7LdZDL8yBHNdDzL5rR0pJiK+L4by0Jiatl4Voz5CJvw47fz54Ug1t3Sg7E4IscjUCLeRB8z8ZgxcQUtnraLdmplv7i0G7ZWojKrGUM883NlzTCrsTSnfvdTFfptpV8HvrxADgwLy2oSKQwdAgAM7mA//qwS2jk2i947ZMaWo4vpz6rZMr5A/XbZD7xY0X1n5okg7ET63Jb4XeHNvaqx88Eix280TMQ1cnS8g/a7XOd4lrKaa+IMvccWa3TOhbZZr/10X04ffS5pOzTOZprSmqyx6yJI4AEBAQE9ii2XwLuc2ySbMYSlo2G1bU6RBm/z8TZPSqdN+9pNlXASCXwFycKEDkf3WcIow8fFxn1K3PfyFSNRs+QRSW6WthKcIiEXDZkKPt6So23+Wnf5tzYXSswuiLGNYuOoMZsfZSNsNipS0vuK295m855cLVbko0mRqNttcRlb624oWoKV4lcTp2nH28yX0v/ysuajEffBtAyZMYuhViJLcpDY44TrxNp9aXDJERu4cnq3ZntldObKcwIqtMs0G+U0ifqMDZ0qkZ3z1mMx2c3Po7n/JX4eIxP93E3cYtdfJ4PblBScXCSJthDpK+cyE5aNZV0fe7dTBGQ7T+60cVvdgIdYO+7vVwIy66mPipF8MxwdPc/avXH0xb7d+wEAc01dC/kCnb9TM5pAlvrI5SkS07oHdkRzNRqgrDGJ/gSAHO+v1+hd0anuTPZ1p7mUwszJpM0PyjwECTwgICDgHzzCCzwgICCgR7HlJpRqldSsXE7NDmJC8bFVqalNgiItOenF7GH8XxO13aiJkmpU/GpLsZ4zSedpyJ5ENTZtQkAmfsSpKTaNyUD6suYaSWrEphMbpZmQlyvC9fyacdwICLknJqUrkY1puNpkVgJROQH12ZbzDwyof774jVsf7iRCNsU0k+bfLcnCbNKwjYqLyCzHNgIy5fIiOf0Gt2XFz67y9iW/tWsns9aGIpvCO9oZ8MnxlointrYhOyWXnOPnwFnzAM9D244/K8/o+hf14+/9F8n2sRMUGfvy0ZeStlaNztFX1vs9mKc1OZwlkvnQYU3POjQ2AgDYPqEmBomszJmozyxvt9j/enFeCevbDhM5eu78uaTtwonXAADdmkZASupf8Xc/efZiss+x2Wb/mBKsVY7qnp5fTNpOnToDQKXkTFPNPNkukaPxspqIppj0HdiFTSNI4AEBAQE9iitK4M65PwLwEwCmvPf3cNsIgD8DsB/AKQA/5b3ffAC/QZclsZaJnBMep9nUNiH+8gljo1/cmKUMm9+gn10GbdBbY5G+em2WeK1UPD5EX/XZRb2MWoPOmXHGPRFyLpYMjRTYYem8XlOCxGXXTnGW21xCnBp3Rvmd6TfDE7IiJ8sNgEjNm8lnYpGWHyW7SYJVYAlLIVMlvW6ppCSVjC0t/0la1KeM10rsqyV8QKV8ST+7ol/J3msvXXLxrDg/+JxrujBSs45RIhO9kZHlqrzn4h62iAlLow7qvpfhRzaTEhEqUxTZ6FlWN11W589zeYV8bNwC+cqynGPIpUjW3uTiSUorbKB5VQ89kGy/+a43AwA6RdV8BoaomMY7fvQdSdv4DiIxy1UiMceNtD13mVxsT585kbR95yT1Mb2gEZD7DtzB/ZPEfvmyFvIolumaLxgXx7OcT2W4z0RW8t+J7RQVPM0urgDQrDMpiaGkbSRPM2KnLXK0jr2s9ZrmfClVad+cCbNddrruN4vNvBH+GMC7V7V9BMDXvPe3A/ga/z8gICAg4CbiihK49/4bzrn9q5rfB+Bh3n4MwBMAfuVaBtBs0JezaFz1ooikuW5X7aQ+JqnBsxO9N9KoFGGwEqpk+BupajL3mclL/FvJ2qbSQ7VINqz8qH6FTzfI1SdrgnC8uDGm5AURV8GOkf5USjT2VO5DpNCVwuVaicalnOtaYaVWkUytzVmQTXJdpIznCvlRNnN+24eVllePMc1lMK1NJPS08Uj/VrK3BSLWg1sRhMPjNcEYLV6ThYLkd9FrShwGLaeSWbsWRNz3zPtYqb/N6qNdpwU+IGOmrOBZwstwAQFnJGtJJWjs3R2QxpPvWpdZLoDC43Arxrjq7yZxwUi+QyP0PG7fowFWA47GUV9U+/KTJ08BAPqHKLfJk099Pdk3zwFezmT1q1RJ0779rjclbTG7EdbZnj+yS43KFQ7qOXDHYe33TfcBAF598fmkbfr8BQDA5CWSvEsVXTuVMs9RRd8t9Yjt7qZU2/goPVdS1q4bm3JybKfPxtpvp61axGZxrW+GCe+9ZGG5CGBio4MDAgICAm48rlu08yQGrfttds496px71jn3rNg6AwICAgKuH9fqRnjJObfDez/pnNsBYGq9A733HwfwcQDYuXPnmhe9qMOWBCuzqmtdBaM8qSaawtNWpV9bQ7PFH4uhHao+VdnlaGaRyCFrGpHUstPzhotNKfLgU4oaCNJSpYqZot008WCxdE/9p5ljbB9iIvJXxxOmwpoupOiFuPSlFVK4UnTmanLxSmRmmvlDrlVIzCtFbqZBjktzMUwrLLFRUQi5SksUSj1Ie3meb6QTN1ZjphATRGSY8pYQkIY8zPAwpAp73ml2kWqT7ofrGlOOpFLtau6gxhK56HlP6v7g2CG9Ts/rv6hRkR3P5poVrqo87rSw0msMyr189qx20eKoyGUlDy+zY8Irz72ctM3OkwPA6OgYAODlF19I9h1+6K0AgLve9OakLc8m1WrZFGdJ3DvFRGRNbdTWNFHe4xyduXPPvqTtyHPP0Pmf/z4AoBtpimOuQ7EibbSXYhpVNUv1D9B6892VzxkAFLgeb9/waNLWbN48E8rnAHyQtz8I4LPX2E9AQEBAwDViM26EfwoiLMecc+cA/DqA3wDwKefchwCcBvBT1zoAkY5sia08G/gtuZVICylVs2PJKWIkiqUOfelnp9V154F7iaz47vPfBQDsm9iR7KtwkvZLp88nbXUueVYwuU18k8Yk0pxNni8uWFaSletacvp1FWIrz5J11rgaJlKulTw5yqLr1pcaLTYrtSY5YRhWepa5t1JaGvHnVmkpVyIbBWkFHWROLdm4mXNbSF923LJtc6FsZo4y9lp4PnKmrFjMJbOKUpovZ93Q1spGsQSomUgbcXPNM7GY9Rp0Um7S+ohbunaiDhPxsQaiLFyg4JjWIu3bflAJvSXQmu/f/ZakTUqOeVNpfY2UfQNS4dw9rs9XpkAd/s1LLyZtJS60IGXcAKBvgDSFIhORlX4tpFAukdueN/lUhPRtGuk2F0vmSHahNJpXkvNoRZwcHT88pG6Bb/qhH6bjWPs58pJqCXkur1YyErhI+SscLyK+f0JU5s0+KZJhnv1cyfa3OWzGC+Wn19n1jnXaAwICAgJuAkIkZkBAQECPYstzocSc+6NlSAXHOUtqxmtFzCRFNlN48+0Rt1chKAAgz+aBxQUle0pMdt59O0VqjY6oenbxdYruimqqrhZYzSoa59y2RGImuRd0X54j/Sr9mvMgxwUfvCkU4XkcCUlr/IfF53tF2gkuVJF1mzONJOdJMRNY04KYKsTMs1nTi8VqM8ZGRRbWG9vUFHHgL7xAhNUDD2gEn/RnTSgbjVNMP7YmphSIsG2WUFo7uORESVM2JiJqfkqJOQn8lb+2CELM0ZPORNpV2E879romM20ymQg5GTe1mMVCnealWbfRorzfTyctnYj6GKrQuvPL6lddmaBr6OTMnPGyyxn1XSyBSXQpUnCVZpU9OzR96qlpIlrPntIoyjL7cFf6THpYjtSUKu87Dqq/dqFCkZXWm63MkZXWB99x5HSTI6mX55U4jdickjFrc2CQ/Lm7dX32D3BBkAcffBAAMD42luyTnEpTl9TPfWmJ7pE3UbNiH8tzPVKX033dxARryPyUqO0rIUjgAQEBAT2KLZfAIyYc4rZhdvK8bSq5JwUOxO0QJhthVvI9aBetDkk0tWUlnaYi2i73EVkxYLIAbhslaXzOm6rSnA0xkzFEA7sLlTg9XdGUchpkAsYMDVPz9JXOmhJOlTJJGQWWwCumVNogSwOWOBWPyW0DSrJsBmnlza5Wyk6LbEzrQyTfNLIx7XjJhAioZHzhAkW/nTunBN3OnTvX7ddenxCwMg4rpUnxBnv8RvMgu3JGD1q4THFrC5MqQYKrqRcHSDIc2nlAd7G0mDGl8QpLtBbml55M2uZmqMxba47ud9Q21c9H6LdjY0oGlgvkzpbLHkza8lXSDgo1kgLnZ5UozIwzgWcEwwKPOxenaDUphSuSY1JcLzfSsr76d9/Qcw6QmvLQQ0qmHn3tFQDAhcvqaDC+jeZhaJCOL5tox0aNJOn6ohK9PqLr66uq1ju2jeZo2wjdg4GyPkvFfiKcy4YoHx6h52poeETbmJgucnnHXeNGAuf1Om0cJC5dJK3nzBnV0ObmuDwiWwas67PkSFrpl3D1GnCQwAMCAgJ6FOEFHhAQENCj2HITSo6rwWeKJtkTjyou6vAyQlaIr60lYJgksBxAISXBVeTIdDE0TgTFru3bk33FfjpuZlDtH0t1UoHaOWOK4H6LjsYxWlC1a9cg9Sc+5QBwW5trZ3aUQCvykOaYBFk2ROsA35KK8THNcLKkg/v3J20L55SYWQ9pJgNL3omZQchM6xe+kcllo1qUaX7gtsiCmEKsT/b4OKm8Bw4cWDNG8Q1PrV1pVHox9Ygpxe4TE40950YRozJtOZNEqo/rJg6Ma+KvTofMbQ1O8A9zj4VwjiJjKrpEUX2XL34naZuaI3JveJDIsu1735jsK+6k9VStaKqhDET11/HHXHMxXzkGACg0VY2PynR8y6yFghQI8VZ/T4k/WIWrLdpx4tgryXa3n+bttgMa7Tg0TKaQUyd1vNNcJWOCn9G8YU6rfI93muRUI8Nk/hjfpiaObUw4ipNCqazmEiHsi0VTACJlLcg0pJmNxEddzg0A+/fuoes7pFGwrx8/DgB49VWah2WTZlocGFbUKo02F+dhESTwgICAgB7Flkvg+w8z8WNIu05CnJkvPhN9GZaAmyYpfsRRTbYAvZcUnyYNZLdB/c02iOzZ2VDiQ6TAtxkXtvFlkpDnWqbYBBOa/XkiNyZK+uUfzhLhUoz0i95ukJTbmFVXsIglpuEhOv/SkEpprWH67WRGSbgWOJfCzIWkbRfW5i3ZDKwr3SXWACRytK9P50Mk2M0Tf+yuZqPe+D6mtVkJWXKg7GcNw0aybpTO1kbqimQl+2w0p2gWdvxyzlptbf6JbofG1mzrXE30kUZUKKhmVCiQVlD3Em2rj1OL12e3q/d97uL3AAB9eZWoR+/6R/R37w8BAOKc5txoepqHullP4klq70STVc8cE2+5gome5Xnpdk1uHd6OnHEccFLUQyRxm9dFDrlKAtzkdekvUy9HXns9acvmaP5+9N3vS9qiFrlJ3nknuQ8eYJdfAKhyaujhIc3rUuX7UjTFOvpY4s6zlLtoCO0kB4otSZcUiTGkuBZCoz8rtBXRNrUly44Ju3erpjgxQfd5127SGL71rW8l+y5PX159Sq0XeRUIEnhAQEBAjyK8wAMCAgJ6FFtuQtl2+24AQM3oI4vsn+2NnuPZhOLY1FIwvEPMVTCKplHSwzqjlhQapPosH6Fq0a++oomrBjOksu2pKgnxwCCpPnXjL95tk0payZK5ob2g5o/mAkXOdXJ6zjYTWw1DYMxMUxSduDYXR0wSG/YvH51QVWyW5+binEbpXasJZWFByc/vfY9UeqnIM2SS+axOzwqkJ6ySbdlnCcuNqulY4kjMH0IyTk9rlOFGSbLSzDvy19a/HGF/4Lk5TRUsRKn4nls0mqT6F9t6bzsF2i47UwpHrlXOnVFTUZfXpMvq8Z2YSPGBAa1QOLj9nQCAeTZZ1Dq6ToqQ/tZWHnJZJXqbjo7LdogMb5oKNxU2HxSMrl5gk0yzqGOTJE8akqmXKbUw7b2Ved6IDD519lSyfYCjk8cGdV23WtTf6Wmd523j9D4osP/3mDGXTDCpu9I8xrVEu4a0blF/YorN5nSMOTZ3rSh3mhCWZo3J34xUetJfdLwkodN+ZX5XRDpzyth7D9O7pWrG/ZnPfwEAsLh4ZWeEjRAk8ICAgIAexZZL4G1J4t+voWL97FozbKrMd7P8pU3KzKu0M8Bf2DETjTVSIakyl1FydFuRizaAjnv6y08k+147Q5F2/SPqWjg6RNNTn1HJrdOlMTXK/GU21xJxKkkpMAEAfX0kcXhTFfz8JZKKckzCubqSPajSb/vzemuiAl1rzboZqdCyBhu5e6W5123UR1paVktKihQikZU2LbD0cZzdqQBgF7uAWSlKxvHd71Ka3zNnziT7Hn744TVjS5P+VhdosC6RIoHPzmoE3/nz57Ee6m26P82uRtpVinSPcnk9Zz+HN2YiyVWj96wb0fnrXdWUChOk3dX7NLpwgTWBDkgLi4wvbCND+womhW2OI5IzdZXAuw3SzFosjdbcncm+pRPkondp6f8lbWdPk9ax3FD31ckZ0noyTa7TCkMQj9H8feAX/k3SNshRwStdEVdC6kkCAEpEXt77xnuSpmofzVt7UbWOl4/TdZ2apLFNzusa/eE3UvTpwf1aV7N/kMaRMWHYHV4LDc6fFJvnRtakuBnTNtcDNWpHm2ufzizTfV9u6HzPL9B4Oy0dd4mLlLaMC6y4TMb8nPf1633fv5f2Pf+iiZrNrK/NrIcggQcEBAT0KLZcAu/fTlKxH1Vp27OdsWS+qpKcvcBS7qjT47dxovfdg5ozYqRIpYoO7dFsZo1lkqxeY6+i0y8cTfad5irU49s0g1rEX+GM8U/ssp2sVSOpp2CyDHZYK+jYTHccZDTQr7a8oSoFGCzMkdTTNiWzqoOkHUQ2boXdwvL1q8+VsBpjJqvaG99IQSOSb8RKtGkug2IvtpK7/PaLX/wiAOCuu+5K9t15J0mCn/rUp5K2Rx55BADw9re/PWmT/k6fpqAWyR4IqLRv7a8yTjtGKQ8nGoDNnSJt1j3R2shXo8kZJN2ySqiSPS7KGTsw8zGO83E48zhlc9yHWQrDBXIraxup1bOUn4k5QMiIVFGK2564ERbzqmG02c11ao4W9mc++1Sy73vf/jYA4NwltbUuNXhdd/U+RpwnyHWYt8joGN/wT+hevfdn/nXSluf5sONYjbZx85xjXsMGUxVYCxs09Tt2jNHCPzFN7p1Pv3gs2Xd+lsb70KyujzsOkDQ+MqzPV35VacOcsYFLqbu0tbNo+KFTrKUcee0k/d9ohXPsAjg/rxrG4iKtFfvs/8LP/xwA4PA99JxZjmliB62F3BF9B3XMfG0WV5TAnXN7nHOPO+eOOOdeds79IrePOOe+4pw7xn+Hr9RXQEBAQMCNw2ZMKF0Av+y9PwzgLQA+7Jw7DOAjAL7mvb8dwNf4/wEBAQEBNwmbKak2CWCSt5ecc0cB7ALwPlCtTAB4DMATAH7lagcwto1UpkafUWlYpa4YVbOaJ5JngM0q1Zph8biSfGNe1eLMdiJehnOau2KaawYuxKSC7T+8P9l39iypTMdPndJ+mUQdG9fouFqTxpbkFvFKQC7wOLqRiTxkHmz3LjXNjI6TsnJ5msYzNqL7hrZvo/FkVD1reOovn7s6ksOqiWKKsGaS4WEah5gb0tzyrCniS1/60ppz/NzPkZr45S9/eU3/99xDhNXLL2s9wYMHiYgSUwqg1b0PcR4JyYkCqHpr08MKAWrHK9cn0ZnWRJKW6tZGna5Gm9N/llpqLuljdXw4p/1GkvY4z3KQcUNrtkkdt7kuurIsjHmixEUeomxjTR/xMvVb7+pan2M3w6W2RpCePEdq/lN/91UAwN994fM6jmk2WThrRpCiKKYmJk+NWIhi4377pgeoPqQrqrvrYpNMBRW//po8cIeSqVOTRBpL2lUAGN/OEanGRFkt0/2+Y5zOdeKCFk04+go9t69f1rVw3xtoHh6+R/Oj7NpJZsJ+jtzMGqfBiNM6t9t67c9zvVJ5PFcAABzwSURBVMu/+Wtd3ydPkTlvZorOnzPk/PbdlPfk0nk1q9SWl/hS9FzT0+RWLEVQO8b0ODZGJt7x8W1J26nT0t/ma2NeFYnpnNsP4H4ATwGY4Jc7AFwEMLHObx51zj3rnHvWPoQBAQEBAdeHTZOYzrk+AH8B4Je894urpDXvXHq9L+/9xwF8HAB27ty55hjHpKQ3BF1/l4iRQxV16dtVpi9Vd5G+uJmCGvx37iWJbXhYj++yxHS2rsREc5gut5Ad4fPo8dE36Qt72kgIzQ5L+5MqDUv+kr4qaQ6WIInY9SpjshdWq/Q1LY2o5DZ5kfprMsPVtjkVcpzPAiagg8thlSsmK/8GWO1SB6hEbUkkCZgRF8Ar5SARd0DrKijbQkRaglOk8bRq9zZASCTq+++/HwBw++23J/ukvzSC1Y5XrkECdCTXCaAEqD2nJUVXYzGmPuomB05bXMCMZF9jcrktQTht7V+Ca+qxrtMGVye/dF4DleqOxt2QQhRm/manaK1fnJtK2uZbRJbNNNQlcuoySYlnX6Dq9O2GSuwqeOv9lFGumIFVqT8yJZ3b/Yepsv1yS69FcoVE8fql6SqmGEPM833y5KmkLcdlCaWgAgDUanT9pTLd4/PHnk/2tVky3WeI+JdfpAyPuSl1x7vnjeS4MDJC74xszki0TqRhFSaPH6N1/exT307a6vyceL5/Q6OqhedZC8ubtTDK2RB3796dtB08eBuNg9figNH6hMA9dlwLhJw2xSA2i01J4M65POjl/Sfe+7/k5kvOuR28fweAqfV+HxAQEBBw47EZLxQH4A8BHPXe/5bZ9TkAH+TtDwL47I0fXkBAQEDAetiMCeVtAH4ewIvOOdFn/iOA3wDwKefchwCcBvBT1zKATJ1Uu92mMMLB7URmHRzYk7T1c5RbbpzzYBiVsFLg9JGmAMQyhyoeu6R+pEcv0nYtT2paPaPqavkgnX/6+1qPcWqazCl9NSUqc0wylXJCBOk4xsZJFey09Pj5GVJ1m+eUdOrsIBPR9Bky7/hjSvKN7KA+cntN2lL+m9lkWXAxMVgziBB4NgXrsWM0H0IeWh/xNKSlZV1taklLHWujIgV2HGIeEfPOiROqVkr05L333rvmnBsVoLCmIhlTWl6XNMxxEYa/+tJnkrYT970BAPDWu9S8M/cyKZ3Lc7SOZuf1vs806PrmjareXJjk43TdtZgErHU44s/kOJnmIidxpP0WORJ5YU7JvQZHCrdnmTHv2pwlsmEKByR2EiO/MZEYcaGIkQklknffRmR0FBlbHxdzbBvCfjXmZ0xOGzahzMxodGuXcwzdeZemjM1vJ1OFr1P/i0uajre2RI4G93c1tqOP5yvzukbWzgzQWl9kpwIbXV3iaO24peMe5FxA+/bp++Y0E4oVTtE7Ojqa7BsdomfzvnvfmbTt5cjKvXs1SlTMKQWOqm629T7WOd+OLeKQyWzu+bbYjBfKN4F13xzvuOozBgQEBATcEGx5JOZDo+RqNDGqcUADFfraFZ2SZd0OuelMLjPxZiiYaolk1PlF/bpPLtFxl1oaTbfALn91zkvSNZ+lg28hCatqKtBf+j591ecWlNiMsiSdlfbQl3nwNiU3WqPU5ms67nMXXgUALL+kpNOePpIg7v+nDwEALj+luUJe4cisakajSjEqJJmVGtePJEwjIEUCt1GOX/nKVwAAd999N4CV5KFIqJY8lD5aRsNYvc9KtvJbS3qKhGxJRMl98vzzpOC9/rom/b/jDpLOrGQj5KWVsgXSr826KOOwBOtGJGaGj78YKxn4f5+nZPxPXnpNr+UEaVU1dmuLzYJqsWQ1Y0h0V5JK5HpcLLIR19lrlVXKnS6wdGmupdSmcdcnVQLvCCkqmTE72ocEczrjnljgkmANk8FPijV4Htsdh7WwycAgrfFsbPvlfDQb5N1pGRJY8nzkzHqaniIy/2VzL9ocBbubI3wLJifQZXbhPXNSteqH7n8j9280urxox3x9RoPxnqTnSr+6Fx/ijIf//Cd/MmmbnSXpfXSMiNCxMXX3G+ZSapWqask5zmFjMxq2OCvi4mJzxf8B1WaksAMA1EVbG17fxXU1Qi6UgICAgB5FeIEHBAQE9Ci23ISyb4xMBX0jqmYv1snscXb6dNLm2Od2KUdqxkxbyaG4Rvuml9WEcnGB1MqaiTaLOF1kiVXIfFUzRlU44fzuu1RVv/DqKQBA1lQArw5wknj+WzqgPqwzoDH1mdS4t72Vkjsd/fLT2u/3SAUc20PkyraHlJR5/m+fpbbXVRXbViUyZLm1fg5Za7pYXdwAULOKNX+kFUtI608gphBbR3K1qSUt6X9axKQN6pIIz5MnKaJQTCmAmnzSSE97rtVV6e11yrhtW5oZSDCWofVxaKdWUH/iuxTleMn4ZEeejitK3VVj4soX6Jo7Y/qIdTgVrW8qUSl1GyU3W8skmCpzMi1v0q02z5IpLr6gpsEMk6e5aC3Bn+V5sQR4NUvkb9eQ81kuXlHiPu49oM9Bc4bU/K6JTE0kv+76ZHCuoFmqCpyOd3BITaVSpb1mip0cefkIAKDB6yNjE1ExyTd9WcnRTJGe26FDGkdYGCQT7PZd++nvuJojq5zKN2vS9srq37dHSUxZ10kKZbP+Yk781Wrq87jM67ptkllJAjQxnZw/rw4SknZ2bk5Nqw5rn8MrIUjgAQEBAT2KLZfAjy9S9FFskqOfnyNJum5IoTZHOXY4L4gVENNKdw1WyC1wm3E1y/FvhljajqoqKUu16uIOlRp27+Vyb5dN5fIsV0RnsqW1oKRgy5HUYMnRcoWjOfdqhNbsKSJHL18myaa4XYnQ4XFy5ZubVCmjL6bflozGAK0xAWDll1+QVozBugq+//3vB6BFFqxUKoSfddUTSdZKz1IYQUhDe87FxcU1/Ypb4He+852kbXWkpJW2pQ9LSp47R5KMzW2yffv2Fee355Q+bEm1yclJrIeRJerj7bs0l8exbzxB/Te136ifpWd2GcSySs8VJiNzFVMkoMnl55Z1/vLs3uf4WiJzH0vTNFfRgmqRi5N0Le15bfPtleu/4+xzw/fDrMk2F3IY69pq93Te7QNEzO1yJgp1kjSjKNNvjgePd303wsFhlbZ9vNaVM89FFZrG1VLW3XGOULTpmqvs0mcl9jPnTgEAKncoAd9q0fO6vET3u2YiIMWdMTYRsh1xhTT5YkRziTiBjS3U0OE26zIr0rYtB3jmLFkQFubpnhUKhvSUMoImp1OpbHNIbw5BAg8ICAjoUYQXeEBAQECPYstNKE+//gIAIFNVlcZJBF9eVbYOkyUVVkMGS8ZXUuo3mqimCqsjhZyqYDVWpaNZUoMbs8Y0wupco27IsgKZD3J5VZs7VTKx5LgCSNRRYqefySwbsIYCqWXFEU2uVBKzCyfzsXnAKn3cx6BeX5mJGqm5mQYxE1hYs4OYQtLSsorJwpKTr71G/s5StQdQk4WYMADg6FHyWxfTiDVdSKSnNW2Jz/fZs5q4RyrxiPnDmm1E5X7ssceSNjG5SBpaAHjXu94FAJiaIpLR1ryUfm1NTDE5veENb8Bq+ClSve+qaHTwj0xQGtwnv/HlpG28xPVLHY0jZ5I9lfLsw51d6wtdMua/gpicWKU2LtyoLZJavrCgfcyzGWbJHLjUou0Gm0lWkNdSbN7kQcuzOXIo0vGW+U2wm6d0np9LABi/h0j2dlWfR7FhOr++P32xqObIASYv50x0pjy31qwiJHSN1+nigq7rbdvIF9v6U0+eJ1PY9nElMev1Gh9H87Y4r84N5TJdg62OkxXzlXlwu2wmifjZWDbPxgybSU4w6Q5omly7JsucAM1xFHkn0nF7TjgW23dFijPBlRAk8ICAgIAexZZL4NN1Thdqkrr3czrW0ZLJdcFfSUk3Obuo0pQEtkWGqGlysYS6yT8gOQ/6RLI236+YD2uar3Cco/1tQ5rs2E+uRhMcKdYtKJERZ6VepkpMXSYp4nklO8cmyM0pKxKKqUqf43SXQ4OazlPcxOLO+tJO07g0iWRtIxVfeeUVACvzjDz7LLksCjlppfhnnnkGAHDpkkb8yX5bNf4Tn/jEiraTRir5/OepsIBI80C6JiCagkhiViITreCll15K2qQOp40q/epXyc1PpGw7H/v2kTvgHuMmthGJ2W3QOesLeu2HhumeVe9/MGnrX+Rxe3bfi/X+lDhNbD7WtdDkfCexkVrrDbpHMbu72vqoUZ7uY2WnSouj2+lctbb20eT5WuKnuW3cAyeYFC8OKkF24ijdj8JFnb9tLIIPD/Mz0qfHj/D2WaPhikAfddcnMQsFfX5Foq0OqCa6vEBkai6n91vygSQpXIxUKmvBuo/K+pybU7fKwaEBPj/N/eCAkqmOn69z59RFucWa05Jx15xl977JC7ROJEKUxkjj7TfRnEW+1inj4rh8jnK3DA+TJtdnNBh5DgqFzaWIXg9BAg8ICAjoUWy5BD5Qpa9YVNCvaswuOfOLKiHEnDS/zUYjK2F12R6XM1+zmO1rkcm90BEpXwJ58ppPZFnKuBnb88AQfTHbZZ2mYc5/kOGMbzWT76HONq68N1kA2/R1LxrXrmG2By6xXb9b12vJcVuxZLIRSi4R0+9qzM+rBCI26m9/WxPUi2RsJRqZQwmckdwogNqo7TyLLdm648k5xKXKnlNgc5CI9FKpqIQnWdsmJsiOmVYx3krPkhfFSurym6effnrNGEVitxnl5JxpkvgI359hE5A1wRxG/offmrRVWFqOWAVcXtR7UPYigaumUWO3trklPW5RpH3O11EzxRhyRVp/kXFvi9iObouAdNm1tcW2bbueyiwZtmx5Qr6+7LzadSc4wAXcf36nBvJM1dhtzgiLOX6WnFtfBizldZ+MqFrR56vDwUPeBmll1n8lidZmA8MuXSLO45lnnkvaDh0ivmLnzh3c59qslUePvJK0zXA2x6axrQ+yphDz81Jr6FzNcobRtOIlBVN2Ls88XJufoZYJHkoyDxr+K5u5upKJQJDAAwICAnoW4QUeEBAQ0KO4ognFOVcC8A0ART7+0977X3fOHQDwSQCjAJ4D8PPe+/X93NZBp04/cStyE9B3JTYmgxpHSnY4N0Inv9b471ckqKc/lbISB21WIy8w4THUp/tEo2nVjdmGU1RO7NBISakofuEiERTzkV6yJLc3zlYYYjV71ESDNWKpf8iuSsuqnuWZgOqbUOJqiiNTa4t63Lb+g7B4/PHHk21RNa1boJgsLAF0kV2fvv71r9O4jEoosC59UsX+gQc01ajUnpQIT3H1Ala6agkkUs2aM8TEIeTrN7/5zWSfpI617oxyDnstcq379+9f8Tt7nfb6HnnkEQDpJhTHuW8yGTXzlMu03jImX0UnpraY3cRc3uTniWjbd5UYK+dIVW8VdY21ueJ8vUnH5QzBmeHK8K22mhjE9bRlogCb4rrJfWVL6r7X4KIJHagJpX+CzFGtAV0fUZnTIw9z6thhzR8yG9MaKDrzupBpiNdX+4eNOVLIziVj3hkYpPW0OK8OCZ6J4Cw/55bka7dz/NfkG2Hzi621eXmK1tiuXbSubPSxuHI2jGlwgclUS+Ifb5ALbJef6WxW3y0SRZkxJg9xBMiY6OA8T1LM19Q1hK9s2yr212LQ3owE3gLwiPf+TQDuA/Bu59xbAPwmgN/23t8GCuz+0NWfPiAgICDgWrGZijweSEqk5/mfB/AIgJ/h9scAfBTA7131CFjg6JggiMTNKq9fuFKVpADJW2C/wvJ1jEwejg7naOiatjZLI1LGqGPyOMTiuG+IjA5LdSMFHcciF4WYYQKoZtwOSywdVYzmkM/Sl7mb1ePmOkze8DXlB9UdaWiQ2mKTyW1hiiSD2SlN/o5VEriURQM04CatkruV1EWifs973rPm+H4OQrDFGMTdb3xcNRKR7OV+2CAfcS20ZKNIwbYPIZZEArLjkMAIm/dEttMkMSE4DxzQkmBCsFqJ3eZbWY0MlzmLjV8qKaKAlXnEWa8Ts8ZoJNQsVy4X8p1+yWsrY9Ypk4BtIfxWuJVxPpWicS1kTbVjtLYm5xGqliXLoGoacYauPVdSYni+yQUoiqqltMq0Bhsl0pq8Uz3Se7rHObPW83l5Hk0w3CoUjRvw9nHWmi5pNsf5Rbp/OVO0oc3OAQVe/3btyPPdMa7BoglYLnV5mdbR8WM0L6eNa6s8miuyd/JfSyKK9pNN/lppmzUSE6gkFepz5pqLRa5ez8+QXcOZJMJKx20DETeLzValz3I9zCkAXwHwOoB57708BecA7Frnt4865551zj1rfX8DAgICAq4Pm3qBe+8j7/19AHYDeDOAO6/wE/vbj3vvH/TeP2hdxwICAgICrg9XZTb33s875x4H8FYAQ865HEvhuwGc3/jX6ehydGHHqtlMynQbqvJmWT2UogZNE70oBRpWVB1nk0jd+OHmWEXqZ/W2a3w7xd/Tm4re4MT+yyXjh1vkYgKsihVMQvs8t2XMh2qZx5YzPudZNhEtsu9vyxBXtTxtj+RVfZ/YQalSc9H6fuDWZCCweRnEZHDfffclbZKC9f77719xDKCEolU1xSfc+uHKtkRK/v7v/36y78UXX6TxT2ieive+971rziWmEDGv2P7F7GFzrEgaXmveET9wIazsuOVcNjJV8q+kgk0oNoFIzGYS60efT+pIsj+/icQEm/piQ5Y154lc81ZsEpNgcpxee1+BzBjWxNDkOIiCWQquStde5iQrBbWMoMVzmququu/YNLiiaAdvttvsB27MAwXO9eKcHi/5SKLO+pGYtv88m0PHxzS/zBzHLnRNVKmYHKVuZ8uYS8TnPGPsJe22RLeaCGo2RUT8LK8o2yn/sWuY/9oiD5LqVtbYCnMJbxeK+pxLrEbBEMgSiSr92vgGyb9i31lptWyvhCtK4M65bc65Id4uA3gngKMAHgfwfj7sgwA+e9VnDwgICAi4ZmxGAt8B4DHnXBb0wv+U9/4LzrkjAD7pnPsvAL4H4A+vZQANditabtpITJLImsbFpiultVjKcUbYEak5MiSAfJltgq8CS+ARf+WzRf0ixkwiLbdNHhPHUZQ5HUeRWYc8R4zlTVGIaj+JPlYaWBIiykSa9kuEZ436zRvJLW7Qtbcb6mrWx+6OBVutelVFsJkZzbgmX3obRSnbfcadUaIzRcodGVHpSGClViEoJeMfoCTmk08+CWClFJGW6U+yF0puFjt2+Wvd/SSPiZXYRXNI0wTEfdDmcElzq5RI0x071F1O0OKMlN7IN+LmGpkMfqUMrzHWIJxxE8vyDcp5EzHMbV2jTXged6nA0qWRtgusCWSckdw46jIyIniH86jEnH8lhq6nYonmo2FcEfMcmeg62ibnzTFBlzVEfMTRxg1v+pBIzHh9CTxjJFpxIOgb0lwoo6NEotvSZJWyUR8AeEMe5tiZQEhEQN387H2Rog0dvi+xSfknrn/2GRVy0brMOo5+zXE202q/krp5fm9ks3pflKjU94Foa2n9e6wtQnMt2IwXygsA7k9pPwGyhwcEBAQEbAFCJGZAQEBAj2LLk1l1lkh96hibSJcJkvKKqvG0neknQiWyQZdsammZeoVl9mPNGBWsjxNEFVg9apgCCS0maAp9qhZVWS2yJFx/H6l4mjTeVBgX4sP48opZx6q1LSa/KgNkzoiNetvlsWUM09XmhDdFQ5CsNqHYaDMh/CyJJGYE68opxKf81vqpit+rNV1IAqiDB9UHXQonCBFq+++kVOqWfq3qKGYMOc6OQ+be+rmvjty01yqmFOvxJKYZW+CiswH51m6yWm4SDWUKa00o8GRaEHU4irTPWlPqgeo5Gxwp2TKmQanL4CWa05gdkqIChuSLeW68eV46nKY2V2DznulDSMxWW80UWTYh5pxR9+WcTKzHXRstSmuxbNcfm1ii7troXUHBxDJ0u5IKVud0z26KCHWxrvV6i+M3+NoLNkKbI1/zWeP/L8+XqXHpmZH17BxguWUHiRnR4wv8XiibBHJyT3N8/pJxCJCCM9Zvu5CX4i9mvH7lhjXxIuFSM+b4UNAhICAg4AcG7nqN6FeDnTt3+kcfffSmnS8gICDgHwI+9rGPPee9f3B1e5DAAwICAnoU4QUeEBAQ0KMIL/CAgICAHkV4gQcEBAT0KG4qiemcuwygBmD6Ssfe4hhDb19Dr48f6P1r6PXxA71/Db00/n3e+22rG2/qCxwAnHPPprGpvYRev4ZeHz/Q+9fQ6+MHev8aen38QDChBAQEBPQswgs8ICAgoEexFS/wj2/BOW80ev0aen38QO9fQ6+PH+j9a+j18d98G3hAQEBAwI1BMKEEBAQE9Chu6gvcOfdu59yrzrnjzrmP3MxzXwucc3ucc48754445152zv0it484577inDvGf4e3eqwbgYtSf8859wX+/wHn3FN8H/7MOVMx4BaEc27IOfdp59wrzrmjzrm39uA9+Pe8hl5yzv2pc650K98H59wfOeemnHMvmbbUOXeE/8nX8YJz7oGtG7linWv4r7yOXnDO/ZVUG+N9v8rX8Kpz7l1bM+qrw017gXNFn98B8GMADgP4aefc4Zt1/mtEF8Ave+8PA3gLgA/zmD8C4Gve+9sBfI3/fyvjF0Fl8AS/CeC3vfe3AZgD8KEtGdXm8T8AfNF7fyeAN4GupWfugXNuF4B/B+BB7/09ALIAPoBb+z78MYB3r2pbb85/DMDt/O9RAL93k8Z4Jfwx1l7DVwDc472/F8BrAH4VAPi5/gCAu/k3v+ukLM8tjJspgb8ZwHHv/QnvfRvAJwG87yae/6rhvZ/03n+Xt5dAL45doHE/xoc9BuCfbc0Irwzn3G4A7wHwB/x/B+ARAJ/mQ2718Q8C+BFwyT7vfdt7P48eugeMHICycy4HoAJgErfwffDefwPA7Krm9eb8fQA+4QnfARU8X1ur7iYj7Rq891/mQuwA8B1QQXaAruGT3vuW9/4kgOPogYpjN/MFvgvAWfP/c9zWE3DO7QeVlnsKwIT3fpJ3XQQwsc7PbgX8dwD/AUndcYwCmDeL+Fa/DwcAXAbwv9kM9AfOuSp66B54788D+G8AzoBe3AsAnkNv3Qdg/Tnv1Wf7XwH4G97uyWsIJOYm4JzrA/AXAH7Je79o93ly47klXXmccz8BYMp7/9xWj+U6kAPwAIDf897fD0rFsMJccivfAwBgW/H7QB+jnQCqWKva9xRu9Tm/EpxzvwYykf7JVo/lenAzX+DnAewx/9/Nbbc0nHN50Mv7T7z3f8nNl0RF5L9T6/1+i/E2AO91zp0CmaweAdmTh1iVB279+3AOwDnv/VP8/0+DXui9cg8A4EcBnPTeX/bedwD8Jeje9NJ9ANaf8556tp1z/xLATwD4Wa9+1D11DYKb+QJ/BsDtzLwXQITB527i+a8abC/+QwBHvfe/ZXZ9DsAHefuDAD57s8e2GXjvf9V7v9t7vx8033/rvf9ZAI8DeD8fdsuOHwC89xcBnHXO3cFN7wBwBD1yDxhnALzFOVfhNSXX0DP3gbHenH8OwC+wN8pbACwYU8stBefcu0Emxfd67+tm1+cAfMA5V3TOHQARsk9vxRivCt77m/YPwI+DmN/XAfzazTz3NY73H4PUxBcAPM//fhxkR/4agGMAvgpgZKvHuolreRjAF3j7IGhxHgfw5wCKWz2+K4z9PgDP8n34DIDhXrsHAD4G4BUALwH4PwCKt/J9APCnIHt9B6QFfWi9OQeV6P0dfq5fBHnb3KrXcBxk65bn+X+Z43+Nr+FVAD+21ePfzL8QiRkQEBDQowgkZkBAQECPIrzAAwICAnoU4QUeEBAQ0KMIL/CAgICAHkV4gQcEBAT0KMILPCAgIKBHEV7gAQEBAT2K8AIPCAgI6FH8fwmVBac2KSPvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " deer  ship  ship   car\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# functions to show an image\n",
    "\n",
    "\n",
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "# get some random training images\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# show images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "# print labels\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Define a Convolutional Neural Network\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "Copy the neural network from the Neural Networks section before and modify it to\n",
    "take 3-channel images (instead of 1-channel images as it was defined).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "\n",
    "net = Net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Define a Loss function and optimizer\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "Let's use a Classification Cross-Entropy loss and SGD with momentum.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Train the network\n",
    "^^^^^^^^^^^^^^^^^^^^\n",
    "\n",
    "This is when things start to get interesting.\n",
    "We simply have to loop over our data iterator, and feed the inputs to the\n",
    "network and optimize.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  2000] loss: 2.226\n",
      "[1,  4000] loss: 1.912\n",
      "[1,  6000] loss: 1.691\n",
      "[1,  8000] loss: 1.593\n",
      "[1, 10000] loss: 1.529\n",
      "[1, 12000] loss: 1.471\n",
      "[2,  2000] loss: 1.398\n",
      "[2,  4000] loss: 1.360\n",
      "[2,  6000] loss: 1.363\n",
      "[2,  8000] loss: 1.326\n",
      "[2, 10000] loss: 1.315\n",
      "[2, 12000] loss: 1.273\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(2):  # loop over the dataset multiple times\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # get the inputs; data is a list of [inputs, labels]\n",
    "        inputs, labels = data\n",
    "\n",
    "        # zero the parameter gradients\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # print statistics\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:    # print every 2000 mini-batches\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Test the network on the test data\n",
    "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
    "\n",
    "We have trained the network for 2 passes over the training dataset.\n",
    "But we need to check if the network has learnt anything at all.\n",
    "\n",
    "We will check this by predicting the class label that the neural network\n",
    "outputs, and checking it against the ground-truth. If the prediction is\n",
    "correct, we add the sample to the list of correct predictions.\n",
    "\n",
    "Okay, first step. Let us display an image from the test set to get familiar.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB5CAYAAAAgYXpDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO19aZAlWXXedzPz7a9e7V1d1XtPd88OMzAMICGEQLIHJIHCJjCyQhrbOCbCIcKSQxEWsn7IRPiHFHZIliNsHBMCgWSFEAYkMMKyYNglDUzPCjM9vUyv1V1d1bVXvf1lXv845+Y5r5bu6oWuftL9Ijoq+2a+zHtv3sw853xnMdZaeHh4eHj0HoLt7oCHh4eHx43Bv8A9PDw8ehT+Be7h4eHRo/AvcA8PD48ehX+Be3h4ePQo/Avcw8PDo0dxUy9wY8xjxpjjxphTxpiP3KpOeXh4eHhcG+ZG/cCNMSGAEwB+CsAkgGcA/Ly19pVb1z0PDw8Pj80Q3cRvHwVwylp7GgCMMZ8G8D4Am77Ai8WiHRgYuIlLenh4ePzDw9TU1Ky1dnRt+828wHcBuKD+PwngzVf7wcDAAJ544ombuKSHh4fHPzx89KMfPbdR+w+dxDTGPGGMOWqMOVqr1X7Yl/Pw8PD4B4ObeYFfBLBH/X83t3XBWvuktfYRa+0jxWLxJi7n4eHh4aFxMy/wZwAcNsYcMMZkAXwQwBdvTbc8PDw8PK6FG7aBW2s7xpgPA/h/AEIAn7DWvny959m39AUAgLFJ2pbNULdMIN+XVqsJAOjEbTomm033xQn91ibiUWOCGAAQhKrP7RLtA+3LZBvpvhDumnKOOOkAANod6VuSGL5AxP0x6b4m75MWIOFxGSOtrRaNIY6jdWMPuG+tRNqq1A3UWnHaVrrvcWh8+MMfTrc7nc66a94KXPf57Jq/uinQbdQauEbtGGXc/CXqeDfPcpKreVNt1G93/Mc+9rF1+/b9OM9t3Enb5q5cBgA0G7JmDt51CAAw0F8BAGRC6U82Qwsvq9t4PUdGrbFOHQBQLmX4HNLXiLdDtYgXFuYBAH19fWlbJpPh89JxJpBzdJIWACDYQFQLjDTWqmTejCJak/l8Pt3XatE5OvwMAkAhX+BrSd9+/3d/p+v8u/fsSLfLI0fod6E8t5W+MgBgpSnruro8x/2l+52oxRDxIApRLm3Lh/wKU89t+gByU5zI+V1botrcNdzY6fo8lxusHcP3zwT6vRBvcBz9Npej/mYD6TcsbZuszF9t7hgA4OtP/2DduTbDzZCYsNZ+GcCXb+YcHh4eHh43hpt6gd8KtFiKsrYujSx95lBKmwLQlyqKWLLWEgV/VU1GGptOakjkCxexhBdyU6TOYRKSitERKcNJw4k6R8uQZBKH9AVt6X1xwOeSr7FhKT6v+hax5BNE1PG43VYd6fCQ5BxO4gzDzS1eYRhuuu9W4UYlej0fqZykpMTEiUyWx2Bln9OIDETakbPcvAS+EcpFureBlcejWaW2pCVEfD5L5y0V6LhIXcatnZxaZIUs33c1lmbsjqN1lVXrxE1RFMm9dZJ9oKR4Nzc51kr1MqnW2nxNgdNeLeS8AV8sw1Kok+oBoN1s8vjUWFiqxFXWRGJFiu+Eg3SujDzTcUgSeJBREnh9lfoWV7kfcr6mpePaSvJt8PwqoRytNmlJAT8T9Zq8W9xzosfnNOIgkOfQOs2FJ1Nr/J1OzMfINY1x7ydZM4ODNOZcoY/PL/csces6J/2IV8u4XvhQeg8PD48ehX+Be3h4ePQott2EYtnEACumC8vkkYlFxUvapNKEBTZTKDXUWQ80kZBlFaljRUVJ2mHXcU4VAgBj1xBpAAwTLjYUVbAek652eY7UrWpL1KLVVWoLrZy3L89kliLhKkUigAo5GmcStNJ9QWoukbG7EbSTzdV+bRL4YZXJ28p5u8wV7vguXdPt0iYfmvNmm+Yj0npzTL8NzUbXTjZo2xquNpaIzViBMmNlQ7pWJpC2XMDmMbdPEZDNOplawlARbhHd93ZTiNAAbDLrUJs18kjGbCrKZgpyvJsHtcYcmRuzGVDHW8xduQIAGBsZlOPZXBJm5VohX8vNs7LkIOLjm4rUdQRruy1taxFY2Rdzf2P1HMSGxpzvk34M7xuj3y4tAADKtdV0X6tB74i4LM9j0k+R3X1ZmXt33YDtrK2mPF/O4SGfl/uSTqlaE24du7+Bstl2eMyJXn58+Wwka7dQYKIXzgwoJprEmWe1DH0DJkovgXt4eHj0KLZdAo9ilrxD+foFLEnkQvV1dwwRfwkDzdTwTztaQnWkTFakl5377wYALC/OAgBm50RSyUQkbQeQL3OrQ9NTtxKAdOwcSTQ2NwwAaIdCyrRYMlhdmk/bLk6zJJFXktXUIgBg70665nCfltKca6GM3QkXsV3vquSgJd9b4T54S6T4tN9KO2BXy44SX9qsCZ08fRoAMLZT3M8SJqNHh0SCzDPxk9xEH682R1mWspOOSG4hS08ZRaBluC2IaR1lM0qqC9lVVWlXmYDubWKUxpWwe2yDyUy1nho89mJR1nDomE0t/vE8VNnF8dlnn0t3tVkTGKy8KW3L5ZjMV1OQurKydhoo9z1jHZkva9ImjsjbXALvQFwdA9BaT0JF4LIWFiptrMRsZKXI9/i5Z9J9rVmSxscfuFv6doWeuaaReSvzwFbqRITm1VhyrJEHw0IYBkxi6ldKs0jnjdqsmbRlslZKdF9yS0tpW7TnPgBAbaA/bUtYq4r5nuUTIUJTjT+WtjC+fnnaS+AeHh4ePQr/Avfw8PDoUWy7CcXp2SaSNLNOve3oCEUmjFqs1mYVORTHTp1TJgY+h/arffNP/hQA4Nm//TsAwCU2pQBAteMiK0W1Ojc5AwA4MykpXnKD4wCA3WMH6Jo5URNbrP5lypL1sdMgtW9u5lLaVhwk88vkKkX3NZQ6PNZHKl4xI2pl3CY1WAebraXvNiIxb0ck5tVNLUyWZVTULPt411eFtF5cIlV3epZMT4U+UYeHOeJQRw060k5HZ27Q2TW92DqybK6z6hwZN/mx9DuEI9upLaP8qttOfU7kHGGF5sFY5ffP/saJi/aNZV2vLpOprVwU0i7g+dZRkRFHLi8yeTm/LKbBAvtJt5Slo9Wma0VZvWaoLeZI544yH7ko6Kzycba8ZpN4c7OennlnEgzU2OMOj1XZLgybOBqG7nsmkbVgRsi0VluRvrXPnKD+GjEzJTxdVedfrp6vbJvjNy4oEp3nQztGNNgcGjZ4ruSSaO6kPtYvi6m0z9Azb/pHZHx83XbgiGEV+8DzHSpSPAqu3yToJXAPDw+PHsW2S+DNgL60SzUVocXSy2BZxIYKk0IRSyCaYErdgBSh4kjOWm0hbfvalyjvyvQiSRTTq/L9OneRjjt3SVKch3mSxuOwkraVKvSlzRRpX5SXL3+OpcR8IGOZbVEU2PjuvWlbg8mV06dJAp9fVDlZdtF594+KJpBhVzqj3LhE/uLxqq+7Ta5P5kwDHzcQALTUHWwggccsZSUsbehoURfhdmVuOW1brtJY6zr/RY1GE+SILK7W5d6Wiyxxqr45eX6rCsb1aiI541zeZL4debmhC2DCkX/KBTBijTFSTGFoaD5srO8ej4+J+1i5mq2u0Lyd19eMXOSySIt7KjRvzmXwxZdeSve97v77AQCJdnGMaX7z2sWWNYF6jTXcSM7fYQ0wjITMb3O+nWZz8xTRsZLOE17DVsuM7HTQ0u6GfN3+FZ6r0bF0X2HHPuqPFfIQ7AppR3amTfUM5za5THlVoFxyq/y82rHhtC2TUJ8aSoMvsRbYWqHxNXWOmgJHvFblvkTDpB2YjHKT5HwnffzTUEn4HUNzbwLlMovrj6b2EriHh4dHj8K/wD08PDx6FNtuQrlSJ7Vhvi0k5jf/5hsAgPuOiCniJ+4ncmCQ/cU1eeKS1gRKHYmZLFHcF86cIz/j+TqpNrY4lO4Ly0yWDYm6X+D6nS2VQrTFxFllkPpWKUsfZy6TSWR5QZEbrOLlC2JqOb9A5GmmQurhzJRUSypfXgEA7KzI8QWXujZR5NcaVGs6GRirkEp1dKl2Q5UYyW279JgqhxSCZP233UWJatvFKqv3jswsKKKrwRFrU8qEMrNA24kiuNpsH6mtEOE7MyvzN3lxCgBw3+GDadtd+3dT/5VffEqmukhabTVx3dZhAlehNkM24SVtMQ8EbLKrL8lYwOYDy0mQwoKMPcv3Kqvm27TJdBZrswNHG5uUOBXzUbVKpoLpaTm+VCnzNVUiL57z1iodl1f+6FcWiQh97gdiVinl6JqHDsqcRmzKadZo/RUilXipSWsrVmmVY/eoNdR8rIWaYpfSNemK1eB96lnOsPkqd+oknf7Zb6f7Om9i05NKy2o5RiO7Is9GAzQPZY63CHNyfFKi8xuriHVOJtc3LO+gzEU2v6zSmsyMibMCLtC+qCJmzsYVmt+wKG3JEfINb3AirECR7tkOTU6kbIP2Kpz8ZvASuIeHh0eP4poSuDHmEwB+BsCMtfYBbhsC8GcA9gM4C+AD1tqFzc5x1Q70kxRQm5NvSTtLROF8TSU7b5FbTyXLbleK+HASZxgKydJokQR7RfFFsyv09S0OEIExOCrEYjUhSWIEKuqNCY9WRqSiRpUklMYqHb9PkSE1lrZnWiING5aGluaV1MXSSJ2/7mFW+j29TNM4tSRS/74R1jCu8oVerMtAy0XSCgKVl8EVp+gSrB254oJcu9K4bvBt38A98fIUuVgODZE2U8iLZNNs0JiLOWnbOUqalFXiWbVGYy2xpNJqqPSfPOjVpoyvk+apUG5tqTuj27dumF0S4dW8H/MuYb86yEngOSX1l5ks7mfyKWB3SADI8T3Oa4GTtaSgIWshTfLPhUFay7LW+kq0b3BINMUzk6Tlnb5wOW07ceopAMDCLEmcqw05R61NNVYiKLdAluwfvPtI2vben34MALCL13MzL+NsVKv8O7lmhQukm/oKNkMmlPXn0kE7MhOQlKqRkiPLC3StziS53VaUNrFyia7fyku0owW9F8zlmbStNMEEZIU1S8izVGD31eyi9LvBxHFndipty/IcdpZprnLz4sjQrrO2VBANZvEMOT9kCyKB940T6epSKVnlMth05LVaw63k+kXwrUjgnwTw2Jq2jwB4ylp7GMBT/H8PDw8Pj9uIa0rg1tpvGWP2r2l+H4B38PanAHwDwK/fSAfuft2jAIDJp4+nbeV++ro/+tY3p23FkOzELZaAtXRpOFtbbCVfRt8Oqrf8wksn5bwDJP3t2keuVVbZ0jIsZSfNubSt1UrWXSvkL+bLL74IAKiohOzFEn35S8oOdunyNIDuPC0hSxVD7P61uCD2u4V52j4zJa5SE2PkIhVlVTTBGkQV0QRilp7bup4c2xbTvxC7pAsO0RKn3cCn0AnoymMxDShx+TKgXDkH2BWr3VbnYqmsWBabopPADQdnGeWylSs4dytVJoyJjS6b4bq+yTUz3Yfw7s1F8Atnz3K/Zb5XlmndxW3RBC5eJO1jgddAdVXswTuGSWoulyQIJ+RiJC2VwS/iXD0B5+KpKum84QajCkucv0T8yZlJ4QmqLfptvp9d2UoyMW4llrIiq02do+CXS5em07Zvf/tvAAD3MtcwOiASZ32VJHtX7gwA2vdSPpLVpc0V71xWxm6dNJ4olZg1mEC5va5y4N3qI68HAFSiN6b7ait0D9oqb5LJ8dyocoOZAl23yu6S2v21zflGMurZqPPcaCe+Otvla6t0zVJBxtLg43Nlec6H+ujdE6t3xSqvXbBbY6GtMhpyn7THb/sGcvvcqA18zFrr9I3LAMaudrCHh4eHx63HTZOYloyPm346jDFPGGOOGmOO6jzFHh4eHh43hxt1I5w2xoxba6eMMeMAZjY70Fr7JIAnAWBiYmLdi77YT6r/voNCqNTZorD3wKG0bYTV8MUzZwEAbR291SFTxKNv/7m0be/BRwAABx48m7Y9+zyZPQbLZJK4NCO5UCJ2K8rpYgLc29WqkFOL86RGDpUz+hDqB5tJRkYlF4orUjC7ICYRw9GKfeyCGIWKyGAV+rULk2nb6CCp2Yd3K1emNfjEH/0vOT/3I6PUuXIfqYCHDghx+6bXkZuTK9tolZnHkYJW20tcjhplJnEEWzZH59fkZDZLJpHhQeXO6GqbqhqDaY6NDJ2j0ZHzLzKpu6hSd64skUrf1q6TTDwOsyvY4UNCMGVctJ4uXB50GVS68O2/fZqHqwqKOOK5Lmvh7GUi2tLalUocGuRK9SVF6ub4uIxyLYzYxS3gmpg1RUBGfA6r8v5cnifiu63Y6GKfc3/jfEGryv2R70ejIf2u9NF53/LGB9O2KqdAbrDL7PnzYhp57bXXaOzK5e3cHM19vSbnjXJCxgNAqSQOAR2eh3as7xkXVlHknWGTUmGMiMrlqozlyhKN3Sj32BbX/MxqMnCRfuNyKeWy8hws8xrPZ9Srz6X5VZGYTY4OBte8XarLmnRpaIoqWrVvN5lsQ23WS+u58r3StRvcm0MtyuQG/AhvVAL/IoDHeftxAF+4wfN4eHh4eNwgtuJG+KcgwnLEGDMJ4LcA/DaAzxhjPgTgHIAP3GgHwhwRAZemj6VtD72Rks+X+uWLHq4QYRSzFBCpclCnLxDR8LbBA3LiIgV79JVUFfGIrlVgt718VpWy5q/vronxtOkVljyyioxZZiLlwB7SGI7cc1+6b36eizdUJCDgErs3GUWaDAyS1LrE0qXOH1Io0m/rK9Lvk+c5uEIRUWOS+oGOr6lgozptZ1RQzQoLsEXVFt97DwCgYZnsURJ4jiUhLbW6wgw6S1//EGkbKVGk3A+dW1SopG0XWaVljYSlkbMcaHVxRhS6+TnSeOp1kdziJkuaKmeKy8mxew/RMXv37E73ldK1oknazSXwF05SP4oF0Xgsa3zNjtyXfs4q6ci6lpJyr6zSPQjVXPXlSePqxEJaGybtQvY1M5EEhuWqJDm22kKOzs878lKX/6K/Lc6xslKVuWqxe+meUXFFHB6kxeMChQBgfoHyqAwPUD8eef396b5JdhVdqssafnWS7kug1vWBNUxYpDKBFvromVtVJdIiVllilYUv4mCXgNdkotwfDRd4idQ13Va7pTIwshYdsWStNR5HXsZKy3Ol2jpqVWYKTDLG67OautwpmY7SBJjh1xkN87HLYMnXUkvOBbJ1e/Vef/bQrXih/Pwmu9513Vfz8PDw8Lhl8JGYHh4eHj2Kbc+FkskTodJoaHWY6w+qCMViyZFCpNrrepnliFSgTz758bTtZ//Zh+kcKnosy7UAXXGIAwd3pftm5omQaqyKGrxzB/mN6wT5Ta5TePAQEax3HRLydel5qkVYXRE10ZEwHRWBVmcTxwDXz4utRIX1D5L611EZ+MOAxjd5SUwLY69DFz7wT/6p9JHJvZLKv+JIk4IyPbnUDMvLnJ+kI6p9hkm1SPm/WlZF68o/2iZ0Ple1WxOnER+fyegIz/VmGOf/2uD8ISWVY2KQ89HELelbPqRxLc6JCWDy4lkAwCEmvsNAmYqsq7iuUu5exeV2mc10VhOF7NtfCGU+du+5i/rv0uZelrU2y6afsTGp75kbIbNOdVH8qROONO0fJPtDLiexDA0ecq0jJpQ8PwdxW9ZYyGSgK3KSyarCEnnafvQNYhI5sm+Czt+StX7mNRrXa8dfAQC89U1CcO7ZQ8eff0ly9rRjl5No85qYWdWPLNeETayYLQtMWndU2t4VjkSNmajM94vpZ6zEJi1F9knFd5W2F67mJ/3VhSg2guVnU5tQYvY1d2l7A3XNrDPcqERLTX6n6NxLEZsQY65A31W3lp8bXZdUm1K3Ci+Be3h4ePQotl0CNxyhVVOSb4MlyIzOgzDHLj6c7ySDxXTf+AB9EU8ek6jLS5OnaKMmpczOTZ4FADy8k6I/d+0TJnBihiSg6imRMoZyJP31DUiZpNdeO0PXnCDpfXFZpKM2f8mnrygJy5EbylWwxhK44dwImrooueyGiURWZg3NR2v2MjZD0hYJIZVA1P5yls5byMuc1jmTXK1N/Th7+qxck0nMvQf2pW1nLtBcfumvnkrb2pwBMs/5Torq/C56rb8iUX0D/SRFPfywqBCjIyR13rWb5jRQ7ntOinJEEyDkVH2HSGcT43SvJnYRCa0z3NXY1axLI7mK6JJhYn10x0TalmcCeXZW3DurHBXswukaKsKyf5TW1i7lCtvXT+OsjIhUPsfEd8wSWVtVKHMuizVF/LXajqAUjSTrMl7m6B5nrGhIO3juRwflHuSZkBsdFNaxwq52c+fPAwDOvXY23bdziNb/0vTTaVuGyetWuPkrJFK5P0LOsphX+VEWZ4iQnV+VHCRXpmh+B/to/T9wn2gCGda+m4rAbbMGoAl4t/5dkZNAEetOCtalAOOUONUsY3duHZ3pFOk55JmL+Hi9dt1vMk4z0g86nz5QLpHxVVxbN4OXwD08PDx6FP4F7uHh4dGj2HYTSpoKVqkj4yOkPml1/GsvkU/2ICeVPzwkKk0+xyROJL7QV2bO0umbElG29y7yEw/5vMWKEEYjY0Qwzc2LurrE5KUuvL1jB6m/EZt3GopsdEmK6krd7/CPO+okjSanquzQ93NYqdSGa+VljYwlxyRPbLsj3TT+4v/8dbqdcIL6QPnQlpkQ7lPmjP2Hacyjw2QyGB6XKM0h7lNeJWNaPEbmpe8fk7qhdeuKR9D/I6XeVvi3h/aKGeatj76BrlUSH+sSq+FOg22pOe2wb3NtSUxmbfajLqhq7QMDZD6Y5uRhs6ooRIEjAsd2yjwXiyoGYA0G2WQWKvNAkwtXGCXzzM9Rn5aXOS2wMvmFHMF37qIkjKosk/mjv1/iBJz/d5NJfKMIvZyLFizJfS9YF7mpc+PSM1EqsHlRVX7fPUzzUlSEYpWr3XeUacYVuzjAJp9jr55O9x05QomroAjLS5fINzw/KGYsQG93k3auuEiizBkrHFNx5YqYBhcX6LwnXvoeAODVF/8u3XfoEMVc7D90b9o2OMJmIGV+cKmTXXEPbZgIUx9y1be0sImqGs8EpBSOUSQpH6958DRyeQN2PCVJu5LF8VnV/dbvkq3CS+AeHh4ePYptl8BdlFR/WQimgT7aNirnxrIlSWJ2gb6EI33S9RITMHEgksfZS2cBAGODkvx9H3/BnXvW956V6M+LUySp95VFKs+wm9PLp86rHrtIQvrbVF/NVY6AG1AJ+DssVk5Nq4TzfdSniF2VikWRsFz+ELSFCI2r1LexHZvnQnnm+R+k24UMEYrNphCsWSbh3vyWN6Vt5y6SJD3HHNID94urWZYJyFpTpPgMay5veIMQkA2O9MuytHj4oETD3s8pRydGROKsFOneJspt9MJligKcWeBiFrNX0n1VJrcXF0UCb3FK14xyiXS5WFykblsRisUBmrcHIOPr7998Lp0kXVORnqFxJelE6o85NWnEEb6JFXkom6Pzj4xIZG+Z13heuWb2c78jvmfavdKyq15HuXf2s4tloKIXE06bGrnoxaZI1v2cgMV2RCuMWatpqUjCOt+PIq/Nc5dl/b3yGml3zaZEeLYbNL821FT55nBSaz4vY7/nbooEPnSvuPPWVkgaf/k5csl9/qgQp9/+FmmAx16RtX7k3ocAAIfvFql8YJDWmyN3w64+uvndIBexJkddCbjO+jKGLjozVqRnkrozbo6udM3GlYGUNaxTTm8VXgL38PDw6FH4F7iHh4dHj2LbTSguOm7nDvHJdjXyEkUGju8m1fwom0YWjaRstSGp2f0jQhT2V9gHMy+q8n42oZQ5he0ffuKP0301vtZyXcivGvvh6syTOzlSsjFP6lw1p69JZp5Xj4s/+vQ0mQOWVXTmwACdsFIidThUpFOGo+PC2sW0bbRE+/vzoqCppJwAgCsXlP/6EJmBdu8W0u6+1x2m8+fkHC+/QETRGKu1ZVWtZ4brA5YqYoIartBx733s7WlbwA7V/f103Miw+K/Pc+rdM+dkPpYWyayzvCTRpytMFi9y2t75ZYmw7DAhm1FpfrNcASdQkWv9FRrXAEduDipzU45NVNmCmKpW60ISr8Uw+3Br3/oyV1dJVDrUTEDzsYP9xY2KQs2yz7Iz7QBAnqMRQ5V31plM0ipEyoTifOBrVVk7LiIwpxalZXNKbYnm++JZme95dj4eKMjxY5xyN5/XNWTZJBKR+SgqCtl9hetT7hmXZ66Pq1UtNzcn3hKVJtYlvbKBbqO+hco3fGCY0rK+7R20dg8dEpPcd775DQDAmTPybFSf5+d2WUxsD76Oqvns2UPn0uma4w6t8Vj1LWFTbVcVqrT+q/sru1y9WE1oO+uH9jl3hGZ6rS4Sk99xygyjTTJbhZfAPTw8PHoU2y6BO9KuMigSeCembuUiccs6woUIjj5LktVyRiLcEkPS3Ngu+ZK/cozcj37kx/9l2vZ3nKi/WiUpsN2Sgg4zl51rnHzTVrmGXaSi3gYDktB3FegcS1dE2umEJPmO7RAiNGbXq7qS+Bp1kjirTJZ1EpGw2g2KRNuREUlvokySUrMjbWsl8IsnXk63l5no+tl/9G/Stsceo+SRX/2auBvuYHJvB1exLyjXtDxHp431iyTWx9t55b7XYanFSZo658vl4yQpnZ8RV7oWF+aI8pI2ta+PSN8dLBG2W+uJo4xKyu9yRujcEX19NJZKpY/3qTqLnI9melrud6OxeXWoIkufbUW0FtglcqAiWk2SpjYmArKg6nymJJWS/hLLbVpucsU03F9FrnX4fndi6evyHI1BP7gZlsBXl0jbm7ok0cdjQzSWgZJEE9dYek6UJtDhMzridBcXKACAu7lO5kP3SZGME6fpeXn+++IIsBY6hXLABReCSLTqDJP4sYpedOlYAyZ1Dx8Rwjxht9upqc+lbQuzNNaTTdHapi9Sfd27DhNJeu/9co4dY0QqR+rd0mlzsQmVYjbmGq/uPm5YAKQrJ8v6/WnKYp4HfYq0eIoS7buiPbcIL4F7eHh49Ci2XQJ3uT8GR0RC6PDXuhFIIYB8mSUJzuB3/oI4/7/tTeQe1liVL2Kxj9z2pi5K7opTJ2i46MgAACAASURBVKgad8dVq1beRVW2u/YNi9vX0hJJPv1lkTjvPkK5GZ558VUAwHPHzkg/fuI9ALqzKJ4+RRL6ospo6FwQG3WSvPeNieRW4KCNoSGRfG1EkkGntbmbUUOVtnrw9dTHd77rnWnb8ADZpn/0zcp+zZJbH2sClbJIxSEXKXBV0wGxteok+0sLZHetsESTqAwsB+9+AACwY7dkbJxfIM2lb0BcC11mO2PXVwx3dlRX6gsAVtkmbFUJLFco4MIU2e6dlgMAbS52ofOjFEubB/JUWVvqUwUdXFDPjMpzs8zBRQlnLTzkAl4ADHD+kDCjpUva1lpKi+tz1Zj7aDSl350WzZVRBSBsk44vKY1kYIA0mEKWbNSRkXUywNpbf5+syRafo6ayLbY4A2jAgSWDSvMqchbPScWzuMLw9999OG27otw/6Vzans/2btW3LO9O9IPIkqmzEbeUNrZ7z34AwP79+9O2Z6bpfndUubcrM4vcH5LOjx17Kd3nApXuukv6PTZGbox9fcL3gAPqGlztPlbPXoY1Lh2049wIdRyPNdpVkUaVnj4tACEIb6CgwzUlcGPMHmPM140xrxhjXjbG/Aq3DxljvmKMOcl/B691Lg8PDw+PW4etmFA6AH7NWnsfgLcA+GVjzH0APgLgKWvtYQBP8f89PDw8PG4TtlJSbQrAFG+vGGOOAdgF4H2gWpkA8CkA3wDw69fbgYRrDPYPSRL/ap3UllosKocjrFytwxMvK9e0Gqkq5ZLk8uBc+zh3QtS+i0zuvPWtlE5Wp+ns4/SwQxPitnR+nswk9aZK5l4idbUySiTPw31Se/EKq9dnz70gY6mRuWFxSa61g6vW91vqz76yuN7tqHARBCMmEZdCtKRUUnHCIxy856F0+4O/9K9pfLGo2cdPEZGYGJVDhsnONqtz84sq6Uvi8sAIXeoKfycQImplmXoSTpOqe0nVs3SFOZKGkEMlJkxPnxTT1hlOYerc8IZGZD6cur+kqtLPzRKRZ5VJJGD3NBO4vCAqspcJ07xOpbu6lgYW5NhlcW5WxvLaAl3TRTECwMAgKZ3j45SPo6Wi9totMsMkVvq4zGauujLvxBwhGbJ5StdedGaSvKruXmD3wYZauwkTf6Uyu6WqdZLlKERN+DpCuKFIO1fp3ZGIbVW0Y3KOImRrqoamIwF3jsv6X4tQmRDSbXVNGJ6vLvc69xuzbp+L4uzrE/NOSi52FetwJjm61sqC3MfnOSXzyy8+k7YNDdN93LlTiNud4/v5mmRWGVam1VEuSGsUUe7uc0eZ9TpMcqZuhNoVkc1XVpnTbLLW5HJtXBeJaYzZD+BhAN8FMMYvdwC4DGBsk988YYw5aow5Wqttzvx7eHh4eFwftkxiGmPKAD4H4Fettcum+4tnjTEbMmzW2icBPAkAExMT645Z4UQcBZXJLc3MlqjyX3z6kSGSzk4Eki1tZp4km7lQvmD9ZfpK3vOAEBOnz5Kk55Lma2Lx8GEiNQ4fuCttOzdFEsfLL38/bZub5aAQTvo/qFzHJl8miX1qVnKQGCZiQxVQNL6H3LH28RTu7RMJK8+lmZoNHWhAEpN2c1qL9//CP0+3B3eSVPTiD0TKdWRQS33lYybVXOkwTaK4UlWxlhC4Lej67HPuEc4SOTsnLoPODU7FbmCgMsD9EUl2fo61DZYCZ2eFsGyy9tFRbpgxl7ULVS6UYp7mOedcDHXFcJf8BiIdFVSWxbVYZGL20kVxxysxuXyPKjDgMjYWOb9Loy5a08ICuZu22zLOGucqKSo3zP4KrftSjv4WFDkZ8TMWKxKz02nxeVV2S1fOKy0+oIoEsBbbVk9eFDIJlyjXVs62OHeFNI3ZOXG5dFkDF1Q+GqdJ5fpEW1oLY7UETn81sWdYatU5QlJJmv86whAA6qvUj8uXpQDEpUu0vVSU4zK8jhwpX1L5V4oRHacJ7YtcROLkWXmn1OtUtKQT07lGRqW4x4MPUkDg4UMisY+O0lqo9IszRq5AmoIFX189e500yaEikn8YJCYAGMpx+jkAf2Kt/Tw3Txtjxnn/OICZzX7v4eHh4XHrsRUvFAPg4wCOWWt/V+36IoDHeftxAF+49d3z8PDw8NgMWzGh/CiAXwTwfWOMY+f+A4DfBvAZY8yHAJwD8IEb6cDpU6S27D0s6SDzAafFbAnRFLEaJESGkJ5lLlJwzz3ih/vVv/4yAKC2JP7ixWEy05+aJGVhz24hPQ/cTYUGckotP7iX9i/OS1GIV7juZsIEyeSCkD3LTL42YjEHLS+SmWaHIkjOzVHb0B4yJ8zllE9ywqSnMpfYiGsBJqKOr/Vifv6Fo+n2S9+n22QgphmXbyLSRQfS1KgZPkZU74jTz+r0ny4fSVb1N2A/8dDSvkpWvEkDNjO1Q6Xuc2SqcttFlnOVtGvsn1wVE1SLST7TVtGZbMNpKZI75mjL6godX1T3cbSf+hEp04WzVGxEZQ6N0joZVIU2XEGCSM3HyioRiaur1N9cTswfjgTU6Ugnxoi8zuVF3XfkpeV8HNWG9KjBBPHiguTnmZsnX+u6Mtfcy2l7M+xb313AgOt1qvXU5Fqek2n0sfhwt9g8VavK+ZcWyZSYVVGlbuxPfe1radvb3/wwuqCKFSTOv7ujIiDZxKLc0WFS8w7tC1Vk6ovPPQsAWF0Qf/Nh9m+/MCVtFfZhz/Jzk6gI5kqZ/dGVf3424kIYORUHEbBZdoHMRmfPSKTz4gLN23NHVe4bjpvYs0eiVSe4QMr4BD37E2Pyvilx2mpTUPU6g81jEzbDVrxQvoPN09y+67qv6OHh4eFxS7DtkZgvnCJpeO8Dj6ZtCejrZzRpx1/wZSZUFheFZBkeIhe69zz2E2nbQ6+nPAif+fyfp22G8xr0c3XwXRPiAlVmci3siOQxtJOmZ/yASFFLnIz/uRdIyp1aVWRuhgjT/nEhdkYOUVtXIQB22zvORSpOXRYJNctsT11FHlZ5GjqJSA3vFuEQAPDtb34l3a5xZrZsRpXiKjoSVW55aDn/havindESOPUjn1MEK7vhZVUWu6hEY81naZw5lc/BpdowKouiI6PbqlBEgwnKVGrVEWx8vC7VlobQKol3oETb/SUaU7kgUm4uQ+fLGLmPRrkDrkWbSTXtdhixi2PcRcy5cnI8f0rMybOUXa/KOOucgbGufECdphNknFuZrPnjx14BAJw7ezZtc1HEVrknTowTYT/EGSHrytvLbS8uCAE5xyRtXWm4LmeP8xRbXBYtKOC5L0aydly+lcuXRcNdK4G3VREJR6KbjpzDRX1q5zkLanOk5+qqTJYrHnL3EdHW3/DQIwCAZ1+SIg9PP0NZNhe5GEjckXuwY5zIyLe97W1pW8T3+ew5cTl++mnKpfTAfRTlXekXZ4hpHvP0tBD2bu3uHBN3wwMH9tP12RGguiJumM4hIBOJ1N/YIAfQteBzoXh4eHj0KPwL3MPDw6NHse0mlBNLpKLPxioVZ4ZU6qClVI7E1ZCjvxPjYkP4sR8hAjKfEbXywD6KrPzp938wbfvsn/8lXesynXdqSZS3RuMUACALUWHn67R96pyoiWA1x46SiWZwTMwJaV08Fe2YsLkhMaLSu+RNSxwpmc+opF2c0rVqVDImJg9tolWsbnVrbFSi06bqROjEsajNFa7TGam+Lc8SObuyXOV+iaqZOPV3o+gwZSbJFOg+2Axd3yUiA4CAbShFldzLVU6P2+vNY+CkSSYrtog8k5EFZc4Y6iO1c4/ywd89Tv63jqdsNkT1Diytp0hFzg1UaN3VJDdVihMnKEXq/fffl7YV2CSipyNgaijh6LtpFYXqkqM168pMwSbBWJlJDh7aDwAY3UH914UGMmy2GVCJpRwBqss8Oh/uV49TGtVVVQDC7dMxBAmbiKorMkc17meNo0VbysTlikecnxai0NUoja9Sx9F2RVhat5HCRVGqIFEkjvjkW1VQ9WJ/7B3v4l3yA1es4chDYoJ94I1U99WVDQ0UhecKjhw8KPEeEc/p/sOSdnZiLxHDBY7o7VcmFDcuV7AEEDPJjlFJi+2SY4VsegoUWxuzQ0Jb2d2SjUNprgovgXt4eHj0KLZdAj++SN+QL3xHoh0f2kfSyM6sGPiLLAWM76Qv3PiISCV3HWQy0orUMMV5ST7x6b9M2559gUghF+nZFdhoHYkk54hzdI1YE3PsmtdhQrQTKJLPzaYqjdRo8XnVlzZiQjNkacuqXCEdpnQy6mvtSmu12ptHatm2SOz9JZIoVhQR2o5JKrvn3gfkNxMkjcxw9N2Mir5b5bwoOv2BkxxtLOctRSRl3PN6StN5SZVKu7JMEn69JRJhnQsp6KjPHLs2lljTGFC5P0a5wvj4hEg2h3aRm9+OnIihq+x6OM9udmFW5q9YItK6rCJehzn/xaUzQlw5tFl6b6yKBhM48lCJkK5YQ8yugidPnkj3rSw5IlkeMVf0IlLic8IheQFHskK5Rg6z1qTJ0RqnIK7XZU4vXJjsOk4F98Gyy2WtJffMSc/VWdFwM9xPV8KuoyIVq+xG2FGuixLJuLnUWFfaR8gukZFVEbL8vHZUhGyH58GdX5dlcwJ9R2kwrrxZS+UgmdjL+YwSTtmaqKIJ/JyfOS+umfWWy6OjCoT0H+i6/sKSXDNiibpU2S+DdfmElmTMl6bn+RzU8ZxKj+0CTE1Z1kdjYfMyf5vBS+AeHh4ePQr/Avfw8PDoUWy7CWWV1YqvPifq54nXKDrz3W8UEumuCVLVz5ymSMi3v0lMAXlWvVdaop595q8oXeRzr0hCopqLAmMTRqBSdzo1J1DRY87sESv1rMmmjTareEb5Fjc5olGTN1G0vn5jkRPvZOEqZKe7EDMJqJNIdZjwy/ZJFZu1qWfmLkniqrhNqlhdqbe1C5TIa0hVAB/lNKsZrgJTUFmn6qGrMKLtTOvV5lqdzC5v56pI998ryZ7OnyfzxNyiRLI2HTmmyK+IiekCs04jirAcKJX4ynIPLs/SWI7PSlIjw0RUZQeZhQoVITiLTHrqNLVlRUqtRYHvWUuZKRy53FXn0fl/s/mhUpHo4Dz71JdLQsKFPK6iiuZ0JouTr1IitKV5Ue2XOGIyVj7fmSxHhKr1lGN93Ljq9Cqac4aJtlpT1POQxzDYL+upxea2Gjupd1SyrCQ1l+h8qDwfZnMZ8Fvf+rqMpUNVcUqRzEfM666tzCSOSHcJvPSz1GZTlX4eHUHYaEpbnFZ44tTMqv7l0ACZZ8tlXRHKVYjXwzNdf3W1eTfmQJlEIk6SFZj1x7khdIU3GH5/FOX4oMHmP0VQXwteAvfw8PDoUWy7BD48Qvkh5hfk8zfFUWN/y3UnASBu7+Mt+tKN7pQoShPSF/Z7RyUa6y+/RpFUzUS++OAvcRCs/27FLBla9Rl27mFaCnBRlBn+8hv9ueQ8DpqkcrUUde6WkK8fWpYorNIEWIrXYvn4TpIW+ypKaqx1S+A7x4fS7cnzkzwmnTyfts+cOJ42LbF7n7t6VbkpVlnaSeIuppeOV6mEW02S2J77DlW7f0dJxvkAj7PeL9KwI+10lG2DCbYljo7UZOq5VynabbYukYGNDF2/sEPGPLiTJKpchcYUqkjMIrvh5YpCiptw86XvXFXjjtwDF8WbdJQ2xmN3JGZBRSoGrBXWVU6R5jxpg+d1MQaeB5dS1eWbAYTszuSV1M+XaLVk/lYWSOJuNFb5rxDP7k7l1Zpv1zklrapf6ghH91eTh87dr6O0D8tSazazObGeV5HA7ZDvi0oRnWMngUS5njo3yoCvqUnjhPPFaKnfRaQmVkXZ8qitqzupqt474T1QdV2jkFM4NyVyNCU0eXi65mabNWKtVbs1Y7qqzHe/Z1oqqtTyORrq9ZELSVuamNiHrcJL4B4eHh49im2XwJ20mlFZ8joNkp7OTIvU1axScMXb30AVzgsDqno8Fz/45nclI1+dbbdtlQ0ux25cTrrYqEJQqKSB9GOqbGM5ltyME4UCdXyOpIyCKuflXI7aKnBlhaUyFwTRVJJe/yC7UI5LYvgy+yfWVeDF2k/v3iOS6WyZXeqqk7PqCM5Kp9zD5vm6WR5zS9m7xe663k2sKwE/4+RLlH/iwopINqMBzUeXBsNSyaqyt1+2JPWdYpvopMqhUSuyBrNXEuqPHSAJJT8grqTpfWCpqFwWTaDI9vBArTF7FdvtMufZqa2IG+HMJVqTjYb0zZVDc3kw9D12mlyggocyHGjmeBFAMkBGbDPXLoNttgPrfCrNJq2dFeWu5m5bqcLuqUrys22a5+aqqnbPuUGWlMTpJG9nXzbK3p3Y9cFcLjeMSTYvMpKo+7haJR6kGOp7QH9jtZhdwFGL3WI7HeVax4UrrJK2JeujPIcdtoHHTttT99oFMWnh2FrqZ7Ohc8PEXcdrzdymfEys2lwQny6K0n3NsKX7zblnBnWhF9qegJfAPTw8PP7ew7/APTw8PHoU1zShGGPyAL4FqiEQAfistfa3jDEHAHwawDCAZwH8orUqFHKLSEkhTeSFpAq2FMkyvUpqznPHiQh6T01UmhVLpoWLC2JiyLMK3anJORqsMroahpGKknP7utzEjHNDkuNs0J2CNZMTl7BVdr1qqZS0zpyizQjOZFLliNDygJhLBjmXQkulwHyVXcwyyn3qjWu0rMqgEHqjY5SfZEqZUFJ1Tv2myWYSVy9Ru+rFV4mw69rDJ26zCl6dlXwZQY5T9CoXtkt8jRdUZftTEc9HmdTy0h4pCjE6QTlthkelZnaOXfNaqieW1fxcxFXYI00kuzZFMl7FV+vyWXJp1VXCnUptdEQtp7N11cm1+pxlc43OA+P2a4KwwyaD1VWuWdrUOUvYhc1olz5aF1lVfGBs1wSfgyImlxfEbbPDBRqsrkDPN63W0mYVZ55wPm9Yd3xGjd0VWqjVlFlvDS5cEKeCk1PUj5KqcRmx7SfuKjdAc+qiLRNFrGc5V45ucyaXWKcG4nl2JKMu1+vIUW2rcvlU9H1x7q5J7KI0FTnJJseunEeuYIVdHznqftlWeZbiIVoXux4UV+l+d0uvIyXKViTwJoB3WmtfD+AhAI8ZY94C4HcA/J619hCABQAf2vplPTw8PDxuFlupyGMBOL+nDP+zAN4JwJVC/xSA/wjgY9fdA0cO6ET5HGySqLwJLh/JmRn64n/iM19O973zHZTU/cwlkf6qzjlffaMyLpMbSwFF5QaU5UIN9RWRnh3RYBXJmGFC0Ul4mrhykl6iCI86u4zpNnfcAEvNwyoJ/JU5CuRYnJUMiIvnKHjp0MED2AyFvEhkOQ4Yyah8IDGTWfrj3kklEx6f3nkVKaCL0mJpZ5XH96qS6vq53NqrDUl8/zJrJ3MVkUyH99C4xg+QtD2gXCJz7JYYqHwWbV4rYaRKk7HEG6VBLXJ8Kj1rF6+rkJhhwq50ypUzdffT52VtLLBOIpNzNNklstOW9eQkal0R3cGR3ZmsLnnHZfA0CcxrMZ9T7ngF+s38HF1TZxnMsEYZ6urnrG12tLS4hoTrClxxBS6UVrPKRUNqVcmnshaBVeX4nDQai9TqpP2uYKCQ3Qitc9VTmhRLviquKZ17q1wF3Y2w4jOYwknZ2tW3w9dvKxI/4XeQdSXv1POQ5jVSHTFYPxbLZHWHAwYrKp/P7gfJGSMycr8XT3A+qN2ibV4LW61KH3I9zBkAXwHwGoBFK2F6kwB2bfLbJ4wxR40xRzfy+vDw8PDwuDFs6QVurY2ttQ8B2A3gUQD3bPUC1tonrbWPWGsfKarcvh4eHh4eN4fr8gO31i4aY74O4K0ABowxEUvhuwFcvJEODHMl7YZKwF/lSLFsKP7ULs2k8+X95vdeSved4fp8i1VhMuZXSQ1WXCBKrI53WI3KqerqTvXOF1SehcD56Iqq7nxWO2wyMNo/lFWqWFVQb7GfakHlv3BJ5YdGyHTSUgRukwsY1HNyzYSj83TF8rVoq4jJKuez6BuQazaqpDbrggExq3tpBlOVytSs1/JTWJUu1zIBVGUf3W+rIhznatQ2p/I9RGNUoXt892jadmCUtof7aV4CFc1ZZdW0oYioiFV5XbMyz1GWEVcHzxdEWMjx3Osox6sh2SAPh1M2rTLlWGZ/UxONOoeL5Iu1CYDXkV53bo05UrXLipW49SQkcMxkcSsj99ZVqHemk0QTlpw7paG0Xzcuq32h3fHO/KD6EfFYbEuI54U5Mou1W5uvyY7yA4/5uFagCVyXF0cXAeEmfpYCdQ9cythEmzrYzJWo9MuOQHbWDH28M4Fpq03i/LOVycyZjVJTi/bvZjMPNMHqzDDqfdDmtM5Dd1PxiF3796T7GlxP87VXJXal0GZLtQSZXxPXlMCNMaPGmAHeLgD4KQDHAHwdwPv5sMcBfGHrl/Xw8PDwuFlsRQIfB/ApQwkFAgCfsdZ+yRjzCoBPG2P+E4DnAXz8RjrQYKkypz4lTZaAMqFIoR3+ELoE9UFBpLSzTF4GimTpsHTUUQRkgzOuVTkSUhM1TioqZUVKKzCxGSipwRGEhSJdX+ekuMKZ5BLlLhQxgTFYEZJx5xBpHTt3Elm3WBVJZZkz960uSRTgACf2n72iIytHoNFWVdbDLI19cFSu2S7TXHbaKvNb4v4ywakkcDdkHZGXSmearXNEG2fra6scJM1+6vddA0LKDA5R9GS5IkuvXKT7lmOCuKHyjbTY7dAq6Tl07p+6H7ydYU1KuxG6YgWaELNXYWkb7HoXafdR55qmXRF57K6wg15PayVr7gB1VUdK8tw7N75YRTa2eR5CpXm1OZ9GrNxdS03SXJzkrXPVNOssvW9Q+izZIKLW9SPS8839np+W/DttjgjVt2Ad9NA5Z0qQlWtmXDbQuKsCBf+U50qdzroMfkoDzLOGMVgR4tuVUHMFSPSchuzymVMarstz0hV9yvfFRaauLKs8Jrw8k0jmaIlTDUYj0o99R4ioHOTo6ouvnkr3zZ6ijKuR6lv+KnllNsNWvFBeAvDwBu2nQfZwDw8PD49tgI/E9PDw8OhRbHsyK6fi5VTSn6IjMtqiOjo3z4S9kHWCnYTVrU5LkU6xSympiSjaTtKUlfL9Wpgn08W8umaFCwH0qyjHCvuO50HmFVddGgAiVvFCVauxycmPXEEAfVynxrUGayrpz+Icj13Y1zxH/DWuEj0YKvVrYJjMO+WS8gNvsklJmVA6sfMNd76/KjEXf9uDrvSYbBZQyZgiVomLbLLo61MRgpw0v5wTMrrEvuHZnKifLd5cZb/1uiJkHdGaV+pqNnQ+06IGB2vME/q+t5ikymYV6ZTZfC5ddG2gzBQZZ7rT5g/um5uhrqLiaWSeSvYUryeSXSSyK+zQasl9r7PpJK6riEkmMUvKzFToJxW9w+NsN+QcwQY2jtQfXhPaadF42iipGIkq1zZdXhaznrNA6TWzFmFHzTHXnUxUBK4F9TeESqHL2xK1qghIY7v+AkDCyepqkSS+k2hqlw5azTdHSzfa0je31k2XL3naST6TCvXk62uCusKpjUePSKxGwO+q4898l645IybQkO+fLsyxkUnrWvASuIeHh0ePwtgbeOvfKCYmJuwTTzxx267n4eHh8fcBH/3oR5+11j6ytt1L4B4eHh49Cv8C9/Dw8OhR+Be4h4eHR4/Cv8A9PDw8ehS3lcQ0xlwBUAUwe61j73CMoLfH0Ov9B3p/DL3ef6D3x9BL/d9nrR1d23hbX+AAYIw5uhGb2kvo9TH0ev+B3h9Dr/cf6P0x9Hr/AW9C8fDw8OhZ+Be4h4eHR49iO17gT27DNW81en0Mvd5/oPfH0Ov9B3p/DL3e/9tvA/fw8PDwuDXwJhQPDw+PHsVtfYEbYx4zxhw3xpwyxnzkdl77RmCM2WOM+box5hVjzMvGmF/h9iFjzFeMMSf57+B29/Vq4KLUzxtjvsT/P2CM+S7fhz8zxmSvdY7thDFmwBjzWWPMq8aYY8aYt/bgPfh3vIZ+YIz5U2NM/k6+D8aYTxhjZowxP1BtG865Ifw3HsdLxpg3bF/PBZuM4T/zOnrJGPPnrtoY7/sNHsNxY8w/3p5eXx9u2wucK/r8dwDvBnAfgJ83xtx3u65/g+gA+DVr7X0A3gLgl7nPHwHwlLX2MICn+P93Mn4FVAbP4XcA/J619hCABQAf2pZebR2/D+CvrLX3AHg9aCw9cw+MMbsA/FsAj1hrHwDVqvkg7uz78EkAj61p22zO3w3gMP97AsDHblMfr4VPYv0YvgLgAWvt6wCcAPAbAMDP9QcB3M+/+R+mK7/snYnbKYE/CuCUtfa0tbYF4NMA3ncbr3/dsNZOWWuf4+0V0ItjF6jfn+LDPgXg57anh9eGMWY3gJ8G8Af8fwPgnQA+y4fc6f3vB/B2cMk+a23LWruIHroHjAhAwRgTASgCmMIdfB+std8CML+mebM5fx+AP7KEp0EFz8dvT083x0ZjsNb+tZUk7U9DSgi/D8CnrbVNa+0ZAKfQAxXHbucLfBeAC+r/k9zWEzDG7AeVlvsugDFr7RTvugxgbJOf3Qn4rwD+PQCX1X4YwKJaxHf6fTgA4AqAP2Qz0B8YY0rooXtgrb0I4L8AOA96cS8BeBa9dR+Azee8V5/tfwXg//J2T47Bk5hbgDGmDOBzAH7VWrus91ly47kjXXmMMT8DYMZa++x29+UmEAF4A4CPWWsfBqVi6DKX3Mn3AADYVvw+0MdoAkAJ61X7nsKdPufXgjHmN0Em0j/Z7r7cDG7nC/wigD3q/7u57Y6GMSYDenn/ibX289w87VRE/juz2e+3GT8K4L3GmLMgk9U7QfbkAVblgTv/PkwCmLTWfpf//1nQC71X7gEA/CSAM9baK9baNoDPg+5NL90HYPM576ln2xjzLwD8DIBfsOJH3VNjcLidL/BnABxm5j0LIgy+eBuvf91ge/HHARyz1v6u2vVFAI/z9uMAvnC7+7YVWGt/w1q721q7+HwugwAAAUVJREFUHzTfX7PW/gKArwN4Px92x/YfAKy1lwFcMMbczU3vAvAKeuQeMM4DeIsxpshryo2hZ+4DY7M5/yKAX2JvlLcAWFKmljsKxpjHQCbF91pra2rXFwF80BiTM8YcABGy39uOPl4XrLW37R+A94CY39cA/ObtvPYN9vdtIDXxJQAv8L/3gOzITwE4CeCrAIa2u69bGMs7AHyJtw+CFucpAP8bQG67+3eNvj8E4Cjfh78AMNhr9wDARwG8CuAHAP4YQO5Ovg8A/hRkr2+DtKAPbTbnoBLA/52f6++DvG3u1DGcAtm63fP8P9Xxv8ljOA7g3dvd/63885GYHh4eHj0KT2J6eHh49Cj8C9zDw8OjR+Ff4B4eHh49Cv8C9/Dw8OhR+Be4h4eHR4/Cv8A9PDw8ehT+Be7h4eHRo/AvcA8PD48exf8HV/T+BepgTjgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GroundTruth:    cat  ship  ship plane\n"
     ]
    }
   ],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# print images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, now let us see what the neural network thinks these examples above are:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "outputs = net(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The outputs are energies for the 10 classes.\n",
    "The higher the energy for a class, the more the network\n",
    "thinks that the image is of the particular class.\n",
    "So, let's get the index of the highest energy:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted:    cat   car   car plane\n"
     ]
    }
   ],
   "source": [
    "_, predicted = torch.max(outputs, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n",
    "                              for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results seem pretty good.\n",
    "\n",
    "Let us look at how the network performs on the whole dataset.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on the 10000 test images: 51 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That looks way better than chance, which is 10% accuracy (randomly picking\n",
    "a class out of 10 classes).\n",
    "Seems like the network learnt something.\n",
    "\n",
    "Hmmm, what are the classes that performed well, and the classes that did\n",
    "not perform well:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of plane : 62 %\n",
      "Accuracy of   car : 72 %\n",
      "Accuracy of  bird : 35 %\n",
      "Accuracy of   cat : 29 %\n",
      "Accuracy of  deer : 43 %\n",
      "Accuracy of   dog : 36 %\n",
      "Accuracy of  frog : 79 %\n",
      "Accuracy of horse : 69 %\n",
      "Accuracy of  ship : 49 %\n",
      "Accuracy of truck : 36 %\n"
     ]
    }
   ],
   "source": [
    "class_correct = list(0. for i in range(10))\n",
    "class_total = list(0. for i in range(10))\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs, 1)\n",
    "        c = (predicted == labels).squeeze()\n",
    "        for i in range(4):\n",
    "            label = labels[i]\n",
    "            class_correct[label] += c[i].item()\n",
    "            class_total[label] += 1\n",
    "\n",
    "\n",
    "for i in range(10):\n",
    "    print('Accuracy of %5s : %2d %%' % (\n",
    "        classes[i], 100 * class_correct[i] / class_total[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, so what next?\n",
    "\n",
    "How do we run these neural networks on the GPU?\n",
    "\n",
    "Training on GPU\n",
    "----------------\n",
    "Just like how you transfer a Tensor onto the GPU, you transfer the neural\n",
    "net onto the GPU.\n",
    "\n",
    "Let's first define our device as the first visible cuda device if we have\n",
    "CUDA available:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cpu\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "# Assuming that we are on a CUDA machine, this should print a CUDA device:\n",
    "\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The rest of this section assumes that ``device`` is a CUDA device.\n",
    "\n",
    "Then these methods will recursively go over all modules and convert their\n",
    "parameters and buffers to CUDA tensors:\n",
    "\n",
    ".. code:: python\n",
    "\n",
    "    net.to(device)\n",
    "\n",
    "\n",
    "Remember that you will have to send the inputs and targets at every step\n",
    "to the GPU too:\n",
    "\n",
    ".. code:: python\n",
    "\n",
    "        inputs, labels = data[0].to(device), data[1].to(device)\n",
    "\n",
    "Why dont I notice MASSIVE speedup compared to CPU? Because your network\n",
    "is really small.\n",
    "\n",
    "**Exercise:** Try increasing the width of your network (argument 2 of\n",
    "the first ``nn.Conv2d``, and argument 1 of the second ``nn.Conv2d`` –\n",
    "they need to be the same number), see what kind of speedup you get.\n",
    "\n",
    "**Goals achieved**:\n",
    "\n",
    "- Understanding PyTorch's Tensor library and neural networks at a high level.\n",
    "- Train a small neural network to classify images\n",
    "\n",
    "Training on multiple GPUs\n",
    "-------------------------\n",
    "If you want to see even more MASSIVE speedup using all of your GPUs,\n",
    "please check out :doc:`data_parallel_tutorial`.\n",
    "\n",
    "Where do I go next?\n",
    "-------------------\n",
    "\n",
    "-  :doc:`Train neural nets to play video games </intermediate/reinforcement_q_learning>`\n",
    "-  `Train a state-of-the-art ResNet network on imagenet`_\n",
    "-  `Train a face generator using Generative Adversarial Networks`_\n",
    "-  `Train a word-level language model using Recurrent LSTM networks`_\n",
    "-  `More examples`_\n",
    "-  `More tutorials`_\n",
    "-  `Discuss PyTorch on the Forums`_\n",
    "-  `Chat with other users on Slack`_\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}