Assignment_Session_7_Sentiment_Analysis_using_LSTM_RNN.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Assignment_Session_7_Sentiment Analysis using LSTM RNN.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "Jy8Os4bY17GT",
        "tp5IzBGsPGHs",
        "yrc88naeieS0",
        "XJ6o_79ISSVb",
        "c2YLB7xwsOjt",
        "4BrsJF9ZgzB1",
        "AKdllP3FST4N",
        "lSNeMezG4TUI",
        "MyvpNWVS-wU_",
        "kpkyCiZJ_DCU"
      ],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/woncoh1/END/blob/main/Assignment_Session_7_Sentiment_Analysis_using_LSTM_RNN.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GYiRsFGD6iUC"
      },
      "source": [
        "# Instructions\n",
        "\n",
        "- Download the StanfordSentimentAnalysis Dataset from this [link](http://nlp.stanford.edu/~socherr/stanfordSentimentTreebank.zip) (it might be troubling to download it, so force download on chrome). Use \"**datasetSentences.txt**\" and \"**sentiment_labels.txt**\" files from the zip you just downloaded as your dataset. This dataset contains just over 10,000 pieces of Stanford data from HTML files of Rotten Tomatoes. The sentiments are rated between 1 and 25, where one is the most negative and 25 is the most positive. \n",
        "  - See **0. Download data** and **1. Process data**\n",
        "- Use **back translation**, **`random_swap`**, and **`random_delete`** to augment the data you are training on. \n",
        "  - See *Data Augmentation* section of **1. Process data**\n",
        "- Train your model and try and achieve **60%+ accuracy**. Upload your colab file on git with the the **training logs**. \n",
        "  - After 10 epochs, validation accuracy of **80.58%** and testing accuracy of **79.96%**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jy8Os4bY17GT"
      },
      "source": [
        "# 0. Download data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B757aBudE6aj",
        "outputId": "ee4dad7f-b365-4597-cbdc-7ac61fb9de62"
      },
      "source": [
        "# download and unzip dataset\r\n",
        "\r\n",
        "!wget -O stanford_sentiment_treebank.zip http://nlp.stanford.edu/~socherr/stanfordSentimentTreebank.zip\r\n",
        "!unzip stanford_sentiment_treebank.zip"
      ],
      "execution_count": 221,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-12-10 13:55:43--  http://nlp.stanford.edu/~socherr/stanfordSentimentTreebank.zip\n",
            "Resolving nlp.stanford.edu (nlp.stanford.edu)... 171.64.67.140\n",
            "Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:80... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://nlp.stanford.edu/~socherr/stanfordSentimentTreebank.zip [following]\n",
            "--2020-12-10 13:55:43--  https://nlp.stanford.edu/~socherr/stanfordSentimentTreebank.zip\n",
            "Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 6372817 (6.1M) [application/zip]\n",
            "Saving to: ‘stanford_sentiment_treebank.zip’\n",
            "\n",
            "stanford_sentiment_ 100%[===================>]   6.08M  13.0MB/s    in 0.5s    \n",
            "\n",
            "2020-12-10 13:55:43 (13.0 MB/s) - ‘stanford_sentiment_treebank.zip’ saved [6372817/6372817]\n",
            "\n",
            "Archive:  stanford_sentiment_treebank.zip\n",
            "replace stanfordSentimentTreebank/datasetSentences.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: A\n",
            "  inflating: stanfordSentimentTreebank/datasetSentences.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._datasetSentences.txt  \n",
            "  inflating: stanfordSentimentTreebank/datasetSplit.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._datasetSplit.txt  \n",
            "  inflating: stanfordSentimentTreebank/dictionary.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._dictionary.txt  \n",
            "  inflating: stanfordSentimentTreebank/original_rt_snippets.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._original_rt_snippets.txt  \n",
            "  inflating: stanfordSentimentTreebank/README.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._README.txt  \n",
            "  inflating: stanfordSentimentTreebank/sentiment_labels.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._sentiment_labels.txt  \n",
            "  inflating: stanfordSentimentTreebank/SOStr.txt  \n",
            "  inflating: stanfordSentimentTreebank/STree.txt  \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uCcGsCpG-pfy"
      },
      "source": [
        "# 1. Process data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tp5IzBGsPGHs"
      },
      "source": [
        "## Dataset Preview"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LLyPUMogtn5W"
      },
      "source": [
        "(From README.txt)\n",
        "\n",
        "**Stanford Sentiment Treebank V1.0**\n",
        "\n",
        "This is the dataset of the paper:\n",
        "\n",
        "*Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank\n",
        "Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng and Christopher Potts\n",
        "Conference on Empirical Methods in Natural Language Processing (EMNLP 2013)*\n",
        "\n",
        "If you use this dataset in your research, please cite the above paper.\n",
        "\n",
        "`@incollection{SocherEtAl2013:RNTN,\n",
        "title = {{Parsing With Compositional Vector Grammars}},\n",
        "author = {Richard Socher and Alex Perelygin and Jean Wu and Jason Chuang and Christopher Manning and Andrew Ng and Christopher Potts},\n",
        "booktitle = {{EMNLP}},\n",
        "year = {2013}}`\n",
        "\n",
        "This file includes:\n",
        "1. original_rt_snippets.txt contains 10,605 processed snippets from the original pool of Rotten Tomatoes HTML files. Please note that some snippet may contain multiple sentences.\n",
        "\n",
        "2. dictionary.txt contains all phrases and their IDs, separated by a vertical line |.\n",
        "\n",
        "3. **sentiment_labels.txt** contains all phrase ids and the corresponding sentiment labels, separated by a vertical line.\n",
        "Note that you can recover the 5 classes by mapping the positivity probability using the following cut-offs:  \n",
        "[0, 0.2], (0.2, 0.4], (0.4, 0.6], (0.6, 0.8], (0.8, 1.0]  \n",
        "for very negative, negative, neutral, positive, very positive, respectively.\n",
        "Please note that phrase ids and sentence ids are not the same.\n",
        "\n",
        "4. SOStr.txt and STree.txt encode the structure of the parse trees. \n",
        "STree encodes the trees in a parent pointer format. Each line corresponds to each sentence in the datasetSentences.txt file. The Matlab code of this paper will show you how to read this format if you are not familiar with it.\n",
        "\n",
        "5. **datasetSentences.txt** contains the sentence index, followed by the sentence string separated by a tab. These are the sentences of the train/dev (validation)/test sets.\n",
        "\n",
        "6. datasetSplit.txt contains the sentence index (corresponding to the index in datasetSentences.txt file) followed by the set label separated by a comma:  \n",
        "\t1 = train  \n",
        "\t2 = test  \n",
        "\t3 = dev (validation)  \n",
        "\n",
        "Please note that the datasetSentences.txt file has more sentences/lines than the original_rt_snippet.txt. \n",
        "Each row in the latter represents a snippet as shown on RT, whereas the former is each sub sentence as determined by the Stanford parser.\n",
        "\n",
        "For comparing research and training models, please use the provided train/dev (validation)/test splits."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o1-Yz-5RRFYc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "49c3e0fc-82bc-4f42-c47b-4527511fcdb1"
      },
      "source": [
        "# load and inspect dataset\n",
        "\n",
        "import pandas as pd\n",
        "\n",
        "dataset_sentences = pd.read_csv('stanfordSentimentTreebank/datasetSentences.txt', sep='\\t')\n",
        "print(dataset_sentences.shape, '\\n', dataset_sentences.head())\n",
        "\n",
        "print()\n",
        "\n",
        "sentiment_labels = pd.read_csv('stanfordSentimentTreebank/sentiment_labels.txt', sep='|')                \n",
        "sentiment_labels = sentiment_labels.rename(columns={'phrase ids': 'phrase_id', \n",
        "                                                    'sentiment values': 'sentiment_value'})\n",
        "print(sentiment_labels.shape, '\\n', sentiment_labels.head())"
      ],
      "execution_count": 222,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(11855, 2) \n",
            "    sentence_index                                           sentence\n",
            "0               1  The Rock is destined to be the 21st Century 's...\n",
            "1               2  The gorgeously elaborate continuation of `` Th...\n",
            "2               3                     Effective but too-tepid biopic\n",
            "3               4  If you sometimes like to go to the movies to h...\n",
            "4               5  Emerges as something rare , an issue movie tha...\n",
            "\n",
            "(239232, 2) \n",
            "    phrase_id  sentiment_value\n",
            "0          0          0.50000\n",
            "1          1          0.50000\n",
            "2          2          0.44444\n",
            "3          3          0.50000\n",
            "4          4          0.42708\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zqRsoF6xYdgl",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "269febaf-547a-4ace-cf1a-dccc677aa346"
      },
      "source": [
        "# histogram of sentiment values for phrases\r\n",
        "# [0,   0.2]: very negative\r\n",
        "# (0.2, 0.4]: negative\r\n",
        "# (0.4, 0.6]: neutral\r\n",
        "# (0.6, 0.8]: positive \r\n",
        "# (0.8, 1.0]: very positive\r\n",
        "\r\n",
        "print(sentiment_labels.sentiment_value.hist(bins=5))"
      ],
      "execution_count": 223,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "AxesSubplot(0.125,0.125;0.775x0.755)\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAD4CAYAAAAZ1BptAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAVHklEQVR4nO3df6zd9X3f8eeruCSE/ICE7iqyWc0Ut5sDq0qugCpS54UKLqyKkUYi0FqcjMVaQ7KsQ2vJ+gdTEqRGHWUBpWm94RkiFqCsm62FzEWEq6jTTIDSQSBNuSNNsEdCEgOZw5LU2Xt/nA/tkXs/9vU5957ry3k+pKP7/b6/n8/3fD6cc+/L3x/nkKpCkqTF/NhqD0CSdOIyJCRJXYaEJKnLkJAkdRkSkqSudas9gOV2xhln1MaNG0fq+73vfY9TTz11eQd0gnPO08E5T4dx5vzII498u6p+4sj6Ky4kNm7cyMMPPzxS3/n5ebZs2bK8AzrBOefp4JynwzhzTvK1xeqebpIkdRkSkqQuQ0KS1GVISJK6DAlJUtcxQyLJziTPJfnSUO23kvxpkseS/Ockpw1t+3CShSRfSXLxUH2u1RaSXDdUPyvJg61+V5KTW/1VbX2hbd+4XJOWJC3NUo4kdgFzR9TuA86uqr8L/BnwYYAkm4ErgLe2Pr+T5KQkJwGfBC4BNgNXtrYAHwduqqq3AM8DV7f61cDzrX5TaydJmqBjhkRVfQE4eETtD6vqcFvdB2xoy1uBO6vqB1X1VWABOK89Fqrq6ar6IXAnsDVJgHcA97T+twGXDe3rtrZ8D3Bhay9JmpDl+DDdPwbuasvrGYTGy/a3GsAzR9TPB94EvDAUOMPt17/cp6oOJ3mxtf/2kQNIsh3YDjAzM8P8/PxIEzl06NDIfdcq5zwdnPN0WIk5jxUSSX4DOAzcsTzDGU1V7QB2AMzOztaonzj0E5rT4ZY7dnPjH31vtYcxUbvmXjt1r/M0vrdXYs4jh0SS9wC/CFxYf/W/tzsAnDnUbEOr0al/Bzgtybp2NDHc/uV97U+yDnhDay9JmpCRboFNMgf8GvDOqnppaNMe4Ip2Z9JZwCbgi8BDwKZ2J9PJDC5u72nh8gBweeu/Ddg9tK9tbfly4PPl/2tVkibqmEcSST4DbAHOSLIfuJ7B3UyvAu5r15L3VdU/raonktwNPMngNNQ1VfWjtp8PAHuBk4CdVfVEe4pfB+5M8jHgUeDWVr8V+HSSBQYXzq9YhvlKko7DMUOiqq5cpHzrIrWX298A3LBI/V7g3kXqTzO4++nI+veBdx1rfJKkleMnriVJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXccMiSQ7kzyX5EtDtTcmuS/JU+3n6a2eJDcnWUjyWJJzh/psa+2fSrJtqP62JI+3PjcnydGeQ5I0OUs5ktgFzB1Ruw64v6o2Afe3dYBLgE3tsR34FAz+4APXA+cD5wHXD/3R/xTwvqF+c8d4DknShBwzJKrqC8DBI8pbgdva8m3AZUP122tgH3BakjcDFwP3VdXBqnoeuA+Ya9teX1X7qqqA24/Y12LPIUmakHUj9pupqmfb8jeAmba8HnhmqN3+Vjtaff8i9aM9x1+TZDuDIxdmZmaYn58/zukMHDp0aOS+a9U0znnmFLj2nMOrPYyJmsbX2Tkvj1FD4i9VVSWp5RjMqM9RVTuAHQCzs7O1ZcuWkZ5nfn6eUfuuVdM451vu2M2Nj4/91l9Tds2dOnWv8zS+t1dizqPe3fTNdqqI9vO5Vj8AnDnUbkOrHa2+YZH60Z5DkjQho4bEHuDlO5S2AbuH6le1u5wuAF5sp4z2AhclOb1dsL4I2Nu2fTfJBe2upquO2NdizyFJmpBjHnMn+QywBTgjyX4Gdyn9JnB3kquBrwHvbs3vBS4FFoCXgPcCVNXBJB8FHmrtPlJVL18Mfz+DO6hOAT7XHhzlOSRJE3LMkKiqKzubLlykbQHXdPazE9i5SP1h4OxF6t9Z7DkkSZPjJ64lSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpa6yQSPKrSZ5I8qUkn0ny6iRnJXkwyUKSu5Kc3Nq+qq0vtO0bh/bz4Vb/SpKLh+pzrbaQ5LpxxipJOn4jh0SS9cA/A2ar6mzgJOAK4OPATVX1FuB54OrW5Wrg+Va/qbUjyebW763AHPA7SU5KchLwSeASYDNwZWsrSZqQcU83rQNOSbIOeA3wLPAO4J62/Tbgsra8ta3Ttl+YJK1+Z1X9oKq+CiwA57XHQlU9XVU/BO5sbSVJE7Ju1I5VdSDJvwG+Dvxf4A+BR4AXqupwa7YfWN+W1wPPtL6Hk7wIvKnV9w3terjPM0fUz19sLEm2A9sBZmZmmJ+fH2lOhw4dGrnvWjWNc545Ba495/CxG76CTOPr7JyXx8ghkeR0Bv+yPwt4Afh9BqeLJq6qdgA7AGZnZ2vLli0j7Wd+fp5R+65V0zjnW+7YzY2Pj/zWX5N2zZ06da/zNL63V2LO45xu+gXgq1X1rar6C+APgLcDp7XTTwAbgANt+QBwJkDb/gbgO8P1I/r06pKkCRknJL4OXJDkNe3awoXAk8ADwOWtzTZgd1ve09Zp2z9fVdXqV7S7n84CNgFfBB4CNrW7pU5mcHF7zxjjlSQdp3GuSTyY5B7gj4HDwKMMTvl8Frgzycda7dbW5Vbg00kWgIMM/uhTVU8kuZtBwBwGrqmqHwEk+QCwl8GdUzur6olRxytJOn5jnZitquuB648oP83gzqQj234feFdnPzcANyxSvxe4d5wxSpJG5yeuJUldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqWuskEhyWpJ7kvxpki8n+bkkb0xyX5Kn2s/TW9skuTnJQpLHkpw7tJ9trf1TSbYN1d+W5PHW5+YkGWe8kqTjM+6RxCeA/1ZVfxv4GeDLwHXA/VW1Cbi/rQNcAmxqj+3ApwCSvBG4HjgfOA+4/uVgaW3eN9RvbszxSpKOw8ghkeQNwM8DtwJU1Q+r6gVgK3Bba3YbcFlb3grcXgP7gNOSvBm4GLivqg5W1fPAfcBc2/b6qtpXVQXcPrQvSdIErBuj71nAt4D/kORngEeADwEzVfVsa/MNYKYtrweeGeq/v9WOVt+/SP2vSbKdwdEJMzMzzM/PjzShQ4cOjdx3rZrGOc+cAteec3i1hzFR0/g6O+flMU5IrAPOBT5YVQ8m+QR/dWoJgKqqJDXOAJeiqnYAOwBmZ2dry5YtI+1nfn6eUfuuVdM451vu2M2Nj4/z1l97ds2dOnWv8zS+t1dizuNck9gP7K+qB9v6PQxC45vtVBHt53Nt+wHgzKH+G1rtaPUNi9QlSRMyckhU1TeAZ5L8dCtdCDwJ7AFevkNpG7C7Le8Brmp3OV0AvNhOS+0FLkpyertgfRGwt237bpIL2l1NVw3tS5I0AeMec38QuCPJycDTwHsZBM/dSa4Gvga8u7W9F7gUWABeam2pqoNJPgo81Np9pKoOtuX3A7uAU4DPtYckaULGComq+hNgdpFNFy7StoBrOvvZCexcpP4wcPY4Y5Qkjc5PXEuSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeqari/Vl6bU4wde5D3XfXa1hzFRu+ZOXe0hvCJ4JCFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnL726actP4nT7XnrPaI5DWDo8kJEldY4dEkpOSPJrkv7b1s5I8mGQhyV1JTm71V7X1hbZ949A+PtzqX0ly8VB9rtUWklw37lglScdnOY4kPgR8eWj948BNVfUW4Hng6la/Gni+1W9q7UiyGbgCeCswB/xOC56TgE8ClwCbgStbW0nShIwVEkk2AP8A+PdtPcA7gHtak9uAy9ry1rZO235ha78VuLOqflBVXwUWgPPaY6Gqnq6qHwJ3traSpAkZ98L1vwV+DXhdW38T8EJVHW7r+4H1bXk98AxAVR1O8mJrvx7YN7TP4T7PHFE/f7FBJNkObAeYmZlhfn5+pMkcOnRo5L5r1cwpcO05h4/d8BXEOU+Hafx9Xok5jxwSSX4ReK6qHkmyZfmGdPyqagewA2B2dra2bBltOPPz84zad6265Y7d3Pj4dN3kdu05h53zFNg1d+rU/T6vxN+wcd41bwfemeRS4NXA64FPAKclWdeOJjYAB1r7A8CZwP4k64A3AN8Zqr9suE+vLkmagJGvSVTVh6tqQ1VtZHDh+fNV9Y+AB4DLW7NtwO62vKet07Z/vqqq1a9odz+dBWwCvgg8BGxqd0ud3J5jz6jjlSQdv5U4/vx14M4kHwMeBW5t9VuBTydZAA4y+KNPVT2R5G7gSeAwcE1V/QggyQeAvcBJwM6qemIFxitJ6liWkKiqeWC+LT/N4M6kI9t8H3hXp/8NwA2L1O8F7l2OMUqSjp+fuJYkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSugwJSVKXISFJ6jIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkroMCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqSukUMiyZlJHkjyZJInknyo1d+Y5L4kT7Wfp7d6ktycZCHJY0nOHdrXttb+qSTbhupvS/J463NzkowzWUnS8RnnSOIwcG1VbQYuAK5Jshm4Dri/qjYB97d1gEuATe2xHfgUDEIFuB44HzgPuP7lYGlt3jfUb26M8UqSjtPIIVFVz1bVH7fl/wN8GVgPbAVua81uAy5ry1uB22tgH3BakjcDFwP3VdXBqnoeuA+Ya9teX1X7qqqA24f2JUmagHXLsZMkG4GfBR4EZqrq2bbpG8BMW14PPDPUbX+rHa2+f5H6Ys+/ncHRCTMzM8zPz480j0OHDo3cd62aOQWuPefwag9jopzzdJjG3+eVmPPYIZHktcB/Av55VX13+LJBVVWSGvc5jqWqdgA7AGZnZ2vLli0j7Wd+fp5R+65Vt9yxmxsfX5Z/K6wZ155z2DlPgV1zp07d7/NK/A0b6+6mJD/OICDuqKo/aOVvtlNFtJ/PtfoB4Myh7hta7Wj1DYvUJUkTMs7dTQFuBb5cVb89tGkP8PIdStuA3UP1q9pdThcAL7bTUnuBi5Kc3i5YXwTsbdu+m+SC9lxXDe1LkjQB4xx/vh34ZeDxJH/Sav8K+E3g7iRXA18D3t223QtcCiwALwHvBaiqg0k+CjzU2n2kqg625fcDu4BTgM+1hyRpQkYOiar6I6D3uYULF2lfwDWdfe0Edi5Sfxg4e9QxSpLG4yeuJUldhoQkqWu67omTNDUeP/Ai77nus6s9jInaNXfqsu/TIwlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLm+BHTKNt8xde85qj0DSicwjCUlSlyEhSeoyJCRJXYaEJKnLkJAkdRkSkqQuQ0KS1GVISJK6DAlJUpchIUnqMiQkSV2GhCSpy5CQJHUZEpKkLkNCktRlSEiSuk74kEgyl+QrSRaSXLfa45GkaXJCh0SSk4BPApcAm4Erk2xe3VFJ0vQ4oUMCOA9YqKqnq+qHwJ3A1lUekyRNjVTVao+hK8nlwFxV/ZO2/svA+VX1gSPabQe2t9WfBr4y4lOeAXx7xL5rlXOeDs55Oowz55+sqp84srhuvPGcGKpqB7Bj3P0kebiqZpdhSGuGc54Oznk6rMScT/TTTQeAM4fWN7SaJGkCTvSQeAjYlOSsJCcDVwB7VnlMkjQ1TujTTVV1OMkHgL3AScDOqnpiBZ9y7FNWa5Bzng7OeTos+5xP6AvXkqTVdaKfbpIkrSJDQpLUNZUhcayv+kjyqiR3te0PJtk4+VEuryXM+V8keTLJY0nuT/KTqzHO5bTUr3RJ8g+TVJI1fbvkUuab5N3tdX4iyX+c9BiX2xLe138zyQNJHm3v7UtXY5zLKcnOJM8l+VJne5Lc3P6bPJbk3LGesKqm6sHgAvj/Av4WcDLwP4HNR7R5P/C7bfkK4K7VHvcE5vz3gde05V+Zhjm3dq8DvgDsA2ZXe9wr/BpvAh4FTm/rf2O1xz2BOe8AfqUtbwb+fLXHvQzz/nngXOBLne2XAp8DAlwAPDjO803jkcRSvupjK3BbW74HuDBJJjjG5XbMOVfVA1X1Ulvdx+AzKWvZUr/S5aPAx4HvT3JwK2Ap830f8Mmqeh6gqp6b8BiX21LmXMDr2/IbgP89wfGtiKr6AnDwKE22ArfXwD7gtCRvHvX5pjEk1gPPDK3vb7VF21TVYeBF4E0TGd3KWMqch13N4F8ia9kx59wOw8+sqs9OcmArZCmv8U8BP5XkvyfZl2RuYqNbGUuZ878GfinJfuBe4IOTGdqqOt7f96M6oT8noclL8kvALPD3VnssKynJjwG/DbxnlYcySesYnHLawuBI8QtJzqmqF1Z1VCvrSmBXVd2Y5OeATyc5u6r+32oPbK2YxiOJpXzVx1+2SbKOwWHqdyYyupWxpK83SfILwG8A76yqH0xobCvlWHN+HXA2MJ/kzxmcu92zhi9eL+U13g/sqaq/qKqvAn/GIDTWqqXM+WrgboCq+h/Aqxl8Cd4r2bJ+ndE0hsRSvupjD7CtLV8OfL7aFaE16phzTvKzwO8xCIi1fq4ajjHnqnqxqs6oqo1VtZHBdZh3VtXDqzPcsS3lff1fGBxFkOQMBqefnp7kIJfZUub8deBCgCR/h0FIfGuio5y8PcBV7S6nC4AXq+rZUXc2daebqvNVH0k+AjxcVXuAWxkcli4wuEB0xeqNeHxLnPNvAa8Ffr9do/96Vb1z1QY9piXO+RVjifPdC1yU5EngR8C/rKo1e4S8xDlfC/y7JL/K4CL2e9b4P/hI8hkGYX9Gu9ZyPfDjAFX1uwyuvVwKLAAvAe8d6/nW+H8vSdIKmsbTTZKkJTIkJEldhoQkqcuQkCR1GRKSpC5DQpLUZUhIkrr+PzIgI1lr61Z6AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yrc88naeieS0"
      },
      "source": [
        "## Mappping Sentiments from Phrases to Sentences"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QB87gui4goUR",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "b652a0fc-643a-474c-9c00-1482a2bfc845"
      },
      "source": [
        "# compute sentiment values for sentences using sentiment values of phrases\n",
        "\n",
        "sentence_sentiment = dataset_sentences\n",
        "phrase_sentiment = dict(zip(list(sentiment_labels.phrase_id), \n",
        "                            list(sentiment_labels.sentiment_value)))\n",
        "\n",
        "# STree.txt connects sentences with phrases\n",
        "with open('stanfordSentimentTreebank/STree.txt') as file:\n",
        "    stree = file.readlines()\n",
        "stree = [line.strip().split('|') for line in stree] \n",
        "\n",
        "from statistics import mean\n",
        "\n",
        "# average sentiments per sentence (otherwise long sentences get high sentiments)\n",
        "sentiments = [mean(phrase_sentiment[int(phrase_id)] for phrase_id in phrase_ids) \n",
        "              for phrase_ids in stree]\n",
        "# define neutral sentiment as values within [0.5, 0.55) \n",
        "sentiments = [2 if sentiment >= 0.5 and sentiment < 0.55 else sentiment \n",
        "              for sentiment in sentiments]\n",
        "# all negative sentiment values to 0, all positive sentiment values to 1\n",
        "sentiments = [int(round(sentiment)) \n",
        "              for sentiment in sentiments]\n",
        "# concatenate sentence sentiment values with corresponding sentence texts\n",
        "sentence_sentiment['sentiment_value'] = sentiments\n",
        "\n",
        "sentence_sentiment"
      ],
      "execution_count": 224,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentence_index</th>\n",
              "      <th>sentence</th>\n",
              "      <th>sentiment_value</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>The Rock is destined to be the 21st Century 's...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>The gorgeously elaborate continuation of `` Th...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>Effective but too-tepid biopic</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>If you sometimes like to go to the movies to h...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>Emerges as something rare , an issue movie tha...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11850</th>\n",
              "      <td>11851</td>\n",
              "      <td>A real snooze .</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11851</th>\n",
              "      <td>11852</td>\n",
              "      <td>No surprises .</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11852</th>\n",
              "      <td>11853</td>\n",
              "      <td>We 've seen the hippie-turned-yuppie plot befo...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11853</th>\n",
              "      <td>11854</td>\n",
              "      <td>Her fans walked out muttering words like `` ho...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11854</th>\n",
              "      <td>11855</td>\n",
              "      <td>In this case zero .</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>11855 rows × 3 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "       sentence_index  ... sentiment_value\n",
              "0                   1  ...               0\n",
              "1                   2  ...               0\n",
              "2                   3  ...               0\n",
              "3                   4  ...               0\n",
              "4                   5  ...               0\n",
              "...               ...  ...             ...\n",
              "11850           11851  ...               0\n",
              "11851           11852  ...               0\n",
              "11852           11853  ...               0\n",
              "11853           11854  ...               0\n",
              "11854           11855  ...               0\n",
              "\n",
              "[11855 rows x 3 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 224
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fdYcCShZ11R9",
        "outputId": "478c574e-a404-406e-b480-bb6e621f65b6"
      },
      "source": [
        "# show percentage of each sentiment class\r\n",
        "# 0: negative\r\n",
        "# 1: positive\r\n",
        "# 2: neutral\r\n",
        "\r\n",
        "print((100 * sentence_sentiment.sentiment_value.value_counts(normalize=True)).map('{:0.2f}%'.format))"
      ],
      "execution_count": 225,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0    56.71%\n",
            "1    28.36%\n",
            "2    14.93%\n",
            "Name: sentiment_value, dtype: object\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "uTa3Watw_2Cs",
        "outputId": "9faa1616-e80b-472b-b74b-71b3ccbb67f4"
      },
      "source": [
        "# rename and select columns\r\n",
        "\r\n",
        "sentence_sentiment = sentence_sentiment.rename(columns={'sentence': 'tweets', 'sentiment_value': 'labels'})\r\n",
        "sentence_sentiment = sentence_sentiment[['tweets', 'labels']]\r\n",
        "sentence_sentiment"
      ],
      "execution_count": 226,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tweets</th>\n",
              "      <th>labels</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>The Rock is destined to be the 21st Century 's...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>The gorgeously elaborate continuation of `` Th...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Effective but too-tepid biopic</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>If you sometimes like to go to the movies to h...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Emerges as something rare , an issue movie tha...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11850</th>\n",
              "      <td>A real snooze .</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11851</th>\n",
              "      <td>No surprises .</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11852</th>\n",
              "      <td>We 've seen the hippie-turned-yuppie plot befo...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11853</th>\n",
              "      <td>Her fans walked out muttering words like `` ho...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11854</th>\n",
              "      <td>In this case zero .</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>11855 rows × 2 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                  tweets  labels\n",
              "0      The Rock is destined to be the 21st Century 's...       0\n",
              "1      The gorgeously elaborate continuation of `` Th...       0\n",
              "2                         Effective but too-tepid biopic       0\n",
              "3      If you sometimes like to go to the movies to h...       0\n",
              "4      Emerges as something rare , an issue movie tha...       0\n",
              "...                                                  ...     ...\n",
              "11850                                    A real snooze .       0\n",
              "11851                                     No surprises .       0\n",
              "11852  We 've seen the hippie-turned-yuppie plot befo...       0\n",
              "11853  Her fans walked out muttering words like `` ho...       0\n",
              "11854                                In this case zero .       0\n",
              "\n",
              "[11855 rows x 2 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 226
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XJ6o_79ISSVb"
      },
      "source": [
        "## Defining Fields"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e63g08ijOrf7"
      },
      "source": [
        "Now we shall be defining LABEL as a LabelField, which is a subclass of Field that sets sequential to False (as it’s our numerical category class). TWEET is a standard Field object, where we have decided to use the spaCy tokenizer and convert all the text to lower‐case."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qk8IP4SK1Lrp",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "04541103-8267-438c-8e12-ca3cd3066883"
      },
      "source": [
        "# Import Library\n",
        "import random\n",
        "import torch, torchtext\n",
        "from torchtext import data \n",
        "\n",
        "# Manual Seed\n",
        "SEED = 43\n",
        "torch.manual_seed(SEED)"
      ],
      "execution_count": 227,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<torch._C.Generator at 0x7fb479851ae0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 227
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "u6bKQax2Mf_U"
      },
      "source": [
        "Tweet = data.Field(sequential=True, tokenize='spacy', batch_first=True, include_lengths=True)\n",
        "Label = data.LabelField(tokenize='spacy', is_target=True, batch_first=True, sequential=False)"
      ],
      "execution_count": 228,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mX-lYIe_O7Vy"
      },
      "source": [
        "Having defined those fields, we now need to produce a list that maps them onto the list of rows that are in the CSV:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VawdWq36O6td"
      },
      "source": [
        "fields = [('tweets', Tweet),('labels',Label)]"
      ],
      "execution_count": 229,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZbtZ-Ph2P1xL"
      },
      "source": [
        "Armed with our declared fields, lets convert from pandas to list to torchtext. We could also use TabularDataset to apply that definition to the CSV directly but showing an alternative approach too."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nT-flpH-P1cd"
      },
      "source": [
        "# Creating dataset\n",
        "#twitterDataset = data.TabularDataset(path=\"tweets.csv\", format=\"CSV\", fields=fields, skip_header=True)\n",
        "\n",
        "example = [data.Example.fromlist([sentence_sentiment.tweets[i],sentence_sentiment.labels[i]], fields) for i in range(sentence_sentiment.shape[0])] \n",
        "twitterDataset = data.Dataset(example, fields)"
      ],
      "execution_count": 230,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c2YLB7xwsOjt"
      },
      "source": [
        "## Splitting Datasets"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g6ZnyCPaR08F"
      },
      "source": [
        "Finally, we can split into training, testing, and validation sets by using the split() method:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uPYXyuKhRpBk"
      },
      "source": [
        "(train, valid, test) = twitterDataset.split(split_ratio=[0.6, 0.2, 0.2], random_state=random.seed(SEED))"
      ],
      "execution_count": 231,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ykvsCGQMR6UD",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d5ed5285-7337-44e7-d403-073c842ae772"
      },
      "source": [
        "(len(train), len(valid), len(test))"
      ],
      "execution_count": 232,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(7113, 2371, 2371)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 232
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kix8P2IKSBaV"
      },
      "source": [
        "An example from the dataset:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dUpEOQruR9JL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "53c9aa11-1eeb-473b-9a72-30cbe9150a27"
      },
      "source": [
        "vars(train.examples[10])"
      ],
      "execution_count": 233,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'labels': 2,\n",
              " 'tweets': ['In',\n",
              "  'other',\n",
              "  'words',\n",
              "  ',',\n",
              "  'about',\n",
              "  'as',\n",
              "  'bad',\n",
              "  'a',\n",
              "  'film',\n",
              "  'you',\n",
              "  \"'re\",\n",
              "  'likely',\n",
              "  'to',\n",
              "  'see',\n",
              "  'all',\n",
              "  'year',\n",
              "  '.']}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 233
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4BrsJF9ZgzB1"
      },
      "source": [
        "## Data Augmentation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Eebw67KXiEvs",
        "outputId": "8b3536cb-3dfb-4271-f6a5-579c9bb4059a"
      },
      "source": [
        "# import libraries and prepare sentence text\n",
        "\n",
        "import random\n",
        "# for back translation\n",
        "!pip install google_trans_new\n",
        "import google_trans_new\n",
        "from google_trans_new import google_translator\n",
        "# for word tokenization\n",
        "import nltk\n",
        "nltk.download('punkt')\n",
        "from nltk.tokenize import word_tokenize"
      ],
      "execution_count": 234,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: google_trans_new in /usr/local/lib/python3.6/dist-packages (1.1.9)\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rj3LXBucg5iy"
      },
      "source": [
        "**Back Translation**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6WOGpmwsgxVC"
      },
      "source": [
        "# translate a sentence to a random language,\n",
        "# and translate back to original language\n",
        "\n",
        "def back_translate(sentence, p=0.1):\n",
        "  # do nothing with probability of 1-p\n",
        "  if random.uniform(0,1) > p:\n",
        "    return sentence\n",
        "\n",
        "  # combine tokenized sentence into one string\n",
        "  sentence = ' '.join(sentence)\n",
        "\n",
        "  # instantiate translator\n",
        "  translator = google_translator()\n",
        "\n",
        "  # choose a target language\n",
        "  available_langs = list(google_trans_new.LANGUAGES.keys()) \n",
        "  trans_lang = random.choice(available_langs) \n",
        "  #print(f\"Translating to {google_trans_new.LANGUAGES[trans_lang]}\")\n",
        "\n",
        "  # translate to the target language\n",
        "  translations = translator.translate(sentence, lang_tgt=trans_lang) \n",
        "  #print(translations)\n",
        "\n",
        "  # translate back to original language\n",
        "  translations_en_random = translator.translate(translations, lang_src=trans_lang, lang_tgt='en') \n",
        "  #print(translations_en_random)\n",
        "\n",
        "  # select only one translation\n",
        "  if len(translations_en_random) > 1:\n",
        "    translations_en_random = translations_en_random[0]\n",
        "\n",
        "  return word_tokenize(translations_en_random)"
      ],
      "execution_count": 235,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OrW1g8Umg5_e"
      },
      "source": [
        "**Random Deletion**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ezy-MNzKhFWV"
      },
      "source": [
        "# randomly delete words from a sentence with a given probability\n",
        "\n",
        "def random_deletion(sentence, p=0.5): \n",
        "    # return if single word\n",
        "    if len(sentence) == 1: \n",
        "        return sentence\n",
        "    # delete words\n",
        "    remaining = list(filter(lambda x: random.uniform(0,1) > p, sentence)) \n",
        "    # if nothing left, sample a random word\n",
        "    if len(remaining) == 0: \n",
        "        return [random.choice(sentence)] \n",
        "    else:\n",
        "        return remaining"
      ],
      "execution_count": 236,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iK257gAHg504"
      },
      "source": [
        "**Random Swap**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dCCh5T5OhF03"
      },
      "source": [
        "# randomly swap a pair of words in a sentence for a given # of times\n",
        "\n",
        "def random_swap(sentence, n=5): \n",
        "    if len(sentence) < 2:\n",
        "      return sentence\n",
        "    length = range(len(sentence)) \n",
        "    for _ in range(n):\n",
        "        idx1, idx2 = random.sample(length, 2)\n",
        "        sentence[idx1], sentence[idx2] = sentence[idx2], sentence[idx1] \n",
        "    return sentence"
      ],
      "execution_count": 237,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kxD0Z3HL14lI"
      },
      "source": [
        "**Carry Out Data Augmentation**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iRReGoIp18WM"
      },
      "source": [
        "for example in train.examples: \n",
        "  example.tweets = back_translate(example.tweets, p=0.01)\n",
        "  example.tweets = random_deletion(example.tweets, p=0.1)\n",
        "  example.tweets = random_swap(example.tweets, n=1)"
      ],
      "execution_count": 238,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AKdllP3FST4N"
      },
      "source": [
        "## Building Vocabulary"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SuvWQ-SpSmSz"
      },
      "source": [
        "At this point we would have built a one-hot encoding of each word that is present in the dataset—a rather tedious process. Thankfully, torchtext will do this for us, and will also allow a max_size parameter to be passed in to limit the vocabulary to the most common words. This is normally done to prevent the construction of a huge, memory-hungry model. We don’t want our GPUs too overwhelmed, after all. \n",
        "\n",
        "Let’s limit the vocabulary to a maximum of 10,000 words in our training set:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mx955u93SGeY"
      },
      "source": [
        "MAX_VOCAB_SIZE = 10_000\n",
        "\n",
        "Tweet.build_vocab(train, max_size = MAX_VOCAB_SIZE)\n",
        "Label.build_vocab(train)"
      ],
      "execution_count": 239,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xvyEeEjXTGhX"
      },
      "source": [
        "By default, torchtext will add two more special tokens, <unk> for unknown words and <pad>, a padding token that will be used to pad all our text to roughly the same size to help with efficient batching on the GPU."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rA3tIESdcJdN",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "019cd112-5732-4d61-bad6-83ed3ae2f0f0"
      },
      "source": [
        "print('Size of input vocab : ', len(Tweet.vocab))\n",
        "print('Size of label vocab : ', len(Label.vocab))\n",
        "print('Top 10 words appreared repeatedly :', list(Tweet.vocab.freqs.most_common(10)))\n",
        "print('Labels : ', Label.vocab.stoi)"
      ],
      "execution_count": 240,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size of input vocab :  10002\n",
            "Size of label vocab :  3\n",
            "Top 10 words appreared repeatedly : [('.', 5969), (',', 5273), ('the', 4519), ('and', 3323), ('of', 3244), ('a', 3208), ('to', 2260), ('-', 2036), (\"'s\", 1905), ('is', 1866)]\n",
            "Labels :  defaultdict(<function _default_unk_index at 0x7fb426d8d9d8>, {0: 0, 1: 1, 2: 2})\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rwjD2-ebTeUX"
      },
      "source": [
        "**Lots of stopwords!!**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lSNeMezG4TUI"
      },
      "source": [
        "## Making batches"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sLWW221gTpNs"
      },
      "source": [
        "Now we need to create a data loader to feed into our training loop. Torchtext \n",
        "provides the BucketIterator method that will produce what it calls a Batch, which is almost, but not quite, like the data loader we used on images."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EQqMhMoDUDmn"
      },
      "source": [
        "But at first declare the device we are using."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zfo2QhGJUK4l"
      },
      "source": [
        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": 241,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zK2ORoqdTNsM"
      },
      "source": [
        "train_iterator, valid_iterator, test_iterator = data.BucketIterator.splits(\n",
        "    (train, valid, test), \n",
        "    batch_size = 32, \n",
        "    sort_key = lambda x: len(x.tweets),\n",
        "    sort_within_batch=True, \n",
        "    device = device)"
      ],
      "execution_count": 242,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gg7gTFQO4fby"
      },
      "source": [
        "Save the vocabulary for later use"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "niE9Cc6-2bD_"
      },
      "source": [
        "import os, pickle\n",
        "with open('tokenizer.pkl', 'wb') as tokens: \n",
        "    pickle.dump(Tweet.vocab.stoi, tokens)"
      ],
      "execution_count": 243,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MyvpNWVS-wU_"
      },
      "source": [
        "# 2. Build the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E4PED4HJWH4t"
      },
      "source": [
        "We use the Embedding and LSTM modules in PyTorch to build a simple model for classifying tweets.\n",
        "\n",
        "In this model we create three layers. \n",
        "1. First, the words in our tweets are pushed into an Embedding layer, which we have established as a 300-dimensional vector embedding. \n",
        "2. That’s then fed into a 2 stacked-LSTMs with 100 hidden features (again, we’re compressing down from the 300-dimensional input like we did with images). We are using 2 LSTMs for using the dropout.\n",
        "3. Finally, the output of the LSTM (the final hidden state after processing the incoming tweet) is pushed through a standard fully connected layer with three outputs to correspond to our three possible classes (negative, positive, or neutral)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "43pVRccMT0bT"
      },
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "\n",
        "class classifier(nn.Module):\n",
        "    \n",
        "    # Define all the layers used in model\n",
        "    def __init__(self, vocab_size, embedding_dim, hidden_dim, output_dim, \n",
        "                 n_layers, dropout):\n",
        "        \n",
        "        super().__init__()          \n",
        "        \n",
        "        # Embedding layer\n",
        "        self.embedding = nn.Embedding(vocab_size, embedding_dim)\n",
        "        \n",
        "        # LSTM layer\n",
        "        self.encoder = nn.LSTM(embedding_dim, \n",
        "                               hidden_dim, \n",
        "                               num_layers=n_layers, \n",
        "                               dropout=dropout,\n",
        "                               batch_first=True)\n",
        "        # try using nn.GRU or nn.RNN here and compare their performances\n",
        "        # try bidirectional and compare their performances\n",
        "        \n",
        "        # Dense layer\n",
        "        self.fc = nn.Linear(hidden_dim, output_dim)\n",
        "        \n",
        "    def forward(self, text, text_lengths):\n",
        "        \n",
        "        # text = [batch size, sent_length]\n",
        "        embedded = self.embedding(text)\n",
        "        # embedded = [batch size, sent_len, emb dim]\n",
        "      \n",
        "        # packed sequence\n",
        "        packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, text_lengths.cpu(), batch_first=True)\n",
        "        \n",
        "        packed_output, (hidden, cell) = self.encoder(packed_embedded)\n",
        "        #hidden = [batch size, num layers * num directions,hid dim]\n",
        "        #cell = [batch size, num layers * num directions,hid dim]\n",
        "    \n",
        "        # Hidden = [batch size, hid dim * num directions]\n",
        "        dense_outputs = self.fc(hidden)   \n",
        "        \n",
        "        # Final activation function (softmax)\n",
        "        output = F.softmax(dense_outputs[0], dim=1)\n",
        "            \n",
        "        return output"
      ],
      "execution_count": 244,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rwBoGE_X_Fl8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "be860e18-a22f-472f-da9b-1266989a37bf"
      },
      "source": [
        "# Define hyperparameters\n",
        "size_of_vocab = len(Tweet.vocab)\n",
        "embedding_dim = 300\n",
        "num_hidden_nodes = 100\n",
        "num_output_nodes = 3\n",
        "num_layers = 2\n",
        "dropout = 0.2\n",
        "\n",
        "# Instantiate the model\n",
        "model = classifier(size_of_vocab, embedding_dim, num_hidden_nodes, num_output_nodes, num_layers, dropout = dropout)\n",
        "\n",
        "print(model)"
      ],
      "execution_count": 245,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "classifier(\n",
            "  (embedding): Embedding(10002, 300)\n",
            "  (encoder): LSTM(300, 100, num_layers=2, batch_first=True, dropout=0.2)\n",
            "  (fc): Linear(in_features=100, out_features=3, bias=True)\n",
            ")\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O-pOMqzJ3eTv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "74a53844-0175-486c-ef89-6c219631e454"
      },
      "source": [
        "# No. of trianable parameters\n",
        "def count_parameters(model):\n",
        "    return sum(p.numel() for p in model.parameters() if p.requires_grad)\n",
        "    \n",
        "print(f'The model has {count_parameters(model):,} trainable parameters')"
      ],
      "execution_count": 246,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The model has 3,242,503 trainable parameters\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kpkyCiZJ_DCU"
      },
      "source": [
        "# 3. Train, evaluate, and test the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eXajorf5Xz7t"
      },
      "source": [
        "## Model Training and Evaluation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PrE9RpMtZ1Vs"
      },
      "source": [
        "**Preparation**  \n",
        "\n",
        "First define the optimizer and loss function, and then push the two into cuda (if available).  \n",
        "\n",
        "Also define an accuracy metric."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-u86JWdlXvu5"
      },
      "source": [
        "import torch.optim as optim\n",
        "\n",
        "# define optimizer and loss\n",
        "optimizer = optim.Adam(model.parameters(), lr=2e-4)\n",
        "criterion = nn.CrossEntropyLoss()\n",
        "\n",
        "# push to cuda if available\n",
        "model = model.to(device)\n",
        "criterion = criterion.to(device)\n",
        "\n",
        "# define metric\n",
        "def binary_accuracy(preds, y):\n",
        "    #round predictions to the closest integer\n",
        "    _, predictions = torch.max(preds, 1)\n",
        "    correct = (predictions == y).float() \n",
        "    acc = correct.sum() / len(correct)\n",
        "    return acc"
      ],
      "execution_count": 247,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3VCJtNb3Zt8w"
      },
      "source": [
        "The main thing to be aware of in this new training loop is that we have to reference `batch.tweets` and `batch.labels` to get the particular fields we’re interested in; they don’t fall out quite as nicely from the enumerator as they do in torchvision."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2WjEPLKsAiS_"
      },
      "source": [
        "**Training Loop**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HDWNnGK3Y5oJ"
      },
      "source": [
        "def train(model, iterator, optimizer, criterion):\n",
        "    \n",
        "    # initialize every epoch \n",
        "    epoch_loss = 0\n",
        "    epoch_acc = 0\n",
        "    \n",
        "    # set the model in training phase\n",
        "    model.train()  \n",
        "    \n",
        "    for batch in iterator:\n",
        "        \n",
        "        # resets the gradients after every batch\n",
        "        optimizer.zero_grad()   \n",
        "        \n",
        "        # retrieve text and no. of words\n",
        "        tweet, tweet_lengths = batch.tweets   \n",
        "        \n",
        "        # convert to 1D tensor\n",
        "        predictions = model(tweet, tweet_lengths).squeeze()  \n",
        "        \n",
        "        # compute the loss\n",
        "        loss = criterion(predictions, batch.labels)        \n",
        "        \n",
        "        # compute the binary accuracy\n",
        "        acc = binary_accuracy(predictions, batch.labels)   \n",
        "        \n",
        "        # backpropage the loss and compute the gradients\n",
        "        loss.backward()       \n",
        "        \n",
        "        # update the weights\n",
        "        optimizer.step()      \n",
        "        \n",
        "        # loss and accuracy\n",
        "        epoch_loss += loss.item()  \n",
        "        epoch_acc += acc.item()    \n",
        "        \n",
        "    return epoch_loss / len(iterator), epoch_acc / len(iterator)"
      ],
      "execution_count": 248,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CZcHhkkvAsCt"
      },
      "source": [
        "**Evaluation Loop**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zHEe-zSVAriL"
      },
      "source": [
        "def evaluate(model, iterator, criterion):\n",
        "    \n",
        "    # initialize every epoch\n",
        "    epoch_loss = 0\n",
        "    epoch_acc = 0\n",
        "\n",
        "    # deactivating dropout layers\n",
        "    model.eval()\n",
        "    \n",
        "    # deactivates autograd\n",
        "    with torch.no_grad():\n",
        "    \n",
        "        for batch in iterator:\n",
        "        \n",
        "            # retrieve text and no. of words\n",
        "            tweet, tweet_lengths = batch.tweets\n",
        "            \n",
        "            # convert to 1d tensor\n",
        "            predictions = model(tweet, tweet_lengths).squeeze()\n",
        "            \n",
        "            # compute loss and accuracy\n",
        "            loss = criterion(predictions, batch.labels)\n",
        "            acc = binary_accuracy(predictions, batch.labels)\n",
        "            \n",
        "            # keep track of loss and accuracy\n",
        "            epoch_loss += loss.item()\n",
        "            epoch_acc += acc.item()\n",
        "        \n",
        "    return epoch_loss / len(iterator), epoch_acc / len(iterator)"
      ],
      "execution_count": 249,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L6LJFW7HaJoV"
      },
      "source": [
        "**Let's Train and Evaluate**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tq330XlnaEU9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "494b71be-28ab-4907-bf43-ce1ac8cbf5d0"
      },
      "source": [
        "N_EPOCHS = 10\n",
        "best_valid_loss = float('inf')\n",
        "\n",
        "for epoch in range(N_EPOCHS):\n",
        "     \n",
        "    # train the model\n",
        "    train_loss, train_acc = train(model, train_iterator, optimizer, criterion)\n",
        "    \n",
        "    # evaluate the model\n",
        "    valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)\n",
        "    \n",
        "    # save the best model\n",
        "    if valid_loss < best_valid_loss:\n",
        "        best_valid_loss = valid_loss\n",
        "        torch.save(model.state_dict(), 'saved_weights.pt')\n",
        "    \n",
        "    print(f'\\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc*100:.2f}%')\n",
        "    print(f'\\t Val. Loss: {valid_loss:.3f} |  Val. Acc: {valid_acc*100:.2f}% \\n')"
      ],
      "execution_count": 250,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\tTrain Loss: 1.011 | Train Acc: 54.40%\n",
            "\t Val. Loss: 0.989 |  Val. Acc: 56.21% \n",
            "\n",
            "\tTrain Loss: 0.977 | Train Acc: 56.70%\n",
            "\t Val. Loss: 0.976 |  Val. Acc: 56.38% \n",
            "\n",
            "\tTrain Loss: 0.900 | Train Acc: 62.97%\n",
            "\t Val. Loss: 0.845 |  Val. Acc: 70.12% \n",
            "\n",
            "\tTrain Loss: 0.806 | Train Acc: 74.52%\n",
            "\t Val. Loss: 0.810 |  Val. Acc: 73.67% \n",
            "\n",
            "\tTrain Loss: 0.779 | Train Acc: 76.99%\n",
            "\t Val. Loss: 0.837 |  Val. Acc: 70.12% \n",
            "\n",
            "\tTrain Loss: 0.763 | Train Acc: 78.24%\n",
            "\t Val. Loss: 0.787 |  Val. Acc: 75.83% \n",
            "\n",
            "\tTrain Loss: 0.750 | Train Acc: 79.56%\n",
            "\t Val. Loss: 0.795 |  Val. Acc: 73.79% \n",
            "\n",
            "\tTrain Loss: 0.735 | Train Acc: 82.14%\n",
            "\t Val. Loss: 0.767 |  Val. Acc: 78.42% \n",
            "\n",
            "\tTrain Loss: 0.721 | Train Acc: 83.73%\n",
            "\t Val. Loss: 0.756 |  Val. Acc: 79.71% \n",
            "\n",
            "\tTrain Loss: 0.712 | Train Acc: 84.32%\n",
            "\t Val. Loss: 0.747 |  Val. Acc: 80.58% \n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "628NJEc838Wk",
        "outputId": "07128d86-3d36-4656-e08b-a25a14f3b733"
      },
      "source": [
        "# test model using testing dataset\n",
        "\n",
        "model.load_state_dict(torch.load('saved_weights.pt'))\n",
        "\n",
        "test_loss, test_acc = evaluate(model, test_iterator, criterion)\n",
        "\n",
        "print(f'Test Loss: {test_loss:.3f} | Test Acc: {test_acc*100:.2f}%')"
      ],
      "execution_count": 251,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test Loss: 0.753 | Test Acc: 79.96%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LZgzB0ZkHVTI"
      },
      "source": [
        "## Model Testing with User Input"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aZZfnWo0abRx"
      },
      "source": [
        "# load weights and tokenizer\n",
        "path='./saved_weights.pt'\n",
        "model.load_state_dict(torch.load(path));\n",
        "model.eval();\n",
        "tokenizer_file = open('./tokenizer.pkl', 'rb')\n",
        "tokenizer = pickle.load(tokenizer_file)\n",
        "\n",
        "# inference \n",
        "import spacy\n",
        "nlp = spacy.load('en')\n",
        "\n",
        "def classify_tweet(tweet):\n",
        "    \n",
        "    categories = {0: \"Negative\", 1:\"Positive\", 2:\"Neutral\"}\n",
        "    \n",
        "    # tokenize the tweet \n",
        "    tokenized = [tok.text for tok in nlp.tokenizer(tweet)] \n",
        "    # convert to integer sequence using predefined tokenizer dictionary\n",
        "    indexed = [tokenizer[t] for t in tokenized]        \n",
        "    # compute no. of words        \n",
        "    length = [len(indexed)]\n",
        "    # convert to tensor                                    \n",
        "    tensor = torch.LongTensor(indexed).to(device)   \n",
        "    # reshape in form of batch, no. of words           \n",
        "    tensor = tensor.unsqueeze(1).T  \n",
        "    # convert to tensor                          \n",
        "    length_tensor = torch.LongTensor(length)\n",
        "    # Get the model prediction                  \n",
        "    prediction = model(tensor, length_tensor)\n",
        "\n",
        "    _, pred = torch.max(prediction, 1) \n",
        "    \n",
        "    return categories[pred.item()]"
      ],
      "execution_count": 252,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yTkHLEipIlM9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "9ffb419e-d6b3-4dbe-c522-448e7e04dbec"
      },
      "source": [
        "classify_tweet(\"A valid explanation for why Trump won't let women on the golf course.\")"
      ],
      "execution_count": 253,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Positive'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 253
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "1RkOoFFhEula",
        "outputId": "d055bfae-6145-4c6d-9924-aea82a1d43f6"
      },
      "source": [
        "classify_tweet(\"Obama has called the GOP budget social Darwinism. Nice try, but they believe in social creationism.\")"
      ],
      "execution_count": 254,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Negative'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 254
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "HJXzaXE5FNCa",
        "outputId": "92dbf8b8-971e-4d53-9c32-09c13a475a78"
      },
      "source": [
        "classify_tweet(\"In his teen years, Obama has been known to use marijuana and cocaine.\")"
      ],
      "execution_count": 255,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Negative'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 255
        }
      ]
    }
  ]
}