Omicron_Sentiment_Analysis_using_Python.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "private_outputs": true,
      "provenance": [],
      "authorship_tag": "ABX9TyNBbU31D6qrtzGHztXfjRw4",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/1MuhammadFarhanAslam/ML-Projects/blob/main/Omicron_Sentiment_Analysis_using_Python.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Mounting Google Drive**"
      ],
      "metadata": {
        "id": "HSQWpQZsk-R5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "\n",
        "# Mount Google Drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "id": "wOfmrnpflItv"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Configure Google Colab to Kaggle through Kaggle API**\n",
        "\n",
        "**To connect Kaggle datasets to Google Colab, you need to follow these steps:**\n",
        "\n",
        "* 1: Install the Kaggle library in Google Colab by running the following command"
      ],
      "metadata": {
        "id": "C4_DSRkLlK9x"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install kaggle"
      ],
      "metadata": {
        "id": "3Ak676wKlPVC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Go to the Kaggle website (https://www.kaggle.com) and sign in to your account (or create a new account if you don't have one).**\n",
        "\n",
        "*Navigate to the dataset you want to use in your Colab notebook.*\n",
        "\n",
        "*Click on the \"Copy API command\" button below the dataset description. This will copy the command to download the dataset using the Kaggle API.*\n",
        "\n",
        "*In your Colab notebook, import the necessary libraries and set up the Kaggle API by running the following code*"
      ],
      "metadata": {
        "id": "LWdhUoRflSsy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "import json\n",
        "\n",
        "# Upload your Kaggle API key file (kaggle.json) to Colab using the file upload feature\n",
        "from google.colab import files\n",
        "files.upload()\n",
        "\n",
        "# Read the contents of the kaggle.json file\n",
        "with open('kaggle.json', 'r') as file:\n",
        "    kaggle_json = json.load(file)"
      ],
      "metadata": {
        "id": "tijhOtQ4lTPy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Important about Kaggle API Security**\n",
        "\n",
        "**The command !chmod 600 ~/.kaggle/kaggle.json is used to change the permissions of the kaggle.json file to restrict access permissions.**\n",
        "\n",
        "*In Linux-based systems, including Google Colab, file permissions are represented by a three-digit number: the first digit represents the owner's permissions, the second digit represents the group's permissions, and the third digit represents other users' permissions.*\n",
        "\n",
        "**Here's a breakdown of what chmod 600 does:**\n",
        "\n",
        "* ***6 means the owner (the user who uploaded the kaggle.json file) has read and write permissions (4 for read and 2 for write), but no execute permissions (0 for execute). 0 means the group and other users have no permissions to read, write, or execute the file.***\n",
        "\n",
        "* ***By setting the permissions to chmod 600, it ensures that only the owner of the file (the user who uploaded the kaggle.json file) has read and write access, and no other users (group or others) can access or modify the file.***\n",
        "\n",
        "* **This step is important to maintain the security of your Kaggle API key, as it contains sensitive information and should not be accessible to other users of the system.**"
      ],
      "metadata": {
        "id": "6bvDraumlY4g"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Move the saved kaggle.json file to the required directory\n",
        "os.makedirs('/root/.kaggle', exist_ok=True)\n",
        "os.rename('kaggle.json', '/root/.kaggle/kaggle.json')\n",
        "\n",
        "# Set the appropriate permissions for the Kaggle API key file\n",
        "os.chmod('/root/.kaggle/kaggle.json', 0o600)"
      ],
      "metadata": {
        "id": "p4thHnsclPlE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**or**"
      ],
      "metadata": {
        "id": "uO18Nn1AlfHU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "\n",
        "# Specify the path to the kaggle.json file\n",
        "kaggle_json_path = os.path.join(os.path.expanduser(\"~\"), \".kaggle\", \"kaggle.json\")\n",
        "\n",
        "# Check if the kaggle.json file already exists\n",
        "if os.path.exists(kaggle_json_path):\n",
        "    print(\"kaggle.json file already exists.\")\n",
        "else:\n",
        "    # Move the uploaded Kaggle API key file to the required directory\n",
        "    !mkdir -p ~/.kaggle    # This command creates a directory named '.kaggle' inside the user's home directory (~). The -p option ensures that the parent directories are also created if they don't exist. If the directory already exists, this command will not throw an error\n",
        "    !mv kaggle.json ~/.kaggle/    # This command moves the file named 'kaggle.json' to the ~/.kaggle/ directory. The mv command is used for file or directory relocation. The first argument, kaggle.json, represents the current name/path of the file, and the second argument, ~/.kaggle/, represents the destination directory where the file should be moved.\n",
        "    !chmod 600 ~/.kaggle/kaggle.json\n",
        "    print(\"kaggle.json file moved and permissions set successfully.\")\n"
      ],
      "metadata": {
        "id": "0PvL_7YvlfWG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Verifying Kaggle API**"
      ],
      "metadata": {
        "id": "gta_VUoWlk1H"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Verify the Kaggle API is working\n",
        "!kaggle datasets list"
      ],
      "metadata": {
        "id": "qDaQynhjllt3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Downloading dataset from URL**"
      ],
      "metadata": {
        "id": "R6ANn-w6loy9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!kaggle datasets download --force gpreda/omicron-rising"
      ],
      "metadata": {
        "id": "iJT9ofnUlqtp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**If the Kaggle API is working correctly, you can download the dataset by running the copied API command in your Colab notebook:**\n",
        "\n",
        "* **The -d flag is useful if you want to download the dataset only once. If you use the -d flag and the dataset already exists in your local directory, Kaggle will not download the dataset again.In your case, the dataset is being updated daily, so you may want to use the --force flag to make sure that you always have the latest version of the dataset.**"
      ],
      "metadata": {
        "id": "OK1LqLUZltYR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**The dataset will be downloaded as a ZIP file. You can unzip the file using the following command**"
      ],
      "metadata": {
        "id": "G7U0UJ4hlxQr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import zipfile\n",
        "\n",
        "# Specify the path to the ZIP file\n",
        "zip_file_path = 'omicron-rising.zip'\n",
        "\n",
        "# creating directory to unzip dataset\n",
        "!mkdir -p omicron-rising\n",
        "\n",
        "# Specify the target directory to extract the files\n",
        "target_directory = 'omicron-rising'\n",
        "\n",
        "# Open the ZIP file\n",
        "with zipfile.ZipFile(zip_file_path, 'r') as zip_ref:\n",
        "    # Extract all the files to the target directory\n",
        "    zip_ref.extractall(target_directory)\n",
        "\n",
        "print(\"ZIP file extracted successfully.\")"
      ],
      "metadata": {
        "id": "nvxTonYLlxxP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "\n",
        "# Specify the directory path\n",
        "directory_path = 'omicron-rising'\n",
        "\n",
        "# Create the directory if it doesn't already exist\n",
        "if not os.path.exists(directory_path):\n",
        "    os.makedirs(directory_path)\n",
        "    print(f\"Directory '{directory_path}' created successfully.\")\n",
        "else:\n",
        "    print(f\"Directory '{directory_path}' already exists.\")\n"
      ],
      "metadata": {
        "id": "DUsqO0dul2YK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "from nltk.sentiment.vader import SentimentIntensityAnalyzer\n",
        "from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator\n",
        "\n",
        "\n",
        "data = pd.read_csv('/content/omicron-rising/omicron.csv')\n",
        "data"
      ],
      "metadata": {
        "id": "YULqwQVeIc52"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.shape"
      ],
      "metadata": {
        "id": "k_nh7aG9JeUC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.info()"
      ],
      "metadata": {
        "id": "RJpAq2cYMxHL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.describe()"
      ],
      "metadata": {
        "id": "ew7t98_Gk4HP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.isnull().sum()"
      ],
      "metadata": {
        "id": "8r6STQMOk7O3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The dataset contains null values in three columns that contains textual data, I will remove all the rows containing the null values:"
      ],
      "metadata": {
        "id": "9WSml2pqmFFO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data = data.dropna()"
      ],
      "metadata": {
        "id": "trm8gzjTlTYh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.isnull().sum()"
      ],
      "metadata": {
        "id": "aSrDwTyImKKO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Sentiment Analysis of Omicron Variant**\n",
        "\n",
        "The text column in the dataset contains the tweets done by people to share their opinions about the Omicron variant. To move further, we need to clean and prepare this column for the task of sentiment analysis. Here’s how we can do that:"
      ],
      "metadata": {
        "id": "yCaJpO-8mdZd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "import re\n",
        "from nltk.corpus import stopwords\n",
        "import string\n",
        "\n",
        "nltk.download('stopwords')\n",
        "stemmer = nltk.SnowballStemmer(\"english\")\n",
        "stopword=set(stopwords.words('english'))"
      ],
      "metadata": {
        "id": "GUajRBuVmRze"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def clean(text):\n",
        "    text = str(text).lower()\n",
        "    text = re.sub('\\[.*?\\]', '', text)\n",
        "    text = re.sub('https?://\\S+|www\\.\\S+', '', text)\n",
        "    text = re.sub('<.*?>+', '', text)\n",
        "    text = re.sub('[%s]' % re.escape(string.punctuation), '', text)\n",
        "    text = re.sub('\\n', '', text)\n",
        "    text = re.sub('\\w*\\d\\w*', '', text)\n",
        "    text = [word for word in text.split(' ') if word not in stopword]\n",
        "    text=\" \".join(text)\n",
        "    text = [stemmer.stem(word) for word in text.split(' ')]\n",
        "    text=\" \".join(text)\n",
        "    return text\n",
        "\n",
        "data[\"text\"] = data[\"text\"].apply(clean)"
      ],
      "metadata": {
        "id": "NS6P_2xNqdl7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.tail(5)"
      ],
      "metadata": {
        "id": "sUr_LsrTq7QE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "As we have cleaned the text column, now let’s have a look at the word cloud of the text column to look at the most number of words used by the people on their tweets:"
      ],
      "metadata": {
        "id": "q4Z1OiS9rPeR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for i in data.text:\n",
        "  print(i)"
      ],
      "metadata": {
        "id": "ipg1wKkVq7-S"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "text = \" \".join(i for i in data.text)\n",
        "text"
      ],
      "metadata": {
        "id": "XYw-c7uGsFJb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "As we have cleaned the text column, now let’s have a look at the word cloud of the text column to look at the most number of words used by the people on their tweets:"
      ],
      "metadata": {
        "id": "pwJYeJX7wqNE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text = \" \".join(i for i in data.text)\n",
        "stopwords = set(STOPWORDS)\n",
        "wordcloud = WordCloud(width=800, height=500, random_state=21, max_font_size=110, stopwords=stopwords, background_color=\"black\").generate(text)\n",
        "plt.figure( figsize=(14,10))\n",
        "plt.imshow(wordcloud, interpolation='bilinear')\n",
        "plt.axis(\"off\")\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "hPa_USI7wtKp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now let’s have a look at the word cloud of the hashtags column to look at the most number of hashtags used by the people on their tweets:"
      ],
      "metadata": {
        "id": "U1XflVQ-wAFf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text = \" \".join(i for i in data.hashtags)\n",
        "stopwords = set(STOPWORDS)\n",
        "\n",
        "wordcloud = WordCloud(width=800, height=500, random_state=21, max_font_size=110, stopwords=stopwords, background_color=\"black\").generate(text)\n",
        "plt.figure( figsize=(14,10))\n",
        "plt.imshow(wordcloud, interpolation='bilinear')\n",
        "plt.axis(\"off\")\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "OHpT04M0wPjI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now I will calculate the sentiment scores of the tweets about the Omicron variant. Here I will add three more columns in this dataset as Positive, Negative, and Neutral by calculating the sentiment scores of the text column:"
      ],
      "metadata": {
        "id": "w6J8Skdxw-yP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "nltk.download('vader_lexicon')\n",
        "sentiments = SentimentIntensityAnalyzer()\n",
        "data[\"Positive\"] = [sentiments.polarity_scores(i)[\"pos\"] for i in data[\"text\"]]\n",
        "data[\"Negative\"] = [sentiments.polarity_scores(i)[\"neg\"] for i in data[\"text\"]]\n",
        "data[\"Neutral\"] = [sentiments.polarity_scores(i)[\"neu\"] for i in data[\"text\"]]\n",
        "data = data[[\"text\", \"Positive\", \"Negative\", \"Neutral\"]]\n",
        "print(data.head())"
      ],
      "metadata": {
        "id": "7x5bkaIQwYFd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now let’s see how most of the people reacted about the Omicron variant:"
      ],
      "metadata": {
        "id": "7urGWW27yhfr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = sum(data[\"Positive\"])\n",
        "y = sum(data[\"Negative\"])\n",
        "z = sum(data[\"Neutral\"])\n",
        "\n",
        "def sentiment_score(a, b, c):\n",
        "    if (a>b) and (a>c):\n",
        "        print(\"1 😊 \")\n",
        "    elif (b>a) and (b>c):\n",
        "        print(\"-1 😠 \")\n",
        "    else:\n",
        "        print(\"0 🙂 \")\n",
        "\n",
        "sentiment_score(x, y, z)"
      ],
      "metadata": {
        "id": "rCIhQHJIyh8A"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "So most of the opinions were Neutral, which means that people were sharing information about the Omicron variant instead of sharing any positive or negative opinions."
      ],
      "metadata": {
        "id": "ryfpS-q6y5wj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the highest polarity score for each row\n",
        "data[\"Max_Sentiment\"] = data[[\"Positive\", \"Negative\", \"Neutral\"]].max(axis=1)\n",
        "\n",
        "# Get the sentiment label\n",
        "def get_sentiment_label(sentiment):\n",
        "  if sentiment == \"Positive\":\n",
        "    return \"Positive\"\n",
        "  elif sentiment == \"Negative\":\n",
        "    return \"Negative\"\n",
        "  else:\n",
        "    return \"Neutral\"\n",
        "\n",
        "data[\"SentimentLabel\"] = data[\"Max_Sentiment\"].map(get_sentiment_label)\n",
        "data"
      ],
      "metadata": {
        "id": "RF8NlOHmy6Ib"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data[data['SentimentLabel']  == \"Neutral\" ]"
      ],
      "metadata": {
        "id": "YSGjScusDksb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data[data['SentimentLabel']  == \"Negative\" ]"
      ],
      "metadata": {
        "id": "Hpzhn-5BEFEb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Creating dataframes containing neutral,positive and negative sentiments"
      ],
      "metadata": {
        "id": "ij4lGATrBZ7u"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "neutral = data[data['SentimentLabel'] == \"Neutral\"]\n",
        "positive = data[data['SentimentLabel'] == \"Positive\"]\n",
        "negative = data[data['SentimentLabel'] == \"Negative\"]"
      ],
      "metadata": {
        "id": "KRllXyaXBcqU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "neutral"
      ],
      "metadata": {
        "id": "6bzmaFBFCDS1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "positive"
      ],
      "metadata": {
        "id": "wPKClGuDCHsa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "negative"
      ],
      "metadata": {
        "id": "0HKLvKJ0CO9e"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Visualization**"
      ],
      "metadata": {
        "id": "j_SdhpJkoIeq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# This code creates a bar chart showing the distribution of sentiment in the dataset of tweets.\n",
        "import plotly.graph_objs as go\n",
        "\n",
        "# Create a list of x-axis labels.\n",
        "x = ['Neutral', 'Positive', 'Negative']\n",
        "\n",
        "# Create a list of y-axis values.\n",
        "y = [len(neutral), len(positive), len(negative)]\n",
        "\n",
        "# Create a bar chart object.\n",
        "fig = go.Figure(data=[go.Bar(x=x, y=y, hovertext=['100% of tweets', '0% of tweets', '0% of tweets'])])\n",
        "\n",
        "# Customize the aspect of the bar chart.\n",
        "fig.update_traces(marker_line_color='midnightblue', marker_line_width=1.)\n",
        "\n",
        "# Set the title of the bar chart.\n",
        "fig.update_layout(title_text='Distribution of sentiments')\n",
        "\n",
        "# Display the bar chart.\n",
        "fig.show()"
      ],
      "metadata": {
        "id": "VD0J6h80_YFC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data['SentimentLabel'].value_counts()"
      ],
      "metadata": {
        "id": "lhJa17iTJc5e"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import plotly.graph_objs as go\n",
        "\n",
        "# Create a list of x-axis labels.\n",
        "x = ['Neutral', 'Positive', 'Negative']\n",
        "\n",
        "# Create a list of y-axis values.\n",
        "y = [len(neutral), len(positive), len(negative)]\n",
        "\n",
        "# Create a pie chart object.\n",
        "fig = go.Figure(data=[go.Pie(labels=x, values=y)])\n",
        "\n",
        "# Customize the aspect of the pie chart.\n",
        "fig.update_traces(hole=0.6, marker_line_color='midnightblue', marker_line_width=1.)\n",
        "\n",
        "# Set the title of the pie chart.\n",
        "fig.update_layout(title_text='Distribution of sentiments')\n",
        "\n",
        "# Display the pie chart.\n",
        "fig.show()\n"
      ],
      "metadata": {
        "id": "yshLKz8KJi7l"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}