Data_Visualization_in_Python_Unsolved.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Data_Visualization_in_Python_Unsolved.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/alitarraf/Data-Science-Training/blob/master/Data_Visualization_in_Python_Unsolved.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-HvCPRPuaUwl",
        "colab_type": "text"
      },
      "source": [
        "# Data Visualization 101\n",
        "\n",
        "We sift through a lot of data for our jobs. Data about website performance, sales performance, product adoption, customer service, marketing campaign results ... the list goes on.\n",
        "\n",
        "When you manage multiple content assets, such as social media or a blog, with multiple sources of data, it can get overwhelming.What should you be tracking? What actually matters? How do you visualize and analyze the data so you can extract insights and actionable information?\n",
        "\n",
        "More importantly, how can you make reporting more efficient when you're busy working on multiple projects at once?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TCp7rMCwrJ4Q",
        "colab_type": "text"
      },
      "source": [
        "<img src=https://i.ibb.co/XYSq34N/good-visuals.png width=\"600\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-INM7DKIg2PO",
        "colab_type": "text"
      },
      "source": [
        "## Why visualization matters?\n",
        "\n",
        "1. Visualization lets you see things that would rather go unnoticed. Any data contain information but if there's no visual data you're missing out on trends, behavior patterns and dependencies.\n",
        "\n",
        "2. Visualization gives answers faster. Looking at a graph and identifying a trend is an instant. Now imagine how much time it will take to eye-scan rows of numbers.\n",
        "\n",
        "3. A good visualization gives way to research data, to play with them, to investigate some curious cause-effect relationships. This is very important for investigation and research work, as in journalism.\n",
        "\n",
        "4. Data volume is growing at a crazy rate. Visualization helps to leverage not only the volume, but the ever increasing diversity of data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sUow03xen2DK",
        "colab_type": "text"
      },
      "source": [
        "# Plotting with Plotly + Cufflinks in Python\n",
        "\n",
        "In this notebook, we will see how to use [plotly](https://plot.ly/python/) and [cufflinks](https://github.com/santosjorge/cufflinks) to create stunning, interactive figures in a single line of Python. This combination of libraries is simple to use, makes excellent charts, and, in my opinion, much more efficient than other methods of plotting in Python.\n",
        "\n",
        "This introduction will show us the basics of using plotly + cufflinks, focusing on what we can do in one line of code (for the most part). \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-3mwleVbwAwD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install plotly_express"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v2hCqO6Fz9fS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install statsmodels==0.10.0rc2 --pre"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BwnaB2KNJFoq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import plotly\n",
        "from plotly.offline import init_notebook_mode\n",
        "import plotly_express as px\n",
        "plotly.__version__"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "13mohnO5JI5f",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import statsmodels.api as sm\n",
        "import warnings"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KREWQMG9n2DL",
        "colab_type": "text"
      },
      "source": [
        "## Plotly + Cufflinks Overview\n",
        "\n",
        "You can create a free account and upload your graphs to share with others (this requires making the graphs and data public). \n",
        "\n",
        "We will run plotly completely in offline mode which means that we won't be publishing any of our graphs online.\n",
        "\n",
        "The plotly Python package is an open-source library built on plotly.js which in turn is built on d3.js. We’ll be using a wrapper on plotly called cufflinks designed to work with Pandas dataframes. So, our entire stack is cufflinks > plotly > plotly.js > d3.js which means we get the efficiency of coding in Python with the incredible interactive graphics capabilities of d3.\n",
        "\n",
        "[Cufflinks is a wrapper ](https://github.com/santosjorge/cufflinks) around the plotly library specifically for plotting with Pandas dataframes. With cufflinks, we don't have to dig into the details of plotly, instead building our charts with minimal code. Basically, you can make charts directly in plotly for more control, or you can use cufflinks to rapidly prototype plots and explore the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p--1nCdb_knl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def enable_plotly_in_cell():\n",
        "  import IPython\n",
        "  from plotly.offline import init_notebook_mode\n",
        "  display(IPython.core.display.HTML('''<script src=\"/static/components/requirejs/require.js\"></script>'''))\n",
        "  init_notebook_mode(connected=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:46.221896Z",
          "start_time": "2019-01-09T03:09:45.418794Z"
        },
        "id": "f6EwSjY_n2DM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# plotly standard imports\n",
        "import plotly.graph_objs as go\n",
        "import plotly.plotly as py\n",
        "\n",
        "# Cufflinks wrapper on plotly\n",
        "import cufflinks\n",
        "\n",
        "# Data science imports\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "# Options for pandas\n",
        "pd.options.display.max_columns = 30\n",
        "\n",
        "# Display all cell outputs\n",
        "from IPython.core.interactiveshell import InteractiveShell\n",
        "InteractiveShell.ast_node_interactivity = 'all'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ngRJZFKn2DP",
        "colab_type": "text"
      },
      "source": [
        "We'll be using plotly + cufflinks in offline mode. We will also set the global `cufflinks` theme to `pearl`. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:46.699508Z",
          "start_time": "2019-01-09T03:09:46.639809Z"
        },
        "id": "CNOyaWXfn2DP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from plotly.offline import iplot\n",
        "cufflinks.go_offline()\n",
        "\n",
        "# Set global theme\n",
        "cufflinks.set_config_file(world_readable=True, theme='pearl')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xH__vPymn2DY",
        "colab_type": "text"
      },
      "source": [
        "After importing cufflinks, plotly plots can be made using `df.iplot()` and then specifying parameters. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o-dfEP2dn2DZ",
        "colab_type": "text"
      },
      "source": [
        "### Data\n",
        "\n",
        "We are using medium article statistics data. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:48.764569Z",
          "start_time": "2019-01-09T03:09:48.700373Z"
        },
        "id": "RKXw3D2yn2DZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df = pd.read_parquet('https://github.com/WillKoehrsen/Data-Analysis/blob/master/plotly/data/medium_data_2019_01_06?raw=true')\n",
        "#df = pd.read_parquet('medium_data_2019_01_06')\n",
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DLiX-GGCr7DL",
        "colab_type": "text"
      },
      "source": [
        "The Read ratio is the difference between your reads and views. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BFgQye7SkmZw",
        "colab_type": "text"
      },
      "source": [
        "Also we will be using the gapminder Dataset, which contains info about countries such as GDP, life Expectency by year and it comes pre-loaded with Plotly express as well as the Tips dataset which contains info about Food servers’ tips in restaurants."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R5CJavKO0-Iq",
        "colab_type": "text"
      },
      "source": [
        "#### Tips dataset\n",
        "\n",
        "Food servers’ tips in restaurants may be influenced by many factors, including the nature of the restaurant, size of the party, and table locations in the restaurant. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l4yat9tU1V5g",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tips = px.data.tips()\n",
        "tips.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eW-s8mjWn2Dd",
        "colab_type": "text"
      },
      "source": [
        "# Univariate (Single Variable) Distributions\n",
        "\n",
        "For single variables, I generally start out with histograms. Plotly has these basic charts well-covered."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5FUHR71cn2De",
        "colab_type": "text"
      },
      "source": [
        "## Histograms"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:52.276558Z",
          "start_time": "2019-01-09T03:09:51.925421Z"
        },
        "id": "Z6mM4DJln2De",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "\n",
        "df['claps'].iplot(\n",
        "    kind='hist',\n",
        "    bins=30,\n",
        "    xTitle='likes',\n",
        "    linecolor='black',\n",
        "    yTitle='count',\n",
        "    title='Claps Distribution')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GbJZQAQen2Dh",
        "colab_type": "text"
      },
      "source": [
        "* Notice that we can hover over any of the bars to get the exact numbers. You can also format the `text` to display different information on hovering.\n",
        "* Histograms use quantitative data (numeric data).\n",
        "* A histogram doesn't have any gaps, otherwise you'll be looking at a bar graph which contains categorical data, We will cover bar graphs during this session.\n",
        "* Bin size determines how many intervals range of data will be divided into."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p7JZRqEOEP4n",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "\n",
        "tips['tip'].iplot(\n",
        "    kind='hist',\n",
        "    bins=10,   #bar width\n",
        "    xTitle='tips',\n",
        "    linecolor='black',\n",
        "    yTitle='count',\n",
        "    title='Tips Distribution')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MjBKH-iln2Di",
        "colab_type": "text"
      },
      "source": [
        "### Percentage Histogram\n",
        "\n",
        "To get the same chart but instead showing the percentage, we simple pass in `percent` as the histnorm parameter."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:54.035862Z",
          "start_time": "2019-01-09T03:09:53.952689Z"
        },
        "id": "BNoWoqp7n2Dj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "\n",
        "tips['total_bill'].iplot(\n",
        "    kind='hist',\n",
        "    bins=11,\n",
        "    xTitle='Total bill',\n",
        "    linecolor='black',\n",
        "    histnorm='percent',\n",
        "    yTitle='percentage (%)',\n",
        "    title='Total bill Distribution in Percent')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YTxarX4xn2Dn",
        "colab_type": "text"
      },
      "source": [
        "## Grouped Histogram\n",
        "\n",
        "When we want to display two different distributions on the same plot, we can group together the data to show it side-by-side. This means setting `barmode` to `group` with two distributions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iFrLTiYsGR0Q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:55.109463Z",
          "start_time": "2019-01-09T03:09:55.106823Z"
        },
        "colab_type": "code",
        "id": "gV2YMN53DD1G",
        "colab": {}
      },
      "source": [
        "def to_time(dt):\n",
        "    return dt.hour + dt.minute / 60"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PpyXrJbPISd-",
        "colab_type": "text"
      },
      "source": [
        "The previous function simply turns time range into a numeric range by dividing minutes by 60 then adding hour\n",
        "\n",
        "Example: 14:24 becomes 14.4"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:55.213009Z",
          "start_time": "2019-01-09T03:09:55.113582Z"
        },
        "code_folding": [],
        "id": "nL1bkWwhn2Dq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df['time_started'] = df['started_date'].apply(to_time)\n",
        "df['time_published'] = df['published_date'].apply(to_time)\n",
        "\n",
        "enable_plotly_in_cell()\n",
        "df[['time_started', 'time_published']].iplot(\n",
        "    kind='hist',\n",
        "    linecolor='black',\n",
        "    bins=24,\n",
        "    histnorm='percent',\n",
        "    bargap=0.1,\n",
        "    opacity=0.8,\n",
        "    barmode='group',\n",
        "    xTitle='Time of Day',\n",
        "    yTitle='(%) of Articles',\n",
        "    title='Time Started and Time Published')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mts58wmRn2Dt",
        "colab_type": "text"
      },
      "source": [
        "## Overlaid Histogram\n",
        "\n",
        "If we prefer the bars to be laid over one another, we specify the `barmode` to be `overlay`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:56.183116Z",
          "start_time": "2019-01-09T03:09:56.090442Z"
        },
        "id": "zhcCn-gPn2Du",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "df[['time_published', 'time_started']].iplot(\n",
        "    kind='hist',\n",
        "    bins=24,\n",
        "    linecolor='black',\n",
        "    opacity=0.8,\n",
        "    histnorm='percent',\n",
        "    barmode='overlay',\n",
        "    xTitle='Time of day',\n",
        "    yTitle='(%) of articles',\n",
        "    title='Time Started and Time Published Overlaid')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DVY91EihIuph",
        "colab_type": "text"
      },
      "source": [
        "**Exercise**: Plot an overlaid histogram showing \"total bill\" and \"tip\" percentage from the tips dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ydx72-e2JQy6",
        "colab_type": "text"
      },
      "source": [
        "**Instructions:**\n",
        "- \"total bill\" can be referenced by column name \"total_bill\"\n",
        "- \"tip\" can be referenced by column name \"tip\"\n",
        "- Use 10 as number of bins\n",
        "- After coding your function, run the cell to check if your result is correct."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zcMbRmZyJGua",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "### START CODE HERE ### (≈ 10 line of code)\n",
        "\n",
        "### END CODE HERE ###\n",
        "\n",
        "#HINT\n",
        "##You can run tips.head() to see how the data looks"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q7GF1y-2PozH",
        "colab_type": "text"
      },
      "source": [
        "**Expected output**:\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "<img src=https://i.ibb.co/HVkh34h/newplot-1.png width=\"800\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cepiYeYDRyUw",
        "colab_type": "text"
      },
      "source": [
        "Why does the plot look like this? Because data is not evenly distibuted"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yq8NfG1Pn2Dw",
        "colab_type": "text"
      },
      "source": [
        "## Bar Plot"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TNMzmqbVn2Dx",
        "colab_type": "text"
      },
      "source": [
        "For a bar plot, we need to apply some sort of aggregation function and then plot. For example, we can show the `count` of articles in each publication with the following."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:58.198343Z",
          "start_time": "2019-01-09T03:09:58.073942Z"
        },
        "id": "cOFVqtadn2Dy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "tips['sex'].value_counts().iplot(\n",
        "    kind='bar', yTitle='Count', linecolor='black', title='Visualize Count of Tips Recorded by Gender')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:58.326699Z",
          "start_time": "2019-01-09T03:09:58.199750Z"
        },
        "id": "rmTXgLQrn2D0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "tips['day'].value_counts().iplot(\n",
        "    kind='bar',\n",
        "    xTitle='Day',\n",
        "    yTitle='Count of tips',\n",
        "    title='Visualize Count of Days for Recorded Tips',\n",
        "    linecolor='black')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aOVBLS69TIiF",
        "colab_type": "text"
      },
      "source": [
        "How a histogram is different than a bar chart?\n",
        "\n",
        "\n",
        "1.   A histogram has numerical groups\n",
        "2.   A bar graph uses cayegorial data as we discussed earlier\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ojqu2Jyjn2D5",
        "colab_type": "text"
      },
      "source": [
        "## Bar Plot with Two Categories\n",
        "\n",
        "Here we'll show two distributions side-by-side. First, we'll set the index to be the date, then resample to month frequency, then take the mean and plot."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bS7kJeQONGr1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oMq6jBGRPUnj",
        "colab_type": "text"
      },
      "source": [
        "We will only be working with the views,reads and published_date columns"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:59.050212Z",
          "start_time": "2019-01-09T03:09:59.036542Z"
        },
        "id": "kdNYf_xLn2D5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df2 = df[['views', 'reads',\n",
        "          'published_date']].set_index('published_date').resample('M').mean()\n",
        "#We set the index to published_date for resampling of time series, the resmaple \n",
        "#function is a pandas function that takes as argument the time range,\n",
        "#pandas is smart enough to understand that 'M' means we weant to sample records \n",
        "#by month and use as representation the last day of each month.\n",
        "\n",
        "#after resampling we can use functions such as sum or mean of each sample.\n",
        "df2.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:09:59.139634Z",
          "start_time": "2019-01-09T03:09:59.051788Z"
        },
        "id": "wKJ2Tm4Nn2D9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "df2.iplot(\n",
        "    kind='bar',\n",
        "    xTitle='Date',\n",
        "    yTitle='Average',\n",
        "    title='Monthly Average Views and Reads')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bFQHGumFn2EC",
        "colab_type": "text"
      },
      "source": [
        "By hovering over any month on the graph, we can make direct comparisons. This is a very handy way to explore your data! "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HLwQiBnxn2ED",
        "colab_type": "text"
      },
      "source": [
        "## Bar Plot with Second Y-Axis"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yD2cTHtmn2EE",
        "colab_type": "text"
      },
      "source": [
        "If we want to put two very different ranges on the same graph, we can just use a secondary y-axis."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:10:02.398171Z",
          "start_time": "2019-01-09T03:10:02.328579Z"
        },
        "id": "mDzhsM6xn2EF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "df2 = df[['views', 'read_time',\n",
        "          'published_date']].set_index('published_date').resample('M').mean()\n",
        "\n",
        "df2.iplot(\n",
        "    kind='bar',\n",
        "    xTitle='Date',\n",
        "    secondary_y='read_time',\n",
        "    secondary_y_title='Average Read Time',\n",
        "    yTitle='Average Views',\n",
        "    title='Monthly Averages')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ndIj7nsCn2EW",
        "colab_type": "text"
      },
      "source": [
        "# Scatter Plots\n",
        "\n",
        "The scatterplot is the heart of most analyses. It allows us to see the evolution of a variable over time or the relationship between two (or more) variables."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zC9jGeBZn2EX",
        "colab_type": "text"
      },
      "source": [
        "## Time-Series"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qub_LSWPn2EY",
        "colab_type": "text"
      },
      "source": [
        "A considerable portion of real-world data has a time element. Luckily, plotly + cufflinks was designed with time-series visualizations in mind.\n",
        "\n",
        "It's very simple to make time-series plots if we set the index to be the datetime. Then we can simply pass in a column as y and plotly will know to use the index to make a date xaxis. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iZkApOex5K_W",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:10:09.684308Z",
          "start_time": "2019-01-09T03:10:09.662521Z"
        },
        "id": "BmcsZbfEn2EY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tds = df[df['publication'] == 'Towards Data Science'].set_index(\n",
        "    'published_date')\n",
        "\n",
        "#here we filter by publication type and set the index to published_Date(time series)\n",
        "\n",
        "tds.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:10:09.801571Z",
          "start_time": "2019-01-09T03:10:09.685771Z"
        },
        "id": "4eczTQYJn2Eb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "tds['reads'].iplot(\n",
        "    mode='lines+markers',\n",
        "    opacity=0.8,\n",
        "    size=8,\n",
        "    symbol=1,\n",
        "    xTitle='Date',\n",
        "    yTitle='No. Readers',\n",
        "    title='Number of Readers Trend')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LLJ3xKvHvjJc",
        "colab_type": "text"
      },
      "source": [
        "Notes about the previous plot:\n",
        "\n",
        "* The first thing to look for is whether the data has a positive or negative trend to it as time progresses.\n",
        "* You have to have enough data (enough time) to assess the trend, having data for 2-3 days is Not a trend.\n",
        "* What's the variation? High points, max points..."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j5ktBEi9n2Ed",
        "colab_type": "text"
      },
      "source": [
        "## Two Variables Time-Series\n",
        "\n",
        "For a second variable, we should add the second variable on a secondary y-axis.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:10:11.998235Z",
          "start_time": "2019-01-09T03:10:11.915437Z"
        },
        "id": "VfDjw-fWn2Eg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "tds[['claps', 'views', 'title']].iplot(\n",
        "    y='claps',\n",
        "    mode='lines+markers',\n",
        "    secondary_y = 'views',\n",
        "    secondary_y_title='views',\n",
        "    opacity=0.8,\n",
        "    size=8,\n",
        "    symbol=1,\n",
        "    xTitle='Date',\n",
        "    yTitle='claps',\n",
        "    text='title',\n",
        "    title='Fans and Views over Time')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z2qSFf_SIMjU",
        "colab_type": "text"
      },
      "source": [
        "Here we are doing quite a few different things all in one line:\n",
        "\n",
        "*   Getting a nicely formatted time-series x-axis automatically\n",
        "*   Adding a secondary y-axis because our variables have different ranges\n",
        "*   Adding in the title of the articles as hover information\n",
        "*  although the ranges are different, can you see the relation between the variables?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XvaL6tZMn2Fa",
        "colab_type": "text"
      },
      "source": [
        "# Heatmap"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IgF1aOAcBEJt",
        "colab_type": "text"
      },
      "source": [
        "To visualize the correlations between numeric variables, we calculate the correlations and then make an annotated heatmap:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:10:25.240851Z",
          "start_time": "2019-01-09T03:10:25.238183Z"
        },
        "id": "zTpYA7utn2Fa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "colorscales = [\n",
        "    'Greys', 'YlGnBu', 'Greens', 'YlOrRd', 'Bluered', 'RdBu', 'Reds', 'Blues',\n",
        "    'Picnic', 'Rainbow', 'Portland', 'Jet', 'Hot', 'Blackbody', 'Earth',\n",
        "    'Electric', 'Viridis', 'Cividis'\n",
        "]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0v0yaD-ik0SY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import plotly.figure_factory as ff\n",
        "corrs = df.corr()\n",
        "#Pandas dataframe.corr() is used to find the pairwise correlation of \n",
        "#all columns in the dataframe. \n",
        "#Any na values are automatically excluded. \n",
        "#For any non-numeric data type columns in the dataframe it is ignored.\n",
        "corrs.head()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:10:25.811828Z",
          "start_time": "2019-01-09T03:10:25.242347Z"
        },
        "id": "saqmkwIFn2Fb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "\n",
        "figure = ff.create_annotated_heatmap(\n",
        "    z=corrs.values,\n",
        "    x=list(corrs.columns),\n",
        "    y=list(corrs.index),#rows\n",
        "    colorscale='Earth',\n",
        "    annotation_text=corrs.round(2).values,#Round a DataFrame to a variable number of decimal places.\n",
        "    showscale=True, #Display colorscale\n",
        "    reversescale=True)\n",
        "\n",
        "figure.layout.margin = dict(l=200, t=200)\n",
        "figure.layout.height = 800\n",
        "figure.layout.width = 1000\n",
        "\n",
        "iplot(figure)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zy5I9JCgn2Fs",
        "colab_type": "text"
      },
      "source": [
        "# Pie Chart"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:10:28.826407Z",
          "start_time": "2019-01-09T03:10:28.796779Z"
        },
        "id": "z8DyDD_kn2Fs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "df.groupby(\n",
        "    'publication', as_index=False)['reads'].count().iplot(\n",
        "        kind='pie', labels='publication', values='reads', title='Percentage of Reads by Publication')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-01-09T03:10:28.840537Z",
          "start_time": "2019-01-09T03:10:28.827996Z"
        },
        "id": "qvUnSQiUn2Fu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "df.groupby(\n",
        "    'publication', as_index=False)['word_count'].sum().iplot(\n",
        "        kind='pie', labels='publication', values='word_count', title='Percentage of Words by Publication')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aZMWtL8Dst93",
        "colab_type": "text"
      },
      "source": [
        "# Plotly express\n",
        "\n",
        "Plotly Express is a terse, consistent, high-level wrapper around Plotly.py for rapid data exploration and figure generation."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "txh5lx2Xs4Gk",
        "colab_type": "text"
      },
      "source": [
        "Let's take a look at the GapMinder Dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c1O6u8_ltHzu",
        "colab_type": "code",
        "outputId": "8c38b98e-d113-4c94-c8a8-3188a91e2b29",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "gapminder = px.data.gapminder()\n",
        "gapminder.head()\n",
        "gapminder2007=gapminder.query(\"year==2007\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>country</th>\n",
              "      <th>continent</th>\n",
              "      <th>year</th>\n",
              "      <th>lifeExp</th>\n",
              "      <th>pop</th>\n",
              "      <th>gdpPercap</th>\n",
              "      <th>iso_alpha</th>\n",
              "      <th>iso_num</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Afghanistan</td>\n",
              "      <td>Asia</td>\n",
              "      <td>1952</td>\n",
              "      <td>28.801</td>\n",
              "      <td>8425333</td>\n",
              "      <td>779.445314</td>\n",
              "      <td>AFG</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Afghanistan</td>\n",
              "      <td>Asia</td>\n",
              "      <td>1957</td>\n",
              "      <td>30.332</td>\n",
              "      <td>9240934</td>\n",
              "      <td>820.853030</td>\n",
              "      <td>AFG</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Afghanistan</td>\n",
              "      <td>Asia</td>\n",
              "      <td>1962</td>\n",
              "      <td>31.997</td>\n",
              "      <td>10267083</td>\n",
              "      <td>853.100710</td>\n",
              "      <td>AFG</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Afghanistan</td>\n",
              "      <td>Asia</td>\n",
              "      <td>1967</td>\n",
              "      <td>34.020</td>\n",
              "      <td>11537966</td>\n",
              "      <td>836.197138</td>\n",
              "      <td>AFG</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Afghanistan</td>\n",
              "      <td>Asia</td>\n",
              "      <td>1972</td>\n",
              "      <td>36.088</td>\n",
              "      <td>13079460</td>\n",
              "      <td>739.981106</td>\n",
              "      <td>AFG</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       country continent  year  lifeExp       pop   gdpPercap iso_alpha  \\\n",
              "0  Afghanistan      Asia  1952   28.801   8425333  779.445314       AFG   \n",
              "1  Afghanistan      Asia  1957   30.332   9240934  820.853030       AFG   \n",
              "2  Afghanistan      Asia  1962   31.997  10267083  853.100710       AFG   \n",
              "3  Afghanistan      Asia  1967   34.020  11537966  836.197138       AFG   \n",
              "4  Afghanistan      Asia  1972   36.088  13079460  739.981106       AFG   \n",
              "\n",
              "   iso_num  \n",
              "0        4  \n",
              "1        4  \n",
              "2        4  \n",
              "3        4  \n",
              "4        4  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KX5WIkAx_YsT",
        "colab_type": "text"
      },
      "source": [
        "##Scatterplot\n",
        "\n",
        "If you want a basic scatter plot, it’s just px.scatter(data, x=\"column_name\", y=\"column_name\")"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JgCgbhwa_qT7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "px.scatter(gapminder2007, x=\"gdpPercap\", y=\"lifeExp\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jx7dXMnSBFP4",
        "colab_type": "text"
      },
      "source": [
        "Notes about the previous plot:\n",
        "\n",
        "* The goal of this scatter plot is to see the relation between gdpPercap and life expectency\n",
        "* We see that when the gdp is very low, life expectancy has a wide range. Some people die young, some live to their 60s\n",
        "* However, As the gdp increses we see that life expectancy definitely increases. Better Income means better Healthcare!\n",
        "* As you can see, We have some outliers in the data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uu610CX4CFr0",
        "colab_type": "text"
      },
      "source": [
        "**Exercise**: Plot a scatterplot on the tips dataset showing \"total bill\" and \"tip\" colored by smoker"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "87Omrr67DFyZ",
        "colab_type": "text"
      },
      "source": [
        "**Instructions:**\n",
        "- \"total bill\" can be referenced by column name \"total_bill\"---> X asis\n",
        "- \"tip\" can be referenced by column name \"tip\"-->Y axis\n",
        "- Use the argument color=\"column name\" to color your plot by a certain column\n",
        "- After coding your function, run the cell to check if your result is correct."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nVp5sSVMDgB4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "### START CODE HERE ### (≈ 1 line of code)\n",
        "\n",
        "### END CODE HERE ###\n",
        "\n",
        "#HINT\n",
        "##You can run tips.head() to see how the data looks"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fWpSMgFFERSL",
        "colab_type": "text"
      },
      "source": [
        "**Expected output**:\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "<img src=https://i.ibb.co/sbLYkJZ/newplot-2.png width=\"800\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s1pLt3MLGgQ_",
        "colab_type": "text"
      },
      "source": [
        "Back to the gapminder dataset,\n",
        "maybe we want to scale the points by the country population… no problem: there’s an arg for that too! Unsurprisingly, it’s called \"size\".\n",
        "Curious about which point is which country? Add a hover_name and you can easily identify any point: never again wonder “what is that outlier?”... just mouse over the point you're interested in!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "byWkBp7_PtKx",
        "colab_type": "text"
      },
      "source": [
        "**Exercise**: Plot a scatterplot on the gapminder 2007 dataset scaled by country population where points are identified by name and colored by continent"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zhG4HMGzQRQT",
        "colab_type": "text"
      },
      "source": [
        "**Instructions:**\n",
        "- use color argument for coloring\n",
        "- use size argument for points size\n",
        "- set size_max attribute to 60\n",
        "- use hover_name argument for point identification"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_QhZ8FpoGscF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "\n",
        "### START CODE HERE ### (≈ 1 line of code)\n",
        "\n",
        "### END CODE HERE ###\n",
        "\n",
        "#HINT\n",
        "##You can run tips.head() to see how the data looks"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "571GaW5gSE5-",
        "colab_type": "text"
      },
      "source": [
        "**Expected output**:\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "<img src=https://i.ibb.co/371xqhL/newplot.png width=\"800\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mtnZ3YCltJpM",
        "colab_type": "text"
      },
      "source": [
        "## Animated Scatter Plot\n",
        "\n",
        "Here’s an example with the Gapminder dataset showing life expectancy vs GPD per capita by country colored By Continent scaled by the country population.\n",
        "\n",
        "Curious about which point is which country? Add a hover_name and you can easily identify any point: never again wonder “what is that outlier?”\n",
        "\n",
        "We also want to see how this chart evolved over time. You can animate it by setting animation_frame=\"year\" and animation_group=\"country\" to identify which circles match which ones across frames. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ViPwh_kayrXt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "px.scatter(gapminder, x=\"gdpPercap\", y=\"lifeExp\",size=\"pop\", size_max=60, color=\"continent\", hover_name=\"country\",\n",
        "           animation_frame=\"year\", animation_group=\"country\", log_x=True, range_x=[100,100000], range_y=[25,90],\n",
        "           labels=dict(pop=\"Population\", gdpPercap=\"GDP per Capita\", lifeExp=\"Life Expectancy\"))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KoqmwWvIvazO",
        "colab_type": "text"
      },
      "source": [
        "## Whisker Plot\n",
        "\n",
        "* Box and whisker plots are ideal for comparing distributions,as it displays the range of data along a number line.\n",
        "* Data is ordered from least to greatest\n",
        "* The median is the middle value that splits the data into two equal groups\n",
        "* lower quartile(often called q1) is the median of the lower half of data\n",
        "* upper quartile(often called q3) is the median of the upper half of data\n",
        "* extremes are the smallest and biggest values of data respectively\n",
        "* the distance between the points in the box tells you about the distribution of the data\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "<img src=https://www150.statcan.gc.ca/n1/edu/power-pouvoir/ch12/img/5214889_01-eng.gif width=\"400\"/>\n",
        "\n",
        "<img src=https://i.stack.imgur.com/mpbTr.gif width=\"400\"/>\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3W7yqE8dv5xA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "px.box(tips,x=\"size\",y=\"tip\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q9JtrszS0OUc",
        "colab_type": "text"
      },
      "source": [
        "Notes about the previous plot:\n",
        "\n",
        "* We notice that groups of 6 people have a high value as the minimum of their tip range.\n",
        "* Why does the box in group 5 look like this?\n",
        "* the median line doesn't have to be in the middle!\n",
        "*Can you spot the difference between upper extreme and outliers?\n",
        "* The benefits of interactivity are that we can explore and subset the data as we like. There’s a lot of information in a boxplot, and without the ability to see the numbers, we’ll miss most of it!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jG3b2QcV8ZJt",
        "colab_type": "text"
      },
      "source": [
        "**Exercise**: Plot a Whisker plot showing \"total bill\" distributed by day colored by Smoking habit and plotted horizontally"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8wDOmezr87-i",
        "colab_type": "text"
      },
      "source": [
        "**Instructions:**\n",
        "- \"total bill\" can be referenced by column name \"total_bill\"\n",
        "- \"days\" can be referenced by column name \"day\"\n",
        "- Color your plot using \"smoker\" column\n",
        "- You can set plot orientation by using argument --->orientation=\"h\"\n",
        "- After coding your function, run the cell to check if your result is correct."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dEDb02_g_EqH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "### START CODE HERE ### (≈ 1 line of code)\n",
        "\n",
        "### END CODE HERE ###\n",
        "\n",
        "#BONUS\n",
        "#If i wanted to order the days I can add argument: category_orders={\"day\":[\"Thur\",\"Fri\",\"Sat\",\"Sun\"]}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nmiBy-Ud_4n1",
        "colab_type": "text"
      },
      "source": [
        "**Expected output**:\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "<img src=https://i.ibb.co/JmkL9GZ/newplot.png width=\"800\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sdJT7ByEHtzb",
        "colab_type": "text"
      },
      "source": [
        "## Scatter and Box and Violin"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xbpXWmIxH0ec",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "warnings.filterwarnings('ignore')\n",
        "enable_plotly_in_cell()\n",
        "gapminder2=gapminder.query(\"year>1980\")\n",
        "px.scatter(gapminder2, x=\"gdpPercap\", y=\"lifeExp\", color=\"continent\", marginal_y=\"violin\",\n",
        "           marginal_x=\"box\", trendline=\"ols\",hover_name=\"country\") #ordinary least square regression"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gMrSw0P9J7ZK",
        "colab_type": "text"
      },
      "source": [
        "Notes about the previous plot:\n",
        "\n",
        "* If we sample the points of 1 continent , it looks like there's a roughly linear relationship\n",
        "* We want to find the best fit regression line, a regression model allows us to predict what future values will look like.\n",
        "<img src=https://i.ibb.co/hXWYZRV/1.png width=\"800\"/>\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "* What's the point of Violin plot if we can use box?\n",
        "\n",
        "<img src=https://datavizcatalogue.com/methods/images/anatomy/SVG/violin_plot.svg width=\"800\"/>\n",
        "\n",
        "* It's like a histogram of the data! Only smoothed to look nicer.\n",
        "* A box plot shows range of data but doesn't show values density.\n",
        "* Looking at two continents that have the same median using box plots only can lead to the incorrect conclusion that they both have the same data distribution."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BXPOeba6LnO6",
        "colab_type": "text"
      },
      "source": [
        "##Parallel Categories Diagram\n",
        "\n",
        "The parallel categories diagram is a visualization of multi-dimensional categorical data sets. Each variable in the data set is represented by a column of rectangles, where each rectangle corresponds to a discrete value taken on by that variable. The relative heights of the rectangles reflect the relative frequency of occurrence of the corresponding value.\n",
        "\n",
        "Combinations of category rectangles across dimensions are connected by ribbons, where the height of the ribbon corresponds to the relative frequency of occurrence of the combination of categories in the data set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jjM4BDmzLXM4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "px.parallel_categories(tips, color=\"size\", color_continuous_scale=px.colors.sequential.Inferno)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evPOotFyMRG8",
        "colab_type": "text"
      },
      "source": [
        "Notes about the previous plot:\n",
        "\n",
        "* We notice that number of male smokers is more than its female counterpart\n",
        "* Highest number of smokers is witnessed on Saturday\n",
        "* Groups of 4 often visit at dinner time.\n",
        "* Apparently people only have lunch on Thursday!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ejKzkgVwSod6",
        "colab_type": "text"
      },
      "source": [
        "## Parallel Coordinates Diagram\n",
        "\n",
        "Parallel coordinates are richly interactive by default, it allows you to visualize relation between variables\n",
        "\n",
        "Drag the lines along the axes to filter values and drag the axis names across the plot to rearrange variables."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "38v-lS6mSo5k",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "enable_plotly_in_cell()\n",
        "px.parallel_coordinates(tips,color=\"size\",color_continuous_scale=[\"red\",\"green\",\"blue\"])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2poMxYRkOZuM",
        "colab_type": "text"
      },
      "source": [
        "## Dash with Tips Showcase\n",
        "<img src=https://aws1.discourse-cdn.com/standard17/uploads/plot/original/2X/e/e1628056dbc0e8b57586b8ab264991da933152ca.gif width=\"900\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fwP624kNvtyk",
        "colab_type": "text"
      },
      "source": [
        "# Where to take it from here?\n",
        "\n",
        "Hopefully you now have an idea of the capabilties of plotly/ Plotly Express + cufflinks. We have only scrachted the surface of this library, so check out the cufflinks documentation and the plotly documentation for plently of more examples.\n",
        "\n",
        "As we have seen, designing an efficient and effective data visualization application is a systematic process. \n",
        "This process involves representing the data of interest, processing the data to extract relevant information for the problem at hand, designing a mapping of this information to a visual representation, rendering this representation, and combining all this functionality in an easy-to-use application."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u5VCnw-ZVxKQ",
        "colab_type": "text"
      },
      "source": [
        "# Data Visualization Showcase\n",
        "\n",
        "\n",
        "* https://www.plotly.express/\n",
        "* [Arlington Visual Budget](http://arlingtonvisualbudget.org/revenues/2019/t/352a9d4d) #Treemap\n",
        "* [Migration Flow](http://download.gsb.bund.de/BIB/global_flow/) #chord diagram\n",
        "* [D3 library](https://github.com/d3/d3/wiki/Gallery) #D3.js is a JavaScript library for producing dynamic, interactive data visualizations in web browsers."
      ]
    }
  ]
}