eda1.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Created by R. David Beales for the [Kelvin Smith Library](https://case.edu/library/) at [Case Western Reserve University](https://case.edu) under [Creative Commons CC BY License](https://creativecommons.org/licenses/by/4.0/)<br />\n",
    "For questions/comments/improvements, email [rdb104@case.edu](mailto:rdb104@case.edu).<br />\n",
    "___"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploratory Data Analysis in Python\n",
    "\n",
    "**Description:** This lesson introduces the basic data import and simple assessment processes using the `pandas` library for Python.  \n",
    "\n",
    "**Use Case:** For Learners (Additional explanation, not ideal for researchers)\n",
    "\n",
    "**Difficulty:** Beginner\n",
    "\n",
    "**Completion time:** 30 minutes\n",
    "\n",
    "**Knowledge Required:** Basic Python\n",
    "\n",
    "**Knowledge Recommended:** csv files\n",
    "\n",
    "**Data Format:** `csv`, `py` \n",
    "\n",
    "**Libraries Used:** `pandas` \n",
    "___"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "Welcome to your first EDA lesson.  This project will use the `pandas` package to introduce the basic EDA workflow.\n",
    "\n",
    "We will be using [2014 Adult Census Data](https://raw.githubusercontent.com/LibraryBeales/Exploratory_Data_Analysis/refs/heads/main/adult.csv) as our example for this tutorial.  The csv file is taken from this [Kaggle Dataset](https://www.kaggle.com/datasets/uciml/adult-census-income?resource=download).  \n",
    "\n",
    "In this project you will:\n",
    "1. Learn how to import the necessary python libraries.\n",
    "2. Use the `read_csv` function in pandas to import a csv file.\n",
    "3. Discover the size of the dataset.\n",
    "4. Discover the types of data in the dataset. \n",
    "5. Learn how to modify data types to improve performance, accuracy and scalability.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ultra Quick Jupyter Notebook Tips\n",
    "\n",
    "These Jupyter Notebooks have markdown cells, which you just read, and code cells, which you can run and edit.  \n",
    "\n",
    "Code cells appear in the light grey box.  You can click on the text in the code cell to edit it.  You can run a code cell by clicking the Run button at the top of the page (pictured) or by clicking Shift + Enter after clicking on the cell.\n",
    "\n",
    "![The Run Button](img/runcellbutton.png) \n",
    "\n",
    "Finally, all the code cells in a notebook must be run in order.  Make sure you start at the top of each lesson and run each cell in order."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing data\n",
    "\n",
    "We'll be using the `pandas` package for most of our exploratory data analysis. Before we can use a Python package, we need to import it. We’ll import `pandas as pd`, allowing us to use the shorter `pd` instead of typing `pandas` each time we call its functions. While this may seem like a small change, it's a common practice in Python, so it's helpful to be familiar with it.\n",
    "\n",
    "Run the cell below to import the pandas package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd #https://pandas.pydata.org/docs/"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the `pandas` package imported, we can use its powerful functions to bring data into our workspace. Our data is in a CSV file—a widely supported format that's easy for humans to read and ideal for data analysis and maintenance. CSV files also allow for efficient storage of large datasets. If you're interested, you can read more about CSV files [here](https://en.wikipedia.org/wiki/Comma-separated_values) if you are curious.\n",
    "\n",
    "Our data can be found at this URL: (https://raw.githubusercontent.com/LibraryBeales/Exploratory_Data_Analysis/refs/heads/main/adult.csv)  This dataset, originally from the 2014 U.S. Census, was compiled by Ronny Kohavi and Barry Becker and is hosted on [Kaggle](https://www.kaggle.com/datasets/uciml/adult-census-income/data).  \n",
    "\n",
    "To import the CSV file, we'll use the `pd.read_csv` function. This function simply requires the file's location, which in our case is the URL above. You can also use `read_csv` to load data from a file path on your computer. For more details on this function, see the official documentation [here](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# URL of the CSV file\n",
    "url = 'https://raw.githubusercontent.com/LibraryBeales/Exploratory_Data_Analysis/refs/heads/main/adult.csv'\n",
    "\n",
    "# Read the CSV file into a DataFrame\n",
    "adultCensus = pd.read_csv(url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Python, the functions `head()`, `info()`, and `describe()` are frequently used for preliminary data exploration and analysis. \n",
    "\n",
    "Here's a quick breakdown of the usage and purpose of each one:\n",
    "\n",
    "1. head()\n",
    "\n",
    "    Purpose: Displays the first few rows of a DataFrame or Series.\n",
    "    Usage: DataFrame.head(n)\n",
    "    Parameter: n (optional) - the number of rows to display (default is 5).\n",
    "\n",
    "2. info()\n",
    "\n",
    "    Purpose: Provides a concise summary of a DataFrame, including the index dtype, column dtypes, non-null values, and memory usage.  Helps in understanding the structure of the DataFrame, checking for null values, and confirming data types.\n",
    "    Usage: DataFrame.info()\n",
    "\n",
    "3. describe()\n",
    "\n",
    "    Purpose: Generates descriptive statistics of numerical columns in a DataFrame, including count, mean, standard deviation, min, 25th percentile, median (50%), 75th percentile, and max.  Useful for quickly getting insights into the distribution and summary statistics of the data.\n",
    "    Usage: DataFrame.describe()\n",
    "\n",
    "Try running all three functions in the code cells below.  What did you learn about our dataset after using these functions?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the first five rows of the DataFrame\n",
    "adultCensus.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Display summary statistics for the numerical columns in a DataFrame.\n",
    "adultCensus.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " # Display a concise summary of the DataFrame, showing information about each column, including data types, non-null counts, and memory usage. \n",
    "adultCensus.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}