08-DataPrivacy&Anonymization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Data Privacy & Anonymization\n",
    "\n",
    "A lot of data, perhaps the vast majority of data typically used in data science, is, directly or indirectly, about people. \n",
    "\n",
    "Individuals have certain privacy rights regarding who can know or share information about specifically identified individuals. This is true in particular about certain classes of sensitive information (for example, health-related information has special protections), but also applies to all other data as well. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Information Privacy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "Information (or Data) Privacy refers to the legal, ethical, and practical issues of collecting, using and releasing data in which there is identifiable information about people included in the dataset. It also deals with when and how to deal with data privacy issues, and how to protect users' privacy.\n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "<a href=\"https://en.wikipedia.org/wiki/Information_privacy\" class=\"alert-link\">Wikipedia</a>\n",
    "has an overview of information privacy.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Anonymization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "Data Anonymization is a type of information sanitization - that is the removal of sensitive information - for the purpose of privacy protection. It is a procedure to modify a data set such that the individuals it reflect are anonymous. Most, typically this means the removal or personally identifiable information of data sets such that the identify of individuals contained in the data set are anonymous.\n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "<a href=\"https://en.wikipedia.org/wiki/Data_anonymization\" class=\"alert-link\">Wikipedia</a>\n",
    "also has an overview of data anonymization.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data protection and anonymization are interdisciplinary components of data science and data practice, including everything from the considerations of ethics & legalities of data use, to the practical and technical challenges of protecting and anonymizing data. \n",
    "\n",
    "Anonymizing data typically comes down to removing any personally identifiable data from a dataset, or, if this information must be kept, separating the identifiable data from sensitive information. \n",
    "\n",
    "Part of the difficulty of data anonymization is that while we can provably demonstrate that a given dataset is anonymized, this rests on particular assumptions - most notably that the dataset is only provable anonymized under the assumption that no extra external information is used to attempt to de-identify it. \n",
    "\n",
    "In practice, de-anonymizing data can often be done by combining multiple datasets - using information from one or several available information sources to decode the individuals included in some other dataset. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regulations\n",
    "\n",
    "There are many official guidelines, rules and standards for data privacy and user identity protection, although much of it is case specific. \n",
    "\n",
    "At the minimum, what is legally required in terms of data protection depends, amongst other things, on:\n",
    "- What the data is / contains, and who it is about, \n",
    "    - Certain data types, and/or populations may have special protections, for example health-related information.\n",
    "- Who owns the data and in what capacity they are acting (company, university, etc.)\n",
    "    - Regulations for scientific research are different than those for companies\n",
    "- User agreements / consent procedures that were in place when the data were collected. \n",
    "    - Individuals have a right to self-determination in terms of what their data is used for. Data should only be used for things that are covered by it's terms of use / terms of collection / consent procedures.\n",
    "- What the data is to be used for.\n",
    "    - Often a combination of the what and the who, there may be specific regulations about how you must deal with, and what you can do, based on the goal of having and using the data.\n",
    "- Where the data was collected and where it is stored, and who it is about.\n",
    "    - Different regions (countries, etc) often have different regulations.\n",
    "\n",
    "Much of these regulations apply more directly to the collection, storage, and release of datasets, but aspects also apply to the use of datasets, including publicly available datasets. Available datasets often have a user agreement for using the data, and, in particular, attempting to identify individuals from datasets may at a minimum break user agreements, and/or (depending on the nature of the data) be illegal based on consumer and research subject protection laws. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Research Standards"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "Data collected and used for research purposes has it's own set of guidelines and requirements regarding the treatment of human subjects, and the collection, storage, use, and dissemination of data. These regulations focus, among other things, on the right to self-determination of human subjects to consent to what data is collected, and how it is used. Data collected for research purposes must follow restrictions based on these consent procedures. \n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "Research protections under the\n",
    "<a href=\"https://en.wikipedia.org/wiki/Declaration_of_Helsinki\" class=\"alert-link\">Declaration of Helsinki</a>.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## HIPAA - Protection for Health Related Information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "The Health Insurance Portability and Accountability Act (HIPAA) is a US federal government regulation that standardizes and protects individuals medical records and health related data. It includes terms for how data must be stored, and how they can be used & shared.\n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "The official US federal government HIPAA information\n",
    "<a href=\"https://www.hhs.gov/hipaa/\" class=\"alert-link\">guidelines</a>\n",
    "include an overview of HIPAA.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Safe Harbour Method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-success\">\n",
    "Safe Harbor is an official agreement regarding how to deal with datasets that have personal data, in particular, specific guidelines on what information to remove from datasets in order to anonymize them. It is a single set of data protection requirements shared across many contexts and countries. \n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "The \n",
    "<a href=\"https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/\" class=\"alert-link\">official documentation</a>\n",
    "for Safe Harbour includes guidelines on how to anonymize data.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Safe Harbor method requires that the following identifiers of the individual or of relatives, employers, or household members of the individual, are removed:\n",
    "- Names\n",
    "- Geographic Subdivisions smaller than a state**\n",
    "- Dates (such as birth dates, etc), and all ages above 90\n",
    "- Telephone Numbers\n",
    "- Vehicle Identification Numbers\n",
    "- Fax numbers\n",
    "- Device identifiers and serial numbers\n",
    "- Email addresses\n",
    "- Web Universal Resource Locators (URLs)\n",
    "- Social security numbers\n",
    "- Internet Protocol (IP) addresses\n",
    "- Medical record numbers\n",
    "- Biometric identifiers, including finger and voice prints\n",
    "- Health plan beneficiary numbers\n",
    "- Full-face photographs and any comparable images\n",
    "- Account numbers\n",
    "- Certificate/license numbers\n",
    "- Any other unique identifying number, characteristic, or code\n",
    "\n",
    "\n",
    "** The first three numbers of the zip code can be kept, provided that more than 20,000 people live in the region covered by all the zip codes that share the same initial three digits (the same geographic subdivision). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Unique Identifiers\n",
    "\n",
    "The logic Safe Harbor (and/or Data Anonymization in general) is to remove any unique information that could be used to identify you. \n",
    "\n",
    "This is perhaps most obvious for things like names, but some of the seemingly less obvious specifications of Safe Harbour also relate to the idea that leaving the specified information in a dataset is a risk for identification of individuals contained in the dataset. \n",
    "\n",
    "For example, while it may be innocuous to talk about a 37 year old male who lives in Los Angeles (as there are many candidates, such that the specific individual is not revealed), it might actually be quite obvious who the person is when talking about a 37 year old male who lives in Potrero, California, a town of about 700 people. This is the same reason ages above 90 have to be removed - even in a fairly large area, say San Diego, it may be fairly obvious who the 98 year old female participant is. \n",
    "\n",
    "Basically - any information that makes you stand out is liable to identify you, and so anonymization attempts to remove these idiosyncracies from data, such that individuals do not stand out in a way that lets someone figure out who they are.\n",
    "\n",
    "This also underlies the difficulty in protecting data in the face of multiple data sources, since collecting observations together makes it much easier to start to pick out people more uniquely - so it may still be relatively easy to identify the 37 year old male from LA if you also happen to know (or figure out) that he has a poodle, is 5'6\", works at UCLA, and was at Griffith Park on Saturday, April 15th, all of which may be relatively easy to figure out by combining publicly available, or easily obtainable, data."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}