Bias in AI Tools for Algorithmic fairness and accountability

Abstract

Abstract:

In an effort to combat bias we need systems that provide transparency in algorithms. These systems would provide information on what an algorithm decided and why it made its decision. Including the obvious missteps of cultural biases to the biases that are more subtle. This workshop will go over the first steps you can take with new open source tools from IBM and Google called AI Fairness 360 and What-If respectively. We will also briefly point out steps and features that Microsoft and Amazon announced for their upcoming AI transparency tools.

Microsoft

Google

IBM

Background Reading

• AI Fairness 360: An extensible toolkit for detecting understanding, and mitigating unwanted bias

• A reductions approach to fair classification

• Improving fairness in machine learning systems, what do industry practitioners need?

• Assessing and addressing algorithmic bias in practice

• Intro to data ethics

• Framework for ethical decision making

• Types of bias

• Trouble with AI Bias

• Algorithm accountability act

• Case Study facial recognition

• AI Fairness 360 toolkit

Datasets

Kaggle

Product Examples

1. Microsoft AI Bot - Tay

2. Norman

3. Contextual Lineup - Facial Recognition Database

• People are unknowingly added to the system

4. Dreamers - ICE Raids

5. Autonomous fleets - pickups

6. Housing Policies

Process

Get involved in the conversation and understanding what is under the hood go back to the first time teach yourself more

Agenda

• Let's talk about machines and automation (30 minutes)

• Algorithmic changes in the workplace

• Overview: What is Algorithmic fairness and why should you care

• Additional Reading (30 minutes)

• Organizations working in the field (30 minutes)

  • AI Now
    • AI Now is a research institution based out of NYU examinging the social implications of aritificial intelligence.
    • Each year AI now holds a symposium, 2018 2017
    • Their reserach includes four key areas:
      • Rights and Liberties
      • Labor and Automation
      • Bias and Inclusion
      • Safety and critical infrastructure
  • Data & Society
    • Data & society is a research institute working out of New York City focused on the social nd cultural issues arising from data centric and automated technologies.
      • On mapping police violence
  • Future of Humanity Institute
    • FHI is a multidisciplinary research institute at the University of Oxford. Academics at FHI bring the tools of mathematics, philosophy, social sciences, and science to bear on big-picture questions about humanity and its prospects. The Institute is led by founding Director Professor Nick Bostrom. Humanity has the potential for a long and flourishing future. Our mission is to shed light on crucial considerations that might shape our future.
  • MIRI - Machine Intelligence Research Institute
    • The Machine Intelligence Research Institute is a research nonprofit studying the mathematical underpinnings of intelligent behavior. Our mission is to develop formal tools for the clean design and analysis of general-purpose AI systems, with the intent of making such systems safer and more reliable when they are developed.
  • OpenAI
    • OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.

• Examination of algorithms in production and specific instances + how these have been unfair

• Company efforts to mitigate bias and tools (1- hour) - What if tool - IBM Tool

• Hand- On Lab (1.5 hours)

• Stakeholders, who is responsible and for what? (1 - hour)

  how to organize product - data engineering and data science workstreams

  

• Data and accountability + feasibility. Who understands the data being piped into these algorithms and why do data analysts + data scientists matter? (30 -minutes)

• Algorithmically enhanced machines - when hardware and software combine

  • autonomous fleets
  • ai robots - boston dynamics
  • surgical robots - two body systems competing
  • arbitrary AI systems