Skip to content

Latest commit

 

History

History
126 lines (59 loc) · 5.95 KB

File metadata and controls

126 lines (59 loc) · 5.95 KB

Identify-and-remove-bias-from-AI-models-using-Watson-Studio

Step 1: Create an IBM Cloud account

Login/Sign-up for IBM Cloud Account: https://ibm.biz/removebias

Step 2: Create Watson Studio service

Go to your IBM Cloud Dashboard, type in search box "Watson studio" select the service, on the service page click create

1

2

Step 3: Create a new Watson Studio project

Click on get started it will take you to a "Cloud Pak for Data" Dashboard.

3

4

Select create a project

5

Click on create an empty project

6

Give your project a unique name and click on create

7

Step 4: Add Data

Go to https://github.com/IBM/bias-mitigation-of-machine-learning-models-using-aif360 and download code in zip file. Un-zip the folder.

8

Click on Assets and select Browse and add fraud_data.csv file from your file system. Repeat the step and add the Pipeline_LabelEncoder-0.1.zip file as an asset.

9

Step 5: Create the notebook

Note: You will create three Notebooks here by repeating the below steps.

From IBM Watson Dashboard click on "Add to project" and select "Notebook"

10

11

Create a Pre-processing Notebook. Select the "From URL" tab and Enter a name for the notebook. Select the runtime (2 vCPU and 8 GB RAM.) Enter this Notebook URL for Pre-processing : https://github.com/IBM/bias-mitigation-of-machine-learning-models-using-aif360/blob/main/notebooks/Pre-processing.ipynb

12

After the noteboob is imported, click on Not Trusted and select the option to trust the source of the notebook.

13

Repeat the above steps for in-processing and Post-processing

Enter this Notebook URL for In-processing : https://github.com/IBM/bias-mitigation-of-machine-learning-models-using-aif360/blob/main/notebooks/In-processing.ipynb Enter this Notebook URL for Post-processing : https://github.com/IBM/bias-mitigation-of-machine-learning-models-using-aif360/blob/main/notebooks/Post-processing.ipynb

Step 6:Insert the data as dataframe

Open Pre-processing Notebook from Dashboard and click on edit

14

15

Click on 0010 icon at the top right side which will bring up the data assets tab. Click on Insert to code dropdown for fraud_data.csv and select the option Insert Pandas Dataframe.

Screenshot 2021-08-20 173819

Step 7: Run the notebook & Analyze Result

Click on Run icon to run the code

16

When a notebook is executed, what is actually happening is that each code cell in the notebook is executed, in order, from top to bottom.

Each code cell is selectable and is preceded by a tag in the left margin. The tag format is In [x]:. Depending on the state of the notebook, the x can be:

A blank, this indicates that the cell has never been executed.
A number, this number represents the relative order this code step was executed.
A *, this indicates that the cell is currently executing.

There are several ways to execute the code cells in your notebook:

One cell at a time.
    Select the cell, and then press the Play button in the toolbar.
Batch mode, in sequential order.
    From the Cell menu bar, there are several options available. For example, you can Run All cells in your notebook, or you can Run All Below, that will start executing from the first cell under the currently selected cell, and then continue executing all cells that follow.

After we run all cells in the notebook, the results are displayed at the end of each notebook per below. Pre-processing results We can observe that, priviledged group had 37% more chance of getting a favorable outcome because of the bias in the dataset.

17

Reference

https://github.com/IBM/bias-mitigation-of-machine-learning-models-using-aif360