imbalance-data-eda-comprehensive-ml-models-shap.ipynb

{
 "metadata": {
  "kernelspec": {
   "language": "python",
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "version": "3.6.4",
   "file_extension": ".py",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "name": "python",
   "mimetype": "text/x-python"
  }
 },
 "nbformat_minor": 4,
 "nbformat": 4,
 "cells": [
  {
   "cell_type": "code",
   "source": [
    "import numpy as np \n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.preprocessing import LabelEncoder,StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "import xgboost as xgb\n",
    "from sklearn.metrics import confusion_matrix, accuracy_score,precision_score, roc_auc_score,classification_report,roc_curve,auc\n",
    "import lightgbm as lgb\n",
    "from catboost import CatBoostClassifier\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from xgboost import XGBClassifier\n",
    "import shap\n",
    "shap.initjs()\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))\n"
   ],
   "metadata": {
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:51.751031Z",
     "iopub.execute_input": "2022-09-28T22:08:51.752296Z",
     "iopub.status.idle": "2022-09-28T22:08:51.776374Z",
     "shell.execute_reply.started": "2022-09-28T22:08:51.752247Z",
     "shell.execute_reply": "2022-09-28T22:08:51.775295Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Introduction\n",
    "\n",
    "Our client is an Insurance company that has provided Health Insurance to its customers now they need your help in building a model to predict whether the policyholders (customers) from past year will also be interested in Vehicle Insurance provided by the company.\n",
    "\n",
    "An insurance policy is an arrangement by which a company undertakes to provide a guarantee of compensation for specified loss, damage, illness, or death in return for the payment of a specified premium. A premium is a sum of money that the customer needs to pay regularly to an insurance company for this guarantee.\n",
    "\n",
    "For example, you may pay a premium of Rs. 5000 each year for a health insurance cover of Rs. 200,000/- so that if, God forbid, you fall ill and need to be hospitalised in that year, the insurance provider company will bear the cost of hospitalisation etc. for upto Rs. 200,000. Now if you are wondering how can company bear such high hospitalisation cost when it charges a premium of only Rs. 5000/-, that is where the concept of probabilities comes in picture. For example, like you, there may be 100 customers who would be paying a premium of Rs. 5000 every year, but only a few of them (say 2-3) would get hospitalised that year and not everyone. This way everyone shares the risk of everyone else.\n",
    "\n",
    "Just like medical insurance, there is vehicle insurance where every year customer needs to pay a premium of certain amount to insurance provider company so that in case of unfortunate accident by the vehicle, the insurance provider company will provide a compensation (called ‘sum assured’) to the customer\n",
    "\n",
    "# Business Goal\n",
    "\n",
    "Building a model to predict whether a customer would be interested in Vehicle Insurance is extremely helpful for the company because it can then accordingly plan its communication strategy to reach out to those customers and optimize its business model and revenue.\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "train=pd.read_csv(r'data/health-insurance-cross-sell-prediction/train.csv')\n",
    "test=pd.read_csv(r'data/health-insurance-cross-sell-prediction/test.csv')\n",
    "train.head()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:55.546635Z",
     "iopub.execute_input": "2022-09-28T22:08:55.547434Z",
     "iopub.status.idle": "2022-09-28T22:08:56.519738Z",
     "shell.execute_reply.started": "2022-09-28T22:08:55.547377Z",
     "shell.execute_reply": "2022-09-28T22:08:56.518356Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "print(train.shape)\n",
    "print(test.shape)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:56.522299Z",
     "iopub.execute_input": "2022-09-28T22:08:56.52279Z",
     "iopub.status.idle": "2022-09-28T22:08:56.529067Z",
     "shell.execute_reply.started": "2022-09-28T22:08:56.522744Z",
     "shell.execute_reply": "2022-09-28T22:08:56.528066Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "train.Response.value_counts()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:56.530254Z",
     "iopub.execute_input": "2022-09-28T22:08:56.531235Z",
     "iopub.status.idle": "2022-09-28T22:08:56.553741Z",
     "shell.execute_reply.started": "2022-09-28T22:08:56.531187Z",
     "shell.execute_reply": "2022-09-28T22:08:56.552465Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Columns description\n",
    "\n",
    "* **id**     Unique ID for the customer\n",
    "* **Gender**     Gender of the customer\n",
    "* **Age**         Age of the customer\n",
    "* **Driving_License**  **0** : Customer does not have DL, **1** : Customer already has DL\n",
    "* **Region_Code**       Unique code for the region of the customer\n",
    "* **Previously_Insured**  -  1 : Customer already has Vehicle Insurance, 0 : Customer doesn't have Vehicle Insurance\n",
    "* **Vehicle_Age**       Age of the Vehicle\n",
    "* **Vehicle_Damage**  1 : Customer got his/her vehicle damaged in the past. 0 : Customer didn't get his/her vehicle damaged in the past.\n",
    "* **Annual_Premium**  The amount customer needs to pay as premium in the year\n",
    "* **PolicySalesChannel**\tAnonymized Code for the channel of outreaching to the customer ie. Different Agents, Over Mail, Over Phone, In Person, etc.\n",
    "* **Vintage**   Number of Days, Customer has been associated with the company\n",
    "* **Response**  1 : Customer is interested, 0 : Customer is not interested"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# **EDA** - Exploratory data analysis"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "train.info()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:56.556465Z",
     "iopub.execute_input": "2022-09-28T22:08:56.557543Z",
     "iopub.status.idle": "2022-09-28T22:08:56.633934Z",
     "shell.execute_reply.started": "2022-09-28T22:08:56.557493Z",
     "shell.execute_reply": "2022-09-28T22:08:56.632963Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "test.isna().sum()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:56.635372Z",
     "iopub.execute_input": "2022-09-28T22:08:56.635923Z",
     "iopub.status.idle": "2022-09-28T22:08:56.663976Z",
     "shell.execute_reply.started": "2022-09-28T22:08:56.635887Z",
     "shell.execute_reply": "2022-09-28T22:08:56.662957Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "train=train.drop(columns='id',axis=1)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:56.752581Z",
     "iopub.execute_input": "2022-09-28T22:08:56.753617Z",
     "iopub.status.idle": "2022-09-28T22:08:56.777971Z",
     "shell.execute_reply.started": "2022-09-28T22:08:56.753571Z",
     "shell.execute_reply": "2022-09-28T22:08:56.776901Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "plt.figure(figsize=(12,6))\n",
    "train.groupby(['Response'])['Previously_Insured'].value_counts().plot(\n",
    "                                                 kind='pie',autopct='%.0f%%')\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:56.909337Z",
     "iopub.execute_input": "2022-09-28T22:08:56.90981Z",
     "iopub.status.idle": "2022-09-28T22:08:57.141173Z",
     "shell.execute_reply.started": "2022-09-28T22:08:56.909766Z",
     "shell.execute_reply": "2022-09-28T22:08:57.139696Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "\n",
    "* **(0,1)** represent to previously insured but this time **not** interested\n",
    "* **(1,0)** represent to previously **not** insured but this time interested\n",
    "* **(0,0)** represent neither previously nor this time interested\n",
    "\n",
    "# Interruption\n",
    "\n",
    "* The company should check the communication strategy and take into account why 46% of customers quite\n",
    "* Customers between 22-30 tend not to buy the service a second time\n",
    "* Fortunately no missing values\n",
    "\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "plt.figure(figsize=(15,5))\n",
    "sns.countplot(train.Age,hue=train.Previously_Insured)\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:57.28065Z",
     "iopub.execute_input": "2022-09-28T22:08:57.282005Z",
     "iopub.status.idle": "2022-09-28T22:08:58.429775Z",
     "shell.execute_reply.started": "2022-09-28T22:08:57.28196Z",
     "shell.execute_reply": "2022-09-28T22:08:58.428573Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "sns.heatmap(train.corr()[['Response']],annot=True,)\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:58.431794Z",
     "iopub.execute_input": "2022-09-28T22:08:58.432237Z",
     "iopub.status.idle": "2022-09-28T22:08:58.794687Z",
     "shell.execute_reply.started": "2022-09-28T22:08:58.432202Z",
     "shell.execute_reply": "2022-09-28T22:08:58.793461Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Scale & Split & Dummies"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "#train\n",
    "le = LabelEncoder()\n",
    "le_count = 0\n",
    "\n",
    "for col in train:\n",
    "    if train[col].dtype == 'object':\n",
    "        if len(list(train[col].unique())) <= 2:\n",
    "            le.fit(train[col])\n",
    "            train[col] = le.transform(train[col])\n",
    "                      \n",
    "            le_count += 1\n",
    "            print('%d columns were label encoded.' % le_count)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:58.797566Z",
     "iopub.execute_input": "2022-09-28T22:08:58.798039Z",
     "iopub.status.idle": "2022-09-28T22:08:59.091743Z",
     "shell.execute_reply.started": "2022-09-28T22:08:58.798002Z",
     "shell.execute_reply": "2022-09-28T22:08:59.090205Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "train_dummies=pd.get_dummies(train)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:59.094877Z",
     "iopub.execute_input": "2022-09-28T22:08:59.095415Z",
     "iopub.status.idle": "2022-09-28T22:08:59.20739Z",
     "shell.execute_reply.started": "2022-09-28T22:08:59.095349Z",
     "shell.execute_reply": "2022-09-28T22:08:59.205852Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "x=train_dummies.drop(columns='Response',axis=1)\n",
    "y=train_dummies['Response']\n",
    "\n",
    "#split\n",
    "X_train, X_test, y_train, y_test = train_test_split(x, y, test_size = 0.2, random_state = 11)\n",
    "#scale\n",
    "sc = StandardScaler()\n",
    "X_train = sc.fit_transform(X_train)\n",
    "X_test = sc.transform(X_test)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:59.208995Z",
     "iopub.execute_input": "2022-09-28T22:08:59.210012Z",
     "iopub.status.idle": "2022-09-28T22:08:59.403643Z",
     "shell.execute_reply.started": "2022-09-28T22:08:59.209971Z",
     "shell.execute_reply": "2022-09-28T22:08:59.402414Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Cat Boost"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "clf = CatBoostClassifier(\n",
    "    iterations=100, \n",
    "    learning_rate=0.001, \n",
    "    verbose=5,\n",
    ")\n",
    "\n",
    "clf.fit(\n",
    "    X_train, y_train,\n",
    "    eval_set=(X_test, y_test),\n",
    ")"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:08:59.405567Z",
     "iopub.execute_input": "2022-09-28T22:08:59.406081Z",
     "iopub.status.idle": "2022-09-28T22:09:02.953303Z",
     "shell.execute_reply.started": "2022-09-28T22:08:59.406031Z",
     "shell.execute_reply": "2022-09-28T22:09:02.952088Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "clf_pred=clf.predict(X_test)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:02.95475Z",
     "iopub.execute_input": "2022-09-28T22:09:02.955206Z",
     "iopub.status.idle": "2022-09-28T22:09:02.988541Z",
     "shell.execute_reply.started": "2022-09-28T22:09:02.95517Z",
     "shell.execute_reply": "2022-09-28T22:09:02.987074Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "cm = confusion_matrix(y_test, clf_pred)\n",
    "print(cm)\n",
    "print(accuracy_score(y_test, clf_pred))\n",
    "plt.figure(figsize=(10,5))\n",
    "sns.heatmap(cm, annot=True, fmt=\"d\", linewidths=.5)\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:02.990112Z",
     "iopub.execute_input": "2022-09-28T22:09:02.990702Z",
     "iopub.status.idle": "2022-09-28T22:09:03.244397Z",
     "shell.execute_reply.started": "2022-09-28T22:09:02.990664Z",
     "shell.execute_reply": "2022-09-28T22:09:03.242809Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Model Interruption\n",
    "* First of all, we get an 87% accuracy score, but it does not mean we got a good model. The model cannot predict true negative(TN) and False Negative (FN) values. That problem comes from an imbalanced data problem\n",
    "* Most customers were not interested (86%), while interested customers occurred (14%) of the time in the data frame.\n",
    "\n",
    "### Sub Sample\n",
    "* In this scenario, our subsample will be a data frame with a 50/50 ratio of interested and non-interested customers. Which means our sub-sample has the same amount of responses.\n",
    "\n",
    "* Our dataset has 46710 cases of non-interested customers, so we can get 46710 cases of interested customers to create our new sub-data frame. We concat the 46710 cases of interested and non-interested customers for creating a new sub-sample."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "plt.figure(figsize=(12,6))\n",
    "sns.countplot(train.Response)\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:03.247963Z",
     "iopub.execute_input": "2022-09-28T22:09:03.248345Z",
     "iopub.status.idle": "2022-09-28T22:09:03.614355Z",
     "shell.execute_reply.started": "2022-09-28T22:09:03.248309Z",
     "shell.execute_reply": "2022-09-28T22:09:03.613364Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "non_interested=train[train.Response==1]\n",
    "interested=train[train.Response==0][:46710]\n",
    "df=pd.concat([non_interested,interested])\n",
    "data=df.sample(frac=1,random_state=42)\n",
    "data.head()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:03.615577Z",
     "iopub.execute_input": "2022-09-28T22:09:03.61615Z",
     "iopub.status.idle": "2022-09-28T22:09:03.686919Z",
     "shell.execute_reply.started": "2022-09-28T22:09:03.616112Z",
     "shell.execute_reply": "2022-09-28T22:09:03.685728Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "plt.figure(figsize=(12,6))\n",
    "sns.countplot(data.Response)\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:03.688628Z",
     "iopub.execute_input": "2022-09-28T22:09:03.689373Z",
     "iopub.status.idle": "2022-09-28T22:09:03.856288Z",
     "shell.execute_reply.started": "2022-09-28T22:09:03.689323Z",
     "shell.execute_reply": "2022-09-28T22:09:03.855162Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "data1=pd.get_dummies(data)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:03.857589Z",
     "iopub.execute_input": "2022-09-28T22:09:03.859088Z",
     "iopub.status.idle": "2022-09-28T22:09:03.893167Z",
     "shell.execute_reply.started": "2022-09-28T22:09:03.859052Z",
     "shell.execute_reply": "2022-09-28T22:09:03.891741Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "x=data1.drop(columns='Response',axis=1)\n",
    "y=data1['Response']\n",
    "\n",
    "#split\n",
    "X_train, X_test, y_train, y_test = train_test_split(x, y, test_size = 0.2, random_state = 11)\n",
    "#scale\n",
    "sc = StandardScaler()\n",
    "X_train = sc.fit_transform(X_train)\n",
    "X_test = sc.transform(X_test)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:03.895258Z",
     "iopub.execute_input": "2022-09-28T22:09:03.895772Z",
     "iopub.status.idle": "2022-09-28T22:09:03.950321Z",
     "shell.execute_reply.started": "2022-09-28T22:09:03.895724Z",
     "shell.execute_reply": "2022-09-28T22:09:03.948972Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Cat Boost"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "\n",
    "clf = CatBoostClassifier(\n",
    "    iterations=100, \n",
    "    learning_rate=0.001, \n",
    "    verbose=20,\n",
    ")\n",
    "\n",
    "clf.fit(\n",
    "    X_train, y_train,\n",
    "    eval_set=(X_test, y_test),\n",
    ")"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:03.9519Z",
     "iopub.execute_input": "2022-09-28T22:09:03.952281Z",
     "iopub.status.idle": "2022-09-28T22:09:04.850718Z",
     "shell.execute_reply.started": "2022-09-28T22:09:03.952241Z",
     "shell.execute_reply": "2022-09-28T22:09:04.849477Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "clf_pred = clf.predict(X_test)\n",
    "cm = confusion_matrix(y_test, clf_pred)\n",
    "print(classification_report(y_test,clf_pred))\n",
    "print('Accuracy',accuracy_score(y_test, clf_pred))\n",
    "plt.figure(figsize=(10,5))\n",
    "sns.heatmap(cm, annot=True, fmt=\"d\", linewidths=.5)\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:04.852193Z",
     "iopub.execute_input": "2022-09-28T22:09:04.852614Z",
     "iopub.status.idle": "2022-09-28T22:09:05.145526Z",
     "shell.execute_reply.started": "2022-09-28T22:09:04.852582Z",
     "shell.execute_reply": "2022-09-28T22:09:05.14398Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Light GBM"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "params = {\n",
    "    'learning_rate': 0.005,\n",
    "    \"max_depth\": 12,\n",
    "    \"num_leaves\": 24,  \n",
    "    \"max_bin\": 512,\n",
    "    \"num_iterations\": 10000\n",
    "}\n",
    "model = lgb.LGBMClassifier(**params)\n",
    "model.fit(X_train,y_train,\n",
    "          eval_set=[(X_test, y_test)],\n",
    "        early_stopping_rounds=1000)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:05.149748Z",
     "iopub.execute_input": "2022-09-28T22:09:05.150131Z",
     "iopub.status.idle": "2022-09-28T22:09:21.611232Z",
     "shell.execute_reply.started": "2022-09-28T22:09:05.150098Z",
     "shell.execute_reply": "2022-09-28T22:09:21.609973Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "model_pred = model.predict(X_test)\n",
    "cm = confusion_matrix(y_test, model_pred)\n",
    "print(classification_report(y_test,model_pred))\n",
    "print('Accuracy',accuracy_score(y_test, model_pred))\n",
    "plt.figure(figsize=(10,5))\n",
    "sns.heatmap(cm, annot=True, fmt=\"d\", linewidths=.5,cmap='turbo_r')\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:21.612974Z",
     "iopub.execute_input": "2022-09-28T22:09:21.614255Z",
     "iopub.status.idle": "2022-09-28T22:09:23.214716Z",
     "shell.execute_reply.started": "2022-09-28T22:09:21.61419Z",
     "shell.execute_reply": "2022-09-28T22:09:23.213478Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# XG Boost"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "xg=xgb.XGBClassifier(\n",
    "                    n_estimators =1000\n",
    "                     )\n",
    "    \n",
    "evaluation = [( X_train, y_train), ( X_test, y_test)]\n",
    "    \n",
    "xg.fit(X_train, y_train,\n",
    "            eval_set=evaluation, eval_metric=\"auc\",\n",
    "            early_stopping_rounds=20,verbose=100)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:23.216294Z",
     "iopub.execute_input": "2022-09-28T22:09:23.216643Z",
     "iopub.status.idle": "2022-09-28T22:09:25.765075Z",
     "shell.execute_reply.started": "2022-09-28T22:09:23.216612Z",
     "shell.execute_reply": "2022-09-28T22:09:25.763709Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "xg_pred = xg.predict(X_test)\n",
    "cm = confusion_matrix(y_test, xg_pred)\n",
    "print(classification_report(y_test,xg_pred))\n",
    "print('Accuracy',accuracy_score(y_test, xg_pred))\n",
    "plt.figure(figsize=(10,5))\n",
    "sns.heatmap(cm, annot=True, fmt=\"d\", linewidths=.5,cmap='tab20')\n",
    "plt.show()"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:25.767313Z",
     "iopub.execute_input": "2022-09-28T22:09:25.767687Z",
     "iopub.status.idle": "2022-09-28T22:09:26.044588Z",
     "shell.execute_reply.started": "2022-09-28T22:09:25.767655Z",
     "shell.execute_reply": "2022-09-28T22:09:26.041992Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "print('Cat Boost',roc_auc_score(y_test, clf_pred))\n",
    "print('XG Boost',roc_auc_score(y_test, xg_pred))\n",
    "print('Light GBM',roc_auc_score(y_test, model_pred))"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:26.046234Z",
     "iopub.execute_input": "2022-09-28T22:09:26.046987Z",
     "iopub.status.idle": "2022-09-28T22:09:26.072039Z",
     "shell.execute_reply.started": "2022-09-28T22:09:26.046939Z",
     "shell.execute_reply": "2022-09-28T22:09:26.071102Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "fpr, tpr, threshold = roc_curve(y_test, clf_pred)\n",
    "fpr1, tpr1, threshold1 = roc_curve(y_test, xg_pred)\n",
    "fpr2, tpr2, threshold1 = roc_curve(y_test, model_pred)\n",
    "roc_auc =auc(fpr, tpr)\n",
    "\n",
    "plt.title('Receiver Operating Characteristic')\n",
    "plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc,color='red')\n",
    "plt.plot(fpr1, tpr1,  label = 'AUC = %0.2f' % roc_auc,color='yellow')\n",
    "plt.plot(fpr2, tpr2,  label = 'AUC = %0.2f' % roc_auc,color='green')\n",
    "\n",
    "\n",
    "plt.plot([0, 1], [0, 1],'r--')\n",
    "plt.xlim([0, 1])\n",
    "plt.ylim([0, 1])\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.show()\n"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:26.075726Z",
     "iopub.execute_input": "2022-09-28T22:09:26.076697Z",
     "iopub.status.idle": "2022-09-28T22:09:26.224936Z",
     "shell.execute_reply.started": "2022-09-28T22:09:26.076658Z",
     "shell.execute_reply": "2022-09-28T22:09:26.223633Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Model Evaluation\n",
    "* Light Gbm performs well than other models at this time. \n",
    "* Accuracy was 79.8%, and we got a 0.80  AUC score which we can interpret as good. \n",
    "* For further interruption, I used shap library plots to understand features' importance and roles"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Shap"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "explainer = shap.Explainer(model, X_train,feature_names=x.columns.values.tolist())\n",
    "shap_values = explainer(X_test)\n",
    "plt.title('Feature Importance')\n",
    "shap.plots.waterfall(shap_values[0],max_display=12)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-09-28T22:09:26.226793Z",
     "iopub.execute_input": "2022-09-28T22:09:26.227557Z"
    },
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 33%|=======             | 6220/18684 [08:50<17:42]       "
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "source": [
    "shap.plots.beeswarm(shap_values,max_display=12)"
   ],
   "metadata": {
    "trusted": true,
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Conclusion\n",
    "* Previously insured, Age, and Vehicle Damage are significant features.\n",
    "* We got fewer Shap Values where Previously Insured value equal to 1. In other words, most previous customers didn't buy the insurance. The company should consider the reason behind that result. The trigger should be communication strategy, irrelevant product, etc.\n",
    "* Shap values get minor numbers, whereas vehicle damage gets a small value. We assume that situation is normal because people used not to think about insurance unless they got an accident. The company should consider starting a marketing campaign for that purpose.\n",
    "* The insurance annual premium prices affected shap value 0.12, which needs considering and further research about pricing strategy."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [],
   "metadata": {
    "pycharm": {
     "is_executing": true
    }
   },
   "execution_count": null,
   "outputs": []
  }
 ]
}