Day-13-b-exercise-cross-validation.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**This notebook is an exercise in the [Intermediate Machine Learning](https://www.kaggle.com/learn/intermediate-machine-learning) course.  You can reference the tutorial at [this link](https://www.kaggle.com/alexisbcook/cross-validation).**\n\n---\n","metadata":{}},{"cell_type":"markdown","source":"In this exercise, you will leverage what you've learned to tune a machine learning model with **cross-validation**.\n\n# Setup\n\nThe questions below will give you feedback on your work. Run the following cell to set up the feedback system.","metadata":{}},{"cell_type":"code","source":"# Set up code checking\nimport os\nif not os.path.exists(\"../input/train.csv\"):\n    os.symlink(\"../input/home-data-for-ml-course/train.csv\", \"../input/train.csv\")  \n    os.symlink(\"../input/home-data-for-ml-course/test.csv\", \"../input/test.csv\") \nfrom learntools.core import binder\nbinder.bind(globals())\nfrom learntools.ml_intermediate.ex5 import *\nprint(\"Setup Complete\")","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:51:20.754730Z","iopub.execute_input":"2021-08-17T08:51:20.755070Z","iopub.status.idle":"2021-08-17T08:51:20.907321Z","shell.execute_reply.started":"2021-08-17T08:51:20.755041Z","shell.execute_reply":"2021-08-17T08:51:20.906057Z"},"trusted":true},"execution_count":2,"outputs":[{"name":"stdout","text":"Setup Complete\n","output_type":"stream"}]},{"cell_type":"markdown","source":"You will work with the [Housing Prices Competition for Kaggle Learn Users](https://www.kaggle.com/c/home-data-for-ml-course) from the previous exercise. \n\n![Ames Housing dataset image](https://i.imgur.com/lTJVG4e.png)\n\nRun the next code cell without changes to load the training and test data in `X` and `X_test`.  For simplicity, we drop categorical variables.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nfrom sklearn.model_selection import train_test_split\n\n# Read the data\ntrain_data = pd.read_csv('../input/train.csv', index_col='Id')\ntest_data = pd.read_csv('../input/test.csv', index_col='Id')\n\n# Remove rows with missing target, separate target from predictors\ntrain_data.dropna(axis=0, subset=['SalePrice'], inplace=True)\ny = train_data.SalePrice              \ntrain_data.drop(['SalePrice'], axis=1, inplace=True)\n\n# Select numeric columns only\nnumeric_cols = [cname for cname in train_data.columns if train_data[cname].dtype in ['int64', 'float64']]\nX = train_data[numeric_cols].copy()\nX_test = test_data[numeric_cols].copy()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:51:20.909337Z","iopub.execute_input":"2021-08-17T08:51:20.909795Z","iopub.status.idle":"2021-08-17T08:51:22.094967Z","shell.execute_reply.started":"2021-08-17T08:51:20.909750Z","shell.execute_reply":"2021-08-17T08:51:22.093410Z"},"trusted":true},"execution_count":3,"outputs":[]},{"cell_type":"markdown","source":"Use the next code cell to print the first several rows of the data.","metadata":{}},{"cell_type":"code","source":"X.head()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:51:22.097030Z","iopub.execute_input":"2021-08-17T08:51:22.097436Z","iopub.status.idle":"2021-08-17T08:51:22.135615Z","shell.execute_reply.started":"2021-08-17T08:51:22.097400Z","shell.execute_reply":"2021-08-17T08:51:22.134613Z"},"trusted":true},"execution_count":4,"outputs":[{"execution_count":4,"output_type":"execute_result","data":{"text/plain":"    MSSubClass  LotFrontage  LotArea  OverallQual  OverallCond  YearBuilt  \\\nId                                                                          \n1           60         65.0     8450            7            5       2003   \n2           20         80.0     9600            6            8       1976   \n3           60         68.0    11250            7            5       2001   \n4           70         60.0     9550            7            5       1915   \n5           60         84.0    14260            8            5       2000   \n\n    YearRemodAdd  MasVnrArea  BsmtFinSF1  BsmtFinSF2  ...  GarageArea  \\\nId                                                    ...               \n1           2003       196.0         706           0  ...         548   \n2           1976         0.0         978           0  ...         460   \n3           2002       162.0         486           0  ...         608   \n4           1970         0.0         216           0  ...         642   \n5           2000       350.0         655           0  ...         836   \n\n    WoodDeckSF  OpenPorchSF  EnclosedPorch  3SsnPorch  ScreenPorch  PoolArea  \\\nId                                                                             \n1            0           61              0          0            0         0   \n2          298            0              0          0            0         0   \n3            0           42              0          0            0         0   \n4            0           35            272          0            0         0   \n5          192           84              0          0            0         0   \n\n    MiscVal  MoSold  YrSold  \nId                           \n1         0       2    2008  \n2         0       5    2007  \n3         0       9    2008  \n4         0       2    2006  \n5         0      12    2008  \n\n[5 rows x 36 columns]","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>MSSubClass</th>\n      <th>LotFrontage</th>\n      <th>LotArea</th>\n      <th>OverallQual</th>\n      <th>OverallCond</th>\n      <th>YearBuilt</th>\n      <th>YearRemodAdd</th>\n      <th>MasVnrArea</th>\n      <th>BsmtFinSF1</th>\n      <th>BsmtFinSF2</th>\n      <th>...</th>\n      <th>GarageArea</th>\n      <th>WoodDeckSF</th>\n      <th>OpenPorchSF</th>\n      <th>EnclosedPorch</th>\n      <th>3SsnPorch</th>\n      <th>ScreenPorch</th>\n      <th>PoolArea</th>\n      <th>MiscVal</th>\n      <th>MoSold</th>\n      <th>YrSold</th>\n    </tr>\n    <tr>\n      <th>Id</th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>1</th>\n      <td>60</td>\n      <td>65.0</td>\n      <td>8450</td>\n      <td>7</td>\n      <td>5</td>\n      <td>2003</td>\n      <td>2003</td>\n      <td>196.0</td>\n      <td>706</td>\n      <td>0</td>\n      <td>...</td>\n      <td>548</td>\n      <td>0</td>\n      <td>61</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>2</td>\n      <td>2008</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>20</td>\n      <td>80.0</td>\n      <td>9600</td>\n      <td>6</td>\n      <td>8</td>\n      <td>1976</td>\n      <td>1976</td>\n      <td>0.0</td>\n      <td>978</td>\n      <td>0</td>\n      <td>...</td>\n      <td>460</td>\n      <td>298</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>5</td>\n      <td>2007</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>60</td>\n      <td>68.0</td>\n      <td>11250</td>\n      <td>7</td>\n      <td>5</td>\n      <td>2001</td>\n      <td>2002</td>\n      <td>162.0</td>\n      <td>486</td>\n      <td>0</td>\n      <td>...</td>\n      <td>608</td>\n      <td>0</td>\n      <td>42</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>9</td>\n      <td>2008</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>70</td>\n      <td>60.0</td>\n      <td>9550</td>\n      <td>7</td>\n      <td>5</td>\n      <td>1915</td>\n      <td>1970</td>\n      <td>0.0</td>\n      <td>216</td>\n      <td>0</td>\n      <td>...</td>\n      <td>642</td>\n      <td>0</td>\n      <td>35</td>\n      <td>272</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>2</td>\n      <td>2006</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>60</td>\n      <td>84.0</td>\n      <td>14260</td>\n      <td>8</td>\n      <td>5</td>\n      <td>2000</td>\n      <td>2000</td>\n      <td>350.0</td>\n      <td>655</td>\n      <td>0</td>\n      <td>...</td>\n      <td>836</td>\n      <td>192</td>\n      <td>84</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>12</td>\n      <td>2008</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 36 columns</p>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"So far, you've learned how to build pipelines with scikit-learn.  For instance, the pipeline below will use [`SimpleImputer()`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html) to replace missing values in the data, before using [`RandomForestRegressor()`](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html) to train a random forest model to make predictions.  We set the number of trees in the random forest model with the `n_estimators` parameter, and setting `random_state` ensures reproducibility.","metadata":{}},{"cell_type":"code","source":"from sklearn.ensemble import RandomForestRegressor\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.impute import SimpleImputer\n\nmy_pipeline = Pipeline(steps=[\n    ('preprocessor', SimpleImputer()),\n    ('model', RandomForestRegressor(n_estimators=50, random_state=0))\n])","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:51:22.137206Z","iopub.execute_input":"2021-08-17T08:51:22.137733Z","iopub.status.idle":"2021-08-17T08:51:22.384458Z","shell.execute_reply.started":"2021-08-17T08:51:22.137695Z","shell.execute_reply":"2021-08-17T08:51:22.383277Z"},"trusted":true},"execution_count":5,"outputs":[]},{"cell_type":"markdown","source":"You have also learned how to use pipelines in cross-validation.  The code below uses the [`cross_val_score()`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html) function to obtain the mean absolute error (MAE), averaged across five different folds.  Recall we set the number of folds with the `cv` parameter.","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import cross_val_score\n\n# Multiply by -1 since sklearn calculates *negative* MAE\nscores = -1 * cross_val_score(my_pipeline, X, y,\n                              cv=5,\n                              scoring='neg_mean_absolute_error')\n\nprint(\"Average MAE score:\", scores.mean())","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:51:23.904263Z","iopub.execute_input":"2021-08-17T08:51:23.904730Z","iopub.status.idle":"2021-08-17T08:51:27.518425Z","shell.execute_reply.started":"2021-08-17T08:51:23.904683Z","shell.execute_reply":"2021-08-17T08:51:27.517418Z"},"trusted":true},"execution_count":6,"outputs":[{"name":"stdout","text":"Average MAE score: 18276.410356164386\n","output_type":"stream"}]},{"cell_type":"markdown","source":"# Step 1: Write a useful function\n\nIn this exercise, you'll use cross-validation to select parameters for a machine learning model.\n\nBegin by writing a function `get_score()` that reports the average (over three cross-validation folds) MAE of a machine learning pipeline that uses:\n- the data in `X` and `y` to create folds,\n- `SimpleImputer()` (with all parameters left as default) to replace missing values, and\n- `RandomForestRegressor()` (with `random_state=0`) to fit a random forest model.\n\nThe `n_estimators` parameter supplied to `get_score()` is used when setting the number of trees in the random forest model.  ","metadata":{}},{"cell_type":"code","source":"def get_score(n_estimators):\n    my_pipeline = Pipeline(steps=[\n        ('preprocessor', SimpleImputer()),\n        ('model', RandomForestRegressor(n_estimators, random_state=0))\n    ])\n    scores = -1 * cross_val_score(my_pipeline, X, y,\n                                  cv=3,\n                                  scoring='neg_mean_absolute_error')\n    return scores.mean()\n\n# Check your answer\nstep_1.check()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:52:34.406568Z","iopub.execute_input":"2021-08-17T08:52:34.406948Z","iopub.status.idle":"2021-08-17T08:52:35.575519Z","shell.execute_reply.started":"2021-08-17T08:52:34.406901Z","shell.execute_reply":"2021-08-17T08:52:35.574428Z"},"trusted":true},"execution_count":9,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"1_GetScore\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Correct","text/markdown":"<span style=\"color:#33cc33\">Correct</span>"},"metadata":{}}]},{"cell_type":"code","source":"# Lines below will give you a hint or solution code\n#step_1.hint()\n#step_1.solution()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:52:50.420945Z","iopub.execute_input":"2021-08-17T08:52:50.421348Z","iopub.status.idle":"2021-08-17T08:52:50.425720Z","shell.execute_reply.started":"2021-08-17T08:52:50.421294Z","shell.execute_reply":"2021-08-17T08:52:50.424534Z"},"trusted":true},"execution_count":10,"outputs":[]},{"cell_type":"markdown","source":"# Step 2: Test different parameter values\n\nNow, you will use the function that you defined in Step 1 to evaluate the model performance corresponding to eight different values for the number of trees in the random forest: 50, 100, 150, ..., 300, 350, 400.\n\nStore your results in a Python dictionary `results`, where `results[i]` is the average MAE returned by `get_score(i)`.","metadata":{}},{"cell_type":"code","source":"results = {}\nfor i in range(1,9):\n    results[50*i] = get_score(50*i)# Check your answer\nstep_2.check()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:53:23.571831Z","iopub.execute_input":"2021-08-17T08:53:23.572410Z","iopub.status.idle":"2021-08-17T08:54:28.301449Z","shell.execute_reply.started":"2021-08-17T08:53:23.572371Z","shell.execute_reply":"2021-08-17T08:54:28.300468Z"},"trusted":true},"execution_count":12,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"2_GetDict\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Correct","text/markdown":"<span style=\"color:#33cc33\">Correct</span>"},"metadata":{}}]},{"cell_type":"code","source":"# Lines below will give you a hint or solution code\n#step_2.hint()\n#step_2.solution()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:54:28.303024Z","iopub.execute_input":"2021-08-17T08:54:28.303311Z","iopub.status.idle":"2021-08-17T08:54:28.306464Z","shell.execute_reply.started":"2021-08-17T08:54:28.303283Z","shell.execute_reply":"2021-08-17T08:54:28.305770Z"},"trusted":true},"execution_count":13,"outputs":[]},{"cell_type":"markdown","source":"Use the next cell to visualize your results from Step 2.  Run the code without changes.","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n%matplotlib inline\n\nplt.plot(list(results.keys()), list(results.values()))\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:54:28.307690Z","iopub.execute_input":"2021-08-17T08:54:28.308094Z","iopub.status.idle":"2021-08-17T08:54:28.515190Z","shell.execute_reply.started":"2021-08-17T08:54:28.308064Z","shell.execute_reply":"2021-08-17T08:54:28.513761Z"},"trusted":true},"execution_count":14,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAYMAAAD5CAYAAADFqlkBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqu0lEQVR4nO3deXxU9b3/8dcnCwkESIAsIgEhohBciBDQ1gULwdrFi1Zbtdpyq1da7SZdrtf2V631dnGrWq/L9apXrVVbrdttaxWQaq1iDcgmi0RECCAJhC1AIMvn98ecwBATyH5mMu/n45FHznzO98x85kDmM+f7/Z5zzN0REZHElhR2AiIiEj4VAxERUTEQEREVAxERQcVARERQMRARESClNY3M7CHg80CFux8fxIqA+4B0oA64yt3/GbXNBOBN4CJ3fzqITQf+X9DkP939kSA+HngY6A38BfiuH2bOa3Z2tg8fPrxVb1JERCLmz5+/2d1zmsatNecZmNkZQDXwaFQxeBm43d1fNLPPAv/u7mcG65KBWUAN8JC7P21mA4FSoBhwYD4w3t23mtk/ge8AbxEpBr9x9xcPlVNxcbGXlpa27t2LiAgAZjbf3YubxlvVTeTurwFVTcNA/2A5E9gQte7bwB+BiqjYp4FZ7l7l7luJFIuzzWww0N/d5wVHA48C57YmLxER6Ryt6iZqwdXAS2Z2K5Gi8kkAMxsCnAd8CpgQ1X4IsC7qcXkQGxIsN42LiEg36cgA8pXATHcfCswEHgzidwDXuHtDB3P7GDObYWalZlZaWVnZ2U8vIpKwOlIMpgPPBMtPAROD5WLgSTNbA1wA3GNm5wLrgaFR2+cHsfXBctP4x7j7/e5e7O7FOTkfG/8QEZF26kgx2ABMCpYnA6sA3H2Euw939+HA00RmGT0HvAScZWYDzGwAcBbwkrtvBHaY2SlmZsBXgec7kJeIiLRRa6eWPgGcCWSbWTlwPXAFcKeZpRCZNTTjUM/h7lVmdiPwdhD6mbs3DkpfxYGppS8GPyIi0k1aNbU0FmlqqYhI23VoaqmEo6a2nt++uYaa2vqwUxGRHq4jU0ulC7k7P3pmCc+8s55eKUlcOGFY2CmJSA+mI4MY9dhba3nmnfUkGcxeXnH4DUREOkBHBjFowdqt/Oz/3uVTo3IYMqA3T88vp6a2nvTU5LBTE5EeSkcGMWZz9V6uemwBgzN7c8eFJ3HWmCOoqW3gjfc3h52aiPRgKgYxpK6+gW8//g5bd+/j3kvHkdknlZMLBtI3LYVZy9RVJCJdR8Ughtzy8kreXL2Fn593AscdmQlAWkoyZxybzZzlm2hoiM9pwCIS+1QMYsRfl27kv19dzSUnD+OC8fkHrSspzKNi516WbtgeUnYi0tOpGMSA9yur+cFTixk7NIvrzhnzsfWfGpUbmVW0bFMI2YlIIlAxCNmuvXV847fz6ZWSxL2XjCMt5eMzhgZk9KL4qIHM0hRTEekiKgYhcneu+eNi3q+s5q6LT+LIrN4tti0Zk8vyjTtYv21PN2YoIolCxSBED/1jDX9avJEffno0p47MPmTbksI8AOYsV1eRiHQ+FYOQ/PODKn7xl+V8+rg8vjGp4LDtC3L6UpCdwSyNG4hIF1AxCEHFjhq++fgCjhrYh1u+OJbIbRwOr2RMHvNWb2FnTW0XZygiiUbFoJvV1jdw1e8WUF1Tx31fGU//9NRWb1tSmEdtvfP3VTobWUQ6l4pBN/vlX1ZQ+uFWfnX+CRyb169N244blkVWn1RNMRWRTqdi0I1eWLSBh/7xAV87dTjTioa0efuU5CQmj8pl7soK6uobuiBDEUlUKgbd5L1NO7nm6cUUHzWAH322sN3PUzImj627a1mwdlvnJSciCa9VxcDMHjKzCjNbGhUrMrN5ZrbQzErNbGIQn2Zmi6Pip0Vtc7OZvWtmy83sNxaMnJrZeDNbYmZl0fGeYkdNLd/47Xz6pqdwzyXjSE1ufw0+/ZhsUpON2ZpiKiKdqLWfSg8DZzeJ3Qzc4O5FwHXBY4A5wNggfhnwAICZfRI4FTgROB6YAEwKtrkXuAI4Jvhp+lpxy935wR8W8WHVbu7+8jhy+6d36Pn6padySsEgFQMR6VStKgbu/hpQ1TQM9A+WM4ENQdtqd2+8vGZG0K6xfTrQC0gDUoFNZjYY6O/u84LtHgXObde7iUH3vbqal5dt4kefLWTiiIGd8pxTx+SxunIX71dWd8rziYh0ZMzgauAWM1sH3Apc27jCzM4zsxXAn4kcHeDubwJzgY3Bz0vuvhwYApRHPW95EIt7/yjbzC0vreDzJw7mslOHd9rzTh6dC+hsZBHpPB0pBlcCM919KDATeLBxhbs/6+6jiXzDvxHAzEYChUA+kQ/7yWZ2elte0MxmBOMQpZWVlR1Ivett2LaHbz/xDkfn9OWm809s9YllrZE/oA+Fg/vr3sgi0mk6UgymA88Ey08BE5s2CLqXCswsGzgPmBd0I1UDLwKfANYTKRCN8oPYx7j7/e5e7O7FOTk5HUi9a+2tq+eq3y1gX10D931lPBlpnX+r6amFuZSuqWLrrn2d/twikng6Ugw2cGAAeDKwCiJHAFGzhMYRGR/YAqwFJplZipmlBtsud/eNwA4zOyXY7qvA8x3IK3Q3/mkZC9dt49YvnsjROX275DWmFObR4DB3pY4ORKTjWvWV1cyeAM4Ess2sHLieyOyfO80sBagBZgTNzwe+ama1wB7gQnd3M3uaSNFYQmQw+a/u/n/BNlcRmbHUm8gRw4sdf2vheHp+OY/NW8vXJxVw9vGDu+x1ThiSSW6/NOYsr+AL4/IPv4GIyCG0qhi4+8UtrBrfTNubgJuaidcDX2/h+UuJTDeNa+9u2M6Pn13CJwoG8cOzRnXpayUlGVMKc/m/RRvZW1ff7E1xRERaS2cgd5Ltu2v5xmPzGdCnF3d9+SRSOnBiWWuVFOZRvbeOt1Y3nfUrItI2KgadoKHBufr37/DR9hruuXQc2X3TuuV1Tx2ZTXpqkqaYikiHqRh0grteKWPuykquO+c4xg0b0G2vm56azGkjc5i9vIID5/mJiLSdikEH/W1lBXfMeY8vjBvCpScP6/bXnzoml/Xb9rB8485uf20R6TlUDDpgXdVuvvvkQkYf0Z+fn3tCp55Y1lqTR+dhprORRaRjVAzaqaa2nit/N58Gd+67dBy9e4UzmyenXxpj87N04ToR6RAVg3Zwd37y3FKWrt/BHRcWcdSgjFDzmTomj0Xl29m0oybUPEQkfqkYtMOTb6/jqfnlfGfySKYU5oWdDiVBDq+s0NnIItI+KgZttGjdNq5//l3OODaH75YcG3Y6AByb15f8Ab11b2QRaTcVgzao2rWPKx+bT06/NO68sIjkpNi4IZuZUVKYx+tlm9mzrz7sdEQkDqkYtFJ9g/OdJ95h86593HfpeAZk9Ao7pYNMHZPH3roGXi/bHHYqIhKHVAxa6fZZ7/F62Wb+c9rxnJCfGXY6HzNh+ED6paWoq0hE2kXFoBVmLdvEf80t4+KJQ/nShKFhp9OsXilJTBqVw5wVFTQ06GxkEWkbFYPD+GDzLr73+4WcmJ/J9eccF3Y6hzR1TB6bq/eyqHxb2KmISJxRMTiE3fvquPKx+SQnG/dcMo701Ni+TPSZx+aSnGQ6AU1E2kzFoAXuzrXPLGHlpp385qKTyB/QJ+yUDiuzTyoThg9g9jKdbyAibaNi0IJH3/yQ5xdu4PtTj+WMY2P3fstNlRTmsXLTTtZV7Q47FRGJIyoGzZj/YRU3/mkZJYW5XHXmyLDTaZPGs5HVVSQibXHYYmBmD5lZhZktjYoVmdk8M1toZqVmNjGITzOzxVHx06K2GWZmL5vZcjNbZmbDg/gIM3vLzMrM7PdmFuoE/sqde7nqdwsYMqA3t32piKQYObGstYZnZzAyt6+KgYi0SWuODB4Gzm4Suxm4wd2LgOuCxwBzgLFB/DLggahtHgVucfdCYCLQ2LF9E3C7u48EtgKXt/lddJK6+ga+9fgCtu+p5b5Lx5PZOzWsVDqkpDCPt1ZXsaOmNuxURCROHLYYuPtrQNOb7DrQP1jOBDYEbav9wC23MoJ2mNkYIMXdZ0W1222RGwBMBp4OtnkEOLfd76aDbn5pJW99UMUvv3AChYP7H36DGFVSmEtdg/PqysqwUxGRONHeMYOrgVvMbB1wK3Bt4wozO8/MVgB/JnJ0AHAssM3MnjGzd8zsFjNLBgYB29y9LmhXDgxpZ04d8pclG7n/tdVM/8RRnHdSfhgpdJqThg1gYEYvdRWJSKu1txhcCcx096HATODBxhXu/qy7jybyDf/GIJwCnA78AJgAFAD/2tYXNbMZwVhEaWVl533rLavYyQ+fWsS4YVn8+HNjOu15w5KcZEwencvcFRXU1jeEnY6IxIH2FoPpwDPB8lNExgAOEnQvFZhZNpFv/AvdfXVwFPAcMA7YAmSZWUqwWT6wvqUXdff73b3Y3Ytzcjpnumf13jq+/tv59O6VzD2XjKdXSs+YYFVSmMuOmjpK12wNOxURiQPt/eTbAEwKlicDqwDMbGQwDoCZjQPSiHzgv03kQz8naptlwfjCXOCCID4deL6dObWZu/PvTy9izZbd3HXxOI7ITO+ul+5ypx+TQ6/kJHUViUirtGZq6RPAm8AoMys3s8uBK4DbzGwR8AtgRtD8fGCpmS0E7gYu9Ih6Il1Ec8xsCWDA/wTbXAN8z8zKiIwh7O9y6moP/P0D/rLkI645exSfOHpQd71st8hIS+GTIwcxe/kmDozpi4g0L+VwDdz94hZWjW+m7U1Epoo29zyzgBObia+mmW6mrjZv9RZ+9dcVfOb4I7ji9ILufvluMaUwj588t5T3K6sZmdsv7HREJIb1jA7yNvpoew3fenwBwwf14ZYvjiXo2epxSgpzAZilaxWJyGEkXDHYV9fANx9fwJ599fz3V8bTN+2wB0dxa3Bmb44f0p85GjcQkcNIuGLwi78sZ/6HW7n5grEJ0XUyZXQe89duZUv13rBTEZEYllDFwN0ZnJnO1ycV8LkTB4edTreYOiYPd3hlhbqKRKRlPbePpBlmxtcnHR12Gt3quCP7c0T/dOYsr+CLxbF5y04RCV9CHRkkIjNjSmEur62qpKa2Pux0RCRGqRgkgJIxeezeV8+bq7eEnYqIxCgVgwTwiYJB9OmVrFlFItIiFYMEkJ6azOnHZDN7WYXORhaRZqkYJIiSwjw+2lHDuxt2hJ2KiMQgFYMEMXl0Lma6N7KINE/FIEEM6pvGuGEDVAxEpFkqBgmkpDCPpet3sHH7nrBTEZEYo2KQQKaOiVy4bs5ynY0sIgdTMUggR+f05ahBfdRVJCIfo2KQQMyMksI83ijbwq69dWGnIyIxRMUgwZQU5rGvvoG/r9ocdioiEkNUDBJM8fAB9E9PUVeRiBykVcXAzB4yswozWxoVKzKzeWa20MxKzWxiEJ9mZouj4qc1ea7+wb2U/ysqNt7MlphZmZn9xnrqrcdiQGpyEp8ancsrKyqob9DZyCIS0dojg4eBs5vEbgZucPci4LrgMcAcYGwQvwx4oMl2NwKvNYndC1wBHBP8NH0t6UQlhXlU7drHwnVbw05FRGJEq4qBu78GVDUNA/2D5UxgQ9C22g9cACcjaAdEjgCAPODlqNhgoL+7zwu2exQ4t83vRFpt0qgcUpJM90YWkf06MmZwNXCLma0DbgWubVxhZueZ2Qrgz0SODjCzJOA24AdNnmcIUB71uDyISRfpn57KyQUDNW4gIvt1pBhcCcx096HATODBxhXu/qy7jybyDf/GIHwV8Bd3L2/6RK1lZjOCcYjSysrK9mculBTmUVZRzZrNu8JORURiQEeKwXTgmWD5KWBi0wZB91KBmWUDnwC+ZWZriBxJfNXMfgWsB/KjNssPYh/j7ve7e7G7F+fk5HQgdSkpzAN04ToRiehIMdgATAqWJwOrAMxsZONsIDMbB6QBW9z9Encf5u7DiXQVPeru/+HuG4EdZnZKsN1Xgec7kJe0wtCBfRiV10/FQEQASGlNIzN7AjgTyDazcuB6IrN/7jSzFKAGmBE0P5/It/5aYA9woR/+jipXEZmx1Bt4MfiRLlYyJpf7Xl3N9t21ZPZJDTsdEQmRxeudr4qLi720tDTsNOLagrVb+cI9b3DnRUVMK9KYvUgiMLP57l7cNK4zkBNYUX4W2X17MWuZuopEEp2KQQJLSjKmjM7j1fcq2VfXEHY6IhIiFYMEN6Uwl501dby9puk5hSKSSFQMEtxpx2STlpKkriKRBKdikOD69Erh1JHZzFmxiXidTCAiHadiIJQU5rGuag/vbaoOOxURCYmKgTClMHJvZJ2AJpK4VAyEvP7pnJifqWIgksBUDASIdBUtXLeNip01YaciIiFQMRAgUgzcYe4K3eNAJBGpGAgAhYP7cWRmOrOXqxiIJCIVAwHAzCgZk8ffV1VSU1sfdjoi0s1UDGS/ksI8amob+EfZ5rBTEZFupmIg+51cMJCMXsnqKhJJQCoGsl9aSjKTRuUwZ/kmGhp0NrJIIlExkIOUFOZRsXMvS9ZvDzsVEelGKgZykE+NyiXJYI5OQBNJKCoGcpABGb0oPmogszRuIJJQDlsMzOwhM6sws6VRsSIzm2dmC82s1MwmBvFpZrY4Kn5aVPs3zezdYP2FUc81wszeMrMyM/u9mfXqijcqrVcyJpflG3dQvnV32KmISDdpzZHBw8DZTWI3Aze4exFwXfAYYA4wNohfBjwQxHcDX3X344LnusPMsoJ1NwG3u/tIYCtweXveiHSeKYV5ALyis5FFEsZhi4G7vwY0vQ2WA/2D5UxgQ9C22g9cFD8jaIe7v+fuq4LlDUAFkGNmBkwGng62eQQ4t71vRjrH0Tl9KcjO0A1vRBJISju3uxp4ycxuJVJQPtm4wszOA34J5AKfa7ph0KXUC3gfGARsc/e6YHU5MKSlFzWzGcAMgGHDhrUzdWmNkjF5/O8/PmBnTS390lPDTkdEulh7B5CvBGa6+1BgJvBg4wp3f9bdRxP5hn9j9EZmNhj4LfA1d2/zHdjd/X53L3b34pycnHamLq0xZXQutfXO31fpbGSRRNDeYjAdeCZYfgqY2LRB0L1UYGbZAGbWH/gz8GN3nxc02wJkmVnjEUo+sL6dOUknGn/UALL6pDJbXUUiCaG9xWADMClYngysAjCzkcE4AGY2DkgDtgQzhJ4FHnX3xvEBgvGFucAFQWg68Hw7c5JOlJKcxORRucxdWUFdfZsP4kQkzrRmaukTwJvAKDMrN7PLgSuA28xsEfALgn584HxgqZktBO4GLgw+8L8EnAH8azDtdKGZFQXbXAN8z8zKiIwh7O9yknBNKcxj6+5aFqzdFnYqItLFDjuA7O4Xt7BqfDNtbyIyVbRp/DHgsRaefzXNdDNJ+M44NpvUZGP28k1MHDEw7HREpAvpDGRpUb/0VE4pGKR7I4skABUDOaSSwjxWV+7i/crqsFMRkS6kYiCHNKUwF9CF60R6OhUDOaT8AX0oHNxfN7wR6eFUDOSwSgpzKV1TxdZd+8JORUS6iIqBHFZJYR4NDnNX6uhApKdSMZDDOmFIJrn90pijriKRHkvFQA4rKcmYUpjLq+9VsreuPux0RKQLqBhIq5QU5lG9t463Vje9mrmI9AQqBtIqp47MJj01SVNMRXooFQNplfTUZE4bmcPs5RUcuH+RiPQUKgbSalPH5LJ+2x6Wb9wZdioi0slUDKTVJo/Ow0xnI4v0RCoG0mo5/dIYm5+lC9eJ9EAqBtImU8fksah8O5t21ISdioh0IhUDaZOSwjwAXlmhE9BEehIVA2mTY/P6kj+gt+6NLNLDqBhIm5gZJYV5vF62mT37dDaySE/RqmJgZg+ZWYWZLY2KFZnZvOB+xqVmNjGITzOzxVHx06K2mW5mq4Kf6VHx8Wa2xMzKzOw3Zmad+Salc00dk8feugZeL9scdioi0klae2TwMHB2k9jNwA3uXgRcFzwGmAOMDeKXAQ8AmNlA4HrgZCL3PL7ezAYE29wLXAEcE/w0fS2JIROGD6RfWoq6ikR6kFYVA3d/DWh6URoH+gfLmcCGoG21HzhFNSNoB/BpYJa7V7n7VmAWcLaZDQb6u/u8YLtHgXPb+X6kG/RKSWLSqBzmrKigoUFnI4v0BB0ZM7gauMXM1gG3Atc2rjCz88xsBfBnIkcHAEOAdVHblwexIcFy0/jHmNmMoOuptLKysgOpS0dNHZPH5uq9LCrfFnYqItIJOlIMrgRmuvtQYCbwYOMKd3/W3UcT+YZ/Y4cyjOLu97t7sbsX5+TkdNbTSjuceWwuyUmmE9BEeoiOFIPpwDPB8lNExgEOEnQvFZhZNrAeGBq1Oj+IrQ+Wm8YlhmX2SWXC8AHMXqbzDUR6go4Ugw3ApGB5MrAKwMxGNs4GMrNxQBqwBXgJOMvMBgQDx2cBL7n7RmCHmZ0SbPdV4PkO5CXdpKQwj5WbdrKuanfYqYhIB7V2aukTwJvAKDMrN7PLicz+uc3MFgG/AGYEzc8HlprZQuBu4EKPqCLSZfR28POzIAZwFZFZR2XA+8CLnfHmpGs1no2sriKR+Gfxem364uJiLy0tDTuNhFfy61fJ65/G7/7tlLBTEZFWMLP57l7cNK4zkKVDSgrzeGt1FTtqasNORUQ6QMVAOqSkMJe6BufVlZrqKxLPVAykQ04aNoCBGb00biAS51QMpEOSk4zJo3OZu6KC2vqGsNMRkXZSMZAOKynMZUdNHaVrtoadioi0k4qBdNjpx+TQKzlJXUUicUzFQDosIy2FT44cxOzlm4jXqcoiiU7FQDrFlMI8Ptyym/crq8NORUTaQcVAOkVJYS4As3StIpG4pGIgnWJwZm9OGJLJE/9cy/Y9OgFNJN6oGEinuf6cMWzYtoern3yHet30RiSuqBhIpykePpDr/+U45q6s5I7Z74Wdjoi0gYqBdKpLTx7GhcVDueuVMv66dGPY6YhIK6kYSKcyM26Ydhxjh2bx/T8sYtWmnWGnJCKtoGIgnS49NZn/vnQ8vXulMOO38zWgLBIHVAykSxyRmc69l45jXdVuDSiLxAEVA+kyEzSgLBI3VAykS2lAWSQ+HLYYmNlDZlZhZkujYkVmNs/MFppZqZlNDOKXmNliM1tiZm+Y2diobWaa2btmttTMnjCz9CA+wszeMrMyM/u9mfXqijcq4dCAskh8aM2RwcPA2U1iNwM3uHsRcF3wGOADYJK7nwDcCNwPYGZDgO8Axe5+PJAMXBRscxNwu7uPBLYCl7f3zUhs0oCySOw7bDFw99eAqqZhoH+wnAlsCNq+4e6NF7WfB+RHbZMC9DazFKAPsMHMDJgMPB20eQQ4t+1vQ2KdBpRFYlt7xwyuBm4xs3XArcC1zbS5HHgRwN3XB+3WAhuB7e7+MjAI2ObudcE25cCQll7UzGYE3VKllZW652680YCySOxqbzG4Epjp7kOBmcCD0SvN7FNEisE1weMBwDRgBHAkkGFml7b1Rd39fncvdvfinJycdqYuYbr05GF8qThfA8oiMaa9xWA68Eyw/BQwsXGFmZ0IPABMc/ctQbgE+MDdK929Ntj2k8AWICvoOoJIt9L6duYkccDM+Nm04zWgLBJj2lsMNgCTguXJwCoAMxtG5IP+K+4e3Q+wFjjFzPoE4wRTgOUeuS3WXOCCoN104Pl25iRxQgPKIrGnNVNLnwDeBEaZWbmZXQ5cAdxmZouAXwAzgubXERkHuKdx2imAu79FZJB4AbAkeN37g22uAb5nZmXBtgd1OUnPpAFlkdhi8XrP2uLiYi8tLQ07Demg3877kJ88t5RvTx7J988aFXY60kncnUgngMQaM5vv7sVN4ynNNRbpLpeePIwl5du465UyjjuyP2cfPzjslKQDtu+p5daXVvLk22vJ6ZtGQU5fCnIyODr4XZDTl8H900lKUqGINSoGEqrGAeWVm6r5/h8WcXROX47J6xd2WtJG7s4LizZw45+WU7VrL+edlE+DO6srq3l2wXp27q3b37Z3ajIjsjP2F4ejczIoyI4Ui4w0fSSFRd1EEhM+2l7D5+96nX7pKTz3zVPJ7J0adkrSSh9s3sV1zy/l76s2c2J+Jr847wSOH5K5f727U7lzL+9X7mL15mpWV+7i/crI7/Ktu4keLjqif3pQJBqPJvpSkJ3BkKzeOproJC11E6kYSMx4e00VF98/j9OPyeaB6RNI1h9/TNtbV899f1vN3X8rIy05iR+ePYpLTj6qTf9uNbX1rK3azfsV1azefKBIvF9Zzc6aA0cTaSlJjMiO7m46cDTRL11fHNpCxUDiggaU48M/yjbzk+eWsnrzLs4ZeyQ/+Vwhuf3TO+353Z3N1ftYXRkpEqsrqyNHFpXVrK06+Ggit1/a/i6nguwMjs7ty9HZfRkyoLe+UDRDA8gSFzSgHNsqd+7l539exnMLN3DUoD48ctlEJh3b+VcDMDNy+qWR0y+NkwsGHbRuX10Da6t2UVZxoNtpdWU1f1688aBzVnqlJDF8UB8KsvtydO6BI4mCnL7qhmyGioHEFA0ox6aGBueJt9dy04sr2FNbz3cmj+SqT40kPTW523PplZLEyNx+jMw9+P+Fu1O1a9/+I4nG7qb3Nu1k1vJNB53Lkt2310GD1wMyIlfON6BxRqwZGEb0DFkzw2h+/YFtLWo9+6fYNq4/0D7SwJo8d/R6i1pP1OuNzc+id6/O3ffqJpKYpAHl2LFsww5+/NwS3lm7jVMKBvKf557AyNy+YafVJrX1DQeNTTQWi9Wbd1G1a1/Y6bXZ7O9Nave/gbqJJK40nqF88f3zuPrJdzSgHIJde+u4Y/Z7PPSPNWT1TuXXXxrLeScNicuTyVKTkzg6py9H53z8A3Trrn1U763DHRyn8fuxEznaaPy6HIk3Xd+4HIk3fUzUtu157sYv6x71HABHZnXe+EwjFQOJWROGD+T6c8bwk+ff5Y7Z72lAuRu99O5H/PSFd9m4vYaLJw7lmrNHk9WnZ96EcEBGr/3dRIlMxUBi2qWnHMWS9ds1oNxNyrfu5qcvLGP28k2MPqIf//Xlkxh/1MCw05JuoGIgMU0Dyt2jtr6Bh17/gDtmrwLgR58dzddOHUFqcnsvbCzxRv/SEvN0yeuuNf/DKs6563V++eIKTh2ZzezvT2LGGUerECQY/WtLXGh6yesGXfK6w7bt3se1zyzm/HvfZMeeWu7/yngemF7MkKzeYacmIVAxkLjROKA8d2Ult+seyu3m7vxxfjmTb3uVP5SWM+OMAmZ9bxJnHXdE2KlJiDRmIHFFA8odU1ZRzf97bgnzVldx0rAsfn7uCYw5sn/YaUkMUDGQuKIB5fapqa3n7rll3Pfq+/ROTeYX553ARROG6kqgsp+6iSTuaEC5bV59r5Kzbn+Nu14p4/MnHsmc75/Jl08epkIgB2nNPZAfMrMKM1saFSsys3mN9zk2s4lB/BIzW2xmS8zsDTMbG7VNlpk9bWYrzGy5mX0iiA80s1lmtir4PaAr3qj0LEdkpnPPJRpQPpSKHTV86/EFTH/on6QkGY//28ncfmEROf3Swk5NYlBrjgweBs5uErsZuMHdi4DrgscAHwCT3P0E4EYO3PQe4E7gr+4+GhgLLA/i/wHMcfdjgDnBY5HDmjhCA8rNqW9wHnljDVNue5WXl23ie1OP5cWrT+eTI7PDTk1i2GHHDNz9NTMb3jQMNI46ZQIbgrZvRLWZB+QDmFkmcAbwr0G7fUDj1aGmAWcGy48AfwOuacubkMSlAeWDLV2/nR89u4TF5ds5/Zhsbpx2PMOzM8JOS+JAeweQrwZeMrNbiRxdfLKZNpcDLwbLI4BK4H+DrqP5wHfdfReQ5+4bg3YfAXktvaiZzQBmAAwbNqydqUtPogHliJ01tdz28ns8+uYaBmak8ZuLT+KcEwfH5UXlJBztHUC+Epjp7kOBmcCD0SvN7FNEikHjN/wUYBxwr7ufBOyime4gj1yir8XOX3e/392L3b04J6fzb6gh8SmRB5Tdnb8s2UjJr1/lkTfXcMnJRzHn+5P4l7FHqhBIm7S3GEwHngmWnwImNq4wsxOBB4Bp7r4lCJcD5e7+VvD4aSLFAWCTmQ0Oth0MVLQzJ0lgiTigvHbLbr728Ntc9bsFDMpI49mrTuXGc4/XvR+kXdpbDDYAk4LlycAqADMbRqRIfMXd94/ouftHwDoza7wG8RRgWbD8ApHiQvD7+XbmJAkuUQaU99U1cPfcMqbe/ipvf1DFTz4/hhe+dSpFQ7PCTk3i2GHHDMzsCSIDvNlmVg5cD1wB3GlmKUANQT8+kZlFg4B7gkPUuqg76nwb+J2Z9QJWA18L4r8C/mBmlwMfAl/qhPclCaqnDyi/tXoLP35uKWUV1Xzm+CO47pwxDM7UtYSk43TbS+lxamrrufD+eZRt2slz3zy1RwwoV+yo4eaXVvL0/HLyB/TmZ9OOY/LoFudaiLSopdteqhhIj7Rx+x7Ouet1+qWnxt09lPfW1fPuhh0sXLuNReXbWLhuGx9u2U1KknHFGQV8Z/IxnX4zdEkcugeyJJTBmb2555LxfPl/IvdQfnD6hJi8/IK7s2bLbhau28rCtZEP/mUbd1BbH/mSltc/jaKhWVw0YRhTx+TF3Y3oJX6oGEiP1Tig/JPn3+X2GLmHctWufSxat4131kU++Bet27Z/KmyfXsmcMCSTy04bwUlDsygaOoAjMjv/xucizVExkB4tzAHlvXX1LNuwg4XBB39jdw9AksGxef34zPFHMHZoFkVDszgmty8puruYhETFQHq07jpDuS3dPUVDszghP5O+afrzk9ihAWRJCJ09oLx11z4WBt09i9ZFBnq37T64u6doWJa6eyTmaABZElpHBpRb091z9nEHunuOzetHcgwOVoscioqBJIzWDCh/rLunfDvLN+xgX30DoO4e6bn0v1gSStMB5ZNHDGJh+bb9/fzNdfd87bTh6u6RHk/FQBJK9IDyNx9/h/rggnbR3T1FQ7MYq+4eSTAqBpJwGi95fcfs9zhqUAZFQ7M4MT+TDHX3SALT/35JSEdkpvOr808MOw2RmKEzXERERMVARERUDEREBBUDERFBxUBERFAxEBERVAxERAQVAxERIY4vYW1mlcCH7dw8G9jciel0tXjKV7l2nXjKN55yhfjKt6O5HuXuOU2DcVsMOsLMSpu7nnesiqd8lWvXiad84ylXiK98uypXdROJiIiKgYiIJG4xuD/sBNoonvJVrl0nnvKNp1whvvLtklwTcsxAREQOlqhHBiIiEiUhioGZrTGzJWa20MxKg9hAM5tlZquC3wNCyu0hM6sws6VRsWZzs4jfmFmZmS02s3Exku9PzWx9sH8Xmtlno9ZdG+S70sw+3c25DjWzuWa2zMzeNbPvBvGY27+HyDVW9226mf3TzBYF+d4QxEeY2VtBXr83s15BPC14XBasHx4DuT5sZh9E7duiIB4Lf2fJZvaOmf0peNz1+9Xde/wPsAbIbhK7GfiPYPk/gJtCyu0MYByw9HC5AZ8FXgQMOAV4K0by/Snwg2bajgEWAWnACOB9ILkbcx0MjAuW+wHvBTnF3P49RK6xum8N6BsspwJvBfvsD8BFQfw+4Mpg+SrgvmD5IuD3MZDrw8AFzbSPhb+z7wGPA38KHnf5fk2II4MWTAMeCZYfAc4NIwl3fw2oahJuKbdpwKMeMQ/IMrPB3ZJooIV8WzINeNLd97r7B0AZMLHLkmvC3Te6+4JgeSewHBhCDO7fQ+TakrD3rbt7dfAwNfhxYDLwdBBvum8b9/nTwBQz65YbTB8i15aE+ndmZvnA54AHgsdGN+zXRCkGDrxsZvPNbEYQy3P3jcHyR0BeOKk1q6XchgDrotqVc+gPjO70reCQ+qGoLreYyTc4fD6JyLfCmN6/TXKFGN23QVfGQqACmEXk6GSbu9c1k9P+fIP124FBYeXq7o379ufBvr3dzNKa5hro7n17B/DvQEPweBDdsF8TpRic5u7jgM8A3zSzM6JXeuQYKyanVcVyblHuBY4GioCNwG2hZtOEmfUF/ghc7e47otfF2v5tJteY3bfuXu/uRUA+kaOS0eFm1LKmuZrZ8cC1RHKeAAwErgkvwwgz+zxQ4e7zu/u1E6IYuPv64HcF8CyR/7ibGg/9gt8V4WX4MS3lth4YGtUuP4iFyt03BX9sDcD/cKC7IvR8zSyVyIfr79z9mSAck/u3uVxjed82cvdtwFzgE0S6VFKayWl/vsH6TGBL92Z6UK5nB11z7u57gf8lNvbtqcC/mNka4Eki3UN30g37tccXAzPLMLN+jcvAWcBS4AVgetBsOvB8OBk2q6XcXgC+Gsx2OAXYHtXdEZom/annEdm/EMn3omDGwwjgGOCf3ZiXAQ8Cy93911GrYm7/tpRrDO/bHDPLCpZ7A1OJjHPMBS4ImjXdt437/ALgleCoLKxcV0R9ITAiffDR+zaU/wfufq2757v7cCIDwq+4+yV0x37t7FHwWPsBCojMulgEvAv8OIgPAuYAq4DZwMCQ8nuCyOF/LZG+wMtbyo3I7Ia7ifTNLgGKYyTf3wb5LA7+cw6Oav/jIN+VwGe6OdfTiHQBLQYWBj+fjcX9e4hcY3Xfngi8E+S1FLguiBcQKUplwFNAWhBPDx6XBesLYiDXV4J9uxR4jAMzjkL/OwvyOJMDs4m6fL/qDGQREen53UQiInJ4KgYiIqJiICIiKgYiIoKKgYiIoGIgIiKoGIiICCoGIiIC/H+6fLqQmAUZSQAAAABJRU5ErkJggg==\n"},"metadata":{"needs_background":"light"}}]},{"cell_type":"markdown","source":"# Step 3: Find the best parameter value\n\nGiven the results, which value for `n_estimators` seems best for the random forest model?  Use your answer to set the value of `n_estimators_best`.","metadata":{}},{"cell_type":"code","source":"n_estimators_best = min(results, key=results.get)\n# Check your answer\nstep_3.check()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:54:37.781975Z","iopub.execute_input":"2021-08-17T08:54:37.782367Z","iopub.status.idle":"2021-08-17T08:54:37.790333Z","shell.execute_reply.started":"2021-08-17T08:54:37.782332Z","shell.execute_reply":"2021-08-17T08:54:37.789212Z"},"trusted":true},"execution_count":16,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"3_BestEst\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Correct","text/markdown":"<span style=\"color:#33cc33\">Correct</span>"},"metadata":{}}]},{"cell_type":"code","source":"# Lines below will give you a hint or solution code\n#step_3.hint()\n#step_3.solution()","metadata":{"execution":{"iopub.status.busy":"2021-08-17T08:54:29.310832Z","iopub.execute_input":"2021-08-17T08:54:29.311263Z","iopub.status.idle":"2021-08-17T08:54:29.320594Z","shell.execute_reply.started":"2021-08-17T08:54:29.311227Z","shell.execute_reply":"2021-08-17T08:54:29.319075Z"},"trusted":true},"execution_count":15,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Javascript object>","application/javascript":"parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"interactionType\": 3, \"questionType\": 2, \"questionId\": \"3_BestEst\", \"learnToolsVersion\": \"0.3.4\", \"valueTowardsCompletion\": 0.0, \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\", \"outcomeType\": 4}}, \"*\")"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Solution: \n```python\nn_estimators_best = min(results, key=results.get)\n```","text/markdown":"<span style=\"color:#33cc99\">Solution:</span> \n```python\nn_estimators_best = min(results, key=results.get)\n```"},"metadata":{}}]},{"cell_type":"markdown","source":"In this exercise, you have explored one method for choosing appropriate parameters in a machine learning model.  \n\nIf you'd like to learn more about [hyperparameter optimization](https://en.wikipedia.org/wiki/Hyperparameter_optimization), you're encouraged to start with **grid search**, which is a straightforward method for determining the best _combination_ of parameters for a machine learning model.  Thankfully, scikit-learn also contains a built-in function [`GridSearchCV()`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html) that can make your grid search code very efficient!\n\n# Keep going\n\nContinue to learn about **[gradient boosting](https://www.kaggle.com/alexisbcook/xgboost)**, a powerful technique that achieves state-of-the-art results on a variety of datasets.","metadata":{}},{"cell_type":"markdown","source":"---\n\n\n\n\n*Have questions or comments? Visit the [Learn Discussion forum](https://www.kaggle.com/learn-forum/161289) to chat with other Learners.*","metadata":{}}]}