-
Notifications
You must be signed in to change notification settings - Fork 1
jibalio/computational-fact-checking
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
 |  | |||
Repository files navigation
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Computational Fact Checking Using Knowledge Graphs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Necessary Import Statements**\n",
"Run this first."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[nltk_data] Downloading package wordnet to\n",
"[nltk_data] C:\\Users\\Leryc\\AppData\\Roaming\\nltk_data...\n",
"[nltk_data] Package wordnet is already up-to-date!\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"C:\\Users\\Leryc\\Envs\\thesis\\lib\\site-packages\\sklearn\\ensemble\\weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release.\n",
" from numpy.core.umath_tests import inner1d\n"
]
}
],
"source": [
"%matplotlib inline\n",
"\n",
"import sys\n",
"from enum import Enum\n",
"from collections import OrderedDict\n",
"\n",
"import csv, math, string\n",
"import numpy as np, pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"\n",
"from scrapers import logger\n",
"from Truth import Page, get_truth_value, get_truth_value_old, get_utfidf, get_truth_matrices, get_mx\n",
"\n",
"\n",
"from sklearn.metrics import roc_curve, confusion_matrix, classification_report\n",
"\n",
"\n",
"from sklearn.svm import LinearSVC\n",
"from sklearn.ensemble import RandomForestClassifier\n",
"from sklearn.model_selection import cross_val_predict, cross_val_score\n",
"\n",
"logger.ignorewarnings = True\n",
"color = plt.cm.Blues"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Phase I: Classification Task (Calibration)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Objective**\n",
"\n",
"This phase is aimed to answer to objectives 1 & 3.\n",
"1. Compare the accuracy of using metric and ultra-metric closure for path evaluation.\n",
"2. Evaluate the accuracy of the path evaluation with cosine similarity and TF-IDF. \n",
"\n",
"**Method**\n",
"\n",
"Statements in the form of \"Politician $p_{i}$ endorses ideology $i_{j}$ are considered. A matrix of all possible combinations of subjects and objects were then obtained. Each path in the matrix was evaluated to obtain its truth value.\n",
"\n",
"Truth matrices were obtained using these 3 algorithms:\n",
"1. Truth value matrix using metric closure, with TF-IDF and cosine similarity\n",
"2. Truth value matrix using ultrametric closure, with TF-IDF and cosine similarity\n",
"3. Truth value matrix using metric closure, using generality only (theoretical framework)\n",
"\n",
"Using the truth values of the politicians mapped to the ideologies as features, the politicians will be classified into either republican (R) or democrat (D). The algorithm which results in the highest accuracy will be used for the succeeding phases of this study."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Get truth matrices**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment this if making a truth matrix from scratch (takes a VERY long time)\n",
"# if truth matrices were already calculated, load from the CSV instead (bottom most part of this ntbk)\n",
"\n",
"#df = pd.read_csv('data/out/i5/i5.csv', sep=';', index_col=0)\n",
"#mt, ut, mold = get_truth_matrices(df)\n",
"#i1_mt = mt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Get target column**"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"\n",
"# Democrat will be the positive class.\n",
"def isDemocrat(entity):\n",
" return entity.endswith('(D)')\n",
"y = [isDemocrat(x) for x in i1_mt.index]"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Anarchism</th>\n",
" <th>Antisemitism</th>\n",
" <th>Capitalism</th>\n",
" <th>Christianity</th>\n",
" <th>Communism</th>\n",
" <th>Conservatism</th>\n",
" <th>Democracy</th>\n",
" <th>Fascism</th>\n",
" <th>Feminism</th>\n",
" <th>Islamophobia</th>\n",
" <th>Left-wing politics</th>\n",
" <th>Liberalism</th>\n",
" <th>Marxism</th>\n",
" <th>Nationalism</th>\n",
" <th>Neo-Nazism</th>\n",
" <th>Protestantism</th>\n",
" <th>Right-wing politics</th>\n",
" <th>Secularism</th>\n",
" <th>Socialism</th>\n",
" <th>White supremacy</th>\n",
" </tr>\n",
" <tr>\n",
" <th>x-entity</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>Jo Bonner (R)</th>\n",
" <td>0.071048</td>\n",
" <td>0.067143</td>\n",
" <td>0.054021</td>\n",
" <td>0.053941</td>\n",
" <td>0.066685</td>\n",
" <td>0.072421</td>\n",
" <td>0.058368</td>\n",
" <td>0.065525</td>\n",
" <td>0.044251</td>\n",
" <td>0.069056</td>\n",
" <td>0.063660</td>\n",
" <td>0.053091</td>\n",
" <td>0.059485</td>\n",
" <td>0.065946</td>\n",
" <td>0.096082</td>\n",
" <td>0.054214</td>\n",
" <td>0.068126</td>\n",
" <td>0.050647</td>\n",
" <td>0.073818</td>\n",
" <td>0.063966</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Martha Roby (R)</th>\n",
" <td>0.067763</td>\n",
" <td>0.063282</td>\n",
" <td>0.064275</td>\n",
" <td>0.055059</td>\n",
" <td>0.067058</td>\n",
" <td>0.098936</td>\n",
" <td>0.083081</td>\n",
" <td>0.077397</td>\n",
" <td>0.062446</td>\n",
" <td>0.057294</td>\n",
" <td>0.074704</td>\n",
" <td>0.075141</td>\n",
" <td>0.054051</td>\n",
" <td>0.060333</td>\n",
" <td>0.089473</td>\n",
" <td>0.058659</td>\n",
" <td>0.089053</td>\n",
" <td>0.064666</td>\n",
" <td>0.087194</td>\n",
" <td>0.065397</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Mike Rogers (R)</th>\n",
" <td>0.069972</td>\n",
" <td>0.066444</td>\n",
" <td>0.064401</td>\n",
" <td>0.050256</td>\n",
" <td>0.058977</td>\n",
" <td>0.055445</td>\n",
" <td>0.054687</td>\n",
" <td>0.070131</td>\n",
" <td>0.044306</td>\n",
" <td>0.056108</td>\n",
" <td>0.069981</td>\n",
" <td>0.060342</td>\n",
" <td>0.046575</td>\n",
" <td>0.076550</td>\n",
" <td>0.089277</td>\n",
" <td>0.061955</td>\n",
" <td>0.056137</td>\n",
" <td>0.051539</td>\n",
" <td>0.070631</td>\n",
" <td>0.057534</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Robert Aderholt (R)</th>\n",
" <td>0.099980</td>\n",
" <td>0.078978</td>\n",
" <td>0.073372</td>\n",
" <td>0.070627</td>\n",
" <td>0.093934</td>\n",
" <td>0.122361</td>\n",
" <td>0.084522</td>\n",
" <td>0.092721</td>\n",
" <td>0.059072</td>\n",
" <td>0.064086</td>\n",
" <td>0.092826</td>\n",
" <td>0.090079</td>\n",
" <td>0.060709</td>\n",
" <td>0.079224</td>\n",
" <td>0.133379</td>\n",
" <td>0.076984</td>\n",
" <td>0.105611</td>\n",
" <td>0.072566</td>\n",
" <td>0.100810</td>\n",
" <td>0.083287</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Mo Brooks (R)</th>\n",
" <td>0.086297</td>\n",
" <td>0.081946</td>\n",
" <td>0.081745</td>\n",
" <td>0.072008</td>\n",
" <td>0.090412</td>\n",
" <td>0.113233</td>\n",
" <td>0.091384</td>\n",
" <td>0.096122</td>\n",
" <td>0.056169</td>\n",
" <td>0.061632</td>\n",
" <td>0.089806</td>\n",
" <td>0.085677</td>\n",
" <td>0.059916</td>\n",
" <td>0.089592</td>\n",
" <td>0.118347</td>\n",
" <td>0.073699</td>\n",
" <td>0.095786</td>\n",
" <td>0.063528</td>\n",
" <td>0.104593</td>\n",
" <td>0.094316</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Anarchism Antisemitism Capitalism Christianity \\\n",
"x-entity \n",
"Jo Bonner (R) 0.071048 0.067143 0.054021 0.053941 \n",
"Martha Roby (R) 0.067763 0.063282 0.064275 0.055059 \n",
"Mike Rogers (R) 0.069972 0.066444 0.064401 0.050256 \n",
"Robert Aderholt (R) 0.099980 0.078978 0.073372 0.070627 \n",
"Mo Brooks (R) 0.086297 0.081946 0.081745 0.072008 \n",
"\n",
" Communism Conservatism Democracy Fascism Feminism \\\n",
"x-entity \n",
"Jo Bonner (R) 0.066685 0.072421 0.058368 0.065525 0.044251 \n",
"Martha Roby (R) 0.067058 0.098936 0.083081 0.077397 0.062446 \n",
"Mike Rogers (R) 0.058977 0.055445 0.054687 0.070131 0.044306 \n",
"Robert Aderholt (R) 0.093934 0.122361 0.084522 0.092721 0.059072 \n",
"Mo Brooks (R) 0.090412 0.113233 0.091384 0.096122 0.056169 \n",
"\n",
" Islamophobia Left-wing politics Liberalism Marxism \\\n",
"x-entity \n",
"Jo Bonner (R) 0.069056 0.063660 0.053091 0.059485 \n",
"Martha Roby (R) 0.057294 0.074704 0.075141 0.054051 \n",
"Mike Rogers (R) 0.056108 0.069981 0.060342 0.046575 \n",
"Robert Aderholt (R) 0.064086 0.092826 0.090079 0.060709 \n",
"Mo Brooks (R) 0.061632 0.089806 0.085677 0.059916 \n",
"\n",
" Nationalism Neo-Nazism Protestantism \\\n",
"x-entity \n",
"Jo Bonner (R) 0.065946 0.096082 0.054214 \n",
"Martha Roby (R) 0.060333 0.089473 0.058659 \n",
"Mike Rogers (R) 0.076550 0.089277 0.061955 \n",
"Robert Aderholt (R) 0.079224 0.133379 0.076984 \n",
"Mo Brooks (R) 0.089592 0.118347 0.073699 \n",
"\n",
" Right-wing politics Secularism Socialism \\\n",
"x-entity \n",
"Jo Bonner (R) 0.068126 0.050647 0.073818 \n",
"Martha Roby (R) 0.089053 0.064666 0.087194 \n",
"Mike Rogers (R) 0.056137 0.051539 0.070631 \n",
"Robert Aderholt (R) 0.105611 0.072566 0.100810 \n",
"Mo Brooks (R) 0.095786 0.063528 0.104593 \n",
"\n",
" White supremacy \n",
"x-entity \n",
"Jo Bonner (R) 0.063966 \n",
"Martha Roby (R) 0.065397 \n",
"Mike Rogers (R) 0.057534 \n",
"Robert Aderholt (R) 0.083287 \n",
"Mo Brooks (R) 0.094316 "
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"i1_mt[:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Classification using Metric Closure WITH TF-IDF/COSINE SIMILARITY MATRIX"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Classify the truth values into R or D using SVM with linear kernel, and assess accuracy using 10-fold cross validation**"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Mean cross-validation accuracy: 0.66\n",
" precision recall f1-score support\n",
"\n",
" False 0.66 0.67 0.67 137\n",
" True 0.67 0.66 0.66 138\n",
"\n",
"avg / total 0.67 0.67 0.67 275\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAECCAYAAAAxcwG5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAABi9JREFUeJzt3LGrX/UdxvHn29wokRapJFkUBJcWFJpCSpcuRboUugvFNYvi5C5dCy0ODuImhf4BjrVQl9ZFMCIdumjrmETRpaFg/HSIpUUsufdnzj3G5/XaQk4OD/fkzTkn93ezZiZAn2/tPQDYh/ihlPihlPihlPihlPih1NHeA+4la61bSd5NcjbJp0leTfLizHy26zCO5X+u31GS95M8PTMf77tqP+78J3NzZi7NzONJfpbk50le2HkTx/ef6/dEko+SPLP3oD2J/0Azcy3JlSTPrrXW3ns4sTeTPLz3iD2J/yuYmfdy+2t4ce8tHN9a60ySJ5O8tveWPYn/q3PXv3ecW2tdTfJhkoeSvL7znl2J/ytYaz2W5FaSa3tv4VhuzsylJI8muS/e+TnEWutCkpeTvDR+OuqeMjOfJHkuyfNrrbN779mL+E/m3Frr6lrrr0n+mOQPSX618yYOMDNvJ3knyVN7b9nLctOCTu78UEr8UEr8UEr8UEr8UEr8B1prXdl7A4dx7W4T/+H8Bbp3uXYRP9Ta7EM+6+jcrPsf3OTcXwfz6T+zjh7Ye8ZmfvC9R/aesJkbN67n/PkLe8/YzAcf/D0f3rhxxx842+x/8ln3P5j7H//lVqdnY39649d7T+BAP/3Jj491nMd+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KHV0pwPWWreSvJvkbJJPk7ya5MWZ+WzjbcCG7hh/kpszcylJ1loXk/w+yYNJXthyGLCtEz32z8y1JFeSPLvWWttMAk7Did/5Z+a9z//cxbs/Bzgtx3ns/zJfetdfa13J7SeD5L7vHHhq4DSc+M6/1nosya0k1774ezPzysxcnpnL6+iBu7EP2MiJ4l9rXUjycpKXZma2mQSchuM89p9ba13Nf7/V97skv910FbC5O8Y/M2dOYwhwunzCD0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0qJH0odbXXiH37/kfz5L7/Z6vRs7Ls/enbvCRzoX3/74FjHufNDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDqaPjHLTWupXk3c+Pfz/J0zPz8ZbDgG0d985/c2YuzcwTST5K8syGm4BTcMhj/5tJHr7bQ4DTdaL411pnkjyZ5LVt5gCn5bjxn1trXU3yYZKHkrz+ZQetta6std5aa711/cb1u7UR2MCJ3vmTPJrkvvyfd/6ZeWVmLs/M5QvnL9ytjcAGTvTYPzOfJHkuyfNrrbPbTAJOw4n/wW9m3k7yTpKn7v4c4LQc6/v8M/PtL/z6F9vMAU6LT/hBKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDKfFDqTUz25x4retJ/rHJyb8ezie5sfcIDvJNv3aPzsyFOx20WfzfdGutt2bm8t47ODnX7jaP/VBK/FBK/Id7Ze8BHMy1i3d+qOXOD6XED6XED6XED6XED6X+Dfrq6RYMBOSnAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 288x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from sklearn.svm import SVC\n",
"from sklearn.model_selection import KFold, StratifiedKFold\n",
"import matplotlib.pyplot as ps\n",
"from sklearn.model_selection import cross_val_score \n",
"\n",
"#from sklearn.neighbors import KNeighborsClassifier\n",
"kf = KFold(n_splits=10)\n",
"#knn = KNeighborsClassifier(n_neighbors=3)\n",
"svm_clf = SVC(C=2**15, kernel='linear')\n",
"\n",
"\n",
"scoresw = cross_val_score(svm_clf,i1_mt,y,cv=kf)\n",
"print(\"Mean cross-validation accuracy: {:.2f}\".format(np.mean(scoresw)))\n",
"\n",
"ypred = cross_val_predict(svm_clf, i1_mt, y, cv=kf)\n",
"print(classification_report(y, ypred, digits=2))\n",
"conf_mx = confusion_matrix(y, ypred, labels=[True, False])\n",
"plt.matshow(conf_mx,cmap=plt.cm.Blues)\n",
"\n",
"plt.xticks(np.arange(2), ['D', 'R'])\n",
"plt.yticks(np.arange(2), ['D', 'R'])\n",
"plt.savefig('figures/classification/metric_tfidf_confmx.pdf')\n",
"plt.show()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Display ROC Curve**"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd4VGX2wPHvMQgsK7IK6AqhJwGSEBADSBP5RSnSVQRlETDAYgEFBVFcBBeVKgrSQZoIrAiKaxRdsCJ1aUuVGFpApUgVCSQ5vz9mMg4hZQKZTDJzPs+Tx8ydm3vPjSEnbzuvqCrGGGMMwHW+DsAYY0z+YUnBGGOMiyUFY4wxLpYUjDHGuFhSMMYY42JJwRhjjIslBWOMMS6WFIwxxrhYUjDGGONSyNcB5FSpUqW0YsWKvg7DGGMKlP/+97/HVbV0ducVuKRQsWJFNm7c6OswjDGmQBGRA56cZ91HxhhjXCwpGGOMcbGkYIwxxqXAjSlk5NKlSyQmJnLhwgVfh2JMjhQtWpTg4GCuv/56X4diDOAnSSExMZHixYtTsWJFRMTX4RjjEVXlxIkTJCYmUqlSJV+HYwzgxe4jEXlHRI6KyPZM3hcRmSAi8SKyTURqX+29Lly4QMmSJS0hmAJFRChZsqS1cE2+4s0xhTlAiyzebwmEOj96A1Ou5WaWEExBZD+3Jr/xWveRqn4jIhWzOKUdME8d+4GuFZG/iMhtqvqTt2Iyxpi88t66g3y05XCuXCs56XeSzp6iTlRVXm4TkSvXzIwvZx+VBQ65vU50HruCiPQWkY0isvHYsWN5EtzVWLZsGSLC7t27Xce++uorWrdufdl53bt3Z8mSJYBjkHzw4MGEhoYSGRlJ3bp1+fTTT685ltdff52QkBCqVq3KihUrMjyncePG1KpVi1q1alGmTBnat28PwIIFC4iKiiIqKooGDRqwdetW19d89tlnVK1alZCQEEaOHOk6HhsbS82aNYmKiuLBBx/k3LlzAMyZM4fSpUu77jNz5kwADhw4wB133EGtWrWIiIhg6tSprmu1aNGCmjVrEhERQZ8+fUhJSQFg4MCBVKtWjaioKDp06MCpU6dcX7Nt2zbq169PREQENWrUcHXJLF68mKioKCIiIhg0aJDr/IMHD9K0aVNuv/12oqKiiIuLy/ZaQ4YMoVy5ctxwww0Zfj+XLFmCiLgWV37xxRfccccd1KhRgzvuuINVq1Zl+f/M+JePthxm509nrvk6v+zeyIp/dmX1tBdITU3Nhciyoape+wAqAtszee8ToJHb65XAHdld84477tD0du7cecUxX+jYsaM2atRIX375ZdexL7/8Ulu1anXZed26ddP3339fVVWff/55ffTRR/XChQuqqvrzzz/r4sWLrymOHTt2aFRUlF64cEETEhK0cuXKmpycnOXX3H///Tp37lxVVV29erX++uuvqqoaFxendevWVVXV5ORkrVy5sv7444+alJSkUVFRumPHDlVVPX36tOta/fv319dff11VVWfPnq1PPvnkFfdLSkpyPfPZs2e1QoUKevjw4cuulZqaqvfff78uXLhQVVVXrFihly5dUlXVQYMG6aBBg1RV9dKlS1qjRg3dsmWLqqoeP35ck5OT9fjx41quXDk9evSoqqo++uij+p///EdVVXv16qWTJ092fb8qVKiQ5bVUVdesWaNHjhzRP//5z1c8z5kzZ7Rx48Zar1493bBhg6qqbtq0yfVM//vf/7RMmTIZfu/zy8+vyd6CtQf0oanfe/QR+fJn+tDU76/6XidPntSePXsqoCEhIfrVV19dU+zARvXg97YvZx8lAuXcXgcDR3wUyzU7d+4cq1ev5ssvv6Rt27YMGzYs2685f/48M2bMYN++fRQpUgSAW2+9lYceeuiaYvnoo4/o3LkzRYoUoVKlSoSEhLB+/Xrq16+f4flnz55l1apVzJ49G4AGDRq43rvzzjtJTEwEYP369YSEhFC5cmUAOnfuzEcffUR4eDg33ngj4Pgj4/fff8+2r7xw4cKuz5OSki77CyjtWsnJyVy8eNF1rWbNml0WV1pr6/PPPycqKoqaNWsCULJkSQASEhIICwujdGlHuZd77rmHDz74gJiYGESEM2ccf8WdPn2aMmXKZHmttHtm5h//+AeDBg1i7NixrmO333676/OIiAguXLhAUlKS6/+1yZ+y6vZZt+9XAOpVujnb64TfdiPtamXY+ZGtlJQUGjRowJ49exg0aBDDhg3jT3/601VdK6d8mRSWA0+JyCKgHnBac2E8YfjHO9h55NqbbO7Cy9yYbT/ehx9+SIsWLQgLC+Pmm29m06ZN1K6d9YSq+Ph4ypcv7/olmJX+/fvz5ZdfXnG8c+fODB48+LJjhw8fvuwXWHBwMIcPZ963uWzZMmJiYjKMY9asWbRs2dJ13XLl/sjjwcHBrFu3zvW6R48exMXFER4ezrhx41zHP/jgA7755hvCwsIYP3686xqHDh2iVatWxMfHM2bMGNcvZoDmzZuzfv16WrZsyYMPPnhFXO+88w6dOnUC4IcffkBEaN68OceOHaNz584MGjSIkJAQdu/ezf79+wkODubDDz/k4sWLAAwbNoxmzZoxceJEfvvtN/7zn/9kea2sbN68mUOHDtG6devLkoK7Dz74gNtvv90SQgGQ1u0TftuV/x7qVbqZdrXK8ki98l6594kTJ7j55psJCgri1VdfpVy5ckRHR3vlXpnxWlIQkYXA3UApEUkEXgauB1DVqUAccB8QD5wHengrlrywcOFCnnnmGcDxi3rhwoXUrl0707+YczrrZPz48R6f62gpen6/hQsX0rNnzyuOf/nll8yaNYvvvvvOo+vOnj2blJQU+vbty+LFi+nRowdt2rTh4YcfpkiRIkydOpVu3bq5+tbLlSvHtm3bOHLkCO3bt+fBBx/k1ltvBWDFihVcuHCBLl26sGrVKu69917XfV599VUKFSpEly5dAEeL4rvvvmPDhg0UK1aMmJgY7rjjDmJiYpgyZQqdOnXiuuuuo0GDBiQkJLieuXv37jz77LOsWbOGrl27sn379iyvlZHU1FT69+/PnDlzMv3+7tixg+eff57PP/8803NM/hJ+240s/nvGLWtvUFUWLFjA008/zciRI+nVqxcdOnTIs/u78+bso4ezeV+BJ3P7vt4emc/IiRMnWLVqFdu3b0dESElJQUQYPXo0JUuW5OTJk5ed/+uvv1KqVClCQkI4ePAgZ8+epXjx4lneIyctheDgYA4d+mMMPzEx8bK/wtPHvn79epYtW3bZ8W3bttGzZ08+/fRTVxeKJ9cNCgqiU6dOjBkzhh49elzW/dKrVy+ef/75K2IoU6YMERERfPvtt5e1CooWLUrbtm356KOPXElh7ty5/Pvf/2blypWuhBQcHEyTJk0oVaoUAPfddx+bNm0iJiaGNm3a0KZNGwCmT59OUFAQ4GgBffbZZwDUr1+fCxcucPz48SyvlZGzZ8+yfft27r77bgB+/vln2rZty/Lly4mOjiYxMZEOHTowb948qlSpkuE1jPdczQygzFoJ3nLo0CH69OlDXFwcd955Jw0bNsyze2fEah/lgiVLlvDoo49y4MAB9u/fz6FDh6hUqRLfffcdoaGhHDlyhF27dgGOWTdbt26lVq1aFCtWjNjYWPr16+fq1vjpp5949913r7jH+PHj2bJlyxUf6RMCQNu2bVm0aBFJSUns27ePvXv3Urdu3Qxjf//992ndujVFixZ1HTt48CD3338/8+fPJywszHW8Tp067N27l3379nHx4kUWLVpE27ZtUVXi4+MBx188H3/8MdWqVXM9T5rly5dTvXp1wJFQfv/9dwBOnjzJ6tWrqVq1KufOnXN9TXJyMnFxca5rffbZZ4waNYrly5dTrFgx13WbN2/Otm3bOH/+PMnJyXz99deEh4cDcPToUdc9Jk+e7GoRlS9fnpUrVwKwa9cuLly4QOnSpbO8VkZKlCjB8ePH2b9/P/v37+fOO+90JYRTp07RqlUrXn/9dZ//Qw8E7607SKdpay77eHHZ/1zjAJ66lrGAnFq4cCERERF89dVXvPnmm3z33XdZ/rzlCU9Go/PTR36cfdSkSRP99NNPLzv21ltvaZ8+fVRV9bvvvtN69eppzZo1NTo6Wj///HPXeUlJSTpw4ECtUqWKRkREaN26dfWzzz675phGjBihlStX1rCwMI2Li3Mdb9mypWtGTGaxx8bG6l/+8hetWbOm1qxZU92/55988omGhoZq5cqVdcSIEaqqmpKSog0aNNDIyEiNiIjQRx55xDWDaPDgwRoeHq5RUVF69913665du1RV9fPPP9caNWpoVFSU1qhRQ6dNm6aqjtlX0dHRWqNGDQ0PD9ennnrKNeOoSpUqGhwc7Irr73//uyuu+fPna3h4uEZEROjAgQNdxzt37qzVq1fX6tWru2YxqTpmHDVo0ECjoqK0Zs2aumLFimyvNXDgQC1btqyKiJYtW/ayWWbu38+02Uf//Oc/tVixYq54a9asqb/88ssVX+Prn19/4T7jx/1jwdoDvg4tU59++qnec889mpCQ4PV74eHsI9EM+onzs+joaE2/yc6uXbtcf4EaU9DYz++1SesiSuv2ycuxgJxKTk5m/PjxXLx4kSFDhgCOP8zzYmW7iPxXVbMdtfaLgnjGmPwpN1f1ZsZ9mmhedftcja1btxIbG8t///tfHnroIVcyyG+lTiwpGGO8JqvpnbnF29NEr1VSUhIjRoxg5MiR3Hzzzbz//vs88MAD+S4ZpPGbpJBXTTBjclNB6769Gvm9S8fb9u7dy6hRo3jkkUd44403LpuRlx/5xeyjokWLcuLEiYD4B2b8hzr3U3Cf+WX8w7lz51iwYAEAkZGR7N69m7lz5+b7hAB+0lIIDg4mMTGR/Fwsz5iMpO28ZvzHF198Qe/evTlw4AC1a9emevXqrtIwBYFfJIXrr7/edq4yxssKwkIwXzp58iTPPfcc77zzDmFhYXz99dcFclaZXyQFY4z3pCWDnBSDS5OXC8F8KSUlhYYNG/LDDz/wwgsvMHTo0ALbLWhJwRg/lVvTQdNP+cyvs3x84fjx464Cdq+99hrly5fPthBmfucXA83GmCvl1iYv9SrdzGsdarD47/UtITipKvPmzSMsLMy1cVT79u0LfEIAaykY43cK0grfgujAgQP8/e9/Z8WKFTRo0IC77rrL1yHlKmspGONn3BNCIPTn56V3332XyMhIvvvuOyZOnMi3337rKtjoL6ylYIwfshaCd5QuXZqGDRsybdo0KlSo4OtwvMKSgjHGZOLSpUuMGzeOS5cu8Y9//IPmzZvTrFkzv66eYEnBmAIsoxlGgbQ2wJs2b95MbGwsmzdvpnPnzvm2gF1uszEFYwqwjGYY2VjCtblw4QIvvvgiderU4ciRI3zwwQcsXLjQ75NBGmspGFPA2fhB7oqPj2fs2LE8+uijjBs3jptuusnXIeUpSwrGmIB37tw5li1bRteuXYmMjGTPnj0BWzrHuo+MKYDS9iPOjcVpgW7FihVERETQrVs3117qgZoQwJKCMQWSrUW4didOnKBbt260aNGCYsWK8e233xbIAna5zbqPjCmgbCzh6qUVsIuPj2fIkCG89NJLBbaAXW6zpGBMPuNJITubdnp1jh07RsmSJQkKCmLUqFFUqFCBWrVq+TqsfMW6j4zJZzwpZGfdRjmjqsyePZuwsDBmzJgBQLt27SwhZMBaCsbkQ9Y1lHv2799P7969+eKLL2jcuDFNmzb1dUj5mrUUjMknbEZR7ps/fz6RkZGsWbOGyZMn89VXXxEWFubrsPI1aykYk0/YjKLcd+utt3LXXXcxdepUype3vSA8YUnBGB9JP6Bs+x9cu0uXLjF69GhSUlIYOnQozZo1o1mzZr4Oq0CxpGDMVbrW7S7T73lsLYRrs2nTJh577DG2bt3KI4884ipgZ3LGq0lBRFoAbwFBwExVHZnu/fLAXOAvznMGq2qcN2MyJre4d/dcDdvzOHf8/vvvDB8+nLFjx1K6dGmWLVtG+/btfR1WgeW1pCAiQcAk4F4gEdggIstVdafbaS8B/1LVKSISDsQBFb0VkzG5zbp7fC8hIYE33niD7t27M2bMmIArYJfbvDn7qC4Qr6oJqnoRWAS0S3eOAml/ZpUAjngxHmOMnzhz5gxz5swBICIigr179zJz5kxLCLnAm0mhLHDI7XWi85i7YcDfRCQRRyuhrxfjMcb4gbi4OCIjI4mNjXUVsPPXrTF9wZtJIaMRHk33+mFgjqoGA/cB80XkiphEpLeIbBSRjceOHfNCqMaY/O748eN07dqVVq1aUbx4cVavXm0F7LzAm0khESjn9jqYK7uHYoF/AajqGqAoUCr9hVR1uqpGq2p06dKlvRSuMSa/Sitgt2jRIoYOHcqmTZu48847fR2WX/Lm7KMNQKiIVAIOA52BR9KdcxCIAeaISHUcScGaAiZfS5uKakXpvO+XX36hdOnSBAUFMXbsWCpUqEBUVJSvw/JrXmspqGoy8BSwAtiFY5bRDhF5RUTaOk97FuglIluBhUB3VU3fxWRMvmIrj71PVZk1axZVq1Zl+vTpALRp08YSQh7w6joF55qDuHTHhrp9vhNo6M0YjPEGm4rqPQkJCfTq1YtVq1bRpEkT7rnnHl+HFFCsIJ4xJt+YO3cuNWrUYMOGDUydOpVVq1YREhLi67ACipW5MMbkG2XKlOH//u//mDJlCsHBwb4OJyBZUjDG+MzFixcZOXIkqampDBs2jHvvvZd7773X12EFNEsKxrixrTDzzoYNG3jsscfYvn07Xbt2tQJ2+YQlBRPQ0ieB9JVLM2Kzjq7N+fPnGTp0KOPHj+e2225j+fLltGnTxtdhGSdLCiagpV9vYJVLvW/fvn1MnDiRXr16MWrUKEqUKOHrkIwbSwom4Li3Dmxjm7xx+vRpli5dSo8ePYiIiCA+Pp5y5cpl/4Umz9mUVBNw0loHYF1BeeGTTz4hIiKCnj17snv3bgBLCPmYtRRMQLLWgfcdO3aMZ555hvfee4/IyEiWLl1KtWrVfB2WyYYlBeMXcrI1ps0e8r6UlBQaNWrEvn37GD58OIMHD6Zw4cK+Dst4wKOkICKFgfKqGu/leIy5KjkpUGddRt7z888/c8sttxAUFMS4ceOoWLEikZGRvg7L5EC2SUFEWgFvAIWBSiJSC3hZVTt4OzhjcsK6hHwnNTWVGTNmMHDgQEaNGsXjjz9O69atfR2WuQqeDDS/AtQDTgGo6hbAipGYfOG9dQfpNG2Na+DY5L34+HhiYmLo06cPderUoXnz5r4OyVwDT7qPLqnqqXQrDa28tfGptDEE98Vm1iWU92bPns0TTzxB4cKFmTFjBrGxsbYquYDzJCnsEpGHgOucG+Y8Daz1bljG/CGjQeT0ycAWm/lG+fLlad68OZMmTaJsWUvK/kCy29NGRP4MDAWaOQ+tAIar6u9eji1D0dHRunHjRl/c2vhIWvdQ+kFkSwZ5Lykpiddff53U1FReeeUVX4djckBE/quq0dmd50lLobmqPg8873bx+4Gl1xCfMZlK3zKwVcf5w7p164iNjWXHjh1069bNCtj5KU8Gml/K4NiQ3A7EmDTuK47BppD62m+//caAAQOoX78+p0+f5t///jdz5syxhOCnMm0piEhzoAVQVkTecHvrRiDV24GZwGYtg/zjwIEDTJ48mT59+jBy5EhuvNEW/vmzrLqPjgLbgQvADrfjZ4HB3gzKBKa0biNbcex7p06dYsmSJfTs2ZPw8HDi4+NtJ7QAkWlSUNXNwGYRWaCqF/IwJhMAPJlRZHzjo48+4vHHH+fo0aM0atSIatWqWUIIIJ4MNJcVkVeBcKBo2kFVDfNaVMbvZdQisOmlvnX06FH69evH4sWLiYqKYvny5VbALgB5khTmACOAsUBLoAc2pmBygY0b5B8pKSk0bNiQgwcPMmLECAYNGsT111/v67CMD3iSFIqp6goRGauqPwIvici33g7MGON9R44c4a9//StBQUG89dZbVKxYkfDwcF+HZXzIkympSeKYe/ajiPQRkTbALV6OyxjjRampqUyZMoVq1aoxdepUAO677z5LCMajlkJ/4AagH/AqUAJ4zJtBGWO854cffqBXr15888033HPPPbRs2dLXIZl8JNukoKrrnJ+eBboCiIhNRTAeyWzzG5t26huzZs3iqaeeomjRorzzzjt0797dFqGZy2TZfSQidUSkvYiUcr6OEJF5WEE846H0q5PT2Cpl36hYsSItW7Zk586d9OjRwxKCuUJWK5pfBx4AtuIYXF6Go0LqKKBP3oRn/IHNMvKdpKQk/vnPfwIwYsQIYmJiiImJ8XFUJj/LqvuoHVBTVX8XkZuBI87Xe/ImNGPMtfj++++JjY1l9+7dPPbYY1bAzngkq6RwIa08tqr+KiK7LSEYT7iPI9jYQd47d+4cQ4YMYeLEiZQrV47PPvvMdkMzHstqTKGyiCx1fiwDKrq99qhstoi0EJE9IhIvIhnWSxKRh0Rkp4jsEJH3ruYhTP7iPo5gYwd57+DBg0ybNo0nn3yS7du3W0IwOZJVS+GBdK/fzsmFRSQImATcCyQCG0RkuarudDsnFHgBaKiqJ0XE1j/4CRtHyFsnT57k/fffp3fv3oSHh5OQkECZMmV8HZYpgLIqiLfyGq9dF4hX1QQAEVmEY5xip9s5vYBJqnrSec+j13hPYwLOsmXLeOKJJzh27BhNmjShatWqlhDMVfNkRfPVKgsccnud6DzmLgwIE5HVIrJWRFpkdCER6S0iG0Vk47Fjx7wUrjEFy88//0zHjh25//77+etf/8r69eupWrWqr8MyBZwnK5qvVkbTHNJvCF0ICAXuBoKBb0UkUlVPXfZFqtOB6eDYozn3QzWmYElJSaFx48YcOnSI1157jeeee84K2Jlc4XFSEJEiqpqUg2snAuXcXgfjmNaa/py1qnoJ2Ccie3AkiQ05uI8xASMxMZEyZcoQFBTEhAkTqFSpkpW3Nrkq2+4jEakrIv8D9jpf1xSRiR5cewMQKiKVRKQw0BlYnu6cD4GmzuuWwtGdlJCD+I0JCKmpqUycOJFq1aoxZcoUAFq2bGkJweQ6T8YUJgCtgRMAqroV5y/yrKhqMvAUsALYBfxLVXeIyCsi0tZ52grghIjsBL4EBqrqiZw/hjH+a/fu3dx1113069ePRo0a0bp1a1+HZPyYJ91H16nqgXQrIVM8ubiqxgFx6Y4NdftcgQHOD2NMOjNnzuSpp56iWLFizJ07l65du9qqZONVniSFQyJSF1Dn2oO+wA/eDcsYA1ClShXatGnD22+/za233urrcEwA8CQpPI6jC6k88AvwH+cxY0wuu3DhAq+88goAr732Gk2bNqVp02x7a43JNZ4khWRV7ez1SEyBl1bzyOodXZ3Vq1cTGxvLnj176NmzpxWwMz7hSVLY4JwquhhYqqpnvRyTKSDSb6Czbt+vANSrdLPVO8qBs2fP8uKLLzJp0iQqVKjAihUraNasma/DMgHKk53XqohIAxxTSoeLyBZgkaou8np0Jl9L3ypISwaP1Cvv48gKlsTERGbOnEnfvn159dVXueGGG3wdkglgHi1eU9Xvge9FZBjwJrAAsKRgrPDdVTpx4gT/+te/ePzxx6levToJCQncdtttvg7LGI8Wr90gIl1E5GNgPXAMaOD1yIzxQ6rKkiVLCA8Pp1+/fuzZ49iixBKCyS88Wby2HbgTGK2qIar6rKqu83Jcxvidn376iQceeICOHTtSrlw5Nm7caAXsTL7jSfdRZVVN9XokxvixtAJ2hw8fZvTo0fTv359ChbxZj9KYq5PpT6WIjFPVZ4EPROSKyqSqer9XIzP5SvqZRmBbbXri0KFDlC1blqCgICZNmkSlSpUICwvzdVjGZCqr7qPFzv++jWMHtfQfJoC4b7GZxrbazFxKSgoTJky4rIBd8+bNLSGYfC+rndfWOz+trqqXbcUpIk8B17ozmylgbKaRZ3bt2kVsbCxr1qyhZcuWtGnTxtchGeMxTwaaH8vgWGxuB2KMP5g+fTq1atXihx9+YP78+XzyySeUL2/rNkzBkdWYQiccC9YqichSt7eKA6cy/ipjAltoaCgdOnRgwoQJ3HLLLb4Ox5gcy2r6w3oceygEc/kYwllgszeDMqag+P333xk2bBgiwsiRI62AnSnwshpT2Afsw1EV1RiTzjfffEPPnj3Zu3cvffr0sQJ2xi9kOqYgIl87/3tSRH51+zgpIr/mXYjG5C9nzpzhiSeeoEmTJqSkpLBy5UqmTJliCcH4hay6j9LawKXyIhCTP1k57CsdOXKEOXPmMGDAAF555RX+/Oc/+zokY3JNpi0Ft1XM5YAgVU0B6gN/B+xfQYBwTwiBvCbh+PHjTJ48GYBq1aqxb98+xo0bZwnB+B1PpqR+iGMrzirAPKA68J5XozL5Str6hEAsia2qLF68mPDwcJ555hl++MGxE61tjWn8lSfFV1JV9ZKI3A+8qaoTRMRmH/mRjEpYpAnkbqMjR47w+OOPs3z5cqKjo1m5cqWtSDZ+z6PtOEWkI9AVaO88dr33QjJ5Lasxg0DtNkpJSeGuu+7i8OHDjB07lqefftoK2JmA4MlP+WPAEzhKZyeISCVgoXfDMnkh/SCylbCAAwcOEBwcTFBQEJMnT6Zy5cqEhIT4Oixj8ky2Ywqquh3oB2wUkWrAIVV91euRGa+zQeQ/pKSk8MYbb1C9enVXAbtmzZpZQjABJ9uWgog0BuYDhwEB/ioiXVV1tbeDM95nLQTYvn07sbGxrF+/ntatW9O+ffvsv8gYP+VJ99F44D5V3QkgItVxJIlobwZmvMfWHvxh6tSp9OvXjxIlSvDee+/RuXNnW4RmAponSaFwWkIAUNVdIlLYizEZL0lLBuv2ORak16t0c8B2G6WVpKhevTodO3bkzTffpHTp0r4Oyxif8yQpbBKRaThaBwBdsIJ4BVJa6yAtGQTiuoPz588zdOhQgoKCGDVqFE2aNKFJkya+DsuYfMOTxWt9gB+BQcDzQAKOVc2mAArkhWhfffUVUVFRjBs3jnPnzqF6xS6zxgTlQC8SAAAWfUlEQVS8LFsKIlIDqAIsU9XReROSuRpZLUBLE6hjCKdPn2bQoEFMnz6dKlWqsGrVKitvbUwmstpk50UcO6xtAuqIyCuq+k5OLi4iLYC3gCBgpqqOzOS8B4H3gTqqujEn9whEGSUA93GCzATq1NOffvqJd999l+eee47hw4dTrFgxX4dkTL6VVUuhCxClqr+JSGkgDvA4KYhIEI7Nee4FEoENIrLcfdDaeV5xHOsg1uU0+ECV0cyhQB4nyMixY8dYtGgRffv2pVq1auzfv98Gko3xQFZJIUlVfwNQ1WMi4sn4g7u6QLyqJgCIyCKgHbAz3Xn/BEYDz+Xw+gEjfcvAViBnTlVZuHAh/fr148yZMzRv3pywsDBLCMZ4KKtf9JVFZKnzYxlQxe310iy+Lk1Z4JDb60TnMRcRuR0op6r/znHkASStZZAmULuBsnPo0CHatGlDly5dCAkJYfPmzVbAzpgcyqql8EC612/n8NoZrQByTfdwtjzGA92zvZBIb6A3QPnygdk9Yi2DrCUnJ3P33Xfz888/M378ePr27UtQUJCvwzKmwMlqj+aV13jtRBwb9KQJBo64vS4ORAJfOVeQ/hVYLiJt0w82q+p0YDpAdHR0wMwjtJXH2du/fz/lypWjUKFCTJs2jcqVK1O5cmVfh2VMgZXTcYKc2ACEikgl5wrozsDytDdV9bSqllLViqpaEVgLXJEQApkVrMtccnIyY8eOpXr16q4d0e655x5LCMZcI68ViFfVZBF5CliBY0rqO6q6Q0ReATaq6vKsrxC4rKR11rZt20ZsbCwbN26kXbt2PPBA+p5OY8zV8jgpiEgRVU3KycVVNQ7HVFb3Y0MzOffunFzbn1kLIXOTJ0/m6aef5qabbmLx4sV07NjRCtgZk4s8KZ1dF5gFlADKi0hNoKeq9vV2cP7Mky0wrYXwh7QCdpGRkXTu3Jnx48dTqlQpX4dljN/xpKUwAWgNfAigqltFxGoEXKWMKpWmZy2EP/z222+89NJLFCpUiDFjxnDXXXdx1113+TosY/yWJ0nhOlU9kK6JnuKlePyeVSr13MqVK+nVqxf79u2jb9++rtaCMcZ7PEkKh5xdSOosXdEX+MG7Yfk36xrK2qlTp3juueeYNWsWoaGhfPPNNzRu3NjXYRkTEDyZkvo4MAAoD/wC3Ok8ZoxX/PLLLyxatIjnn3+erVu3WkIwJg9l21JQ1aM41hgY4zVpieDpp5+matWq7N+/3waSjfEBT2YfzcCtPEUaVe3tlYhMQFFVFixYwNNPP825c+e47777CA0NtYRgjI940n30H2Cl82M1cAuQo/UKxmTk4MGDtGrViq5du1K1alW2bNlCaGior8MyJqB50n202P21iMwHvvBaRH7AkzUIgS6tgN3Ro0eZMGECTzzxhBWwMyYfuJoyF5WACrkdiD/JqohdoK9BSEhIoEKFChQqVIgZM2ZQpUoVKlas6OuwjDFOnowpnOSPMYXrgF+Bwd4Myh/YtNPLJScnM27cOF5++WVGjx5Nv379iImJ8XVYxph0skwK4lgpVBNI6wtJVdWAKV2dU1bqOmNbtmwhNjaWTZs20aFDBzp27OjrkIwxmchyoNmZAJapaorzwxJCFqyQ3ZXefvtt6tSpw+HDh1myZAlLly7ltttu83VYxphMeDKmsF5EaqvqJq9HU0BZqesrpZWkiIqKokuXLrzxxhvcfPOVdZ6MMflLpklBRAqpajLQCOglIj8Cv+HYZlNVtXYexZjvWQvhD+fOnWPIkCFcf/31jB071grYGVPAZNVSWA/UBtrnUSwFmrUQ4PPPP6d3794cPHjQCtgZU0BllRQEQFV/zKNYChT3tQiBPrB88uRJBgwYwJw5c6hatSrffPMNjRo18nVYxpirkFVSKC0iAzJ7U1Xf8EI8+V5G+yEEerfR0aNHWbJkCS+88AJDhw6laNGivg7JGHOVskoKQcANOFsMgS6jZBDI+yH8/PPPLFy4kP79+7sK2JUsWdLXYRljrlFWSeEnVX0lzyLJ52xzHAdVZd68efTv35/z58/TunVrQkNDLSEY4yeyWqdgLYR00gaTAzUh7N+/nxYtWtC9e3fCw8OtgJ0xfiirloLVIMBWKadJTk6madOmHD9+nEmTJtGnTx+uu86TIrvGmIIk06Sgqr/mZSD5VaCvQYiPj6dSpUoUKlSId955h8qVK1OhgtVDNMZfXU2VVL+X0XTTQFuDcOnSJcaMGcPw4cMZM2YM/fr1o2nTpr4OyxjjZdb+z0Ba6wACs9T1pk2bqFu3LkOGDKFdu3Z06tTJ1yEZY/KItRQyEYitA4AJEyYwYMAASpcuzdKlS+nQoYOvQzLG5CFrKRjAMdUU4Pbbb+fRRx9l586dlhCMCUDWUnATiDONzp49ywsvvECRIkUYN24cjRs3pnHjxr4OyxjjI9ZScBNoM40+++wzIiMjmTx5MqqKbZdhjLGWQjqBMJZw4sQJBgwYwLx586hevTqrV6+mfn3/fmZjjGespYCj26jTtDWuGUf+7sSJEyxbtox//OMfbN682RKCMcbFqy0FEWkBvIWjuN5MVR2Z7v0BQE8gGTgGPKaqB7wZk7vMitz5o59++okFCxbw7LPPEhYWxoEDB7jpppt8HZYxJp/xWlIQkSBgEnAvkAhsEJHlqrrT7bTNQLSqnheRx4HRQJ5Nig+EIneqyuzZsxkwYABJSUm0a9eO0NBQSwjGmAx5s/uoLhCvqgmqehFYBLRzP0FVv1TV886Xa4FgL8aTIX8ucrdv3z6aNWtGbGwsNWvWZOvWrVbAzhiTJW92H5UFDrm9TgTqZXF+LPBpRm+ISG+gN0D58v73y9sbkpOT+b//+z9OnDjBlClT6N27txWwM8Zky5tJIaPS2xnOeRSRvwHRQJOM3lfV6cB0gOjoaJs3mYW9e/dSuXJlChUqxOzZs6lSpQrlypXzdVjGmALCm386JgLuv42CgSPpTxKRe4AhQFtVTfJiPH7t0qVLjBgxgsjISN5++20A7r77bksIxpgc8WZLYQMQKiKVgMNAZ+AR9xNE5HZgGtBCVY96MRa/tnHjRmJjY9m2bRudO3fm4Ycf9nVIxpgCymstBVVNBp4CVgC7gH+p6g4ReUVE2jpPG4NjH+j3RWSLiCz3Vjz+6q233qJevXocP36cjz76iIULF3LLLbf4OixjTAHl1XUKqhoHxKU7NtTt83u8eX9/pqqICNHR0cTGxjJ69Gj+8pe/+DosY0wBZ2UuCpgzZ87w/PPPU7RoUcaPH0/Dhg1p2LChr8MyxviJgJyjWFDLWsTFxREREcH06dMpVKiQFbAzxuS6gEwKBa0a6vHjx/nb3/5Gq1atKFGiBN9//z1jxoxBJKNZv8YYc/UCtvuoIFVDPXnyJB9//DEvv/wyL774IoULF/Z1SMYYPxWwSSG/O3z4MAsWLGDgwIGEhoZy4MABG0g2xnhdQHYf5WeqyowZMwgPD2fYsGH8+OOPAJYQjDF5wpJCPvLjjz8SExND7969qV27Ntu2bSMkJMTXYRljAoh1H+UTycnJxMTE8OuvvzJt2jR69uxpBeyMMXnOkoKP7dmzhypVqlCoUCHmzp1LlSpVCA7O8wrixhgDWPeRz1y8eJHhw4dTo0YNJk2aBECTJk0sIRhjfMpaCj6wfv16YmNj2b59O4888ghdunTxdUjGGANYSyHPvfnmm9SvX9+19mDBggWUKlXK12EZYwxgSSHPpJWkqFu3Lr169WLHjh20bt3ax1EZY8zlrPvIy06fPs2gQYP405/+xJtvvkmDBg1o0KCBr8MyxpgMWUvBiz7++GPCw8OZOXMmRYoUsQJ2xph8z5KCFxw7doxHHnmEtm3bUrJkSdauXcuoUaOsgJ0xJt+zpOAFp0+fJi4ujuHDh7Nx40bq1Knj65CMMcYjNqaQSw4dOsS7777L4MGDCQkJ4cCBA5QoUcLXYRljTI5YS+EapaamMnXqVCIiIhgxYoSrgJ0lBGNMQRQwLYX31h3koy2HAVwb7FyrvXv30qtXL77++mtiYmKYPn06lStXvubrGmOMrwRMUnDfbS03dlxLTk7m3nvv5dSpU8yaNYsePXrYQLIxpsALmKQAubPb2q5duwgNDaVQoULMnz+fKlWqUKZMmVyK0BhjfMvGFDyUlJTEyy+/TFRUFG+//TYAjRs3toRgjPErAdVSuFpr164lNjaWnTt30rVrV7p27errkIwxxiuspZCNcePG0aBBA86ePUtcXBzz5s2jZMmSvg7LGGO8wpJCJlJTUwGoX78+ffr0Yfv27bRs2dLHURljjHdZ91E6p06d4tlnn6VYsWJMnDjRCtgZYwKKtRTcfPjhh4SHhzN37lyKFy9uBeyMMQHHkgJw9OhRHnroITp06MCtt97K+vXree2112zdgTEm4FhSAM6cOcMXX3zBq6++yvr166ldu7avQzLGGJ/walIQkRYiskdE4kVkcAbvFxGRxc7314lIRW/G4+7gwYO8+uqrqCohISEcPHiQF198keuvvz6vQjDGmHzHa0lBRIKASUBLIBx4WETC050WC5xU1RBgPDDKW/GkSU1NZfLkyURERPDaa6+5CtgVL17c27c2xph8z5sthbpAvKomqOpFYBHQLt057YC5zs+XADHipY788DI3ckvqCe6++26efPJJ6tevz44dOwgJCfHG7YwxpkDy5pTUssAht9eJQL3MzlHVZBE5DZQEjud2MENaViUkpBWnT59m9uzZdOvWzQaSjTEmHW8mhYx+46af4+nJOYhIb6A3QPny5a8qmEKFCvHuu+9SpUoVbrvttqu6hjHG+Dtvdh8lAuXcXgcDRzI7R0QKASWAX9NfSFWnq2q0qkaXLl36qgNq1KiRJQRjjMmCN5PCBiBURCqJSGGgM7A83TnLgW7Ozx8EVqmtGDPGGJ/xWveRc4zgKWAFEAS8o6o7ROQVYKOqLgdmAfNFJB5HC6Gzt+IxxhiTPa/WPlLVOCAu3bGhbp9fADp6MwZjjDGesxXNxhhjXCwpGGOMcbGkYIwxxsWSgjHGGBdLCsYYY1ykoC0LEJFjwIGr/PJSeKGERj5nzxwY7JkDw7U8cwVVzXb1b4FLCtdCRDaqarSv48hL9syBwZ45MOTFM1v3kTHGGBdLCsYYY1wCLSlM93UAPmDPHBjsmQOD1585oMYUjDHGZC3QWgrGGGOy4JdJQURaiMgeEYkXkcEZvF9ERBY7318nIhXzPsrc5cEzDxCRnSKyTURWikgFX8SZm7J7ZrfzHhQRFZECP1PFk2cWkYec/693iMh7eR1jbvPgZ7u8iHwpIpudP9/3+SLO3CIi74jIURHZnsn7IiITnN+PbSJSO1cDUFW/+sBRpvtHoDJQGNgKhKc75wlgqvPzzsBiX8edB8/cFCjm/PzxQHhm53nFgW+AtUC0r+POg//PocBm4Cbn61t8HXcePPN04HHn5+HAfl/HfY3PfBdQG9ieyfv3AZ/i2LnyTmBdbt7fH1sKdYF4VU1Q1YvAIqBdunPaAXOdny8BYqRgb9ic7TOr6peqet75ci2OnfAKMk/+PwP8ExgNXMjL4LzEk2fuBUxS1ZMAqno0j2PMbZ48swI3Oj8vwZU7PBYoqvoNGexA6aYdME8d1gJ/EZFc21LSH5NCWeCQ2+tE57EMz1HVZOA0UDJPovMOT57ZXSyOvzQKsmyfWURuB8qp6r/zMjAv8uT/cxgQJiKrRWStiLTIs+i8w5NnHgb8TUQScezf0jdvQvOZnP57zxGvbrLjIxn9xZ9+ipUn5xQkHj+PiPwNiAaaeDUi78vymUXkOmA80D2vAsoDnvx/LoSjC+luHK3Bb0UkUlVPeTk2b/HkmR8G5qjqOBGpj2M3x0hVTfV+eD7h1d9f/thSSATKub0O5srmpOscESmEo8mZVXMtv/PkmRGRe4AhQFtVTcqj2Lwlu2cuDkQCX4nIfhx9r8sL+GCzpz/bH6nqJVXdB+zBkSQKKk+eORb4F4CqrgGK4qgR5K88+vd+tfwxKWwAQkWkkogUxjGQvDzdOcuBbs7PHwRWqXMEp4DK9pmdXSnTcCSEgt7PDNk8s6qeVtVSqlpRVSviGEdpq6obfRNurvDkZ/tDHJMKEJFSOLqTEvI0ytzlyTMfBGIARKQ6jqRwLE+jzFvLgUeds5DuBE6r6k+5dXG/6z5S1WQReQpYgWPmwjuqukNEXgE2qupyYBaOJmY8jhZCZ99FfO08fOYxwA3A+84x9YOq2tZnQV8jD5/Zr3j4zCuAZiKyE0gBBqrqCd9FfW08fOZngRki0h9HN0r3gvxHnogsxNH9V8o5TvIycD2Aqk7FMW5yHxAPnAd65Or9C/D3zhhjTC7zx+4jY4wxV8mSgjHGGBdLCsYYY1wsKRhjjHGxpGCMMcbFkoLJd0QkRUS2uH1UzOLciplVk8zhPb9yVuLc6iwRUfUqrtFHRB51ft5dRMq4vTdTRMJzOc4NIlLLg695RkSKXeu9TWCwpGDyo99VtZbbx/48um8XVa2Jo1jimJx+sapOVdV5zpfdgTJu7/VU1Z25EuUfcU7GszifASwpGI9YUjAFgrNF8K2IbHJ+NMjgnAgRWe9sXWwTkVDn8b+5HZ8mIkHZ3O4bIMT5tTHOOv3/c9a5L+I8PlL+2J9irPPYMBF5TkQexFFfaoHznn9y/oUfLSKPi8hot5i7i8jEq4xzDW6F0ERkiohsFMc+CsOdx/rhSE5fisiXzmPNRGSN8/v4vojckM19TACxpGDyoz+5dR0tcx47CtyrqrWBTsCEDL6uD/CWqtbC8Us50Vn2oBPQ0Hk8BeiSzf3bAP8TkaLAHKCTqtbAUQHgcRG5GegARKhqFDDC/YtVdQmwEcdf9LVU9Xe3t5cA97u97gQsvso4W+Aoa5FmiKpGA1FAExGJUtUJOOriNFXVps7SFy8B9zi/lxuBAdncxwQQvytzYfzC785fjO6uB9529qGn4Kjpk94aYIiIBANLVXWviMQAdwAbnOU9/oQjwWRkgYj8DuzHUX65KrBPVX9wvj8XeBJ4G8f+DDNF5BPA49LcqnpMRBKcNWv2Ou+x2nndnMT5ZxxlH9x33XpIRHrj+Hd9G44NZ7al+9o7ncdXO+9TGMf3zRjAkoIpOPoDvwA1cbRwr9g0R1XfE5F1QCtghYj0xFFmeK6qvuDBPbq4F8wTkQz32HDW46mLowhbZ+Ap4P9y8CyLgYeA3cAyVVVx/Ib2OE4cO5CNBCYB94tIJeA5oI6qnhSROTgKw6UnwBeq+nAO4jUBxLqPTEFRAvjJWSO/K46/ki8jIpWBBGeXyXIc3SgrgQdF5BbnOTeL5/tT7wYqikiI83VX4GtnH3wJVY3DMYib0QygszjKd2dkKdAexz4Ai53HchSnql7C0Q10p7Pr6UbgN+C0iNwKtMwklrVAw7RnEpFiIpJRq8sEKEsKpqCYDHQTkbU4uo5+y+CcTsB2EdkCVMOxZeFOHL88PxeRbcAXOLpWsqWqF3BUoHxfRP4HpAJTcfyC/bfzel/jaMWkNweYmjbQnO66J4GdQAVVXe88luM4nWMV44DnVHUrjr2ZdwDv4OiSSjMd+FREvlTVYzhmRi103mctju+VMYBVSTXGGOPGWgrGGGNcLCkYY4xxsaRgjDHGxZKCMcYYF0sKxhhjXCwpGGOMcbGkYIwxxsWSgjHGGJf/B9PduJtdsmkpAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from sklearn.metrics import roc_curve\n",
"from sklearn.metrics import roc_auc_score\n",
"\n",
"y_scores = cross_val_predict(svm_clf, i1_mt, y, cv=kf, method=\"decision_function\")\n",
"fpr, tpr, thresholds = roc_curve(y, y_scores)\n",
"\n",
"fpr, tpr, thresh = roc_curve(y, y_scores)\n",
"auc = roc_auc_score(y, y_scores)\n",
"plt.plot(fpr,tpr,label=\"AUC = \"+repr(auc))\n",
"\n",
"plt.plot([0,1],[0,1],'k--')\n",
"plt.xlabel(\"False Positive Rate\")\n",
"plt.ylabel(\"True Positive Rate\")\n",
"plt.legend(loc=0)\n",
"\n",
"plt.savefig('figures/classification/metric_tfidf_roc.pdf', format='pdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**F1 SCORE**"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"F1 Score: 0.6654501216545012\n"
]
}
],
"source": [
"from sklearn.metrics import f1_score\n",
"m_f1 = f1_score(y, ypred, average='macro') \n",
"print(f\"F1 Score: {m_f1}\")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"275"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(i1_ut)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Classification Using Ultrametric Closure with TF-IDF/Cosine Similarity"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Mean cross-validation accuracy: 0.57\n",
" precision recall f1-score support\n",
"\n",
" False 0.5714285714 0.5547445255 0.5629629630 137\n",
" True 0.5704225352 0.5869565217 0.5785714286 138\n",
"\n",
"avg / total 0.5709237242 0.5709090909 0.5707955748 275\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAECCAYAAAAxcwG5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAABi1JREFUeJzt3DGoXvUdxvHn39ykXFAK1rhYKhVKaeuQQqBDR+lS6FpcXLMoTu7SsUtxcBA3KXQpXYQutWuRgmBEumrtmFytTlmMvw6xtIgl977m3GN8Pp8t5OTwcE++nHNy35s1MwH6fGPvAcA+xA+lxA+lxA+lxA+lxA+ljvYecD9Za91O8k6Si0k+SfJqkhdn5tNdh3Eq/3P9jpK8l+Tpmflo31X7cec/m1szc2Vmfpzk50l+keSFnTdxev+5fk8k+TDJM3sP2pP4DzQzN5JcS/LsWmvtvYczeyPJo3uP2JP4v4SZeTd3voaP7L2F01trXUjyZJLX9t6yJ/F/ee7694/jtdb1JB8keSjJ6zvv2ZX4v4S11uNJbie5sfcWTuXWzFxJ8liSS/HOzyHWWpeTvJzkpfHTUfeVmfk4yXNJnl9rXdx7z17EfzbHa63ra62/J/lLkj8n+fXOmzjAzLyV5O0kT+29ZS/LTQs6ufNDKfFDKfFDKfFDKfFDKfEfaK11be8NHMa1u0P8h/MX6P7l2kX8UGuzD/mso+NZlx7c5NxfBfPJrayj471nbOYnP/zu3hM2c/PkZi4/fHnvGZt5//1/5OTk5K4/cLbZ/+SzLj2Yb/7gV1udno399W8v7T2BA/3sp1dPdZzHfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfiglfih1dLcD1lq3k7yT5GKST5K8muTFmfl0423Ahu4af5JbM3MlSdZajyT5fZJvJXlhy2HAts702D8zN5JcS/LsWmttMwk4D2d+55+Zdz/7c4/c+znAeTnNY/8X+cK7/lrrWu48GSQXHzjw1MB5OPOdf631eJLbSW58/vdm5pWZuTozV9fR8b3YB2zkTPGvtS4neTnJSzMz20wCzsNpHvuP11rX899v9f0uyW83XQVs7q7xz8yF8xgCnC+f8INS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odS4odSR1ud+Eff/07+8KffbHV6Nva9Z/649wQOdPLPf53qOHd+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KCV+KHV0moPWWreTvPPZ8e8leXpmPtpyGLCt0975b83MlZl5IsmHSZ7ZcBNwDg557H8jyaP3eghwvs4U/1rrQpInk7y2zRzgvJw2/uO11vUkHyR5KMnrX3TQWuvaWuvNtdabH35wcq82Ahs40zt/kseSXMr/eeefmVdm5urMXH3o2w/fq43ABs702D8zHyd5Lsnza62L20wCzsOZ/8FvZt5K8naSp+79HOC8nOr7/DPzwOd+/ctt5gDnxSf8oJT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4oZT4odSamW1OvNbNJO9vcvKvhoeTnOw9goN83a/dYzNz+W4HbRb/191a682Zubr3Ds7OtbvDYz+UEj+UEv/hXtl7AAdz7eKdH2q580Mp8UMp8UMp8UMp8UOpfwO8MOlpIwsmKgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 288x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"F1 Score: 0.5707671957671958\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd0lVXWwOHfMaEMDjJShYSSCgkBESIBQaqACFJG0QiCJTQRRFSUIggI0rv0olgAEUWYzwjMCIgiVdpAAAkJJXRCExlKkv39kWIIKRe4yXvLftbKWrec3LtPctmc7PcUIyIopZRyLfdZHYBSSin70+SulFIuSJO7Ukq5IE3uSinlgjS5K6WUC9LkrpRSLkiTu1JKuSBN7kop5YI0uSullAvytOqNixcvLhUqVLDq7ZVSyin99ttv50SkRE7tLEvuFSpUYNu2bVa9vVJKOSVjzBFb2mlZRimlXJAmd6WUckGa3JVSygVZVnPPzM2bN4mLi+PatWtWh6KUXRQsWBBvb2/y5ctndSjKzThUco+Li6Nw4cJUqFABY4zV4Sh1T0SE+Ph44uLi8PHxsToc5WZyLMsYY+YbY84YY/Zk8bwxxkwxxkQbY3YbY6rfbTDXrl2jWLFimtiVSzDGUKxYMf1LVFnClpr7p8CT2TzfHAhI+eoKzLiXgDSxK1ein2dllRzLMiKy3hhTIZsmrYHPJPm8vk3GmH8YY0qLyEk7xaiUUk5t4eajLN95nITr/+P6Hxd5tGpFPni6cq6+pz1my3gBx9Ldj0t57DbGmK7GmG3GmG1nz561w1vnjmXLlmGMYf/+/WmPrVu3jpYtW97S7uWXX2bp0qVA8sXgfv36ERAQQEhICDVr1uSHH36451hGjhyJv78/FStWZNWqVZm2EREGDhxIYGAgQUFBTJky5Zbnt27dioeHR1qsAAsWLCAgIICAgAAWLFhw22u2atWKkJCQtPuDBg2iatWqVKtWjaZNm3LixAkA9u/fT+3atSlQoADjxo275TUuXrzIs88+S6VKlQgKCmLjxo0AnD9/niZNmhAQEECTJk24cOECABcuXKBt27ZUrVqVmjVrsmfPX5XAlStXUrFiRfz9/Rk1alTa4x9//DH+/v4YYzh37lza48uXL0+LNzQ0lF9++SXHvg8cOJCyZcvy97///ZZ+rF+/nurVq+Pp6XnLzxDg3XffpXLlygQFBfHGG2+gZxKrzCzfeZwtv65n1Ycd2TCrP0lJSbn/piKS4xdQAdiTxXPfA3XT3f8RqJHTa9aoUUMyioqKuu0xK7Rr107q1q0rH3zwQdpja9eulRYtWtzS7qWXXpKvv/5aRETee+896dSpk1y7dk1ERE6dOiVfffXVPcWxd+9eqVq1qly7dk1iYmLE19dXEhISbms3f/586dixoyQmJoqIyOnTp9OeS0hIkIYNG0rz5s3TYo2PjxcfHx+Jj4+X8+fPi4+Pj5w/fz7te7755ht54YUXpHLlymmPXbp0Ke325MmTpVu3bmnvtWXLFhkwYICMHTv2lrg6deokc+bMERGR69evy4ULF0REpG/fvjJy5EgRERk5cqS8++67IiLyzjvvyJAhQ0REZN++fdKoUaO0Pvj6+sqhQ4fk+vXrUrVqVdm7d6+IiGzfvl1iY2OlfPnycvbs2bT3/uOPPyQpKUlERHbt2iUVK1bMse8bN26UEydOyP33339LP2JjY2XXrl3SsWPHtJ+hiMiGDRvksccek4SEBElISJBatWrJ2rVrb/v9OMrnWuW+Lzcdkedm/nrLV5sJq+TBR54UQPz9/WXdunX39B7ANrEhb9tjtkwcUDbdfW/ghB1e1xJXrlxhw4YNrF27llatWjFkyJAcv+fq1avMmTOH2NhYChQoAECpUqV47rnn7imW5cuXEx4eToECBfDx8cHf358tW7ZQu3btW9rNmDGDhQsXct99yX+IlSxZMu25qVOn8swzz7B169a0x1atWkWTJk0oWrQoAE2aNGHlypW88MILXLlyhQkTJjB79uxb4n/ggQfSbv/5559pteSSJUtSsmRJvv/++1tiunz5MuvXr+fTTz8FIH/+/OTPnz+tX+vWrQPgpZdeokGDBowePZqoqCj69+8PQKVKlTh8+DCnT58mJiYGf39/fH19AQgPD2f58uUEBwfzyCOPZPqzSz/6Th9vdn2vVatWpq+VugdS6s83lTGGa9eucePGDUSEmzdvUqpUqUxfQ7me1FJLeptjzwMQ5pP8+UpKSmTNmG78cfooLV/szpLZE/jb3/6WJ/HZI7mvAHoaYxYDYcAlsUO9fei/9hJ14vI9B5decJkHcqxzfffddzz55JMEBgZStGhRtm/fTvXq2U8Aio6Oply5crckwKz06dOHtWvX3vZ4eHg4/fr1u+Wx48eP35JwvL29OX78eMZv5dChQ3z11VcsW7aMEiVKMGXKFAICAjh+/DjLli1jzZo1tyT348ePU7Zs2Uxfd9CgQbz99tsUKlTotvcZOHAgn332GUWKFMm0D+nFxMRQokQJXnnlFXbt2kWNGjWYPHky999/P6dPn6Z06dIAlC5dmjNnzgDw8MMP8+2331K3bl22bNnCkSNHiIuLyzTezZs3Z/v+kFxe69+/P2fOnEn7zye7vt+p2rVr07BhQ0qXLo2I0LNnT4KCgu7qtZTzWb7zOFEnLxNc+q9/92E+RWldzYtm/vdTtGhRjDEse2gCZcuWJTQ0NE/jyzG5G2MWAQ2A4saYOOADIB+AiMwEIoGngGjgKvBKbgWbFxYtWsSbb74JJCfcRYsWUb169SxnPdzpbIiJEyfa3FYyqd9m9n7Xr1+nYMGCbNu2jW+//ZZXX32Vn3/+mTfffJPRo0fj4eFh0+vu3LmT6OhoJk6cyOHDh29rM2LECEaMGMHIkSP5+OOPGTp0aJaxJyQksH37dqZOnUpYWBi9e/dm1KhRfPjhh1l+T79+/ejduzfVqlWjSpUqPPLII3h6etr8c8iobdu2tG3blvXr1zNo0CD+85//3PVrZSY6Opp9+/YRFxcHJP8VsH79eurVq3dXr6ccU2YjdCAtsX/V7a+/pEWEL7/8ksCnkj/vXbp0oW3btnkZbhpbZsu8kMPzArxut4hS5PaV5MzEx8ezZs0a9uzZgzGGxMREjDGMGTOGYsWKpV34S3X+/HmKFy+Ov78/R48e5Y8//qBw4cLZvsedjNy9vb05duyva9VxcXGUKVPmtu/19vbmmWeeAZIT2iuvJP//um3bNsLDwwE4d+4ckZGReHp64u3tnVYWSX3dBg0asHHjRn777TcqVKhAQkICZ86coUGDBre0BWjfvj0tWrTINrl7e3vj7e1NWFgYAM8++2zahdBSpUpx8uRJSpcuzcmTJ9PKSA888ACffPIJkPyPxMfHBx8fH65evWrTzyEr9erV49ChQ5w7dy7Lvt+NZcuWUatWrbQSUPPmzdm0aZMmdydmS6klVXDpB2hd7a+5I8eOHaN79+5ERkZSq1Yt6tSpk/sBZ8OhVqhabenSpXTq1IlZs2alPVa/fn1++eUXatasyYkTJ9i3bx9BQUEcOXKEXbt2Ua1aNQoVKkRERARvvPEGs2bNIn/+/Jw8eZIff/yRF1988Zb3uJORe6tWrWjfvj1vvfUWJ06c4ODBg9SsWfO2dm3atGHNmjW8+uqr/PTTTwQGBgIQGxub1ubll1+mZcuWtGnThvPnzzNgwIC0/6xWr17NyJEjKVq0KK+99hoAhw8fpmXLlmmJ8ODBgwQEBACwYsUKKlWqlG3sDz30EGXLluXAgQNUrFiRH3/8keDg4LR+LViwgH79+rFgwQJat24NJM+uKVSoEPnz52fu3LnUq1ePBx54gEcffZSDBw8SGxuLl5cXixcvZuHChdm+f3R0NH5+fhhj2L59Ozdu3KBYsWI0a9Ys077fjXLlyjFnzhz69++PiPDTTz+l/dWnHE9WI/D0MkvkqaWW9mHlsvy+RYsW0a1bNxITE5k0aRI9e/a87S/mPGfLVdfc+HLE2TL169eXH3744ZbHJk+eLN27dxcRkV9++UXCwsLk4YcfltDQUFm9enVau+vXr0vfvn3Fz89PKleuLDVr1pSVK1fec0zDhw8XX19fCQwMlMjIyLTHmzdvLsePHxcRkQsXLshTTz0lISEhUqtWLdm5c+dtr5N+Zo+IyLx588TPz0/8/Pxk/vz5t7WPjY29ZbbMP//5T6lcubJUqVJFWrZsKXFxcSIicvLkSfHy8pLChQtLkSJFxMvLK21mzY4dO6RGjRpSpUoVad26ddqslHPnzkmjRo3E399fGjVqJPHx8SIi8uuvv4q/v79UrFhR2rZte8sMnu+//14CAgLE19dXhg8fnvb45MmTxcvLSzw8PKR06dISEREhIiKjRo2S4OBgefjhh6VWrVry888/59j3vn37ipeXlxhjxMvLK2221JYtW8TLy0sKFSokRYsWleDgYBFJnsXTtWtXqVSpkgQFBUmfPn0y/R1a/blWyZ6b+auEfLDyttksGb++3HTkjl/7hx9+kCeeeEJiYmJyIfJbYeNsGSMWzcsNDQ2VjId1pI6KlXIl+rl2DM/PSl5nkb5GfrcSEhKYOHEiN27cYODAgUDyQDkvViQbY34TkRyvzmpZRinldGwpsWSUcWbL3dq1axcRERH89ttvPPfcc2lJ3dG2mtD93JVSTid1GuKdyHgB9E5dv36dQYMGERoayrFjx/j6669ZvHixwyX1VA43cs+rP22UygtWlT3dQcZpiLnt4MGDjB49mvbt2zNhwgSKFSuWZ+99NxwquRcsWJD4+Hjd9le5BEnZz71gwYJWh+L0MpZh7FViycmVK1dYvnw5HTp0ICQkhP3796etlHZ0DpXcvb29iYuLw5E3FVPqTqSexKTuTcbVoPdaYrHFv//9b7p27cqRI0eoXr06QUFBTpPYwcGSe758+fTEGqXcVHYXSTNbDZpbLly4wDvvvMP8+fMJDAzkp59+csrZTg6V3JVSzutuZrCkl9VKUMibkTpAYmIiderU4ffff6d///4MHjzYactqmtyVUjm629Wdd8KWlaC55dy5cxQtWhQPDw8++ugjypUrl+OGgY5Ok7tSKkeZ7YCYkZXJ+W6JCJ9//jlvvvkmo0aNomvXrrRp08bqsOxCk7tSyiZ5PfUwtx05coRu3bqxatUqHnvsMZfb8E2Tu1IqTU7b27qKL774gtdeew0RYerUqfTo0eO2w1icnWv1Ril1T7Ja+ZlXFzTzSokSJahTpw579+6lZ8+eLpfYQUfuSin+GrHn5ZTDvHTz5k3Gjx/PzZs3GTRoEM2aNaNp06YuvVhSk7tSbianAylcaYQOsGPHDiIiItixYwfh4eEOu9GXvWlyV8rNZHf2pzPNdMnJtWvXGDZsGGPGjKF48eJ88803/POf/7Q6rDyjyV0pN+HqpZeMoqOjGTduHJ06dWL8+PE8+OCDVoeUpzS5K+WC3K30kurKlSssW7aMjh07EhISwoEDB9x2SxNN7kq5IHcpvaS3atUqunbtyrFjxwgNDSUoKMhtEztoclfKZblD6QUgPj6et956i88++4xKlSrx888/O+VGX/amyV0p5bRSN/qKjo5m4MCBvP/++0670Ze9aXJXSjmds2fPUqxYMTw8PBg9ejTly5enWrVqVoflUFxvWZZSymWJCJ988gmBgYHMmTMHgNatW2tiz4Qmd6WUUzh8+DDNmjXj1VdfpUqVKjRs2NDqkByaJnelXMjCzUd5ftbGTPeHcWaff/45ISEhbNy4kenTp7Nu3ToCAwOtDsuhac1dKReSfgqkK81lL1WqFPXq1WPmzJmUK+eaUzntTZO7Ui7GFaZA3rx5kzFjxpCYmMjgwYNp2rQpTZs2tTosp6JlGaWUQ9m+fTuPPvoo77//PgcOHEBErA7JKdk0cjfGPAlMBjyAuSIyKsPz5YAFwD9S2vQTkUg7x6qUSuGKh2r873//Y+jQoYwbN44SJUqwbNkylznyzgo5jtyNMR7ANKA5EAy8YIwJztDsfWCJiDwChAPT7R2oUuovrnioRkxMDBMmTODll18mKipKE/s9smXkXhOIFpEYAGPMYqA1EJWujQCpw4UiwAl7BqmUup0r1NYvX77Mt99+y8svv0zlypU5ePAg5cuXtzosl2BLzd0LOJbuflzKY+kNAV40xsQBkUAvu0SnlHJZkZGRhISEEBERwb59+wA0sduRLck9s+NKMl7heAH4VES8gaeAz40xt722MaarMWabMWbb2bNn7zxapZTTO3fuHB07dqRFixYULlyYDRs26EZfucCWskwcUDbdfW9uL7tEAE8CiMhGY0xBoDhwJn0jEZkNzAYIDQ3VS+BK3YH0F1Gd9cJp6kZfMTExDB48mAEDBlCgQAGrw3JJtiT3rUCAMcYHOE7yBdP2GdocBRoDnxpjgoCCgA7NlboHGWfEpD9sw9kunJ4+fZoSJUrg4eHBuHHjKF++PFWrVrU6LJeWY3IXkQRjTE9gFcnTHOeLyF5jzDBgm4isAN4G5hhj+pBcsnlZdHKqUvck44EbznjYhogwf/583n77bUaNGkX37t15+umnrQ7LLdg0zz1lznpkhscGp7sdBdSxb2hKKWeeERMTE0OXLl1Ys2YN9evX54knnrA6JLei2w8o5QAyW5TkrHV1gAULFtCjRw88PDyYOXMmXbp04b77dEF8XtKftlIOILNFSc5WV0+vTJkyNGrUiKioKLp166aJ3QI6clfKQThzCebGjRuMGjWKpKQkhgwZQpMmTWjSpInVYbk1Te5KWSBjGcaZSzBbt27l1VdfZc+ePXTs2BERwZjMlseovKTJXalcktXmXnDrtEZwzhLM1atXGTx4MBMnTqR06dKsWLFCZ8I4EE3uSuWSjFMZ03PGaY0ZxcbGMnXqVLp06cLo0aMpUqSI1SGpdDS5K2VnqSP21MTurHX0zFy6dIlvv/2WV155hcqVKxMdHU3ZsmVz/kaV5/QStlJ25qpH3X3//fdUrlyZzp07s3//fgBN7A5Mk7tSuSB1xO7MZZdUZ8+epUOHDrRs2ZIHH3yQjRs3UqlSJavDUjnQsoxSKkuJiYnUrVuX2NhYhg4dSr9+/cifP7/VYSkbaHJXSt3m1KlTlCxZEg8PD8aPH0+FChUICQmxOix1B7Qso5RKk5SUxKxZswgMDGTWrFkAtGzZUhO7E9LkrpSdLNx8lOdnbcz0bFNnEB0dTePGjenevTuPPvoozZo1szokdQ80uStlJ848S+aTTz6hSpUqbN++nTlz5vCf//wHX19fq8NS90Br7krZkbPOay9XrhzNmjVj2rRpeHk5139MKnOa3JVyQ9evX2fkyJEkJSUxbNgwGjduTOPGja0OS9mRlmWUcjObN2+mRo0aDB06lKNHj6KHprkmHbkrdRec8XCNP//8k0GDBjFp0iS8vLz4v//7P1q0aGF1WCqX6MhdqbvgjIdrHDlyhOnTp9O9e3f27t2rid3F6chdqTvgbJuCXbx4kaVLl9K5c2eCg4OJjo7G29vb6rBUHtCRu1J3wJmmOy5fvpzg4GC6d++ettGXJnb3ocldqTvk6JuCnTlzhvDwcNq0aUOJEiXYtGmTbvTlhrQso5QNMpZjHFViYiJ16tTh6NGjDB8+nHfffZd8+fJZHZaygCZ3pWzg6OWYEydO8NBDD+Hh4cHkyZOpUKECwcHBVoelLKRlGaVs5IjlmKSkJGbMmEGlSpWYOXMmAE899ZQmdqXJXSln9fvvv9OwYUN69OhBWFgYzZs3tzok5UA0uSvlhObNm8fDDz/M7t27mT9/PqtXr8bHx8fqsJQD0Zq7Utlw1AupFSpUoHnz5kybNo3SpUtbHY5yQJrclcqGo1xIvX79Oh9++CEAw4cP142+VI40uSuVCUdaifrrr78SERHB/v37efXVVxERjDGWxaOcg9bclcqEI4zYr1y5Qu/evalbty5Xr15l5cqVzJs3TxO7solNyd0Y86Qx5oAxJtoY0y+LNs8ZY6KMMXuNMQvtG6ZSec/qqY9Hjx5l1qxZvP766+zZs0ePvVN3JMeyjDHGA5gGNAHigK3GmBUiEpWuTQDQH6gjIheMMSVzK2ClXNmFCxf4+uuv6dq1K8HBwcTExFCmTBmrw1JOyJaae00gWkRiAIwxi4HWQFS6Nl2AaSJyAUBEztg7UKVyi6Pszb5s2TJ69OjB2bNnqV+/PhUrVtTEru6aLcndCziW7n4cEJahTSCAMWYD4AEMEZGVGV/IGNMV6ArJZzYqlVsyS9hZ2Rx7HoAwn6Jpj+Vlrf3UqVP06tWLpUuXUq1aNb7//nsqVqyYJ++tXJctyT2zqzcZz+XyBAKABoA38LMxJkRELt7yTSKzgdkAoaGheraXyjV3Mjc9zKcorat5WVJbT0xM5PHHH+fYsWN89NFHvPPOO7rRl7ILW5J7HFA23X1v4EQmbTaJyE0g1hhzgORkv9UuUSqVg4wjdUeYwpiduLg4ypQpg4eHB1OmTMHHx0e35VV2Zctsma1AgDHGxxiTHwgHVmRo8x3QEMAYU5zkMk2MPQNVKjsZj72zetFRVpKSkpg6dSqVKlVixowZADRv3lwTu7K7HEfuIpJgjOkJrCK5nj5fRPYaY4YB20RkRcpzTY0xUUAi0FdE4nMzcKUycuSROsD+/fvp3LkzGzZsoFmzZrRs2dLqkJQLs2mFqohEApEZHhuc7rYAb6V8KZWrHGV2y52YO3cuPXv2pFChQixYsICOHTvqYiSVq3SFqnI6GUsw4LhlmFR+fn48/fTT7Nu3j06dOmliV7lO95ZRTsnRSzDXrl1j2LBhAHz00Uc0bNiQhg0bWhyVciea3JVDym6euqOXYDZs2EBERAQHDhygc+fOutGXsoQmd2WprJJ4ZguLUjlqCeaPP/5gwIABTJs2jfLly7Nq1SqaNm1qdVjKTWlyV3kms0SeVRK3cmHR3YqLi2Pu3Ln06tWLESNG8Pe//93qkJQb0+Su8kxmq0adMYmnFx8fz5IlS3jttdcICgoiJiZGT0ZSDkGTu8p1jnTwhb2ICN988w2vv/4658+fp1GjRlSsWFETu3IYmtyV3WUsv6QvvThirfxOnTx5ktdff51ly5ZRo0YNVq9erRt9KYejyV3ZXcbyi7OXXtJL3ejr+PHjjBkzhj59+uDpqf+MlOPRT6W6I7ZspetK5ZdUx44dw8vLCw8PD6ZNm4aPjw+BgYFWh6VUlnSFqrojma0OzchRpyrejcTERKZMmXLLRl/NmjXTxK4cno7c1R1ztVF5Vvbt20dERAQbN26kefPmPP3001aHpJTNNLmrLDnjBl32Mnv2bHr16kXhwoX5/PPP6dChg64yVU5FyzIqS864QZe9BAQE0LZtW6KionjxxRc1sSunoyN3lS13KcH873//Y8iQIRhjGDVqlG70pZyeJneVJquj6lzd+vXr6dy5MwcPHqR79+660ZdyCZrcVVpSz7jPi6uXYC5fvky/fv2YMWMGvr6+/PjjjzRq1MjqsJSyC03uLs6WeekZV5C6wmIjW5w4cYJPP/2Ut956i2HDhnH//fdbHZJSdqPJ3cVltllXRu6U1M+dO8eSJUvo0aMHlSpVIjY2llKlSlkdllJ2p8ndDbjLRdHsiAhLliyhV69eXLx4kSeeeILAwEBN7Mpl6VRIF7Vw81Gen7Uxx9Wk7uDEiRO0adOG8PBwypcvz2+//aYrTJXL05G7i0pfjnHli6I5SUxMpF69ehw/fpxx48bRu3dv3ehLuQX9lLuA7FaSums55siRI3h7e+Ph4cH06dPx9fXF39/f6rCUyjNalnEB7rySNKPExEQmTJhAUFBQ2kZfTZs21cSu3I6O3F2EO4/SU+3Zs4eIiAi2bNlCy5YtadOmjdUhKWUZTe5OyF1XkmZn5syZvPHGGxQpUoSFCxcSHh6uq0yVW9Pk7iTSJ3R3W0mandStAoKCgmjXrh2TJk2iRIkSVoellOU0uTuJ9LNf3GnRUVauXr3K4MGD8fDwYPTo0dSvX5/69etbHZZSDkOTuxPRunqydevW0blzZw4dOkSPHj10oy+lMqGzZRycLkb6y6VLl+jWrVvaVrxr1qxh2rRpmtiVyoRNyd0Y86Qx5oAxJtoY0y+bds8aY8QYE2q/EN2bLkb6y8mTJ/niiy9455132L17t+63rlQ2cizLGGM8gGlAEyAO2GqMWSEiURnaFQbeADbnRqDuzJ3LMWfPnmXx4sX06tWLSpUqcfjwYb1gqpQNbBm51wSiRSRGRG4Ai4HWmbT7EBgDXLNjfMpNiQgLFy4kKCiIt99+m99//x1AE7tSNrIluXsBx9Ldj0t5LI0x5hGgrIj8nx1jU27q2LFjPP3003To0AF/f3927NihG30pdYdsmS2T2dUqSXvSmPuAicDLOb6QMV2BrgDlyrnvND6VtYSEBBo0aMCpU6eYOHEivXr1wsPDw+qwlHI6tiT3OKBsuvvewIl09wsDIcC6lFkLDwErjDGtRGRb+hcSkdnAbIDQ0FBBZSl10ZK7rD49fPgwZcuWxdPTk1mzZuHr64uvr6/VYSnltGwpy2wFAowxPsaY/EA4sCL1SRG5JCLFRaSCiFQANgG3JXZlm9SpjwOW/ZfNseddfpZMQkIC48aNIygoiOnTpwPwxBNPaGJX6h7lOHIXkQRjTE9gFeABzBeRvcaYYcA2EVmR/SsoW2R2SLWrr0LdvXs3ERERbNu2jdatW/PMM89YHZJSLsOmFaoiEglEZnhscBZtG9x7WO4ntQTjDkkdYPr06fTu3ZsHH3yQr776inbt2uliJKXsSLcfcCDuMJ89dauAkJAQwsPDmThxIsWLF7c6LKVcjib3XJTZCUlZcfULp3/++Sfvv/8+np6ejB07lnr16lGvXj2rw1LKZeneMrkosxOSsuLKF05//PFHqlSpwqRJk7h+/ToiOlFKqdymI/dc5g6llqxcvHiRd955h3nz5hEQEMD69et5/PHHrQ5LKbegI3eVa06fPs3ixYt577332LVrlyZ2pfKQjtyVXaUm9N69e1OxYkUOHz6sF0yVsoCO3JVdiAhffPEFwcHBvPvuuxw8eBBAE7tSFtHkru7Z0aNHadEgtDmBAAAODElEQVSiBR07dqRixYrs3LmTgIAAq8NSyq1pWUbdk9SNvs6cOcOUKVPo0aOHbvSllAPQ5K7uSkxMDOXLl8fT05M5c+bg5+dHhQoVrA5LKZVCyzJ2lrrxl6uee5qQkMDo0aMJDg5m2rRpADRu3FgTu1IORpO7naVfuORqC5N27txJWFgY/fr146mnnqJdu3ZWh6SUyoKWZXKBKy5c+vjjj+nTpw/FihVj6dKluoOjUg5OR+4qW6lbBVStWpUOHToQFRWliV0pJ6Ajd5WpK1euMHDgQPLly8e4ceN0oy+lnIyO3NVtVq9eTUhICFOnTuXmzZu60ZdSTkiTu0pz4cIFXnnlFZo1a0bBggVZv349kydP1kM0lHJCmtxVmjNnzrB06VL69+/Pzp07qVu3rtUhKaXuktbc3dypU6dYtGgRffr0Sdvoq1ixYlaHpZS6Rzpyd1MiwoIFCwgODqZ///5pG31pYlfKNejI/R5lPErPGY7LO3z4MN26dWP16tXUqVOHuXPn6kZfSrkYTe53ILMzUTfHngcgzKco4PirUhMSEmjYsCHnzp1j2rRpdO/enfvu0z/glHI1mtzvQOrWAulH5mE+RWldzYv2YeUsjCxn0dHR+Pj44Onpyfz58/H19aV8+fJWh6WUyiWa3O+Qs20tcPPmTcaOHcvQoUMZO3Ysb7zxBg0bNrQ6LKVULtPkng1nrKent337diIiIti5cyft2rXj+eeftzokpVQe0WJrNtLv8AiOX09Pb8qUKdSsWZNTp07x7bffsmTJEkqVKmV1WEqpPKIj90ykjthTR+rOVIYREYwxPPLII3Tq1Inx48fz4IMPWh2WUiqPaXLPRPrE7iwj9T/++IP+/ftToEABxo8fz+OPP87jjz9udVhKKYtoWSYLqSN2R58FA7By5UpCQkKYPn06IqIbfSmlNLk7s/j4eF566SWaN2/O/fffz4YNG5gwYYJu9KWU0uSeXur5p85y9ml8fDzLli1j0KBB7Nixg9q1nefagFIqd9mU3I0xTxpjDhhjoo0x/TJ5/i1jTJQxZrcx5kdjjFOujnGGWvvJkycZN24cIkJgYCBHjhxh2LBhFChQwOrQlFIOJMfkbozxAKYBzYFg4AVjTHCGZjuAUBGpCiwFxtg70LziqLV2EWH+/PkEBQUxaNAgoqOjAXQmjFIqU7aM3GsC0SISIyI3gMVA6/QNRGStiFxNubsJ8LZvmO4tNjaWpk2bEhERwcMPP8yuXbt0oy+lVLZsmQrpBRxLdz8OCMumfQTwQ2ZPGGO6Al0BypVzrJGxo0pISKBRo0bEx8czY8YMunbtqht9KaVyZEtyz2zqRaZz7YwxLwKhQP3MnheR2cBsgNDQUJ2vl42DBw/i6+uLp6cnn3zyCX5+fpQtW9bqsJRSTsKWIWAckD6reAMnMjYyxjwBDARaich1+4Tnfm7evMnw4cMJCQnh448/BqBBgwaa2JVSd8SWkftWIMAY4wMcB8KB9ukbGGMeAWYBT4rIGbtH6Sa2bdtGREQEu3fvJjw8nBdeeMHqkJRSTirHkbuIJAA9gVXAPmCJiOw1xgwzxrRKaTYW+DvwtTFmpzFmRa5F7KImT55MWFgY586dY/ny5SxatIiSJUtaHZZSyknZtLeMiEQCkRkeG5zu9hN2jsttpG70FRoaSkREBGPGjOEf//iH1WEppZycbhxmkcuXL/Pee+9RsGBBJk6cSJ06dahTp47VYSmlXITOqbNAZGQklStXZvbs2Xh6eupGX0opu9PknofOnTvHiy++SIsWLShSpAi//vorY8eO1Y2+lFJ2p8k9D124cIF//etffPDBB2zfvp2wsOzWgiml1N3TmnsuO378OF9++SV9+/YlICCAI0eO6AVTpVSu05F7LhER5syZQ3BwMEOGDOHQoUMAmtiVUnnC7UbuqeejZiZ1u997dejQIbp06cLatWtp0KABc+bMwd/f/55fVymlbOXSyT2zRL459jwAYT5Fb2tvj33cExISaNy4MefPn2fWrFl07txZN/pSSuU5l07u6Q/fSBXmU5TW1bzsvl/7gQMH8PPzw9PTkwULFuDn54e3t+58rJSyhksnd/jr8I3ccuPGDUaOHMmIESMYO3YsvXv3pn79TDfFVEqpPOPyyT03bdmyhYiICPbs2UP79u3p0KGD1SEppRSgs2Xu2qRJk6hdu3ba3PUvv/yS4sWLWx2WUkoBmtzvWOpWATVr1qRLly7s3buXli1bWhyVUkrdSssyNrp06RLvvvsuf/vb35g0aRKPPfYYjz32mNVhKaVUpnTkboN//etfBAcHM3fuXAoUKKAbfSmlHJ4m92ycPXuW9u3b06pVK4oVK8amTZsYPXq0bvSllHJ4mtyzcenSJSIjIxk6dCjbtm3j0UcftTokpZSyidbcMzh27BhffPEF/fr1w9/fnyNHjlCkSBGrw1JKqTuiI/cUSUlJzJw5k8qVKzN8+PC0jb40sSulnJFLJveFm4/y/KyNRJ28bFP7gwcP0qhRI1577TVq1qzJf//7X93oSynl1FyyLJN+T5mcNgJLSEigSZMmXLx4kXnz5vHKK6/oBVOllNNzieSecffH1MSe3Z4y+/btIyAgAE9PTz7//HP8/PwoU6ZMXoSrlFK5ziXKMqkj9VTZjdivX7/OBx98QNWqVfn4448BePzxxzWxK6VcikuM3MG23R83bdpEREQEUVFRdOzYkY4dO+ZRdEoplbecLrlndgCHLScojR8/nr59++Lt7U1kZCTNmzfPzTCVUspSTleWyViCgezLMElJSQDUrl2b7t27s2fPHk3sSimX53Qjd7CtBHPx4kXefvttChUqxNSpU3WjL6WUW3G6kbstvvvuO4KDg1mwYAGFCxfWjb6UUm7HpZL7mTNneO6552jbti2lSpViy5YtfPTRRzpvXSnldlwquV++fJl///vfjBgxgi1btlC9enWrQ1JKKUvYlNyNMU8aYw4YY6KNMf0yeb6AMearlOc3G2Mq2DvQrBw9epQRI0YgIvj7+3P06FEGDBhAvnz58ioEpZRyODkmd2OMBzANaA4EAy8YY4IzNIsALoiIPzARGG3vQDNKSkpi+vTpVK5cmY8++ihto6/ChQvn9lsrpZTDs2XkXhOIFpEYEbkBLAZaZ2jTGliQcnsp0NjkUqE7uMwDlEyKp0GDBrz++uvUrl2bvXv36kZfSimVji1TIb2AY+nuxwFhWbURkQRjzCWgGHDOHkGmN7B5Rfz9W3Dp0iU++eQTXnrpJb1gqpRSGdiS3DPLnBnnFtrSBmNMV6ArQLly5Wx469t5enryxRdf4OfnR+nSpe/qNZRSytXZUpaJA8qmu+8NnMiqjTHGEygCnM/4QiIyW0RCRSS0RIkSdxcxULduXU3sSimVDVuS+1YgwBjjY4zJD4QDKzK0WQG8lHL7WWCN6MohpZSyTI5lmZQaek9gFeABzBeRvcaYYcA2EVkBzAM+N8ZEkzxiD8/NoJVSSmXPpr1lRCQSiMzw2OB0t68B7ewbmlJKqbvlUitUlVJKJdPkrpRSLkiTu1JKuSBN7kop5YI0uSullAsyVk1HN8acBY7c5bcXJxe2NnBw2mf3oH12D/fS5/IikuMqUMuS+70wxmwTkVCr48hL2mf3oH12D3nRZy3LKKWUC9LkrpRSLshZk/tsqwOwgPbZPWif3UOu99kpa+5KKaWy56wjd6WUUtlw6OTuyAdz5xYb+vyWMSbKGLPbGPOjMaa8FXHaU059TtfuWWOMGGOcfmaFLX02xjyX8rvea4xZmNcx2psNn+1yxpi1xpgdKZ/vp6yI016MMfONMWeMMXuyeN4YY6ak/Dx2G2Oq2zUAEXHIL5K3Fz4E+AL5gV1AcIY2PYCZKbfDga+sjjsP+twQKJRy+zV36HNKu8LAemATEGp13Hnwew4AdgAPptwvaXXcedDn2cBrKbeDgcNWx32Pfa4HVAf2ZPH8U8APJJ9kVwvYbM/3d+SRu0MdzJ1HcuyziKwVkaspdzeRfDKWM7Pl9wzwITAGuJaXweUSW/rcBZgmIhcARORMHsdob7b0WYAHUm4X4fYT35yKiKwnkxPp0mkNfCbJNgH/MMbY7Yg5R07umR3M7ZVVGxFJAFIP5nZWtvQ5vQiS/+d3Zjn22RjzCFBWRP4vLwPLRbb8ngOBQGPMBmPMJmPMk3kWXe6wpc9DgBeNMXEknx/RK29Cs8yd/nu/IzYd1mERux3M7URs7o8x5kUgFKifqxHlvmz7bIy5D5gIvJxXAeUBW37PniSXZhqQ/NfZz8aYEBG5mMux5RZb+vwC8KmIjDfG1Cb5dLcQEUnK/fAskav5y5FH7nY7mNuJ2NJnjDFPAAOBViJyPY9iyy059bkwEAKsM8YcJrk2ucLJL6ra+tleLiI3RSQWOEBysndWtvQ5AlgCICIbgYIk78Hiqmz69363HDm5u+PB3Dn2OaVEMYvkxO7sdVjIoc8icklEiotIBRGpQPJ1hlYiss2acO3Cls/2dyRfPMcYU5zkMk1MnkZpX7b0+SjQGMAYE0Rycj+bp1HmrRVAp5RZM7WASyJy0m6vbvUV5RyuNj8F/E7yVfaBKY8NI/kfNyT/8r8GooEtgK/VMedBn/8DnAZ2pnytsDrm3O5zhrbrcPLZMjb+ng0wAYgC/guEWx1zHvQ5GNhA8kyanUBTq2O+x/4uAk4CN0kepUcA3YHu6X7H01J+Hv+19+daV6gqpZQLcuSyjFJKqbukyV0ppVyQJnellHJBmtyVUsoFaXJXSikXpMldKaVckCZ3pZRyQZrclVLKBf0/HsZ2uKLkygAAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def isDemocrat(entity):\n",
" return entity.endswith('(R)')\n",
"\n",
"from sklearn.svm import SVC\n",
"svm_clf = SVC(C=2**14, kernel='linear')\n",
"\n",
"from sklearn.model_selection import cross_val_score \n",
"scoresw = cross_val_score(svm_clf,i1_ut,y,cv=kf)\n",
"print(\"Mean cross-validation accuracy: {:.2f}\".format(np.mean(scoresw)))\n",
"\n",
"ypred = cross_val_predict(svm_clf, i1_ut, y, cv=kf)\n",
"print(classification_report(y, ypred, digits=10))\n",
"conf_mx = confusion_matrix(y, ypred, labels=[True, False])\n",
"plt.matshow(conf_mx,cmap=plt.cm.Blues)\n",
"plt.xticks(np.arange(2), ['D', 'R'])\n",
"plt.yticks(np.arange(2), ['D', 'R'])\n",
"plt.savefig('figures/classification/ultrametric_tfidf_confmx.pdf')\n",
"plt.show()\n",
"\n",
"from sklearn.metrics import roc_curve\n",
"from sklearn.metrics import roc_auc_score\n",
"\n",
"yw_scores = cross_val_predict(svm_clf, i1_ut, y, cv=kf, method=\"decision_function\")\n",
"fpr, tpr, thresholds = roc_curve(y, yw_scores)\n",
"\n",
"fpr, tpr, thresh = roc_curve(y, yw_scores)\n",
"auc = roc_auc_score(y, yw_scores)\n",
"plt.plot(fpr,tpr,label=\"AUC = \"+repr(auc))\n",
"\n",
"plt.plot([0,1],[0,1],'k--')\n",
"\n",
"plt.legend(loc=0)\n",
"plt.savefig('figures/classification/ultrametric_tfidf_roc.pdf')\n",
"from sklearn.metrics import f1_score\n",
"u_f1 = f1_score(y, ypred, average='macro') \n",
"print(f\"F1 Score: {u_f1}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Classification Task using METRIC Closure as generality (theoretical framework)\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Mean cross-validation accuracy: 0.61\n",
" precision recall f1-score support\n",
"\n",
" False 0.5892857143 0.7226277372 0.6491803279 137\n",
" True 0.6448598131 0.5000000000 0.5632653061 138\n",
"\n",
"avg / total 0.6171738075 0.6109090909 0.6060666079 275\n",
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAECCAYAAAAxcwG5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAABh1JREFUeJzt3LGrXvUdx/HPryZioNpGvBkUKrjqkCH9C6RLoVAoFBfXLIqT0FE6dikODuImhQ6FLo61u0vAiHRVurSQqFE7xILx20FLiyi590nOPcbP67WFnHv4kHPfnPM897lZMxOgzw/2HgDsQ/xQSvxQSvxQSvxQSvxQ6szeA+4la61bSd5NcjbJ50leT/LyzHyx6zCO5f+u35kk7yd5dmY+3nfVftz5T+bmzFycmSeT/CzJz5O8tPMmju+/1++pJB8leW7vQXsS/4Fm5lqSy0meX2utvfdwYm8leWzvEXsS/x2Ymffy5b/hhb23cHxrrfuSPJ3kjb237En8d85d/95xbq11NcmHSR5O8ubOe3Yl/juw1noiya0k1/bewrHcnJmLSR5Pcn+85ucQa62jJK8meWX8dtQ9ZWY+SfJCkhfXWmf33rMX8Z/MubXW1bXW35L8Nclfkvx2500cYGbeTvJOkmf23rKX5aYFndz5oZT4oZT4oZT4oZT4oZT4D7TWurz3Bg7j2n1J/IfzDXTvcu0ifqi12Yd8Hnjo/Dx49Ogm5/4u+OzTG3ngofN7z+AA3/dr96/r/8hnn9647S+cbfY/+Tx49Gh+9bs/bXV64Fv8+Te/PtZxHvuhlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPihlPih1JnbHbDWupXk3SRnk3ye5PUkL8/MFxtvAzZ02/iT3JyZi0my1rqQ5I9JfpTkpS2HAds60WP/zFxLcjnJ82uttc0k4DSc+DX/zLz31ddduPtzgNNy6Bt+33jXX2tdXmtdWWtd+ezTG3cwC9jaieNfaz2R5FaSa1//u5l5bWYuzcylBx46fzf2ARs5UfxrraMkryZ5ZWZmm0nAaTjOu/3n1lpX878f9f0hye83XQVs7rbxz8x9pzEEOF0+4QelxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lxA+lzmx14p/8+Fxe/uWTW52ejZ3/6fN7T+BA//7n9WMd584PpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpcQPpc4c56C11q0k7351/PtJnp2Zj7ccBmzruHf+mzNzcWaeSvJRkuc23AScgkMe+99K8tjdHgKcrhPFv9a6L8nTSd7YZg5wWo4b/7m11tUkHyZ5OMmb33TQWuvyWuvKWuvK9Q+u362NwAZO9Jo/yeNJ7s+3vOafmddm5tLMXDp65OhubQQ2cKLH/pn5JMkLSV5ca53dZhJwGk78ht/MvJ3knSTP3P05wGk51s/5Z+aHX/vzL7aZA5wWn/CDUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUuKHUmtmtjnxWteT/H2Tk383PJLkg71HcJDv+7V7fGaObnfQZvF/3621rszMpb13cHKu3Zc89kMp8UMp8R/utb0HcDDXLl7zQy13figlfiglfiglfiglfij1H2MQ69FXGAS7AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 288x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"F1 Score: 0.6062228169956507\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd0VOX28PHvNlG4KPKTogKhk0AKASFSVECKFAU7xYKgAS7FBhZQEMuriAiiIFVAxAIognCv2K7lgihipF1ASugJKL2pQMp+/5jJOISUCZmSyezPWllkzpw5Z59JyJ6nnP2IqmKMMcYAXBDoAIwxxhQdlhSMMca4WFIwxhjjYknBGGOMiyUFY4wxLpYUjDHGuFhSMMYY42JJwRhjjIslBWOMMS7hgQ6goMqXL6/Vq1cPdBjGGBNUfvnll4OqWiG//YIuKVSvXp2kpKRAh2GMMUFFRHZ5sp91HxljjHGxpGCMMcbFkoIxxhiXoBtTyElaWhopKSmcOnUq0KEY4zUlS5YkIiKCCy+8MNChmBBSLJJCSkoKpUuXpnr16ohIoMMxptBUlUOHDpGSkkKNGjUCHY4JIT7rPhKRmSKyX0TW5/K8iMh4EUkWkXUi0vB8z3Xq1CnKlStnCcEUGyJCuXLlrPVr/M6XYwqzgA55PN8RiHR+9QUmF+ZklhBMcWO/0yYQfNZ9pKpLRaR6HrvcAsxWx3qgK0Tk/0Skoqru81VMxhgTTD74aTeL1qSSfvovTp84ytXxdXi2c6xPzxnI2UeVgT1uj1Oc284hIn1FJElEkg4cOOCX4M7HwoULERE2bdrk2vbdd9/RqVOns/br1asX8+fPBxyD5EOHDiUyMpK4uDgaN27MZ599VuhYXn75ZWrXrk2dOnX44osvctxHVRk2bBhRUVFER0czfvx4ABYtWkR8fDwNGjQgISGB77//3vWasLAwGjRoQIMGDbj55ptd2xMTE6lfvz7x8fHceeednDx5EoBZs2ZRoUIF12umT58OwK5du2jUqBENGjQgNjaWKVOmuI41Z84c6tWrR3x8PB06dODgwYMAPPfcc1SuXNl1rCVLlgCwc+dO/vGPf7i29+vXz3WsefPmER8fT2xsLE8++aRr+9KlS2nYsCHh4eGun0WWJ598ktjYWKKjo3n44YdRVU6cOOE6foMGDShfvjyPPvponteY1/v1zTff0LBhQ+Li4ujZsyfp6en5/kxN6Fm0JpWVPyzli//Xg+VTnyIzM9P3J1VVn30B1YH1uTz3KXCd2+OvgUb5HbNRo0aa3caNG8/ZFghdunTR6667Tp999lnXtm+//VZvuumms/br2bOnfvTRR6qqOmTIEL3vvvv01KlTqqr622+/6bx58woVx4YNGzQ+Pl5PnTql27dv15o1a2p6evo5+82cOVN79OihGRkZqqr6+++/q6rqiRMnNDMzU1VV165dq3Xq1HG95uKLL87xnMeOHXN9P2jQIH355ZdVVfXtt9/WgQMHnrP/6dOnXdd84sQJrVatmqampmpaWppWqFBBDxw4oKqqTzzxhOv9fPbZZ/XVV18951g7duzQ2NjYc7YfPHhQq1Spovv371dV1fvuu0//85//uF6zdu1a7dGjh+tnoaq6fPlyveaaazQ9PV3T09O1adOm+u23355z7IYNG+p///vfPK9RNef3KyMjQyMiInTz5s2qqvrMM8/o9OnTc3x9UfndNv71/opdeutrX+hlV3VQQGvXrq3fffddoY4JJKkHf7cD2VJIAaq4PY4A9gYolkI7efIky5cvZ8aMGcydO9ej1/z555+89dZbTJgwgRIlSgBwxRVX0LVr10LFsmjRIrp3706JEiWoUaMGtWvXZuXKlefsN3nyZEaMGMEFFzh+DS6//HIALrnkEld/9h9//OFR3/all14KOD5k/PXXX/m+5qKLLnJd8+nTp12fgLJ+Mf/44w9UlePHj1OpUiUPr/xs27dvJyoqigoVHOVe2rZty8cffww4yqXEx8e7rj2LiHDq1CnOnDnD6dOnSUtL44orrjhrn61bt7J//36aN29+XnEdOnSIEiVKEBUVBcANN9zgissYgIWrdvPZyD4cXfMlne7tx7p162jZsqVfzh3IKamLgQdFZC7QBDimXhhPeP5fG9i493ihg3MXU+nSfPvxPvnkEzp06EBUVBRly5Zl1apVNGyY94Sq5ORkqlat6vqDmpdBgwbx7bffnrO9e/fuDB069KxtqampNG3a1PU4IiKC1NTUc167bds25s2bx8KFC6lQoQLjx48nMjIScHSFPfXUU+zfv59PP/3U9ZpTp06RkJBAeHg4Q4cO5dZbb3U9d//997NkyRJiYmIYO3asa/vHH3/M0qVLiYqKYty4cVSp4vgssGfPHm666SaSk5N59dVXXX/8J0+eTL169bj44ouJjIxk4sSJrmO9+eabzJ49m4SEBMaOHctll10GwI4dO7jqqqu49NJLefHFF2nevDm1a9dm06ZN7Ny5k4iICD755BPOnDmT5/vcrFkzWrVqRcWKFVFVHnzwQaKjo8/aZ86cOXTr1u2sxJfbNeb0fpUvX560tDSSkpJISEhg/vz57NmzB2MOHTpE2bJlueCCMBrd0Z83erclISHBrzH4ckrqHOBHoI6IpIhIooj0E5GsDt8lwHYgGXgLGOCrWPxhzpw5dO/eHXD8oZ4zZw6Q+wySgs4sGTduHGvWrDnnK3tCALK64/I93+nTpylZsiRJSUn06dOHBx54wPXcbbfdxqZNm/jkk0945plnXNt3795NUlISH3zwAY8++ijbtm1zPff222+zd+9eoqOjmTdvHgCdO3dm586drFu3jrZt29KzZ0/X/lWqVGHdunUkJyfzzjvv8Pvvv5OWlsbkyZNZvXo1e/fuJT4+npdffhmA/v37s23bNtasWUPFihV57LHHAKhYsSK7d+9m9erVvPbaa9x9990cP36cyy67jMmTJ9OtWzeaN29O9erVCQ/P+3NQcnIyv/76KykpKaSmpvLNN9+wdOnSs/aZO3cud911l+txXteY0/slIsydO5dBgwbRuHFjSpcunW9cpnj64KfddJv6I12n/EDTB56lUrWaXN3jKTbuO07EVS39nhDAt7OP7srneQUGevu8vh6Zz8mhQ4f45ptvWL9+PSJCRkYGIsLo0aMpV64cR44cOWv/w4cPU758eWrXrs3u3bs5ceIEpUuXzvMcBWkpREREnPXJMyUlJccumIiICO644w7AkQTuv//+c/Zp0aIF27Zt4+DBg5QvX951nJo1a3L99dezevVqatWq5do/LCyMbt268eqrr3L//fdTrlw513N9+vRhyJAh55yjUqVKxMbGsmzZMqpVqwbgOmbXrl0ZNWoUwFndOH369HEN4JcoUcLVFdWoUSNq1arFli1bSEhIoHPnznTu3BmAadOmERYWds753S1cuJCmTZtyySWXANCxY0dWrFhBixYtAFi7di3p6ek0atTI9Zq8rjG396tZs2YsW7YMgC+//JItW7bkGZcpnhatSWXNpm2c+M8k9q3/kXI1YilfK54yFS/llgY5zrvxPU8GHorSV1EcaJ4yZYr27dv3rG0tWrTQpUuX6qlTp7R69equGHfu3KlVq1bVo0ePqqpjILVXr156+vRpVVXdu3evvvvuu4WKZ/369WcNNNeoUSPHgeYhQ4bojBkzVNUxIJ6QkKCqqlu3bnUNNP/yyy9aqVIlzczM1MOHD7sGhw8cOKC1a9fWDRs2aGZmpm7dulVVVTMzM/Wxxx7Txx57zHU9WRYsWKBNmjRRVdU9e/bon3/+qaqqhw8f1sjISF23bp2mpqbqlVde6RocHj58uA4ePPicY7322mvarVs3VVXdv3+/6/q2bdumlSpV0kOHDqnq34Pnhw8f1vr167sGd7O4D/qrqs6dO1fbtGmjaWlpeubMGW3durUuXrz4rPdsxIgRZx0jt2vM7f1yj+vUqVPaunVr/frrr8/5+agG/nfbeM/7K3Zp1yk/nPUVcfsQDS9ZSkuVKqWvv/56jv9PvQUPB5oD/ke+oF9FMSm0bNlSP/vss7O2vfHGG9qvXz9VVf3++++1SZMmWr9+fU1ISNAvv/zStd/p06f1iSee0Fq1amlsbKw2btxYP//880LH9OKLL2rNmjU1KipKlyxZ4tresWNHTU1NVVXVI0eO6I033qhxcXHatGlTXbNmjaqqjho1SmNiYrR+/fratGlTXbZsmao6ZubExcVpfHy8xsXFuWbMZGRk6DXXXKNxcXEaGxurd999t2s20tChQzUmJkbj4+P1+uuv119//VVVVb/88kutV6+exsfHa7169XTq1KmuGCdPnqx169bVevXqaadOnfTgwYOqqnrvvfdqXFyc1qtXTzt37uz6Yzx//nzXOa666qqz/oh3795do6OjNTo6WufMmePavnLlSq1cubKWKlVKy5YtqzExMaqqmp6ern379tW6detqdHS0Dho06Kz3tUaNGq5ryJLbNeb2fqmqPv7441q3bl2NiorScePG5fpzDPTvtvGerlN+0LhnPz8rKTR/6DWNu/o63b59u8/P72lSEM2h/7koS0hI0OyL7Pz666/nDAYaUxzY77b/Zd0w5m0b9x2n7uWlaPzHCs6cOcOwYcMAxwdzf9y9LiK/qGq+gxQ2umWMMfydDH7acRiAJjXKevX4lTJ+55cJY5i/6X907drVlQyKWjkTSwrGGINj0HfjvuM0qVGWWxpU5u4mVb1y3NOnT/Piiy8yatQoypYty0cffcQdd9xR5JJBlmKTFPzVBDPGX4KtazcYuXcVbdx3nJiKlzLvn828eo6tW7fyyiuvcPfdd/Paa6+dNVutKCoWSaFkyZIcOnTIymebYkPVsZ5CyZIlAx1KkVeYMQD3rqIYL04DPXnyJIsWLeKee+4hLi6OTZs2UbNmTa8c29eKRVKIiIggJSWFolwsz5iCylp5zeQtq9snpmL+lQGy83ZXEcBXX31F37592bVrFw0bNiQ6OjpoEgIUk6Rw4YUX2upUxoQwX3T7FNSRI0d4/PHHmTlzJlFRUfz3v/8NypljxSIpGGNMIGVkZHDttdeyZcsWnnrqKUaMGBG0XX+WFIwx5jwdPHiQsmXLEhYWxsiRI6latWq+hTCLukCWzjbGmKCkqsyePZuoqCjXokq33npr0CcEsKRgjDEFsmvXLjp27EjPnj2Jjo52FUssLiwpGGOCUlbZ6Y37vLt+Sl7ee+894uLi+P7775kwYQLLli2jbt26fju/P9iYgjEmKLlPRfVXmekKFSpw7bXXMnXqVFeZ9+LGkoIxpsjK68Y0X92B7C4tLY2xY8eSlpbGM888Q/v27WnXrl2xvknWkoIxpsjxpDidr1sIq1evJjExkdWrV9O9e/ciW8DO2ywpGGOKHF8Vp/PEqVOneOGFFxg9ejTly5fn448/5vbbb/fb+QPNkoIxpkgK1F3KycnJjBkzhvvuu4+xY8dy2WWX+T2GQLKkYIwJeSdPnmThwoX06NGDuLg4Nm/eHLKlcywpGGMCwpNBZH/44osv6Nu3L3v27CEhIYHo6OiQTQhg9ykYYwIka9wgJ/6YZnro0CF69uxJhw4dKFWqFMuWLQvKAnbeZi0FY0zABGrcIKuAXXJyMsOGDWP48OFBW8DO2ywpGGNCxoEDByhXrhxhYWG88sorVKtWjQYNGgQ6rCLFuo+MMcWeqvL2228TFRXFW2+9BcAtt9xiCSEHlhSMMX7l75pFO3fupH379jzwwAPUq1ePVq1a+eW8wcq6j4wxPpPTDCP3u5R9PZj87rvv0r9/f0SESZMm8c9//pMLLrDPwnmxpGCM8Zmc1k/2513KV1xxBS1atGDKlClUreq/u6KDmSUFY4xP+XOGUVpaGqNHjyYjI4MRI0bQrl072rVr55dzFxfWjjLGFAurVq3i6quvZvjw4WzevBlVDXRIQcmnSUFEOojIZhFJFpGhOTxfVUS+FZHVIrJORG70ZTzGmOLnr7/+YujQoTRu3Jjff/+dhQsX8v777xf7aqa+4rOkICJhwESgIxAD3CUiMdl2Gw58qKpXAd2BSb6KxxhTPG3fvp3XXnuNXr16sXHjRm699dZAhxTUfNlSaAwkq+p2VT0DzAVuybaPAlkjUGWAvT6MxxjjJ76ednr8+HFmzZoFQGxsLFu3bmX69OkhV9HUF3w50FwZ2OP2OAVokm2f54AvReQh4GKgrQ/jMcb4WE6L43h72umSJUvo168fqampNGnShOjo6GK7NGYg+DIp5NShl33k5y5glqqOFZFmwLsiEqeqmWcdSKQv0BewaWXGFGG+XBzn4MGDDBo0iPfee4+YmBiWL19uBex8wJdJIQWo4vY4gnO7hxKBDgCq+qOIlATKA/vdd1LVacA0gISEBJtSYEwR5ospqFkF7LZv386IESN4+umnKVGihFfPYRx8mRR+BiJFpAaQimMg+e5s++wG2gCzRCQaKAkc8GFMxpgg8vvvv1OhQgXCwsIYM2YM1apVIz4+PtBhFWs+G2hW1XTgQeAL4Fccs4w2iMgLInKzc7fHgD4ishaYA/RSm1xsTMhTVWbMmEGdOnWYNm0aAJ07d7aE4Ac+vaNZVZcAS7JtG+H2/UbgWl/GYIwJLtu3b6dPnz588803tGzZkrZtbf6JP1mZC2OMR/JaPjNLYZfRfOeddxgwYABhYWFMmTKFPn36WAE7P7N32xjjkbyWz8xS2GU0K1WqROvWrdm4caNVNA0QaykYYzzm7ZlFZ86cYdSoUWRmZvLcc89xww03cMMNN3jt+KbgLCkYY3KUvbuosF1D2f3888888MADrF+/nh49eqCqVq+oCLC2mTEmR9m7iwrbNZTlzz//5PHHH6dp06YcOXKExYsXM3v2bEsIRYS1FIwxufLFjWg7duxgwoQJ9OnTh1deeYUyZcp49fimcCwpGGNc3LuMvNlddOzYMRYsWMD9999PbGwsycnJVKlSJf8XGr+zpGBMCMs+buBeyM5b3UWffvop//znP9m3bx/NmjWjbt26lhCKMEsKxoSw7Gsoe7OQ3YEDB3j00Uf54IMPiIuLY8GCBdStW7fQxzW+ZUnBmBDnqwJ21113HTt27OD5559n6NChXHTRRV49h/ENj5KCiFwEVFXVZB/HY4zxg6xuI29PM/3tt9+4/PLLCQsLY+zYsVSvXp24uDivHd/4Xr5TUkXkJuB/wFfOxw1EZKGvAzPG+I57QvDGuEFmZiZTp04lKiqKqVOnAtCpUydLCEHIk5bCCzhWTPsWQFXXiEhtn0ZljPGJ7C0Eb3QbJScn06dPH7777jtat25N+/btvRCpCRRPbl5LU9Wj2bZZeWtjgpC3Wwhvv/029erVY9WqVbz11lv85z//oWbNml6I1ASKJy2FX0WkK3CBc8GcR4AVvg3LGOMr3hxYrlq1Ku3bt2fixIlUruzdtZhNYHjSUngQaARkAguAUzgSgzEmxJw+fZrnnnuOESMcy6K0adOGTz75xBJCMeJJUmivqkNU9Srn11Cgo68DM8YULT/99BONGjXi+eefZ/fu3dgiicWTJ0lheA7bhnk7EGNM0fTHH38wePBgmjVrxrFjx/j3v//NrFmzrIBdMZXrmIKItAc6AJVF5DW3py7F0ZVkjAkBu3btYtKkSfTr149Ro0Zx6aXeu6/BFD15DTTvB9bjGEPY4Lb9BDDUl0EZY/LmydKYOfH0ZrWjR48yf/58evfuTUxMDMnJyURERJxPqCbI5JoUVHU1sFpE3lfVU36MyRiTTV6F6wrCk6moixYton///uzfv5/rrruOunXrWkIIIZ5MSa0sIi8BMUDJrI2qGuWzqIwxZ/Fl4bos+/fv5+GHH2bevHnEx8ezePFiK2AXgjxJCrOAF4ExOGYd3Y+NKRjjMzl1DXnzDuScZGRkcO2117J7925efPFFnnzySS688EKfnMsUbZ4khVKq+oWIjFHVbcBwEVnm68CMCRWedA156w7k7Pbu3cuVV15JWFgYb7zxBtWrVycmJsbr5zHBw5OkcFocc8+2iUg/IBW43LdhGRM6/NE1lF1WAbshQ4YwatQoBgwYwI033uiz85ng4UlSGARcAjwMvASUAR7wZVDGhBpfdg1lt2XLFvr06cPSpUtp27YtHTvavajmb/kmBVX9yfntCaAHgIjYVARjgtCMGTN48MEHKVmyJDNnzqRXr152E5o5S55JQUSuBioD36vqQRGJBYYArQFLDMYUUF6DyP5QvXp1OnbsyMSJE6lYsaJfzmmCS65lLkTkZeB94B7gcxEZhmNNhbWATUc15jxkjR+489UgMjgK2A0fPpzhwx3Vatq0acOCBQssIZhc5dVSuAWor6p/iUhZYK/z8Wb/hGZM8MrtjmNfTy1198MPP5CYmMimTZt44IEHUFXrKjL5yispnFLVvwBU9bCIbLKEYEzespJBbncc+7JVkOXkyZMMGzaMCRMmUKVKFT7//HNbDc14LK+kUFNEFji/F6C622NU9fb8Di4iHYA3gDBguqqOymGfrsBzOFZzW6uqd3sevjFFS1b3kD+mleZm9+7dTJ06lYEDBzJy5EhKly7t9xhM8MorKdyR7fGbBTmwiIQBE4EbgBTgZxFZrKob3faJBJ4CrlXVIyJi9z+YoOfP6aVZjhw5wkcffUTfvn2JiYlh+/btVKpUya8xmOIhr4J4Xxfy2I2BZFXdDiAic3GMU2x026cPMFFVjzjPub+Q5zQm5CxcuJABAwZw4MABWrZsSZ06dSwhmPPmySI756sysMftcYpzm7soIEpElovICmd30zlEpK+IJIlI0oEDB3wUrjHB5bfffqNLly7cfvvtXHnllaxcuZI6deoEOiwT5Dy5o/l85TTNIfv6feFAJHA9jvselolInKoePetFqtOAaQAJCQm2BqAJeRkZGTRv3pw9e/YwcuRIHn/8cStgZ7zC46QgIiVU9XQBjp0CVHF7HIFjWmv2fVaoahqwQ0Q240gSPxfgPMb4hScL2/j6RrSUlBQqVapEWFgY48ePp0aNGlbe2nhVvt1HItJYRP4HbHU+ri8iEzw49s9ApIjUEJGLgO7A4mz7fAK0ch63PI7upO0FiN8Yn/rgp910m/oj3ab+yNML/+eaapobX005zczMZMKECdStW5fJkycD0LFjR0sIxus8aSmMBzrh+AOOqq4VkVb5vUhV00XkQeALHFNSZ6rqBhF5AUhS1cXO59qJyEYgA3hCVQ+d57UY43XuFUwDNc1006ZN9O7dm+XLl9O+fXs6derk1/Ob0OJJUrhAVXdluxMyw5ODq+oSYEm2bSPcvldgsPPLGK8633WM3fnzDuScTJ8+nQcffJBSpUrxzjvv0KNHD7sr2fiUJ0lhj4g0BtR578FDwBbfhmXM+XFPBOe7jrE7f9yBnJdatWrRuXNn3nzzTa644oqAxWFChzg+rOexg+OGsvFAW+em/wAPqupBH8eWo4SEBE1KSgrEqU0Q6Db1x7MGewN1V/H5OnXqFC+88AIAI0eODHA0pjgRkV9UNSG//TxpKaSrancvxGSM12XvIgp0d09hLF++nMTERDZv3kzv3r2tgJ0JCE+Sws/OqaLzgAWqesLHMRmTq/zWMw50d8/5OHHiBE8//TQTJ06kWrVqfPHFF7Rr1y7QYZkQ5cnKa7VE5BocU0qfF5E1wFxVnevz6Iwh73GCQBae85aUlBSmT5/OQw89xEsvvcQll1wS6JBMCMt3TOGsnR3rKrwO3KOqYT6LKg82phB6gn2cICeHDh3iww8/pH///gDs27fPFr4xPuW1MQURuQRHIbvuQDSwCLim0BEak4+sFkIwjxNkp6p8/PHHDBw4kMOHD9O6dWvq1KljCcEUGZ6MKawH/gWMVtVlPo7HhJi87iVw7yoKtnGCnOzbt4+BAweycOFCGjVqxJdffmkF7EyR40lSqKmqmT6PxIQk95ZAdsVhvCBLVgG71NRURo8ezaBBgwgP92U9SmPOT66/lSIyVlUfAz4WkXMGHjxZec2Y7IrTFFJP7Nmzh8qVKxMWFsbEiROpUaMGUVFRgQ7LmFzl9VFlnvPfAq24ZkxOclu7OBinkHoiIyODiRMn8tRTTzF69GgGDhxo6ySboJDXymsrnd9Gq+pZicFZ6K6wK7OZEFIU1i72l19//ZXExER+/PFHOnbsSOfOnQMdkjEe82TltQdy2Jbo7UBM8ZfVTVScE8K0adNo0KABW7Zs4d133+XTTz+latXie72m+MlrTKEbjmmoNURkgdtTpYGjOb/KFFeFrTjq68VniorIyEhuu+02xo8fz+WXXx7ocIwpsLzGFFYCh3CsmDbRbfsJYLUvgzJFT16zhDxRXMcO/vrrL5577jlEhFGjRtGqVStatcp3uRFjiqy8xhR2ADtwVEU1Iao43kDmLUuXLqV3795s3bqVfv36WQE7Uyzk1X30X1VtKSJHAPcpqYJjfZzzL1JvipRQuoHMG44fP87QoUOZPHkyNWvW5Ouvv6Z169aBDssYr8ir+yirDVzeH4GYwAmVG8i8Ze/evcyaNYvBgwfzwgsvcPHFFwc6JGO8Jq/uo6y7mKsAe1X1jIhcB8QD7wHH/RCf8RPrGsrbwYMH+fDDDxkwYAB169Zlx44dthKaKZY8mZL6CY6lOGsBs3EUxfvAp1EZv/jgp92uCqQmZ6rKvHnziImJ4dFHH2XLFsdKtJYQTHHlSfGVTFVNE5HbgddVdbyI2OyjIJbT3cU2XnCuvXv30r9/fxYvXkxCQgJff/21lagwxZ5Hy3GKSBegB3Crc9uFvgvJ+Foo3V18vjIyMmjRogWpqamMGTOGRx55xArYmZDgyW/5A8AAHKWzt4tIDWCOb8MyvmDTS/O3a9cuIiIiCAsLY9KkSdSsWZPatWsHOixj/CbfMQVVXQ88DCSJSF1gj6q+5PPIjNe5JwTrLjpbRkYGr732GtHR0UyePBmAdu3aWUIwIceTldeaA+8CqTjuUbhSRHqo6nJfB2e8z1oI51q/fj2JiYmsXLmSTp06ceutt+b/ImOKKU+6j8YBN6rqRgARicaRJPJd69MEnvuNaaFSf6ggpkyZwsMPP0yZMmX44IMP6N69u92VbEJUk6TrAAAXG0lEQVSaJ0nhoqyEAKCqv4rIRT6MyRRC9ruT3WcYWbfR37JKUkRHR9OlSxdef/11KlSoEOiwjAk4UT1nUbWzdxCZBZzG0ToAuAcopao9fRtazhISEjQpKSkQpy7SclvEBrAZRm7+/PNPRowYQVhYGK+88kqgwzHGb0TkF1XNt4fHk5ZCPxwDzU/iGFNYCkwoXHjG22yaaf6+++47evfuzbZt2xgwYIAVsDMmB3kmBRGpB9QCFqrqaP+EZM6XDSLn7NixYzz55JNMmzaNWrVq8c0331h5a2NykeuUVBF5GkeJi3uAr0QkpxXY8iQiHURks4gki8jQPPa7U0RURGzwuoCsVEX+9u3bx3vvvcfjjz/OunXrLCEYk4e8Wgr3APGq+oeIVACWADM9PbCIhOFYnOcGIAX4WUQWuw9aO/crjaN76qeCBm/s3oPcHDhwgLlz5/LQQw9Rt25ddu7caQPJxnggr6RwWlX/AFDVAyLiSfE8d42BZFXdDiAic4FbgI3Z9vt/wGjg8QIeP2TlNM3Uuo0cVJU5c+bw8MMPc/z4cdq3b09UVJQlBGM8lFdSqOm2NrMAtdzXalbV2/M5dmVgj9vjFKCJ+w4ichVQRVX/LSKWFPKR0wwjayH8bc+ePfTv359PP/2UJk2aMGPGDCtgZ0wB5ZUU7sj2+M0CHjunaR2u+a/Olsc4oFe+BxLpC/QFqFo1dGfV2Ayj3KWnp3P99dfz22+/MW7cOB566CHCwsICHZYxQSevRXa+LuSxU3As0JMlAtjr9rg0EAd855wWeCWwWERuVtWzbkRQ1WnANHDcp1DIuIJG9hvRrKvoXDt37qRKlSqEh4czdepUatasSc2aNQMdljFBq6DjBAXxMxApIjWcd0B3BxZnPamqx1S1vKpWV9XqwArgnIQQyrJaBlmsq+hv6enpjBkzhujoaCZNmgRA27ZtLSEYU0g+KxCvquki8iDwBRAGzFTVDSLyApCkqovzPkLwy/5Jv6CsZZCzdevWkZiYSFJSErfccgt33JG9p9MYc748TgoiUkJVTxfk4Kq6BMdUVvdtI3LZ9/qCHDsYuE8XPR/WMjjXpEmTeOSRR7jsssuYN28eXbp0sbuSjfEiT0pnNwZmAGWAqiJSH+itqg/5OrjiwD7pe0dWSYq4uDi6d+/OuHHjKF++fKDDMqbY8aSlMB7ohOPuZlR1rYjYLaHGL/744w+GDx9OeHg4r776Ki1atKBFixaBDsuYYsuTgeYLVHVXtm0ZvgjGGHdff/019erV4/XXX+f06dPkV9HXGFN4niSFPc4uJBWRMBF5FNji47iCmtUjKpyjR4/Su3dv2rZtS3h4OEuXLmX8+PE2dmCMH3iSFPoDg4GqwO9AU+c2kwurR1Q4v//+O3PnzmXIkCGsXbuW5s2bBzokY0JGvmMKqrofxz0GpgBsgLlgshLBI488Qp06ddi5c6cNJBsTAJ7MPnoLt/IUWVS1r08iCkK53Xls8qeqvP/++zzyyCOcPHmSG2+8kcjISEsIxgSIJ91H/wG+dn4tBy7HsTyncbI7j8/P7t27uemmm+jRowd16tRhzZo1REZGBjosY0KaJ91H89wfi8i7wFc+iyhIWXdRwWQVsNu/fz/jx49nwIABVsDOmCLgfMpc1ACqeTsQExq2b99OtWrVCA8P56233qJWrVpUr1490GEZY5zy7T4SkSMictj5dRRHK+Fp34dmipP09HReeeUVYmJimDhxIgBt2rSxhGBMEZNnS0EcE8PrA1mjqJlqdxCZAlqzZg2JiYmsWrWK2267jS5dugQ6JGNMLvJsKTgTwEJVzXB+WUIwBfLmm29y9dVXk5qayvz581mwYAEVK1YMdFjGmFx4MvtopYg09HkkpljJ+vwQHx/PPffcw8aNG63EtTFBINfuIxEJV9V04Dqgj4hsA/7AscymqmpIJwr3exPsvoS/nTx5kmHDhnHhhRcyZswYK2BnTJDJq6Ww0vnvrUAd4EagC3Cn89+Q5n5vgt2X4PDll18SFxfHhAkTSEtLswJ2xgShvAaaBUBVt/kplqCQ1UKwVdH+duTIEQYPHsysWbOoU6cOS5cu5brrrgt0WMaY85BXUqggIoNze1JVX/NBPEWeFbs71/79+5k/fz5PPfUUI0aMoGTJkoEOyRhznvJKCmHAJThbDOZv1kKA3377jTlz5jBo0CBXAbty5coFOixjTCHllRT2qeoLfovEBAVVZfbs2QwaNIg///yTTp06ERkZaQnBmGIir4FmayG4sYVzYOfOnXTo0IFevXoRExNjBeyMKYbyaim08VsURVjWwPJPOw4D0KRG2ZAcS0hPT6dVq1YcPHiQiRMn0q9fPy64wJPbXIwxwSTXpKCqh/0ZSFGVNbCclQzublI10CH5VXJyMjVq1CA8PJyZM2dSs2ZNqlWzeojGFFf2Uc8DWQPLoZQQ0tLSGDlyJLGxsa4Cdq1atbKEYEwxdz6ls4u9UL9bedWqVSQmJrJmzRq6dOlCt27dAh2SMcZPrKWQg1C+W3n8+PE0btyY3377jQULFvDhhx9yxRVXBDosY4yfWEvBTSjfrayqiAhXXXUV9913H2PHjuWyyy4LdFjGGD8L6aTg3k0EhOQMoxMnTvDUU09RokQJxo4dS/PmzWnevHmgwzLGBEhIdx+5dxOBIxmMvK1eyAwqf/7558TFxTFp0iRU1QrYGWNCu6UAoVmy4tChQwwePJjZs2cTHR3N8uXLadYstN4DY0zOQrKlEOp3Jx86dIiFCxfyzDPPsHr1aksIxhgXnyYFEekgIptFJFlEhubw/GAR2Sgi60TkaxHxyyT4UKx0um/fPsaMGYOqEhUVxa5du3jhhRcoUaJEoEMzxhQhPksKIhIGTAQ6AjHAXSISk2231UCCqsYD84HRvoonu1C5IU1VmTlzJtHR0TzzzDMkJycD2MwiY0yOfNlSaAwkq+p2VT0DzAVucd9BVb9V1T+dD1cAET6MJ+Ts2LGDdu3akZiYSP369Vm7dq0VsDPG5MmXA82VgT1uj1OAJnnsnwh8ltMTItIX6AtQtWrx/mTvLenp6bRu3ZpDhw4xefJk+vbtawXsjDH58mVSyKn0do5zHkXkXiABaJnT86o6DZgGkJCQYPMm87B161Zq1qxJeHg4b7/9NrVq1aJKlSqBDssYEyR8+dExBXD/axQB7M2+k4i0BYYBN6vqaR/GU6ylpaXx4osvEhcXx5tvvgnA9ddfbwnBGFMgvmwp/AxEikgNIBXoDtztvoOIXAVMBTqo6n4fxlKsJSUlkZiYyLp16+jevTt33XVXoEMyxgQpn7UUVDUdeBD4AvgV+FBVN4jICyJys3O3V3GsA/2RiKwRkcW+iqe4euONN2jSpAkHDx5k0aJFzJkzh8svvzzQYRljgpRP72hW1SXAkmzbRrh939aX5y/OsgrYJSQkkJiYyOjRo/m///u/QIdljAlyIV/mItgcP36cIUOGULJkScaNG8e1117LtddeG+iwjDHFhM1RDCJLliwhNjaWadOmER4ebgXsjDFeZ0khCBw8eJB7772Xm266iTJlyvDDDz/w6quvIpLTrF9jjDl/lhSCwJEjR/jXv/7Fs88+y6pVq2jSJK97AI0x5vzZmEIRlZqayvvvv88TTzxBZGQku3btsoFkY4zPWUuhiFFV3nrrLWJiYnjuuefYtm0bgCUEY4xfWFIoQrZt20abNm3o27cvDRs2ZN26ddSuXTvQYRljQoh1HxUR6enptGnThsOHDzN16lR69+5tBeyMMX5nSSHANm/eTK1atQgPD+edd96hVq1aRERYBXFjTGDYR9EAOXPmDM8//zz16tVj4sSJALRs2dISgjEmoKylEAArV64kMTGR9evXc/fdd3PPPfcEOiRjjAGspeB3r7/+Os2aNXPde/D+++9Tvnz5QIdljDGAJQW/ySpJ0bhxY/r06cOGDRvo1KlTgKMyxpizWfeRjx07downn3ySf/zjH7z++utcc801XHPNNYEOyxhjcmQtBR/617/+RUxMDNOnT6dEiRJWwM4YU+RZUvCBAwcOcPfdd3PzzTdTrlw5VqxYwSuvvGIF7IwxRZ4lBR84duwYS5Ys4fnnnycpKYmrr7460CEZY4xHbEzBS/bs2cN7773H0KFDqV27Nrt27aJMmTKBDssYYwrEWgqFlJmZyZQpU4iNjeXFF190FbCzhGCMCUaWFAph69attG7dmv79+9O4cWP+97//WQE7Y0xQs+6j85Sens4NN9zA0aNHmTFjBvfff78NJBtjgp4lhQL69ddfiYyMJDw8nHfffZdatWpRqVKlQIdljDFeYd1HHjp9+jTPPvss8fHxvPnmmwA0b97cEoIxplixloIHVqxYQWJiIhs3bqRHjx706NEj0CEZY4xPWEshH2PHjuWaa67hxIkTLFmyhNmzZ1OuXLlAh2WMMT5hSSEXmZmZADRr1ox+/fqxfv16OnbsGOCojDHGt6z7KJujR4/y2GOPUapUKSZMmGAF7IwxISVkWgof/LSbblN/pNvUH9m473iO+3zyySfExMTwzjvvULp0aStgZ4wJOSGTFBatSXUlg5iKl3JLg8qu5/bv30/Xrl257bbbuOKKK1i5ciUjR460+w6MMSEnpLqPYipeyrx/Njtn+/Hjx/nqq6946aWXeOKJJ7jwwgsDEJ0xxgSeT1sKItJBRDaLSLKIDM3h+RIiMs/5/E8iUt2X8bjbvXs3L730EqpK7dq12b17N08//bQlBGNMSPNZUhCRMGAi0BGIAe4SkZhsuyUCR1S1NjAOeMVX8WTJzMxk0qRJxMbGMnLkSFcBu9KlS/v61MYYU+T5sqXQGEhW1e2qegaYC9ySbZ9bgHec388H2oiPOvJjKl3K5ZmHuP766xk4cCDNmjVjw4YNVsDOGGPc+HJMoTKwx+1xCtAkt31UNV1EjgHlgIPeDmZYxzrUrn0Tx44d4+2336Znz542kGyMMdn4Mink9Bc3+xxPT/ZBRPoCfQGqVq16XsGEh4fz3nvvUatWLSpWrHhexzDGmOLOl91HKUAVt8cRwN7c9hGRcKAMcDj7gVR1mqomqGpChQoVzjug6667zhKCMcbkwZdJ4WcgUkRqiMhFQHdgcbZ9FgM9nd/fCXyjdseYMcYEjM+6j5xjBA8CXwBhwExV3SAiLwBJqroYmAG8KyLJOFoI3X0VjzHGmPz59OY1VV0CLMm2bYTb96eALr6MwRhjjOdCpsyFMcaY/FlSMMYY42JJwRhjjIslBWOMMS6WFIwxxrhIsN0WICIHgF3n+fLy+KCERhFn1xwa7JpDQ2GuuZqq5nv3b9AlhcIQkSRVTQh0HP5k1xwa7JpDgz+u2bqPjDHGuFhSMMYY4xJqSWFaoAMIALvm0GDXHBp8fs0hNaZgjDEmb6HWUjDGGJOHYpkURKSDiGwWkWQRGZrD8yVEZJ7z+Z9EpLr/o/QuD655sIhsFJF1IvK1iFQLRJzelN81u+13p4ioiAT9TBVPrllEujp/1htE5AN/x+htHvxuVxWRb0VktfP3+8ZAxOktIjJTRPaLyPpcnhcRGe98P9aJSEOvBqCqxeoLR5nubUBN4CJgLRCTbZ8BwBTn992BeYGO2w/X3Aoo5fy+fyhcs3O/0sBSYAWQEOi4/fBzjgRWA5c5H18e6Lj9cM3TgP7O72OAnYGOu5DX3AJoCKzP5fkbgc9wrFzZFPjJm+cvji2FxkCyqm5X1TPAXOCWbPvcArzj/H4+0EaCe8HmfK9ZVb9V1T+dD1fgWAkvmHnycwb4f8Bo4JQ/g/MRT665DzBRVY8AqOp+P8fobZ5cswKXOr8vw7krPAYVVV1KDitQurkFmK0OK4D/ExGvLSlZHJNCZWCP2+MU57Yc91HVdOAYUM4v0fmGJ9fsLhHHJ41glu81i8hVQBVV/bc/A/MhT37OUUCUiCwXkRUi0sFv0fmGJ9f8HHCviKTgWL/lIf+EFjAF/f9eID5dZCdAcvrEn32KlSf7BBOPr0dE7gUSgJY+jcj38rxmEbkAGAf08ldAfuDJzzkcRxfS9Thag8tEJE5Vj/o4Nl/x5JrvAmap6lgRaYZjNcc4Vc30fXgB4dO/X8WxpZACVHF7HMG5zUnXPiISjqPJmVdzrajz5JoRkbbAMOBmVT3tp9h8Jb9rLg3EAd+JyE4cfa+Lg3yw2dPf7UWqmqaqO4DNOJJEsPLkmhOBDwFU9UegJI4aQcWVR//fz1dxTAo/A5EiUkNELsIxkLw42z6LgZ7O7+8EvlHnCE6QyveanV0pU3EkhGDvZ4Z8rllVj6lqeVWtrqrVcYyj3KyqSYEJ1ys8+d3+BMekAkSkPI7upO1+jdK7PLnm3UAbABGJxpEUDvg1Sv9aDNznnIXUFDimqvu8dfBi132kquki8iDwBY6ZCzNVdYOIvAAkqepiYAaOJmYyjhZC98BFXHgeXvOrwCXAR84x9d2qenPAgi4kD6+5WPHwmr8A2onIRiADeEJVDwUu6sLx8JofA94SkUE4ulF6BfOHPBGZg6P7r7xznORZ4EIAVZ2CY9zkRiAZ+BO436vnD+L3zhhjjJcVx+4jY4wx58mSgjHGGBdLCsYYY1wsKRhjjHGxpGCMMcbFkoIpckQkQ0TWuH1Vz2Pf6rlVkyzgOb9zVuJc6ywRUec8jtFPRO5zft9LRCq5PTddRGK8HOfPItLAg9c8KiKlCntuExosKZii6C9VbeD2tdNP571HVevjKJb4akFfrKpTVHW282EvoJLbc71VdaNXovw7zkl4FuejgCUF4xFLCiYoOFsEy0RklfPrmhz2iRWRlc7WxToRiXRuv9dt+1QRCcvndEuB2s7XtnHW6f+fs859Cef2UfL3+hRjnNueE5HHReROHPWl3nee8x/OT/gJItJfREa7xdxLRCacZ5w/4lYITUQmi0iSONZReN657WEcyelbEfnWua2diPzofB8/EpFL8jmPCSGWFExR9A+3rqOFzm37gRtUtSHQDRifw+v6AW+oagMcf5RTnGUPugHXOrdnAPfkc/7OwP9EpCQwC+imqvVwVADoLyJlgduAWFWNB150f7GqzgeScHyib6Cqf7k9PR+43e1xN2DeecbZAUdZiyzDVDUBiAdaiki8qo7HURenlaq2cpa+GA60db6XScDgfM5jQkixK3NhioW/nH8Y3V0IvOnsQ8/AUdMnux+BYSISASxQ1a0i0gZoBPzsLO/xDxwJJifvi8hfwE4c5ZfrADtUdYvz+XeAgcCbONZnmC4inwIel+ZW1QMist1Zs2ar8xzLncctSJwX4yj74L7qVlcR6Yvj/3VFHAvOrMv22qbO7cud57kIx/tmDGBJwQSPQcDvQH0cLdxzFs1R1Q9E5CfgJuALEemNo8zwO6r6lAfnuMe9YJ6I5LjGhrMeT2McRdi6Aw8CrQtwLfOArsAmYKGqqjj+QnscJ44VyEYBE4HbRaQG8DhwtaoeEZFZOArDZSfAV6p6VwHiNSHEuo9MsCgD7HPWyO+B41PyWUSkJrDd2WWyGEc3ytfAnSJyuXOfsuL5+tSbgOoiUtv5uAfwX2cffBlVXYJjEDenGUAncJTvzskC4FYc6wDMc24rUJyqmoajG6ips+vpUuAP4JiIXAF0zCWWFcC1WdckIqVEJKdWlwlRlhRMsJgE9BSRFTi6jv7IYZ9uwHoRWQPUxbFk4UYcfzy/FJF1wFc4ulbypaqncFSg/EhE/gdkAlNw/IH9t/N4/8XRisluFjAla6A523GPABuBaqq60rmtwHE6xyrGAo+r6locazNvAGbi6JLKMg34TES+VdUDOGZGzXGeZwWO98oYwKqkGmOMcWMtBWOMMS6WFIwxxrhYUjDGGONiScEYY4yLJQVjjDEulhSMMca4WFIwxhjjYknBGGOMy/8HK262S1N9OLsAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"def isDemocrat(entity):\n",
" return entity.endswith('(R)')\n",
"\n",
"from sklearn.svm import SVC\n",
"svm_clf = SVC(C=2**15, kernel='linear')\n",
"\n",
"from sklearn.model_selection import cross_val_score \n",
"scoresw = cross_val_score(svm_clf,i1_mold,y,cv=kf)\n",
"print(\"Mean cross-validation accuracy: {:.2f}\".format(np.mean(scoresw)))\n",
"\n",
"ypred = cross_val_predict(svm_clf, i1_mold, y, cv=kf)\n",
"\n",
"print(classification_report(y, ypred, digits=10))\n",
"conf_mx = confusion_matrix(y, ypred, labels=[True,False])\n",
"plt.matshow(conf_mx,cmap=plt.cm.Blues)\n",
"plt.xticks(np.arange(2), ['D', 'R'])\n",
"plt.yticks(np.arange(2), ['D', 'R'])\n",
"plt.savefig('figures/classification/mold_confmx.pdf')\n",
"plt.show()\n",
"\n",
"from sklearn.metrics import roc_curve\n",
"from sklearn.metrics import roc_auc_score\n",
"\n",
"y_scores = cross_val_predict(svm_clf, i1_mold, y, cv=kf, method=\"decision_function\")\n",
"\n",
"fpr, tpr, thresholds = roc_curve(y, y_scores)\n",
"\n",
"fpr, tpr, thresh = roc_curve(y, y_scores)\n",
"auc = roc_auc_score(y, y_scores)\n",
"plt.plot(fpr,tpr,label=\"AUC = \"+repr(auc))\n",
"\n",
"plt.plot([0,1],[0,1],'k--')\n",
"\n",
"plt.legend(loc=0)\n",
"plt.xlabel(\"False Positive Rate\")\n",
"plt.ylabel(\"True Positive Rate\")\n",
"plt.savefig('figures/classification/mold_roc.pdf')\n",
"\n",
"# F1 Score\n",
"from sklearn.metrics import f1_score\n",
"\n",
"mold_f1 = f1_score(y, ypred, average='macro') \n",
"print(f\"F1 Score: {mold_f1}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Results\n",
"The first algorithm, Metric Closure with TF-IDF+Cosine Similarity was more accurate in classifying the politicians, hence, Metric Closure with TF-IDF+Cosine Similarity is better at calculating the semantic proximity and truth values of the politicans to the ideologies, and will be used in the succeeding parts of this paper."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Validation on Factual Statements\n",
"\n",
"Simple statements various subject areas were tested. Statements in the form of \"$p_{i}$ was married to $s_{j}$\" were considered, where $p_{i}$ is a US president, and $s_{j}$ is a spouse of a US president. A matrix of statements was then obtained by getting all the combinations of subjects and objects."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Get the paths taken by the statements in the knowledge graph.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment these line(s) below if making a truth matrix from scratch (takes a VERY long time)\n",
"# if truth matrices were already calculated, load from the CSV instead (bottom most part of this ntbk)\n",
"#df = pd.read_csv('data/out/t2_output.csv', sep=';', index_col=0)\n",
"#print(df.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Map the truth value function to the Path matrix.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": false
},
"outputs": [],
"source": [
"# Uncomment these line(s) below if making a truth matrix from scratch (takes a VERY long time)\n",
"# if truth matrices were already calculated, load from the CSV instead (bottom most part of this ntbk)\n",
"#w_new, w_u_new, w_mold = get_truth_matrices(df)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Get the confusion matrix of the truth values**"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Confusion Matrix\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQoAAAEGCAYAAACdCduyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAFh9JREFUeJzt3XmYXXV9x/H3J5OZyTLZE/btQYtRFiME+wRBBAEXqIoLIDwKtjWKKFUerWitxrpQrYgKKI1bQQtoLQIVEHGrVYPKjrGtVpaCoGQjZJ1MZr79456xQ5iZ3/2d3HPmTvJ5Pc88M/fM+d3v7+Qm35zte76KCMzMRjNhrCdgZu3PicLMkpwozCzJicLMkpwozCzJicLMksZNopDUL+muIV/nVxRnV0lXSrpP0u2Slkk6uYI4lW6PpJD0lSGvJ0paIelbrYxTvPf6Vr/nMDEG/7zulnSHpCMqiBGSLhzy+p2SllQQZ/02r8+SdEmLY7T085/YuqlVblNELKgygCQB1wKXR8TpxbJ9gZdVEK7q7dkAHCRpckRsAo4HfldhvKr98c9L0ouAC4CjWxyjF3ilpAsiYmWL37tuLf38x80eRU2OBbZExGWDCyLiwYi4eAzntD1uAk4sfn4tcNUYzqWVpgNrKnjfrcBS4B0VvPdYaNnnP54SxeRtdtVPrSDGgcAdFbzvcOrYnquB0yRNAg4BflZBjLoM/nn9F/AF4EMVxbkUOEPSjIreH7b57IG/qyhOyz5/H3qMQtKlwJE09jIOb/HbV749EXGPpP1o/G9yY5WxajD00GMRcIWkg6LFNQgR8YSkK4BzgU2tfO8hnvTZSzoLWNjqIK38/MfTHkUdlgOHDr6IiHOAFwLzxmxG2+964BPsOIcdRMQyYC7VfS6fAv4CmFrR+9epJZ+/E8WTfR+YJOnsIcumjNVkWuRLwN9FxL1jPZFWkTQf6ABWVfH+EbEa+DqNZDHeteTzH0+HHpOL47lB346Ill5SjIiQ9ArgIkl/Daygcfb43a2MU6h8ewAi4mHg061+321MkfTwkNefjIhPtjjG0D8vAWdGRH+LYwx1IfDWCt+/Fq36/OUyczNL8aGHmSU5UZhZkhOFmSU5UZhZ0rhLFJIWO077xagrzo60LXXFaUWMcZcogFo+wB0sjrdl546zUyYKM6tZ295H0Tl1RnTP2u0py/s2rKVz6vD1Oj2TOrPjbOkfGHZ577o1dE+b9ZTl+8yanB+jb/gYAGtWr2LW7DlPWT5hgrLjTOwYfsyqlSuZM3fusL/rHWVuuXFWr1rJ7DlPjbO1P//vWG4MgDJ/lQcGhh800ufSmFve/68Do0xszaqVzBpmezpKfP79I23LCDEAlt9z58qISN4K37Z3ZnbP2o2D37Y0a8wRz9w1O85Dqzdmrf+5Vx+SHeP+FRuyx/RMyv9o5vR0ZY+5f0Xe9gPMnZYX57G1vdkxdpnRnT2mTEJ6YlNf9phdZ0zKWn/jlvwbSGdMzv/8H9+Yvy3P3KPnwWbW86GHmSU5UZhZUmWJQtJFkt4+5PXNkr4w5PWFks6rKr6ZtU6VexQ/BY4AkDSBxvMDDhzy+yOAn1QY38xapMpE8ROKREEjQfwSWCdplqRu4JnAnRXGN7MWqeyqR0Q8ImmrpH1oJIxlwJ7AImAtcE9EbBk6priDbDFA18z8KxhmVo2qT2YO7lUMJoplQ17/dNuVI2JpRCyMiIUj3SthZvWrOlEMnqc4mMahx6009ih8fsJsHKljj+IkYHVE9BfPIpxJI1ksqzi2mbVI1YniXhpXO27dZtnaHaATk9lOo9JbuIuHn07fZtlZVcY0s9Zr21qPp+/Sw/VvyetDe9wnfpQd5+tnL8pav8x9+xv78sdMm5xf4FamkGzmlPw4ucrUbUzp6sges3WEoqjRKP+PLFvXxPwdd5WYWJlCsmb5Fm4zS3KiMLMkJwozS3KiMLMkJwozS3KiMLMkJwozS3KiMLMkJwozS3KiMLMkJwozS3KiMLOkti0KmyAxpTtveje946jsOAf/1b9mrX/Pp1+VHWPPmfndxcoUK5UpCpranV98ldspa6QOVqOZmvnZA5TpeTehxB90d2fe9q/fvDU7RpnPZW2JBkDN8h6FmSU5UZhZUm2JQtJukq6W9FtJv5J0o6QD6opvZuXVkijUeArHN4EfRsTTIuJZwHsBP5PfbByo62TmMUBfRFw2uCAi7qoptpltp7oOPQ4Cbk+tJGmxpNsk3bZy5YoapmVmzWirk5lDGwDNnTtvrKdjZoW6EsVy4LCaYplZi9WVKL4PdEt64+ACSYdLOrqm+Ga2HWpJFBERwMnA8cXl0eXAEuCROuKb2fap7RbuiHgEOKWueGbWOm1b6xEBfVsHssZ0duTft/+bz74ma/2TLn1KE/akC195SPaY/eZNyR7TlVmDAbC2RB3GlO68OH39eZ8jlGvmMxD5Y6LEmNyiku4SDYDq2v5mtdVVDzNrT04UZpbkRGFmSU4UZpbkRGFmSU4UZpbkRGFmSU4UZpbkRGFmSU4UZpbkRGFmSU4UZpbUtkVh/QPBE5mNU3pKNE3JbQBz87lHZsfY5fQvZ4+597LTs8fMmtqVPaZMc5rcBkC9ff3ZMeb05G/LE5vyt6VMcyJl/p1Zua43O8YBu0/LHtNZoiiwWd6jMLMkJwozS3KiMLOk2s5RSOoH7h2y6BUR8UBd8c2svDpPZm6KiAU1xjOzFvGhh5kl1ZkoJku6q/j65nArDO0UtnqVO4WZtYu2OvSIiKXAUoBDFhxW3ZNCzSyLDz3MLMmJwsySnCjMLKm2RBERPXXFMrPWatuisIkdYvbUzqwxZYqCpmQWkmXWAwFw0wWvyB7zl1fekT3m3968KHtMmUKy3M5fvX35ncI2lxgzqTO/KHDtxr7sMdOmVf/Ppkyx2qYt+cV3zfKhh5klOVGYWZIThZklOVGYWZIThZklOVGYWZIThZklOVGYWZIThZklOVGYWZIThZklOVGYWVLbFoX1DwQbevOKXNaV6Ho1EHnFN2WKddZtyZ/XRScfnD1m/nnXZY/55Bufmz3muXvPyVr/9kfXZMc4cO707DEzpuQVEQI8unZz9pjJXXnFZ2WKtTo78qsPOye6U5iZjaFkopD0sWaWmdmOq5k9iuOHWfaSZgNI6i+evL1c0t2SzpPkPRmzcWTEcxSSzgbeAuwv6Z4hv5oG/CQjxh+fvi1pF+BKYAbwgfzpmtlYGO1k5pXATcAFwPlDlq+LiNVlgkXEY5IWA7+QtCQi80yimY2JEQ8BImJtRDwQEa8FHgb6gAB6JO1TNmBE3FfE3WXb3z25AdDKsiHMrMWSl0clvRVYAvwBGHyQYQCHbEfcYa/9DG0A9OznuAGQWbto5j6KtwPPiIhVrQgoaX+gH3isFe9nZtVr5urDQ8DaVgSTNA+4DLjE5yfMxo9m9ijuA34o6Qagd3BhRHyyyRiTJd0FdAJbga8AzY41szbQTKL43+Krq/jKEhH5zRbMrK0kE0VEfBBA0tSI2FD9lMys3TRz1WMR8EWgB9hH0rOBN0XEW6qcmAQdE/IKYyZ15t/w2ZVZSDOhRKuwjVvzi8ImduRvy3ffd0L2mMPO/Vr2mG9/NK/z2cTMzxFgj1mTs8c8XqLrV093fl1kbsHWYyWKFcsUeK3blL/9zWpmNp8CXgSsAoiIu4HnVzYjM2s7TaWtiHhom0XVNTk0s7bTzH7XQ5KOAEJSF3Au8J/VTsvM2kkzexRvBs4B9qRxK/eC4rWZ7SSaueqxEjijhrmYWZsarcz8Yho1HcOKiHMrmZGZtZ3RDj1uA24HJgGHAr8pvhbgk5lmO5UR9ygi4nIASWcBx0REX/H6MuA7tczOzNpCMycz96DxVKtBPcUyM9tJNHN59O+BOyX9oHh9NI3nU5jZTqKZqx5flnQT8KfFovMj4vfVTsvM2smIhx6S5hffD6VxqPFQ8bVHsczMdhKj7VGcBywGLhzmdwEcW8mMBgMEbO0fSK84RG4RGUBHZpHXxBIdnPaf0ZM9ZvmK/GcFvWT+btljPvy2F2SP+cEDeQ87O2H/edkxerfmffZQrrvWlhJx1mzIK74qM6/cv/uQX+CYY7SrHouL78dUFt3MxoVmOoW9RtK04uf3SbpG0nOqn5qZtYtm9lX+NiLWSTqSRrn55TSee5lN0voy48xsbDWTKAbvwjwR+FxEXEeJR+KZ2fjVTKL4naR/BE4BbpTU3eQ4M9tBNPMP/hTgZuDFEfE4MBt4VxWTGdopbJU7hZm1jWSiiIiNNJr1HFks2kqjOKzlImJpRCyMiIVz5sytIoSZldDMVY8PAO8G3lMs6gS+WuWkzKy9NHPocTLwMmADQEQ8wpOLxMxsB9dMothStP8LaPT32I54UyQ9POTrvO14LzOrSTPVo18vrnrMlPRG4M+Bz5cJFhG+WmI2DjVTPfoJSccDTwDPAN4fEbdUPTEpvwnK5k35jVZy7whRiQZAW0rct3/4nrOzx/xhbW96pW3Mn5Nfh7LPzClZ65/w4fznHH3r/OOyx5SpdRgo0St745a8B7yt681vzDN/ev7RfZnmVM0aNVFI6gBujojjgMqTg5m1p1FTcET0AxslzahpPmbWhpo5R7EZuFfSLRRXPsBP4TbbmTSTKG4ovsxsJ5U6R/EcGnsRyyPCbQTNdlKjPQrv/cDXgFcBNxSXRs1sJzTaHsWpwIKI2ChpDvBtSt4/YWbj22hXPTYXBWFExKrEuma2Axttj+Jpkq4vftY2r4mIl1U6MzNrG6Mlipdv8/oTVU7EzNrXaE/h/vc6J2Jm7cvnHcwsqZkbrsZMiXqdbL19eQVbAwP5k5rTk/8s4k2ZhUcAu8zozh4zZ0P+3Db35c3t4rMXZcc45r3XZY/50rvzC8k29uUXEh4wK69ga1p3Z3aM35co8NvYW6IosklZexSSZqlM+aSZjWuj3nA1pP9od9HN/LfAHyTlp24zG7dG26M4Ffjv4uczi+/zgKOBj1Y5KTNrL6MlisFH4EGjQ9jVEdFf1HxknduQtKukKyXdJ+l2ScsknVx20mZWr9ESRa+kgyTNA44Bhj6mqOlHHBXnNK4FfhQR+0fEYcBpwF5lJmxm9Rttz+DtwDdoHG5cFBH3A0h6KXBnRoxjaeyd/LFfaUQ8CFycP10zGwuj3XB1KzB/mOU3AjdmxDgQuKOZFSUtBhYD7LX3PhkhzKxKIyaKYR6lH8BK4MeDexdlSLqURtexLRFx+JMCRCwFlgIsOPSwGu6iMLNmjHaOYto2X9OBhcBNkk7LiLEcOHTwRUScA7yQxiGNmY0Dox16fHC45ZJmA98Frm4yxveBj0o6OyI+VyzLe967mY2p7FqPiFhNo+y82fUDeAVwtKT7Jf0cuJxGP1MzGweyaz0kHQusyRkTEY/SuCRqZuPQaCcz76XoNzrEbOAR4PVVTqqIn935aWp3R3aciR15Mcp0o+rdmt8prExB3KTO/O3v688PNG96XvHZjCn5RVE//nj+/XiLzvtG9pgbPpT//KXcvwN/WJ9f4LX7tEnZY0rUKzZttD2Kk7Z5HcCqiNgw3MpmtuMa7WTmg3VOxMzalx9cY2ZJThRmluREYWZJThRmluREYWZJThRmluREYWZJThRmluREYWZJThRmltS2ncIGBiK7W9bW/vziq9x+Rn0lCrzWbOjLHtNdovisZ1L+xzmtxJjcz+XBNRuzY5QpiipT4PXi93wze8x1H86Pk6u/RIXXzBLFd83yHoWZJTlRmFlSLYcekvqBe4t49wOvi4jH64htZtuvrj2KTRGxICIOAlYD59QU18xaYCwOPZYBe45BXDMrqdZEIamDxqP6rx/h94sl3SbpttWrVtY5NTMbRV2JYrKku4BVNJ67ectwK0XE0ohYGBELZ8+ZW9PUzCyl1nMUwL5AFz5HYTau1HroERFrgXOBd0qq7u4QM2up2k9mRsSdwN24z4fZuFHLfRQR0bPN6z+rI66ZtYbvzDSzpLYtCpsgMakzL49tyavvAmDihLxBEzvyg8yeWuJ0TGaxGkDmpgCwuS+vwAtgVldX1vplCtw6SmzM3J68eQFcteSl2WNefsaw/btHdM1X358d43lv+Ez2mO8tre4agfcozCzJicLMkpwozCzJicLMkpwozCzJicLMkpwozCzJicLMkpwozCzJicLMkpwozCzJicLMktq2KKx/IHhi09asMd2ZRWQA/ZHXkalL+TEeWrUpe8xuM/M7ZXWW6BTV15/fkSq3I9m0rvx5bSnRkW1yV0f2mL4SHbn+45qPZK1/1Cl5RWQAP/3Gkuwxv12zPntMs7xHYWZJThRmllRLopDUL+kuSXdLukPSEXXENbPWqOscxeBTuJH0IuAC4OiaYpvZdhqLQ4/pwJoxiGtmJdW1RzHYAGgSsDtw7HArSVoMLAbYc6+9a5qamaXU3aR4PvBi4ArpqQ+FfHKnsHk1Tc3MUsair8cyYC7gTGA2TtSeKCTNBzpo9CE1s3Gg7nMUAALOjIj858Sb2Zioq1NY/r21ZtY22rbWo2OCmD45b3pl6hZyG81MKNGYZr95U7LH9JeoQSjRM4gpJeojNvTm1eB0lmiaNHdad/aY3HkB7Dd9avaYEz/ynaz1r/riu7JjHH3+tdljrl9yYvaYZvkWbjNLcqIwsyQnCjNLcqIwsyQnCjNLcqIwsyQnCjNLcqIwsyQnCjNLcqIwsyQnCjNLcqIws6S2LQobiGBzX14TmDLFR1szC8liYn6x1u/X9maPmdPTlT2mjKmZzXwA5k3PK9jKLbwrO6Y38+8LlCvy++6Sl2Stf82vHs2OUabA63UX/zh7TLO8R2FmSU4UZpbkBkBmluQGQGaW5AZAZpbUtg2A9tp7n5qmZmYpbdwAaG5NUzOzFDcAMrMkNwAysyQ3ADKzJDcAMrMk35lpZkmKyC9yqoOkFcCDw/xqLrCyhinsSHG8LTt3nNFi7BsRyQsLbZsoRiLptohY6DjtFaOuODvSttQVpxUxfOhhZklOFGaWNB4TxVLHacsYdcXZkbalrjjbHWPcnaOwakj6G+B0oB8YAN4UET8b21lZu2jbR+FZfSQtAk4CDo2IXklzgXqexWfjwng89LDW2x1YGRG9ABGxMiIekfSApI9J+nnx9XQASftK+p6ke4rv+xTL/0nSqwffVNL64vvukn5UPLzol5KOKpafIGlZ8TCjf5HUU/uWW1OcKAzgO8Dekn4t6bOShj5U6ImIeC5wCfCpYtklwBURcQjwz8BnEu9/OnBz8fCiZwN3FXst7wOOi4hDgduA81q3SdZKPvQwImK9pMOAo4BjgK9JOr/49VVDvl9U/LwIeGXx81eAjydC/AL4kqRO4NqIuKtIRs8CflI8caALWNaK7bHWc6IwAIoivR8CP5R0L3Dm4K+GrjbS8OL7Voq91OJ5I13Fe/9I0vOBE4GvSPoHGk85uyUiXtvK7bBq+NDDkPQMSX8yZNEC/v/2+VOHfB/8H/+nwGnFz2cAgw0lHgAOK35+OdBZvP++wGMR8Xngi8ChwK3A84ac95gi6YAWbpa1kPcoDKAHuFjSTBp7Bf9D45GEJwHdkn5G4z+Vwf/9z6VxKPEuYAXwhmL554HrJP0c+B6woVj+AuBdkvqA9cDrI2KFpLOAqyQNdhR6H/DryrbSSvN9FDYiSQ8ACyOijuIoa2M+9DCzJO9RmFmS9yjMLMmJwsySnCjMLMmJwsySnCjMLOn/AHtfQZSj7Ee8AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig, ax = plt.subplots(1,1)\n",
"plt.xlabel('Spouse')\n",
"plt.ylabel('US President')\n",
"\n",
"ax.matshow(w_new, cmap=color)\n",
"ax.set_xticklabels([' ']+[x[0] for x in w_new.keys()][::2])\n",
"ax.set_yticklabels([' ']+[x[0] for x in w_new.index][::2])\n",
"print(\"Confusion Matrix\")\n",
"\n",
"fig.savefig('figures/phase2/president_spouse.pdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Get F1 Score**"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"F1 Score: 0.9142857142857143\n"
]
}
],
"source": [
"from sklearn.metrics import roc_curve\n",
"from sklearn.metrics import roc_auc_score\n",
"\n",
"vfs_w_y, vfs_w_y_score = get_mx(w_new)\n",
"ypred = cross_val_predict(svm_clf, vfs_w_y_score.reshape(-1,1), vfs_w_y, cv=kf)\n",
"\n",
"from sklearn.metrics import f1_score\n",
"m_f1 = f1_score(vfs_w_y, ypred) \n",
"print(f\"F1 Score: {m_f1}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Validation of World Capitals"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.\n",
"#df_capitals = pd.read_csv('data/out/t3_countries.csv', sep=';', index_col=0)\n",
"#df_capitals.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Map the truth value function to the matrix**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.\n",
"#c_new, c_u_new, c_u_mold = get_truth_matrices(df_capitals)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Show confusion matrix**"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Confusion Matrix\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQ0AAAEGCAYAAAB/1cDLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHvtJREFUeJzt3XuUpWV15/HvrkvXvaqr+gLNRRtRVIKx0QYdiQoSo3EgiKMmjMvB8dImQiJeJhIzGcnKhLi8AFlm1IWCoEFmMIDiJVFjMGh0KQ0ioKgwTkOAppvuruquS1d1Xfb8UaewLM/e73mbPnVOye+zVq+qOs953/ep55zafc7Zz7Mfc3dERGrV0ugOiMjKoqAhIqUoaIhIKQoaIlKKgoaIlKKgISKlrKigYWYvN7Ofmtl9ZnZhE/Rnm5ndZWZ3mNnWBvXhSjPbaWZ3L7ptyMy+bmb3Vr4ONrg/F5nZQ5VxusPMXrGM/TnazG42s3vM7Edm9vbK7Q0Zo6Q/DRujsmylzNMws1bgZ8BLgQeBW4Fz3P3HDezTNmCzu+9qYB9eBIwBn3b3Eyq3fQDY4+7vrwTXQXd/TwP7cxEw5u4fWo4+LOnPBmCDu99uZn3AbcArgTfQgDFK+vNaGjRGZa2kVxonA/e5+8/d/QDwv4GzGtynhnP3W4A9S24+C7i68v3VzD8pG9mfhnH37e5+e+X7UeAe4EgaNEZJf1aMlRQ0jgT+fdHPD9L4wXbga2Z2m5ltaXBfFjvM3bfD/JMUWN/g/gCcb2Z3Vt6+LNvbpcXMbCNwIvA9mmCMlvQHmmCMarGSgoZVua3R761OcffnAL8LnFd5aS6/6mPAscAmYDvw4eXugJn1AtcDF7j7vuW+fg39afgY1WolBY0HgaMX/XwU8HCD+gKAuz9c+boTuJH5t1DNYEflvfPCe+idjeyMu+9w91l3nwM+wTKPk5m1M/8Heo2731C5uWFjVK0/jR6jMlZS0LgVeJqZHWNmq4A/AG5qVGfMrKfyQRZm1gP8DnB3ftSyuQk4t/L9ucAXGtiXhT/KBWezjONkZgZcAdzj7pcsamrIGEX9aeQYlbVisicAlTTUZUArcKW7/3UD+/IU5l9dALQBn21Ef8zsWuBUYC2wA3gf8HngOuBJwAPAa9x9WT6cDPpzKvMvux3YBrx14fOEZejPbwHfAu4C5io3v5f5zxGWfYyS/pxDg8aorBUVNESk8VbS2xMRaQIKGiJSioKGiJSioCEipazIoNFksy/VnwLN1h9ovj41W38yKzJoAM02wOpPrtn6A83Xp2brT2ilBg0RaZAVMU+jrXvA2wcOe+znmYm9tHUPAPCUdb0Hfd6JmZmwrdpClwWt9suxdu/wbgYG1zz2c1tLfPRcMt5tLQcfw23RJYd372JwzdrHfj4wM1fliMo1W5Pf9BA9NYb37GZw6BfjMz0X9wdgVWtr3NYWj9H+A7Nh29JxX/qYdbbH18wes7m5ZJCyJxGw+NCRPbtYPfSLxyx5CmHJiYueQtHQP/zg/Qzv2V3Q43lttdzpUDOzlwN/y/zMzk+6+/uz+7cPHMbGN3ykattn/+gFBdeK2+7YMRK2ZX/4q1etSq+5pitu3z8TP7EHk+Oy3wPyP6ZteybCtsN6O8K22ewP4nHYMT6Zth/V3x22PXlt3Hb3g/E6tLHp+D8IgGce3he2TUzFj9n4VHze1uwvH5hIglwWxLLnZl9X/ic9Nlm9v+f8xxenxy227G9PKsV0/hfzK0OPB84xs+OXux8icnAa8ZmGiumIrGCNCBo1FdMxsy1mttXMts5M7F22zolIrhFBo6ZiOu5+ubtvdvfNCx96ikjjNSJoNF0xHRGpXSOyJ48V0wEeYr6Yzn/ODjhmXQ/X/OF/qNr20e89kF7s3S86Jmw7oqcrbFvXE2cVshQcQEfyyXf3XDzk7Un688c78wp1Jx09FLYd0d+ZHhtpa83/TynKDkR6DuRPu5bkvFmWqGvVwaVqIc8UZZmr7LisP5CnwrNjs2GfnM7T2QPd7VVvL/NYLnvQcPcZMzsf+Cq/KKbzo+Xuh4gcnIbM03D3rwBfacS1ReTx0TRyESlFQUNESlHQEJFSFDREpBQFDREppSHZk0Npy0lHp+2v/uh3w7b3nPX0sO2o1fEcjh37ptJrtiWrF7NseHdH/HCs78rnWmQrJkf2T4dt/Z3V8/YA09PxOQFmkjkKB2bj+QK7Jw+k581W+04mfRo/EK84HS64Zk9HPC9i93h8bFsyiWPvRDzukM/nmUp+z+yx7kmeQxDP8Siae/RL56j5niIiKGiISEkKGiJSioKGiJSioCEipShoiEgpKyLlOjcH40Fx16IlvZ/6ryeFba+4+Gth2xcvfGnYtnsyT7ke1RcXv82WUme/ylxBafDpZJn13qk49dedpP0eTzHjrMp9d1u+ZDxbGp+lebMq50UV0LOM45qeOAWcpWM7CkoLZI9ZlgLt64z/bHuTNoB9+6unpcvUkNYrDREpRUFDREpR0BCRUhQ0RKQUBQ0RKUVBQ0RKWREp1+m5OR6Z2F+17Rlr+9Nj+5O9La/4kxeGbS9613Vh2z9/4NXpNYd64xTd/XvGw7asAnWW9oM8/ZmlVbOVtfXaHHywIJebVWXPutS/Kl6xW6QzGfusavhgd/y4dLbn/ydH6U+AmWSVcFYlvqiC/KGoRq5XGiJSioKGiJSioCEipShoiEgpChoiUoqChoiUsiJSru2tLeFmzdNJagrg0dF4FeJ3H9wbtn3rkteGbWd/+JvpNT93wYvDtqGkaG4mWx0LMDYZp+9aknLGo0nR4aKMa9Eq2Mi9I6Np+6bOwbAtS0Xu3h8/1tuDlP2CIwfiQtLZNbMNl4tS1mNT8WPWk6SAx5PHOktXQ5zmnS1YBbxYQ4KGmW0DRoFZYMbdNzeiHyJSXiNfaZzm7rsaeH0ROQj6TENESmlU0HDga2Z2m5ltqXYHM9tiZlvNbOvIHr0gEWkWjQoap7j7c4DfBc4zsxctvYO7X+7um9198+qhtcvfQxGpqiFBw90frnzdCdwInNyIfohIecseNMysx8z6Fr4Hfge4e7n7ISIHpxHZk8OAG20+yd8GfNbd/yk9wstVS14s2xR40+G9YVtW1fmKt70gveaZF381bPvSn78sbGtJJj4UbdCbLaufmInHYHVXvJz8Z3vy+RTHDMTjly0nL6oMns2LmErO25VUOZ+ZzcevPVlSvnM0rj4/MhXPDTmiN577AfnckdUz8XyevlXxc3MqmTcCydyREn9fyx403P3nwLOX+7oicmgo5SoipShoiEgpChoiUoqChoiUoqAhIqWsiKXxba3GmqDCd0dBxeeo+jLkKbpsKfqG/jyV9m8Xnxm2nfyuG8K2Wy95VXreTEdSjby1pSdsy6qYb9oQL1GHfMPqbPz6u4bS8/YnaeDDV3eGbVl5gFP68lnFWYr92PVxajlLDxdV+O5Lfs+p6eobnkP+nO5OUu8AIxPVSyGoGrmI1I2ChoiUoqAhIqUoaIhIKQoaIlKKgoaIlLIiUq5z7kwGKaiiKt3ZqshHJ+LVi8d0xmnKrII3wESSvfrC/3hF2PbGv789bLv0Vc9Kr5mlTh8ZnQzbBjri9J0VlBvP0o2ZWx8ZSdtP2xinR7NNle/cGVeX72vPn+odbQNhW1bxfnwqTo1mjwnkVcWzZ3X2lC/6exger/7cnSmxjFyvNESkFAUNESlFQUNESlHQEJFSFDREpBQFDREpZUWkXA0LV+EVJYqypGFPkobLVs8WbXzc2R6vNOzuiK/58T/YFLb93iW3pNf88rsPbtPprKBuS8HKR/f42KyQ7/OPyFfPdiTjl/V3bWdH2DY5G6c3IV/luW9/nFZtS44r2ow5S8lmz7Hsmtlq56xPZTbz1isNESlFQUNESlHQEJFSFDREpBQFDREpRUFDREpR0BCRUuo2T8PMrgTOAHa6+wmV24aA/wNsBLYBr3X34aJzOR7m/fdP5/n31Unl5h8+Oha2ZbnwPZPxpr8ATxroDtuy+QBZBeobL3hhes3zr78zbHvnqceGbet74rkNxN0B8k2e25L5FPeOxOMO8NyueB7H+FT8eK9LfpexqbxKd/a4ZFXDf/Bw/PQ9fl1/es1sjtF0Mra7kpIOewvKNkR/D0VlEBar5yuNq4CXL7ntQuAb7v404BuVn0VkBalb0HD3W4A9S24+C7i68v3VwCvrdX0RqY/l/kzjMHffDlD5uj66o5ltMbOtZrZ1ePfuZeugiOSa9oNQd7/c3Te7++bBNWsa3R0RqVjuoLHDzDYAVL7uXObri8jjtNxB4ybg3Mr35wJfWObri8jjVM+U67XAqcBaM3sQeB/wfuA6M3sT8ADwmlrONTI5w00/3VG17TUnbEiPnU6WaPevilNp2RLsLB0LeVqwdTo+NisInVXhBrjw9KeFbed+5Fth2yfPOyVsK6pQ3Zkswx7fH4/Bgdk8l3v/8ETYduLRq8O2uaS/j0zsT6/5FOLq8yPjcYp9ei5OjRZVBt+2dzxsG0iem9lzerY1v+YDI9XHNkufL1W3oOHu5wRNp9frmiJSf037QaiINCcFDREpRUFDREpR0BCRUhQ0RKSUFVGNfKCjjZcdu65qW/EmxXF6byppyypJT8zkKcMsJduS9LejJV6JWZQS61sVP5Tn/ad48+g//tRtYdul5z4nveaBZGPkviRlWCSrnp6uEk7GaGBVfE7I06NZdfTjBvvCtqI9ldd0xn1qs/j/82w1787xeLNvgMGO6tcsmkawmF5piEgpChoiUoqChoiUoqAhIqUoaIhIKQoaIlJKmnKtFAIOufvScn51MevO6IHqBVN7OvKCsVmB22wVpyfpsr5k42iAH+4cDdt+c12comtti9NeWaoR8lTuvsk4RfzRN24O2976ie+n17zyrc8P2+aSsrnfeWBfet7Tjokf09VjcSo3G6Ox6bzgbja8u8aSQr4H4hWw67s602t+/idxOZnj1nWFbUf0xinXJ/XHq3UBHhqtvtp3Jlmtu1TRPI3bmC+aXG1IHXhKzVcSkV8LadBw92OWqyMisjLUPCPUzAaBpwGPveaqVBwXkSeQmoKGmb0ZeDtwFHAH8Hzgu8BL6tc1EWlGtWZP3g6cBNzv7qcBJwKP1q1XItK0ag0ak+4+CWBmHe7+E+Dp9euWiDSrWj/TeNDMVgOfB75uZsPAw/Xrlog0q5qChrufXfn2IjO7GRgA/rFuvVqiBehsq567z+ZTQL7kuSeZb9HTGbdlG/ACnHR4XDE7W8mfzRWINsD+xXnj9o1DcV4/GleAj7355PSaf3r9XWHbxa/6jbDtxRvj8YF4+TbAQLKh9yP74mXhswVPlEdH4/kWfR0HV0FiumBt/KueedhBnXd1siF1UQmFaDl+a0vt8zxruqeZfWbhe3f/V3e/Cbiy5quIyK+NWsPLL/23YWatwHMPfXdEpNmlQcPM/szMRoHfNLN9lX+jzG+nqN3RRJ6A0qDh7n/j7n3AB929v/Kvz93XuPufLVMfRaSJFC1Ye0Ylvfo5M/uVgpHufnvdeiYiTanoY+F3AluAD1dpczQjVOQJp2jB2pbK19OWpzvVtbW2cHh/9WXGBcXI0yXjQ5Nxai9Lfx7ely95zqpXe5L6yzadLtyMOdkg+uTWuMLBYG88BvsP5FXXP/n6+LPw93zpx2Hbf3vJsel51w/EKeJMlhrtbOtNjx3sidOYWdo+K72QVbQvOm/2XJhOqsB3F6SHB7qr96n9UFcjN7NOM3unmd1gZteb2QVmlv7lmNmVZrbTzO5edNtFZvaQmd1R+feKmnsqIk2h1pTrp5lPu34E+DvgeOAz6RFwFfDyKrdf6u6bKv++UmtHRaQ51DrV7enu/uxFP99sZj/MDnD3W8xs48F2TESaU62vNH5gZo/VdjOz5wH/dpDXPN/M7qy8fRmM7mRmW8xsq5ltHd696yAvJSKHWq1B43nAd8xsm5ltY76WxovN7C4zu7PE9T4GHAtsArZTPSsDgLtf7u6b3X3z4Jq1JS4hIvVU69uTap9NlObuOxa+N7NPAF86FOcVkeVT6yrX+wHMbD2/XO7vgTIXM7MN7r698uPZwN3Z/RfMzDm7RquvLM1WPUK+Ce8Do+NhW5a+e7RglWu2SrNrVZaOjc85PB6vwgToT1Y+Zqtys/Rd0QrirPr3X7z0uLDtrL/+anrem//nGQd1zUfG4lWuRSnr9f1xmnfswEzYtjsZ26JNsP/f3rGwLXsOZeft78pTp3snqv8uWfp3qVrL/f0e828ljmB+3cmTgXtYspBtyTHXAqcCa83sQeB9wKlmton5iWHbgLfW3FMRaQq1vj35K+brgv6zu59oZqcB52QHuHu19itK9k9EmkytH4ROu/tuoMXMWtz9ZuY/zBSRJ5haX2mMmFkvcAtwjZntBOI3eiLya6toletTgcOAs4D9wDuA1zH/mcYf1713ItJ0it6eXAaMuvu4u8+5+4y7Xw18Bbio7r0TkaZTFDQ2uvuvTN5y963Axrr0SESaWtFnGtlK1nhb60Osq72V44/qr9pWtKA3q858ek9cDTpbmrxhMP/Vu5Jl6nv3xx8F9XTEczg2rM6X42fVtvuTuSzZkvoiWW5/Lmn71sVnpudtb4v7lF3zWUcMhG2jybgD9CXV59uSJe7Z0viix6w7mbPTm/Qnm1tTNN1iw+rqz4VszJcquuetZvaWpTea2ZuY31FeRJ5gil5pXADcaGav4xdBYjOwivkZnSLyBFNUuWsH8ILKZK4TKjd/2d3/pe49E5GmVOvak5uBm+vcFxFZAQ7+UzAReUJS0BCRUg5uZ9tlNj07x4691ZcgZ0ulIU8pbh+Ol1JnacqiTXYHuuJhXZWkttKl8RPT6TVXJWnBrKr4TJK+K1ounY19duxIwe+yti9eFv7C98XL6r950cvCtsnpvLL6+FScks2GYSI57tF9eQmFLJWbjd/eZPyetLY7veb4VPVxKLEyXq80RKQcBQ0RKUVBQ0RKUdAQkVIUNESkFAUNESllRaRcZ+ec4bHq1bi7k5WhAEmBau4bGQ3bNkzHK1mn5/KU67bhuH2gI07lrmqJY3hLQWo5S+Xeuyf+PY/qjVN0U7N5mtKSNcYPjU+EbdkqTYD21ni16g8+EFcqf86FXw7bLn3Lyek1B7rDfbvYGaT7Ae7evTdse9ba+PcA2LU/Pm9nW/y8zqqR7x7Nq9aPTVZPEWep96X0SkNESlHQEJFSFDREpBQFDREpRUFDREpR0BCRUlZEyrXFLCy0Ole0S3FiY39v2JZtCPzQyP70vP1JSqyjPU6ltSerHot+zWwcsrRqds3O9nwD40xnWzy2RSuTs/Rxduzn3nFq2Paem/K9xj/0ymeFbWPT8UrW9qQ/2/bFG4wDHDfUF7ZNTccp0NVZoeikWDHE41f0mCxWt1caZna0md1sZveY2Y/M7O2V24fM7Otmdm/la5wgF5GmU8+3JzPAu9z9mcxvHn2emR0PXAh8w92fBnyj8rOIrBB1Cxruvt3db698PwrcAxzJ/BaPV1fudjXwynr1QUQOvWX5INTMNgInAt8DDnP37TAfWID1wTFbzGyrmW0d3rNrObopIjWoe9Co7DZ/PXCBu++r9Th3v9zdN7v75sGhtfXroIiUUtegYWbtzAeMa9z9hsrNO8xsQ6V9A7Cznn0QkUOrntkTA64A7nH3SxY13QScW/n+XOAL9eqDiBx69ZyncQrweuAuM7ujctt7gfcD11X2g30AeE3RidpajcHe6hWqs0rbAJ7MX4iWCUNevXp1Vz5/oTOZi5Hl0acKKmZnsvkfLRZXr85KCzyeuSFZ3v/B4Xyey1DwWENepbsrGduPvnZTes0X//kXw7Z/et8rwrZr7hgJ2179rHiDccg3eR7ojsdvX8Fm1pno+dditc/TqFvQcPdvE2/qfnq9risi9aVp5CJSioKGiJSioCEipShoiEgpChoiUsqKWBo/5/Gmy1GV8gU9SVrrgbF46fKsx8vJewqWH2dpwSyzlaWPi9JsR69JqqenmzzH/2/MzOY512wj7Inkd9kzmW+MvG46LkvQ1hKfN1vdfc+ufDLyLRefGbY97x3/ELZd9q7TwrbZgpx1Nn67x+PndZbyL9q0O7qmU3uJCb3SEJFSFDREpBQFDREpRUFDREpR0BCRUhQ0RKSUFZFyxT3coPbIoTjVCLB3Il7h+Yyh/rBtdU+c1soqRUOeVrWkMVulmbVBvrI2WwGbnXe6IOXanlQN72iP2yZm8vRxttl1a2t83mwMsorskK94/sbfxBUpT331fw/b/uEzf5FeM1tZOtgdr/TNqrW3FVQV7+uq/iffFNXIReTXk4KGiJSioCEipShoiEgpChoiUoqChoiUsiJSrmYWptOyVBnkqaRdE/Fqy6y4bbZqFGBkf5zmXdNzcKm0LHUM+e+ZFSweSzJtRatcs5RrUbo2kyX/kv2qmUpWjU7N5kWbZ5LVoT3t8Z/JOy/+k7Dt7Vfdnl7z8xe8MGzLHs9sdWx7Uiga4tXSRatjF9MrDREpRUFDREpR0BCRUhQ0RKQUBQ0RKUVBQ0RKUdAQkVLqNk/DzI4GPg0cDswBl7v735rZRcBbgEcrd32vu38lO9fM7Bw79k5G10n7MZ3ktH++N65GvqE/XnJfVNm6qy3Olfeuioc8y78PT+ZV17NjfzoyGrYNrEpKABTMbTiiJ15uvvdA3N/79kyk513bHVcjz+aG/DR5XIoqg5+wfiA+NpnDcPKRcXmFs897QXrNMz70zbDt3a/5jbDtuMHesO3wvs70mvePVB/7bMPzpeo5uWsGeJe7325mfcBtZvb1Stul7v6hOl5bROqknrvGbwe2V74fNbN7gCPrdT0RWR7L8pmGmW0ETgS+V7npfDO708yuNLPB4JgtZrbVzLYO79m9HN0UkRrUPWiYWS9wPXCBu+8DPgYcC2xi/pXIh6sd5+6Xu/tmd988OLSm3t0UkRrVNWiYWTvzAeMad78BwN13uPusu88BnwBOrmcfROTQqlvQsPm0xhXAPe5+yaLbNyy629nA3fXqg4gcevXMnpwCvB64y8zuqNz2XuAcM9sEOLANeGvhmSxeKlxURbmnI04prtoTH5ud9piBOOUFMHogXsaeLd/uDypFA/S05Q9Vb7LR9YbuOH3cl6SAR6byNO+qpDK4pQvcc1lytDupnt7eEvenO2mDvCzBxFScjsyqfxc9N88/+/iw7bIbfxK2XbXl+WFbVuEcYKizemmG1oLxWaye2ZNvU700QjonQ0Sam2aEikgpChoiUoqChoiUoqAhIqUoaIhIKSujGjkWppIKFi+mqz/7khWe2cbIo0FF5wXZSsO2JE2Zpe+sJ0+lZdm9wc4k7ZykGrva8821s42ae2fip9b6pNI75NXTPXnAB4N0IhSvEs6KcWfV5zta4+dJNrYApxw5FLad+OaTwrYz/ypOQH7tL89Ir9kVVPUvsf+zXmmISDkKGiJSioKGiJSioCEipShoiEgpChoiUsqKSLnOuYep06zQLORpzOm5OJWWpWoLFhKmmzVnKx/7uuLU6OSBvPBrW7Iz8lSSMsxWRRZtCTyXjNGMx22dSdoZ4Pvbh8O2J62Nixlnj2eWGoV80+ksHTuXpICnpvONwrM0cHeQGgU442UnhG1vu/YH6TU/8frnBi2151z1SkNESlHQEJFSFDREpBQFDREpRUFDREpR0BCRUhQ0RKSUFTFPA+I5A0UVn7P27mSj5mzT3/0z+ZyJbJPnrD/Zsu9sPgBAWzIPYe9UPG+kI5kzUfR7ZnMf2iw+73179qfnfflT14dtu0fjuQ17knkPM8kcDoCWlrjCfHsyB2aoK16On5UOANi2L94IuzcpzXDuc+PdTY86/anpNc+67FtVb79/51h63GJ6pSEipShoiEgpChoiUoqChoiUoqAhIqUoaIhIKZal+ZqFmT0K3L/oprXArgZ1pxr1J9ds/YHm61Oj+/Nkd19Xyx1XRNBYysy2uvvmRvdjgfqTa7b+QPP1qdn6k9HbExEpRUFDREpZqUHj8kZ3YAn1J9ds/YHm61Oz9Se0Ij/TkPows8OBy4CTgClgG3CBu//sEJ3/VOCAu3/nUJxPGmOlvtKQQ8zMDLgR+Ka7H+vuxwPvBQ47hJc5FXhBcP0Vs3jyiU5BQxacBky7+8cXbnD3O4Bvm9kHzexuM7vLzH4f5l81mNmXFu5rZn9nZm+ofL/NzP7SzG6vHPMMM9sI/CHwDjO7w8xeaGZXmdklZnYz8EEzu9fM1lXO0WJm95nZ2uUaAKmNorssOAG4rcrtrwI2Ac9mfi7BrWZ2Sw3n2+XuzzGztwHvdvc3m9nHgTF3/xCAmb0JOA74bXefNbMR4HXMv0X6beCH7t5McykEvdKQYr8FXOvus+6+A/hX5j/zKHJD5ettwMbkfp9z94XCHVcC/6Xy/RuBT5XvrtSbgoYs+BFQbSedqJLMDL/8/Olc0j5V+TpL/op2fOEbd/93YIeZvQR4HvCPWYelMRQ0ZMG/AB1m9paFG8zsJGAY+H0za6183vAi4PvMT+s/3sw6zGwAOL2Ga4wCfQX3+STw98B1i16BSBNR0BAAfD73fjbwUjP7v2b2I+Ai4LPAncAPmQ8sf+ruj1ReFVxXabsGyPcDnPdF4OyFD0KD+9wE9KK3Jk1L8zSkqZjZZuBSd4+CijSYsifSNMzsQuCPmM+gSJPSKw0RKUWfaYhIKQoaIlKKgoaIlKKgISKlKGiISCn/H+D2VLWuOVZ7AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig, ax = plt.subplots(1,1)\n",
"plt.xlabel('Country')\n",
"plt.ylabel('Capital')\n",
"ax.matshow(c_new, cmap=color)\n",
"print(\"Confusion Matrix\")\n",
"fig.savefig('figures/phase2/capital_country.pdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Calculate F1 Score**"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"F1 Score: 0.9655172413793104\n"
]
}
],
"source": [
"vfs2_y, vfs2_y_score = get_mx(c_new)\n",
"ypred = cross_val_predict(svm_clf, vfs2_y_score.reshape(-1,1), vfs2_y, cv=kf)\n",
"from sklearn.metrics import f1_score\n",
"t2_f1 = f1_score(vfs2_y, ypred) \n",
"print(f\"F1 Score: {t2_f1}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Validation of Directors / Movies"
]
},
{
"cell_type": "code",
"execution_count": 216,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.df_movies = pd.read_csv('data/out/t4_movies.csv', sep=';', index_col=0, encoding='utf-8')\n",
"#df_movies.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Map the truth value function to the matrix**"
]
},
{
"cell_type": "code",
"execution_count": 217,
"metadata": {
"scrolled": false
},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.\n",
"#d_new, d_u_new, d_mold = get_truth_matrices(df_movies)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Confusion matrix\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQ0AAAEGCAYAAAB/1cDLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAH71JREFUeJzt3XuU3HWZ5/H30/dr+pLudO4EAggoEkxQd/GCF/AyF0TFkVUHd1hx5+iOuLq7LLOr6Myc9XiQ0T27uoMrTthRFFcR3GEcER1RB9GAENCoYAghF5J0dzrd6Vv68uwfXQlt29/nV7+Q6qqOn9c5Od1d3/7+fk/9qvpJVT3fi7k7IiLFqip3ACKyuChpiEguShoikouShojkoqQhIrkoaYhILosqaZjZa83sl2b2uJldW+54jjKzHWb2iJk9ZGZbyhTDzWa238wenXVbp5ndbWaPFb52VEhc15vZ7sL1esjMXr/AMa0xs++a2TYz+5mZva9we1mvVxBXWa/Xb8W5WMZpmFk18CvgYmAX8BPgCnf/eVkDYyZpAJvcvbeMMbwMOAzc4u7PK9z2caDf3T9WSLId7v6fKiCu64HD7n7DQsYyK6YVwAp3f9DMWoEHgDcA76SM1yuI6y2U8XrNtZheabwQeNzdt7v7EeBLwKVljqliuPu9QP+cmy8FNhe+38zME3BBJeIqK3ff6+4PFr4fArYBqyjz9QriqiiLKWmsAp6a9fMuKueCOvAtM3vAzK4udzCz9Lj7Xph5QgLLyhzPbO81s62Fty8L/rbpKDNbB5wP3E8FXa85cUGFXC9YXEnD5rmtUt5bXejuLwBeB7yn8JJc0j4DrAc2AHuBT5QjCDNrAb4KXOPug+WIYT7zxFUR1+uoxZQ0dgFrZv28GthTplh+g7vvKXzdD9zOzFupSrCv8D756Pvl/WWOBwB33+fuU+4+DXyWMlwvM6tl5g/zC+7+tcLNZb9e88VVCddrtsWUNH4CnGFmp5pZHfBW4M4yx4SZNRc+tMLMmoFLgEfjXgvmTuDKwvdXAneUMZZjjv5hFlzGAl8vMzPgc8A2d79xVlNZr1cqrnJfr7kWTfUEoFBq+iRQDdzs7n9V5pAws9OYeXUBUAN8sRxxmdmtwEVAF7AP+DDwdeA2YC2wE7jc3Rf0Q8lEXBcx81LbgR3Au49+lrBAMb0E+D7wCDBduPk6Zj4/KNv1CuK6gjJer7kWVdIQkfJbTG9PRKQCKGmISC5KGiKSi5KGiOSyKJNGhY26BCozJqjMuCoxJqjMuCoxpkWZNICKu5BUZkxQmXFVYkxQmXFVXEyLNWmISJksinEaHZ1LfeXqU479fLC/l47OrmM/11TPNy1lxsTUdLINoLY6nTenM67N9KxDz42puiodE0BVkK6npuLzVgXHtjlNfb29LO16Jq6JyfSxs+5vdN6arPs7K7De3gN0dXUf+3lyOn6MpoOwss57ZDJ97LnPm/6+XjqXPnOtpoITV8290HM8mz+r2Yc+2NdLx6yYskRxRX/ru3ftpL+vN75TBTVFR3MCmdlrgU8xM7Lzf7v7x6LfX7n6FG79++8l25ctqU+27T44Fsayor0h2TY2MRX2PTw2mWxra6oN+zbVVSfbBkYmwr5LGtPHrg0SKMTXY2Q8fX8AWhrST5f25rqwb3R/Dw4fCfuOHkk/Dlnnfap3JNm2rC39vJmJK/04NNen7w/AZJD4s5JzQ2362FEiy4prPEigb7zkJeFxZ1vwtyeFxXT+JzMzQs8BrjCzcxY6DhE5PuX4TEOL6YgsYuVIGkUtpmNmV5vZFjPbcrC/bKvoicgc5UgaRS2m4+43ufsmd980+wNGESmvciSNil1MR0SylSNpVORiOiJSnAUvubr7pJm9F/hHnllM52fP5pjf2JZej6SzMS7JLWlMX4LB0bgEuWdoNNl2WlVz2Dcqq+0ZiMvENcHYkqyS69NBzBMZ40Mi0XgIgM6W9OMwPB6XtgdH06XPqPwMMDaVPnZW+XJoPH3ewbG4LD40kW5vrY1jXtKQbs8aBtA/nG4bmUw/n8czxjPNVpZxGu5+F3BXOc4tIs+OhpGLSC5KGiKSi5KGiOSipCEiuShpiEguZame5GUG9UGJ8oKVncm2f/H+28Jjf/+GNyfbopmKAIeDstrYRFzCimay/nTfQNi3LSgzZs2+PDA6nmzbPZhuAzi3e0mybWlGaTuaEXxwJJ7lunc4XSaur4n/33t84HCyrS0obQJsefpQsu2Mjqaw745D6Zj3H453gHzp2vRWrXtH0scFaKyJH/+UyYzy82x6pSEiuShpiEguShoikouShojkoqQhIrkoaYhILkoaIpLLohinUVdTxerOxmT7zmDF6ds/+ofhsb/0aHpa/WVn9YR9o8p21tYQ48EU5+Zg5W6Ahtp0rq/LGLfQWZ9egXugLl4K4NCR9HiKrqaM1cjrj38JgsGMVdIjT/Snx55sXB73bW9IPw67D8fLF0RLFKxpj6/VrsPp53PfSHwtlrek2+qCJRXiZ/Nv0isNEclFSUNEclHSEJFclDREJBclDRHJRUlDRHJZFCXXvuEjbN7yZLL9956Trp198/F4d7arNq5Jtv1kT1/Yt6k2ffmizZIBulvTpc8lvXHfaPXuqLQJsC+YWr1uSTzde017uj2rxHwgmHafNZ3//J72ZFvWdb70Oemyedbm0RuWpc+79UC8fMFEMNV8bWt8nRuq09fj1Lb4Om/rG0q2TU6ly7VRvHPplYaI5KKkISK5KGmISC5KGiKSi5KGiOSipCEiuShpiEguZRmnYWY7gCFgCph0903R7zfWVrOhuy3ZHu1Yvml1axhLtAT+RacuC/t+4vtPJNs2rkwvQw9wJNilu6YqzuVtTeml92uq4l3jOxvS40Na6o7/6ZA1JT8aBlATTtmOt4Oozri/E8F1jnvC/uH02JIldfH2B7XBYxi1QbwNwehkvGv8mtb0EhJjU+m+dRkxzVbOwV2vcPd45JWIVBy9PRGRXMqVNBz4lpk9YGZXlykGETkO5Xp7cqG77zGzZcDdZvYLd7939i8UksnVAD0rV5cjRhGZR1leabj7nsLX/cDtwAvn+Z2b3H2Tu29q7+xa6BBFJGHBk4aZNZtZ69HvgUuARxc6DhE5PuV4e9ID3G5mR8//RXf/ZtShrrqKNUvT04m/98SBZFtrMH0dYHA0vXt7Y8aq4G88J12S/eedcWHootPSfbNijqqMWVPUlzc3JNsOjMS7xkelws6WeJp5Z3O6RDkU7CgPMDAWrILeGp+3byx9nzqPxH0tuM4rmtOlTYChI+n7NB6UPiEuuY5MxtequzFdUj8yHZX5swrQs3636N88Qdx9O3DeQp9XRE4MlVxFJBclDRHJRUlDRHJR0hCRXJQ0RCQXJQ0RyWVRbGFQU21hPX7jivQ09F/1ppd0B6ivTdfEo60CAIYn0jXzC1YtDfv+8S0PJNuuveSMsG+0FMCzmSr+2MHhsO8pbenrkbWVQDTmJZr6DnBwPD1O49BIepwNwP5g7Mkpbc3Hfd72+niMx+GJOK5IQ036Oo9kTI1vC8KKxtlYNChlDr3SEJFclDREJBclDRHJRUlDRHJR0hCRXJQ0RCSXRVFyHZ+Y5tf70uXAw8HU6q37DofHbgpKrll+fiAd09Jg1W+AT77x3GTbaz4arhTAN667JNmWtfn31t5Dybb9h+My4dJgFfRoiQGAkfH0Y9Q7Gk/Jf6wvvdN9W11c+txxMH3sszvjaea7g53udw7EMQ8F5frpjOULTulIP3f2DcXXeUdd+lpFMQ2OF18i1isNEclFSUNEclHSEJFclDREJBclDRHJRUlDRHJR0hCRXBbFOI2aaqMjWAJ/MhiccFZ3eusDgO5gSf8sp3ema9vRrvAAK4Ml8G9630vDvjsG0+NDXtQWT8lf2ZK+vxNT8fiBlmBMS23Gzu/dS+JxK7H08gar2uKtBM4KtjBoqY+f/qd1pJ87NRlTyfuDafWDY/H09uctbUu2dTeNhX3bgt3so60Tbs0xXkmvNEQkFyUNEclFSUNEclHSEJFclDREJBclDRHJZVGUXKvMaA7KY+PB6txrW+OSa3sw3XtsIi6NLW9Kl/uWt8el3Khgt7Iljjk69v07+8K+Z3a2JtsaquOyW1t9+lpFJXGAhtr0/09twWMA8NI1ncd1XIhLkEsyzts+km6PrgXEO86vj5YMJ75P3cSl62jH+WlP/w3VBCuVz1WyVxpmdrOZ7TezR2fd1mlmd5vZY4Wv6b0HRKQilfLtyd8Cr51z27XAPe5+BnBP4WcRWURKljTc/V6gf87NlwKbC99vBt5QqvOLSGks9AehPe6+F6DwdVnqF83sajPbYmZbensPLFiAIhKr2OqJu9/k7pvcfVNXV3e5wxGRgoVOGvvMbAVA4ev+BT6/iDxLC5007gSuLHx/JXDHAp9fRJ6lko3TMLNbgYuALjPbBXwY+Bhwm5ldBewELi/mWEcmp9nVn16afc9guu2OX8Sfh7xqfbrqOxZMJQa4+5dzP+d9xpvP7Qn7dgS7jn/pZ0+Hfd927opk27ol8U7or/yv/y/Z9obXPy/su64zPS7l3J74vN2N6bElvxpIT30HeGBXeimA158RLwXwpYf3JtsuCabNA/xwx2CybVlrPNZi98H0FPa6YCwFwJpgC4O9g+kp9wDNdeljTwRLSPRnbEExW8mShrtfkWh6VanOKSKlV7EfhIpIZVLSEJFclDREJBclDRHJRUlDRHJZFFPjG2qrOH15y3H1/YPnxKNJV7SmS4FZu3t3b0iXxlYuiVfJrqtJ5+uXn9oenzdY2Ttr1/jb//w1ybY/3bwl7Puv3vGCZNuyYJVziKe/NwVlQoCmoESZdZ1fc1a6JLtxRTzJujWYVl8VL0bOc5ellzeozZiG3tGQLuceGI3LxD1BaXsqeD7/fUPxqSDzlYaZVZnZW4o+ooic1DKThrtPA+9dgFhEZBEo9jONu83sg2a2prCQTqeZpZdTEpGTVrFvZP6k8PU9s25z4LQTG46IVLqikoa7n1rqQERkcSgqaZhZLfCnwMsKN/0T8DfuXvwsFxE5KRT79uQzQC3w6cLP7yjc9m9KEZSIVK5ik8YF7n7erJ+/Y2YPlyKg+UxOO31D6fp0/0h6uvAjB9LTmwEmptPbH0x5vPP7o/vTU7ZfHG5SAL3BtOzHetNT/QFWt6THJkTjPwAe7U1fj5e/YHXY9/1ffCjZ9vE/Oi/ZlmX7QPo6Avz4qfTU+RXN8TiNbwfLF6zJ2CriR0+lr9Xy1ngLg6cOpZ+TR4ItNwDWBlPjnxqIp8Yvb00/d2qCwSUjGdt1zFZs9WTKzNYf/cHMTgOKP4uInDSKfaXxH4Dvmtl2Zvb5OYVnKioi8juk2KTxA+AM4DnMJI1flCwiEaloxb49uc/dx919q7s/7O7jwH2lDExEKlP4SsPMlgOrgEYzO59ntiBdAsSfIonISSnr7clrgHcCq4EbZ90+CFxXophEpIKFScPdNwObzexN7v7VBYrpt1SZ0RxM3V3Wmi5Rre/I2IG9OT2VOGOWeWjFkniq+Or2dKlwRbAbPcCKjnT7REY577zutmRbZ2NcRrzsnPQyAzf+0/aw739/47nJttba+P+uF69dkmzrybjObwpWhe9uiXdg37AyvcL6qub4eXV212SybTJj/YKGoGx+XnJPwhlLG9PT6ocn0jF9vjZenmC2Yj/T2GhmxxZ5MLMOM/vLos8iIieNYpPG69x94OgP7n4QeH1pQhKRSlZs0qg2s2Ov5cysEYhf24nISanYcRp/B9xjZp9n5q3+nwCbSxaViFSsYqfGf9zMtgKvZqbs+hfu/o8ljUxEKlKehYW3AZPu/m0zazKzVnePN+EUkZNOUZ9pmNm7gP8L/E3hplXA10sVlIhUrmJfabwHeCFwP4C7P2ZmYcXYzG4Gfh/Y7+7PK9x2PfAu4OhW7te5+11ZJ3ec6aC2XVOdzn3LgiXdIV5aP8uRyfSxo5gAaqvT05Sztk6IrkVTfVxvbwzq8VnjJZpq0u2fvCw9DgNg90B6yvbq9njMQ9WhdN+MnQQYnUpPxs66VtF4mWiaeZamjDERNZZ+7ox7PLl8Yio9Tmd0Mt03a+uL2Yqtnoy7+7GJ/GZWQ/bYp78FXjvP7X/t7hsK/zIThohUlmKTxvfM7Dpm5qBcDHwF+EbUwd3vBdIroIjIolRs0riWmbcUjwDvBu4C/stxnvO9ZrbVzG42s3iLKxGpOMXssFYN3OLun3X3y939zYXvj2dqxmeA9cAGYC/wieC8V5vZFjPb0tfbexynEpFSKGaHtSmg28zSM2GK5O773H2qsGvbZ5n5cDX1uze5+yZ337S0q+vZnlpETpBiqyc7gB+a2Z3AsVVg3f3GZI95mNkKd99b+PEy4NE8/UWk/IpNGnsK/6qA1mI6mNmtwEVAl5ntAj4MXGRmG5ipvOxg5vORTO4wMZV+N7S993Cy7Y5fHEi2AZy/Kj39ef/heFuXp4fS7a9eH39c82RQRtwzGK84/ab69MMWXSeAn+4fSLbd+9jBsO/SYAmCl6xLT18HWLekJdm28Z2fCfu+7o9ekWy7YuOKsO9tDz6dbGsOSsgAn//xrmRbe3P8wjt6916dUY7vaU0f+5d74/GUm9a1J9uiUv5IUI6dq9hh5B8p+ojP9Llinps/l/c4IlJZspb7+6S7X2Nm32CecRnu/ocli0xEKlLWK43/U/h6Q6kDEZHFIWu5vwcKX79nZt2F7+MPCUTkpBZ+ImMzrjezXmb2OvmVmR0wsw8tTHgiUmmyxmlcA1zIzF6uS929A3gRcKGZvb/k0YlIxclKGn8MXOHuTxy9wd23A28vtInI75isD0Jr3f23xnC7+wEzO/455TmZQTQTOZrWm7WLekdDuiZel1FPPzCcXhK+oTqe/hyNp8gaazGesU3B8Z53eDx9fwBagi0OsuYUtAZbUHz8Q28K+/7H629Ltl15wZVh3+h589RQeqwMxM+d07ribSZ29I8l2/qHxsO+y5ekn5NNwRgdgJpgyYWB0WhqfPGzQrJeaUSjjOIRSCJyUsp6pXGemQ3Oc7sB8eo2InJSyiq5Fr/tkoj8Tih2PQ0REUBJQ0RyUtIQkVzy7HtSVtVB7aytPl0KXNcR7x7ZHvSdmI5Lm6va0qWx+oyS60vXLE22/WhPvLRqVL48MByX83pa0jE/Z0W86kG023nWLupRSW99e3p5AoA/ePvFybbvPhFP51/TlT72mR3p6foAe3rSBcKG2vj/29ODkuzDY/GSC+f2pGOuzxgGcHpH+ryjremSa2PG0ITZ9EpDRHJR0hCRXJQ0RCQXJQ0RyUVJQ0RyUdIQkVyUNEQkl0UxTmN6GobG0tO2o+XixzKmkVdbevxH1i7qtVXpOr5nTBbfczg9LXtoPF5OfioYL7GsJZ5H+OTgcLIta3b02vb0mJfxYHd2gKpgr63hyXhK/tql6TEgb3luvIXBq/7z19N9/+rSsG9DMHahszEeh/PzfenHd3l7PK1+PNj5PXi6AnBwPD0G5Mn+9BiesckTNzVeROQ3KGmISC5KGiKSi5KGiOSipCEiuShpiEgui6LkahavDN3WlJ7evrYtnhrf1pDu25xRch2aSJcKu1vi8y5fki6NjmSUIJcEq4JnrSi+ojld7qtbFZcRO4JlBLqb4/vbHEznX9saT6ufWp0uBzbWxTHf8P6XJ9seH4h3YF8TLH3Q3RiXtmtWHP//xx31wQr5S+PjNgZLMrT2pB+Dipgab2ZrzOy7ZrbNzH5mZu8r3N5pZneb2WOFrx2likFETrxSvj2ZBD7g7mcDLwbeY2bnANcC97j7GcA9hZ9FZJEoWdJw973u/mDh+yFgG7AKuBTYXPi1zcAbShWDiJx4C/JBqJmtA84H7gd63H0vzCQWYFmiz9VmtsXMtvT1/dYmbyJSJiVPGmbWAnwVuMbd59t4aV7ufpO7b3L3TUuXdpUuQBHJpaRJo7Df61eBL7j71wo37zOzFYX2FcD+UsYgIidWKasnBnwO2ObuN85quhM4umvvlcAdpYpBRE68Uo7TuBB4B/CImT1UuO064GPAbWZ2FbATuDzrQNVVFi7bPxnshH5GR7wsf7QLd7RtAsCpnl5qPmv8QH1tuv309jjm5vp035bgOgFUBXOrs8altAfjQ7Lub3QlO4NtFSDe6T7r/p7duSTZtjpY7h/gI3c/lmz7wEtPDftG22rUZ4yJiMYkRdcC4uds9HRuqCl+B9aSJQ13/wHp58qrSnVeESktDSMXkVyUNEQkFyUNEclFSUNEclHSEJFcFsXU+MNjk/zg8b50+0R6BebP3fdUeOy3bkqvZn1wNJ5m/u1t6eHtl79gedh3MFhx/PYH94Z9P/DK9cm2rNLZlx95Otn2q92Hwr7nrElPSD40ml6ZHeBVZ6b79o3E1/knO9JxXfb8eWchHPM/7tmebPuzV6evI8DgSPo+vejdnw/7bnzl+cm2+qDMD7CqM71UwJP7D4d9m4IS9PBo+u9k58GR8Liz6ZWGiOSipCEiuShpiEguShoikouShojkoqQhIrkoaYhILhbtuF4pnr9ho3/jnh8m2w8HO8r/uj+ua5/ZlZ6GPj4R7zj/5KH0Duzr2tPT5iGe4vzEQBzz2d3p6d7R9gYAj+5Nj3nYPzIW9l3Vkh4/0DeW3pEcYMOK9DiNoWD8AMATwXU+t6ct7Hv3r9NrPG1aHi+E3x/cp+mMP5s//8ojybarLo7Hh5zZmX7uHBiNr3Okvjr9GuG6t72eX//84Yw96WfolYaI5KKkISK5KGmISC5KGiKSi5KGiOSipCEiuSyKqfFmUBOUi4aCKex7DsdlxPZgh+6sEuTj/aPJtqaa+NIOjKenXW/dF5dc13e0JNv6D8dT1HcOpadA3/v4QNj3rJ70sc/tiUvMUVl1R1BSBfjhk+ky8armeEXx+55I931BT3vY9x8e60+2TUzG5fiXnb8q2Xbtn30i7PvfPvXvk20P746fGxecki7Hbx9KP59HM+7PbHqlISK5KGmISC5KGiKSi5KGiOSipCEiuShpiEguShoikkvJxmmY2RrgFmA5MA3c5O6fMrPrgXcBBwq/ep273xUdyx0mp9J15IHx9BiAfUPxtOu2+vS4hYwNutnelx6ncXZXPG6hu7Eh2bZv8GDYd2g8PS6lKdiNHuDgSHrrhPEj6TaA/mA8zMhk3Hf/cHpK9/BEvIXBweH0Y/jrQ/G4hV0H0u3RWBmIx2LUZuz8PjyWjvnSa64K+37yaz9Ptp333J6w733b0+NS6uvSz42J4O9rrlIO7poEPuDuD5pZK/CAmd1daPtrd7+hhOcWkRIpWdJw973A3sL3Q2a2DUgPkxORRWFBPtMws3XA+cD9hZvea2ZbzexmM4uXTxKRilLypGFmLcBXgWvcfRD4DLAe2MDMK5F5B+Kb2dVmtsXMtvT3HZjvV0SkDEqaNMyslpmE8QV3/xqAu+9z9yl3nwY+C7xwvr7ufpO7b3L3TZ1Lu0sZpojkULKkYWYGfA7Y5u43zrp99o7LlwGPlioGETnxSlk9uRB4B/CImT1UuO064Aoz2wA4sAN4d9aBpt0ZDcqBI5Ppkt3Og/H09lM706XP7X1x38GRdFlt8Ehc6nXS9dydvXEZcSg4dl1V/P/AzoPp0uf+gXQJGaCxLv106R+tD/u216WXIHhgdzw1/ulgR/OxyXg18oGB9GOY9Rg9/vRgsq2nPZ6S3zeYPu/M/6dpF25anWz72qe/HPa95F+/Mdk2HWwMnzXVf7ZSVk9+AMx3dcIxGSJS2TQiVERyUdIQkVyUNEQkFyUNEclFSUNEclHSEJFcFsUWBtPTHu4MP+3pMQ9jE/GU7bpga4S1HemxBQDf+0V6XEPWVPGepvT4kEND8c7g0bG7Mvb97j98/LuOR9PBm2rj/3+6mtLjOJa1xDvdbw/OW1Md3+EjwfieyYyt35vq038e0VINAL3B9hYtLfHzavxI+rxnv/bisO+3vvydZNsr3/yKZFvGKhC/Qa80RCQXJQ0RyUVJQ0RyUdIQkVyUNEQkFyUNEcllUZRcq6qMloZ0qE3D6bb6mnh17khGRY4zV6anZa9uiadOf/qfn0y2nbU2XgFxTUtTsm0sY4rz6s50XFnl2Ki8WVcdX+fhI+mS+UM70ytoA7Q2xiXZSHd3elX49vq49FkVVHM7muOlAJZH580ouUZLEJyyMr0rPMAF7/79ZNt3frwz2TYerHA/l15piEguShoikouShojkoqQhIrkoaYhILkoaIpKLkoaI5GIeTCuvFGZ2AJg9sKEL6C1TOCmVGBNUZlyVGBNUZlwLFdMp7l7UrmSLImnMZWZb3H1TueOYrRJjgsqMqxJjgsqMqxJj0tsTEclFSUNEclmsSeOmcgcwj0qMCSozrkqMCSozroqLaVF+piELx8wc+Dt3f0fh5xpgL3C/u6dnR6WP92+BEXe/5cRGKgtlUcxylbIaBp5nZo3uPgpcDOw+3oO5+/86YZFJWSzWtyeysP4B+L3C91cAtx5tMLNOM/u6mW01sx+Z2fPNrMrMdphZ+6zfe9zMeszsejP7YOG29Wb2TTN7wMy+b2ZnLei9kuOipCHF+BLwVjNrAJ4P3D+r7SPAT939+cB1wC3uPg3cAVwGYGYvAna4+745x70J+HfuvhH4IPDp0t4NORH09kQyuftWM1vHzKuMu+Y0vwR4U+H3vmNmS82sDfgy8CHg88BbCz8fY2YtwL8EvmJ2bLWbeGUbqQhKGlKsO4EbgIuApbNun299KwfuA043s27gDcBfzvmdKmDA3Tec+FCllPT2RIp1M/BRd39kzu33Am8DMLOLgF53H/SZstztwI3ANnfvm93J3QeBJ8zs8kJfM7PzSnwf5ARQ0pCiuPsud//UPE3XA5vMbCvwMeDKWW1fBt7OnLcms7wNuMrMHgZ+Blx64iKWUtE4DRHJRa80RCQXJQ0RyUVJQ0RyUdIQkVyUNEQkFyUNEclFSUNEcvn/7Tw/gOfN2EIAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig, ax = plt.subplots(1,1)\n",
"plt.xlabel('Movie')\n",
"plt.ylabel('Director')\n",
"ax.matshow(d_new, cmap=color)\n",
"print(\"Confusion matrix\")\n",
"fig.savefig('figures/phase2/2_confmx_movies.pdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Calculate F1 Score**\n",
"\n",
"F1 Score was not so high, since the results for this task has a high false positive rate (see results and discussions)."
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"F1 Score: 0.3888888888888889\n"
]
}
],
"source": [
"vfs2_y, vfs2_y_score = get_mx(d_new)\n",
"ypred = cross_val_predict(svm_clf, vfs2_y_score.reshape(-1,1), vfs2_y, cv=kf)\n",
"from sklearn.metrics import f1_score\n",
"t2_f1 = f1_score(vfs2_y, ypred) \n",
"print(f\"F1 Score: {t2_f1}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Validation of US State Capitals"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.\n",
"#df_states = pd.read_csv('data/out/t5_states.csv', sep=';', index_col=0, encoding='utf-8')\n",
"#df_states.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.\n",
"#s_new, s_u_new, s_mold = get_truth_matrices(df_states)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Confusion matrix\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQ0AAAEGCAYAAAB/1cDLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHSdJREFUeJzt3XmU3HWZ7/H303t6TXc6CU2IBJTFMMrWAwwoElcYvQQYZYZxYeY6E2ZGRnC5Zxj0QMaF8ei4cDxz8IQLF/AgLkeF6EVGhCjqqENALqBBBQxhCQnpztKdTnd6ee4fXUjR07/nW78k1VWln9c5nHTXU9/6ffvX1Q9V9X1+z9fcHRGRUtVVegIiUluUNEQkFyUNEclFSUNEclHSEJFclDREJJeaShpmdqaZ/crMHjWzyyo9n2JmttHMHjKzB8xsfYXncr2ZbTWzh4tu6zGzO83sN4V/u6tobqvN7OnCuXvAzP60QnNbambrzGyDmf3CzC4p3F7RcxfMqyLnzWqlTsPM6oFfA28AngLuBS5w919WdGIFZrYR6Hf3bVUwl9OBYeAmd/+jwm2fBAbd/ROFhNvt7v9UJXNbDQy7+7/N9XxmzK0P6HP3+82sA7gPOAf4Kyp47oJ5nU8FzlstvdI4CXjU3R93973Al4GVFZ5TVXL3e4DBGTevBG4sfH0j00+6OZcxt6rg7pvd/f7C10PABmAJFT53wbwqopaSxhLgyaLvn6KCJ24WDnzXzO4zs1WVnswsFrv7Zph+EgKLKjyfmS42swcLb18q8tapmJktA44HfkYVnbsZ84IKnLdaSho2y23V9N7qNHc/ATgLeE/hZbiU5hrgpcBxwGbg05WcjJm1A18HLnX3XZWcS7FZ5lWR81ZLSeMpYGnR94cAz1RoLv+Nuz9T+Hcr8E2m305Vky2F98bPv0feWuH5/I67b3H3SXefAq6lgufOzBqZ/sO82d2/Ubi54udutnlV6rzVUtK4FzjCzA4zsybgL4C1FZ4TAGbWVviACjNrA94IPByPmnNrgQsLX18I3FbBubzI83+QBedSoXNnZgZcB2xw988UhSp67rLmVanzVjOrJwCFJaXPAfXA9e7+8QpPCQAzO5zpVxcADcCXKjk3M7sFOAPoBbYAVwK3Al8FXgJsAt7m7nP+gWTG3M5g+iW2AxuBi57/DGGO5/Yq4IfAQ8BU4ebLmf78oGLnLpjXBVTgvNVU0hCRyqultyciUgWUNEQkFyUNEclFSUNEcqnJpFGlFZdA9c6tWucFmtu+qtTcajJpAFX7i6R651at8wLNbV8paYhI9auJOo26lk5v6HjhGqGp0Z3UtXT97vtDFraF45vqs3NjQ91sl7S8YHIqPj82Y/j2gQG6Fyz43ffR8PjI0FAf32PvxFQYt6LJ7Rjcxvye3hc/fvCzz/y5Zkqdl/rEeR2ffGH8bHMbm5zMHNva2BBP7gA+pbcPbqO7aG67xsbD+8+f1xjGp+JfGY0N8XmbKDpvgwPb6FnwwtymUn/LQfiZpzexY3Ag9ZQEpqsX55yZnQlczXRl5/92909E92/oWETvn30yM37VRSeHx1va3poZW9DeFI7dORI/SRqChAQwOp795E8lrIWdzWF80+BIGG+si+cW/exNDfHY1Hnpao3/eDbvGA3jj+0czoyd2BdfzFnO/w/e8Vh82cl5y/vC+MhY9vMBoK+7JYxv2TmWGduzN37s6AXCu84+IxxbbM7fnhSa6fw701eDLgcuMLPlcz0PEdk3lfhMQ810RGpYJZJGSc10zGyVma03s/VTozvnbHIiEqtE0iipmY67r3H3fnfvL/7QU0QqqxJJo6qb6YhIrBJJo2qb6YhI2pwvubr7hJldDPwHLzTT+UU05pCFrXx8VXYnsw9c+1/hMddd8abMWGpte2DP3jB+aHf2ci4kllwTy7WNiXiDxfGuRM1AZxBPLd8V1wvMpi5R6LF3Mi5YOHJ+R2YsVQMyr6k+jO9OLHt2zcv+szh2Ufa8AJoTS9XjifOWWoaPlk3bW+I/55GxiexgSRUa0ypSp+HutwO3V+LYIrJ/VEYuIrkoaYhILkoaIpKLkoaI5KKkISK5KGmISC4VWXLNq62xgf4lPZnx2y97fTj+zE9+PzN29V+fGI5d1h336ki0lWAyWFdvaYxzdqpnRWp8qh/H/oxN1ULUJeoNFrTFLQminz112X6qz0jqvEVzXzo/rstJGQvqduDFPVBmk6rtiUTnLXXcYnqlISK5KGmISC5KGiKSi5KGiOSipCEiuShpiEguNbHkOjY5xcbB3Znxnz4VtwP8+DtemRk7/5LrwrGf/thfhvHjF80P46vveCQz9vpjFmXGAJYlOlM/OxRftt/RHC+LLmzNXvbsa5sXjt22J7srNkD77viy/CeH407qj27bkxl77WELMmMAV33vN2H8gyteGsaj1cehvYku7E3xUvJvd2V3WQf41cBQGG8Ptm+Itn0A+OVz2X9DO0bjn6uYXmmISC5KGiKSi5KGiOSipCEiuShpiEguShoikouShojkYlFL9GpxzLEn+Fdvvycz/sT2eM3/yKDt/O7RoK078D9vXB/Gv/ju7K0VAIaDx09cPZ7ceX1T4uc+qCOu84gub0+12t+xO64R6e2Id7xP7Rq/fSz78Y/obQ/H5rnMezZtQX3L4HD8c0fbQgDsjrYRAA7pietjNg1k1688sm1XOPZl3dnn7e3/4zX88sGfl3Ti9EpDRHJR0hCRXJQ0RCQXJQ0RyUVJQ0RyUdIQkVyUNEQkl5rop2HEa++L2uKagGeDmoBFnfHYNe+Mtzh47ervhPFr//HVmbHmRDv61Jr/6ETcPyHVyj9qh5+qIUlt3TCWOHZKZ1P2z57a2iFVezSViDcG2zc8uTOujVk+ryuM1yVqSEbH4/MW/ewt9XH/lC3D2T1QJhJ1OcUqkjTMbCMwBEwCE+7eX4l5iEh+lXylscLdt1Xw+CKyD/SZhojkUqmk4cB3zew+M1s12x3MbJWZrTez9YMDekEiUi0qlTROc/cTgLOA95jZ6TPv4O5r3L3f3ft7FvTO/QxFZFYVSRru/kzh363AN4H4UlERqRpznjTMrM3MOp7/Gngj8PBcz0NE9k0lVk8WA98s1F00AF9y9zuiATtGx1n7yLOZ8VOWxHuPRGvjd/92azj2uIXxY/+fS/7bO6sXOe+Kb2XGvv3xleHY8cl4zX4sEU/VaUxOZe91UZ8o1EjVQgztiffRiPplADzwbPb+HyuPXhyOve2RLWH8LUfG+81s35099+7meF+TrTvjPiGpPiXPBbUUAB3N2X+yfe1x/5SodqYuVZhTZM6Thrs/Dhw718cVkQNDS64ikouShojkoqQhIrkoaYhILkoaIpJLTVwa397UwKuX9mTGG5OXmGf/mL3tC8Oxjw5kL/0BvKIvXpK97aNnZ8b+7rp7w7Fr3xcv5zYklsm2jsTLd6/oy76M+7Ftw+HY+sQl3ks641b8d22MLw04dWn2eU1dGv9nx/SF8YnEUvX8oCXBQ5t3hGNTz4fUUvRB8+Nl02e2Zy/pNjfGfwcj40ErhRxbmeiVhojkoqQhIrkoaYhILkoaIpKLkoaI5KKkISK5KGmISC41UafR3FDHsoVt+zw+WtdvbojzZn1dZxjvaIlP4eG97ZmxWy/N3t4AYMWV8fYI373iTWF8flt8GXdXUL9ybEtcb7ArUW/QnTj2yYl2Bkvnt2bG2hPnPFWvMLRnIox3tWbXaRy1MH4+9LTF206knm/twaXvEP/sqedia1P2FgeNiXkV0ysNEclFSUNEclHSEJFclDREJBclDRHJRUlDRHJR0hCRXGqiTmNyytk5kl0XkGqnH7XjH9oTj42OC9CSqAl4dkd2/4PU2J/+61vC+Kkfvj2M3/GhN4TxqCPGc7viXhxRO/xS7E30tNi0fSQzdsj8uFdH61T8tE5t7TA8ml3HMbo36EmRGAswlIg3JeolRsayx6d2IRgczt42YiKxtcKLjlPyPUVEUNIQkZyUNEQkFyUNEclFSUNEclHSEJFclDREJJeaqNNwh/FgbX0qUacR9SBI7aHR3JjdgwBgPLG+PTaVva7fOJVYWE+44b1xP46T//6mML72U+dnxg7tjvuXXH33o2H8r09cEsZTbnlwc2bsvGMWhWNbGuLfWer50t2S3QskqvmB9PMpVSPyzPY9YTx6+IbE/j9PD2fXvoxPlV53U7ZXGmZ2vZltNbOHi27rMbM7zew3hX+7y3V8ESmPcr49uQE4c8ZtlwF3ufsRwF2F70WkhpQtabj7PcDgjJtXAjcWvr4ROKdcxxeR8pjrD0IXu/tmgMK/mW9OzWyVma03s/XbB+N9P0Vk7lTt6om7r3H3fnfv7+7prfR0RKRgrpPGFjPrAyj8u3WOjy8i+2muk8Za4MLC1xcCt83x8UVkP5WtTsPMbgHOAHrN7CngSuATwFfN7N3AJuBtpTxWXR20BbUWqbXzqcTaeaQz2BsE4r0kAOYHa/6pfSpSx+5oivfY+Nan/jyMn3PltzNjD/57PPaMI+J9S9qb4rnvHIv7lBy2ILtnxqLWlnBsqidFal+UqIwj9ditiX1LJhLPxTx9LWZK9WdZOC/7vDXUlf76oWxJw90vyAi9rlzHFJHyq9oPQkWkOilpiEguShoikouShojkoqQhIrnUxKXxhtFUH2xDkGgLv7grWGoai5drU5cyp5bvoiXZrzz0TDj2rcf0hfFHtu8K4ysOiy8h33zTuzJjR78/LqG5+4o3hfEF7dlLzQCP7RgO4+e+/KDMWNTqAKCrNV6KHhmLtyGIHj916fr+bKcBcFDwXAV4dmf2lhipx+6al31e6q30Ng3h2Tezniju7jMvSBOR33OpVxr3Ac7s++o4cPgBn5GIVLUwabj7YXM1ERGpDSV/plHosnUE8Ls3XYWeGSLyB6SkpGFmfwNcAhwCPACcAvwEeG35piYi1ajUJddLgD8GnnD3FcDxwHNlm5WIVK1Sk8aou48CmFmzuz8CHFW+aYlItSr1M42nzGw+cCtwp5ltB+IigzmUulx5dDx7Xb4usT69vy3pd49l15Cctzy7FgHSl8Yv7+kM41uCNX2Ia0z+86NnhWOPfv0Hw/jNN3wojB/S3hrGnx3Knvv88bgGpDHRyn/b0FgYj+odtg3tDcem6lNStRR7J+PnkwXP19G9cf1JQ1DrlKNMo7Sk4e7nFr5cbWbrgC7gO6UfRkR+X5T09sTMvvj81+7+A3dfC1xftlmJSNUq9TONY4q/MbN64MQDPx0RqXZh0jCzfzazIeCVZrar8N8Q0w2B1d9T5A9QmDTc/V/dvQP4lLt3Fv7rcPcF7v7PczRHEakiqQvWji4sr37NzE6YGXf3+8s2MxGpSqnVk/cDq4BPzxJzVBEq8gcndcHaqsK/K+ZmOrObcmc40QMh4sHYVC+OVFv41Lr98N7sx29I1BOk2t1vH423AWhOPP6eYF0/VU9w99c+Fsavuvs3Yfz9p8cXSN/2SHbB8d+f9JJw7HDid5raZmA8qJVIbTsxlqjbic45pOt+oi0Opqb2vTdMog3Ii5R67UkL8A/Aq5h+hfFD4AvPV4mKyB+OUitCbwKGgM8Xvr8A+CIlbnYkIr8/Sk0aR7n7sUXfrzOz/1eOCYlIdSu1uOvnZnbK89+Y2cnAj8szJRGpZqW+0jgZeJeZbSp8/xJgg5k9BLi7v7IssxORqlNq0jizrLMQkZpR6lWuTwCY2SJe3O5vU+YgEfm9VOqS69lMF3gdzPR1J4cCG5hxIVulbBuO+yMc0j0vM5aqR9i+O66FWNjZHMbbGrNP8bd+tSUce/ZRi8N4ogyDCY/X/KM9WZ7YNhKOTZw2Pn/eK8L4az5yZxi/+eJXZcZStQytzdk/F8BEomfFVNCfpS7xg6dqRFL9WQ4OnqsQ77tiFj8hokPnKNMo+YPQjzLdF/TXhQ7lryPxQaiZXW9mW83s4aLbVpvZ02b2QOG/P80xVxGpAqUmjXF3HwDqzKzO3dcBxyXG3MDsn4V81t2PK/x3e465ikgVKPWD0B1m1g7cA9xsZluB8HWYu99jZsv2b3oiUm1S/TReZmanASuBEeB9wB3AAPCP+3jMi83swcLbl+7g2KvMbL2ZrR8c2LaPhxKRAy319uRzwJC773b3KXefcPcbgduB1ftwvGuAlzL91mYzs189C4C7r3H3fnfv71nQuw+HEpFySCWNZe7+4Mwb3X09sCzvwdx9i7tPuvsUcC1wUt7HEJHKSiWNaN/7eG1oFmbWV/TtucDDWfcVkeqU+iD0XjP7W3e/tvhGM3s30zvKZzKzW4AzgF4zewq4EjjDzI5jell4I3BRKZOccmcs2Lukq6UxHL9zJLvWItXfINVP47ldcY3IU7uz6x1et2xBOLY90bthw0Dcy6MlUcjx5ED2mn9Xa3xOv/f41jB+kvWE8YvPWx7G37z625mxdVetDMcO7YlrJaL9XgBGgr1qouchQGN9/DuL+mFAuo4j2qcnNTasT8nRUCOVNC4Fvmlmb+eFJNEPNDH9SiGYg18wy83XlTwzEalKqc5dW4BTzWwF8EeFm/+vu99d9pmJSFUq9dqTdcC6Ms9FRGpAqRWhIiKAkoaI5KSkISK5lHrtSWV5vJw0ryW+FDq6BDyxAhYuvwEs7opKWeJjp5b+UvGF8+LL8rua42XTaO6pBbjtI/HS4+KueG6nL42Xm9de+ebM2Bv/Jb7O8c7V8cXTiZVJxoItL1LL4PMTS9Wj4/Fl+Sm7g+dj6rL66G/IUr0OiuiVhojkoqQhIrkoaYhILkoaIpKLkoaI5KKkISK5KGmISC41UadRV2e0NmdPNdU2PtqmYHA4cXl5Y1wDMpq4VHrr7uxL53tbm8Kx0WXQsH+XUQNsGsi+bH9RYmuG1xwaX/q+dWfcMmAssY3AM0FLgR987C3h2P6VHw7j62/7WBiP2gJs3x0/X1LnPNWKYSLxO42ej6m5jQT1J1Op4pUieqUhIrkoaYhILkoaIpKLkoaI5KKkISK5KGmISC5KGiKSS03UaaS2MGhq2Pfc15ho859YdmfvRFxv0B1sr5DqGt/WHNeIpGodtuweDeMv623PjEXbPgD8dudwGH/VoQvD+OMD8fjvP7YzM3b0gs5w7L23xnUYf/GFn4bxL//dKZmxqOYHoCXonwLp50tve1y7E9XWtDbFf847gt9p6VUaeqUhIjkpaYhILkoaIpKLkoaI5KKkISK5KGmISC5KGiKSS9nqNMxsKXATcBAwBaxx96vNrAf4CrAM2Aic7+7bw0nW1dETrF+PJfaSmBesnTck6jQmErUQqZ4W0bp+qoVBfWJuh3TG+1ykzkt0/M558f4dR/fEtRJDiR4nva1xv46VL8+u84j6XUC6l8fn33F8GH/7tT/LjF3zjhPDsU31cR1Hau6DiZ4Y0b4rDYkaktJ3NomV85XGBPABd385cArwHjNbDlwG3OXuRwB3Fb4XkRpRtqTh7pvd/f7C10PABmAJsBK4sXC3G4FzyjUHETnw5uQzDTNbBhwP/AxY7O6bYTqxAIvmYg4icmCUPWmYWTvwdeBSd9+VY9wqM1tvZusHB54r3wRFJJeyJg0za2Q6Ydzs7t8o3LzFzPoK8T5g62xj3X2Nu/e7e3/PgvjiJxGZO2VLGmZmwHXABnf/TFFoLXBh4esLgdvKNQcROfDKeWn8acA7gYfM7IHCbZcDnwC+ambvBjYBb0s90JR72H59PLEs2hAsg6WWTFPLltFjA4wG48en9m85d+dofPl6T2KLhJbG7P9njE/Gx05d1t85L35qPRFc4g0w4dnnJrUNwEHzW8J46vnyqT9/ZWbszR+5PRz7wGfPC+Obd8TtCg7ujpfRo5YFA4nl2rHJYAuD1C+0SNmShrv/iOyl4deV67giUl6qCBWRXJQ0RCQXJQ0RyUVJQ0RyUdIQkVyUNEQkl5rYwsAMmoOagtQ2A6l1/Uhd4nLj6FLl1LEHRuJ5eWLtvLEuzvmprR1aGqN2+/Hcot8HgCV+KYNjcU3B4fPbMmNRqwOA4cRl+ak6jSVdrZmxL1/2hnDsB9b+Moy/59RDw3hqbtHzqSXx+24Mnst1qT+i4vuWfE8REZQ0RCQnJQ0RyUVJQ0RyUdIQkVyUNEQkFyUNEcmlJuo0Jqdg157stffGRE+L5qAeIbVFQaqvRCq+dyL78Xvmxf0uou0PIN3TIlUrEdVx7NoT9+pI1b50zIsn19ca97yIpPqMtCVqZ5Ljm6PnSzzvS047LIzf8MDTYfxzK5eH8WjuUa8NSDxXS2+noVcaIpKPkoaI5KKkISK5KGmISC5KGiKSi5KGiOSipCEiudREnUadxXt0pGoGop4Xu0fjOo2xoM4C0r0domMnygWS8clEoUaqr0RU39JYH///JDW3qD4F0r0+ntyVvS/KUc2d4dihRI1Ja3P8tI/qHaJ6IYCDu+M6jv91+uFh/K3X3RvGv3D+sZmxnva47ifaa8ZzFGrolYaI5KKkISK5KGmISC5KGiKSi5KGiOSipCEiuShpiEguZavTMLOlwE3AQcAUsMbdrzaz1cDfAs8V7nq5u98eP1aibiBeng5rClKr01FvBYCmRC+P4dHsIzQk+mVs3r4njKfGp+Ye7XWR6uUR1c2Ucuxdid4PSzqy9x5J/dyNiX4aqb4TvR3NmbFUD5OJRH+V1Hn70oUnhvETPnRHZuyHV8R7sizuzK4hSdXlFCtncdcE8AF3v9/MOoD7zOzOQuyz7v5vZTy2iJRJ2ZKGu28GNhe+HjKzDcCSch1PRObGnHymYWbLgOOBnxVuutjMHjSz682sO2PMKjNbb2brBwe2zcU0RaQEZU8aZtYOfB241N13AdcALwWOY/qVyKdnG+fua9y93937exb0lnuaIlKisiYNM2tkOmHc7O7fAHD3Le4+6e5TwLXASeWcg4gcWGVLGjbdCvs6YIO7f6bo9r6iu50LPFyuOYjIgVfO1ZPTgHcCD5nZA4XbLgcuMLPjmF7t3AhclHqgySlPttSPREuLqaXB1GX3I4l4dHl6c+Ly8AXB0h/Ar7buCuMdiaXHrtbGzFjqsvrUkmzK3qn40nkLTmuqXcHg8N4w3t2W/XMDjAfbWuxILNcu7op/Z2s3bA7jrzk0fit+0z+cmhk78pyrwrF3X/++zFhqW4di5Vw9+REw2zMrrMkQkeqmilARyUVJQ0RyUdIQkVyUNEQkFyUNEclFSUNEcqmJLQzq6yysKUgtMUclBalLglPr19G8IL5UOigfAWB+4rG7muKeAKltAsaDuaUu8Y7GAljih2tOnPfh8ew6kYWdcS3EUX3tYXx0PK7zaA1qd8YSY9sS2yOc/fK+ML64K94C4bEtw5mxX996eTj2b275eWbsmZ2j4dhieqUhIrkoaYhILkoaIpKLkoaI5KKkISK5KGmISC5KGiKSi3mqJ3sVMLPngCeKbuoFqrVxaLXOrVrnBZrbvjqQczvU3ReWcseaSBozmdl6d++v9DxmU61zq9Z5gea2ryo1N709EZFclDREJJdaTRprKj2BQLXOrVrnBZrbvqrI3GryMw2Ze2b2IeAvgUmm9+a9CPgTpvfoHUmMvbSU+0ltUNKQJDP7E+AzwBnuPmZmvUxvu/2fQL+7h5/gm9nGUu4ntaFW357I3OoDtrn7GEDhj/+twMHAOjNbB2Bm1xS20vyFmf1L4bb3znK/N5rZT8zsfjP7WmEXPqkReqUhSYU/6h8BrcD3gK+4+w9mvoIwsx53HzSzeuAu4L3u/mDx/QqvUr4BnOXuu83sn4Bmd/9IBX402Qc10YRHKsvdh83sRODVwArgK2Z22Sx3Pd/MVjH9vOoDlgMPzrjPKYXbf1xo1NME/KRcc5cDT0lDSuLuk8D3ge+b2UPAhcVxMzsM+CDwx+6+3cxuAGZrQ2XAne5+QXlnLOWizzQkycyOMrMjim46jumy/iGgo3BbJ7Ab2Glmi4Gziu5ffL+fAqeZ2csKj91qZkeWc/5yYOmVhpSiHfi8mc0HJoBHgVXABcB3zGyzu68ws58DvwAeB35cNH7NjPv9FXCLmT3f7PPDwK/n6GeR/aQPQkUkF709EZFclDREJBclDRHJRUlDRHJR0hCRXJQ0RCQXJQ0RyeX/A9/HE8flnYnRAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"fig, ax = plt.subplots(1,1)\n",
"plt.xlabel('State')\n",
"plt.ylabel('Capital')\n",
"ax.matshow(s_new, cmap=color)\n",
"print(\"Confusion matrix\")\n",
"fig.savefig('figures/phase2/2_confmx_states.exe', format='pdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Get F1 Score**"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"F1 Score: 0.9090909090909091\n"
]
}
],
"source": [
"vfs2_y, vfs2_y_score = get_mx(s_new)\n",
"ypred = cross_val_predict(svm_clf, vfs2_y_score.reshape(-1,1), vfs2_y, cv=kf)\n",
"from sklearn.metrics import f1_score\n",
"t2_f1 = f1_score(vfs2_y, ypred) \n",
"print(f\"F1 Score: {t2_f1}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Aggregation"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[0.34989304 0.26557676 0.25118927 0.24763552 0.24631537 0.23613277\n",
" 0.22471282 0.22328377 0.08567668]\n"
]
},
{
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x16be112d630>"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAF3CAYAAABKeVdaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XdYXFX6wPHvmQYDM7RQQkJ6NTE9puvG2Ht3LWtfXfvaXde6lrWtZe3G+rP3El27xp5oYhKNMb1CIPTOwLTz++PQhoFhCDMMM5zP8/AQ7rn38qIw79xT3iOklGiapmkagCHSAWiapmm9h04KmqZpWjOdFDRN07RmOilomqZpzXRS0DRN05rppKBpmqY100lB0zRNa6aTgqZpmtZMJwVN0zStmU4KmqZpWjNTpAPoqvT0dDl06NBIh6FpmhZVfvnllxIpZUZn50VdUhg6dCjLly+PdBiapmlRRQixPZjzwtZ9JIR4VghRJIT4vYN2IYR4SAixSQjxmxBiarhi0TRN04ITzjGF54GDA7QfAoxq/DgPeDyMsWiapmlBCFtSkFJ+C5QFOOUo4AWpLAVShBDZ4YpH0zRN61wkxxQGArmtvs5rPFawuzfMz19IYeErzV87C5w4i5y7HaCmaUq9SbIzzYsXGFRuIN4lqIqXFCZ5/c4dUmrA4hFUJEiKbf7tw0oMmLyCskQvpYn++7kMLzZilFBi81Ke4N8+ssiIALYKE5X1vi9hArC6BABOI7gNssP2BpPEI1rapFAfTZd4Be0KdbtBuMnJ2ozR4AagpjaF0oqBeAUMG/AH4PszVNekUVaZjTR4GZq9tvlmth0e1lZP5aJ/LG7/GwcpklNS2/tP1u6OP0KI84QQy4UQy4uLizu8YWHhK9TUrGr+2lnkxFPj6XagmtbXVcdLKhIkbmPLH6kU4DH6f0jR1C4Dtns7uL5JZ+21dWY8DcbmF/OmD2j5/m3b2r7AtG7zttfe5qOtULQnWKtJtZdgNjkxGt0YDS2J1Ghwq2OtPgzC23ytOl99eJM9zPp8czvfpWsi+aSQBwxq9XUOkN/eiVLKhcBCgOnTpwfcKs5mm8yUKV8DsPLylQBM+XpKt4PVtL7syfx8Lt+wgfz5s8mOi4t0OADcOF99/vrr0Nxv/kr1evH1lJ59vcjLe5hNmy5lv323YLFkdfn6r57azIHnDeHTh2oZcXP3d9KM5JPCIuD0xllIs4BKKeVudx1pmqZFI7t9GoMHX4fZnNn1i1etYt6VM/gPV5H/eD7bbt3W7XjC9qQghHgVmA+kCyHygJsBM4CU8gngI+BQYBNQB5wVrlg0TeuenLg4DkpNJc6giyCEWnLyHJKT53T9whUrYP/9EXY78vRLsD5VhP2w3UgsbYQtKUgpT+6kXQIXhev7a5oWOof168dh/fpFOoyYVFu7hvj4YRiNCcFf9MsvcMABkJSEefFizq3NYPmjudgm27odj077mqZpEeLxOFi2bAK5ufcGf5HTCcccA0lJ8PXX1GcPY/PXdbgQOilomtYzXi4sZPCSJRQ69RTvUHI4NgMSq3VM8BdZLPDaa2qEfehQvvgCJl+SyVaLHesYa7dj0klB07RO1Xg85DY04JXdn92itXA41gOQkDC685N//hkebyz8MGcOtCkMOu7lcRhM3X9J10lB0zQtQurqNgBgtY4KfOJPP6kxhPvug9radk+JHxofkph0UtA0zU+9x0OV2938tcOjF4GGg8OxAYslG5PJ3vFJS5fCgQdCRgYsXgyJiT7NrjIXAPXb6kMSU9SVzu7I63/cQ1blN2wzTuPyxkUoNWfWAGBr/FrTYkn+2/0o+iQ15Pf1SqhuTALJJrWEuNpjBzmZsUYnQvSOcYWa9VZsYxzMX7nJ57hZOhjiXoFTWNlhUsWXB7tXEidrfM5ziCTyTJMAGOL+BaOjjOHx8ZSWqjW0ZnMGSUkzACgv/xKv1/dF12LJxm5X9y8r+xQp3T7tcXGDsNkmAlBa+j+/+OPjh5Gd/TfS0g7r+IdcsgQOOgiyslRCyMnhww/h6adh7ly4+mrwbKsDknFXuTu+TxfETFJwlL4BwGpLoMKsmhY7ij5JbX5hDCV347hBnKGlZoTVYEBKEB3U84kE2xgHmQeX+x2f1/Acf2p4mgLDGJ60vwrAQfX3MdDzh895241TeM72DACHO+4gw7sN6mD1atWelnYoEyeqF/O1a8/A6dzpc31GxomMH/86AGvWnIjHU+XT3r//OYwd+zQAq1cfQdtCFzk5lzFy5AOBf8hVq6B/f5UQBg6ksBBOPRUSEmB04zDENFnOLNyMPyA58L2CJGSUDRxNnz5dtrfJzvPfTQfgzL1b2lbO12UutNg1f776HKoyD02u2byZ/+blUbv33piicLHaL7/MREoPe+zxfyQmjgfUWgCPp87nPKPRRmLiHgDU1Kz2exIwmZKbB4Crq1chpcun3WxOw2od0di+Aik9bdrTsVqHAVBVtcwvTosli/j4we3/EA4HWBtnEtXVqSwAnHsuPP88/P47jGmcsPT7Mb9Tu7aWmetmdvSfBAAhxC9SyukBTyKGnhQ0TQuN32tr2SMhISoTgstVQXX1coYMub45IQA+/26PzTYhYLvdPrmT9sB7hCUl7RWw3cd338Gf/wzvvAOzZjUnhJUr4Zln4LLLWhICQM2qGuwzA4xJdJFOCpqm+RgUF8eebQYzo0Vl5TeAl9TU/SIdyu759ls49FAYNAiGDPFpslrh2GPhpptajnnqPHjrvSFZtNZEJwVN03w8OaYLC6l6Gbt9BqNHP0lS0qxIh9J133wDhx2mEsLixWosoZWxY+Gtt3wvMSYYmVMwB6/bf9+K3RV9z4eapoVNtI0xthUXl82AAedhMPSO8t5BW71aPSEMGaIGiVolBIcDLr0UcnM7vjwUi9aa7xWyO2maFvUWFhQwZMkSSqKwnIXTWUxBwTM4nSWRDqXr9thDvfJ/9ZWaftrK/ffDww/Dxo3+l225fgsbLt4Q0lBirvsof2E+ha8UAmoAJpR9bZoW61bX1FDhdtPPbI50KF1WXv4569f/lalTJ2GxpEc6nOB8+y2MGgXZ2XDnnc2Ha36rYfWRqymqNXFH6VQOGtfAggUJuEpd/Dz+5+bz3GVuUvcL7VqVmEsKha8UNicD22QbWad0fScjTeurVtfWsmdiIqI3LUgIUnn5l5hMKdjtUTIF/Ysv4Igj4PDD4c03fZqqf6mmYXsDzw7bA3eZ4LZLHEACIk6QfrRvwut/mu/YQ3fFXFIAsE226bUJWp9V4nTyXkkJTUOPf8nKIsFoZHlVFStqavzOP6t/f8wGA0sqK/m1poaTMru/UUtPk1JSXv4lKSn7IoSx8wsi7fPP4cgj1Qq0xx7za3ZXuPkDO4u2pvCPf8Be56u9LEw2E2OeCO9EgJhMClrfsXAhvPJKpKOIjFWrYHI70+fv2LGDB/Pymr8+sl8/EoxGFpWWctv27X7nn5qVhRl4o7iYSo+HucmhWRnbk+rrt9DQsJ1Bg66KdCid+/RTOOooNZ3oiy8gveWd/yuvwPvvw91XZTD/hQT+/ovkn//s2ac2nRS0qPbKKx2/OMa6yZPhlFP8j39cWsp+KSm8sIdarZthsQBw1aBBnD9ggN/51sZFarcMHcp1gweT2Xh+b+J2V7Fq1QJcLt9B5OTkeYwb9xJebz0mU7/evz7B64UbblADy198Aa12s1u3Ds44Q9W9E/fEM26veB48redD1ElBi3qTJ4e+1EO0KnW5KHA6OX/AAAbE+U7LTDKZSDJ1/CefHKAt0oSwMGjQ5RQVvYbJ1PJC2rRSOTFxPEOH3kJCwthIhRgcgwE+/BBMJp+EAHDllWrx8sqVEL+5kvLNXlIXhL7gYWd672+Bpmld1s9spmTuXFxRvt6gLaMxnqysU8nKOrXDc3JyLu7BiLroo4/gxRfhhRf8ppwCfPyxOuXee1Xz6vN2UL+9nr1WdaE8RojodQqaFmPMBgMJxigYbO2CvLxHqK1dE+kwds+HH6o9lTduVMXt2vHBB2pm6qWXqq/d5W5MKZF5z66TgqbFCLfXy+wVK3ijqCjSoYRUQ8NONm26hLKyTyIdStd98IEqWDRxohpD6GAQ/9FH4Ycf1PbLoGYf6aSgaVq3/FRdzdKqqpj7oy4v/wqAlJRePojc1qJFcNxxatDr888hJcXvlNJS2L5d7VORkdFy3F3hxpwamQWEsfb7o2l91qdlZRiA/VJ7fnAynMrLv8RsTm/exSxqZGWpTS86SAigKp7uuSeUlfkej+STgh5o1rQY8UlZGTOTkkiNwhIVHZFSUlHRtCgtSt7DbtoEI0fCzJnw2WcdnrZ6NTzxBFx4IaSl+bZNXjwZU6ruPtI0bTeVOJ0sr67moLavLlHO6czH5SohJWVBpEMJzttvqzUIL70U8DQp4fLL1RDDLbf4t9un2bEOt4Ynxk7oJwVNixI/VVVx49ateNpMN7192DD6Wyz8OTOTw9vMfY92cXEDmTu3HAjdfgFh89ZbcNJJMGOGKmERwKJF8OWX8NBDfssVcJW7KH6rmNQDUrEO7fnEoJ8UtKi0cKHqrl21KtKR9ByHx0OZy0WD14tTyuYPCQyzWnl13Dim2UO3LWNvYTTGYzQmRDqMwN58UyWEWbNUGYukpICn//abGks4/3z/tvqt9Ww4bwO1v9aGKdjA9JOCFpVal7dor9RDLJqfmsry6Z3uux71amvXUlr6PwDKyj5mwIALyMw8PsJRBbB1q/olnD1brUALkJhrasBmgxtvhKuvhvaGf9wVbgA90KxpXdXXylsUNDSQajIRH2ML09rasuU6Skvfb/xKkJPz94jG06lhw+CNN+CAA9Qrfgc8Hpg7F/bdFx58EOLj2z8v0klBdx9pWpT406pVnLluXaTDCLuGhh2kpR3MvHnV7L13LenpgfvnI+a119ReyqBWLAdICABPP626jebODXxbd7l+UtA0rRNSSnY2NMTcQHJ7Ro9eiBACkymyuya6yl14G3wHuI02IyabCfniS3DmGXj3PRD3HnOa2012E8ZEI163F1eJq/l4RSXc8E8L8+bC8ccLvE4vrjIXbZmSTRF/UtBJQdOiQKXbTZ3Xy8C4KNuQfjckJUV+3KTi+wpW7e0/i2HobUMZOvR7OPMMKryTWP3lpXizlzS3j3hgBIMuG4Rjg4Nl45c1H3+UEZSSw437lSBEBtUra1gxa4Xf/ce9Po7sv2aTdkgaRntkugl1UtC0KJDvdAIwsBfudRBKLlcppaUfkZq6P3Fx2RGLI3luMuNeH4dzlxNhadnkJq3wQzj9AuQ+83Ecu5ARFt8po8lzVG0jS38Lox4fBUBdg+DrG7M4aWIdc09Ss6jih8Y3t7dmn2bHlGzClBy5l2adFDQtCuxsaADw2yMh1tTU/Mq6daczadKXEU0KQggyT2xnW9KzlsKCBRgWLWJAQsfTZM1pZgaeP7D567WnghCJJDb2/lmyLD7tvYlOCpoWBUZarfxnxAjGBnghigUNDWob0bi4QRGLYdf/7aLq5ypG3j8SQ1zjXJz6ejVd6OmnwekEa3CLynbuhOxsnx03ez2dFDSti+o9Hmo8Hr/jKSYTJoMBh8dDbTvtqWYzRiGo83ioa6c9zWzGIAS1Hg+ONu12o5ErB0XuhbKnNDTkAmolc6TsfGQn0i1bEsIzz8Ddd8M336hX+CATgssF++8PkyapiUrRQicFTeuiFwoL+duGDX7H182YwZiEBB7Lz+eqzZv92nfOns2AuDju2bGDf23f7tdeNW8edpOJm7du5b68PL92OX9+SOLvzerrczGZ+kVsBXPN7zVUL69m5IMj1YGnn4Zzz4WDDoIuVp99/HG17/Ldd4ch0DDSSUHrMQsXqpXIodC0mrknvV5UhEdK5iQl8fDIkX7tmY3LUxekpLTb3rQH8qH9+pHezlLWOIN6Z3p0ejpDO1rZFOMaGvKIj49g19FzuxBmQeapmeoX9m9/g4MPhnff7Xi1WTu++gr+/ne1nu2II8IYcBjopKD1mNalKborEuUtnsjPxy0l302Zwp4BFipNsduZEqDUwYykJGYEqI0zLyWFeR3U348kr9eNEAaEMOD1upDS7XeOwRCPECKIdidS+nehjRnzFG53ZVji74zX5aXwpUL6Hd4Py1fvqYRw6KGq8mkXk/QTT6jPDz2kNtCJJjopaD0qmktT1Hu92GO8xERHPJ46li4dxtSpS7Bah7Njx91s23aj33lz55ZhNqeydesN5Obe49e+zz4uhDCxadPl5Oc/5tNmMMSzzz6OiM068tR4SD8mnYxjMmB6tqptfeedsBszvsaMgXPOgbFjwxBomOmkoGlBcng8zV1EfU1d3TpcriIqK3/Eah1OauoCDAb//xYGgxqETUs7BLPZf2+Hpo1y0tOPIj5+cJu2yL4cmVPNjDl8I8wfqhLB/ffv9r1uuy10cfU0nRQ0LUgOrxeroW+WC6urUwPrNtskAJKT55CcPKfD81NT55OaOr/D9rS0A0lLOzCkMXaHq8yF+44Hsd5/jXo6+Mc/Ih1SxPTN33BN2w0OrxdrH+0+cjg2AAKr1X8APRbU/PXfWO+/BveCw+GKK7p9v8MOg4suCkFgEaCfFDQtSKumTyfKxgxDpq5uA3FxgzEaI7NFZDjJBx4g9d1bKE+ZT+rHb0MISols2hRwW4VeTScFTQtSWh8dTwCwWDJITd030mGEXkkJ3HIrxeyN89bnSA1RbSmHA6J18blOCpoWpOu3bGH/1FT27eIiplgwcuQDkQ4hPNLT2X7ka+x43cTsv4RuFbXDEfTC515HJwVN64CUkkd37gTUtvH/3rEDq8HQJ5NCzPnPf6ChAfnPf1K4NIV+x9gwp4buSVAnBU2LIbdv28Y+KSnsnZzMJZs2+bSNiNa/9G6orPyBtWtPZ9y413vFXgfdds89cO21cOKJCCmZvnp6825noXLIITBxYkhv2WN0UtBCKlApi0iUpugqj5TcuG0b/xo6lL2Tkyme02pXLSFI6YPjCnV166iv34LZHAO7vt11F1x3HZx0Erz4IhgMGOPBmN31WWVSwmOPwR9/+B4fOhTefDM04UaCTgpaSAUqZRGJ0hRd1VSdNN5gQAhBeoxvahOMuroNCGHxW2zWG9Tn1uPMd5I0U5UNqfimAk+db/kMS6YF+zR7c0Jo+NOx1Jz6IN5F5Wy9YSujHhtF6vyudwk6nbBkCbz6KqS1Wqc3bRpcfXW3fqyI0klBC7loLmXh8Ko9efvSIrWCgueorPze59ioUQ9jNCZQVPQmxcVvYbWORIjetUZDSslvB/2GMcnItKXTAFh3zjrqN9f7nNfvyH5MeH8CZGZSFH8Qa7+5EPnNWtVogLgBu7dxUVwcvPSSeuCItvpGgeikoGmt1PfBpLB581VI6cRkainC11TMzuHYgJROMjNPj1R4HapaWkXd2joGX9/yBDP+rfHIBulznrla7dHA2WdjnXg8U1o9SJjSTCSM6vrc0cceg9mzYcqU2EoIoJOCpvloflLoIyuXpfQwaNBV2O1TSUs7yK99yJDrGTLk+ghE1rldz+/CkGBg8LUtScE+uc2KsVtvVWUrli6FSZOwT++4Om2wfv0VLr1UFbx78slu367X0UlB01oZabVSMW8ecbH29q8DQhgZMuS6SIfRZZ46D0WvFZFxfAYmewcvY7fcAv/6F5xxBuy5Z0i+r9cLF1yg9tu5886Q3LLXCeszshDiYCHEeiHEJiGEX4UpIcRgIcRiIcRKIcRvQohDwxmPpnXGIATJJhPxfeRJwekspKEhHyll5yf3IjUra5BOSf8z+/s3Sgk336wSwllnqe00Q/T/87nn1ODyvff6Di7HkrAlBaFGpR4FDgHGAScLIca1Oe0G4A0p5RTgJOAxNC2CNtbVcc3mzWxzOCIdSo/Iy/svS5cOaXdDnN4seW4yswtmk/KndjYjWrRIdRudfbbaTjNECaGkBK65BubNg9N73xBLyITzSWEGsElKuUVK6QReA45qc44Emjr5koH8MMajaZ3a5HBwb24uhS5XpEPpEXV1G4iPH97u3gi9lfSopxpzihlhaKeb74gj4IUX4KmnIIQTBmw2uOQStfdyLM9DCOeYwkAgt9XXecDMNufcAnwmhLgESAT2D2M8mtahu3fs4MPSUkobk0FfmX3kcGwgIWFMpMPoku3/3k7pB6VM/nYyxvjGpwApVemKE05Qq8dOOy3k3zc+Xg1TxLpw/ua3N1LXtuPyZOB5KWUOcCjwomjamqn1jYQ4TwixXAixvLi4OAyhan2dEbAIQbbFwvEZGYzqA+UspPTicGzEah0d6VCCJqVk1/O7MNqMvgnhuutU386zz4b8e7rdcNRR8NlnIb91rxTOJ4U8YFCrr3Pw7x46BzgYQEq5RAgRD6QDRa1PklIuBBYCTJ8+PbpGxGJYeyUtoqGURXuuGjyYqwb3vhW74dTQkIfXW09CQvQkhcrvKqnfUs/Qm4eqA1KqOkb33qumBbV6K19fD//9LxQWtlx/zTXQvz98+y28957//W+8Uc0s+vxz+PhjdWz7djVMceaZ4fqpepdwJoVlwCghxDBgJ2oguW2Rgx3AfsDzQog9gHhAPwpEifZKWkRDKYu2zl63jvV1dfwwdWqkQ+lRJlMyY8e+GHBbzd5m13O7MNqNZByXoRLCNdeobqMLL4RHHvFZSXbXXWoCUuvNbs47TyWFNWvUGHRbV1yhksLKlb7t55wDRx8dxh+sFwlbUpBSuoUQFwOfop7On5VSrhFC3Aosl1IuAq4EnhJCXI7qWjpTRtvcuD4umktaNKn2eKhwR8/sGyk91NauQUo3iYl7YjBYaGgowOks8DvXZpuEEEbq6/NwuYr82rOyTkVEyZoMd42bojeLyDwpE2OiEerq1C/fRRfBww/7JASPBz76SNW9e/VV/3tdcIH66Mg116iPviisi9eklB8BH7U5dlOrf/8BzA1nDJoWa3bufIxNmy4FYPbsncTFDaCgYCHbtt3id+68eZWYTEnk5T1IXt59fu1/+pM33OGGjDAKRv13FLYpiaoaXUICLF4MiYl+tSaMRvjxR6itjVCwUUyvaNa0KFNRsZi4uMGMGvUwJpNaQZWZeRI22xS/cw0GNWCenX0WKSn79GicoWa0Gsk+uz9cdhmsX686+m02v/OWL4eRIyElBZKTIxBolNNJQdOiiJSSysofSUs7kPT0I5uPJySMCTi1NDFxPImJ43sixLBwbHNQuqiEAWvuxrDwUZUY2tnborpazRSaOLFloFjrGp0UtD7vT8nJUTMFtb5+Gy5XIUlJsyMdSo/a9UwBltuvxMB7cPnlcN997ZYnvflmKCiAd96JQJAxQicFrc+7OCcn0iEELS4uhylTlhAfPzTSofQY6ZVYHrqFgbwHV16ppp+2kxB+/RUeekjNMJrZdpmsFjSdFLQ+zeX1UuxykW2xRMUsHIPBTHLyrEiH0aMqFleQX/Unko/vj+3eu9tNCE3VS9PSYrd6aU/pG2v5Na0Dx61Zw8AlS5i9YkWkQwlKbu79VFR8G+kweobXC4sWseu5AuqTR2N98c4Od7SprlbrC+69V33Wdp9OClqftrq2ltlJSTw+uvev6vV4atm8+RrKyvpAvQWvF84/H446iriN35N1alZLWYt2JCfDhx/GdvXSnqK7j7QuaV3aIlpLWjSRUpLf0MCJGRlMsds7vyDCqquXAx6Sk2N8kNnrhb/9TS0p/uc/GX77uXgbK6NWVcG//+17+q5dcNNNMHx4BGKNQTopaF3SurRFNJa0aK3E5cIpJQPjdm/j9p5WWfkjAElJMTym4PXCuefCs88i/3E9zgtvwFsvuOoqwbHHwh57wIMP+l5iMMD48XD11ZEJOdbopKB1WSyUtgCwGAzcP2IE81Pa2ailF6qqWoLVOhqzuV+kQwmfpUvh+efhppsoHH4+64f+xKhvp/HYYzZGjID99lOF7rTw0UlB67OSTSYuHzSo8xN7SG3tOtavP9vv+NCht5CWdiAOx+aoKl63W+bMgZUrkRMmkDf1F6xjrCSOS4x0VH2KTgpan1XQ0EC1x8NIqxVDL5iOKoQBo9G/bIMQ6s90r71+x+ut6+mwws/jUYPKRx8Nhx0GEydS8XU5NatqGP3U6KiYKhxLdFLQ+qwn8/P51/btOPfZJ+JJobp6FSZTMpMmdTyzSAiB0Rhj75o9HrVRwUsvqZHiww4DIO+BPMzpZrJOzaK2b+yM2mvoKalan5XvdJJpNmPuBVtvrl9/Fhs3XhLpMHqW263mkL70Etx+u9o9DXAWOSn7uIwBFwzAaFXTUOPiVOVTLfz0k4LWZ+1saOg1M4/q63Nje1ZRWx6PSgivvqrmmDYmBABLpoUZG2ZgtKkskJSkB5d7kk4KWsx4rbCQEVYreyUlUe5y8dyuXX7n7JeayiSbjSKnk9W1tUxqp/RyT/N46nC7S4mLi54aTN1mMKilx3fdpbbTbCSlRAiBdWh0FCiMRTopaDGh2Onk5LVr+XNGBq+NH0+Jy8WVmzf7nff4qFFMstnY2dBAbkMDZ/XvH4FofTU07AQgLq73zIQKG7dbrTbLyfHbPhMg7/48yr8sZ/zb45u7jhwOtZbt5JPhkEMiEXTfopOCFhPcjbu4zmncVWW41UrlvHl+58U3jh9MtNmonDePJFPk/wQaGnKBPpAUXC449VRYsgR+/91vBxyv20veQ3nED4tvTghNl734olofo5NC+EX+L0Lz07qURG/T20tbxDW+6BuFCPiC31l7T0pMnMD48W9js02KdCjh43Kp5e9vvaX2QkhOpqICHnsMGhrUKbVrHBTvyGbBKWk07SF3661qK2at5/SOvwrNR+tSEr1NtJe26I0slgwyMo6NdBjh43Kpvp+334b771eb5ADbtsHnn7deHZ8IJCLyJWc0Hrn9dnW52QyjRvV45H2STgq9VKyUktA6V1n5I1K6o34P5Q7ddptKCA8+CH//e/PhyZNh8WL178qllaycvZKRD40k55KWAXens6eD1XRS0GJClsVC1bx5zd1H0aC6egV1devIzb0f8DJ9enTs6dBlV12lKtb9+c8+h+vrVf07qxUSRicw/K7h9D9wsDAuAAAgAElEQVQz8gP/fV30/AVpWgAGIbCbTFiiKCmsXn0Ea9eeSk3NLyQkjIt0OKHldMK//qUGBJKS/BICwH/+A4mJakKSOc3M4GsHY7Lr96mRFj1/QZoWQJXbzVWbNrGksjLSoQTN660nM/NUZsxYz9ixz0Y6nNBpaIDjj4dbboFPP+3wNIdDrVIuemYnRW8U9Vx8WkA6KWgxodbj4b68PH6rrY10KEHba6/fGTXqERISRmMwWCIdTmg0NMBxx8EHH6ipRcccg6vCxbqz1lH1cxUA1SurWXPCGnLfKCVOeNhyzRaK3yqOcOBaE50UNC1C4uKyMZujYy+HoNTXw7HHwv/+B088ARdcAEDNLzXsen4XFd9WAOCp8VD7Ry01pR7ipBfrCCtDrh8Syci1VnRS0KLeByUlHLp6daTD6LIdO+6ltPTjSIcROvn5sGIFPPmkWoLcRtLMJABS9k5hxpoZ2I/IJCnHzPSV07FNiny5EU3RozpaVMqtr+eKzZu5IicHi8FAf4uFI/v1Y98o2UUNYMeOu8jKOoV+/aJ8ma7TqRYSDB8O69ergeUgHHUUTJnS+Xlaz9JJQYtKVR4PbxUXc2JGBidkZnJQWlqkQ+qbHA61Oc6UKaq4XZAJAVRPk9b76KQQYe2VtOitq5k1zYfDod7uf/EFnHSSX/Nzz8HEiTDKZqRhfAoPvG4l7ruWdqtV7a+TmtpzIWud00khwtoraaFLSWi9Xl2dSghffgnPPqte3VtZvx7OPhuuvhruuSeJrJcmc2s7XUXr1qkhCK330EmhF9AlLbSoIqWadvrll/D882qznDaWLlWfTztNfZ44saXwXWtmc/jC1HaPTgpaVDILwaC4OKxRtIK5rVmztiFEFP4JCqEeA045peVVv42ffgK7HcaNg6qfq9h40UZGLxyNfYq9h4PVuioKfyM1DUYnJLBj9uxIh9GhP/44hQEDLiAlZW8KC19j48YL/M6ZPPk7bLY9IxDdbqqthWXLYP58OOGEgKf+9BPstZdaseyp9lC9vBpPjadn4tS6RScFTQsDVfnURUrK3litw8nK8u9iMZujaIS1thYOO0y92m/ZAtnZHZ7qckFuLhx0UA/Gp4WMTgpaVNpeX8/f1q/nn0OGsE8vXJsghAGDQe0znJQ0g6SkGRGOqBtqalRC+P57ePnlgAkB1DjBrl3tjyFovV/0dshqfVKJ08k2h4P1dXV8Wl5OoS64H17V1WoPzB9+UFPl2pl62h6DQU051aKPflLQospVmzfzf4WFzV9H0/4JUemFF9Seyq++2uk4QpNrrlFPC3fcob42pZhI3jsZU5J+uYkG+v+SFlX+mp3N/MbuIqvRqFcyh9uFF8LcuV1aTfn669B6DoB9mp0p3+p6FtFCJwUtalS73YywWpmbnIwQItLhBDRz5kagd8fYoaoqtRjtzjthzJguJYSCAtixAy67LHzhaeGln717yMKFaiZf249VqyIaVlS5Py+PAUuWRDqMoAhhRIgo/POqrFTThj74ADZs6NKlRUVqszWAmTNb3XJJJT+N/Ymq5VUhDFQLlyj8rY1OTeUs2tIlLWLT1q03smvX/0U6jK6pqIADD4Tly+HNN+GII7p0eX6+KlmRleVb/dRb58Wx3oHX4Q1xwFo46O6jHqTLWfQdhYUvk5w8j/79z4h0KMFpSgirVsFbb6m6Rl2wejUkJKiep/h4Xb4imgWVFIQQFmCwlHJTmOPRNC0STCZIToa33/Z7Qth0xSZKPyj1OTby4ZH0O7gfFd9VsP7s9fx5+0RGxdVxe3/1EjHmuTGkzEuh9JNS1p+zvsd+DK37Ok0KQojDgPsBCzBMCDEZuFlKeUy4g9O07nA4tjR24XgxGm0MHnwtALt2vYTD4ftCZTKlMWjQ5QAUFDxLff1Wn3aLJZuBAy8EYOfOx3E6833a4+OHkp19DgB5ef/F5SoLx48UeuXlKiHY7fDZZ6quURvF7xQjjIKkWS17JZjT1KOAKdmEfYYdQ5EBc4r6N9A8/dScaiZlfgqmJBO2KXp3tWgQzJPCrcBMYDGAlHKVEGJkWKPStHYckJqKzWgM+vz8/IXk5t4NGLBYspqTQnHxW5SWfuBzrtU6sjkpFBa+TEXF1z7tNtuU5qSwa9dzVFf/4tOekrJPc1LIz38Sj6eaxMTxXfnxel5ZGRxwAKSldZgQAGZumol0SowJ/v/tbRNtjHt5HJaxkDQ5jnEvJ/u0J81MYtzL48ISvhYewSQFl5Syos0UQBmmeDStQ3OSk5mTnNz5iY08nhpMpjTmzfPt+pgw4b2A102e/GXA9mnTfg7YPmPGH8EFGEmlpbD//rB2Lbz7bocJAcBgMujRxz4kmNlHa4UQJwIGIcQwIcSDwNIwx6VpzVZWV3PX9u1cv2ULv9bUIGVw70m8XgdGY0KYo4tCrRPCe++pMhYdcNe42XDhBiq+r+jBALVICib/XwzcBHiBd4BPgevCGZSmtXbj1q38r0z10b9eVMTG1pPgAxg58gE8ntpwhhadTj9dJYT33++0lKmrxEX+4/nY97KTMq/jwoP33ae31YwVwSSFg6SU1wLXNh0QQhyLShCaFnZuKdnLbufbyZMxGwxBr2Y2mZIwmYLfSL7PeOABtex4//07PdVd7gZU/aJADjssJJFpvUAw3Uc3tHPs+lAHommBGIB4oxFjF8pbFBQ8R0HB82GLKaoUF8O996qtNEePDiohALgrGpNCauCk8NNPsGZNt6PUeoEO/08LIQ4CDgYGCiHub9WUhOpK6tMWLlSrlIO1alWXSshorTw2ejTeIMcRWisoeAaDIY7s7DNDH1Qv9McfsHVry7v2p58GhwOs1UUc88gCkkq28FPakcw7ZwwAjz8ObrfvPfbcE/bdF7xeePRRqPnVRAEDWfaxlfjVMHWqqo/X0KD+BprcdhssWACvvdZDP6wWNoHSfxHwO1APtH4PUA38I5xBRYOmshXBvtDrcha7b/huFub3eh2YTL1vA55wKC+HffaBiRNbksL11wNFhXzFAqxs5SA+ZMDiMcxTM2e58kqVNFo7/3yVFKSESy8FsKuPe1T71VerpOBwNLW3GDQofD+f1nM6TApSypXASiHEy1LK+h6MKWroshU944OSEhq8Xo7PzOzSdV5vXZ+ZfXTrrSox3HZby7F13xRiP3oBxh1bqXrlf7y5975YLC3tubn+94mLU58NBigpUf9umu0lhCA+Xh1LSmppb6KrmMeGYAaaBwoh7gDGAfFNB6WUo8MWlaa18ujOnVS43V1OCh6Po3lLzFi2bh088gj89a/qXXyT1B2/QmE+fPQRyfPn+13Xr1/H9xSidbv/OI7BEPh6LXoFM9D8PPAc6jfjEOANIKieQyHEwUKI9UKITUKIdruchBAnCiH+EEKsEUJ0oZde09rn8dQipewzTwpXXKGK0TU/Jbhc6vOBB6pBhnYSQrAKni1g4yUbux2jFj2CSQoJUspPAaSUm6WUNwD7dnaREMIIPIpKJOOAk4UQ49qcMwq15mGulHI8oLfm0LrF46nlxx+z8XiqmDVrOyNG3BfpkMLK44E5c9R+OJmZqPrVU6ao0tcAKd0bU6n4poKSD0o6P1GLGcF0HzUINTF8sxDifGAnEMxz/Axgk5RyC4AQ4jXgKKB1DYBzgUellOUAUsqirgSvaW25XKV4PNUUF7/VXIsolhmNcEPTpPH8fDVKnJ8P2dkhub+7wt3pGgUttgTzpHA5YAMuBeaiXsjPDuK6gUDroay8xmOtjQZGCyF+EEIsFUIcHMR9tT5mz8TEoM/1eOoAMBhiv9voxRdVpWspgZ07VTdRfj588gnMmxeS76GTQt/T6f9tKeVPjf+sBk4DEELkBHHv9lYZtZ1sbgJGAfOBHOA7IcSeUkqfQitCiPOA8wAGDx4cxLfWYsllOTmMDzIxeL1qjmWsDzCXlKgpoTNmwLELKlRCKCyETz9V/Ukh4i53Ez88vvMTtZgR8ElBCLGXEOJoIUR649fjhRAvEFxBvDyg9czlHCC/nXPel1K6pJRbgfWoJOFDSrlQSjldSjk9IyMjiG+txZKc+HjOCrI7xOtVTwqxPsB8001QXa0qVoiUZPjLX0KeEAAMCQbiBsSF9J5a7xZoRfOdwHHAr8ANQoh3gb8DdwPnB3HvZcAoIcQw1DjESUDb5VvvAScDzzcmntHAlq7+EFrsKnE62XvVKm4aMoSTs7KCuMJAfPzwsC1a8zg8SE/LA6/BYsBgMSC9Ek+dxz+aztrjDBjMBqRH4nF03O51e/HWq0ICq9fAk08aue6k7Yx1VYOYgPf6G1V7je8SZaPViDAKvC4v3gb/QgSdtU/9cSrCEHxpES36Beo+OgqYJKV0CCHSUO/yJ0kpg9pbT0rpFkJcjKqqagSelVKuEULcCiyXUi5qbDtQCPEH4AGullKWdnzX7ulqaYpAdNmKnuEB1tXVUdG2HkMHkpNnM2vW5rDEUvZ5Gb8d/JtPkZehtw5l6I1DadjZwNLB/g/QIx4YwaDLBlG3ro5l45f5tY95egyGw7N5/cF6Bt21glRc7CCBX1H7RmSfm03S9CRq1zoY8eBK7Li5i7GMpYbrXtkH+bUVtq+n9MNy1hzjX3xo8reTSdk7haLXilh3+jq/9mkrp2GfbKfgqQI2XuQ/9XTmpplYR8R2V5zmK1BSqJdSOgCklGVCiHXBJoQmUsqPgI/aHLup1b8lcEXjR9h1tTRFILpsRd9Tt7YOvCoRGKyq5zV5rnrxNiWbGH7vcL9rUvZWTyzmTHO77dapdv50IPz2m5X3LhnB8MFOlv2cyP1vNq4Me6rxg0Q+v3IEI/q7OOSzXZz5w1FYvbU4H30do8lE4vjEdu8fP0yNB9in2tttb+oaSpqd1G67qZ8eZO5rAv0fHy6EaCqPLYChrb5GSnlsWCMLE12aIraVlv6P3Nz7GTfuVSyWrq2A7kz/s/qTdnAa1pFWvy4VU5KJwVd1PAnCkm5pt/2BB+C33+CFF+CQP/fHYoGL6uDUB/3vkZGRjXnnNv7+yNEISxV8/iXx06cDkDAqIeD3TxyfSOL4jgfr7VPs2KfYO2zX+o5ASeG4Nl8/Es5ANC0U6uu3UVHxVVjubbKbMNlD98555041YHzIIWqcuKkqeEKC+mjXLbcgqqrgiy9g2rSQxaJpTQIVxAu8Ua2m9QCLEBySlsbg+OCmRXo84ZuSWvxOMa5SFwPOHRCS+918sypd/cgjAbdI9vXYY6pU6fjxIYlB09rSHYZar1FW9gX5+U/4HX9n9IPEx/ejpORDdu163q99zJgnMZv7UVT0ZnN7OJJC4YuFODY5QpYU7rkHjj0Whvt35fvavBmuvRaeeQaSk3VC0MJKJwWt1/B4Kqmr850h45aSYmcNg+LB7S7zawfwelUBOJdL1ejJyDgRgyH0v9ruCnenO5AFw+lUVUbT0uDQQzs5edMmVbqirg7y8lRS0LQwCvo3XAgRJ6VsCGcwWt+WkXEcGRktQ1lbHA5G/PQTrNjFG+PSOKH/6fTvf3qH1w8ceAEDB14QtvjcFW7iBnV/Idedd8L778N330HAhdobN6qEUF8PX32lnxC0HtFp7SMhxAwhxGpgY+PXk4QQD4c9Mq1PKSl5nxUr5uB0Fjcfy29Q70FuHjKEw3pB8f5Q1AHatEklhbFjO0kIGzao0hUNDSohTJrUre+racEKpiDeQ8DhQCmAlPJXgiidrWld4XQWUVW1BCmdzcccXrVKbL/UVBKMxkiF1qy7SUFKuPhisFjgvs4qeptMqtLpV1+pPTY1rYcE8xtukFJuF77TI/zX42taiDUlBashmPcu4Tdr+6xuXf/226o80X//G6CydX4+9O+vRp+XLevCtCRNC41gkkKuEGIGIBs3zrkE2BDesLqvvZIWujRFdBkWH88VOTkMiOsdBdlMSd3rOlq4UO1/c+GFHZywdi0sWABnnqn6mHRC0CIgmLdgF6DKUAwGCoFZjcd6taaSFq3p0hTRZYLNxn0jR/aKpOAscbL56s3U/FrT5WsbH3j48EM1wGxqL7f88YcaVJYSTjute8FqWjcE89bHLaU8KeyRhIEuaRE9LJZMkpJmI4Sl+Vi9x4NLSmxGIyLC75qdO53k/ieXpNlJ2CbZgr6uslLtd3PHHXDkkTBoUDsnNSUEgwEWL1aj0JoWIcE8KSwTQnwkhDhDCKGLo2hhkZ5+FFOn/ojF0rJfxuP5+SR9/z2VQVZIDSd3hYqhqwPNN94Ia9bAgI7WuzU0wMEH64Sg9RqdJgUp5QjgdmAasFoI8Z4QIiqfHLTo0jzQ3AtmHrnK1QK5riSFFSvg0UfhggugsW6dv7g4eOop9UirE4LWCwQ1rUNK+aOU8lJgKlAFvBzWqLQ+xeUqZ/v2O1i2bAJOZ2HzcYfXi0DVP4q0rj4peL0qGaSnq64jP6tXw6uvqn8fdBCMGROiSDWtezr9DRdC2FAb7pwE7AG8D4R2zz+tT1u//lxKSt4GDD5jCv8rLcUsRMTHEwA81WoWdrBlLj77DH7+GV58EVLabgL366+w335q9dpRRwUoiappPS+Y3/DfgQ+Ae6SU34U5Hq2PkVJSWfkN/fodwfDhd2E2pza3raqpQQa4NhSqV1XjqfaQMCYBS6YFV4WL2tW1fudlnZLFgL8NQJiDS1AHHwzff9/OlslNCcFqhS+/1AlB63WCSQrDpZT+m7f2Ql6nl8pvKkn+ky4aFi0cjs24XCX063c4iYnjfNpyZ88Oa9dR1U9VrJi1AoA9Xt6DrFOyqFlVw6/7/up37p6L9iT9iPSg7ltYCFlZMHdum4ZVq1qeEBYvhhEjuvsjaFrIdZgUhBD3SSmvBN4WQvi9YeuNO69Jlwoz65QsCNFezFp4mc1pjB79FGlpB/i1DQzz+oSyz8tAwIQPJmCbpqaZ2ibZmPSFf52hxImBChW1+PZbOOAAtSbhgLY/0qefgs2mEkKn9bI1LTICPSm83vg5qnZcS/5TMgPOG6CTQpQwm9MYMOCvfscLGhp4qqCAkzMzGRWmLpaKxRXYJtnod1hLsT1zqpnU/VIDXNUxl0utVs7ObvOU4HarFWvXXgvnnQepu3d/TesJgXZe+7nxn3tIKX0SgxDiYqBX7czWUGLGWRzPuatGYpuvS1pEi+Lid7HbpxIfP8Tn+CaHg5u3bWNOUlJIk8Lvv6sPr9vL9u9MnHCRekL4+mvYtcv33KSklv0OvvgCSkp82/v1a3ka+OQT9XSwZg0sWtRqqGD5cjj5ZHjrLVXpVCcErZcLZkzhbPyfFs5p51hEucpMeOtbZtjqkha9n9tdzZo1xzNkyA0MG/Yvn7ayxgVraWZzSL/n8cfD+vUABnIGjOfyK1V57rvuUr07rY0b15IUbrkFfvjBt33mzJakcO218Ntv6v5HHNF4wrJlcOCBavqR3xQkTeudAo0p/Bk1DXWYEOKdVk12oCLcge0OQ7yHpyZvYsrXUyIdihaE6uplgJekpNl+beUutVgstd1CQbuvvh6OPlrVmzOZBPE5au/np56C2jaTjiwts2N5+WVwOHzbW28b/d57ake1UaMaD/z8s0oIaWlqDGGI75OQpvVWgf7ifkbtoZADPNrqeDWwMpxBaX1DZeWPACQl+ZekDteTQmKimhkU9+pWEvdMhJGZQAc1iVrp7DV92LBWX/z+u3qESE9XCWHw4O4FrWk9KNCYwlZgK/BFz4Wj9SVVVUtISBiH2ezftVLmcmEEkkJc4mLNGvDUe/g+ZQcDLxpI5gmZIb0/ACNHwkknwQ03dJ5tNK2X6bDMhRDim8bP5UKIslYf5UKIsp4LUYtFUkqqqpa223UE8K9hwyiaOzcsq5mrllYhGyQp80Pcz798OZSVqX6lJ5/UCUGLSoG6j5q23AxuxY6mdeKWrVt5s1jtwfznzEz+MeMP3J56xv/8s9+5Z/Xvz1Vh6Ha55BLI3OJhbwMk7x3CRY4//KCWMR96KLz+eufna1ovFaj7qGkV8yAgX0rpFELMAyYCL6EK42la0J4qKMBqMHCm8R2mFP+CZeiXmKRkXKL/xjVZrUd5Q+iDD2BiveCQKTbMKSEar/j+ezjkELVA4f77Q3NPTYuQYKZ2vAfsJYQYAbwA/A+1NOzwcAamxZYyl4t8p5N7hg/n8NoiKio2AmAQgjfHj+/BSCTSC6kLQrRe4LvvVEIYOFANKne4cYKmRYdgSmd7pZQu4FjgQSnlJcDA8IalxZpil4vJNhtTbDbc7lLM5n6dXxQWgn6H9mP43SEoM+H1wkUXQU6OWv2mE4IWA4LajlMIcQJwGnB047HQzhPUYt6YhARWNu40s2JbCSZTpJKCBEJUjttgUP1RFovqOtK0GBDsiuYLUaWztwghhgGvhjcsLZa5XKXYbB1P/N+4Ue1B09Yhh6iK02vXqo+2jjgCzGZ17caN/u3HHAO2Cgf82gB0o/to8WJVtuLhh/WiNC3mdJoUpJS/CyEuBUYKIcYCm6SU7e0lpWkdOnL1aobEx/PwqFEkJIzDZpvo075pk6ottNdeqobQFVf43yM3V/XUvPkm3Hyzf3tFBSQnq41t7r3Xv91Z7eHRumXkHJDDbieFr76Cww9Xq9UqK3UtIy3mBLPz2t7Ai8BOQAD9hRCnSSl/CHylpilSSr6rrCSnsRT2hAnv+Z3z5JPw0ENQXg5/+YvadqCtzMZ1Zn/7mypV0ZZN1bbj8svVPdqqXlqFdHVjfcKXX6rHkeHDVXLQCUGLQcF0Hz0AHCql/ANACLEHKkl0tBW5pvnY2dBAhdvNnokd70nw9deqwFxCgvrIyOj4fllZ6qMj2dntd/FvvbECjJA8bzfWJ3zxhUoIo0ap5BAoQE2LYsEkBUtTQgCQUq4VrTfS1WKKu8pN0atFeJ0tm+0lz0nGPs2Ou9LNrhd2+V2TMj8F2wQbzmInRa8V+bUv30dNchvtsLD1ye/JH3oG6fm3kFij1kea9u7HihVWrr3YRd7DhX7Xpx+dTvygeOo21FH2qf9i+swTM7FkWahdU0v5V+V+7VmnZmFOM1PxXQX2aXZMSbtZZG/SJDWwrBOCFsOC+etYIYR4EvV0AHAquiBezCp8uZCNF/qO0g6/ezi79jBSme9g9f2bmo9bHdC/EEY9Noq84YLKHbX80ao9oQ6yimDluxmQAiPyBdsf+hUe3ULBE/nwszp35z/teL1WZvSvY9Olm2grcVwi8YPiqVlZ02570swkLFkWqpZWtdueun8q5jQz6Uek423o4s6yBQXqsWP//VWfVhi3B9W03iCYpHA+cClwDWpM4Vvg4XAGpUWOMcGIfaadCR9OaJ62aUgwsOfqX1hXVwfPtZx7sC2VRcPGYUg0MHPFT+x0On3aj0/uxyuDxzLC7Obd9XVkjimn4dU17CqDiW/tjT1hLwD+cbuJuDg48CI75vPabmwMxiRVFC/96HTmlnTcnnlqJulH+1dlMSar9oEXD0RYuvCi/umnasrSiy/CccfphKD1CQGTghBiAjACeFdKeU/PhKRFUv8z+tP/jP4AvLRrF/fk5vLt5Mk8OHIk1Y3lrJvPtViaS0U8OWYMDo/Hpz0nLg5zsplBmHl7/HjWr9uPqqqlGAzx2LJGYraoa2//N5x6GiQkGQi0ntIQZ8AQ13G7Md6IMb7jqqqBrvXz8ccqIYwbB/PnB3+dpkW5QJvs/BO1w9oKVJmLW6WUz/ZYZFrE5TudrK6txWwwcFBaWsBzD+sXeDHaSKuVimF34HaXk5p6ECaTrbnNaoWpU0MScmh89JFKCHvuCZ9/rjbK0bQ+ItBbp1OBiVLKE4C9gAt6JiQtkn4e9zPb79oOgMOr+t+thi68w+6AEILU1AVkZBznkxC+/x7++U+1xqBX2LxZJYQJE9SMI50QtD4m0F97g5SyFkBKWdzJuVoMcFe7qVtb1zyW4PB4sAiBoZt96Q7HFjZuvAyn039m0TvvwAMP+G5tGVEjRsATT6gnBL0OQeuDAo0pDG+1N7MARrTeq1lKeWxYI9N6XP32egDih6pX6HqvNyRPCXl5/yU//3EGD77Gr23xYpg9uxckhQ8+ULOMpk+Hs86KcDCaFjmBksJxbb5+JJyBaJFXv9U3KYxKSODAbnafuFzlFBQ8Q2bmycTF+VYRdbvh119V91FEvfcenHgiLFgAn3wS4WA0LbICbbLzZU8GokVe/bbGpDBMJYWLBg7kooHdq5JeUPAUXm8tgwb5FzPyekFKtYI5Yt59VyWEadP0jmmahh4n0FqJHxxPxvEZmDNCUxnd63WSl/cQKSn7YbNNavec1NQIdh29845KCHvtBZ99pqrpaVoft5vr/bVYlH5UOulHtSz+OmPtWircbt6fMGG37ufx1JCWdgCZmSe3226xqH3uI0JKeOUVmDFDrUlISopQIJrWuwSdFIQQcVLKhnAGo0WW1+XFYDawpraWVwsL+baykoHd2CvZbE5j7NjnOj+xp3k8YDTCyy+D0wl2e6Qj0rReo9PuIyHEDCHEamBj49eThBC6zEUM+rH/j2z+x2bu2bGDO3bsILe+nmm7+YJZU/Mb1dW/BDzH5YITTlC9OD3m9dfV00FpKcTF6YSgaW0E86TwEHA48B6AlPJXIcS+YY1KC7u8vIfZtOkypkz5luTkuezc8izuN/9KroAzC+FM1Dzk6dmrGs9/hE2bLvW7z8yZm7Fah7F9+51s3Xp9qxaJ2ZzB7Nl5GAztP21IqTYwmzIl5D9e+157TW20MGeOSgiapvkJJikYpJTb2+xp6+noZC061NX9gcEQT1zcIADM5WPgpb+QflwGieNa9j0wm9XONklJezFkyA1+9zGZ1IY1yclz/dpTUhZ0mBB63CuvwGmnwbx58L//tezIo2maj2CSQq4QYgYghRBG4BJgQ3jD0nqC0WgjPn4wAIa8MfDc2Qy+YCpJw/wHXZOSZpKUNLPDe6Wk7ENKyj5hi7Vb3n1XJYR99lF7fQ8Om8EAACAASURBVAbY7EfT+rpgpqReAFwBDAYKgVnoOkgxp/UahQWrVnFfbm6EIwqhGTPgzDN1QtC0IHT6pCClLAJO6oFYtAhKnJRIzmU5mNJMfLemklkhnKK5Zo3aq6Y1k0l17Q8aFMbZoN9+C3PnwsCB8MwzYfommhZbOk0KQoinANn2uJTyvLBEpOF1etly3Rbc5S37F2Sfl03yrGTq1tex4+4dftcMvGQg9il2an6tIe+/eX7tg64eROIeiVT9VEX+k/m47Udhj9uHdc+sA2DIjUMY+cBIatxu3FKSagrdEpb//Aeef973WGqqWqOww/9HCY3/+z9Vw+juu+Hqq8P0TTQt9gTzl/9Fq3/HA8cAQfUtCCEOBv4LGIGnpZR3dXDe8cCbwF5SyuXB3DuWlX9VTt79eVj6WxBmNcDf70i1X4Gr3EX5F/77EGeerAaEncXOdtuz/6p2snfuampPBpKpRZ2bc1kOAGWNG+mkmbu3qnnNGjjwQPUG/brr4JxzfNtDmHP8Pfec+ob77w8XXxzGb6RpsSeY7iOfgjBCiBeBzzu7rnFQ+lHgACAPWCaEWCSl/KPNeXbUdp8/dSHumObc5cScbmbmppkYE313EkuelczsHbM7vDZt/7SA7U2rlquqfqK+PpfMzON92stcLnWfbr5qV1dDfr6adjp6tProEc8+C3/9KxxwgCp0Z7X20DfWtNiwO7WPhgFDgjhvBrBJSrlFSukEXgOOaue824B7gPrdiCUmZZ+ZzZzCOX4JIZR27XqejRsv8jtuFIL5KSkM6uY8fodDfe7R1+TiYvj73+Ggg+D993VC0LTdEMyYQjktYwoGoAz4RxD3HohvN1Me4DOnUQgxBRgkpfxQCHFVUBH3EcIQ/k3iqzweRv3k+4D22cSJLJ48udv3jkhSyMiAb75R+ypHfIMGTYtOAZOCUCvWJgE7Gw95pZR+g84dXd7OseZrhRAG4AHU4tnANxLiPOA8gMGDBwf57aNT6SelbL5qM3u+uycJo8JbU9rp9SKlZGar6T/xIdhUB3o4KSxcqDZnuPBCXBMmkJeXR329fvDU+qb4+HhycnIw7+a4YMCkIKWUQoh3pZTTduPeecCgVl/nAPmtvrYDewJfN66W7g8sEkIc2XawWUq5EFgIMH369GCTUlSq+KoCx0YHcQO7X4ahoSGf3Nz7Ub13YLWOJCdHlarIz3+SVFMqP0ydSlY3it51JDtb1TVKT+/83G554gm44AI44gg4/3zy8vKw2+0MHToU0c1tRDUt2kgpKS0tJS8vj2HDhu3WPYIZTfxZCDFVSrmii/deBowSQgxDPWmcBJzS1CilrASaXzKEEF8DV/X12UcVX1eQNDMJY0L3xxNKSt4jL+8+jMZkhDCQnDy3OSnYbJOIjx8aloQAag3CnDlhuXWLxx6Diy5SCeHNN8FgoL6+XicErc8SQtCvXz+Ki4t3+x4dJgUhhElK6QbmAecKITYDtahuISmlnBroxlJKtxDiYuBT1JTUZ6WUa4QQtwLLpZSLdjvqGOWudFP9SzVDrg9mHL9zHk8dALNn52Iy+VYDnT59JY/t3El+WVm3t9yMiEcfVdNNjzoK3nhDbc7QSCcErS/r7u9/oA7knxs/Hw2MAQ4FTgCOb/zcKSnlR1LK0VLKEVLKOxqP3dReQpBSzu/rTwmV31eCF1L2TQnJ/dLTj2TcuDcwGtsfm7ht+3be7sY7ikDuuw/+n70zj4ui/B/4e1jkEhAJUQQVU0yBZQHB+8gLrzJvM/PMNEut/GXSt0OzWy0vTDNvrTQty/JIRdE0L1Qkb1BRMUQRBZF7d35/DIzALofCAi7zfr32JewzO/OZWZnPPNf7cXAAozXtZ2ZC3756CaEyoFKp8PX1xdvbm0GDBpGamlrqfS5ZsoQ1a9bovR8TE4O3t/dj73fevHkliu/QoUO8+uqrpKamMmzYMNRqNd7e3rRr146UlJTHPr4ps3r1ajw8PPDw8GD16tUGtzl16hStW7dGrVbz/PPPk5ycDEBmZiajR49GrVaj0WgICwsrt7iLaj4SAERRvFROsVR5zJ8yx3mYM/atysb7YGPTBBsb40wQiIuD6Gjp55o1Ife+dPQoZGTA6dOQlGQEQ3V8PNSuDW+/LQ0/LaOO8bLE2tqaiAhJOT5s2DCWLFnClCkP16gWRRFRFDF7hNhfe+21Mo8TpKTw8ssvY1PMQtk7duygR48ezJ8/n9q1a/Pvv/8CcOHChcfu0DRlEhMT+fjjjwkPD0cQBJo3b06fPn2oWbNmvu3Gjh3LnDlz6NixIytWrGD27Nl88sknfP/99wD8+++/3Lp1i549e3Ls2LFH+j/zuBR1hFqCIEwp7GX0yKogNVrVwHOdJyrrspmfkJJymnv39pXJvgrSs6ckHe3QAd7JM5h44EDpvVWr4KmnoExbcubOlWbBnc2Z/1gJE0JB2rdvT3R0NDExMTRr1ozXX38df39/rl+/zs6dO2ndujX+/v4MGjRIfuIODg7G09MTHx8f3sm5uDNmzGDOnDkAHD9+HI1GQ+vWrVm0aJF8LK1Wy9SpUwkMDMTHx4fvvvsOgLCwMJ599lkGDhxI06ZNGTZsGKIosmDBAv777z86depEp06d0Gq1jBo1Cm9vb9RqNXPnzpX3HRoaSteuXYmLi8PV1VV+/5lnnsHS0pKYmBiaNm3KyJEj8fHxYeDAgXINJDQ0FD8/P9RqNWPGjCEjQ1rA0d3dnYSEBADCw8N59tlnAdi3bx++vr74+vri5+fH/fv3AZg9e7Z8btOnTwfgwYMH9O7dG41Gg7e3Nxs25Jtrq8cff/xBy5Yt8fPzo2vXrsTHx+tdXwBvb29iYmIAWLNmDT4+Pmg0GoYPH16i7/2vv/6iW7duODo6UrNmTbp168aOHTv0trtw4QIdOkh24W7duvHLL78AcPbsWbp06QKAs7MzDg4OhIeXT0NKUTUFFWCL4aGlCmWMNlVL5s1MrBpalVmb+I0bC7hz5w/atIkrfuNH5P596NJFUljk7ZL48UeppgDQoGy6RiS++Qb+7/+krOPhUaKPRL0VRUpE2TZt2Pra4jGvZMfPzs5m+/bt9OjRA5BuACtXruTbb78lISGBTz/9lN27d1O9enW++uorvvnmGyZOnMjmzZs5f/48giBw7949vf2OHj2ahQsX0rFjR6bm8TotX76cGjVqcOzYMTIyMmjbti1BQUEAnDx5kjNnzlC3bl3atm3LwYMHmTx5Mt988w179+7FycmJ48ePc+PGDU6fPg0gHzshIYFq1apRo0YNxowZQ1BQEJs2baJLly6MHDkSj5zv48KFCyxfvpy2bdsyZswYvv32WyZOnMioUaMIDQ2lSZMmjBgxgsWLF/PWW28Vet3mzJnDokWLaNu2LSkpKVhZWbFz506ioqI4evQooijSp08f9u/fz+3bt6lbty5bt24FICkpqcjvpF27dhw+fBhBEFi2bBmzZs3i66+/LnT7M2fO8Nlnn3Hw4EGcnJxIzFlU/IcffmD27Nl62zdu3JhNmzZx48YN6tV7OPjSzc2NGzdu6G3v7e3Nli1beOGFF9i4cSPXc+zEGo2G33//nRdffJHr169z/Phxrl+/TosWLYo8v7KgqKQQJ4riTKNHoADA3dC7nO5zGt/9vji0L5s+Ba02FTMz4811cHGREkNe2rUzwoHmzJGkdoMGSesqV/LmirS0NHxzJgC2b9+eV155hf/++48GDRrQqlUrAA4fPszZs2dp27YtILUht27dGnt7e6ysrBg7diy9e/fmueeey7fvpKQk7t27R8eOHQEYPnw427dvB2Dnzp1ERkayadMmeduoqCgsLCxo0aIFbm6S38rX15eYmBjaFfiynn76aS5fvsykSZPo3bu3nFB27twp/+zr68vly5fZuXMnu3fvJjAwkEOHDmFtbU29evXk83n55ZdZsGAB3bp1o2HDhjTJ8ZyMHDmSRYsWFZkU2rZty5QpUxg2bBj9+/fHzc2NnTt3snPnTvxylulLSUkhKiqK9u3b88477zBt2jSee+452rdvX+R3Exsby5AhQ4iLiyMzM7PYYZt79uxh4MCBOOWMrXbMeQIaNmwYw4YNK/RzhqZzGXrYW7FiBZMnT2bmzJn06dMHi5z+sTFjxnDu3DkCAgJo0KABbdq0wdyowrCHFNunoFA+3Au7h2ApYBdY+jWDEzIzic3IwFyXikplw5W0NO5mZ+fbxlwQOBUQgOVjNsH8/HM5LW/8++9SQhg8WEoIj/CHUdIn+rImb59CXqrnWctBFEW6devGTz/9pLfd0aNHCQ0NZf369YSEhLBnz558nyusJimKIgsXLqR79+753g8LC8MyT+eOSqUiu8D/B4CaNWty6tQp/vrrLxYtWsTPP//MihUr2L59e74+EVtbW/r370///v0xMzNj27ZtDBgwQC8uQRAM3hxzMTc3R6fTAeSbbBgcHEzv3r3Ztm0brVq1Yvfu3YiiyHvvvcf48eP19nP8+HG2bdvGe++9R1BQEB999FGhx5w0aRJTpkyhT58+hIWFMWPGDL1Y8sZT2PUurqbg5uaWr3M4NjZWbh7LS9OmTdm5cycAFy9elGs85ubm+Zrv2rRpI9fIjE1Rd4QuRZQplDH3wu5Ro3UNVFal70/oEBFB25Mn0enSMDOz5sMrV2h+/Hi+V9CpUzhbWFDjMZ8+mjcvJ8ldr14wf/4jJ4TKTqtWrTh48CDROb31qampXLx4kZSUFJKSkujVqxfz5s3TSy4ODg7UqFGDAwcOANLNKZfu3buzePFisnKkhhcvXuTBgwdFxmFnZye32SckJKDT6RgwYACffPIJJ06cQBRFIiMj5ZrPwYMHuXtXMutmZmZy9uxZGuS0E167do1Dhw4B8NNPP9GuXTuaNm1KTEyMfJ5r166Vaznu7u4cP34cQG5LB7h06RJqtZpp06YREBDA+fPn6d69OytWrJD7XW7cuMGtW7f477//sLGx4eWXX+add97hxAlpOtV7773H5s2b9c43KSlJ7hPJOyLI3d1d/uyJEye4cuUKAF26dOHnn3/mzp07AHLz0bBhw4iIiNB75dbSunfvzs6dO7l79y53795l586deska4NatWwDodDo+/fRTeUBBamqq/N3t2rULc3NzPD09i/wuy4pC/8pEUUwslwgUyLqbRcrJFNynu5fJ/m5mZtKtZk202VLz0ZR69Rjs7Jxvm8etIeTyyy/g7AzF1NYfn6VLpTkItWvD5MlGOkjFUatWLVatWsXQoUPljtdPP/0UOzs7XnjhBdLT0xFFMd/TYi4rV65kzJgx2NjY5LvRjB07lpiYGPz9/RFFkVq1avHbb78VGce4cePo2bMnLi4uzJs3j9GjR8tPzF988QXHjx/Hz89Pflq+dOkSEyZMQBRFdDodvXv3ZsCAAVy9epVmzZqxevVqxo8fj4eHBxMmTMDKyoqVK1cyaNAgsrOzCQwMlG9806dP55VXXuHzzz+nZcuHWrR58+axd+9eVCoVnp6e9OzZE0tLS86dO0fr1pIB2NbWlnXr1hEdHc3UqVMxMzOjWrVqLF68GJBG7fTp00fvfGfMmMGgQYNwdXWlVatW8s1/wIABrFmzBl9fXwIDA+XmLi8vL95//306duyISqXCz8+PVQUXBzGAo6MjH374IYGBgQB89NFHctPT2LFjee211wgICOCnn36SBwv079+f0aNHA1Ky6N69O2ZmZri6urJ27dpij1lm5A6Pe1JezZs3Fw2xcFk7ccF3bcUTHU8YLK/M3P79triXveLdfXcL3SYzM0F88OB8nleUXJaWdj1fWaOwteJH5/eIycnhYlLSUaPE/PTTovjyy0bZtSh++qkogii+994jf/Ts2bNGCKjq8sknn4g//fRTsdtduXJF9PLyKoeISkZQUFBFh1ChGPo7QJo0XOw91nTq408wNdrWwHO9J/YtDc9P0OmyOXy4EVrtw5EVFhZ15FFFUVETuHPnT7lsGZB62x27Z64YNW6j8Mkn8NFH8PLL0s8KFcoHH3xQ0SE8Fn/99VdFh/DEUqWSwrXZ14j/IV6v48j9E3ecnnPi/sn7XBhzQe9zjb5uRM3ONbl34B7Rk6L1ypssaYJ9S3sSdyZyedplvfKma5piq7bl9ubbXJ15Va/c6xcvnIc4673/EC06XTrOzi/x1FPSaBQzs4dq6Hr13sHZWdJKZel0jLxwnoFPPZ4Mq0KZOROmT4fhw6XV01TGW09CoWxxd3eXh7IqPNlUqaRQZ2QdMuMzSYtKy/d+7mI2ZpZmWNbXn4JrZi21v6usVQbLBUspyZjZFPJ5i5zP2xbyefOiB3qZmVnSsWPhvggHh475ft9dR0RXyLaVlgcP4KefYORIaQ1PJSEoKFQIVSopWDhb0HhO40LLq3tWR/27utByu+Z2RZY7tHPAoV3hcwwcuzni2M348jlBEHhibqmiCDodVK8OBw5IwiQlISgoVBiV3xNQhsStjOPm2psVHcYjo9Nlce7cSBIS/ih22/jMTF69cIGjOWItY7F1K3z2WSl3IopS/8HQodIiOU89pSQEBYUKpmolhWVxxK+Jr+gwHgMd8fFrePCg+Dbb25mZLIuL46oRVx47e1Z6uC/VIniiCB9+CJ9+Cvb2T4THSEGhKqD8JZoY6TljzK2NeJN980145ZVS7EAU4f33parG2LHSnAQTSgqKOlsBSqbOnjFjBq6urrIAcNu2bQBkZWUxcuRI1Go1zZo144svvii3uKtUn0JlZGVcHE9bW9PRwYHbmZnMzhFi5aWfo+STSNZm8+4lfZP5UGdn/OzsuJyWxtzYWKDs1lo2RFoaFGNaLpqZM+GLL2DcOFi82KQSAijqbIWSq7MB3n77bdmGm8vGjRvJyMjg33//JTU1FU9PT4YOHYq7u7vRYzetv8YnDK0oMubCBRbm3MjvZWcTcuOG3utszpNcSrbWYPnFNGk01Y2MDDYnJOBcrRoepbprF01aGlhbl2IH3brBlCkmmRAKoqizFXV2UerswhAEgQcPHpCdnU1aWhoWFhbY25fNOivFodQUKpB7OVKyDg7SiCUPGxtSc9zqedHpMjgU44SbtYPB8lzaOxRdXlY8VlIQRWl0Ufv25bSAs8TJZ0/qvec82BnX113RpmqJ7BWpV15nVB1cRrmQmZDJmYFn8pX5hfmV+NiKOltRZ+dSmDobICQkhDVr1hAQEMDXX39NzZo1GThwIL///jsuLi6kpqYyd+5cWZNhbEz7Ma0APn/54P3b47e/ljWJOeIyx2JEb2ZmlrRte5t69Qr/QypPUlMfMSmIomQ67dAB9u41WlyVhVx1dkBAAPXr1+eVnA6YwtTZvr6+rF69mqtXr+ZTZ//66696zTqG1Nm57Ny5U/b3tGzZkjt37hAVFQUgq7PNzMxkdXZB8qqzd+zYIT+ZGlJnT506lcTERAIDAzl37hyAnjr7wIEDXLhwQU+dvX///iKvX646e8GCBdy7dw9zc/N86mx/f3/Onz9PVFQUarWa3bt3M23aNP7++29q1KhR5L5jY2Pp3r07arWa2bNnc+bMmSK3L0qdXZQQTyyhOnvChAlcunSJiIgIXFxc+L//+z9AMuWqVCr+++8/rly5wtdff83ly/oTY41BlaopmNtWntMVRS03o19hNlG4xNoSL4yndu2hZGTEcf78CL3tXV0n4uT0QgVEqs+330qjR0uEKEqL48ydC5MmgQF9sDEp6sleZaMqstzCyeKRaga5KOpsCUWdHSa/X5g6u3bt2vLPr776qrx+xo8//kiPHj2oVq0azs7OtG3blvDwcJ5++ulCz62sqFI1hRuLbvDf0v8qOgwAMjJi0Saupw43MRfTEcWsnBIRrTZV76XTZRW5v/KkVy/II7UsHFGU1lKeO1casjR/fhmvz/nkoqizFXU2QFzcw1URN2/eLI8kq1+/Pnv27EEURR48eMDhw4dp2rRpUV9lmVF5Hp3Lgfgf41HZqKg7rm5Fh4JWK3UOt2nyJW51XsIip8PV0rIu/v4HKzK0YvnrL2kthWIWrZL6EObPh7fekpbTVBKCjKLOVtTZAQEBvPvuu0RERCAIAu7u7vLggDfeeIPRo0fj7e2NKIqMHj0aHx+fYo9bJpREpVqZXqVRZx9vc1yM6BpRaHl5kpoaLZ482Um8e3dfRYdSLDqdKJ48Kb3CwyWr9cyZJfzw339LOygnFHV22aKos59MFHX2E4i1dSNu1NvIhsQkPq9ReDtxZSA7G/wKNK0XOhAit1O5Tx+pY9koizYrlBeKOrvqoSSFCmRXYiIrb97ki3LoPHocYmLgxx9h2DDI2zxrbg5dDC3WqtPBxInS/ANraykpKFQJFHW26aAkhQoiMXEnnW6+yjHVlxUdSqFcuCDZKDp1gr59i9lYp4PXX4fvvoNp06RZywoKCk8cVSop+P396MMLjUF8ZiaTzh7mde017C0r71eQM1G6+DkJOh289hp8/z0EB8PnnyudygoKTyhVakiqYCYgmFX8zcpCENDYSMlgnGujCo6mcEqcFERRWiTnf/9TEoKCwhNO5X1MLSXxP8Zzbti5fO8JlgKNv26M6xuuhXyqfKhZrRqjatcgKhl613Kr0FiKIleeWWhS0Ong7l1pJtuaNZLHSEkICgpPNCZbU6jRvgYNP2tIg+kN5Ff94PrYBdhVaFxpWi1eR49y5J602I+ZmfHEdaUlt6Zg0K2n1Ur+7NatITlZWhxHSQgA3Lx5kxdffJFGjRrh6elJr169uHjxYpkeIywsjH/++afQ8i1btvDll4/fX9WjRw9u3LhBVlYWwcHBeHh44O3tTYsWLdi+fftj7TOv/nvVqlX891/xE0lHjRolTwirrBw/fhy1Wk3jxo2ZPHmywVncd+/epV+/fvj4+NCiRQu5U/7ChQuy/M/X1xd7e3vmzZtX3qeQD5OtKVjVs6LB/xpUdBiANBfk3r0wsrOTuK/NJj01ifv2bng89RwqVWl0o8Zl3Dh48UUDw09zE8Lq1TB9urRIjgIgfdf9+vVj5MiRrF+/HoCIiAji4+PlCVFlQVhYGLa2trQxIBbMzs6mT58+BidvlYS0tDQSExNxdXUlODiYuLg4Tp8+jaWlJfHx8ezbt++x9ptX/71q1Sq8vb2pW7fiJ5KWlgkTJrB06VJatWpFr1692LFjBz179sy3zeeff46vr68sO3zjjTcIDQ3lmWeekWeta7VaXF1d6devX0WchozJ1hTun7xP3Kq44jcsBx48OMOpU505c6Yf184PojdbybB/HrX6DwSh8i4/aWEBTk4F7NZaLYweLSWEGTOkl4LM3r17qVatWr4boK+vL+3bt0cURaZOnSqrqXM1z2FhYbLzBmDixInyrFl3d3emT5+Ov78/arWa8+fPExMTw5IlS5g7dy6+vr78/fffjBo1iilTptCpUyemTZvGqlWrmDhxIgC3b99mwIABBAYGEhgYyMGD0oz5whTVuart1NRUvv/+exYuXCi7k2rXrs3gwYMB6WYYEBCAl5eXrLLOjXnatGm0aNGCFi1ayIqLXD31pk2bCA8PZ9iwYfj6+pKWlsbMmTMJDAzE29ubcePGGXzaNqQUL4yYmBjat2+Pv78//v7+cq2qqGt97Ngx2rRpg0ajoUWLFvL1KIq4uDiSk5Np3bo1giAwYsQIg7PIz549S5eccdy56o9cbXcuoaGhNGrUSNaGVBQmW1NI+C2BqzOv4jLKpaJDQaeTGucbN15AsmUAP52J5ZsnYB2BrVvh6FH4+OM8b86cCWvXSm8WIR6rDLwVFUVEGa8K5mtry7wcVbQhTp8+TfPmzQ2W/frrr0RERHDq1CkSEhIIDAykQwnmcjg5OXHixAm+/fZb5syZw7Jly3jttdewtbWVb47Lly/n4sWL7N69G5VKlU/F8Oabb/L222/Trl07rl27Rvfu3Tl37pxBRTXA9u3b6du3L9HR0dSvX79Qj/9nn32Go6MjWq2WLl26EBkZKasY7O3tOXr0KGvWrOGtt97izz//lD83cOBAQkJCmDNnDgEBAYB0c84V2Q0fPpw///yT559/Xv5MYmJisUrxvDg7O7Nr1y6srKyIiopi6NChhIeHF7p9ZmYmQ4YMYcOGDQQGBpKcnIy1tTUXLlxgyJAhBj8TFhbGjRs3cHN72C9YmCJbo9Hw66+/0q5dO44ePcrVq1eJjY3NJ8Rbv349Q4cOLfK8ygOTTQqVCzPMzGywsWnCXQs1CWSUarnMtDRpYlleateWmnnS0yFH55IPFxdwcJA6j69e1S93cwM7O0hJgdzF3zZsgC1bCiSFiRPB1VVqW1J4JA4cOMDQoUNRqVTUrl2bjh07cuzYsWIXT+nfvz8AzZs359dffy10u0GDBqFS6dc8d+/ezdmzZ+Xfk5OTuX//vqyoHjZsGP3795dvbgcPHmTOnDmcP3++yLh+/vlnli5dSnZ2NnFxcZw9e1ZOCrk3t6FDh/L2228XuR+QalizZs0iNTWVxMREvLy88iWFvErx3r1753vaN0RWVhYTJ04kIiIClUpVbJ/OhQsXcHFxkV1Fud9J3uYdQxiq0RiyEwQHB/Pmm2/i6+uLWq3Gz88P8zzK/MzMTLZs2VKuy24WhpIUygF7+wA6dJBslXdSUwmqWRPXPCrjR2XIEPjjj/zvhYTAG29IE85yhJb5WL0aRoyAkycNmyd+/RX69ZMcdnmbQ93dkTwXISHS5LRatZ6YhFDUE72x8PLyKrRj1NANBArXNueS23RTmPI6l7xq7rzodDoOHTqEdYFhZIYU1RYWFtSrVw8LCwsaN27MtWvXuH//PnZ2+QdoXLlyhTlz5nDs2DFq1qzJqFGj8sWd98ZYnMIlPT2d119/nfDwcOrVq8eMGTP0roG5uXmRSvGCzJ07l9q1a3Pq1Cl0Op1cC3pURXZxNQU3Nzdic1ZOBEmRbaifxN7enpUrV8rHatiwhyWhbwAAIABJREFUIQ3zGCW3b9+Ov79/vppDRaEkhXLGw8aGvzSaUu3jmWekmkHXrg/f8/eX/m3QAHL6N/ORK6Fs0sRwec4DEhpN/nLPJtkwfLj0Zr16MGBAqWI3dTp37sz//vc/vv/+e1599VVAaqtOTU2lQ4cOfPfdd4wcOZLExET279/P7NmzycrK4uzZs2RkZJCenk5oaCjtinFG2dnZkZycXKKYgoKCCAkJkVdpi4iIwNfXV1ZUq9VqDh06xPnz57lx44a8UpyNjQ2vvPIKkydP5rvvvsPCwoK4uDhCQ0NRq9VUr16dGjVqEB8fz/bt2/OtF7BhwwaCg4PZsGGDbDYtGH9um33ujdnJyYmUlBQ2bdrEwIED822fkpJCamoqvXr1olWrVjRu3BiQdNNHjx7Ve8JOSkqSFxVavXo1Wq0WkBY6MnStmzZtyn///cexY8cIDAzk/v37WFtbF1tTcHBwwM7OjsOHD9OyZUvWrFnDpEmT9La7d+8eNjY2WFhYsGzZMjp06JCvhvjTTz9ViqYjUJKC0Tn/4AGJD6LIjvscc+dJmNn44GFjw1OlWOzcwNoeMg4OUk2iMGrVKrrcxSVPeXa2JD76+Wf46islIZQAQRDYvHkzb731Fl9++SVWVla4u7szb948OnTowKFDh9BoNAiCwKxZs6hTpw4AgwcPxsfHBw8PD/wK2gcN8Pzzz8tLNi5cuLDIbRcsWMAbb7yBj48P2dnZdOjQgSVLlhhUVA8cODDf/j799FM++OADPD09sbKyonr16sycORONRoOfnx9eXl48/fTT8opruWRkZNCyZUt0Op3BhYRGjRrFa6+9hrW1NYcOHeLVV19FrVbj7u4uN+Hk5f79+waV4pcuXTLY/Pb6668zYMAANm7cSKdOneRaVL169QxeawsLCzZs2MCkSZNIS0vD2tqa3bt3Y2trW8w3AYsXL2bUqFGkpaXRs2dPeeTRkiVLAGnU1blz5xgxYoR8rZcvXy5/PjU1lV27dsna7AqnJCrVyvQqqTo742aGeD/yftF+2XKg16lTYtO934p79yK23PuFyN69osvBg2JMWlpFh1Y0mZmiOGiQ5MmeNauioykxijr78UlPTxcL+/t6FBo0aCDevn27DCIqnmHDhom3bt0ql2M9SSjqbANY1LbAorZFRYfBzIYNSagZD5dgZgN3dPZqHMzNaZDTxvk4vPACWFlJHcFG4/Jl2LkT5syRltNUMHksLS2LHKFTGVm3bl1Fh2BymGxSSD6STPLRZNwmVZxGIjk7mzoWFjS2t+ck0NzenqdKvLhx4SQkFDLLuCzQ6aSJCc88AxcvgrOzkQ6kYKrEFBwap/BEUfkHyz8md7bdIXpydIXGMC82FrdDhyh86fJKRmYmDBwo9R+AkhAUFKogJpsUKhNmQjWqVauNIFR8c1ahZGZKPcybN5dAi6qgoGCqmGzzkTG4eXMtly+/p/e+n9/fWFs35MaNxVy9+pn8fkutlp/JxsLyNG3b3izPUB+NzEwYNEiaqbZwoTRBTUFBoUqiJIVHwNKyHo6OPfTeV6mkBn4rq4b5yq+m3OdESgovmD3+RDVDdO0KpZj7lh9RhMGDpYSQOwNOQUGhyqI0H5UAUYQrVxK5fbshFhbLsLdfRtOm0svGZhmxsbW5fBmSknpgYbEMBwepLMrpK+bc/IBr12pw+TLy69ath/u+coV8ZZcvw+3bD49bsOzyZZg8WVrPpkwQBOjdG779VkkIZYSizjZMVVZnz549W5YPent7o1KpSExMBGDMmDE4Ozvj7e1d3qEbpiTjVivTq6TzFDITM8XUK6klHNVbNKtXi+L48VPF7dutRRDFZ599WNa4sTSUP+/r+eelskP37ol2tbL1yl966eHnbWz0Pz9+vFSm0+mXgSi+804ZnFRamigeO1YGO6pcVPQ8BZ1OJ7Zq1UpcvHix/N7JkyfF/fv3l+lxpk+fLs6ePdtgWVZWVqn2nZqaKgYGBoqiKIrTpk0TR4wYIaanp4uiKIo3b94UN2zYUKr9i6IoduzYUTxWgv9/I0eOFDdu3Fjq4xmTwMBA8Z9//hF1Op3Yo0cPcdu2bUVuv2XLFrFTp07y7/v27ROPHz8uenl5lVlMyjyFPIhZIvcjpOnzNk3LZtxm7pO7hYXkEMqrJ5k1CwoadnOlia1q1GBZiCSpy0se5Qnffy9NHM5LXu3+6tX68Xh6Plr8eqSnQ//+sG8fXLoEObNqFUpPYepskB7A3n33XbZv344gCHzwwQcMGTKEsLAw5syZI5tEJ06cSEBAAKNGjcLd3Z2RI0fyxx9/kJWVxcaNG7GysmLJkiWoVCrWrVvHwoULWb58OY6Ojpw8eVLWbIeHhxMSEsLt27d57bXXuHbtGgDz5s2jbdu27Nu3jzfffBOQZmLv378fOzs7PXX2lStXClVnHzt2jLS0NAYOHMjHOeZEd3d3hgwZwt69ewH48ccfady4MTNmzMDW1hZ3d3dZnZ07o3n27Nn88ccfpKWl0aZNG7777js9F1FwcDBbtmzB3NycoKAg5syZU+j3EBMTw/Dhw3nwQHKOhYSE0KZNmyKv9bFjx3jzzTd58OABlpaWhIaG6jmfCpJXnQ3I6uyC6ynkpaDSokOHDpVqGK/JJYWUyBSO+x3Hoq4F9abWo95b9QxuFxIbS3RaGo2trZmYcxf/7OpVbmdm5tvOs3p1GjSoS1YWiGYiJ1pHAbBd+gf/1naMyLmpTrt0iQydjhPAlihI1+kY37Muvra2hUrBXnqp8HMRBEliV6akp0PfvvDXX1JGMvGE8OzJk3rvDXZ25nVXV1K1WnpFRuqVj6pTh1EuLiRkZjLwzJl8ZWHFKCgUdbaizjakzs4lNTWVHTt2EBISUmT8FYnJJQUxW8ThWQdc33TFyt3wrOE0rZZJ0dFYCgJda9aUk8Lm27eJzl2DMofujo5sGFiXS5fg0nWRVTfzjyK6r9XKSeGH+HhScsRbAFmiSERKCodybXUVTVqalBB27YJly6TV0xTKDUWdXTimrs7O5Y8//qBt27Y46i1nWHkwuaQAYFnfklp9axVanpqjzp3VqBGT82T58JynlsKwMjPjXvv2hZbHGlgasVKxZMnDhDBmTEVHUy4U9WRvo1IVWe5kYVFszaAgijpbQlFnG6ayLKRTFCY3+sjcwRxbTdFmw7Scp/mSLnSzdCmMH/8Cbm5flzq+CmXyZAgLqzIJoSLo3LkzGRkZfP/99/J7x44dY9++fXTo0IENGzag1Wq5ffs2+/fvp0WLFvl0zklJSYSGhhZ7nLzq6eLIVWfnkvvkm6vOnjZtGgEBAZw/f57t27cbVGdn5jSrxsXFsW7dOpKTk/XU2XnJXWr0cdXZBUlJSSEpKYlevXoxb948+Rw2b97Me+/pzx1KSkrCxcUFMzMz1q5da1Cdnfda51Vng2Rlzc7OlmsKhl4ODg64uLjI6mxRFFmzZg0vvPCCwe8hKSmJffv2FVpeWTC5pGDzjA31phjuR8jFXBDo7ehIwxJK6e7fh9DQttSqNaEsQixfUlOlJHDtGqhUUII2bIXHJ1edvWvXLho1aoSXlxczZsygbt269OvXDx8fHzQaDZ07d5bV2Xl1zsOGDSuxOnvz5s3yGs1FsWDBAsLDw/Hx8cHT01NWOs+bNw9vb280Gg3W1tb07NmTHTt2yEkBJHV2rVq18PT0xNvbm759+1KrVq186uwxY8YUqs6eP3++rLnOS64629fXF0tLS1md3bdv30LV2c899xw+Pj507NixROrs1atX06pVKy5evGhQnZ33WudVZ2s0Grp166ZXWymMxYsXM3bsWBo3bkyjRo3yqbNzrzVICSwoKEivRjd06FBat27NhQsXcHNzy6fVrhBKMkSpMr1KOiS1LJkzRxRr1owT4+NPl/m+jUpKiih26iSKgiCKlXxYX1lR0UNSn2QUdbbpoAxJzUPKiRSi3ozCY/6jLcX44MF5EhI2Y239NM7OUhtibGwIWu196tSB6dO3c+5cOM7OqcYIu+x58ACeew7274c1ayTRnYJCESjqbAUwwY5mXZaO7HuFd8YBHExKYtCZM/zi5UXrGjUAuHbtC+Lj1+Do2ENOCtevf0VGRiyurtJa9VZWaqPHXyY8eCDNUv77b1i7tuhxrwoKZUxlGnOv8OgYtU9BEIQegiBcEAQhWhCEYAPlUwRBOCsIQqQgCKGCIDQwZjy53E3PJu6GwK0bArkz7bXa+1haelKz5u/ExkJsLLi5RePhkU5ycjrffpuOWl340LRKRUaG1Jewbp2SEBQUFB4Jo9UUBEFQAYuAbkAscEwQhC2iKJ7Ns9lJIEAUxVRBECYAs4AiVhAuG2a/UR1+aU1f4KmnpEVrdLo0rl6tTps2efXWlri7S36iPn2MHVUZkJIC1aqBoyMcOiR1LCsoKCg8AsZsPmoBRIuieBlAEIT1wAuAnBREUdybZ/vDwMtGjEfmTrwArql88j8VDeykMeBabSq1a9uQZyQhACVYt7tycP8+9OwJTk7SmghKQlBQUHgMjJkUXIHreX6PBVoWsf0rwOPpF/NQzbEa9i2LniGakJUNjlmMGGNF/ZxRqR4eiwAdnTqVNoIKIDlZSghHjsD69ZIfQ0FBQeExMGafgqE7k8EpnYIgvAwEALMLKR8nCEK4IAjht3PtdIVg7WGN6+uuRW7zv0+yeeXjVOrlWZTA1tYbW1ufIj9XKUlOhh494OhR2LBBGWVUCVCpVPj6+uLl5YVGo+Gbb76RZ9GGh4czefLkMjlOSfXTJSUrK0v2NgmCwPDhw+Wy7OxsatWqVaxeojDyarOfNBITE+nWrRseHh5069aNu3fvGtzu3XffxcvLi2bNmuVTaGdmZjJu3DiaNGlC06ZN+eWXX8oz/EfGmEkhFsg7i8wN0PsfLAhCV+B9oI8oihmGdiSK4lJRFANEUQyoVatwfUVJWBUXR7PmWpb1q5tvWvutWz+TlHSoVPuuEF56CY4dkxLCgAEVHY0CYG1tTUREBGfOnGHXrl1s27ZNNogGBASwYMECvc8Upa8ojMdJCto8bq6CHDhwgDY5qpbq1atz+vRp0nJcYLt27cLVteiHraJ47bXXGFHmdsfy4csvv6RLly5ERUXRpUsXg+tU/PPPPxw8eJDIyEhOnz4tz2IHSRzo7OzMxYsXOXv2LB07dizvU3gkjJkUjgEegiA0FKTFiV8EtuTdQBAEP+A7pIRwy8A+Hpn74fe58NqFQstnxMTwzdZkcr4vmejoN7l5c2VZhFC+fPIJbNokqbAVKh3Ozs4sXbqUkJAQRFEkLCxMftqeMWMG48aNIygoiBEjRqDVapk6dSqBgYH4+Pjw3XffyfuZNWsWarUajUZDcHAwmzZtkvXTvr6+pKWlERoaip+fH2q1mjFjxpCRIT1jubu7M3PmTNq1a8fGjRtZsGABnp6e+Pj48OKLL8rH2LFjRz7lc8+ePdm6dSugr3tOTEykb9+++Pj40KpVKyIjI9HpdLi7u+czmDZu3Jj4+HhmzJghq64vXbpEjx49aN68Oe3bt5fFexs3bpRnWJfEHtu3b1+aN2+Ol5cXS5culd+3zdMRuGnTJkaNGgVAfHw8/fr1Q6PRoNFoilykKC+///47I0eOBGDkyJH89ttvetsIgkB6ejqZmZlkZGSQlZVF7RzH/ooVK2QVh5mZGU5OTiU6bkVhtD4FURSzBUGYCPwFqIAVoiieEQRhJtLMui1IzUW2wMacp/ZroiiWapyPqBXRpemK3CZiUS0+qCYN489Fq03FzKxs1l8wOvfuwS+/SJZTPz/ppaDHW29BEYLLx8LXF+bNe7TPPP300+h0Om7d0n/uOX78OAcOHMDa2pqlS5dSo0YNjh07RkZGBm3btiUoKIjz58/z22+/ceTIEWxsbEhMTMTR0TGffjo9PZ1Ro0YRGhpKkyZNGDFiBIsXL+att94CwMrKigMHDgBQt25deY2EvDfwvXv3Mn36dPn3F198kZkzZ/Lcc88RGRnJmDFjZKXG9OnT8fPz47fffmPPnj2MGDGCiIgIXnjhBTZv3szo0aM5cuQI7u7u8s0xl3HjxrFkyRI8PDw4cuQIr7/+Onv27GHmzJn89ddfuLq6FqvGBulm6+joSFpaGoGBgQwYMICnnnqq0O0nT55Mx44d2bx5M1qtlpSUFADat29v0CM1Z84cunbtSnx8PC4uLgC4uLgY/B5bt25Np06dcHFxQRRFJk6cSLNmzeTz+PDDDwkLC6NRo0aEhIToXZPKhFEnr4miuA3YVuC9j/L83NWYx38UdLo0VCrr4jesaO7dg6Ag6W7Xvn3+FXkUKi257csF6dOnj2wv3blzJ5GRkbIQLikpiaioKHbv3s3o0aOxsZEeWgxply9cuEDDhg1pkvP/YeTIkSxatEhOCnlNn7nen759+9K3b18A/vvvPxwdHeVj5G4XExPDTz/9RK9evfId78CBA3LbeOfOnblz5w5JSUkMGTKEmTNnMnr0aNavX69nGE1JSeGff/5h0KBB8nu5NZq2bdsyatQoBg8eLOvCi2LBggVs3rwZgOvXrxMVFVVkUtizZ4/cr6FSqaiRM3G1OHdUSYiOjubcuXOyMbVbt27s378fT09PYmNjadu2Ld988w3ffPMN77zzDmvXri31MY2Fyc1oLgmWFilYVNOh1ZqhUlVHp8tGFLMqf03h7l0pIURGwq+/KgmhGB71id5YXL58GZVKhbOzM+fOnctXlleOJooiCxcupHv37vm22bFjR7H66cKSjqHjbN26lf3797NlyxY++eQTzpw5w/bt2/WOC1LSeueddwgLC+POnTtFHk8QBFq3bk10dDS3b9/mt99+44MPPsi3jU6nw8HBweAaBUuWLOHIkSNs3boVX19fIiIiCr3Jh4WFsXv3bg4dOoSNjQ3PPvusLLDLe61KIrUrrqZQu3Zt4uLicHFxIS4uDmdnZ71tN2/eTKtWreSmq549e3L48GHat2+PjY0N/fr1A6Q1LypceFcMJmdJpeiWI3plr2XVzDp8+KE9J05InWqimA2oUKmKXnqvQklMhK5dHyaExxwFolC+5C6FOXHixGJv7N27d2fx4sVkZWUBcPHiRR48eEBQUBArVqwgNVXybuUu+J5XP920aVNiYmKIjo4GYO3atQY7NHU6HdevX6dTp07MmjWLe/fukZKSotefkMuYMWP46KOPUKvzK146dOjADz/8AEg3aCcnJ+zt7REEgX79+jFlyhSaNWumd1O3t7enYcOGbNy4EZCSy6lTpwCpr6Fly5bMnDkTJycnrl+/zo0bN+jSpYteXElJSdSsWRMbGxvOnz/P4cOH5bLatWtz7tw5dDqdXJMA6NKlC4sXLwakDvfk5GRAqikYUmN37So1ZPTp04fVOevirl692qD6un79+uzbt4/s7GyysrLYt28fzZo1QxAEnn/+ecLCwgAIDQ3Fs9Tr6RoXk0sKlvUtsahtYbAsNRXGPpVNRmZ1tm+fQ716/weASmWFp+eP1Kkz3ODnKgV//w3nzsFvv0leI4VKS1pamjwktWvXrgQFBeVrqy+MsWPH4unpib+/P97e3owfP57s7Gx69OhBnz59CAgIwNfXV+6wzaufFkWRlStXMmjQINRqNWZmZvnWic5Fq9Xy8ssvo1ar8fPz4+2338bOzo6oqCiaNm2qt72bm5u8jnNeZsyYIeu4g4OD5ZsmSE1V69atK3Rxmh9++IHly5ej0Wjw8vLi999/B2Dq1Kmo1Wq8vb3p0KEDGo2GuLg4zM31GzR69OhBdnY2Pj4+fPjhh7Rq1Uou+/LLL3nuuefo3Lmz3BcAMH/+fPbu3YtaraZ58+acKbDUamEEBweza9cuPDw82LVrF8HBkrEnPDycsWPHAtISo40aNZIHA2g0GnnluK+++ooZM2bg4+PD2rVr+frryr0ui1BctbOyERAQIBoyOYYsb4+YLdJm5Xz8//FHMMv/VJaUJEnt/v57AsnJv+LoGI/6SfDb6XSQuxjQzZsmv6ZyaTl37hzNmjWr6DCeKA4cOMC6devyuf8rCyEhIdSvX58+T4RnpvJg6O9AEITjoigWvbwkptSnkJPbsm5n6SUEkJLCgwew7LCaF9qrUHuXc3yPQ0KC1Ew0Y4Y0QU1JCApGoF27drRr166iwzDIxIkTKzqEKofpJIUS8u2Vzph17UZQRQdSHAkJ0KULXLyoeIwUFBTKDZPrUygOAR2fuZeLofvxuX0bOneWEsIff0C3bhUdkYKCQhWhyiWFd9Wf8u/RotdwrlCSkqSEEB0Nf/4pjThSUFBQKCeqTFKoVg0cm6VjZlHMmNWKxs5OSgp//ik1HykoKCiUI6bTp5DTt2zuaPiUXFxg++FMkmKsQX+eSsUTHy+NmW3YEObPr+hoFBQUqiimVVMQQFW98E7ZFvb2NLCyKseASsjNm9CpkzTSqAiLpcKTgaLO1qcqqLNzv3dfX998Q2hfeeUVNBoNPj4+DBw4UHYuVVZMKymIoMsw3Dx08yZ4+2s5EZVVzkEVQ1yclBCuXYPFi5WRRiaAos7Wx9TV2fDwe4+IiGDLlodC6Llz53Lq1CkiIyOpX78+ISEh5RX6Y2E6SSF3nsIt/Zt+cvJRYmNHoUsPZ3W0H66ur5dzcIWQmxCuX4ft26EEumCFJwtFnV011NlFYW8vrQQpiiJpaWnF6k4qGpPpUxABUYQHOhU5ShOqV5cevG/cWEVKylpq1BjEkVvdcXdvXJGhPmTqVLhxA3bsgEo6eehJ59ln9d8bPBhef13qwikg/wRg1CjplZCgv5BdjsLmkVDU2aavzgZJvhcQEIC5uTnBwcGygRZg9OjRbNu2DU9Pz0qvuTCZpKBLM0PUCnSNCQTJiMu//4K3N5w/n0pGhhuHD/fGsVtUxQaal5AQSfofUOzMc4UnHEWdLWGq6myAa9euUbduXS5fvkznzp1Rq9U0atQIgJUrV6LVapk0aRIbNmxg9OjRZXJMY2AySUEUAUFksmMMbv+rz87Eu3ydcpcuN21p6JzK/fs21H4nBvMOhjuJyo0bN+Djj6URRg4OSkIwMkU92dvYFF3u5PR4NYOCKOrsh5iqOhukGhhINcNnn32WkydPykkBpEQ0ZMgQZs+eXamTgsn0KZjb6FBZaRnqcJMe41PZ3e1ffs2KI/z+fezt03Byskbo9x8d69oWvzNjcf06dOworaccVYlqLApGQ1FnVw119t27d+UaT0JCAgcPHsTT0xNRFOXvRBRF/vjjD4M22sqEydQU5HkKT5mTmjP87ydPT3o99RQ3bw5Bq00hzrVNxcV37ZrUqZyQADt3go9PxcWiYFRy1dlZWVmYm5szfPhwpkyZUuznxo4dS0xMDP7+/oiiSK1atfjtt9/o0aMHERERBAQEYGFhQa9evfj8889ldba1tTWHDh2S1dnZ2dkEBgYWqc5OSkpCFMVSqbNHjx6Nj48PNjY2eurswMBAVq1aZfA8f/jhByZMmMCnn35KVlYWL774IhqNhqlTpxIVFYUoinTp0gWNRsPx48cLVWcvWbIEHx8fnnnmGYPq7Hr16uHt7S33HcyfP59x48axfPlyVCoVixcvpnXr1sV+L8HBwQwePJjly5dTv359OaGFh4ezZMkSli1bxrlz5xg/fjxmZmbodDqCg4Px9PREp9MxcuRIkpOTEUURjUYjJ6bKismosxd82wFEaL9xPkmb3el06hR7NBo61axZAVEW4No1qcczMVFKCC1aVHREJouizn50FHW26aGoswFdtoAggC5dh4O5OX2dnKhjIS22k5kZj5mZNebm9hUT3P37YG4Ou3ZBYGDFxKCgUAiKOlshLyaRFLRpWmlMqiCtp9Dczo7N3g8XTDh5sj12ds3x9PypfANLTISaNcHLC86elRKDgoKCQiXGJDqa7x9/OHLgqef1RyvodGmYmdnovW9UrlwBf3/4/HPpdyUhKCgoPAGYRFIgT7eIxzwP1ty8Sa2DB7mRMxpAq03FzMy6/OK5fFnqQ7h/3/DsKAUFBYVKiuk8vgqQmx2SsrNJyMrCQhDQatPIzk5EpSqnmsKlS9IoowcPIDQUfH3L57gKCgoKZYBJ1BTsW9qjstJiZikNRU3LGZJqbWZGenoMANWqFT7TscxIT5cWxUlNhT17lISgoKDwxGESScHMIv9ppOt09OcXrkW9gqVlPXx8duDqOsn4gVhZwaxZUg1BozH+8RQqJYo6W5+qoM6eNm0a3t7eeHt7s2HDBvn9PXv24O/vj7e3NyNHjpSNuHfv3qVfv374+PjQokULTp8+nW9/Wq0WPz+/fNc8NDQUf39/fH19adeunTwxriwxiaSQfjUdbaYZuizpdDbevk0TLpKctB9zc1scHbsbt/koKgq2bZN+HjRISQhVHEWdrY+pq7O3bt3KiRMniIiI4MiRI8yePZvk5GR58tr69es5ffo0DRo0kCf6ff755/j6+hIZGcmaNWv0JgnOnz9fb67BhAkT+OGHH4iIiOCll17i008/LfPzNY2kcC0ddAKiTprW/EGDBrSxrVY+ncsXL0rqinHjIOcPSEEhF0WdXTXU2WfPnqVjx46Ym5tTvXp1NBoNO3bs4M6dO1haWsqiwm7duskiwbNnz8oKj1xNSXx8PACxsbFs3bqVsWPH5juOIAiyniMpKUn2LZUlptPRnIchzs5E3oTMTCN3Ll+4IHUqa7XSxDTrchzhpFAsUVFvkZKiL14rDba2vnh4zHukzyjqbNNXZ2s0Gj7++GOmTJlCamoqe/fuxdPTEycnJ7KysggPDycgIIBNmzZx/fp1+TO//vor7dq14+jRo1y9epXY2Fhq167NW2+9xaxZs/TiWrZsGb169cLa2hp7e/t8zqeywiRqCobQ6dJQqYx4kz5/Xhp2qtXC3r3SBDUFhUIoqTp7zZo1+Pr60rJlS+7cuVMqdfb+/fvlckPq7HXr1sk8BCm7AAAQn0lEQVReocdRZ+f2ORRUZ+e2pxenzvb19WX8+PHExcUBD9XZ33//fZHNXLksWLAAjUZDq1atZHV2UezZs4cJEyYA+ursooR4JSEoKIhevXrRpk0bhg4dSuvWrTE3N0cQBNavX8/bb79NixYtsLOzk695cHAwd+/exdfXl4ULF+Ln54e5uTl//vknzs7Ocv9OXubOncu2bduIjY1l9OjRJXJqPSomUVMw9AdnYeGCIBjx9H78UfJ1790Lnp7GO47CY/OoT/TGQlFnP8SU1dnvv/8+77//PgAvvfQSHh4eALRu3VquYe3cuZOLFy8CkjF25cqVgHQ9GzZsSMOGDVm/fj1btmxh27ZtpKenk5yczMsvvywv69myZUtASvQ9evQo9vweFZOoKaQjSvMUhIf/UT0919Gs2aqyP1juH8PHH8OJE0pCUCgSRZ1dNdTZWq1WTpqRkZFERkYSFBQEIDc3ZWRk8NVXX8n22nv37pGZmQlIzUIdOnTA3t6eL774gtjYWGJiYli/fj2dO3dm3bp11KxZk6SkJDmp7Nq1yyjyR5OoKdRq44DZ+eKrm6XmzBkYPlxaD8HDA4zQyaPw5KOos6ueOjsrK4v27dsDUuLL2zQ3e/Zs/vzzT3Q6HRMmTKBz586AZDIdMWIEKpUKT09Pli9fXmQc5ubmfP/99wwYMAAzMzNq1qzJihUrio3/UTEZdfbCpZLlcdI4qTPt33/7UqNGO+rXf6dsDnz6NHTuLDmM9u6FZ54pm/0qlCmKOvvRUdTZpkeVV2ffjLqPNtOMvLXzpKS/sbKqVzYH+PdfKSFYWEgJIaczT0HBFFDU2Qp5MYmkcDkuBXQCtnaJHD/eCtCRnX23bOYpnDsnJQRLSykh5HQeKSgoKJgiptPRDKRn2ODg0JFq1ZxwdOyFk1P/0u/czU2aixAWpiQEBQUFk8ckagrp6ECErCwrGjX6qmx2eu4c1K8Pdnbw889ls08FBQWFSo5J1BQkBGxt75KQ8Hvpd3XyJLRrB+PHl35fCgoKCk8QppMUBBEnxzguX36vdPs5cQK6dAFbW5g5s2xiU1BQUHhCMImk0KFNHcystZiptKVbdvP4cSkh2NtLfQhPP11mMSpUHRR1tj5VQZ197do1goKCaNasGZ6ensTExOQrnzRpUj5ZX2XFJJKCrbk5ZmYCZird4/uOtFppYpqDg5QQGjYs0xgVqg6KOlsfU1dnA4wYMYKpU6dy7tw5jh49mk+HER4eXiLJX2XAJJJCdGIG2Q/MENA9fk1BpYJff5USgrt7WYanUIVR1NlVR52dnZ1Nt27d5Bhy5YK53+usWbNKdLyKxiRGH529I62nYCboHn0xnSNH4M8/pf4DA9P8FZ5sTp58Vu89Z+fBuLq+jlabSmRkL73yOnVG4eIyiszMBM6cGZivzM8v7JFjUNTZpq/OvnjxIg4ODvTv358rV67QtWtXvvzyS1QqFSEhIfTp00feR2XHJJJCRs7aNjE3mvFsx29K/sHDh6F7d3BygrffBgNKYgWFsqCk6uzIyEg2bdoESNK30qizFy1aJCcFQ+rsvn370rdvX+Dx1Nm5i8UUVGfPnDmT0aNHF6vOziW3RpOrzh48eDD9+xc/x2jBggWy8C5XnV1UUtizZ4/cr1FQnV1asrOz+fvvvzl58iT169dnyJAhrFq1ip49e7Jx40bCwsJKfYzywjSSQrrkt8jKssbauoR9AYcOSQnB2VlqMlISgklS1JO9SmVTZLmFhdNj1QwKoqizH2Kq6mw3Nzf8/Px4OmdwSt++fTl8+DB16tQhOjqaxo0bA5Camkrjxo2NsrZyWWESfQoZ6QJWVg9wc7lAUtLB4j/wzz8QFAS1a0sJwc3N6DEqVE0UdXbVUGcHBgZy9+5dbt++DUi1Ek9PT3r37s3NmzeJiYkhJiYGGxubSp0QwESSQrVqIs7O13GscatkSSE+XpqtrCQEBSOQq8728vKia9euBAUF5WurL4yxY8fi6emJv78/3t7ejB8/nuzsbHr06EGfPn0ICAjA19dX7rDNVWf7+voiiqKszlar1ZiZmRWpzlar1fj5+ZVKnR0eHo6Pjw/BwcF66ux169bpNR3l8sMPP7B8+XI0Gg1eXl78/rs04XTq1Kmo1Wq8vb3p0KEDGo2GuLi4QtXZ2dnZ+Pj48OGHHxpUZ3fu3DlfO/78+fPZu3cvarWa5s2bc+bMmcK+inwEBweza9cuPDw82LVrF8HBwYA0oih3DWWVSsWcOXPo0qULarUaURR59dVXS7T/yoZJqLPTtFp++V0DohnD+p8q/Ins3j1pyClAVhZUq2bkaBXKG0Wd/ego6mzTo8qrs61VKp6pZ46VlXvhCWH/fnjhBVi3Dnr3VhKCgkIOijpbIS8m0XxULPv2Qc+eUKcOGFgMW0FBQUFBwiRqCgBNmnxreP2EsDCpZtCgAezZIyUGBQUFBQWDmExSqFGjjf6b0dHQq5ekrNizRxptpGDyiKJY7EgfBQVTpbT9xCbTfJSYuJukpALT1hs1gk8+URJCFcLKyoo7d+6U+g9DQeFJRBRF7ty5g5WV1WPvw2RqCpcuvYOVlTtq9W9SEnBxgWbN4P/+r6JDUyhH3NzciI2NlceLKyhUNaysrHArxVB7oyYFQRB6APMBFbBMFMUvC5RbAmuA5sAdYIgoijGlOuiuXdCnD3ToAH/9VapdKTx5VKtWjYaK4VZB4bExWvORIAgqYBHQE/AEhgqC4Flgs1eAu6IoNgbmAqVaS9P24C14/nlo0gRyZlsqKCgoKJQcY/YptACiRVG8LIpiJrAeKDg//AUgdyrkJqCL8Jg9hPaHkmnw5hGpyWjPHklyp6CgoKDwSBgzKbgC1/P8HpvznsFtRFHMBpKAwjWHhSGK1NmQSEYjO9i9G4owJSooKCgoFI4x+xQMPfEXHBJSkm0QBGEcMC7n1xRBEC4UckwnnJwSSh6iyeEEKOdftanq10A5/8LPv0FJdmDMpBAL1MvzuxtQcO3A3G1iBUEwB2oAiQV3JIriUmBpwfcLIghCeEncHqaKcv5V+/xBuQbK+Zf+/I3ZfHQM8BAEoaEgCBbAi8CWAttsAUbm/DwQ2CMqA8wVFBQUKgyj1RREUcwWBGEi8BfSkNQVoiieEQRhJhAuiuIWYDmwVhCEaKQawouF71FBQUFBwdgYdZ6CKIrbgG0F3vsoz8/pwKCCnysFxTYxmTjK+StU9WugnH8peeLWU1BQUFBQMB4m4z5SUFBQUCg9T1xSEAShhyAIFwRBiBYEIdhAuaUgCBtyyo8IguBe/lEalxJcgymCIJwVBCFSEIRQQRBKNBTtSaG488+z3UBBEERBEExqNEpJzl8QhME5/wfOCILwY3nHaGxK8DdQXxCEvYIgnMz5O+hVEXEaC0EQVgiCcEsQhNOFlAuCICzIuT6RgiD4l3jnoig+MS+kDutLwNOABXAK8CywzevAkpyfXwQ2VHTcFXANOgE2OT9PMKVrUJLzz9nODtgPHAYCKjrucv7+PYCTQM2c350rOu4KuAZLgQk5P3sCMRUddxlfgw6AP3C6kPJewHakuWCtgCMl3feTVlMoV3VGJaXYayCK4l5RFFNzfj2MNEfEVCjJ/wGAT4BZQHp5BlcOlOT8XwUWiaJ4F0AUxVvlHKOxKck1EAH7nJ9roD9H6olGFMX9GJjTlYcXgDWixGHAQRAEl5Ls+0lLCuWnzqi8lOQa5OUVpCcGU6HY8xcEwQ+oJ4rin+UZWDlRku+/CdBEEISDgiAczrEVmxIluQYzgJcFQYhFGgE5qXxCqzQ86n1C5klbT6HM1BlPMCU+P0EQXgYCgI5Gjah8KfL8BUEwQzLujiqvgMqZknz/5khNSM8i1RL/FgTBWxTFe0aOrbwoyTUYCqwSRfFrQRBaI82H8hZFUWf88CoFj30ffNJqCo+izqAodcYTTEmuAYIgdAXeB/qIophRTrGVB8Wdvx3gDYQJghCD1J665f/bu98QqaowjuPfH2GpSYJIkQRtYVhJupSF5IswS/pDUiJusWkbSShFaNmLMKigF5L1IjNbS0IDE1O0pD+YhFnImi7hn1osQ0UCKQmRsA1k+/XinB2nbXTv6vpndp4PDMycmXvPuQfmPnOfOzynD91sLvod+MT2cdv7gZ9IQaKvKDIHTwAfAdhuAfqT6gLVikLniUqqLShE6YwCc5DTJ0tIAaGv5ZNPefy2j9oearvOdh3pnsok263nZ7i9rsh34GPSnw2QNJSUTtp3Tkd5dhWZg4PABABJN5CCQi0tx7cemJ7/hTQWOGr7UJENqyp95CidUXQOFgCDgNX5HvtB25PO26B7UcHj77MKHv8GYKKkNqADeN72H+dv1L2r4Bw8B7wnaQ4pbdLUl34cSlpJSg8OzfdNXgL6AdhuJt1HuQ/4BfgLeLzwvvvQPIUQQjhD1ZY+CiGEcBZFUAghhFASQSGEEEJJBIUQQgglERRCCCGURFAIFxxJHZJ2lD3qTvHZupNViuxhn1/nqps7c3mIEaexj5mSpufnTZKGlb23VNKNvTzO7ZLqC2wzW9LAM+071IYICuFC1G67vuxx4Bz122h7NKmg4oKebmy72fYH+WUTMKzsvRm223pllCfGuZhi45wNRFAIhURQCFUhXxF8K+n7/Li9wmdGStqWry52Sboutz9a1r5E0kXddPcNMDxvOyHX5N+da9hfktvn68SaFa/ntpclzZU0hVRzakXuc0D+hT9G0ixJr5WNuUnSW6c5zhbKipxJekdSq9IaCq/ktmdIwWmTpE25baKkljyPqyUN6qafUEMiKIQL0YCy1NG63PY7cLftm4EGYGGF7WYCb9quJ52Uf80lDhqAcbm9A2jspv8HgN2S+gPLgAbbN5EqAMySNAR4CBhpexTwavnGttcAraRf9PW228veXgNMLnvdAKw6zXHeQypp0Wme7THAKOAOSaNsLyTVvBlve3wue/EicFeey1bg2W76CTWkqspchJrRnk+M5foBi3IOvYNUz6erFmCepKuAtbb3SpoA3AJszyU/BpACTCUrJLUDB0illkcA+23/nN9fDjwFLCKt07BU0mdA4RLdtg9L2pfr0ezNfWzJ++3JOC8llXgoX1FrqqQnSd/rK0mLy+zqsu3Y3L4l93Mxad5CACIohOoxB/gNGE26wv3f4jm2P5T0HXA/sEHSDFIJ4eW2XyjQR2N54TxJFdfhyLV3biMVXHsYeBq4swfHsgqYCuwB1tm20hm68DhJq43NB94GJku6BpgL3Gr7iKRlpCJwXQnYaPuRHow31JBIH4VqMRg4lOvhTyP9Sv4PSdcC+3LKZD0pjfIVMEXS5fkzQ1R8zeo9QJ2k4fn1NGBzzsEPtv056SZupX8A/Ukq413JWuBBUs3/VbmtR+O0fZyUBhqbU0+XAceAo5KuAO49yVi2AuM6j0nSQEmVrrpCjYqgEKrFYuAxSVtJqaNjFT7TAPwgaQdwPWk5wjbSyfNLSbuAjaTUSrds/02qLrla0m7gH6CZdIL9NO9vM+kqpqtlQHPnjeYu+z0CtAFX296W23o8znyv4g1gru2dpHWZfwTeJ6WkOr0LfCFpk+3DpH9Grcz9bCXNVQhAVEkNIYRQJq4UQgghlERQCCGEUBJBIYQQQkkEhRBCCCURFEIIIZREUAghhFASQSGEEEJJBIUQQggl/wLR0DyTmmSqCgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x432 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from sklearn import metrics\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"fig, ax = plt.subplots(figsize=(6,6))\n",
"\n",
"#======SPOUSES===================================================================\n",
"#plt.figure().clf()\n",
"#ax.figure()\n",
"y, y_score = get_mx(w_new)\n",
"fpr, tpr, thresh = metrics.roc_curve(y, y_score)\n",
"auc = metrics.roc_auc_score(y, y_score)\n",
"ax.plot(fpr,tpr,'m-',label=\"Presidents/Spouses, auc={:.2f}\".format(auc))\n",
"#------------------------------------------------------------------\n",
"y, y_score = get_mx(w_mold)\n",
"fpr, tpr, thresh = metrics.roc_curve(y, y_score)\n",
"auc = metrics.roc_auc_score(y, y_score)\n",
"ax.plot(fpr,tpr,'m--',label=\"Presidents/Spouses, auc={:.2f}\".format(auc))\n",
"#======COUNTRIES===================================================================\n",
"y, y_score = get_mx(c_new)\n",
"fpr, tpr, thresh = metrics.roc_curve(y, y_score)\n",
"auc = metrics.roc_auc_score(y, y_score)\n",
"ax.plot(fpr,tpr,'c-',label=\"Countries/Capitals, auc={:.2f}\".format(auc))\n",
"\n",
"#------------------------------------------------------------------\n",
"y, y_score = get_mx(c_u_mold)\n",
"fpr, tpr, thresh = metrics.roc_curve(y, y_score)\n",
"auc = metrics.roc_auc_score(y, y_score)\n",
"ax.plot(fpr,tpr,'c--',label=\"Countries/Capitals, auc={:.2f}\".format(auc))\n",
"#======DIRECTORS===================================================================\n",
"y, y_score = get_mx(d_new)\n",
"fpr, tpr, thresh = metrics.roc_curve(y, y_score)\n",
"auc = metrics.roc_auc_score(y, y_score)\n",
"ax.plot(fpr,tpr,'b-',label=\"Directors/Movies, auc={:.2f}\".format(auc))\n",
"\n",
"#------------------------------------------------------------------\n",
"y, y_score = get_mx(d_mold)\n",
"fpr, tpr, thresh = metrics.roc_curve(y, y_score)\n",
"auc = metrics.roc_auc_score(y, y_score)\n",
"ax.plot(fpr,tpr,'b--',label=\"Directors/Movies, auc={:.2f}\".format(auc))\n",
"#======STATES===================================================================\n",
"y, y_score = get_mx(s_new)\n",
"fpr, tpr, thresh = metrics.roc_curve(y, y_score)\n",
"auc = metrics.roc_auc_score(y, y_score)\n",
"ax.plot(fpr,tpr,'y-',label=\"Directors/Movies, auc={:.5f}\".format(auc))\n",
"\n",
"print(thresh)\n",
"\n",
"#------------------------------------------------------------------\n",
"y, y_score = get_mx(s_mold)\n",
"fpr, tpr, thresh = metrics.roc_curve(y, y_score)\n",
"auc = metrics.roc_auc_score(y, y_score)\n",
"ax.plot(fpr,tpr,'y--',label=\"Directors/Movies, auc={:.2f}\".format(auc))\n",
"\n",
"\n",
"ax.plot([0,1],[0,1],'r--')\n",
"\n",
"\n",
"\n",
"ax.set_xlabel('False Positive Rate')\n",
"ax.set_ylabel('True Positive Rate')\n",
"#ax.tight_layout()\n",
"\n",
"# remove this code to remove the border\n",
"ax.set_ylim(0, 1.02)\n",
"ax.set_xlim(-0.02, 1.02)\n",
"\n",
"ax.legend(loc=0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Validation on annotated corpus"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Read the list of GREC statements and their corresponding paths**"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>predicate</th>\n",
" <th>object</th>\n",
" <th>score</th>\n",
" <th>path</th>\n",
" </tr>\n",
" <tr>\n",
" <th>subject</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>Morris S. Miller</th>\n",
" <td>/people/person/place_of_birth</td>\n",
" <td>New York City</td>\n",
" <td>5</td>\n",
" <td>Morris S. Miller>Buffalo, New York>New York City</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Max Ferguson (painter)</th>\n",
" <td>/people/person/place_of_birth</td>\n",
" <td>New York City</td>\n",
" <td>5</td>\n",
" <td>Max Ferguson (painter)>New York University>Ber...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>John J. Dunnigan</th>\n",
" <td>/people/person/place_of_birth</td>\n",
" <td>New York City</td>\n",
" <td>5</td>\n",
" <td>John J. Dunnigan>New York (state)>Arabic>New Y...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Frederick Juengling</th>\n",
" <td>/people/person/place_of_birth</td>\n",
" <td>New York City</td>\n",
" <td>5</td>\n",
" <td>Frederick Juengling>Berlin>New York City</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Reuel Denney</th>\n",
" <td>/people/person/place_of_birth</td>\n",
" <td>New York City</td>\n",
" <td>5</td>\n",
" <td>Reuel Denney>Buffalo, New York>New York City</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" predicate object score \\\n",
"subject \n",
"Morris S. Miller /people/person/place_of_birth New York City 5 \n",
"Max Ferguson (painter) /people/person/place_of_birth New York City 5 \n",
"John J. Dunnigan /people/person/place_of_birth New York City 5 \n",
"Frederick Juengling /people/person/place_of_birth New York City 5 \n",
"Reuel Denney /people/person/place_of_birth New York City 5 \n",
"\n",
" path \n",
"subject \n",
"Morris S. Miller Morris S. Miller>Buffalo, New York>New York City \n",
"Max Ferguson (painter) Max Ferguson (painter)>New York University>Ber... \n",
"John J. Dunnigan John J. Dunnigan>New York (state)>Arabic>New Y... \n",
"Frederick Juengling Frederick Juengling>Berlin>New York City \n",
"Reuel Denney Reuel Denney>Buffalo, New York>New York City "
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"grec_paths = pd.read_csv(\"data/GREC/grec_final/gt_with_paths.csv\", encoding='utf-8', sep=';', index_col=0)\n",
"grec_paths[:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Get truth series**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.\n",
"#grec_m, grec_u, grec_u2 = get_grec_truth_series(grec_paths)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.\n",
"#WRITE \n",
"#grec.to_csv('data/GREC/pob1k/gt_with_truthvals.csv', encoding='utf-8', sep=';')\n",
"#READ\n",
"#grec_m = pd.read_csv('data/GREC/pob1k/gt_with_truthvals.csv', encoding='utf-8', sep=';', index_col=0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": false
},
"outputs": [],
"source": [
"# Uncomment these line(s) below if calculating truth values from scratch.\n",
"# Otherwise, load the CSVs at the bottom of this ntbk.\n",
"# Run only if truth_series was not yet calculated. Otherwise skip to 3rd cell.\n",
"#grec_m, grec_u, grec_mold = get_grec_truth_series(grec)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**See the truth scores**\n",
"\n",
"Column \"path\" is the predicted score given by the algorithm, and Column \"score\" is the aggregated score of human fact checkers, and can range from -5 to +5 (-5 with all raters having rated the statement 'false', and +5 with all raters having rated the statement 'true')"
]
},
{
"cell_type": "code",
"execution_count": 228,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>subject</th>\n",
" <th>predicate</th>\n",
" <th>object</th>\n",
" <th>score</th>\n",
" <th>path</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>Morris S. Miller</td>\n",
" <td>/people/person/place_of_birth</td>\n",
" <td>New York City</td>\n",
" <td>5.0</td>\n",
" <td>0.193146</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Max Ferguson (painter)</td>\n",
" <td>/people/person/place_of_birth</td>\n",
" <td>New York City</td>\n",
" <td>5.0</td>\n",
" <td>0.295168</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>John J. Dunnigan</td>\n",
" <td>/people/person/place_of_birth</td>\n",
" <td>New York City</td>\n",
" <td>5.0</td>\n",
" <td>0.184989</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" subject predicate object \\\n",
"0 Morris S. Miller /people/person/place_of_birth New York City \n",
"1 Max Ferguson (painter) /people/person/place_of_birth New York City \n",
"2 John J. Dunnigan /people/person/place_of_birth New York City \n",
"\n",
" score path \n",
"0 5.0 0.193146 \n",
"1 5.0 0.295168 \n",
"2 5.0 0.184989 "
]
},
"execution_count": 228,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"grec_m[\"path\"] = grec_m[\"path\"].astype('float64')\n",
"grec_m[:3]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Drop all nan values**\n",
"\n",
"These nan values were unretrievable paths (due to Wikidata entity no longer existing)"
]
},
{
"cell_type": "code",
"execution_count": 229,
"metadata": {},
"outputs": [],
"source": [
"grec_m = grec_m.dropna()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**See correlation Graph**\n",
"\n",
"The correlation of human fact checker scores vs. the computational fact checking algorithm."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAF3CAYAAABUsGfpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xm8lHXd//HXhyPIwQ1UtNhEzTARFT0qmvsGZKBZbnd2W9nt3WJ7lNxaLll6R7d3m5VL3m2mieIRM0Vz66emchCRMEkkFw4WKCAkJ9k+vz++18CcObNc55y5Zrvez8fjPJj5zjUznxnOmc9c3+XzNXdHRESkmD7VDkBERGqfkoWIiJSkZCEiIiUpWYiISElKFiIiUpKShYiIlKRkISIiJSlZiIhISUoWIiJSkpKFiIiUtFW1AyiXnXfe2UeOHFntMERE6sqcOXNed/fBpY5rmGQxcuRI2traqh2GiEhdMbOX4xynbigRESlJyUJEREpSshARkZKULEREpCQlCxERKUnJQkRESlKyEBGRkpQsRESkJCULEREpSclCRERKUrIQEZGSlCxERKQkJQsRESkp0WRhZhPMbKGZLTKzC/Pc/kkzm29mz5jZo2a2T9ZtU6P7LTSz8UnGKSIixSWWLMysCbgGmAjsA5ydnQwiv3H3Me5+APAd4OrovvsAZwGjgQnAj6PHExGRKkjyzOIQYJG7L3b3dcAtwCnZB7j76qyr2wAeXT4FuMXd33b3vwGLoscTEZEqSHLzo6HAq1nXlwCH5h5kZp8BvgT0A47Luu8TOfcdmkyYIiJSSpJnFpanzbs0uF/j7nsCXwMu7s59zex8M2szs7bly5f3KlgRESksyWSxBBiedX0YsLTI8bcAp3bnvu5+nbu3uHvL4MElt5AVEZEeSjJZzAb2MrPdzawfYcB6ZvYBZrZX1tWTgReiyzOBs8xsazPbHdgLeCrBWEVEpIjExizcfYOZXQDMApqAG919gZldDrS5+0zgAjM7AVgPrATOje67wMxuBZ4DNgCfcfeNScUqIlKPWue2M23WQpau6mDIwGamjB/FqWOTGd419y5DAXWppaXF29raqh2GiEhFtM5tZ+qM+XSs3/I9urlvE1eeNqZbCcPM5rh7S6njtIJbRKQOTZu1sFOiAOhYv5FpsxYm8nxKFiIidWjpqo5utfeWkoWISB0aMrC5W+29pWQhIlKHpowfRXPfzlWQmvs2MWX8qESeL8kV3CIikpDMIHalZkMpWYiI1KlTxw5NLDnkUjeUiIiUpGQhIiIlKVmIiEhJShYiIlKSkoWIiJSkZCEiIiUpWYiISElKFiIiUpKShYiIlKRkISIiJanch4g0hEruGpdGShYiUvdyd41rX9XB1BnzAZQwykTdUCJS9yq9a1waKVmISN2r9K5xaaRkISJ1r9K7xqWRkoWI1L1K7xqXRhrgFpG6V+ld49JIyUJEGkIld41LI3VDiYhISUoWIiJSkpKFiIiUpGQhIiIlaYBbRKROVbIelpKFiEgdqnQ9LHVDiYjUoUrXw9KZhYg0hLSVKK90PSydWYhI3ct0ybSv6sDZ0iXTOre92qElptL1sJQsRKTupbFEeaXrYakbSkTqXhpLlFe6HpaShYjUvSEDm2nPkxgavUR5JethqRtKROqeSpQnT2cWIlL3VKI8eUoWItIQVKI8WeqGEhGRknRmISJSp1QbSkREilJtKBERKanSCxGVLERE6lC+dSXF2ntLyUJEpA41mXWrvbeULERE6tBG926195aShYhIHRpaoJRJofbeSjRZmNkEM1toZovM7MI8t3/JzJ4zs2fN7AEz2y3rto1m9kz0MzPJOEVE6k3DVJ01sybgGuBEYAkw28xmuvtzWYfNBVrcfa2ZfQr4DnBmdFuHux+QVHwiIvWskarOHgIscvfFAGZ2C3AKsDlZuPtDWcc/AZyTYDwiIg2lUarODgVezbq+JGor5Dzgnqzr/c2szcyeMLNTkwhQRETiSfLMIt/8rbzD9GZ2DtACHJ3VPMLdl5rZHsCDZjbf3V/Mud/5wPkAI0aMKE/UIiLSRZJnFkuA4VnXhwFLcw8ysxOAi4DJ7v52pt3dl0b/LgYeBsbm3tfdr3P3FndvGTx4cHmjFxGRzZJMFrOBvcxsdzPrB5wFdJrVZGZjgWsJiWJZVvsgM9s6urwz8F6yxjpERKSyEuuGcvcNZnYBMAtoAm509wVmdjnQ5u4zgWnAtsB0C6sOX3H3ycB7gGvNbBMhoV2VM4tKREQqyDyh1X6V1tLS4m1tbdUOQ0SkrpjZHHdvKXWcVnCLiEhJShYiIlKSkoWIiJSkZCEiIiUpWYiISElKFiIiUpKShYiIlKRkISIiJSlZiIhISUlWnRURkQS1zm1viM2PREQkIa1z25k6Yz4d6zcC0L6qg6kz5gMkkjDUDSUiUoemzVq4OVFkdKzfyLRZCxN5PiULEZE6tHRVR7fae6tosjCzJjOblsgzi4hIjw0Z2Nyt9t4qmizcfSNwkEWbTYiISG2YMn4UzX2bOrU1921iyvhRiTxfnAHuucCdZjYdeCvT6O4zEolIRERKygxi19JsqB2BN4DjstocULIQEUmJksnC3T9WiUBERCS+mps6a2bvNrMHzOzP0fX9zOziskciIiKx1eLU2euBqcB6AHd/FjgrkWhERCSWmpo6Gxng7k/ltG1IIhgREYmnpqbORl43sz0Jg9qY2YeA1xKJRkREYjl278Hdau+tOLOhPgNcB+xtZu3A34BzEolGRERieej55d1q7604yaLd3U8ws22APu6+xsx2TCQaERGJpRbHLGaY2Vbu/laUKN4B3J9INCIiEkstjlm0ArdFdaJGAvcRZkeJiEiVTBk/ij45hZj6GImV+yiZLNz9esKZRCtwF/BJd78vkWhERCSWtpdXsMk7t23y0J6EgsnCzL6U+QH6A8OBZ4BxUZuIiFTJTU+80q323io2wL1dzvU7CrSLiEiFeTfbe6tgsnD3yxJ6ThERqTNxakPdb2YDs64PMrNZyYYlIiLFbNOvqVvtvRVnNtRgd1+VueLuK4FdEolGRERi+cCB+SvLFmrvrTjJYqOZjchcMbPdSK5bTEREYrj72fxVlwq191acFdwXAY+a2SPR9aOA8xOJRkREYlm5dn232nsrzuZH95rZgcA4wIAvuvvriUQjIiI1Kc4AtwETgAPd/S5ggJkdknhkIiJSM+KMWfwYOAw4O7q+BrgmsYhERKTmxBmzONTdDzSzuRBmQ5lZv4TjEhGRGhLnzGK9mTWxZfOjwcCmRKMSEZGimsy61d5bcZLFDwilPnYxs28BjwLfTiQaERGJZdweg7rV3ltxZkPdZGZzgOMJs6FOdfe/JBKNiIjE8tIb+Tc5KtTeW3HGLABeAFZnjjezEe6eTGlDEREpqdI75ZVMFmb2WeAS4B/ARsLZhQP7JRKRiIiUNGRgM+15EkM1d8r7PDDK3Ue7+37uPsbdlShERKpoyvhRNPftXDSwuW9T9XbKA14F3kzk2UVEpEdOHTuUDx40dPPspyYzPnjQUE4dm0whwYLdUFm74S0GHjazu4G3M7e7+9WJRCQiIiW1zm3n9jntbPRQ13WjO7fPaadltx0TSRjFziy2i35eIezB3S+rTbvliYhU0bRZC+lYv7FTW8f6jUybtTCR5ys2wP0dYDt3X5bdaGa7ErNbyswmAN8HmoAb3P2qnNu/BHwC2AAsBz7u7i9Ht50LXBwdeoW7/yLOc4qIpEG+we1i7b1V7Mzi+8ARedpPAP631ANHq76vASYC+wBnm9k+OYfNBVqiAfPbCAkKM9uRMAPrUOAQ4BIzS2aliYhIHaqlFdxHuPuM3EZ3v4mwp0UphwCL3H2xu68DbgFOyXmsh9x9bXT1CWBYdHk8cL+7r4h25rufUPlWRERg81hF3PbeKpYsiqWnOLOohhJmUmUsidoKOQ+4p4f3FRFJlaEF1lMUau+tYh/6y/LtW2FmBxPGF0rJl2zypjwzOwdoAaZ1575mdr6ZtZlZ2/LlcUISEWkMlV5nUWyAewpwq5n9HJgTtbUA/w6cFeOxlwDDs64PA5bmHmRmJxC2bj3a3d/Ouu8xOfd9OPe+7n4dcB1AS0uL9gUXkdTITI+dNmshS1d1MGRgM1PGj0psnUXBMwt3f4ow7mDAR6MfI+xv8WSMx54N7GVmu0f7X5wFzMw+wMzGAtcCk3NmXc0CTjKzQdHA9klRm4iIRKa3vUL7qg6cMAtqeltyJfuK1oaKPsAv6ckDu/sGM7uA8CHfBNzo7gvM7HKgzd1nErqdtgWmh91becXdJ7v7CjP7JiHhAFzu7it6EoeISCP68PV/4rEXO38sPvbiCj58/Z+46T8OK/vzxa062yPu/nvg9zlt38i6fEKR+94I3JhcdCIi9Ss3UZRq7604s5pERCTlSiYLMzs9TpuIiDSuOGcWU2O2iYhIgypWdXYi8D5gqJn9IOum7Qm1nEREJCWKDXAvBdqAyWxZZwGwBvhikkGJiEhtKZgs3H0eMM/M7gDecveNsLlA4NYVik9ERGpAnDGL+4DsYiPNwB+SCUdERGpRnGTR393/mbkSXR6QXEgiIlJr4iSLt8zswMwVMzsISGZ3DRERqUlxVnB/gVCOI1ME8J3AmcmFJCIitaZksnD32Wa2NzCKUEjweXdfn3hkImXSOre9YpU5RSrFyL/nQzL75MWvDTWKsDVqf2CsmeHuv0woJpGyaZ3bztQZ8zdvbN++qoOpM+YDKGFIXSu0J0NSezXEKfdxCfDD6OdYwj7ZkxOKR6Ssps1auDlRZHSs38i0WQurFJFIfYozwP0h4Hjg7+7+MWB/tM5C6sTSVfnnYhRqF5H84iSLDnffBGwws+2BZcAeyYYlUh5DCuxHXKhdRPKLkyzazGwgcD2h7MfTwFOJRiVSJpXep1ikUcWZDfXp6OJPzexeYHt3fzbZsETKo9L7FIs0qmJVZy9w9x9Fl0e7+wJ3f6likYmUyaljhyo5iPRSsW6oj2dd/lXSgYiISO2Ku61qUus8RESkBwp9KFdjUd5AM/sAIaFsb2anZd/o7jMSiklEREo4fM8deezFFXnbk1AsWTzClsV3fwQmZd3mgJKFiNSMtJV1eemN/GuFCrX3VrHNjz6WyDOKiJRZGsu6tBdYWFqovbfijlmIiNSsNJZ1sQKDE4Xae0vJQkTqXhrLuniBioGF2ntLyUJE6p7KuiSv2KK80wrdBpoNJSK1Y8r4UZ3GLEBlXcqt2GyoSUVu02woEakZKuuSPM2GEpGGoLIuyYq1U56ZnQyMJuyUB4C7X55UUCIiUltKJgsz+ykwgLBL3g2EzZBUolykhqVtgZokL85sqMPd/d+Ble5+GXAYMDzZsESkpzIL1NpXdeBsWaDWOre92qFJGQ0a0Ldb7b0Va6e86N+1ZjYEWA/snkg0ItJraVyglkaXTBpNn5wFeH0stCchzpjF76Kd8qYRdslzQneUiNSgNC5Qk+SVPLNw92+6+yp3vx3YDdjb3b+efGgi0hNaoJYOF90xn005q7U3eWhPQqwV3GZ2uJn9G3AmcIqZ/Xsi0YhIr2nf8XR4a93GbrX3VpzZUL8C9gSeATJROPDLRCISkV7RAjVJQpwxixZgH/ekylOJSLlpgZqUW5xuqD8D70g6EBERqV1xzix2Bp4zs6eAtzON7j658F1ERKSRxEkWlyYdhIiI1LY4U2cfAZ4Htot+/hK1iYhIlWzTr6lb7b1VMlmY2RmEWlCnA2cAT5rZhxKJRkREYqm5qbPARcDB7r4MwMwGA38AbkskIhERqTlxZkP1ySSKyBsx7yciIg0izpnFvWY2C7g5un4m8PvkQhIRkVpTMlm4+xQz+yDwXsCA69z9jsQjExGRmhGrO8ndb3f3L7n7F7uTKMxsgpktNLNFZnZhntuPMrOnzWxD7qC5mW00s2ein5lxn1NERMqv4JmFmT3q7keY2RpCLajNNwHu7tsXe2AzawKuAU4ElgCzzWymuz+XddgrwEeBr+R5iA53PyDeyxARkSQVTBbufkT073Y9fOxDgEXuvhjAzG4BTgE2Jwt3fym6bVMPn0NERCogzjqLX8Vpy2Mo8GrW9SVRW1z9zazNzJ4ws1O7cT8RESmzOLOhOu3RZ2ZbAQfFuJ/laetO5doR7r7UzPYAHjSz+e7+Yk4s5wPnA4wYMaIbDy3S2FrntqtEuZRVwTMLM5sajVfsZ2aro581wD+AO2M89hJgeNb1YcDSuIG5+9Lo38XAw8DYPMdc5+4t7t4yePDguA8t0tBa57YzdcZ82ld14ED7qg6mzphP69z2aocmdaxgsnD3K6Pximnuvn30s5277+TuU2M89mxgLzPb3cz6AWcBsWY1mdkgM9s6urwzYdruc8XvJSIQNj3qWN+55EPH+o1Mm7WwShFJI4izzmKqmQ0C9gL6Z7X/scT9NpjZBcAsoAm40d0XmNnlQJu7zzSzg4E7gEHAJDO7zN1HA+8Bro0GvvsAV+XMohKRApau6uhWu0gccbZV/QTweUI30jPAOOBPwHGl7uvuvydntbe7fyPr8uzocXPv9zgwptTji0hXQwY2054nMQwZ2FyFaKRRxFmU93ngYOBldz+WMHawPNGoRKTHpowfRXPfzmWqm/s2MWX8qCpFJI0gzmyof7n7v8wMM9va3Z83M/3WidSozKwnzYaScoqTLJaY2UCgFbjfzFbSjVlNIlJ5p44dquQgZRVngPsD0cVLzewhYAfgnkSjEhGRmtKtFdzu/oi7zwRuTDQqERGpKXEGuHNXcDcRbwW3iIg0iGJVZ6cC/wU0m9nqTDOwDriuArGJSA+p3IeUW5IruEWkClTuQ5IQZzbUPWZ2VG5jqRXcIlIdxcp96OxCeipOspiSdbk/YZ+KOcRYwS0iladyH5KEOFNnJ2VfN7PhwHcSi0hEekXlPiQJsfbgzrEE2LfcgYhIeajchyQhTiHBH7Jl06I+wAHAvCSDkmRppkxjU7kPSUKcMYu2rMsbgJvd/bGE4pGEZWbKZAZAMzNlAH2YNBCV+5ByK9kN5e6/AG4G5gLPEjY1kjqljXFEpCfidEO9D7gWeJGwKG93M/tPd1d9qDqUb+CzWHsjSGO3WxpfsyQrTjfU1cCx7r4IwMz2BO5GxQTrUpMZG93ztjeiNHa7pfE1S/LizIZalkkUkcXAsoTikYTlSxTF2utdGrvd0viaJXlxziwWmNnvgVsJs6JOB2ab2WkA7j4jwfikzAYN6MvKtevztjeiNC5QS+NrluTFSRb9gX8AR0fXlwM7ApMIyUPJoo4UOoFo0BMLdmjuy6qOrslxh+bGTI6Q3kV5GqdJVpwV3B+rRCBSGW/m+eAs1l7vCg3FNOgQDRAW5WWPWUDjL8rTOE3y4syG2h34LDAy+3h3n5xcWJKUtH3rXJWny61YeyNI46I8FU9MXpxuqFbgZ8BdwKZkw5Gkpe1bZ9qSY0baFuWlcUp4pcVJFv9y9x8kHolURNq+daYtOaZV2qaEV0OcZPF9M7sEuA94O9Po7k8nFpVImaQtOaZV2qaEV0OcZDEG+Ahh/4pMN5Sj/SzqUhoHAtPWJZNGQwt0Nw5txO7G9na48Ubu/dnP2Pv1lwEY+bXfJf60cZLFB4A93H1d0sFI8jQQKI3o2L0H8+snXsnbXrcWLoRbb4WHHoJ99oH3vhd++Uu4914A9q5wOHGSxTxgIFq13RC0YEsa0UPPL+9We0154QV47bUwn/uvf4Vf/QoeeaTzMbNnw047wRlnwIknQns7py17J3OHjMKtJ9sSdV+cZLEr8LyZzabzmIWmztahNC5Sk8ZXF1+C3noLli6FBx6AT32q6+3HHAMTJsC//hWuDxkSksMZZ8C4cV0WBz194d3Jx5wlTrK4JPEopGLSuEhNGl9NTZF+4w246Sa48UaYl7VP3F57wZe/3DVRDBkCV18dupmGDYOvfa2y8cYUZwX3I2a2K3Bw1PSUu6tLqk7lqwtVrF2kHlRlivQLL8D06eFnp52gXz/47Gfhfe/reuy++8J554WzhIkTYfjwuvuGFmcF9xnANOBhwn4WPzSzKe5+W8KxSQI0H10aUaJTpBctCmcIy5fDscfCpz8NDz7Y9bhDDoGRI+G734VttoExY+Cww6BPzpjCoEG9j6kK4nRDXQQcnDmbMLPBwB8AJYs6pPno0qh6PUV6xQro2xd+8xu44QZoa+t6zP33h7GHffYJx59xBpx+Ohx++Jak8J739DyGGhYnWfTJ6XZ6g3j7YEgNStV8dJF8Vq6ERx+FSy+Fp3PWFt9yC3z96+EsIuMd7wjjDOeeCyNGwAknVDTcWhEnWdxrZrMI+3ADnIl2yatbDTkfXSSfxYvDeMKtt4akcPzxcNZZ8OMfw9y5XY+fNg0OOAD+/GfYeeeu3UcpF2eAe0q00dERhDGL69z9jsQjk0TU9Xx0kXwWL4Y77wzjBLvsAnPmhISwYkXn41avDgngm98M17fbLsxAamqqfMx1qGCyMLN3Abu6+2PRbngzovajzGxPd3+xUkFK+ag6p9StlSth2bLQRTR9OvwgT33T978/zDaaNAkefxxOOimMKRxxhJJCLxU7s/ge8F952tdGt01KJCJJlGZDSc375z/D7KNvfQvuyenx3ndfOP/8MACdMXhwSAif+1xYy9CnT5ixJGVVLFmMdPdncxvdvc3MRiYWkSRKs6GkZrz8chhPmD49lLMYPDiMFVxxBXzwg12P/4//CO3HHAOf+UyXMYWLW+dz88/vZaM7TWacfehwrjh1TGVeSwoUSxb9i9ymqTN1SrOhpOJeegluuy10H739NnzkIzB5cih9kW358tCFdPDB8LvfhZlH++wTq/vo4tb5nSZubHTffF0JozyKJYvZZvYf7n59dqOZnQfMSTYsSYo2A0qH1rntld/D4403QqXUPfaA//5v+N73uh4zYECYkZSZfjpgQFircOSRsFXWx9Hw4d166puffLVgu5JFeRRLFl8A7jCzD7MlObQA/Qhly6UOaTOgxpf4niXu8NRT4dv/FVd0vX3GjNC9lLHjjiEhnHPOlhXN48b1Po4s6l5NXsFk4e7/AA43s2OBfaPmu909zzp3qSfaDKixlW3PktdfD/snTJ8OTzwR2oYOhalTw9TU557rfPzo0XDVVSEhLFlS0dpHZiGH5WuX8oizzuIh4KEKxCIiZdDtct2vvhrGFO65J8w2GjMmlLW4+eauxx53XJhx9MtfhhpHI0Z07j6qkuat+rB2/aa87VIe1f9fFpGyKlSu+wDWhLUH69bB3/8O//d/cN99nQ964IEwBfX974f99w+D0JMnw9FH10RSKKQjT6Io1i7dV7v/+yJlUpXB3ir62tEj+N7Nj9Pywhy+c2/OwrX/BsaOhY9/PHQzQThDOP308HPssXW5eE2beiVPySKF0vThmfhgbzWtXg233x4WqD3++Obmybvuyju/8A0O/mFOohg4MJTPPvbYMGPpggsqHHBytKlX8hJNFmY2Afg+0ATc4O5X5dx+FGE1+H7AWdl7ZJjZucDF0dUr3P0XScaaFg394ZlH2QZ7q6m9PYwpTJ8O/fuHMtpf+1r40M81fDh8+tPMGrY/Uz55A69sNxhr2qrhF6itKrB5V6F26b7EkoWZNQHXACcCSwjrNma6e/YUileAjwJfybnvjoTtXFsAB+ZE912ZVLxp0RAfnt1QV7Ww2ttDEbzXX4ejjgq7rt17b9fjDjwwrHS+8krYfvsw4HzMMSGJkLVAbYd3hONTsECtprZVbVBJnlkcAixy98UAZnYLcAqwOVm4+0vRbbmjUOOB+919RXT7/cAEtpRJlx6qi43ty6gma2GtWRP6R267DX796zConOuuu+DNN2G//UJZjNNPD2sVspIC++7b9X7Ab57sWoI+096oyUKLTZOXZLIYCmQvq1wCHNqL+zbe194qGDigb979tgcOaMyBwKou1lq9Oixeu/RSeOyxzrdde23YP2HRoi1t228P//mf4WfkyDAjqQc2FXhphdobgRabJi/JZJHvq1vcX9dY9zWz84HzAUaMGBE/shQr9BnZqAtdK1ILa+nSMNB8661hB7YJE8LPHXfAI490Pf7b3w4L1x59NHQn1eHso1qkxabJSjJZLAGyC7wMA5YWODbffY/Jue/DuQe5+3XAdQAtLS0N+nFXXm/mmV5YrL3elbV7YunSkAAGDIBttw0F8n7wg7BaOfc4s7DS+YtfhObmMBjdtzHP3iQdkkwWs4G9zGx3oB04C/i3mPedBXzbzAZF108CppY/xPRJ20Bgj7on1qwJH/hr1oQ9mf/nf7oec+SRYa3C0UeHUhhHHx3GFI47TklBGlJiycLdN5jZBYQP/ibgRndfYGaXA23uPtPMDgbuAAYBk8zsMncf7e4rzOybhIQDcHlmsFt6J40DgQW7Jzo6YP58uPpq+O1vO982fHjYfvNHP9rStu22YaD5858Pg8tNTfDRjyYae0+oDL0kIdF1Fu7+e+D3OW3fyLo8m9DFlO++NwI3JhkfpGuBGqR0IPDvf98ypvDHP4YVyzvtFBLBhAldjz/rrPAzcWKolFpnYwojd8qfLEbupGQhPZfqFdxpW6CW0bADgZmk8OqrsHEjfPjDcMop8ErOVNKVK0MiGDUqjEHstluomNqvX3XiLrMnFudfjlSoXSSOVCeLtC1Qy6j7s6k334S//jV8yH/3u2GQ+e23Ox+z9dahAN6RR4YzA7PQhXTCCeG2jJEjKxp6JWhvB0lCqpNF2haoQZ2dTbnDvHkwaxZceGHX23/+c2htDWcREGYdnX562LbzmGNCldQjj6xkxDWhJhciSt1LdbH3QgvRGnWBGhQ/m6qa1avhJz8J00vNws/IkWEjnU99qmuiGDYs7MY2fnzYxnP9+pBY1q6FX/winD3UcDntpJ19aP4tSQu1i8SR3r8o4F85H5ql2htBVc+mli0LH/J33hk22Nl99zAb6Sc/6XrsYYfBe94TBqAHDAjJI7v7SApq2W1HfvPEK2TX0OkTtTeyuu9erXGpThZp3DClIuU+li2Dv/0N3nor/Fx7Ldx9d+dj7r0XPvYxOPlkGDIkrGuYOBFOOklJoZemzVquaS+NAAATIElEQVRI7m/wpqi9UT8866p7tU6lOlmkUVnLfbz9NqxYEbbg/NznwsBztj33hK9/fctspH79wsK1M84IXUgNMvuo1qRxLC6tk1UqKdXJYlCBb9mDGnjMIt9uYsXagXB28LvfwfXXd66Quu22YYXzued2Pr5fv1Akb/z4MD0193ZJVNpW6UM6E2SlpTpZnLzfOzfX+c9tT6Xly8O6g+nTwwyjbbYJm+wcdVTXU49ttoGvfhUOPRT+8hd417tSPahcS9K4Sj+NCbLSUv3XffezrxVsb9S6/wCD1r7J/q/9lR07VvPUsNFc+odrOeHF2WF/5mz77gvbbRdKbA8cGNY1jB8fdmvLtvPOlQpdYkjjKv00JshKS3WyyNcFVay97qxdG84Q7rkn1D6aMYOXcg753KQpbLuugwW77MHoFa+E8YTTTw9JoTn6Vrb//pWOXKRb0pggKy3VyaJhrF0LzzwDl10G9923pd0MvvOdMK4wZ87m5vV9mvjtfifxfwdN5uVB72TmPkdzzrgRDX02lSZpnRnUsGVsaoSSRT15442wYvnWW0NSmDQJxo2DJ5+EmTO7Hn/ppaF09jnnhMJ5fftu2Z9ZGlZaZwZpnUWylCxq0YoVYaC5qSnsjbBmTaiB9OKLnY/785/DAPPnPhdKXGy9NRx/fFjEVsDNT75asF1nFo0hjTOD0no2VUlKFtXU0REqpK5bBzfdFMpb5BozBi66CA48MMxIOvjgMKYwcWLRpFCIisw1vjTODErr2VQlpTpZbNOvibfWdS3tsU2/Mu9fsG4dLFgAP/4x3HBD59u23z7UM/rudzu3n3EGfOEL0NISzi7OPLMsoRj5N0JXibnGkcaZQWk8m6q0VCeLTQW+TRdqL2nlyjCmMH16mIG0/fawyy6h3MXxx3c9fuLEUPJi/Piw8K0CK5oHFEiQA8qdIKVq0jgzSL/XyUt1suhxbaiVK0MxvBdeCF1DZ58Np50GixZ1Pm716lDraLfdwqD08OGhW2mbbcr0Crov3x9UsXapT2mbGbS2wO9voXbpvlQni5LWrg2b7Oy6ayh18dOfwms5C/m22iqscD7ssPDT0RG6kN73vs5JYc89Kxt7AWnc6+Di1vnc/OSrbHSnyYyzDx3e8IP5aZsZVKgvQCNx5ZPqZJFdG+pdr7/CuFfmc8X9Ubns7NXM3/9+6F5avXpL24c+FKakTpwYuo/y7eVcg9I2wJ07VXij++brjZow0jgzKI1fgrYy2JDnz3arhF5yqjc/umTSaJr6hHf26w/esCVRZGy7bVj5fNpp0NYG//xn6HZyD+MSp5xSd5VTBzYX2PCpQHu9+82T+deUFGpvBDW5wVXC0rjh06IrT+6SGLay0J6EVCcL2PIGXHXMRznx49ew35dvp/XpJSEhrFkTupSGDYM+jfFWrd+YfzymUHu921TghKlQeyNI48ygK04dwznjRmw+k2gyS0VVgu+ecQBDBzZjwNCBzXz3jAMSe67G+ATsoWmzFrI++tT4yy578MLg3Vi91dYN/Q1MA9yNr9B6ikZeZ5FGrXPbmXLbPNpXdeCE7sYpt82jdW57Is+X6mSRxm9gadPcN/+veKH2RnDs3oO71d4IMmNTmXGLzNjUxa3zqxxZci67awHrN3Y+RV6/0bnsrgWJPF/j/sXEoG9gje/AEQO71d4IHnp+ebfaG0GxMjaNqtJVs1OdLKaMH0Vz386Ldhp9pWvaPLF4ZbfaG0Eaz5jTNsuvGlKdLE4dO5QrTxvTaYDoytPGNOz0Qig8lbBRpxim8UMkjWfMafu9roZUr7OA9K10HbfHIB57cUXe9kaUxvn3aawNdfahw/OW3m/kqbOVlvpkkbaVrk/9rWuiKNZe79L4IZLG2lCZKbJpW6lfSalOFq1z2/nib5/ZXBKgfVUHX/ztM0DjrnQtVPaqVDmsepXWD5G0nTFD+L9u9P/XbNkVKHLbk5DqZPHV2+Z1qR3jUXva/tAaWdo+RCQdLpk0mi9Pn8fGrBWmTX2MSyaNTuT5Uj3AvW5j/kHOQu0iIrUk9wM8yQ/0VJ9ZSDqkbVxK0iG7AkXG+k2e2O6AShbS0FrntjNl+rzNf1TtqzqYMn0e0LjjUpIOlV5Pk+puqDQaWmCufaH2enfpzAV5v31dOjOZkggilVLp9TRKFimTtlXrqzrylz4o1N4oWue2896rHmT3C+/mvVc9mFhxOameSv8tqxsqZdI4Bz9t0rj5URpV+m9ZySKF0jQHv9Jz0WtBsc2P0vL/nhaV/FtWN5Q0tOzdEDOSnIteC9JYSFCSl+pkkca9DtKoknPRa0EaCwlK8hr976aojgI1Lgq1N4o0DX4Wm4veqNI2iUEqI9VjFmmsSJq2wc/2Al0vhdobgSYxSBJSnSzSuNdB2gY/0/iFANI1iUEqI9XdUGlboAbpG/xM4xcCkSSkOlmksW83bYOfafxCIJKEVCeLNG6rmrYEmbbXK5KUVI9ZQPr6dtM2+Jm21yuSFPME+27NbALwfaAJuMHdr8q5fWvgl8BBwBvAme7+kpmNBP4CZOY3PuHunyz2XC0tLd7W1lbeFyAi0uDMbI67t5Q6LrEzCzNrAq4BTgSWALPNbKa7P5d12HnASnd/l5mdBfw3cGZ024vufkBS8YmISHxJjlkcAixy98Xuvg64BTgl55hTgF9El28Djjdr8DmNIiJ1KMkxi6HAq1nXlwCHFjrG3TeY2ZvATtFtu5vZXGA1cLG7/78kgtQuaiIipSWZLPKdIeQOkBQ65jVghLu/YWYHAa1mNtrdV3e6s9n5wPkAI0aM6HaAaVvNLCLSU0l2Qy0BhmddHwYsLXSMmW0F7ACscPe33f0NAHefA7wIvDv3Cdz9OndvcfeWwYMHdzvAYquZRURkiySTxWxgLzPb3cz6AWcBM3OOmQmcG13+EPCgu7uZDY4GyDGzPYC9gMXlDjBtq5lFRHoqsW6oaAziAmAWYersje6+wMwuB9rcfSbwM+BXZrYIWEFIKABHAZeb2QZgI/BJd19R7hiHDGzOW1CuUVczi4j0VKLrLCqpJ+sscscsIKzubfRV3CIiGVVfZ1EPtLpXRCSeVCcLSF+5DxGRnkh1IUEREYlHyUJEREpSshARkZKULEREpCQlCxERKUnJQkRESlKyEBGRkpQsRESkJCULEREpSclCRERKUrIQEZGSlCxERKQkJQsRESlJyUJEREpSshARkZKULEREpCQlCxERKUnJQkRESlKyEBGRkpQsRESkJHP3asdQFma2HHi52nH0wM7A69UOosL0mtMhba+5Xl/vbu4+uNRBDZMs6pWZtbl7S7XjqCS95nRI22tu9NerbigRESlJyUJEREpSsqi+66odQBXoNadD2l5zQ79ejVmIiEhJOrMQEZGSlCxqiJl9xczczHaudixJM7NpZva8mT1rZneY2cBqx5QEM5tgZgvNbJGZXVjteJJmZsPN7CEz+4uZLTCzz1c7pkoxsyYzm2tmv6t2LElQsqgRZjYcOBF4pdqxVMj9wL7uvh/wV2BqleMpOzNrAq4BJgL7AGeb2T7VjSpxG4Avu/t7gHHAZ1LwmjM+D/yl2kEkRcmidvwv8FUgFYNI7n6fu2+Irj4BDKtmPAk5BFjk7ovdfR1wC3BKlWNKlLu/5u5PR5fXED48h1Y3quSZ2TDgZOCGaseSFCWLGmBmk4F2d59X7Viq5OPAPdUOIgFDgVezri8hBR+cGWY2EhgLPFndSCrie4Qve5uqHUhStqp2AGlhZn8A3pHnpouA/wJOqmxEySv2mt39zuiYiwhdFzdVMrYKsTxtqThzNLNtgduBL7j76mrHkyQzez+wzN3nmNkx1Y4nKUoWFeLuJ+RrN7MxwO7APDOD0B3ztJkd4u5/r2CIZVfoNWeY2bnA+4HjvTHncC8BhmddHwYsrVIsFWNmfQmJ4iZ3n1HteCrgvcBkM3sf0B/Y3sx+7e7nVDmustI6ixpjZi8BLe5ejwXJYjOzCcDVwNHuvrza8STBzLYiDN4fD7QDs4F/c/cFVQ0sQRa+8fwCWOHuX6h2PJUWnVl8xd3fX+1Yyk1jFlItPwK2A+43s2fM7KfVDqjcogH8C4BZhIHeWxs5UUTeC3wEOC76f30m+sYtdU5nFiIiUpLOLEREpCQlCxERKUnJQkRESlKyEBGRkpQsRESkJCULAcDMdjWz35jZYjObY2Z/MrMPVCGOl3Kr7prZk9EUzFfMbHnWlMyR3Xjc48xsXNb1X5vZqTHut42ZPWxmfaLro8zsbjN7MXqfHjSzI6LbPpEV3/Nm9rmsx7nCzNqzYn/GzLaLbhtnZo9G1WmfN7PrzKw5J44TzOzNqKrpX83skThTUnNfdzmY2Y5m9sms6+8ws7vL+RxSe5QsJLOQqhX4o7vv4e4HAWeRp7hftNCsotz9UHc/APgG8Ft3PyD6eSkntqYiD3McoQpqd30CmO7um8xsAPA74Mfuvmf0Pn0B2CPr+JuiWI8ELjWzd2bdNi0r9gPcfU10+2+BL7n7KEJ12geAbfPE8pC7j3X3dwNfBH5iZkeXiL+nr7uYHYHNySKqNLDCzA4t8/N0UY3fPwmULATCB8o6d9+8MM7dX3b3HwKY2UfNbLqZ3QXcZ8E0M/uzmc03szOj447JruVvZj8ys49Gl18ys8vM7OnoPntH7TuZ2X3RN+ZryV9PKS8z28rMVkXf2p8CDjGzJRbtjRF9Y/+Dme1J+NCfEn2jPzx6iGPN7PHobKrQWdSHgTujyx8hJNTN36Ld/Vl3/2XunaJV6YuBd+beluOzwM/c/anofpvc/belVrVHlV2/RVj0h5mdEp2BzY3ez13yve58x0X3P87M5kXHPW1m20TtF5rZUxb2HflG9PRXAaOiY6+K2lqj96oTMxsanTU9E/2+HB61nxw9zzwzuy9q29nMZkbP9biZ7Ru1X2Fm15rZ/cD/Rf/vV2fF9YkS77GUgZKFAIwGni5xzGHAue5+HHAacACwP3ACMC3nG3Qhr7v7gcBPgK9EbZcAj7r7WGAmMKKbse8APO3uh7j7n/Id4O4vEkpHZ77ZPx7dtAthxfGpwJW59zOz/sAwd18SNcV5nzL3HQk0AX/Oas58aD9jocgiwL7AnDiPmcfTwN7R5T8C46L3cQZhT4l8r7vLcZnYgPOjs6KjgH9F3VwjgEMJ/9+HRx/2FwILo8fMbOjURjibynUOcFf0uPsDz5rZOwi/Ax9w9/0JZ7EA3wSejPY4uRT4edbjjAUmuftHgPMJhfsOAQ4m7JnR3d8b6Sad0kkXZnYNcAThbOPgqPl+d18RXT4CuNndNwL/MLNHCH+0paqLZorKzSEkHAgfTKcBuPvdZraym+GuA+7o5n0yWqMChs+aWb7S4bsAK/K0A2BmM4E9gQXufkbU/GEzOxEYBXws2sciY5q7f6+HseYNIevyCODW6IN4a0JNqnwKHfcY8D0z+w1wu7v/08xOImzcNDc6Zlvg3cCyPI+7DBiSp302cG2UeFvdfV70/jzk7i8D5PxenRy13WdmP8+c4QB3uvu/ossnAe8xs0yS2QHYi/RsHFYVOrMQgAXAgZkr7v4ZQvG7wVnHvJV1uVBX0QY6/071z7n97ejfjXT+otKbmjMdORVrs2PIff5cb2ddzveaOnIeI/d9mgycR+jDz7jJ3UcDxwDfz3TzFLEAOKjEMYWMZcvObNcA/+vuY4BPU/i15z3O3a8A/pOQEGab2V6E9+SKrDGWd7n7zws8bn/C+9WJuz9IeC9eA24ysw9Hj5vv/zz3/yD7eu7v36ez4trd3R8oEJeUiZKFADwI9DezT2W1DShy/B+BMy3sOTyYcHbwFPAysI+ZbW1mOxASTil/JOrrNrOJwKCevIAsL7Hlw/eDWe1rCIULY4vGDfqbWb+o6VfAMWZ2ctZhed8nd38UuJkwJlHMD4HzzKwFwmQDMzs3el8LMrMDCPugXBM17QC0m5kB52Ydmvu68x5nZntG4y9XEs4kRhEKIJ6XNX4xzMJMtXzv5bvp3OWWedzdgL+7+3WEbqWxhLOY46LbMLNMss3+XTgBWOLub+U+ZhTXpy0a7LYwQ605z3FSRuqGEtzdLUwj/V8z+yqwnPBN7msF7nIHYQxjHuEb4lcze2+Y2a3As8ALbOm+KOYy4GYzexp4hN53JVwKXG9mfycksIw7gelmdhrwmW483gPA4cDD7r7WzCYB/2NmPwT+Qeh6+3aB+14FPJU1CDzFogH/yCR3f9XM/o1wFrIT4f18GLg1z+Mda2ZzCQnqH4Rv149kve47CHtoPMWWgfXc113ouK+Y2ZGEnd6eBe5z93UWJiI8EXILawgl1l8yszYzmw/cHY1bHAvkmz57PPAlM1sP/BM4x93/EX0xuTNKWksJ3V3fIAxgPxsd+7EC7+u1hO60Z6K4ltHg29XWAlWdFSnCzA4mfCgX+uBKvegD//8BJ7v7m9WOR5KhbiiRItx9NvCoRYvyJK9dgO8oUTQ2nVmIiEhJ+rYkIiIlKVmIiEhJShYiIlKSkoWIiJSkZCEiIiUpWYiISEn/HwcjFje1rMFZAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x432 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from numpy.polynomial.polynomial import polyfit\n",
"import matplotlib.pyplot as plt\n",
"x = grec_m['score']\n",
"y = grec_m['path']\n",
"\n",
"fig, ax = plt.subplots(1,1,figsize=(6,6))\n",
"bb, m = polyfit(x, y, 1)\n",
"\n",
"ax.scatter(x,y)\n",
"ax.plot(x, bb + m * x, 'r--')\n",
"ax.set_ylabel(\"Computational Fact Checker\")\n",
"ax.set_xlabel(\"Ground Truth (GREC Dataset) score\")\n",
"fig.savefig('figures/grec/pob_correlation.pdf', format='pdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Calculate Spearman and Kendall's $\\tau$ Correlation value between**"
]
},
{
"cell_type": "code",
"execution_count": 231,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Spearman Correlation\n",
"0.40276840498587074\n",
"6.349433264873441e-21\n",
"Tau Correlation\n",
"0.3171971308063854\n",
"2.0270123085891602e-19\n"
]
}
],
"source": [
"from scipy.stats import pearsonr, spearmanr, kendalltau\n",
"r, pval = spearmanr(grec_m[\"score\"], grec_m[\"path\"])\n",
"r_tau, pval_tau = kendalltau(grec_m[\"score\"], grec_m[\"path\"])\n",
"\n",
"print(\"Spearman Correlation\")\n",
"print (r)\n",
"print (pval)\n",
"print(\"Tau Correlation\")\n",
"print (r_tau)\n",
"print (pval_tau)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There is a correlation between the human fact checkers and the computational fact checking method, thus satisfying objective #3. For reference, here is the ROC curve, using the GREC dataset as ground truth."
]
},
{
"cell_type": "code",
"execution_count": 242,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAF3CAYAAABKeVdaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VNX5x/HPww6yuEBdiAgKKNAqasS1BcUFUUERARVcK0prLWittu5oq1UqatUKoqVaAXFHikVkcfuBQlUUoSKilQgiCiJlkSXP748ziSFkmSQzc2f5vl+vvJi5czP3mSGZb845955j7o6IiAhAragLEBGR9KFQEBGRYgoFEREpplAQEZFiCgURESmmUBARkWIKBRERKaZQEBGRYgoFEREpplAQEZFidaIuoKqaN2/urVu3jroMEZGM8u9///trd29R2X4ZFwqtW7dm3rx5UZchIpJRzOy/8eyXtO4jM3vUzL4yswXlPG5mdp+ZLTGz983skGTVIiIi8UnmmMJYoEcFj58MtIt9DQb+msRaREQkDkkLBXd/DVhdwS69gcc8mAPsbGZ7JqseERGpXJRjCi2BZSXuF8S2rajqE23ZsoWCggI2bdqUqNqkEg0aNCAvL4+6detGXYpIWho9GsaNS93x9lm/kLyDW/CH0ZWOJVcoylCwMraVueKPmQ0mdDHRqlWrHR4vKCigSZMmtG7dGrOynlYSyd355ptvKCgooE2bNlGXI7KDVH8gl+XVV8O/Xbum5njr6uzCCfNuB+6u0fNEGQoFwN4l7ucBy8va0d1HA6MB8vPzdwiOTZs2KRBSyMzYbbfdWLVqVdSlSIKkw4doIqX6A7ksXbvCOefA4MFJPtAnn8A++0CdPalpIEC0oTAJuNzMJgCHA2vdvcpdR0UUCKml9zszxPthnw4foomUsg/kqL33HnTvDoMGwT33JOQpkxYKZjYe6AY0N7MC4CagLoC7PwRMAXoCS4ANwIXJqiVVnnvuOfr06cOiRYs44IADAJg1axYjRoxg8uTJxftdcMEFnHrqqfTt25ctW7Zwww038Mwzz1C/fn0aNWrELbfcwsknn1yjWm6//XYeeeQRateuzX333cdJJ520wz7Tp0/n6quvprCwkMaNGzN27Fjatm1b/PjTTz/NWWedxdy5c8nPz69RPZJ48Xzgx/thnzMfotnknXfg+OOhSRO44oqEPW3SQsHdz67kcQd+mazjR2H8+PEcc8wxTJgwgZtvvjmu77nhhhtYsWIFCxYsoH79+qxcuZJXi36Tq2nhwoVMmDCBDz/8kOXLl3P88cezePFiateuvd1+Q4YM4YUXXqBDhw48+OCD3HbbbYwdOxaAdevWcd9993H44YfXqBZJnnHjwh+KnTuXv48+7LPUv/8NJ5wATZvCzJmQwLG9jLuiOV3973//480332TmzJn06tUrrlDYsGEDDz/8MJ9++in169cHYPfdd6dfv341quWFF15gwIAB1K9fnzZt2tC2bVvefvttjjzyyO32MzO+++47ANauXctee+1V/NgNN9zAb3/7W0aMGFGjWiS5OneGWbOirkJSavNmOOOMEAizZkGCp/3JulAYOjT89ZRInTtX3l33/PPP06NHD9q3b8+uu+7KO++8wyGHVHyR9pIlS2jVqhVNmzattIZhw4Yxc+bMHbYPGDCAa6+9drttX3zxBUcccUTx/by8PL744osdvnfMmDH07NmThg0b0rRpU+bMmQPAu+++y7Jlyzj11FMVCuVIh4HZyloJkqXq1YMJE2CvvRIeCJCFoRCV8ePHM3ToUCB8UI8fP55DDjmk3AHZqg7Ujhw5Mu59Q89c5ccbOXIkU6ZM4fDDD+euu+7iyiuvZPTo0QwbNqy4G0nKFk/XTbJ17hy6hiRHvP126DYaMgSOOipph8m6UEjQAHyVfPPNN8yYMYMFCxZgZmzbtg0z484772S33XZjzZo12+2/evVqmjdvTtu2bfn8889Zt24dTZo0qfAYVWkp5OXlsWzZD9cFFhQUbNc1BLBq1Srmz59fPGbQv39/evTowbp161iwYAHdunUD4Msvv6RXr15MmjQpJwab420BFAWCum4kJd56C048EVq0gPPOg512Stqhsi4UovD0009z3nnnMWrUqOJtXbt25Y033qBLly4sX76cRYsW0aFDB/773/8yf/58OnfuTKNGjbj44ou54oorGDVqFPXq1WPFihVMnz6dgQMHbneMqrQUevXqxTnnnMOVV17J8uXL+fjjj+nSpct2++yyyy6sXbuWxYsX0759e6ZNm0aHDh1o1qwZX3/9dfF+3bp1Y8SIEVkbCKVDIN6zdfRXuqTMnDlw0kkhEGbOTGoggEIhIcaPH7/DX+tnnnkm48aN46c//Sn/+Mc/uPDCC9m0aRN169ZlzJgxNGvWDIDbbruN66+/no4dO9KgQQN22mknhg8fXqN6OnXqRL9+/ejYsSN16tThgQceKD7zqGfPnowZM4a99tqLhx9+mDPPPJNatWqxyy678Oijj9bouJmodDeQztaRtDJ7dgiE3XcPgZCXl/RDWln9z+ksPz/fS6+nUPRXuKRWNrzvsV4ydQNJevrrX2HkyBAILVvW6KnM7N/uXmmTX8txioikm40bw79DhoSmbA0DoSoUCiIi6eT112G//cJYAkCjRik9vEJBRCRdvPYanHwyNGsWJrmLQNaEQqaNjWQ6vd8iCfbqq9CzJ+y9dxhD2DOaNceyIhQaNGjAN998ow+qFClaT6FBgwZRlyKSHT74IATCPvuEsx722COyUrLilNS8vDwKCgo0v38KFa28JiIJ0KFDmOl06NBw+mmEsiIU6tatqxXARCTzvPYatGsXuopuvz3qaoAs6T4SEck4r7wSLkxL4FoIiaBQEBFJtWnT4LTToH17ePDBqKvZjkJBRCSVpk4NgbD//jB9epjTKI1kxZiCSDzKmgE16umvJccUFsL114eB5Vdegd12i7qiHSgUJONUd4GbsmZA1WynklK1asHkyVCnTloGAigUJANVd4EbzYAqkZkyBR5/HB57LPJTTiujUJCMpAVuJGNMngxnngk/+Qls2BCmsEhjGmgWEUmWF1+EPn3gwAPDGEKaBwKopSBpToPDkrEmTYK+fcMP68svw847R11RXBQKkjbKCgANDkvG2n33sIrTU09lRAuhSFasvCbZoVu3slsBGhyWjLJkCbRtG3UVO4h35TW1FCQSFXULaQBZMtYzz8CAAfC3v8HAgVFXUy0aaJZIjBv3Q9dQEXULSUZ7+mno3x8OOwx69Yq6mmpTS0Ei07WrWgWSJZ56Cs4+G444Al56CZo0ibqialNLQUSkJj79NDRxjzwy4wMB1FIQEamZNm1g4kQ44QRo3DjqampMoSApUXpgWdcaSMabMCGcdnrssXDGGVFXkzDqPpKUKJqvqIgGlSWjPfEEnHsujBgRdSUJp5aCJJxON5Ws9o9/wPnnhzMlJk6MupqEUyhIjegqZMkpjz0GF1wQuoxefBEaNYq6ooRTKEiNlDWNtaaolqw1cyYcd1yY1ygLAwEUCpIA6haSrLdpEzRoAGPGwObN0LBh1BUljQaaRUQq8sgjYerrFSugdu2sDgRQKIiIlG/MGPj5z2HffWGXXaKuJiUUCiIiZRk9Gi65BHr0gOefD91HOUBjChI3LXgjOWPiRLj0UujZM8x8miOBAGopSBWUvgANdKqpZKnu3WHYMHj22ZwKBFBLQapIZxpJVps8OcxhtNtucPfdUVcTCbUUREQA7r8fTjsNRo6MupJIqaUg5So9hvDqq9tfpSySNe67D379a+jdG668MupqIqWWgpSr9BhC0ZXKIlnlnntCIJx+ehhgrlcv6ooipZaCVEhjCJLVvv4ahg+HPn3CVNh160ZdUeQUCiKSu5o3h9mzw8VpCgRA3UcikotGjIA//CHc3n9/BUIJCgURyS133glXXw3vvw+FhVFXk3YUCiKSO+64A665BgYMCKun1dJHYGkaUxBAU1hIDrjjDvjd7+Dss8NiOXX08VcWxaQAmsJCcsCPfgSDBikQKqF3Rorp9FPJSkuXhrOLLroILrwQzKKuKK2ppSAi2Wv4cOjUCebPD/cVCJVSKIhIdrr5ZrjpJujfH37846iryRhJ7T4ysx7AvUBtYIy731Hq8VbA34GdY/tc6+5TklmTaFBZspx7CIThw0N30cMPh2U0JS5JaymYWW3gAeBkoCNwtpl1LLXb9cBEdz8YGAA8mKx65AcaVJasNmlSCISLLgrLaSoQqiSZLYUuwBJ3XwpgZhOA3sDCEvs40DR2uxmwPIn1SAkaVJasddpp4Qyjc8/VdQjVkMx3rCWwrMT9gti2km4GBppZATAF+FUS6xGRbOUOd90Fn30WgmDQIAVCNSXzXStrmN9L3T8bGOvueUBP4HEz26EmMxtsZvPMbN6qVauSUKqIZCz3cFHab38Ljz4adTUZL5mhUADsXeJ+Hjt2D10MTARw99lAA6B56Sdy99Hunu/u+S1atEhSuSKScdzDtBV/+hMMGRIGmKVGkhkKc4F2ZtbGzOoRBpInldrnc6A7gJl1IISCmgIiUjn30Dq46y74xS/ggQfUZZQASXsH3X0rcDkwFVhEOMvoQzMbbma9YrtdBVxiZvOB8cAF7l66i0lEZEcbN4azJX75y7C+si5MS4ikXqcQu+ZgSqltN5a4vRA4Opk1iEiWcYctW6BRI5g5E3baSYGQQGpriUjmcIehQ6FXL9i8GRo3ViAkmEIhB4weDd26/fBV+sI1kYzgDr/+Ndx3H3TooNXSkkShkANKX8Gsq5cl47jDr34Ff/kLDBsGd9+tFkKSaOrsHKErmCWj/f734eyiq64KZxspEJJGoSAi6e/ss6FhQ7jhBgVCkikURCQ9FRbC5MlhLqMDDwxfknQaUxCR9FNYCJddBr17w/TpUVeTU9RSEJH0UlgIl14apr3+/e+he/eoK8opaimISPooLIRLLgmBcP31cNttGkNIMYWCiKSPOXNg7Fi48cawUI4CIeXUfSQi6eOoo+DddzWoHCG1FEQkWtu2hS6jf/4z3FcgREqhICLR2bYNLrggjCG8/37U1QjqPhKRqGzdCuefH+Zhue22sHqaRE6hICKpt20bnHcejB8Pf/yjAiGNKBREJPVq1YJddoE77gjLaUraUCiISOps3Qpffgl5eVotLU1poFlEUmPLljBn+5FHwtq1CoQ0pVAQkeQrCoSnngrrITRrFnVFUg51H4lIcm3ZEqa+fuaZsDjOsGFRVyQVUCiISHLdemsIhHvuCctpSlpTKIhIcv3mN9CpE/TvH3UlEgeNKYhI4m3eDLfcAhs2QNOmCoQMolAQkcT6/nvo2xduvhmmTo26GqkidR+JSOJ8/z2ceWaY3O7BB+GMM6KuSKpIoSAiibFpUwiEKVPgoYfC6mmScRQKIpIYy5fDO+/AqFEweHDU1Ug1KRSyzOjRYdLJkt57Dzp3jqYeyQGbN0PdurDvvvDRR2FgWTKWBpqzzLhxIQRK6tw5XEwqknAbN8Jpp/0wy6kCIeOppZCFOneGWbOirkKy3saN0Ls3vPIKDBgQdTWSIAoFEam6DRtCIEyfDo8+GlZPk6ygUBCRqnEPZxlNnw5jx4bFciRrKBREpGrM4KKLwkDVoEFRVyMJplAQkfisXw9z50K3bnDWWVFXI0mis49EpHLr18Mpp8DJJ8OKFVFXI0mkloKIVOx//wuB8MYb8MQTsOeeUVckSaRQEJHyrVsHPXvC7NnhIhjNdpr1FAoiUr7HHguBMH68xhFyhEJBRMr3i1/A0UdrnpQcooHmDDZ6dDgRpORX6SkuRKrsu++gT58wj5GZAiHHKBQymOY5koRbuxZOOglefBEWL466GomAuo8ynOY5koT59tsQCO+8A089FSa6k5yjUBCREAgnnhiank8/HeY1kpwUV/eRmdUzs7bJLkZEIlKnDjRrBs88o0DIcZW2FMzsFOBuoB7Qxsw6Aze5uxZfFcl0a9aEQGjSBF5+OQwsS06Lp6UwHDgc+BbA3d8D1GoQyXSrV8Pxx4czjdwVCALEN6awxd2/te1/YDxJ9Ug5tMymJNQ334RAWLQInntOgSDF4mkpLDKzfkAtM2tjZvcAc5Jcl5Si008lYUoGwvPPh0nuRGLiaSlcDtwIFALPAlOB3yWzKNmxZVDUKtDpp1Jj550XAuGFF8IpqCIlxBMKJ7n7NcA1RRvMrA8hICRJiloGRd1DahVIwowcCZ9/HloLIqXEEwrXs2MAXFfGNkkwtQwkYVatCktn/uY30L59+BIpQ7mhYGYnAT2AlmZ2d4mHmhK6kkQkE3z1FRx3HCxdCr16wf77R12RpLGKWgpfAQuATcCHJbavA65NZlEikiArV4ZA+PRTmDxZgSCVKjcU3P1d4F0ze8LdN6WwJhFJhJKB8M9/wrHHRl2RZIB4xhRamtkfgI5Ag6KN7q5OSZF0Nn8+LF8OU6aEedVF4hDPdQpjgb8BBpwMTAQmxPPkZtbDzD4ysyVmVmaXk5n1M7OFZvahmY0rax8RqYItW8K/J54YWgkKBKmCeEKhkbtPBXD3T9z9eqDSdqiZ1QYeIARJR+BsM+tYap92hGsejnb3TsDQKtYvIiUtXw4HHxymvgbYeedo65GME0/30fcW5rj4xMwuA74AfhTH93UBlrj7UgAzmwD0BhaW2OcS4AF3XwPg7l9VpXgRKWH58jBusHw57Lln1NVIhoqnpTAMaAxcARxN+CC/KI7vawksK3G/ILatpPZAezN708zmmFmPOJ5XREr74ovQTbR8OfzrX3DMMVFXJBmq0paCu78Vu7kOGARgZnlxPHdZM2yVnkivDtAO6AbkAa+b2Y/d/dvtnshsMDAYoFWrVnEcWiSHfPttCISVK2HqVDjqqKgrkgxWYUvBzA4zs9PNrHnsficze4z4JsQrAPYucT8PWF7GPi+4+xZ3/xT4iBAS23H30e6e7+75LVq0iOPQIjmkWTMYOFCBIAlRbiiY2e3AE8C5wL/M7DpgJjCf0O1TmblAu9jMqvWAAcCkUvs8T2zQOhY87YGlVX0RIjnp88/hww/DtNc33QRHHhl1RZIFKuo+6g0c5O4bzWxXwl/5B7n7R/E8sbtvNbPLCbOq1gYedfcPzWw4MM/dJ8UeO9HMFgLbgKvd/ZuavCCRnPDf/4ZB5Tp1YOHC8K9IAlT0k7TJ3TcCuPtqM/tPvIFQxN2nAFNKbbuxxG0Hrox9iUg8PvssBMKaNTBtmgJBEqqin6Z9zaxoJlQDWpe4j7v3SWplIrKjzz4Lg8pr18Irr0B+ftQVSZapKBTOLHX//mQWIiJxuPlm+O67EAiHHhp1NZKFKpoQb3oqCxGRODz4IFx9NXTqFHUlkqXiuXhNRKL0ySfQt2/oMmrUSIEgSaURKpF0tmRJGFTesAEKCsI1CSJJFHdLwczqJ7MQESnl44/DoPLGjTBjhloIkhKVhoKZdTGzD4CPY/cPMrO/JL0ykVy2eHEIhO+/D4Fw0EFRVyQ5Ip6Wwn3AqcA3AO4+nzimzhaRGqhTJ8x0OmMGHHhg1NVIDolnTKGWu/83zJ5dbFuS6hHJbcuXwx57wL77wty5YQoLkRSKp6WwzMy6AG5mtc1sKLA4yXWJ5J5Fi8K1B9ddF+4rECQC8YTCEMI0FK2AlcARsW0ikigLF4azjNxh0KCoq5EcFk/30VZ3H5D0SkRyVVEg1KoFM2fCAQdEXZHksHhaCnPNbIqZnW9mTZJekUgu+f576NFDgSBpo9JQcPf9gNuAQ4EPzOx5M1PLQSQR6teHhx+GWbMUCJIW4rp4zd3/z92vAA4BviMsviMi1fXBBzB+fLh90kmw//7R1iMSU+mYgpk1Jiy4MwDoALwAaM0/keqaPx+6d4eddoLevcN8RiJpIp6B5gXAi8Cd7v56kusRyW5FgdCwIUyfrkCQtBNPKOzr7oVJr0Qk27333g8thJkzYb/9oq5IZAflhoKZ/dndrwKeMTMv/bhWXhOpoqlToXHjEAj77ht1NSJlqqil8GTsX624JlITW7eGuYyuuQYGD4Zddom6IpFylXv2kbu/HbvZwd2nl/wiDDiLSGXmzYMOHcJYAigQJO3Fc0rqRWVsuzjRhYhknblz4YQTQkth552jrkYkLhWNKfQnnIbaxsyeLfFQE+DbZBcmktHefhtOPBF23TWMIeyzT9QVicSlojGFtwlrKOQBD5TYvg54N5lFiWS0BQtCC6F58xAIrVpFXZFI3MoNBXf/FPgUeCV15YhkgbZtYcAAuP562HvvqKsRqZKKuo9edfeuZrYGKHlKqgHu7rsmvTqRTDJvXjjVdNddYdSoqKsRqZaKBpqLltxsDrQo8VV0X0SKvPlmmP56iJYakcxW0SmpRVcx7w3UdvdtwJHApcBOKahNJDO88UaY/nrPPeHuu6OuRqRG4jkl9XnCUpz7AY8RrlEYl9SqRDLF66+HQNhrrzD9dcuWUVckUiPxhEKhu28B+gD3uPuvAP3kixQWwi9/CXl5IRD22ivqikRqLK7lOM3sLGAQcHpsW93klZR7Ro+GcaXaXu+9B507R1OPxKlWLXjxRahXL3QdiWSBeK9oPpYwdfZSM2sDjE9uWbll3LgQAiV17gznnBNNPVKJmTNDC6GwMFyUpkCQLFJpS8HdF5jZFUBbMzsAWOLuf0h+abmlc+fQAyFpbsYMOPVUaNMG1q7VXEaSdeJZee2nwOPAF4RrFPYws0Hu/mayixNJK9Onw2mnhWsRZsxQIEhWimdMYSTQ090XAphZB0JI5CezMJG08sorIRDatQvh0EKX6kh2imdMoV5RIAC4+yKgXvJKEklTBx2kQJCsF09L4R0zG0VoHQCciybEqzadaZRhVqwIA8nHHx+W0jSLuiKRpIqnpXAZ8AnwW+AaYCnhqmapBp1plEGmTg3rKD/zTLivQJAcUGFLwcx+AuwHPOfud6ampOynM40ywEsvwRlnQMeO0K1b1NWIpEy5LQUz+z1hiotzgWlmVtYKbCLZZ8oUOP106NQpDDDvtlvUFYmkTEUthXOBA919vZm1AKYAj6amLJGIfPJJaCH85CcwbZpOO5WcU9GYwvfuvh7A3VdVsq9IdthvP3joIQWC5KyKWgr7llib2YD9Sq7V7O59klqZSCq9+GI4yyg/Hy68MOpqRCJTUSicWer+/cksRCQyzz8P/frBccfBv/4VdTUikapojebpqSxEJBLPPRcC4dBD4ckno65GJHIaJ5Dc9eyzIRAOOwxefhmaNYu6IpHIKRQkN7mHKwm7dAldRk2bRl2RSFqIZ5oLAMysvrt/n8xiso2mtEhT27ZB7drwxBOweTM0aRJ1RSJpo9KWgpl1MbMPgI9j9w8ys78kvbIsoCkt0tCTT4bWwTffQP36CgSRUuJpKdwHnEq4uhl3n29mxya1qiyiKS3SyIQJMHAgHHVUCAQR2UE8Ywq13P2/pbZtS0YxIkkzbhycey4cfXSYxqJx46grEklL8bQUlplZF8DNrDbwK2BxcssSSaDnnoNBg+BnP4PJk2GnnaKuSCRtxdNSGAJcCbQCVgJHxLaJZIYuXeCCCxQIInGotKXg7l8BA1JQi0hivfZa6C5q2RIeeSTqakQyQqWhYGYPA156u7sPTkpFIonw97+HOYz+9Ce4+uqoqxHJGPF0H70CTI99vQn8CIjregUz62FmH5nZEjO7toL9+pqZm1l+PM8rUqG//S0EwvHHw+WXR12NSEaJp/touwlhzOxxYFpl3xcblH4AOAEoAOaa2SR3X1hqvybAFcBbVahbpGyPPgo//zmccEKY6K5hw6grEsko1Znmog2wTxz7dQGWuPtSd98MTAB6l7HfrcCdwKZq1CLyg1Wr4Ne/hpNOghdeUCCIVEM8Ywpr+GFMoRawGii3K6iElsCyEvcLgMNLPffBwN7uPtnMfhNXxWms9LQWmtIixVq0gFdfDesqN2gQdTUiGanCUDAzAw4CvohtKnT3HQady/v2MrYVf6+Z1QJGAhdU+kRmg4HBAK1atYrz8KlXNK1FURBoSosUGT0atm6FX/wCDjkk6mpEMlqFoeDubmbPufuh1XjuAmDvEvfzgOUl7jcBfgzMCtnDHsAkM+vl7vNK1TEaGA2Qn58fbyhFQtNapNhDD8GQIXDaaXDZZVBLE/+K1EQ8v0Fvm1l1/vyaC7QzszZmVo9wrcOkogfdfa27N3f31u7eGpgD7BAIIuV68MEfAuGppxQIIglQbkvBzOq4+1bgGOASM/sEWE/oFnJ3rzAo3H2rmV0OTAVqA4+6+4dmNhyY5+6TKvp+kQo98EA43bR3b5g4EerVi7oikaxQUffR28AhwOnVfXJ3nwJMKbXtxnL27Vbd40gO2rwZTj89TIWtQBBJmIpCwQDc/ZMU1SJSuZUrYffdYdiwcPqpuoxEEqqiUGhhZleW96C7352EekTKN3Ik3HwzzJ4dTjtVIIgkXEWhUBtoTNmnloqk1t13w1VXQd++0K5d1NWIZK2KQmGFuw9PWSUZRusvp9CIEWFSu7POCusq160bdUUiWaui9rdaCBXQ+ssp8sILIRD69QtvugJBJKkqail0T1kVGUoXqqVAz55w773hauU68SwUKCI1UW5Lwd1Xp7IQke2MHh3ONKpbF664QoEgkiI6fUPSzx/+AJdeGloIIpJSCgVJL7feCtdfDwMHhtsiklIKBUkfw4fDjTfCoEEwdizUrh11RSI5R6Eg6WH9ehg/Hs4/PyynqUAQiYRG7yRa7lBYCDvtBG+8ATvvrEAQiZBaChId99BddPbZYZGc3XZTIIhETKEg0XCHG26A226Dpk01j5FImlD3kaSeO1x3Hdx+O/z85zBqlEJBJE3oN1FSb/jwEAiDBysQRNKMWgqSeiecAN99B3fdpUAQSTP6jZTUcIfXXw+3jzoK/vxnBYJIGtJvpSSfe5jp9Gc/g5kzo65GRCqg7iNJLvewOM7IkfCrX0G3blFXJCIVUEtBksc9rKU8cmRYT/nee8G0TIdIOlNLoZSyVlQri1ZZi8Mbb4QgGDo0LKepQBBJewqFUopWVKvsA1+rrMXhpz8Ng8tHH61AEMkQCoUyaEWZmzg/AAAQ3klEQVS1GigaVO7VKwwsH3NM1BWJSBUoFCRxCgvh8svhr3+Fhg1DKIhIRlEoSGIUFoZ1lEeNgmuuCVcti0jGUShIzRUWwmWXwcMPw7XXwh//qDEEkQyV86FQ+mwjnVVUDe5hkZzf/z7MeqpAEMlYOR8Kpc820llFVVBYCGvWhHUQHnssTFuhQBDJaDkVCmVdg1AUCDrbqIq2bQvTXr/5JsybF9ZEEJGMl1NXNBe1CkpSy6Aatm2Diy+GsWPDm6dAEMkaOdVSALUKamzbNrjwQnj8cbj5ZrjppqgrEpEEyrlQkBoaPjwEwi23hPWVRSSrZG0oVDR+IDVw+eXQsmVYNU1Esk7Wjilo/CCBtm6Fe+6BzZuhRQsFgkgWy9qWAmj8ICG2boVBg2DCBNh7bzjzzKgrEpEkyupQkBrauhXOPRcmToQ//UmBIJIDFApSti1bQiA89RTceWeY+VREsp5CQcq2dCm8/DKMGBGW0xSRnKBQkO0VFobpKvbfHxYvhh/9KOqKRCSFsvbsI6mGzZuhb98wfgAKBJEcpFCQYPNm6N8fnnsuLJAjIjlJ3UcSAuGss2DSJPjLX8IFaiKSkxQKuc4d+vULgXD//fDLX0ZdkYhESKGQ68zglFPgpJNgyJCoqxGRiCkUctWmTbBgAeTnwyWXRF2NiKQJDTTnok2boE8f6NoVvvwy6mpEJI2opZBrNm2C00+HqVPh4Ydhjz2irkhE0ohCIZds3BgCYdo0GDMmrJ4mIlKCQiGXPPTQD4Fw0UVRVyMiaUihkEuuuAIOPRR+9rOoKxGRNKWB5my3YUNoFXz+OdSurUAQkQopFLLZ+vVw6qkwdiy8/XbU1YhIBlD3UbYqCoTXXoPHHgsT3YmIVEKhkI3Wrw9XKb/+Ojz+uBamFpG4JbX7yMx6mNlHZrbEzK4t4/ErzWyhmb1vZtPNbJ9k1pMzvv8+jCX84x8KBBGpkqS1FMysNvAAcAJQAMw1s0nuvrDEbu8C+e6+wcyGAHcC/ZNVU9b73/+gbl3YdVeYPTsMLIuIVEEyWwpdgCXuvtTdNwMTgN4ld3D3me6+IXZ3DpCXxHqy27p10KNHWBPBXYEgItWSzFBoCSwrcb8gtq08FwMvJbGe7PXddyEQ5syBgQPDzKciItWQzIHmsj6ZvMwdzQYC+UDXch4fDAwGaNWqVaLqyw5FgTB3Ljz5JJx5ZtQViUgGS2ZLoQDYu8T9PGB56Z3M7HjgOqCXu39f1hO5+2h3z3f3/BYtWiSl2Ix1zjkKBBFJmGSGwlygnZm1MbN6wABgUskdzOxgYBQhEL5KYi3Z69Zb4emnw1TYIiI1lLRQcPetwOXAVGARMNHdPzSz4WbWK7bbXUBj4Ckze8/MJpXzdFLSt9/CI4+E2wcfDL17V7y/iEicknrxmrtPAaaU2nZjidvHJ/P4Wenbb+HEE+G99+CnP4X27aOuSESyiOY+yiRr1sAJJ8D8+fDsswoEEUk4TXORKVavDoGwYEEIhFNOiboiEclCCoVM8frrsGgRPP88nHxy1NWISJZSKKS7wkKoVSsMJi9dqjWVRSSpNKaQzr7+Go46Cv71r3BfgSAiSaaWQrr6+mvo3h0WL9Y8RiKSMgqFdLRqVQiEjz+GF1+E43XmroikhkIh3axdC8cdB598ApMnh3AQEUkRjSmkmyZNQigoEEQkAmoppIuVK8NqaW3awL33Rl2NiOQohUI6+PLL0Dowg/ff18CyiERGoRC1FStCICxbBlOmKBBEJFIKhSitWAHHHgsFBfDSS2GCOxGRCCkUonT11fDFF+HitGOOiboaERGdfRSp+++HmTMVCCKSNhQKqfbFFzB4MGzcCDvvDPn5UVckIlJMoZBKy5ZB165hPeWPP466GhGRHSgUUuXzz6FbtzCFxcsvw4EHRl2RiMgONNCcCkWBsHo1TJsGXbpEXZGISJkUCqmwbh3UqRMC4bDDoq5GRKRcCoVkWr0adtkFOnWChQtDMIiIpDGNKSTLp5/CIYfAH/8Y7isQRCQDKBSSYenSMIawbh307Bl1NSIicdOfr4n2ySdh6or162H6dOjcOeqKRETiplBIpE2bwippGzbAjBlw0EFRVyQiUiUKhURq0ADuvBPat1cgiEhGUigkwscfh6+ePeGss6KuRkSk2hQKNbV4cRhUrlUrBEPDhlFXJCJSbQqFmvjoozCovG1buDBNgSAiGU6hUF3/+U8IhMLCMP11x45RVyQiUmO6TqG6xo0DdwWCiGQVhUJVuYd/b7kF3nlHgSAiWUWhUBUffgiHHhoGlM1gr72irkhEJKGyZkxh9OjQo1PkvfcSfDHxggVw3HFhDqPCwgQ+sYhI+sialsK4cSEIinTuDOeck6An/+CDMKhcty7MmgX775+gJxYRSS9Z01KAEASzZiX4SRctCi2E+vXDoHK7dgk+gIhI+sialkLS5OWFVsKsWQoEEcl6WdVSSKhFi6BVK2jSBCZOjLoaEZGUUEuhLO++C8ccA5deGnUlIiIppVAo7Z13oHt3aNwYhg+PuhoRkZRSKJT073+HQGjaNIwh7Ltv1BWJiKSUQqHItm0waBDsvHMIhDZtoq5IRCTlNNBcpHZtePbZMNPpPvtEXY2ISCTUUnjrLbjhhjCn0QEHKBBEJKfldijMmQMnnhguh16zJupqREQil7uhMHt2CIQWLeDVV2HXXaOuSEQkcrkZCv/3fyEQdt89DCrn5UVdkYhIWsjNUFi5MlytrEAQEdlOboXCt9+Gf884I0yp2rJltPWIiKSZ3AmF114L1x7885/hft260dYjIpKGciMUXn0VTj4Z9tgjrJwmIiJlyv5QmDULevYM1x/MnBmCQUREypTdobBkSQiE1q0VCCIiccjuUNhvP7j1VpgxI5x+KiIiFcrOUJgxIyySYwZXXaVAEBGJU1JDwcx6mNlHZrbEzK4t4/H6ZvZk7PG3zKx1jQ86bRqccgoMHVrjpxIRyTVJCwUzqw08AJwMdATONrOOpXa7GFjj7m2BkcCfanLMw1ZPhdNOg/bt4YknavJUIiI5KZkthS7AEndf6u6bgQlA71L79Ab+Hrv9NNDdzKxaB1v9L/6woDd06BC6j5o3r3bhIiK5Kpmh0BJYVuJ+QWxbmfu4+1ZgLbBblY/kzoXf3cvK5h3hlVdgt6o/hYiIJHeRnbL+4vdq7IOZDQYGx+7+z8w+KueYzWne/Ov4S8w6zQG9/tyW6++BXn/5rz+uxWKSGQoFwN4l7ucBy8vZp8DM6gDNgNWln8jdRwOjKzugmc1z9/xqV5zh9Ppz+/WD3gO9/pq//mR2H80F2plZGzOrBwwAJpXaZxJwfux2X2CGu+/QUhARkdRIWkvB3bea2eXAVKA28Ki7f2hmw4F57j4JeAR43MyWEFoIA5JVj4iIVC6Z3Ue4+xRgSqltN5a4vQk4K4GHrLSLKcvp9Uuuvwd6/TVk6q0REZEi2TnNhYiIVEvGhUIkU2ekmTjegyvNbKGZvW9m080srlPRMkVlr7/Efn3NzM0sq85Gief1m1m/2M/Ah2Y2LtU1JlscvwOtzGymmb0b+z3oGUWdyWJmj5rZV2a2oJzHzczui70/75vZIXE/ubtnzBdhwPoTYF+gHjAf6Fhqn18AD8VuDwCejLruCN6DY4FGsdtDsuk9iOf1x/ZrArwGzAHyo647xf//7YB3gV1i938Udd0RvAejgSGx2x2Bz6KuO8Hvwc+AQ4AF5TzeE3iJcC3YEcBb8T53prUUUjp1Rpqq9D1w95nuviF2dw7hGpFsEc/PAMCtwJ3AplQWlwLxvP5LgAfcfQ2Au3+V4hqTLZ73wIGmsdvN2PEaqYzm7q9RxjVdJfQGHvNgDrCzme0Zz3NnWiikbuqM9BXPe1DSxYS/GLJFpa/fzA4G9nb3yaksLEXi+f9vD7Q3szfNbI6Z9UhZdakRz3twMzDQzAoIZ0D+KjWlpY2qfk4US+opqUmQsKkzMljcr8/MBgL5QNekVpRaFb5+M6tFmHH3glQVlGLx/P/XIXQhdSO0El83sx+7+7dJri1V4nkPzgbGuvufzexIwvVQP3b3wuSXlxaq/TmYaS2FqkydQUVTZ2SweN4DzOx44Dqgl7t/n6LaUqGy198E+DEwy8w+I/SnTsqiweZ4fwdecPct7v4p8BEhJLJFPO/BxcBEAHefDTQgzAuUK+L6nChLpoWCps6I4z2IdZ+MIgRCtvUnV/j63X2tuzd399bu3powptLL3edFU27CxfM78DzhZAPMrDmhO2lpSqtMrnjeg8+B7gBm1oEQCqtSWmW0JgHnxc5COgJY6+4r4vnGjOo+ck2dEe97cBfQGHgqNsb+ubv3iqzoBIrz9WetOF//VOBEM1sIbAOudvdvoqs6seJ8D64CHjazYYRukwuy6Y9DMxtP6B5sHhs3uQmoC+DuDxHGUXoCS4ANwIVxP3cWvU8iIlJDmdZ9JCIiSaRQEBGRYgoFEREpplAQEZFiCgURESmmUJC0Y2bbzOy9El+tK9i3dXkzRVbxmLNis27Oj00PsX81nuMyMzsvdvsCM9urxGNjzKxjguuca2ad4/ieoWbWqKbHltygUJB0tNHdO5f4+ixFxz3X3Q8iTKh4V1W/2d0fcvfHYncvAPYq8djP3X1hQqr8oc4Hia/OoYBCQeKiUJCMEGsRvG5m78S+jipjn05m9nasdfG+mbWLbR9YYvsoM6tdyeFeA9rGvrd7bE7+D2Jz2NePbb/DflizYkRs281m9hsz60uYc+qJ2DEbxv7CzzezIWZ2Z4maLzCzv1SzztmUmOTMzP5qZvMsrKFwS2zbFYRwmmlmM2PbTjSz2bH38Skza1zJcSSHKBQkHTUs0XX0XGzbV8AJ7n4I0B+4r4zvuwy41907Ez6UC2JTHPQHjo5t3wacW8nxTwM+MLMGwFigv7v/hDADwBAz2xU4A+jk7gcCt5X8Znd/GphH+Iu+s7tvLPHw00CfEvf7A09Ws84ehCktilzn7vnAgUBXMzvQ3e8jzHlzrLsfG5v24nrg+Nh7OQ+4spLjSA7JqGkuJGdsjH0wllQXuD/Wh76NMJ9PabOB68wsD3jW3T82s+7AocDc2JQfDQkBU5YnzGwj8BlhquX9gU/dfXHs8b8DvwTuJ6zTMMbM/gnEPUW3u68ys6Wx+Wg+jh3jzdjzVqXOnQhTPJRcUaufmQ0m/F7vSVhc5v1S33tEbPubsePUI7xvIoBCQTLHMGAlcBChhbvD4jnuPs7M3gJOAaaa2c8JUwj/3d1/F8cxzi05cZ6ZlbkOR2zunS6ECdcGAJcDx1XhtTwJ9AP+Azzn7m7hEzruOgmrjd0BPAD0MbM2wG+Aw9x9jZmNJUwCV5oB09z97CrUKzlE3UeSKZoBK2Lz4Q8i/JW8HTPbF1ga6zKZROhGmQ70NbMfxfbZ1eJfs/o/QGszaxu7Pwh4NdYH38zdpxAGccs6A2gdYRrvsjwLnE6Y8//J2LYq1enuWwjdQEfEup6aAuuBtWa2O3ByObXMAY4uek1m1sjMymp1SY5SKEimeBA438zmELqO1pexT39ggZm9BxxAWI5wIeHD82Uzex+YRuhaqZS7byLMLvmUmX0AFAIPET5gJ8ee71VCK6a0scBDRQPNpZ53DbAQ2Mfd345tq3KdsbGKPwO/cff5hHWZPwQeJXRJFRkNvGRmM919FeHMqPGx48whvFcigGZJFRGREtRSEBGRYgoFEREpplAQEZFiCgURESmmUBARkWIKBRERKaZQEBGRYgoFEREp9v8Ymb975B8vwgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x432 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from sklearn.metrics import roc_curve\n",
"from sklearn.metrics import roc_auc_score\n",
"\n",
"z = lambda x: x>0\n",
"fig, ax = plt.subplots(figsize=(6,6))\n",
"y = [z(x) for x in grec_m['score']]\n",
"fpr, tpr, thresh = roc_curve(y, grec_m['path'])\n",
"auc = roc_auc_score(y, grec_m['path'])\n",
"ax.plot(fpr,tpr,'b-',label=\"AUC = {:.2f}\".format(auc))\n",
"ax.plot([0,1],[0,1],'r--')\n",
"ax.set_xlabel('False Positive Rate')\n",
"ax.set_ylabel('True Positive Rate')\n",
"#ax.tight_layout()\n",
"# remove this code to remove the border\n",
"ax.set_ylim(0, 1.02)\n",
"ax.set_xlim(-0.02, 1.02)\n",
"\n",
"ax.legend(loc=0)\n",
"fig.savefig(\"figures/Validation on Annotated Corpus/ROC_Validation_on_Annotated_Corpus.pdf\", bbox_inches='tight')\n",
"fig.savefig(\"figures/Validation on Annotated Corpus/ROC_Validation_on_Annotated_Corpus.png\", bbox_inches='tight', format='png')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Validation on Degrees"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"grec_paths = pd.read_csv(\"data/GREC/GREC_degree/gt_with_paths.csv\", encoding='utf-8', sep=';', index_col=0)\n",
"grec_paths[:5]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"grec_m2, grec_u2, grec_mold2 = get_grec_truth_series(grec_paths)"
]
},
{
"cell_type": "code",
"execution_count": 124,
"metadata": {},
"outputs": [],
"source": [
"grec_m2 = grec_m2.dropna()"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Spearman Correlation\n",
"0.24988641492145977\n",
"1.4810703681758453e-08\n",
"Tau Correlation\n",
"0.19662283194111033\n",
"2.6569565626479824e-08\n"
]
}
],
"source": [
"from scipy.stats import pearsonr, spearmanr, kendalltau\n",
"r, pval = spearmanr(grec_m2[\"score\"], grec_m2[\"path\"])\n",
"r_tau, pval_tau = kendalltau(grec_m2[\"score\"], grec_m2[\"path\"])\n",
"\n",
"print(\"Spearman Correlation\")\n",
"print (r)\n",
"print (pval)\n",
"print(\"Tau Correlation\")\n",
"print (r_tau)\n",
"print (pval_tau)"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAF3CAYAAABUsGfpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmYXGWZ/vHvnU6AhK1ZAgwNMREhSAgQaRYBQRYF1EBAUFARHBRnFB3HMRrUkUVnQDMjOoo/wXFFBJElRkGDCoIwAukQEgiLExkI6SgEIYgQyPb8/jin0tWVqq7T6T5V3XXuz3XV1VVvnVP1VKdTz3l3RQRmZmZ9GdHsAMzMbOhzsjAzs7qcLMzMrC4nCzMzq8vJwszM6nKyMDOzupwszMysLicLMzOry8nCzMzqcrIwM7O6RjY7gMGy/fbbx/jx45sdhpnZsDJv3rxnImJsveNaJlmMHz+erq6uZodhZjasSHoiy3FuhjIzs7qcLMzMrC4nCzMzq8vJwszM6nKyMDOzupwszMysLicLMzOry8nCzMzqcrIwM7O6nCzMzKwuJwszM6vLycLMzOrKNVlIOk7So5IWS5pR5fnDJd0naY2kU6o8v5WkbklfzzNOMzPrW27JQlIbcBlwPLAXcLqkvSoOWwKcBfyoxst8Hrg9rxjNzCybPGsWBwKLI+KxiFgFXAOcWH5ARDweEQuBdZUnS9of2BG4JccYzcwsgzyTRQfwZNnjpWlZXZJGAP8JTM8hLjMz66c8k4WqlEXGcz8E3BwRT/Z1kKRzJHVJ6lq+fHm/AzQzs2zy3ClvKbBr2eNdgGUZz3098AZJHwK2ADaR9LeI6NVJHhFXAFcAdHZ2Zk1EZmbWT3kmi7nA7pImAN3AacC7spwYEe8u3Zd0FtBZmSjMzKxxcmuGiog1wLnAHOBh4NqIWCTpIkknAEg6QNJS4FTgckmL8orHzMw2niJao/Wms7Mzurq6mh2GmdmwImleRHTWO84zuM3MrC4nCzMzq8vJwszM6nKyMDOzupwszMysLicLMzOry8nCzMzqcrIwM7O6nCzMzKwuJwszM6srz4UEzcwsR7PmdzNzzqMsW7GSndtHM/3YiUybkmnboH5zsjAzG4Zmze/mvBseYOXqtQB0r1jJeTc8AJBLwnAzlJnZMDRzzqPrE0XJytVrmTnn0Vzez8nCzGwYWrZiZb/KB8rJwsxsGNq5fXS/ygfKycLMbBiafuxERo9q61U2elQb04+dmMv7uYPbzGwYKnViezSUmZn1adqUjtySQyU3Q5mZWV1OFmZmVpeThZmZ1eVkYWZmdTlZmJlZXU4WZmZWl5OFmZnV5WRhZmZ1OVmYmVldThZmZlaXk4WZmdXlZGFmZnU5WZiZWV1OFmZmVpeThZmZ1eVkYWZmdTlZmJlZXU4WZmZWl5OFmZnV5WRhZmZ15ZosJB0n6VFJiyXNqPL84ZLuk7RG0ill5ftJ+r2kRZIWSnpnnnGamVnfcksWktqAy4Djgb2A0yXtVXHYEuAs4EcV5S8B742IScBxwFcktecVq5mZ9W1kjq99ILA4Ih4DkHQNcCLwUOmAiHg8fW5d+YkR8Yey+8skPQ2MBVbkGK+ZmdWQZzNUB/Bk2eOlaVm/SDoQ2AT44yDFZWZm/ZRnslCVsujXC0h/B1wJvC8i1lV5/hxJXZK6li9fvpFhmplZPXkmi6XArmWPdwGWZT1Z0lbATcBnI+LuasdExBUR0RkRnWPHjh1QsGZmVlueyWIusLukCZI2AU4DZmc5MT3+RuAHEfGTHGM0M7MMcksWEbEGOBeYAzwMXBsRiyRdJOkEAEkHSFoKnApcLmlRevo7gMOBsyTdn972yytWMzPrmyL61Y0wZHV2dkZXV1ezwzAzG1YkzYuIznrHeQa3mZnV5WRhZmZ1OVmYmVldThZmZlaXk4WZmdXlZGFmZnU5WZiZWV1OFmZmVpeThZmZ1eVkYWZmdTlZmJlZXXnulGdmTTJrfjcz5zzKshUr2bl9NNOPnci0Kf3ee8xsPScLsxYza343593wACtXrwWge8VKzrvhAQAnDNtoboYyazEz5zy6PlGUrFy9lplzHm1SRNYKnCzMWsyyFSv7VW6WhZOFWYvZuX10v8rNsnCyMGsx04+dyOhRbb3KRo9qY/qxE5sUkbUCJwuzFjNtSgdv37+DNgmANom379/hzm0bECcLsxYza34318/rZm26ZfLaCK6f182s+d1NjsyGMycLsxbj0VCWB8+zMGsxHg1VHI2cfOmahVmL8WioYihNvuxesZKgZ/JlXs2NThZmLaaoo6Fmze/m0EtuZcKMmzj0kltbvo+m0c2NfSYLSW2SZubyzmaWi2lTOrj45Ml0tI9GQEf7aC4+eXJLj4Zq9FX2UNDo5sY++ywiYq2k/SUpIh1aYWZD3rQpxRoq29dVdqv+HnZuH013lcSQV3Njlmao+cBPJZ0h6eTSLZdozMw2QhE79Rvd3JhlNNS2wF+Ao8rKArghl4jMzPqp0VfZQ0GpxtSo0VB1k0VEvC+XdzYzGyTTj53Ya1l2KEanfiObG+s2Q0naQ9JvJD2YPt5H0mfzD83MLJsiduo3mur1W0u6HZgOXB4RU9KyByNi7wbEl1lnZ2d0dXU1Owwzs2FF0ryI6Kx3XJYO7jERcW9F2ZqNC8vMzIajLMniGUm7kXRqI+kU4E+5RmVmZkNKltFQHwauAPaU1A38H/CeXKMyM7MhJUuy6I6IYyRtDoyIiBckbZt3YGZm1rehtpDgDZJGRsSLaaLYCfhVLtGYmVkmQ3EhwVnAdek6UeOBW4DzconGzMwyafRCglkm5X1L0iYkSWM88MGI+J9cojEzs0yGzEKCkj5e/hDYFbgfOFjSwRHx5VwiMjOzuobSQoJblt22AG4EFpeV1SXpOEmPSlosaUaV5w+XdJ+kNemQ3PLnzpT0v+ntzKwfyMysCIbMQoIRceFAXlhSG3AZ8CZgKTBX0uyIeKjssCXAWcAnKs7dFjgf6CSZ3zEvPfe5gcRkZtYqhtxCgpJ+BZwaESvSx9sA10TEsXVOPRBYHBGPpeddA5wIrE8WEfF4+ty6inOPBX4VEc+WxXAccHWGz2RmVghDaiFBYGwpUQCkV/c7ZDivA3iy7PHStCyLgZxrZmaDLEuyWCtpXOmBpFeRLv1Rh6qUZd1tL9O5ks6R1CWpa/ny5Rlf2szM+ivLDO7PAHemq88CHA6ck+G8pSQjqEp2AZZljGsp8MaKc39beVBEXEGyFAmdnZ3e9tXMCqWRM7izzLP4paTXAQeTXPH/c0Q8k+G15wK7S5oAdAOnAe/KGNcc4N/T/hGAN+OJgGZm682a3830nyxg9brkOrl7xUqm/2QBQC4JI8vmRyLpXH5dRPwMGCPpwHrnRcQa4FySL/6HgWsjYpGkiySdkL72AZKWAqcCl0talJ77LPB5koQzF7io1NltZmZwwexF6xNFyep1wQWzF+Xyflmaob4BrCPZg/si4AXgeuCAeidGxM3AzRVlnyu7P5ekianaud8BvpMhPjOzwlmxcnW/ygcqS7I4KCJeJ2k+JKOh0uU/zMysILKMhlqdTrArbX40lqSmYWZmTTJmVPWv71rlA5XlVf+LZKmPHST9G3An8O+5RGNmZplsWrHUR73ygcoyGuoqSfOAo0lGQ02LiIdzicbMzDJZ8VKNPosa5QOVpc8C4H+Bv5aOlzQuIpbkEpGZmdU1lFadBUDSR4CnSHbH+zlwU/rTzMya5Mg9x/arfKCy9Fn8EzAxIiZFxD4RMTki9sklGjMzy+SmhX/qV/lAZUkWTwLP5/LuZma2UZ6r0TdRq3ygsuyU9xjwW0k3Aa+UnvdOeWZmxdFXB3dpN7wl6W2T9GY2rDRysTWzRmkfParqbO320aNyeb++ksWXgC0j4unyQkk74mYpGyZmze/mvBseYOXqtUCy2Np5NzwA5LPYmlmjXHDCpF4LCQKMGiEuOGFSLu/XV5/FV4HDqpQfA1yaSzRmg2zmnEfXJ4qSlavXMnPOo02KyGxwTJvSwcxT96WjfTQCOtpHM/PUfZuyRPlhEbHBvhXpJL1P5xKN2SBbVmUcel/lZsPJUNlWtdpudVnOMxsyak1Qymviklmr6utL/+lq+1ZIOgDwHqY2LEw/diKjK9bKGT2qjenHTmxSRGbDU1/NUNOBayV9D5iXlnUC7yXZ9c5syCtV0T0aymxgaiaLiLg3rVl8GDgrLV5Esr/F07XOMxtqGtmua9aq+lxIME0K5zcoFjMzG6KyrjprZsOIJyLaYHOyMGsxnohoeciyRPmpWcrMbGjwRETLQ5b5EudlLDOzIcATEYtj1vxuDr3kVibMuIlDL7mVWfO7c3uvvladPR54C9Ah6b/KntoKWJNbRGY2II3eQc2ao9HNjX3VLJYBXcDLJPMsSrfZwLGDHomZDQpPRCyGRjc39jXPYgGwQNKNwIsRsRZAUhuwaS7RmNmAeSJiMTS6uTHLaKhbSFaa/Vv6eHRadkguEZnZgHkiYutrdHNjlg7uzSKilChI74/JJRozM8vkyD3H9qt8oLIkixclva70QNL+gIdVmJk10W2PVF/PtVb5QGVphvoY8BNJy9LHfwe8M5dozMwskyHXZxERcyXtCUwk2ePikYjYcONXMzNrmPYxo3jupSp7cI9p/B7c5SYCewGbAVMkERE/yCUiMzOr6+WKYbP1ygeqbrKQdD7wRpJkcTNwPHAn4GRhZtYkK1ev61f5QGXp4D4FOBr4c0S8D9gXz7MwMyuULMliZUSsA9ZI2gp4Gnh1vmGZmdlQkqXPoktSO/AtkuU+/gbcm2tUZmY2pGQZDfWh9O43Jf0S2CoiFuYblplZ/3jDp3zVbIaSdG7Z/UkAEfG4E4WZDTWlFVi7V6wk6FmBNc8lu4umrz6Lvy+7f2XegZiZbSxv+JS/LB3ckEzG6zdJx0l6VNJiSTOqPL+ppB+nz98jaXxaPkrS9yU9IOlhSd5sycxq8oZP+eurz6Jd0kkkCWUrSSeXPxkRN/T1wulS5pcBbwKWAnMlzY6Ih8oOOxt4LiJeI+k04IskS4mcCmwaEZMljQEeknR1RDzez89nZgXgDZ/y11fN4nbgBOBtwB3A1LLb2zK89oHA4oh4LCJWAdcAJ1YccyLw/fT+dcDRkgQEsLmkkSRLoq8C/prpE5lZ4XjDp/z1tfnR+wb42h3Ak2WPlwIH1TomItZIeh7YjiRxnAj8iWQ59H+OiGcHGI+Ztagibvi0TY21obZp8tpQG6NaP0dkPOZAYC2wM7AN8DtJv46Ix3qdLJ0DnAMwbty4AQdsZsNX0TZ8On/qJKZft4DVa3u+Vke1ifOnTsrl/bJ2cG+MpcCuZY93IdnXu+oxaZPT1sCzwLuAX0bE6oh4GrgL6Kx8g4i4IiI6I6Jz7Nh8NvwwMxuKpk3pYOYp+9LRPhoBHe2jmXnKvrklzDyTxVxgd0kTJG0CnAbMrjhmNnBmev8U4NaICGAJcJQSmwMHA4/kGKuZmfWhZjNU5einSvVGQ6V9EOcCc4A24DsRsUjSRUBXRMwGvg1cKWkxSY3itPT0y4DvAg+SNFV915MBzcx6lCYiluaXlCYiArnULpRcyFd5QvpuH+dFRPx9H883XGdnZ3R1dTU7DDOzhjj0klurDhfuaB/NXTOOyvw6kuZFxAbN/JXyHA1lZmY5qZYo+iofqEyjoSS9FZhEslMeABFxUS4RmZlZXW0Sa6u0DLVpoxbcqKtuB7ekb5LMqv4ISf/BqcCrconGzMwyqZYo+iofqCyjoQ6JiPeSLMtxIfB6eg+JNTOzFpdpp7z050uSdgZWAxPyC8nMzIaaLH0WP093ypsJ3Ecyw/q/c43KbBB5UxyzgcuyU97n07vXS/o5sFlEPJ9vWGaDo9Fj0c1aVdbRUIcA40vHSyIifpBjXGaDoq9NcZwszLKrmywkXQnsBtxPsrgfJE1RThY25HlTnOJwc2O+stQsOoG9otZUb7MhzJviFIObG/OXZTTUg8BOeQdilgdvilMM3oM7f1lqFtuTbGt6L/BKqTAiTsgtKrNBUsRNcYrIzY35y5IsLsg7CLM8FW1TnCJyc2P+6jZDRcTtJHtJbJneHk7LzMyGBDc35i/L2lDvAO4lWRPqHcA9kk7JOzAzs6ymTeng4pMn99o17uKTJ7tGOYiyNEN9Bjgg3d4USWOBXwPX5RmYmVl/uLkxX1lGQ40oJYrUXzKeZ2ZmLSJLzeKXkuYAV6eP3wncnF9IZmY21GRZG2q6pLcDh5LsZ3FFRNyYe2RmZjZkZFobKiKuB67PORYzMxuiaiYLSXdGxGGSXiBZC2r9U0BExFa5R2dmG8XrJNlgq5ksIuKw9OeWjQvHGsFfJK3N6yRZHrLMs7gyS5kND6Uvku4VKwl6vkhmze9udmg2SLxOkuUhS5/FpPIHkkYC++cTjuWtiPs7FK0m5XWSLA81axaSzkv7K/aR9Nf09gLwFPDThkVog6poXyRFrEnVWg/J6yTZQNRMFhFxcdpfMTMitkpvW0bEdhFxXgNjtEFUtC+SIjbJTD92IqPa1KtsVJu8TpINSJaFBM+TtI2kAyUdXro1IjgbfEfuObZf5cNd0WpS61VuVeaty2yAsnRwvx+4A5gDXJj+vCDfsCwvtz2yvF/lw13RalKQ1KZWr+udHVavi5auTVn+sqzx9E/AAcATEXEkMAVozW+WAijalXYRl64u2r+xNUaWZPFyRLwMIGnTiHgEaN3/aS2uaFfaRVy6un3MqH6Vm2WRZejsUkntwCzgV5KeA5blG5bl5cg9x/LDu5dULW9VRVu6Omr0T9QqN8siy0KCJ6V3L5B0G7A18Itco7LcFK3PooieX7m6X+VmWdRNFpKujIgzYP0Wq6UZ3GfkHJvlwO3ZrW/r0aNYUSUxbD26tZuhijb5stE2ZgZ3Gy00g7tof2De2L71rVqztl/lrcDrYeVvY2ZwP02LzOAu4uzeIo4OKpqXVq/rV3krKOLky0Yr9AzuIv6BFXF0kLU+N6/mL0sz1C+qzdiOiDtyiKehivoHVrTRQUWzzZhRPPfShn0W27Tw0Fk3r+YvyzyL6WW3fwV+RovM4C7anAMrhvOnTqq6NtT5UyfVOGP4m37sREaNqPjMI7we1mDKsjbU1LLbm4C9SVaerUvScZIelbRY0owqz28q6cfp8/dIGl/23D6Sfi9pkaQHJG2W/WNl4wXXrBVNm9LBzFP27dXUOPOUfVu/Nqk6j21AMu3BXWEpScLoUzpq6jLgTek5cyXNjoiHyg47G3guIl4j6TTgi8A70z0zfgicERELJG0H5DNI3AuuWQsqWlPjzDmPsnptxXpYa6Ol92lptCzzLL5Gz1foCGA/YEGG1z4QWBwRj6Wvcw1wIlCeLE6kp0nrOuDrkgS8GVgYEQsAIuIvGd6v3/pacK2V/8CKNlzYWl+1/oq+yq3/stQsusrurwGujoi7MpzXATxZ9ngpcFCtYyJijaTnge2APYCQNAcYC1wTEV/K8J79UsQObo9Ht1bUJrG2ynombXJb1GDJ0mfxfeBqYD6wEJib8bWr/StV/mvWOmYkcBjw7vTnSZKO3uANpHMkdUnqWr68/8tV1JrR2sozXYs4XNhaX7VE0Vd5y4hg09WvNOStsjRDvQW4HPgjyZf7BEkfjIh660MtBXYte7wLGy5AWDpmadpPsTXwbFp+e0Q8k8ZwM/A64DflJ0fEFcAVAJ2dnf3+q6h10dHKFyNFrE1Z6+uoMXS2YziObIyApUth4cLk1tEB730vLFkCr3rV+sMeLztl/Kd+nntYWZqhvgwcGRGLASTtBtxE/cUE5wK7S5oAdAOnAe+qOGY2cCbwe+AU4NaIKDU/fVLSGGAVcARwabaPlN2KKmPR+ypvBR6Pbq1oWKymvG4d3HEH3H03LFiQJIKHHoIpU+C+++BPf4Kdd65+7nvfC6urfy99d/+pOQbdI0uyeLqUKFKPkSz50ae0D+Jckp312oDvRMQiSRcBXRExG/g2cKWkxSQ1itPSc5+T9GWShBPAzRFxU38+WBZFXHBt+rETmf6TBb069j0e3Ya7ny/4U83yL0ybnM+bRsCyZckV/0EHwYgRcPrpcM01Gx67bh2sWQNHHrnhc/PnJz/b23vKdtoJ9tknub35zUnZbrv1Wmd+/IxB/0rsU5ZksShtBrqW5Iv7VJJhsCcDRMQNtU6MiJuBmyvKPld2/+X09aqd+0OS4bO5KWIzFODx6NZyql309VVeUykBLFyYXP0vWADPPw9XXw1bbw1bbQUvvLDheU89BTvsUD1RALzyCmy2GXzyk0ni2Hff5DZxImyySXLM6NFDetORLMliM5JJeEekj5cD2wJTSZJHzWQx1BWxGcrj0YvBw6PLRCS3ESPg17+G73wnSQaLFvUcs2gR7LVXckw1jz2WNBe9UtGZvMMOcMwxMGZM8njpUhg7ticBVPriFwf+eZoky+ZH72tEIM1QxPZ7d3C3vkIMj45I2vgXLoQHH2TCyh35v9Hbcs491/Pp336397FfBG64AU46Kbn97W8bvt7zzyc/P/EJ+P73k6v+ffZJfk6ZkiQS2DBZVOpokd9vFVlGQ00APgKMLz8+Ik7IL6zGmH7sRKZft6DXlXarL/dRxARZNH0Njx7yyeLFF+HOO3uagNJkwOzZMHUqTJ8O//EfG5w29c3v57+mTOPQJ2rMF54wIfn5m9/AI48kSWDPPWHTTXsfN3NmcrMNZGmGmkXSEf0zoPUWxC/Ych/Tj53Y66oTvJ9Fqxlytcc//zn5udNOsHhxcnX/4IO9j3n/++Fb34KvfAU++9kNX+OGG5JkUbrCB9h++/Vt/ze8lKxAdOapF/bqdBTwf5e8teecAw9MbtZvWZLFyxHxX7lH0gRFXO6j9Lncnt26GlJ7/POfe678FyyAI46AD3wA7r8/abaptP32sHx5MvKnMlFA0rkLcPLJcPvtMHlyTzPQa1/bUwN43/uSW4W45FZYsXKD0SmuMQ+eLMniq5LOB24B1jfYRcR9uUXVIEPuCqxBirbI3GdnPcDV9zzJ2gjaJE4/aNf8hlMOAQOqPb7yCnz3u3DXXT2TwgD23x+6upLEsN9+G5531VVJslizZsPntt8ePv/55P5b3pK8xh57JKODKr32tXDLLRk+ZW+uMecvS7KYDJwBHEVPM1Skj4e19hqbxLS38CYxRfPZWQ/0mqy1NmL941ZNGOW1x1Xdyzj0pWWc/tptOGjKcUky2G+/pN2+UgT85S/wj/+44XPz5iU/d94ZttgiudIv7wQ+7LDk+c7Ovod/br55cs4gc405f1mSxUnAqyNiVd7BNNorq6tvYF+r3Iafq+95smb5sE0WzzyTNOeUzwUYMQLuuScZsikxDZhWed4//z2MGlU9Uey2W/Jzhx2SDuRtt02SwF579a4BjB1bfZ6BtbwsyWIB0E6GWdvDTRE3ti+aYbPA3HPPJVfso0bBN7+Z3BZUjOxZsgR23TX5wq5m+fINh26WvvSPOSa54m9rS5aNGFnjv/7IkfAv/zLwz9NghRgu3GRZksWOwCOS5tK7z2LYD5211tfUpauXL+/pBF68GD7zmZ5F4a68csPjf/e7pDmnWjMQ9Izx/+QnYe7cnmagffZJagClTuKI6pPySgmiVqIYxob1cOFhIstfzfm5R9EkY0aNqFqLGDMqy9bkNhycftCuVReYO/2gXascndGSJXD99b3nAqxbB7/9bTIq6I1vTEb0VDriCHjnO3s6jcvtsktSa4Dk+VdegUmTehJAuTqzgGfN7+41f6h7xUqmX5fUUlr1i7Oog1UaKcsM7tsl7QgckBbdGxEt0SS1yci2qslik5FtTYjG8lDql+hzNNQzzyRf+rvtBuPHJyuDHnlkkgDKfe5zcOGFMGNGslZQpQceSBLC1KlJsmhv71kDaJ994Pjjk+Puv7/voCcPrC/lwp8tqrqky4U/W9SyycKTTfOXZQb3O4CZwG9J5rh8TdL0iLgu59hy93yNRcZqldswUkoACxdyzm2/571d8/nuPsdxxxtP4vhlD4CqjMg54oikdnDffRsmCujpD/jUp5J+g1IT0KRJPWsDQdLm38R2/2oj/PoqbwUeOpu/LM1QnwEOKNUmJI0Ffk2yZ/awNnIEVOvLHulWqKFp1apktM+KFcks39//PkkIa9MviMMOS9r9f/GLZDx/alz684KnL2fifsdxxaInObT8ddvbky/9z3wmefwP/wDnnNM7AZTbd1/46lcH+9PZAEyb0kHXE8/2qkG+ff9izSfKW5ZkMaKi2ekvZNiOdTioNejJg6Ea6Nlne9r9t9gCzj47qRXUGvETAU88AZddtuFzS9K+iUmTYPfdYaeduG7NdnRttQsP7zCBR8aOB+D2jr059OLfcNeMGlOFqk0WsyFt1vxurp/XvX4ww9oIrp/XTeertnXCGCRZksUv053rSo2076T+LnlWdPfe23PlX0oG48bBH/8IL72UTM6q5uyza7/maaclPydOhGuvTbaYnDRpw9caNw7+8AcAps+4qepyX+74bC0eDZW/LB3c09ONjg4j6bO4IiJuzD0yG1qeey75ot9332QuwLnnwuWXb7i8wwsvJDWEgw7a8DUeeyxpSiof4bPVVj2zgI9Kr/S3377vWcCbbQanVt0zawPu+CwGj4bKX81kIek1wI4RcVe6G94NafnhknaLiD82Ksi8HLrbttz1x2erlhfCc8/1Xgzuqafg29+GHXdMrtgfemjDcx55JLmyr9YMBMmQzy22SEYO/eUvPSOBJk1KyksaNCmuiB2fRRwS7ouC/PVVs/gK8Okq5S+lzzVml/AcXfWB1/Pub/2+V8I4dLdtueoDr29iVAMUkay8ec898N//3dMMtCpdreWee5IlmkeO7OkYLrdoUZIs/vrX3uVbbgmHH97Tl7BkCWyzTe8EUO7CCwfvMw1AEdcMOnn/XarOLTl5/12aEE1jeG/5/PWVLMZHxAazhyKiS9L43CJqsFM7x/H4X1au/yI5tXNc/ZMabcWKnqv/t70t2cjl+9+Hs87a8NhvfSvZG+CDH9xwuQhIahMA558PX/pSTxNQ6bb//snzT1ZfU2m9XQcwqc2hs1lRAAAT2ElEQVRyddsjy/tV3jK8t3yu+koWfQ0JaYm6XVPXk3nlleQqv7wZaMGCpDZwxhnwta/BRz+64XkPPwzf+AbcfXf11y0tCPe97yWvv88+ySSvyhrAv/5rcmtxRVwzqFpzTF/lrcB7y+evr0bMuZI+UFko6WxgXn4hNU5fIyg2yooVyb7AkGzcfvjhSWeu1HObmrbe/ehHySSwj340SRBz5yZNRVdckTw/fnzP626+ORxySDL+/yMfScq+8Y2ejejLb0cemTy/335J7eL1r6/dVFQAg/5vPAyMqHFFXau8FbiDO3991Sw+Btwo6d30JIdOYBOSZcuHvUx/YCtWJMs4lGoAe++dfMEvXpyM5a8mIqkB/O53Gz5XGsP/hjfA0Ucnr1FqCtp772R0ECRJpa9O4EYshNcCivglsq7Gn02t8lbgDu781UwWEfEUcIikI4G90+KbIuLWhkTWAKXNj0568FYunvN1NluziiVb78iJH/9BMvGr/Oq+3Ec/Wn05iDFjevYPPvTQpLawxx49CaDca14Dv/71oH0Wq85fIsVw5J5jq3bqH7lnjcmd1m9Z5lncBtzWgFga7sVXkjkCl9705fVl455/KinffvtkBNCaNb13BHvDG5ID99ij7yv/MWOSXcOsqYo4dFZQdSJiK9dFC9up30Ctt7B9P6xKO8RePf2nbL76ZV7YNJ0JvDaSfoLK4aM27BRxzaBalzAt3ApVyObGRmvdWTr9sG5EW0+isJYya343P577ZK81g34890lmze9ucmT5aR9dfQ/5WuWtoFazopsbB4+ThbW0vvZ2aFWr1lTfQ75WeSuo1TfhPovBU+hksfkm1Tc5qlVuw08R93Yo4t7y7rPIX6H7LF5aVf1Kq1Z5q6i6P3MLt+Fb6yviRMRGK3TNoojtnKUZzd0rVhL0zGhu1Tb8IrbfbzOm+merVd4K2mrMO6pVbv1X6GQx/diJjKqY1trqi48VbUbzBSdMqvpvfMEJk5oUUf7OnzqJUW0Vn7lNnD+1dT/z2hrD2GuVW/8VuhkKKNziY0UbYljEVWeL+Jk7aky+7GjhVoJGK3SyKOLiY0Wc0TxtSmvPq7BiTr5stEI3QxXtKhuS/1SjR/Ue7eX/VK2laP1SkFwQXHzyZDraRyOSGsXFJ0/2RcIgKnTNoqhX2VCsJoqiKep+1K5B5qvQyaKoVVf/p2ptRawxg4eE563QycJX2daKilhjLuImV42Wa5+FpOMkPSppsaQZVZ7fVNKP0+fvqdyuVdI4SX+T9Im8Ypw2pYO7ZhzF/13yVu6acZT/sGzYK2K/VNGGhDdDbslCUhtwGXA8sBdwuqS9Kg47G3guIl4DXAp8seL5S4Ff5BWjWSsqYmdvUZveGinPZqgDgcUR8RiApGuAE4GHyo45EbggvX8d8HVJioiQNA14DHgxxxjNWlLR+qWK2PTWaHk2Q3UAT5Y9XpqWVT0mItYAzwPbSdoc+BRwYY7xmbWsWfO7OfSSW5kw4yYOveTWlh42C8Vsemu0PGsW1eZCV869r3XMhcClEfE39bG2i6RzgHMAxo0bt1FBfnbWA702xjn9oF35wrTJG/Vaw4VHjbS2Inb2erBK/vKsWSwFdi17vAuwrNYxkkYCWwPPAgcBX5L0OPAx4NOSzq18g4i4IiI6I6Jz7Nj+r1v/2VkP8MO7l/TaGOeHdy/hs7Me6PdrDRdFnLBVNO7sLYYdt9ykX+UDlWeymAvsLmmCpE2A04DZFcfMBs5M758C3BqJN0TE+IgYD3wF+PeI+PpgB3j1PU/2q7wV+Iuk9RWxs7eIF0HnvaVyvFDf5QOVW7JI+yDOBeYADwPXRsQiSRdJOiE97NskfRSLgY8DGwyvzVMRV6os6hdJkdrvt66x/Hqt8lZQxIugWp8tr8+c66S8iLgZuLmi7HNl918GTq3zGhfkEhzJWvfVEkMrr4FftFEjRWy/L+K2qkW8CGr0Zy70QoKnH7Rrv8pbQdFGjRTxirOI26q219jYqVZ5K2j05m2FThZfmDaZ9xw8bn1Nok3iPQePa+nRUEWbsFXEK84iemV19VpTrfJW0OgLv0KvDQVJwmjl5FBNkSZsFa3ZDZLtU597aXXV8lZVxNpUo4cLFz5ZWGsr4srC50+dxPTrFvTa2KvVt1UtqkZe+DlZWEsr4mStIn7m9tGjWLFyw9pUewuPAGs0RYsME+3s7Iyurq5+n+fZzGbD36z53Uz/yQJWryurTY0QM0/d1/+f65A0LyI66x1X6JpFEYdVWjEU7SKoiLWpRit0sijq9pPW2op6EVSkgRvNUOihs9VGyfRVbjYcFHFuieWv0Mmi1kTtFp7AbQXguSWWh0Ini1p9+y3S528F1eiZvVYMhU4WZq2oaEu6WGMUuoN7803aeHHVhssBbL5JW5WjzYYHjwyyPBQ6Wayr0d5Uq9xsuPDIIBtshU4WK2usG1Or3Gy4KOJ2wZavQicLs1ZU2i64pLRdMOCEYRut0B3ctVbhbOXVOa31FXG7YMtfoZPF+VMnMaqt96QKr85pw10Rtwu2/BW6GcqjRqwVFXG7YMtfoZMFeNSItZ7TD9q1V59FebnZxip8sjBrNaVObI+GssFU+P0szMyKLOt+FoXu4DYzs2wK3wxVtE1izMw2RqGTRVE3iTEz669CN0N5kxgzs2wKnSy8SYyZWTaFThbtNZb1qFVuZlZUhe6z8E55xeBBDGYDV+hk8fzK1f0qt+HHgxjMBkehm6G8V3Hr8yAGs8FR6GThvYpbnwcxmA2OQieLaVM6uPjkyXS0j0ZAR/toLj55spsnWohrj2aDo9B9FuBVZ1vd9GMn9uqzANcezTZG4ZOFtTbvWWI2OJwsrOW59mg2cIXuszAzs2xyTRaSjpP0qKTFkmZUeX5TST9On79H0vi0/E2S5kl6IP15VJ5xmplZ33JLFpLagMuA44G9gNMl7VVx2NnAcxHxGuBS4Itp+TPA1IiYDJwJXJlXnGZmVl+eNYsDgcUR8VhErAKuAU6sOOZE4Pvp/euAoyUpIuZHxLK0fBGwmaRNc4zVzMz6kGey6ACeLHu8NC2rekxErAGeB7arOObtwPyIeCWnOM3MrI48R0OpSlnlEn19HiNpEknT1JurvoF0DnAOwLhx4zYuSjMzqyvPmsVSYNeyx7sAy2odI2kksDXwbPp4F+BG4L0R8cdqbxARV0REZ0R0jh07dpDDNzOzkjyTxVxgd0kTJG0CnAbMrjhmNkkHNsApwK0REZLagZuA8yLirhxjNDOzDHJLFmkfxLnAHOBh4NqIWCTpIkknpId9G9hO0mLg40BpeO25wGuAf5V0f3rbIa9Yzcysb4oW2emns7Mzurq6mh2GmdmwImleRHTWO84zuM3MrC4nCzMzq8vJwszM6nKyMDOzupwszMysLicLMzOry8nCzMzqcrIwM7O6nCzMzKwuJwszM6vLycLMzOpysjAzs7qcLMzMrC4nCzMzq8vJwszM6nKyMDOzupwszMysLicLMzOry8nCzMzqcrIwM7O6nCzMzKwuJwszM6tLEdHsGAaFpOXAE82OYyNsDzzT7CAazJ+5GIr2mYfr531VRIytd1DLJIvhSlJXRHQ2O45G8mcuhqJ95lb/vG6GMjOzupwszMysLieL5rui2QE0gT9zMRTtM7f053WfhZmZ1eWahZmZ1eVkMYRI+oSkkLR9s2PJm6SZkh6RtFDSjZLamx1THiQdJ+lRSYslzWh2PHmTtKuk2yQ9LGmRpH9qdkyNIqlN0nxJP292LHlwshgiJO0KvAlY0uxYGuRXwN4RsQ/wB+C8Jscz6CS1AZcBxwN7AadL2qu5UeVuDfAvEfFa4GDgwwX4zCX/BDzc7CDy4mQxdFwKfBIoRCdSRNwSEWvSh3cDuzQznpwcCCyOiMciYhVwDXBik2PKVUT8KSLuS++/QPLl2dHcqPInaRfgrcB/NzuWvDhZDAGSTgC6I2JBs2Npkr8HftHsIHLQATxZ9ngpBfjiLJE0HpgC3NPcSBriKyQXe+uaHUheRjY7gKKQ9GtgpypPfQb4NPDmxkaUv74+c0T8ND3mMyRNF1c1MrYGUZWyQtQcJW0BXA98LCL+2ux48iTpbcDTETFP0hubHU9enCwaJCKOqVYuaTIwAVggCZLmmPskHRgRf25giIOu1mcukXQm8Dbg6GjNMdxLgV3LHu8CLGtSLA0jaRRJorgqIm5odjwNcChwgqS3AJsBW0n6YUS8p8lxDSrPsxhiJD0OdEbEcFyQLDNJxwFfBo6IiOXNjicPkkaSdN4fDXQDc4F3RcSipgaWIyVXPN8Hno2IjzU7nkZLaxafiIi3NTuWweY+C2uWrwNbAr+SdL+kbzY7oMGWduCfC8wh6ei9tpUTRepQ4AzgqPTf9f70ituGOdcszMysLtcszMysLicLMzOry8nCzMzqcrIwM7O6nCzMzKwuJwsDQNKOkn4k6TFJ8yT9XtJJTYjj8cpVdyXdkw7BXCJpedmQzPH9eN2jJB1c9viHkqZlOG9zSb+VNCJ9PFHSTZL+mP6ebpV0WPrc+8vie0TSR8te5wuSustiv1/SlulzB0u6M12d9hFJV0gaXRHHMZKeT1c1/YOk27MMSa383INB0raS/qHs8U6SbhrM97Chx8nCShOpZgF3RMSrI2J/4DSqLO6XTjRrqIg4KCL2Az4H/Dgi9ktvj1fE1tbHyxxFsgpqf70f+ElErJM0Bvg58I2I2C39PX0MeHXZ8Velsb4BuEDS35U9N7Ms9v0i4oX0+R8DH4+IiSSr0/4G2KJKLLdFxJSI2AP4Z+D/STqiTvwb+7n7si2wPlmkKw08K+mgQX6fDTTj788SThYGyRfKqohYPzEuIp6IiK8BSDpL0k8k/Qy4RYmZkh6U9ICkd6bHvbF8LX9JX5d0Vnr/cUkXSrovPWfPtHw7SbekV8yXU309paokjZS0Ir1qvxc4UNJSpXtjpFfsv5a0G8mX/vT0iv6Q9CWOlPQ/aW2qVi3q3cBP0/tnkCTU9VfREbEwIn5QeVI6K/0x4O8qn6vwEeDbEXFvet66iPhxvVnt6cqu/0Yy6Q9JJ6Y1sPnp73OHap+72nHp+UdJWpAed5+kzdPyGZLuVbLvyOfSt78EmJgee0laNiv9XfUiqSOtNd2f/r0ckpa/NX2fBZJuScu2lzQ7fa//kbR3Wv4FSZdL+hXw3fTf/ctlcb2/zu/YBoGThQFMAu6rc8zrgTMj4ijgZGA/YF/gGGBmxRV0Lc9ExOuA/wd8Ii07H7gzIqYAs4Fx/Yx9a+C+iDgwIn5f7YCI+CPJ0tGlK/v/SZ/agWTG8TTg4srzJG0G7BIRS9OiLL+n0rnjgTbgwbLi0pf2/UoWWQTYG5iX5TWruA/YM71/B3Bw+nu8gWRPiWqfe4PjSrEB56S1osOBl9NmrnHAQST/3oekX/YzgEfT1yxt6NRFUpuq9B7gZ+nr7gsslLQTyd/ASRGxL0ktFuDzwD3pHicXAN8re50pwNSIOAM4h2ThvgOBA0j2zOjv3431k6t0tgFJlwGHkdQ2DkiLfxURz6b3DwOujoi1wFOSbif5T1tvddHSonLzSBIOJF9MJwNExE2SnutnuKuAG/t5TsmsdAHDhZKqLR2+A/BslXIAJM0GdgMWRcQ70uJ3S3oTMBF4X7qPRcnMiPjKRsZaNYSy++OAa9Mv4k1J1qSqptZxdwFfkfQj4PqI+JukN5Ns3DQ/PWYLYA/g6Sqv+zSwc5XyucDlaeKdFREL0t/PbRHxBEDF39Vb07JbJH2vVMMBfhoRL6f33wy8VlIpyWwN7E5xNg5rCtcsDGAR8LrSg4j4MMnid2PLjnmx7H6tpqI19P6b2qzi+VfSn2vpfaEykDVnVlasWFseQ+X7V3ql7H61z7Sy4jUqf08nAGeTtOGXXBURk4A3Al8tNfP0YRGwf51japlCz85slwGXRsRk4EPU/uxVj4uILwAfJEkIcyXtTvI7+UJZH8trIuJ7NV53M5LfVy8RcSvJ7+JPwFWS3p2+brV/88p/g/LHlX9/HyqLa0JE/KZGXDZInCwM4FZgM0n/WFY2po/j7wDeqWTP4bEktYN7gSeAvSRtKmlrkoRTzx2kbd2Sjge22ZgPUOZxer58315W/gLJwoWZpf0Gm0naJC26EnijpLeWHVb19xQRdwJXk/RJ9OVrwNmSOiEZbCDpzPT3WpOk/Uj2QbksLdoa6JYk4MyyQys/d9XjJO2W9r9cTFKTmEiyAOLZZf0XuygZqVbtd7kHvZvcSq/7KuDPEXEFSbPSFJJazFHpc0gqJdvyv4VjgKUR8WLla6ZxfUhpZ7eSEWqjqxxng8jNUEZEhJJhpJdK+iSwnORK7lM1TrmRpA9jAckV4idLe29IuhZYCPwvPc0XfbkQuFrSfcDtDLwp4QLgW5L+TJLASn4K/ETSycCH+/F6vwEOAX4bES9Jmgr8p6SvAU+RNL39e41zLwHuLesEnq60wz81NSKelPQuklrIdiS/z98C11Z5vSMlzSdJUE+RXF3fXva5byTZQ+NeejrWKz93reM+IekNJDu9LQRuiYhVSgYi3J3kFl4gWWL9cUldkh4Abkr7LY4Eqg2fPRr4uKTVwN+A90TEU+mFyU/TpLWMpLnrcyQd2AvTY99X4/d6OUlz2v1pXE/T4tvVDgVeddasD5IOIPlSrvXFVXjpF/7vgLdGxPPNjsfy4WYosz5ExFzgTqWT8qyqHYAvOVG0NtcszMysLl8tmZlZXU4WZmZWl5OFmZnV5WRhZmZ1OVmYmVldThZmZlbX/wcbGuxlgTePiQAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x432 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from numpy.polynomial.polynomial import polyfit\n",
"import matplotlib.pyplot as plt\n",
"x = grec_m2['score'].astype('float64')\n",
"y = grec_m2['path'].astype('float64')\n",
"\n",
"fig, ax = plt.subplots(1,1,figsize=(6,6))\n",
"bb, m = polyfit(x, y, 1)\n",
"\n",
"ax.scatter(x,y)\n",
"ax.plot(x, bb + m * x, 'r--')\n",
"ax.set_ylabel(\"Computational Fact Checker\")\n",
"ax.set_xlabel(\"Ground Truth (GREC Dataset) score\")\n",
"fig.savefig('figures/pob_correlation2.pdf', format='pdf')"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAF3CAYAAABKeVdaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XecVNXdx/HPj65SVMAGIiAlgsGNIrFFUCwEC9gQC3ZRjDEa46NPLEE0MVEjaqyrMcYCWFFEDCJFjA8IBFERoyAaXSBqBJTQYX/PH2cW12V3dnZ37twp3/frNa+duXNn5jcDO98959x7jrk7IiIiAPXiLkBERLKHQkFERLZQKIiIyBYKBRER2UKhICIiWygURERkC4WCiIhsoVAQEZEtFAoiIrKFQkFERLZoEHcBNdWqVStv37593GWIiOSUf/zjH/9x99bV7ZdzodC+fXvmzJkTdxkiIjnFzP6Vyn6RdR+Z2SNm9qWZza/ifjOzu81skZm9a2b7RlWLiIikJsoxhUeBfknu/ynQOXEZCtwfYS0iIpKCyELB3acDy5PsMgB4zIOZwPZmtmtU9YiISPXiHFNoA3xe7nZJYtuymj7Rxo0bKSkpYd26demqTarRpEkT2rZtS8OGDeMuRSSnFBfDqFHpf949Vi+g7Y9a89viaseSk4ozFKySbZWu+GNmQwldTLRr126r+0tKSmjWrBnt27fHrLKnlXRyd77++mtKSkro0KFD3OWI1FpUX9DJvP56+Nm7d3qfd1WDHThyzi3AHXV6njhDoQTYvdzttsDSynZ092KgGKBnz55bBce6desUCBlkZrRs2ZKvvvoq7lJEqpXsiz+qL+hkeveG00+HoUPT9IQffwx77AENdqWugQDxhsI44FIzGwP8GPjG3WvcdVRGgZBZ+rwlV4waBfPmQVHR1vel/Qs60+bNg759YcgQuPPOtDxllIekjgZmAF3NrMTMzjezi83s4sQuE4DFwCLgIeCSqGrJlLFjx2Jm/POf/9yybdq0aRx77LHf2++cc87h2WefBcJ4yDXXXEPnzp3Ze++96dWrF6+88kqda7nlllvo1KkTXbt2ZeLEiZXu85Of/ISioiKKiorYbbfdGDhwIBC6hy677DI6depEjx49mDt3bp3rEYlScTH06VP5pSwQpk2r/JKzgTB3Lhx+ODRtCpddlranjayl4O6nVXO/Az+L6vXjMHr0aA455BDGjBnD8OHDU3rM9ddfz7Jly5g/fz6NGzfmiy++4PWyNm0tLViwgDFjxvD++++zdOlSjjjiCD766CPq16//vf3eeOONLddPOukkBgwYAMArr7zCwoULWbhwIW+99RbDhg3jrbfeqlNNIlFK1hooKgqtgbzyj3/AkUdC8+YwdSqkcWwv585ozlb//e9/efPNN5k6dSrHH398SqGwZs0aHnroIT755BMaN24MwM4778ygQYPqVMuLL77I4MGDady4MR06dKBTp07MmjWLAw88sNL9V61axZQpU/jLX/6y5fFnnXUWZsYBBxzAypUrWbZsGbvuqiOGpW6iGtgt3xrIexs2wAknhECYNg3SPO1P3oXC5ZeH/yDpVFRUfXfdCy+8QL9+/ejSpQs77rgjc+fOZd99k5+kvWjRItq1a0fz5s2rreGKK65g6tSpW20fPHgw11xzzfe2LVmyhAMOOGDL7bZt27JkyZIqn3vs2LH07dt3Sx1Llixh992/Owag7PEKBamrZH/R10Vetgaq0qgRjBkDu+2W9kCAPAyFuIwePZrLL78cCF/Uo0ePZt99961yQLamA7UjR45Med/QM5f6640ePZoLLrig1o8XqYmC+Ys+3WbNCt1Gw4bBQQdF9jJ5FwppGoCvka+//popU6Ywf/58zIzNmzdjZtx66620bNmSFStWfG//5cuX06pVKzp16sRnn33GqlWraNasWdLXqElLoW3btnz++XfnBZaUlLDbbrtVWfusWbMYO3ZsrR4vIhnw1ltw1FHQujWcdRZst110r+XuOXXZb7/9vKIFCxZstS2THnjgAR86dOj3th166KE+ffp0X7dunbdv335LjZ9++qm3a9fOV65c6e7uV111lZ9zzjm+fv16d3dfunSpP/7443WqZ/78+d6jRw9ft26dL1682Dt06OCbNm2qdN/777/fzzrrrO9tGz9+vPfr189LS0t9xowZvv/++1f62Lg/d8k9vXuHi9TAjBnuzZu777mn+2ef1fppgDmewnesFtlJg9GjR3PCCSd8b9tJJ53EqFGjaNy4MU888QTnnnsuRUVFnHzyyTz88MO0aNECgJtvvpnWrVvTrVs39t57bwYOHEjr1nU7Tb179+4MGjSIbt260a9fP+69994tRx7179+fpUu/O0dwzJgxnHba9w8U69+/Px07dqRTp05ceOGF3HfffXWqR0RqacaM0ELYaafQ57b77tU+pK7MK+k/zmY9e/b0iuspfPDBB+y1114xVVS49LlLTfXpE35qTCFF998PI0eGw07btKnTU5nZP9y9Z3X7qaUgIpJt1q4NP4cNC4dr1TEQakKhICKSTd54A/bcE2bODLe33TajL69QEBHJFtOnw09/Ci1ahEnuYpA3oZBrYyO5Tp+3SJq9/jr07x8Gk6dOhZhOFs2LUGjSpAlff/21vqgyxBPrKTRp0iTuUkTyw3vvhUDYY48wCr/LLrGVkhcnr7Vt25aSkhLN759BZSuviUga7LVXmOn08sth551jLSUvQqFhw4ZaAUxEcs/06dC5c+gquuWWuKsB8qT7SEQk57z2Ghx9dFrXQkgHhYKISKZNmgTHHQddukCWzRigUBARyaSJE0MgdO0KkyeHSe6yiEJBRCRTSkvhuuvCwPLkydCqVdwVbSUvBppFRHJCvXowfjw0aAAtW8ZdTaXUUhARidqECXDaabBxYzjkNEsDARQKIiLRGj8+rKm8cCGsWRN3NdVSKIiIROWll+DEE6FHj3AIamIdlWymUBARicK4cXDSSWFR6kmTYPvt464oJQoFEZEo7LxzWFUohwIBFAoiIum1aFH4+eMfw6uv5kSXUXkKBRGRdHnuuXAOwhNPxF1JrSkURETS4dln4dRTYf/94fjj466m1hQKIiJ19cwzMHgwHHBAmMaiefO4K6o1hYKISF188gmcfjoceCC88go0axZ3RXWiaS5EROqiQwd4+mk48kho2jTuaupMoSAiaVNcDKNGVX3/vHnhsP28MGZMOOz0sMPCGct5Qt1HIpI2o0aFL/6qFBWFnpac9+STcMYZcPvtcVeSdmopiEhaFRWFtefz1hNPwNlnQ+/eodsoz6ilICKSqsceg7POCmcqjx8P220Xd0Vpp1AQEUnV1Klw+OFhorttt427mkio+0hEaiTZYHJeDSSXt24dNGkCDz8MGzbANtvEXVFk1FIQkRpJNpicNwPJ5f35z2Hq62XLoH79vA4EUEtBRGoh7weTyzz8MFx4IRx9NOywQ9zVZIRCQaQAVXc+QTJ520VUUXExXHQR9OsHY8eG7qMCoO4jkQJU3fkEyeRlF1FFTz8dAqF//4IKBFBLQaRgFUwXUG307QtXXAG33AKNG8ddTUappSAiUmb8eFi/Hlq2hDvuKLhAAIWCiEhwzz1w3HEwcmTclcRK3Ucieaogzyeorbvvhl/8AgYMgF/+Mu5qYqWWgkieKrjzCWrrzjtDIAwcGAaYGzWKu6JYqaUgksc0mFyN//wHRoyAE08MU2E3bBh3RbFTKIhI4WrVCmbMgI4dFQgJCgWRmNXlRLJkNG6QxO23h6OMrr0WunaNu5qsojEFkZjV5USyZDRuUIVbb4WrroJ334XS0riryTpqKYhkAfX9Z8jvfw//+78weDA8/jjU09/FFekTEZHCUBYIp50WAqGB/iaujEJBRArDTjvBkCFh9TQFQpUUCiKS3xYvDj/POw/++lcFQjUUCiKSv0aMgO7d4Z13wm2zeOvJAQoFEclPw4fDb34Dp54Ke+8ddzU5I9JQMLN+ZvahmS0ys2squb+dmU01s7fN7F0z6x9lPSJSANxDGNx4I5x7blhOs379uKvKGZF1rplZfeBe4EigBJhtZuPcfUG53a4Dnnb3+82sGzABaB9VTSJR0UpmWWTcuNBtdN558NBDOuy0hqL8tHoBi9x9sbtvAMYAAyrs40DzxPUWwNII6xGJjFYyyyLHHReOMFIg1EqUw/BtgM/L3S4Bflxhn+HAq2b2c2A74IgI6xGJlE5Ai5F7mLrilFOgfftw6KnUSpQxWtkwv1e4fRrwqLu3BfoDj5vZVjWZ2VAzm2Nmc7766qsIShWRnOUeTkr7n/+BRx6Ju5qcF2UolAC7l7vdlq27h84HngZw9xlAE6BVxSdy92J37+nuPVu3bh1RuSKSc9zh6qvhD3+AYcPCEUdSJ1GGwmygs5l1MLNGwGBgXIV9PgP6ApjZXoRQUFNARKrnHloHt90Gl1wC996rMYQ0iOwTdPdNwKXAROADwlFG75vZCDM7PrHblcCFZvYOMBo4x90rdjGJiGxt7dowiPOzn4X1lXViWlpEer63u08gHGZaftsN5a4vAA6OsgYRyTPusHEjbLstTJ0K222nQEgjtbVEJHe4w+WXw/HHw4YN0LSpAiHNFAoikhvc4Re/gLvvhr320vKZEdF0gSKS/dzh5z8Pg8lXXAF//KNaCBFRS0FEst+vfx0C4corFQgRU0tBRLLfaafBNtvA9dcrECKmUBCR7FRaCuPHh7mMevQIF4mcuo9EJPuUlsLFF8OAATB5ctzVFBS1FEQku5SWwkUXwcMPh7GEvn3jrqigqKUgItmjtBQuvDAEwnXXwc03awwhwxQKIpI9Zs6ERx+FG24IC+UoEDJO3Ucikj0OOgjefluDyjFSS0FE4rV5c+gyevnlcFuBECuFgojEZ/NmOOecMIbw7rtxVyOo+0hE4rJpE5x9dljg+uabw+ppEjuFgohk3ubNcNZZMHo0/O53CoQsolAQkcyrVw922AF+//uwnKZkDYWCSIqKi0NPR2XmzYOioszWk5M2bYJ//xvattVqaVlKA80iKRo1Knz5V6aoCE4/PbP15JyNG8OHdOCB8M03CoQspZaCFJxkf/EnU9YamDYt7SXlv7JAePbZMPV1ixZxVyRVUChI3qnuS//118PP3r1r9rxqDdTSxo1h6uvnnoM77giL5EjWUihI3inr5qmqj7937/DlPnRoZusqWDfdFALhzjvDcpqS1RQKkpfUzZNFfvUr6N4dTj017kokBRpoFpH027ABbrwR1qyB5s0VCDlEoSAi6bV+PZx8MgwfDhMnxl2N1JC6j0Qkfdavh5NOCpPb3XcfnHBC3BVJDSkURCQ91q0LgTBhAjzwQFg9TXKOQkFE0mPpUpg7Fx58UId25TCFguQkTTmRRTZsgIYNoWNH+PDDMLAsOUsDzZKTNOVElli7Fo477rtZThUIOU8tBclZOhchZmvXwoAB8NprMHhw3NVImigURKTm1qwJgTB5MjzySFg9TfKCQkFEasY9HGU0eTI8+mhYLEfyhkJBRGrGDM47LwzcDBkSdzWSZgoFEUnN6tUwezb06QOnnBJ3NRIRHX0kItVbvRqOOQZ++lNYtizuaiRCailI1tK5CFniv/8NgfD3v8OTT8Kuu8ZdkURIoSCRqu0qZ5B8MRydi5Ahq1ZB//4wY0b4h9Rsp3lPoSB1luyLv7arnJU9RovhxOyxx0IgjB6tcYQCoVCQOku20pm+2HPcJZfAwQerr66AKBQkLXR2cR759ttwMtott0DXrgqEAqOjj0TkO998A0cfDS+9BB99FHc1EgO1FEQkWLkyBMLcufDMM2GiOyk4CgURCYFw1FFhcOjZZ8O8RlKQUuo+MrNGZtYp6mJEJCYNGkCLFvDccwqEAldtS8HMjgHuABoBHcysCPiNu2vxVZFct2JFCIRmzeDVV8O8RlLQUmkpjAB+DKwEcPd5gFoNIrlu+XI44gg48cQw86kCQUhtTGGju6+07/+H8YjqEZFM+PrrEAgffABjxyoQZItUQuEDMxsE1DOzDsAvgJnRliUikSkfCC+8AP36xV2RZJFUuo8uBfYDSoHngXWEYBCRXHTWWSEQXnxRgSBbSaWlcLS7Xw1cXbbBzE4kBISI5JqRI+Gzz0JrQaSCVFoK11Wy7dp0FyIiEfrqK7jttjCg3KWLAkGqVGVLwcyOBvoBbczsjnJ3NSd0JYlILvjySzj8cFi8GI4/PsxnJFKFZN1HXwLzCWMI75fbvgq4JsqiRCRNvvgiBMInn8D48QoEqVaVoeDubwNvm9mT7r4ugzVJlqluoRytgpalygfCyy/DYYfFXZHkgFTGFNqY2Rgze9fMPiq7RF6ZZI2y9RKqolXQstQ778DSpTBhggJBUpbK0UePAjcDtwM/Bc4lxTEFM+sH3AXUBx52999Xss8gYDjhhLh33F1fL1lI6yXkkI0boWHDMMHdJ5/A9tvHXZHkkFRaCtu6+0QAd//Y3a8Dqv2zw8zqA/cSgqQbcJqZdauwT2fgf4GD3b07cHkN6xeR8pYuhR/9KEx9DQoEqbFUWgrrLcxx8bGZXQwsAXZK4XG9gEXuvhjAzMYAA4AF5fa5ELjX3VcAuPuXNSleRMpZujR0Ey1dCrvuGnc1kqNSaSlcATQFLgMOJnyRn5fC49oAn5e7XZLYVl4XoIuZvWlmMxPdTSJSU0uWQJ8+IRD+9jc45JC4K5IcVW1Lwd3fSlxdBQwBMLO2KTx3ZTNsVZxIrwHQGegDtAXeMLO93X3l957IbCgwFKBdu3YpvLRIAVm5MgTCF1/AxIlw0EFxVyQ5LGlLwcz2N7OBZtYqcbu7mT1GahPilQC7l7vdFlhayT4vuvtGd/8E+JAQEt/j7sXu3tPde7Zu3TqFlxYpIC1awJlnKhAkLaoMBTO7BXgSOAP4m5ldC0wF3iF0+1RnNtDZzDqYWSNgMDCuwj4vkBi0TgRPF2BxTd+ESEH67DN4//0w7fVvfgMHHhh3RZIHknUfDQD2cfe1ZrYj4a/8fdz9w1Se2N03mdmlwETCIamPuPv7ZjYCmOPu4xL3HWVmC4DNwFXu/nVd3pBIQfjXv8KgcoMGsGBB+CmSBsn+J61z97UA7r7czP6ZaiCUcfcJwIQK224od92BXyYuIpKKTz8NgbBiBUyapECQtEr2v6mjmZVNj21A+3K3cfcTI61MRLb26adhUPmbb+C116Bnz7grkjyTLBROqnD7nigLEZEUDB8O334bAmG//eKuRvJQsgnxJmeyEBFJwX33wVVXQffucVcieSqVk9dEJE4ffwwnnxy6jLbdVoEgkdIIlUg2W7QoDCqvWQMlJeGcBJEIpdxSMLPGURYiIhUsXBgGldeuhSlT1EKQjKg2FMysl5m9ByxM3N7HzP4UeWUiheyjj0IgrF8fAmGffeKuSApEKi2Fu4Fjga8B3P0dUpg6W0TqoEGDMNPplCnQo0fc1UgBSWVMoZ67/yvMnr3F5ojqESlsS5fCLrtAx44we3aYwkIkg1JpKXxuZr0AN7P6ZnY5oOU4RdLtgw/CuQfXXhtuKxAkBqmEwjDCNBTtgC+AAxLbRCRdFiwIRxm5w5AhcVcjBSyV7qNN7j448kpEClVZINSrB1Onwg9+EHdFUsBSaSnMNrMJZna2mTWLvCKRQrJ+PfTrp0CQrJHKymt7mtlBhPUQbjSzecAYdx8TeXWSVsXFMGpUzR83bx4UFaW/HgEaN4aHHoL27aFr17irEUnt5DV3/z93vwzYF/iWsPiO5JhRo8IXfE0VFcHpp6e/noL23nswenS4fvTRCgTJGtW2FMysKWHBncHAXsCLgNb8y1FFRTBtWtxVFLh33oG+fWG77WDAgDCfkUiWSGWgeT7wEnCru78RcT0i+a0sELbZBiZPViBI1kklFDq6e2nklYjku3nzvmshTJ0Ke+4Zd0UiW6kyFMzsj+5+JfCcmXnF+7XymkgNTZwITZuGQOjYMe5qRCqVrKXwVOKnVlwTqYtNm8JcRldfDUOHwg47xF2RSJWqPPrI3Wclru7l7pPLXwgDziJSnTlzYK+9wlgCKBAk66VySOp5lWw7P92FiOSd2bPhyCNDS2H77eOuRiQlycYUTiUchtrBzJ4vd1czYGXUhYnktFmz4KijYMcdwxjCHnvEXZFISpKNKcwirKHQFri33PZVwNtRFiWS0+bPDy2EVq1CILRrF3dFIimrMhTc/RPgE+C1zJUjkgc6dYLBg+G662D33eOuRqRGknUfve7uvc1sBVD+kFQD3N13jLw6kVwyZ0441HTHHeHBB+OuRqRWkg00ly252QpoXe5SdltEyrz5Zpj+epiWGpHcluyQ1LKzmHcH6rv7ZuBA4CJguwzUJpIb/v73MP31rrvCHXfEXY1InaRySOoLhKU49wQeI5yjUIsJmEXy0BtvhEDYbbcw02CbNnFXJFInqYRCqbtvBE4E7nT3nwP6ny9SWgo/+xm0bRsCYbfd4q5IpM5SWo7TzE4BhgADE9saRleSSI6oVw9eegkaNQpdRyJ5INUzmg8jTJ292Mw6AKOjLUski02dGloIpaXhpDQFguSRVJbjnG9mlwGdzOwHwCJ3/230pYlkoSlT4NhjoUMH+OYbzWUkeSeVldd+AjwOLCGco7CLmQ1x9zejLk4kq0yeDMcdF85FmDJFgSB5KZUxhZFAf3dfAGBmexFComeUhYlklddeC4HQuXMIh9Y6VUfyUypjCo3KAgHA3T8AGkVXkkiW2mcfBYLkvVRaCnPN7EFC6wDgDDQhXlYqLoZRSc4gmTcPiooyV09eWLYsDCQfcURYStMs7opEIpVKS+Fi4GPgf4CrgcWEs5oly4waFb74q1JUBKefnrl6ct7EiWEd5eeeC7cVCFIAkrYUzOyHwJ7AWHe/NTMlSV0UFYXzqKSOXnkFTjgBunWDPn3irkYkY6psKZjZrwlTXJwBTDKzylZgE8k/EybAwIHQvXsYYG7ZMu6KRDImWUvhDKCHu682s9bABOCRzJQlEpOPPw4thB/+ECZN0mGnUnCShcJ6d18N4O5fmVkq4w8SsWSDyRpIToM994QHHggtBQWCFKBkodCx3NrMBuxZfq1mdz8x0sqkUmWDyZV9+WsguQ5eeikcZdSzJ5x7btzViMQmWSicVOH2PVEWIqnTYHKavfACDBoEhx8Of/tb3NWIxCrZGs2TM1mISCzGjg2BsN9+8NRTcVcjEjuNE0jhev75EAj77w+vvgotWsRdkUjsFApSmNzDAE2vXqHLqHnzuCsSyQqpTHMBgJk1dvf1URYjkhGbN0P9+vDkk7BhAzRrFndFIlkjlamzewF/BloA7cxsH+CCxLKcEgEddhqhp56CW28N3UUtW0LjxnFXJJJVUuk+uhs4FvgawN3fIazEJhFJNoeRDjutgzFj4IwzYLvtFAYiVUil+6ieu//Lvj8Z2OaI6pEEHXaaZqNGwZAhcMgh8PLL0LRp3BWJZKVUQuHzRBeSm1l94OfAR9GWJZJGY8eGQDj0UBg/PrQURKRSqXQfDQN+CbQDvgAOSGwTyQ29esE55ygQRFJQbUvB3b8EBmegFpH0mj4dDj4Y2rSBP/857mpEckIqRx89BHjF7e4+NJKKRNLhr38Ncxj94Q9w1VVxVyOSM1LpPnoNmJy4vAnsBKR0voKZ9TOzD81skZldk2S/k83MzaxnKs8rktRf/hIC4Ygj4NJL465GJKek0n30vQlhzOxxYFJ1j0sMSt8LHAmUALPNbJy7L6iwXzPgMuCtGtQtUrlHHoELLoAjjwwT3W2zTdwVieSU2kxz0QHYI4X9egGL3H2xu28AxgADKtnvJuBWYF0tahH5zldfwS9+AUcfDS++qEAQqYVUxhRW8N2YQj1gOVBlV1A5bYDPy90uAX5c4bl/BOzu7uPN7FcpVSxSldat4fXXw7rKTZrEXY1ITkoaChbOWNsHWJLYVOruWw06V/XwSrZteWxiJbeRwDnVPpHZUGAoQLt27VJ8eSkYxcWwaRNccgnsu2/c1YjktKTdR4kAGOvumxOXVAMBQstg93K32wJLy91uBuwNTDOzTwnnP4yrbLDZ3Yvdvae792zdunUNSpC898ADcNFFYabT0tK4qxHJeamMKcwys9r8+TUb6GxmHcysEeFch3Fld7r7N+7eyt3bu3t7YCZwvLvPqcVrSSG67z4YNgyOOw6eeQbqaSZ4kbqqsvvIzBq4+ybgEOBCM/sYWE3oFnJ3TxoU7r7JzC4FJgL1gUfc/X0zGwHMcfdxyR4vktS994bDTQcMgKefhkaN4q5IJC8kG1OYBewLDKztk7v7BGBChW03VLFvn9q+jhSgDRtg4MAwFbYCQSRtkoWCAbj7xxmqRaR6X3wBO+8MV1wRDj9Vl5FIWiULhdZm9suq7nT3OyKoR6RqI0fC8OEwY0Y47FSBIJJ2yUKhPtCUyg8tFcmsO+6AK6+Ek0+Gzp3jrkYkbyULhWXuPiJjlYhU5fbbw6R2p5wS1lVu2DDuikTyVrL2t1oIEr8XXwyBMGhQWD1NgSASqWSh0DdjVYhUpX9/uOuu0EJokMpCgSJSF1WGgrsvz2QhIt9TXByONGrYEC67TIEgkiE6fEOyz29/G6auuOuuuCsRKTgKBckuN90E110HZ54ZrotIRikUJHuMGAE33ABDhsCjj0L9+nFXJFJwFAqSHVavhtGj4eyzw3KaCgSRWGj0TuLlHqa83m47+PvfYfvtFQgiMVJLQeLjHrqLTjstLJLTsqUCQSRmCgWJhztcfz3cfDM0b655jESyhLqPJPPc4dpr4ZZb4IIL4MEHFQoiWUK/iZJ5I0aEQBg6VIEgkmXUUpDMO/JI+PZbuO02BYJIltFvpGSGO7zxRrh+0EHwxz8qEESykH4rJXruYabTQw+FqVPjrkZEklD3UQyKi8Ms0FWZNw+KijJXT6Tcw+I4I0fCz38OffrEXZGIJKGWQgxGjQpf/FUpKoLTT89cPZFxD2spjxwZ1lO+6y4wLdMhks3UUohJURFMmxZ3FRH7+99DEFx+eVhOU4EgkvUUChKdn/wkDC4ffLACQSRHqPtI0ssdfvUrmD493D7kEAWCSA5RS0HSp7QULr3T9Jl1AAARAElEQVQU7r8fttkmHG0kIjlFoSDpUVoKl1wSzlC++upw1rKI5ByFgtRdaSlcfDE89BBccw387nfqMhLJURpTkLpzD4vk/PrXCgSRHKeWgtReaSmsWBHWQXjssTBthQJBJKeppSC1s3kznH8+HHhgmNyufn0FgkgeUChIzZUFwqOPhlOvmzePuyIRSRN1H0nNbN4M554Ljz8Ow4fDb34Td0UikkYKBamZESNCINx4Y1hfWUTyikJBaubSS6FNm7BqmojkHY0pSPU2bYI774QNG6B1awWCSB5TKEhymzbBkCFhCuyXXoq7GhGJmLqPpGqbNsEZZ8DTT8Mf/gAnnRR3RSISMYVCRJKtrpYTK6tt3BgC4Zln4NZbw3KaIpL31H0UkWSrq+XEymqLF8Orr8LttysQRAqIWgoRysnV1UpLw3QVXbvCRx/BTjvFXZGIZJBaCvKdDRvg5JPD+AEoEEQKkEJBgg0b4NRTYezYsECOiBQkdR9JCIRTToFx4+BPfwonqIlIQVIoFDp3GDQoBMI998DPfhZ3RSISI4VCoTODY46Bo4+GYcPirkZEYqZQKFTr1sH8+dCzJ1x4YdzViEiW0EBzIVq3Dk48EXr3hn//O+5qRCSLqKVQaNatg4EDYeJEeOgh2GWXuCsSkSyiUCgka9eGQJg0CR5+OKyeJiJSjkKhkDzwwHeBcN55cVcjIllIoVBILrsM9tsPDj007kpEJEtpoDnfrVkTWgWffQb16ysQRCQphUI+W70ajj0WHn0UZs2KuxoRyQHqPspXZYEwfTo89liY6E5EpBoKhXy0enU4S/mNN+Dxx3Ng8QYRyRaRdh+ZWT8z+9DMFpnZNZXc/0szW2Bm75rZZDPbI8p6Csb69WEs4YknFAgiUiORtRTMrD5wL3AkUALMNrNx7r6g3G5vAz3dfY2ZDQNuBU6Nqqa899//QsOGsOOOMGNGGFgWEamBKFsKvYBF7r7Y3TcAY4AB5Xdw96nuviZxcybQNsJ68tuqVdCvX1gTwV2BICK1EmUotAE+L3e7JLGtKucDr0RYT/769tsQCDNnwplnhplPRURqIcqB5sq+mbzSHc3OBHoCvau4fygwFKBdu3bpqq/Oioth1KjK75s3L6zRHLmyQJg9G556Ck46KQMvKiL5KsqWQgmwe7nbbYGlFXcysyOAa4Hj3X19ZU/k7sXu3tPde7Zu3TqSYmtj1Kjw5V+ZoqIMjfGefroCQUTSJsqWwmygs5l1AJYAg4HvfU2a2Y+AB4F+7v5lhLVEpqgIpk2LsYCbbgrrIQwYUP2+IiLViKyl4O6bgEuBicAHwNPu/r6ZjTCz4xO73QY0BZ4xs3lmNi6qevLKypXw5z+H6z/6kQJBRNIm0pPX3H0CMKHCthvKXT8iytfPSytXwlFHhX6rn/wEunSJuyIRySOa+yiXrFgBRx4J77wDzz+vQBCRtNM0F0kkO7oIMniEEcDy5SEQ5s8PgXDMMRl6YREpJGopJJHs6CLI4BFGEOYx+uADeOEFBYKIREYthWrEfnRRaSnUqxcGkxcv1prKIhKpgg+FrDgBrSr/+U+Y/nr48HCCmgJBRCJW8N1HWXECWmX+8x/o2zcMKmseIxHJkIJvKUAWdBFV9NVXIRAWLoSXXoIjdOSuiGSGQiHbfPMNHH44fPwxjB8fwkFEJEMKvvso6zRrFkJBgSAiMVBLIVt88UVYLa1DB7jrrrirEZECpVDIBv/+d2gdmMG772pgWURio1CI27JlIRA+/xwmTFAgiEisFApxWrYMDjsMSkrglVfCBHciIjFSKMTpqqtgyRL429/gkEPirkZEREcfxeqee2DqVAWCiGQNhUKmLVkCQ4fC2rWw/fbQs2fcFYmIbKFQyKTPP4fevcN6ygsXxl2NiMhWFAqZ8tln0KdPmMLi1VehR4+4KxIR2YoGmjOhLBCWL4dJk6BXr7grEhGplEIhE1atggYNQiDsv3/c1YiIVEmhEKXly2GHHaB7d1iwIASDiEgW05hCVD75BPbdF373u3BbgSAiOUChEIXFi8MYwqpV0L9/3NWIiKRMf76m28cfh6krVq+GyZNjXs9TRKRmFArptG5dWCVtzRqYMgX22SfuikREakShkE5NmsCtt0KXLgoEEclJCoV0WLgwXPr3h1NOibsaEZFaUyjU1UcfhUHlevVCMGyzTdwViYjUmkKhLj78MAwqb94cTkxTIIhIjlMo1NY//xkCobQ0TH/drVvcFYmI1JnOU6itUaPAXYEgInlFoVBT7uHnjTfC3LkKBBHJKwqFmnj/fdhvvzCgbAa77RZ3RSIiaaUxhVTNnw+HHx7mMCotjbsaEZFIqKWQivfeC4PKDRvCtGnQtWvcFYmIRKIgWgrFxWFcuDLz5lUzPdEHH4QWQuPGYVC5c+dIahQRyQYF0VIYNSp8+VemqAhOPz3Jg9u2Da2EadMUCCKS9wqipQDhy3/atBo84IMPoF07aNYMnn46qrJERLJKQbQUauztt+GQQ+Cii+KuREQkoxQKFc2dC337QtOmMGJE3NWIiGSUQqG8f/wjBELz5qGvqWPHuCsSEckohUKZzZthyBDYfvsQCB06xF2RiEjGFcxAc7Xq14fnnw8zne6xR9zViIjEQi2Ft96C668Pcxr94AcKBBEpaIUdCjNnwlFHhRMZVqyIuxoRkdgVbijMmBECoXVreP112HHHuCsSEYldYYbC//1fCISddw6Dym3bxl2RiEhWKMxQ+OKLcLayAkFE5HsKKxRWrgw/TzghTIbUpk289YiIZJmCCYUeK6eHcw9efjlsaNgw3oJERLJQ3pynkGx67O3mvM4f1vaHLu3CymkiIlKpvGkpVDU9dtHKaTy7tj/rdt4jrIewyy6ZL05EJEfkTUsBKpkee9Ei6NEfftCBbaZMCUcbiYhIlfKmpVCpPfeEm24CBYKISEryMxSmTAmL5JjBlVcqEEREUhRpKJhZPzP70MwWmdk1ldzf2MyeStz/lpm1r/OLTpoExxwDl19e56cSESk0kYWCmdUH7gV+CnQDTjOzbhV2Ox9Y4e6dgJHAH+rymvsvnwjHHQddusCTT9blqUREClKULYVewCJ3X+zuG4AxwIAK+wwA/pq4/izQ18ysVi+2/G/8dv4A2Guv0H3UqlWtCxcRKVRRhkIb4PNyt0sS2yrdx903Ad8ALWv8Su6c++1dfNGqG7z2GrSs+VOIiEi0h6RW9he/12IfzGwoMDRx879m9mEVr9mKVq3+k3qJeacVoPdf2Ar9M9D7r/r9p7RYTJShUALsXu52W2BpFfuUmFkDoAWwvOITuXsxUFzdC5rZHHfvWeuKc5zef2G/f9BnoPdf9/cfZffRbKCzmXUws0bAYGBchX3GAWcnrp8MTHH3rVoKIiKSGZG1FNx9k5ldCkwE6gOPuPv7ZjYCmOPu44A/A4+b2SJCC2FwVPWIiEj1Ip3mwt0nABMqbLuh3PV1wClpfMlqu5jynN6/FPpnoPdfR6beGhERKZOf01yIiEit5FwoxDJ1RpZJ4TP4pZktMLN3zWyymaV0KFquqO79l9vvZDNzM8uro1FSef9mNijxf+B9M6tipZHclcLvQDszm2pmbyd+D/rHUWdUzOwRM/vSzOZXcb+Z2d2Jz+ddM9s35Sd395y5EAasPwY6Ao2Ad4BuFfa5BHggcX0w8FTcdcfwGRwGbJu4PiyfPoNU3n9iv2bAdGAm0DPuujP8798ZeBvYIXF7p7jrjuEzKAaGJa53Az6Nu+40fwaHAvsC86u4vz/wCuFcsAOAt1J97lxrKWR06owsVe1n4O5T3X1N4uZMwjki+SKV/wMANwG3AusyWVwGpPL+LwTudfcVAO7+ZYZrjFoqn4EDzRPXW7D1OVI5zd2nU8k5XeUMAB7zYCawvZntmspz51ooZG7qjOyVymdQ3vmEvxjyRbXv38x+BOzu7uMzWViGpPLv3wXoYmZvmtlMM+uXseoyI5XPYDhwppmVEI6A/HlmSssaNf2e2CLXVl5L29QZOSzl92dmZwI9gd6RVpRZSd+/mdUjzLh7TqYKyrBU/v0bELqQ+hBaiW+Y2d7uvjLi2jIllc/gNOBRd/+jmR1IOB9qb3cvjb68rFDr78FcaynUZOoMkk2dkcNS+QwwsyOAa4Hj3X19hmrLhOrefzNgb2CamX1K6E8dl0eDzan+Drzo7hvd/RPgQ0JI5ItUPoPzgacB3H0G0IQwL1ChSOl7ojK5FgqaOiOFzyDRffIgIRDyrT856ft392/cvZW7t3f39oQxlePdfU485aZdKr8DLxAONsDMWhG6kxZntMpopfIZfAb0BTCzvQih8FVGq4zXOOCsxFFIBwDfuPuyVB6YU91HrqkzUv0MbgOaAs8kxtg/c/fjYys6jVJ8/3krxfc/ETjKzBYAm4Gr3P3r+KpOrxQ/gyuBh8zsCkK3yTn59MehmY0mdA+2Soyb/AZoCODuDxDGUfoDi4A1wLkpP3cefU4iIlJHudZ9JCIiEVIoiIjIFgoFERHZQqEgIiJbKBRERGQLhYJkHTPbbGbzyl3aJ9m3fVUzRdbwNaclZt18JzE9RNdaPMfFZnZW4vo5ZrZbufseNrNuaa5ztpkVpfCYy81s27q+thQGhYJko7XuXlTu8mmGXvcMd9+HMKHibTV9sLs/4O6PJW6eA+xW7r4L3H1BWqr8rs77SK3OywGFgqREoSA5IdEieMPM5iYuB1WyT3czm5VoXbxrZp0T288st/1BM6tfzctNBzolHts3MSf/e4k57Bsntv/evluz4vbEtuFm9iszO5kw59STidfcJvEXfk8zG2Zmt5ar+Rwz+1Mt65xBuUnOzOx+M5tjYQ2FGxPbLiOE01Qzm5rYdpSZzUh8js+YWdNqXkcKiEJBstE25bqOxia2fQkc6e77AqcCd1fyuIuBu9y9iPClXJKY4uBU4ODE9s3AGdW8/nHAe2bWBHgUONXdf0iYAWCYme0InAB0d/cewM3lH+zuzwJzCH/RF7n72nJ3PwucWO72qcBTtayzH2FKizLXuntPoAfQ28x6uPvdhDlvDnP3wxLTXlwHHJH4LOcAv6zmdaSA5NQ0F1Iw1ia+GMtrCNyT6EPfTJjPp6IZwLVm1hZ43t0XmllfYD9gdmLKj20IAVOZJ81sLfApYarlrsAn7v5R4v6/Aj8D7iGs0/Cwmb0MpDxFt7t/ZWaLE/PRLEy8xpuJ561JndsRpngov6LWIDMbSvi93pWwuMy7FR57QGL7m4nXaUT43EQAhYLkjiuAL4B9CC3crRbPcfdRZvYWcAww0cwuIEwh/Fd3/98UXuOM8hPnmVml63Ak5t7pRZhwbTBwKXB4Dd7LU8Ag4J/AWHd3C9/QKddJWG3s98C9wIlm1gH4FbC/u68ws0cJk8BVZMAkdz+tBvVKAVH3keSKFsCyxHz4Qwh/JX+PmXUEFie6TMYRulEmAyeb2U6JfXa01Nes/ifQ3sw6JW4PAV5P9MG3cPcJhEHcyo4AWkWYxrsyzwMDCXP+P5XYVqM63X0joRvogETXU3NgNfCNme0M/LSKWmYCB5e9JzPb1swqa3VJgVIoSK64DzjbzGYSuo5WV7LPqcB8M5sH/ICwHOECwpfnq2b2LjCJ0LVSLXdfR5hd8hkzew8oBR4gfMGOTzzf64RWTEWPAg+UDTRXeN4VwAJgD3efldhW4zoTYxV/BH7l7u8Q1mV+H3iE0CVVphh4xcymuvtXhCOjRideZybhsxIBNEuqiIiUo5aCiIhsoVAQEZEtFAoiIrKFQkFERLZQKIiIyBYKBRER2UKhICIiWygURERki/8HpegUsGxFeIAAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x432 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from sklearn.metrics import roc_curve\n",
"from sklearn.metrics import roc_auc_score\n",
"\n",
"z = lambda x: x>0\n",
"fig, ax = plt.subplots(figsize=(6,6))\n",
"y = [z(x) for x in grec_m2['score']]\n",
"fpr, tpr, thresh = roc_curve(y, grec_m2['path'])\n",
"auc = roc_auc_score(y, grec_m2['path'])\n",
"ax.plot(fpr,tpr,'b-',label=\"AUC = {:.2f}\".format(auc))\n",
"ax.plot([0,1],[0,1],'r--')\n",
"ax.set_xlabel('False Positive Rate')\n",
"ax.set_ylabel('True Positive Rate')\n",
"#ax.tight_layout()\n",
"# remove this code to remove the border\n",
"ax.set_ylim(0, 1.02)\n",
"ax.set_xlim(-0.02, 1.02)\n",
"\n",
"ax.legend(loc=0)\n",
"fig.savefig(\"figures/grec_roc2.pdf\", bbox_inches='tight')\n",
"#fig.savefig(\"figures/Validation on Annotated Corpus/ROC_Validation_on_Annotated_Corpus.png2\", bbox_inches='tight', format='png')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Other Utilities"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"import time\n",
"from Truth import get_truth_value, get_utfidf, get_truth_value_old\n",
"def get_grec_truth_series(df):\n",
" \"\"\"\n",
" Returns 3 pandas dataframes:\n",
" - metric_tfidf\n",
" - umetric_tfidf\n",
" - metric_old\n",
" \"\"\"\n",
" metric_tfidf = df.copy()\n",
" umetric_tfidf = df.copy()\n",
" metric_old = df.copy()\n",
" metric_tfidf.is_copy=False\n",
" umetric_tfidf.is_copy=False\n",
" metric_old.is_copy=False\n",
" for idx, rowidx in enumerate(df.index):\n",
" print('>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>'+str(rowidx))\n",
" start = time.time()\n",
" print(f\"[{idx+1}/{len(df.index)}] {df.loc[rowidx, 'path']}\")\n",
" try:\n",
" metric_tfidf.loc[rowidx,\"path\"] = get_truth_value(metric_tfidf.loc[rowidx,\"path\"], display=False)\n",
" umetric_tfidf.loc[rowidx,\"path\"] = get_utfidf(umetric_tfidf.loc[rowidx,\"path\"],display=False)\n",
" metric_old.loc[rowidx, \"path\"] = get_truth_value_old(metric_old.loc[rowidx,\"path\"],display=False)\n",
" except:\n",
" print(f\"Value error occured @ {metric_tfidf.loc[rowidx,'path']}.\")\n",
" print(f\"Putting nan as value\")\n",
" metric_tfidf.loc[rowidx,\"path\"] = np.nan\n",
" umetric_tfidf.loc[rowidx,\"path\"] = np.nan\n",
" metric_old.loc[rowidx, \"path\"] = np.nan\n",
" \n",
" print(f\"Done {time.time()-start} seconds. m:{metric_tfidf.loc[rowidx]['path']}; u:{umetric_tfidf.loc[rowidx]['path']}; mo:{metric_old.loc[rowidx]['path']}\")\n",
" metric_tfidf.loc['path']= metric_tfidf['path'].astype('float64')\n",
" umetric_tfidf.loc['path']=umetric_tfidf['path'].astype('float64')\n",
" metric_old.loc['path']=metric_old['path'].astype('float64')\n",
" return (metric_tfidf.drop(\"path\", axis=0), umetric_tfidf.drop(\"path\", axis=0), metric_old.drop(\"path\", axis=0))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Other Utilities"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Save the loaded CSVs for future use. Only uncomment the variables needed."
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"# Uncomment the variables you're gonna be saving.\n",
"# CAUTION: This will override existing CSV.\n",
"\n",
"#=========================================\n",
"# Phase I: Classification Task\n",
"#=========================================\n",
"#mold.to_csv('data/out/degmatrices/i2_mold.csv', sep=';')\n",
"#mt.to_csv('data/out/degmatrices/i2_mt.csv', sep=';')\n",
"#ut.to_csv('data/out/degmatrices/i2_ut.csv', sep=';')\n",
"\n",
"#=========================================\n",
"# Phase II: Validation on Factual Data\n",
"#=========================================\n",
"# Countries/Capitals\n",
"#c_new.to_csv('data/out/degmatrices/t3_countries/c_new.csv', sep=';')\n",
"#c_u_new.to_csv('data/out/degmatrices/t3_countries/c_u_new.csv', sep=';')\n",
"#c_u_mold.to_csv('data/out/degmatrices/t3_countries/c_u_mold.csv', sep=';')\n",
"#-----------------------------------------\n",
"# Presidents/Spouses\n",
"#w_new.to_csv('data/out/degmatrices/t2_presidents/w_new.csv', sep=';')\n",
"#w_u_new.to_csv('data/out/degmatrices/t2_presidents/w_u_new.csv', sep=';')\n",
"#w_mold.to_csv('data/out/degmatrices/t2_presidents/w_mold.csv', sep=';')\n",
"#-----------------------------------------\n",
"# Director/Movie\n",
"#d_new.to_csv('data/out/degmatrices/t4_movies/d_new.csv', sep=';')\n",
"#d_u_new.to_csv('data/out/degmatrices/t4_movies/d_u_new.csv', sep=';')\n",
"#d_mold.to_csv('data/out/degmatrices/t4_movies/d_mold.csv', sep=';')\n",
"#-----------------------------------------\n",
"# US State Capitals\n",
"#s_new.to_csv('data/out/degmatrices/t5_states/s_new.csv', sep=';')\n",
"#s_u_new.to_csv('data/out/degmatrices/t5_states/s_u_new.csv', sep=';')\n",
"#s_mold.to_csv('data/out/degmatrices/t5_states/s_mold.csv', sep=';')\n",
"\n",
"#=========================================\n",
"# Phase II: Validation on Annotated Data\n",
"#=========================================\n",
"#grec_m.to_csv('data/GREC/grec_final/gt_with_truthvals.csv', sep=';')\n",
"#grec_m2.to_csv('data/GREC/GREC_degree/gt_with_truthvals.csv', sep=';')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Load the , and concat"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"# Uncomment the variables that will be used.\n",
"import pandas as pd\n",
"#=========================================\n",
"# Phase I: Classification Task\n",
"#=========================================\n",
"i1_mt = pd.read_csv('data/out/degmatrices/i1_mt.csv', sep=';', index_col=0)\n",
"i2_mt = pd.read_csv('data/out/degmatrices/i2_mt.csv', sep=';', index_col=0)\n",
"i1_mt = pd.concat([i1_mt,i2_mt])\n",
"i1_ut = pd.read_csv('data/out/degmatrices/i1_ut.csv', sep=';', index_col=0)\n",
"i2_ut = pd.read_csv('data/out/degmatrices/i2_ut.csv', sep=';', index_col=0)\n",
"i1_ut = pd.concat([i1_ut,i2_ut])\n",
"i1_mold = pd.read_csv('data/out/degmatrices/i1_mold.csv', sep=';', index_col=0)\n",
"i2_mold = pd.read_csv('data/out/degmatrices/i2_mold.csv', sep=';', index_col=0)\n",
"i1_mold = pd.concat([i1_mold,i2_mold])\n",
"i1_mold.to_csv('data/out/degmatrices/ide_mold.csv', sep=';')\n",
"#i1_mt = pd.read_csv('data/out/degmatrices/ide.csv', sep=';', index_col=0)\n",
"\n",
"#=========================================\n",
"# Phase II: Validation on Factual Data\n",
"#=========================================\n",
"# Presidents/Spouses\n",
"w_new = pd.read_csv('data/out/degmatrices/t2_presidents/w_new.csv', sep=';', index_col=0)\n",
"w_u_new = pd.read_csv('data/out/degmatrices/t2_presidents/w_u_new.csv', sep=';', index_col=0)\n",
"w_mold = pd.read_csv('data/out/degmatrices/t2_presidents/w_mold.csv', sep=';', index_col=0)\n",
"#-----------------------------------------\n",
"# Countries/Capitals\n",
"c_new = pd.read_csv('data/out/degmatrices/t3_countries/c_new.csv', sep=',', index_col=0)\n",
"c_u_new = pd.read_csv('data/out/degmatrices/t3_countries/c_u_new.csv', sep=',', index_col=0)\n",
"c_u_mold = pd.read_csv('data/out/degmatrices/t3_countries/c_u_mold.csv', sep=',', index_col=0)\n",
"#-----------------------------------------\n",
"# Director/Movie\n",
"d_new = pd.read_csv('data/out/degmatrices/t4_movies/d_new.csv', sep=';', index_col=0)\n",
"d_u_new = pd.read_csv('data/out/degmatrices/t4_movies/d_u_new.csv', sep=';', index_col=0)\n",
"d_mold = pd.read_csv('data/out/degmatrices/t4_movies/d_mold.csv', sep=';', index_col=0)\n",
"#-----------------------------------------\n",
"# US State Capitals\n",
"s_new = pd.read_csv('data/out/degmatrices/t5_states/s_new.csv', sep=';', index_col=0)\n",
"s_u_new = pd.read_csv('data/out/degmatrices/t5_states/s_u_new.csv', sep=';', index_col=0)\n",
"s_mold = pd.read_csv('data/out/degmatrices/t5_states/s_mold.csv', sep=';', index_col=0)\n",
"\n",
"#=========================================\n",
"# Phase II: Validation on Annotated Data\n",
"#=========================================\n",
"grec_m = pd.read_csv('data/GREC/grec_final/gt_with_truthvals.csv', sep=';', index_col = 0)\n",
"grec_m2 = pd.read_csv('data/GREC/GREC_degree/gt_with_truthvals.csv', sep=';', index_col = 0)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"mold.to_csv('data/classification/new_mold.csv', sep=';', encoding='utf-8')\n",
"ut.to_csv('data/classification/new_ut.csv', sep=';', encoding='utf-8')\n",
"mt.to_csv('data/classification/new_mt.csv', sep=';', encoding='utf-8')"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"mt.to_csv('data/out/degmatrices/i5_m.csv', sep=';', encoding='utf-8')\n",
"ut.to_csv('data/out/degmatrices/i5_u.csv', sep=';', encoding='utf-8')\n",
"mold.to_csv('data/out/degmatrices/i5_mold.csv', sep=';', encoding='utf-8')"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"grec_m2.to_csv('data/final datasets/3. validation on annotated data/3.2 Education-degree.csv', sep=',')"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(275, 20)"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"i1_mold.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": false
},
"outputs": [],
"source": [
"i1_mt"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "ThesisVirtualEnvPy3",
"language": "python",
"name": "thesisvirtualenvpy3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.0"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
About
No description, website, or topics provided.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published