Pandas.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.3"},"colab":{"name":"Pandas.ipynb","provenance":[]}},"cells":[{"cell_type":"markdown","metadata":{"id":"FbNvYfa6TeVg","colab_type":"text"},"source":["<img src=\"https://www.politecnicos.com.br/img/075.jpg\" alt=\"Grupo Turing\" height=\"420\" width=\"420\">\n","\n","Notebook de conceitos e funções básicas da biblioteca pandas do Grupo Turing\n","\n","Autor: Ariel Guerreiro \n","\n","![Pandas](https://pandas.pydata.org/_static/pandas_logo.png)\n","\n"]},{"cell_type":"markdown","metadata":{"id":"-cTbFr23TeS6","colab_type":"text"},"source":["# Introdução"]},{"cell_type":"markdown","metadata":{"id":"98ZaTA5cTeS8","colab_type":"text"},"source":["   Pandas é uma biblioteca para Python muito utilizada na área de Data Science, sendo a melhor forma de se trabalhar com os dados uma vez que permite limpar, organizar, transformar e analisar o dataset, com o auxílio de outras bibliotecas de visualização, como Matplotlib ou Seaborn, ou com bibliotecas de IA, como SKLearn.\n","\n","   \n","   \n","   Os dados em pandas podem ser analisados de duas formas: **Series** e **Dataframes**. Series são listas unidimensionais, que podem guardar dados de qualquer tipo, enquanto os Dataframes são estruturas bidimensionais, como uma tabela, consistindo de linhas e colunas."]},{"cell_type":"markdown","metadata":{"id":"wfRIHfyuTeS-","colab_type":"text"},"source":["# Importando a biblioteca"]},{"cell_type":"markdown","metadata":{"id":"6i2hc6qaTeTB","colab_type":"text"},"source":["A biblioteca pode ser baixada por meio do pip, com o Anaconda ou diversas outras formas, como disponível no [tutorial oficial](https://pandas.pydata.org/pandas-docs/stable/install.html). Uma vez disponível, a biblioteca é importada da seguinte forma:"]},{"cell_type":"code","metadata":{"id":"IXoS1yNLTeTD","colab_type":"code","colab":{}},"source":["import pandas as pd # é normal utilizar 'pd' como abreviação para pandas"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"8xpaGp5nTeTK","colab_type":"text"},"source":["# Criando e visualizando um dataset"]},{"cell_type":"markdown","metadata":{"id":"VEeMjy3WTeTM","colab_type":"text"},"source":["É costumeiro criar um dataset através de um arquivo [CSV](https://pt.wikipedia.org/wiki/Comma-separated_values), mas outras formas de arquivos podem ser utilizados. Para a criação de um dataframe '**df**', utilizamos a função **read_csv**. Alguns de seus parâmetros são:\n","* **filepath_or_buffer**: nome do arquivo (com a extensão utilizada), com o caminho explicitado ou implícito, caso esteja na mesma pasta do notebook\n","* **sep, delimiter** : ',' como padrão\n","* **header**: lista para informar onde os nomes das colunas estão\n","* **names**: lista para ser utilizada como nome das colunas, muitas vezes já disponível como a primeira linha do CSV\n","* **usecols**: lista para fornecer quais colunas devem ou não devem ser utilizadas no dataframe\n","\n","Para este notebook, será utilizado o dataframe 'Titanic', que possui informações de passageiros e se sobreviveram ou não, bastante utilizado para machine learning. O dataset foi retirado [deste link](https://www.kaggle.com/c/titanic/data)"]},{"cell_type":"code","metadata":{"id":"oBL4bNMbTeTQ","colab_type":"code","colab":{}},"source":["df = pd.read_csv('titanic.csv')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"LeF_b9WJTeTX","colab_type":"text"},"source":["Agora é interessante vizualizar alguns detalhes sobre o dataset. São utilizadas as funções:\n","* **df.head()**: imprime as 5 primeiras linhas (pode ser alterado), análogo ao **df.tail()**, que imprime as últimas linhas\n","* **df.describe()**: mostra medidas estatísticas de cada coluna (onde tais medidas são possíveis)\n","* **df.info()**: fornece informação de número de linhas preeenchidas e tipo de dados"]},{"cell_type":"code","metadata":{"id":"-COHUxcNTeTZ","colab_type":"code","outputId":"0557c30b-c5b1-4b35-b586-c0fe8d69754c","colab":{}},"source":["df.head()"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PassengerId</th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Name</th>\n","      <th>Sex</th>\n","      <th>Age</th>\n","      <th>SibSp</th>\n","      <th>Parch</th>\n","      <th>Ticket</th>\n","      <th>Fare</th>\n","      <th>Cabin</th>\n","      <th>Embarked</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Braund, Mr. Owen Harris</td>\n","      <td>male</td>\n","      <td>22.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>A/5 21171</td>\n","      <td>7.2500</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>2</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n","      <td>female</td>\n","      <td>38.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>PC 17599</td>\n","      <td>71.2833</td>\n","      <td>C85</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>3</td>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>Heikkinen, Miss. Laina</td>\n","      <td>female</td>\n","      <td>26.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>STON/O2. 3101282</td>\n","      <td>7.9250</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>4</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n","      <td>female</td>\n","      <td>35.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>113803</td>\n","      <td>53.1000</td>\n","      <td>C123</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>5</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Allen, Mr. William Henry</td>\n","      <td>male</td>\n","      <td>35.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>373450</td>\n","      <td>8.0500</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   PassengerId  Survived  Pclass  \\\n","0            1         0       3   \n","1            2         1       1   \n","2            3         1       3   \n","3            4         1       1   \n","4            5         0       3   \n","\n","                                                Name     Sex   Age  SibSp  \\\n","0                            Braund, Mr. Owen Harris    male  22.0      1   \n","1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n","2                             Heikkinen, Miss. Laina  female  26.0      0   \n","3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n","4                           Allen, Mr. William Henry    male  35.0      0   \n","\n","   Parch            Ticket     Fare Cabin Embarked  \n","0      0         A/5 21171   7.2500   NaN        S  \n","1      0          PC 17599  71.2833   C85        C  \n","2      0  STON/O2. 3101282   7.9250   NaN        S  \n","3      0            113803  53.1000  C123        S  \n","4      0            373450   8.0500   NaN        S  "]},"metadata":{"tags":[]},"execution_count":3}]},{"cell_type":"code","metadata":{"id":"KhREoZ0cTeTi","colab_type":"code","outputId":"5df2f216-4694-4a6a-b693-d09d58bf6ee2","colab":{}},"source":["df.describe()"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PassengerId</th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Age</th>\n","      <th>SibSp</th>\n","      <th>Parch</th>\n","      <th>Fare</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>count</th>\n","      <td>891.000000</td>\n","      <td>891.000000</td>\n","      <td>891.000000</td>\n","      <td>714.000000</td>\n","      <td>891.000000</td>\n","      <td>891.000000</td>\n","      <td>891.000000</td>\n","    </tr>\n","    <tr>\n","      <th>mean</th>\n","      <td>446.000000</td>\n","      <td>0.383838</td>\n","      <td>2.308642</td>\n","      <td>29.699118</td>\n","      <td>0.523008</td>\n","      <td>0.381594</td>\n","      <td>32.204208</td>\n","    </tr>\n","    <tr>\n","      <th>std</th>\n","      <td>257.353842</td>\n","      <td>0.486592</td>\n","      <td>0.836071</td>\n","      <td>14.526497</td>\n","      <td>1.102743</td>\n","      <td>0.806057</td>\n","      <td>49.693429</td>\n","    </tr>\n","    <tr>\n","      <th>min</th>\n","      <td>1.000000</td>\n","      <td>0.000000</td>\n","      <td>1.000000</td>\n","      <td>0.420000</td>\n","      <td>0.000000</td>\n","      <td>0.000000</td>\n","      <td>0.000000</td>\n","    </tr>\n","    <tr>\n","      <th>25%</th>\n","      <td>223.500000</td>\n","      <td>0.000000</td>\n","      <td>2.000000</td>\n","      <td>20.125000</td>\n","      <td>0.000000</td>\n","      <td>0.000000</td>\n","      <td>7.910400</td>\n","    </tr>\n","    <tr>\n","      <th>50%</th>\n","      <td>446.000000</td>\n","      <td>0.000000</td>\n","      <td>3.000000</td>\n","      <td>28.000000</td>\n","      <td>0.000000</td>\n","      <td>0.000000</td>\n","      <td>14.454200</td>\n","    </tr>\n","    <tr>\n","      <th>75%</th>\n","      <td>668.500000</td>\n","      <td>1.000000</td>\n","      <td>3.000000</td>\n","      <td>38.000000</td>\n","      <td>1.000000</td>\n","      <td>0.000000</td>\n","      <td>31.000000</td>\n","    </tr>\n","    <tr>\n","      <th>max</th>\n","      <td>891.000000</td>\n","      <td>1.000000</td>\n","      <td>3.000000</td>\n","      <td>80.000000</td>\n","      <td>8.000000</td>\n","      <td>6.000000</td>\n","      <td>512.329200</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["       PassengerId    Survived      Pclass         Age       SibSp  \\\n","count   891.000000  891.000000  891.000000  714.000000  891.000000   \n","mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n","std     257.353842    0.486592    0.836071   14.526497    1.102743   \n","min       1.000000    0.000000    1.000000    0.420000    0.000000   \n","25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n","50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n","75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n","max     891.000000    1.000000    3.000000   80.000000    8.000000   \n","\n","            Parch        Fare  \n","count  891.000000  891.000000  \n","mean     0.381594   32.204208  \n","std      0.806057   49.693429  \n","min      0.000000    0.000000  \n","25%      0.000000    7.910400  \n","50%      0.000000   14.454200  \n","75%      0.000000   31.000000  \n","max      6.000000  512.329200  "]},"metadata":{"tags":[]},"execution_count":4}]},{"cell_type":"code","metadata":{"id":"SM0ngH28TeTo","colab_type":"code","outputId":"748327ca-d668-41f8-ff9d-f813d11768c4","colab":{}},"source":["df.info()"],"execution_count":0,"outputs":[{"output_type":"stream","text":["<class 'pandas.core.frame.DataFrame'>\n","RangeIndex: 891 entries, 0 to 890\n","Data columns (total 12 columns):\n","PassengerId    891 non-null int64\n","Survived       891 non-null int64\n","Pclass         891 non-null int64\n","Name           891 non-null object\n","Sex            891 non-null object\n","Age            714 non-null float64\n","SibSp          891 non-null int64\n","Parch          891 non-null int64\n","Ticket         891 non-null object\n","Fare           891 non-null float64\n","Cabin          204 non-null object\n","Embarked       889 non-null object\n","dtypes: float64(2), int64(5), object(5)\n","memory usage: 83.6+ KB\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"qpWAnyFYTeTt","colab_type":"text"},"source":["Caso se queira separar parte do dataframe, para uma análise ou manipulação, existem funções dentro do pandas para tal: \n","* **df.loc[ ]**: pega linhas (ou colunas) com uma **label** particular do index\n","* **df.iloc[ ]**: pega linhas (ou colunas) com uma **posição** particular do index (portanto somente recebe inteiros) \n","* **df.ix[ ]** tenta se comportar como loc, mas se comporta como iloc se não recebe o label\n"]},{"cell_type":"code","metadata":{"id":"qoWtnMRCTeTx","colab_type":"code","outputId":"efe6378b-47db-4642-e567-e9916058f406","colab":{}},"source":["df.iloc[:5,:5] #separa as 5 primeiras linhas e as 5 primeiras colunas"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PassengerId</th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Name</th>\n","      <th>Sex</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Braund, Mr. Owen Harris</td>\n","      <td>male</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>2</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n","      <td>female</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>3</td>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>Heikkinen, Miss. Laina</td>\n","      <td>female</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>4</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n","      <td>female</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>5</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Allen, Mr. William Henry</td>\n","      <td>male</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   PassengerId  Survived  Pclass  \\\n","0            1         0       3   \n","1            2         1       1   \n","2            3         1       3   \n","3            4         1       1   \n","4            5         0       3   \n","\n","                                                Name     Sex  \n","0                            Braund, Mr. Owen Harris    male  \n","1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  \n","2                             Heikkinen, Miss. Laina  female  \n","3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  \n","4                           Allen, Mr. William Henry    male  "]},"metadata":{"tags":[]},"execution_count":6}]},{"cell_type":"code","metadata":{"id":"BRDV3PuiTeT2","colab_type":"code","outputId":"17e5a636-0de7-4755-ec4a-980db2d3bec3","colab":{}},"source":["df.loc[0:5, 'Survived':'Age'] #separa as 6 primeiras linhas e as colunas de 'Survived' até 'Age' "],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Name</th>\n","      <th>Sex</th>\n","      <th>Age</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Braund, Mr. Owen Harris</td>\n","      <td>male</td>\n","      <td>22.0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n","      <td>female</td>\n","      <td>38.0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>Heikkinen, Miss. Laina</td>\n","      <td>female</td>\n","      <td>26.0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n","      <td>female</td>\n","      <td>35.0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Allen, Mr. William Henry</td>\n","      <td>male</td>\n","      <td>35.0</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Moran, Mr. James</td>\n","      <td>male</td>\n","      <td>NaN</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   Survived  Pclass                                               Name  \\\n","0         0       3                            Braund, Mr. Owen Harris   \n","1         1       1  Cumings, Mrs. John Bradley (Florence Briggs Th...   \n","2         1       3                             Heikkinen, Miss. Laina   \n","3         1       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)   \n","4         0       3                           Allen, Mr. William Henry   \n","5         0       3                                   Moran, Mr. James   \n","\n","      Sex   Age  \n","0    male  22.0  \n","1  female  38.0  \n","2  female  26.0  \n","3  female  35.0  \n","4    male  35.0  \n","5    male   NaN  "]},"metadata":{"tags":[]},"execution_count":7}]},{"cell_type":"markdown","metadata":{"id":"kLOS2JeJTeT8","colab_type":"text"},"source":["Também é possível separar o dataframe por meio de condições, podendo ser utilizadas ferramentas lógicas como o **& (and)** e **| (ou)**. Por exemplo, é possível separar o dataset com os passageiros que possuem menos de 25 anos e estavam na 1a classe:"]},{"cell_type":"code","metadata":{"id":"RUvanbpBTeT9","colab_type":"code","outputId":"d152b893-535a-49ac-c290-58e3f96fbc43","colab":{}},"source":["df[(df['Age']<25) & (df['Pclass']==1)]"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PassengerId</th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Name</th>\n","      <th>Sex</th>\n","      <th>Age</th>\n","      <th>SibSp</th>\n","      <th>Parch</th>\n","      <th>Ticket</th>\n","      <th>Fare</th>\n","      <th>Cabin</th>\n","      <th>Embarked</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>27</th>\n","      <td>28</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Fortune, Mr. Charles Alexander</td>\n","      <td>male</td>\n","      <td>19.00</td>\n","      <td>3</td>\n","      <td>2</td>\n","      <td>19950</td>\n","      <td>263.0000</td>\n","      <td>C23 C25 C27</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>88</th>\n","      <td>89</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Fortune, Miss. Mabel Helen</td>\n","      <td>female</td>\n","      <td>23.00</td>\n","      <td>3</td>\n","      <td>2</td>\n","      <td>19950</td>\n","      <td>263.0000</td>\n","      <td>C23 C25 C27</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>97</th>\n","      <td>98</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Greenfield, Mr. William Bertram</td>\n","      <td>male</td>\n","      <td>23.00</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>PC 17759</td>\n","      <td>63.3583</td>\n","      <td>D10 D12</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>102</th>\n","      <td>103</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>White, Mr. Richard Frasar</td>\n","      <td>male</td>\n","      <td>21.00</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>35281</td>\n","      <td>77.2875</td>\n","      <td>D26</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>118</th>\n","      <td>119</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Baxter, Mr. Quigg Edmond</td>\n","      <td>male</td>\n","      <td>24.00</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>PC 17558</td>\n","      <td>247.5208</td>\n","      <td>B58 B60</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>136</th>\n","      <td>137</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Newsom, Miss. Helen Monypeny</td>\n","      <td>female</td>\n","      <td>19.00</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>11752</td>\n","      <td>26.2833</td>\n","      <td>D47</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>139</th>\n","      <td>140</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Giglio, Mr. Victor</td>\n","      <td>male</td>\n","      <td>24.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>PC 17593</td>\n","      <td>79.2000</td>\n","      <td>B86</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>151</th>\n","      <td>152</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Pears, Mrs. Thomas (Edith Wearne)</td>\n","      <td>female</td>\n","      <td>22.00</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>113776</td>\n","      <td>66.6000</td>\n","      <td>C2</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>291</th>\n","      <td>292</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Bishop, Mrs. Dickinson H (Helen Walton)</td>\n","      <td>female</td>\n","      <td>19.00</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>11967</td>\n","      <td>91.0792</td>\n","      <td>B49</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>297</th>\n","      <td>298</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Allison, Miss. Helen Loraine</td>\n","      <td>female</td>\n","      <td>2.00</td>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>113781</td>\n","      <td>151.5500</td>\n","      <td>C22 C26</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>305</th>\n","      <td>306</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Allison, Master. Hudson Trevor</td>\n","      <td>male</td>\n","      <td>0.92</td>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>113781</td>\n","      <td>151.5500</td>\n","      <td>C22 C26</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>307</th>\n","      <td>308</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Penasco y Castellana, Mrs. Victor de Satode (M...</td>\n","      <td>female</td>\n","      <td>17.00</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>PC 17758</td>\n","      <td>108.9000</td>\n","      <td>C65</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>310</th>\n","      <td>311</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Hays, Miss. Margaret Bechstein</td>\n","      <td>female</td>\n","      <td>24.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>11767</td>\n","      <td>83.1583</td>\n","      <td>C54</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>311</th>\n","      <td>312</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Ryerson, Miss. Emily Borie</td>\n","      <td>female</td>\n","      <td>18.00</td>\n","      <td>2</td>\n","      <td>2</td>\n","      <td>PC 17608</td>\n","      <td>262.3750</td>\n","      <td>B57 B59 B63 B66</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>329</th>\n","      <td>330</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Hippach, Miss. Jean Gertrude</td>\n","      <td>female</td>\n","      <td>16.00</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>111361</td>\n","      <td>57.9792</td>\n","      <td>B18</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>341</th>\n","      <td>342</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Fortune, Miss. Alice Elizabeth</td>\n","      <td>female</td>\n","      <td>24.00</td>\n","      <td>3</td>\n","      <td>2</td>\n","      <td>19950</td>\n","      <td>263.0000</td>\n","      <td>C23 C25 C27</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>356</th>\n","      <td>357</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Bowerman, Miss. Elsie Edith</td>\n","      <td>female</td>\n","      <td>22.00</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>113505</td>\n","      <td>55.0000</td>\n","      <td>E33</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>369</th>\n","      <td>370</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Aubart, Mme. Leontine Pauline</td>\n","      <td>female</td>\n","      <td>24.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>PC 17477</td>\n","      <td>69.3000</td>\n","      <td>B35</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>373</th>\n","      <td>374</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Ringhini, Mr. Sante</td>\n","      <td>male</td>\n","      <td>22.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>PC 17760</td>\n","      <td>135.6333</td>\n","      <td>NaN</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>393</th>\n","      <td>394</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Newell, Miss. Marjorie</td>\n","      <td>female</td>\n","      <td>23.00</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>35273</td>\n","      <td>113.2750</td>\n","      <td>D36</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>435</th>\n","      <td>436</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Carter, Miss. Lucile Polk</td>\n","      <td>female</td>\n","      <td>14.00</td>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>113760</td>\n","      <td>120.0000</td>\n","      <td>B96 B98</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>445</th>\n","      <td>446</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Dodge, Master. Washington</td>\n","      <td>male</td>\n","      <td>4.00</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>33638</td>\n","      <td>81.8583</td>\n","      <td>A34</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>504</th>\n","      <td>505</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Maioni, Miss. Roberta</td>\n","      <td>female</td>\n","      <td>16.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>110152</td>\n","      <td>86.5000</td>\n","      <td>B79</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>505</th>\n","      <td>506</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Penasco y Castellana, Mr. Victor de Satode</td>\n","      <td>male</td>\n","      <td>18.00</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>PC 17758</td>\n","      <td>108.9000</td>\n","      <td>C65</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>539</th>\n","      <td>540</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Frolicher, Miss. Hedwig Margaritha</td>\n","      <td>female</td>\n","      <td>22.00</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>13568</td>\n","      <td>49.5000</td>\n","      <td>B39</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>550</th>\n","      <td>551</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Thayer, Mr. John Borland Jr</td>\n","      <td>male</td>\n","      <td>17.00</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>17421</td>\n","      <td>110.8833</td>\n","      <td>C70</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>585</th>\n","      <td>586</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Taussig, Miss. Ruth</td>\n","      <td>female</td>\n","      <td>18.00</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>110413</td>\n","      <td>79.6500</td>\n","      <td>E68</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>627</th>\n","      <td>628</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Longley, Miss. Gretchen Fiske</td>\n","      <td>female</td>\n","      <td>21.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>13502</td>\n","      <td>77.9583</td>\n","      <td>D9</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>641</th>\n","      <td>642</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Sagesser, Mlle. Emma</td>\n","      <td>female</td>\n","      <td>24.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>PC 17477</td>\n","      <td>69.3000</td>\n","      <td>B35</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>689</th>\n","      <td>690</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Madill, Miss. Georgette Alexandra</td>\n","      <td>female</td>\n","      <td>15.00</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>24160</td>\n","      <td>211.3375</td>\n","      <td>B5</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>700</th>\n","      <td>701</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Astor, Mrs. John Jacob (Madeleine Talmadge Force)</td>\n","      <td>female</td>\n","      <td>18.00</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>PC 17757</td>\n","      <td>227.5250</td>\n","      <td>C62 C64</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>708</th>\n","      <td>709</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Cleaver, Miss. Alice</td>\n","      <td>female</td>\n","      <td>22.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>113781</td>\n","      <td>151.5500</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>710</th>\n","      <td>711</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Mayne, Mlle. Berthe Antonine (\"Mrs de Villiers\")</td>\n","      <td>female</td>\n","      <td>24.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>PC 17482</td>\n","      <td>49.5042</td>\n","      <td>C90</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>742</th>\n","      <td>743</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Ryerson, Miss. Susan Parker \"Suzette\"</td>\n","      <td>female</td>\n","      <td>21.00</td>\n","      <td>2</td>\n","      <td>2</td>\n","      <td>PC 17608</td>\n","      <td>262.3750</td>\n","      <td>B57 B59 B63 B66</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>748</th>\n","      <td>749</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Marvin, Mr. Daniel Warner</td>\n","      <td>male</td>\n","      <td>19.00</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>113773</td>\n","      <td>53.1000</td>\n","      <td>D30</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>781</th>\n","      <td>782</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Dick, Mrs. Albert Adrian (Vera Gillespie)</td>\n","      <td>female</td>\n","      <td>17.00</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>17474</td>\n","      <td>57.0000</td>\n","      <td>B20</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>802</th>\n","      <td>803</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Carter, Master. William Thornton II</td>\n","      <td>male</td>\n","      <td>11.00</td>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>113760</td>\n","      <td>120.0000</td>\n","      <td>B96 B98</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>853</th>\n","      <td>854</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Lines, Miss. Mary Conover</td>\n","      <td>female</td>\n","      <td>16.00</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>PC 17592</td>\n","      <td>39.4000</td>\n","      <td>D28</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>887</th>\n","      <td>888</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Graham, Miss. Margaret Edith</td>\n","      <td>female</td>\n","      <td>19.00</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>112053</td>\n","      <td>30.0000</td>\n","      <td>B42</td>\n","      <td>S</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["     PassengerId  Survived  Pclass  \\\n","27            28         0       1   \n","88            89         1       1   \n","97            98         1       1   \n","102          103         0       1   \n","118          119         0       1   \n","136          137         1       1   \n","139          140         0       1   \n","151          152         1       1   \n","291          292         1       1   \n","297          298         0       1   \n","305          306         1       1   \n","307          308         1       1   \n","310          311         1       1   \n","311          312         1       1   \n","329          330         1       1   \n","341          342         1       1   \n","356          357         1       1   \n","369          370         1       1   \n","373          374         0       1   \n","393          394         1       1   \n","435          436         1       1   \n","445          446         1       1   \n","504          505         1       1   \n","505          506         0       1   \n","539          540         1       1   \n","550          551         1       1   \n","585          586         1       1   \n","627          628         1       1   \n","641          642         1       1   \n","689          690         1       1   \n","700          701         1       1   \n","708          709         1       1   \n","710          711         1       1   \n","742          743         1       1   \n","748          749         0       1   \n","781          782         1       1   \n","802          803         1       1   \n","853          854         1       1   \n","887          888         1       1   \n","\n","                                                  Name     Sex    Age  SibSp  \\\n","27                      Fortune, Mr. Charles Alexander    male  19.00      3   \n","88                          Fortune, Miss. Mabel Helen  female  23.00      3   \n","97                     Greenfield, Mr. William Bertram    male  23.00      0   \n","102                          White, Mr. Richard Frasar    male  21.00      0   \n","118                           Baxter, Mr. Quigg Edmond    male  24.00      0   \n","136                       Newsom, Miss. Helen Monypeny  female  19.00      0   \n","139                                 Giglio, Mr. Victor    male  24.00      0   \n","151                  Pears, Mrs. Thomas (Edith Wearne)  female  22.00      1   \n","291            Bishop, Mrs. Dickinson H (Helen Walton)  female  19.00      1   \n","297                       Allison, Miss. Helen Loraine  female   2.00      1   \n","305                     Allison, Master. Hudson Trevor    male   0.92      1   \n","307  Penasco y Castellana, Mrs. Victor de Satode (M...  female  17.00      1   \n","310                     Hays, Miss. Margaret Bechstein  female  24.00      0   \n","311                         Ryerson, Miss. Emily Borie  female  18.00      2   \n","329                       Hippach, Miss. Jean Gertrude  female  16.00      0   \n","341                     Fortune, Miss. Alice Elizabeth  female  24.00      3   \n","356                        Bowerman, Miss. Elsie Edith  female  22.00      0   \n","369                      Aubart, Mme. Leontine Pauline  female  24.00      0   \n","373                                Ringhini, Mr. Sante    male  22.00      0   \n","393                             Newell, Miss. Marjorie  female  23.00      1   \n","435                          Carter, Miss. Lucile Polk  female  14.00      1   \n","445                          Dodge, Master. Washington    male   4.00      0   \n","504                              Maioni, Miss. Roberta  female  16.00      0   \n","505         Penasco y Castellana, Mr. Victor de Satode    male  18.00      1   \n","539                 Frolicher, Miss. Hedwig Margaritha  female  22.00      0   \n","550                        Thayer, Mr. John Borland Jr    male  17.00      0   \n","585                                Taussig, Miss. Ruth  female  18.00      0   \n","627                      Longley, Miss. Gretchen Fiske  female  21.00      0   \n","641                               Sagesser, Mlle. Emma  female  24.00      0   \n","689                  Madill, Miss. Georgette Alexandra  female  15.00      0   \n","700  Astor, Mrs. John Jacob (Madeleine Talmadge Force)  female  18.00      1   \n","708                               Cleaver, Miss. Alice  female  22.00      0   \n","710   Mayne, Mlle. Berthe Antonine (\"Mrs de Villiers\")  female  24.00      0   \n","742              Ryerson, Miss. Susan Parker \"Suzette\"  female  21.00      2   \n","748                          Marvin, Mr. Daniel Warner    male  19.00      1   \n","781          Dick, Mrs. Albert Adrian (Vera Gillespie)  female  17.00      1   \n","802                Carter, Master. William Thornton II    male  11.00      1   \n","853                          Lines, Miss. Mary Conover  female  16.00      0   \n","887                       Graham, Miss. Margaret Edith  female  19.00      0   \n","\n","     Parch    Ticket      Fare            Cabin Embarked  \n","27       2     19950  263.0000      C23 C25 C27        S  \n","88       2     19950  263.0000      C23 C25 C27        S  \n","97       1  PC 17759   63.3583          D10 D12        C  \n","102      1     35281   77.2875              D26        S  \n","118      1  PC 17558  247.5208          B58 B60        C  \n","136      2     11752   26.2833              D47        S  \n","139      0  PC 17593   79.2000              B86        C  \n","151      0    113776   66.6000               C2        S  \n","291      0     11967   91.0792              B49        C  \n","297      2    113781  151.5500          C22 C26        S  \n","305      2    113781  151.5500          C22 C26        S  \n","307      0  PC 17758  108.9000              C65        C  \n","310      0     11767   83.1583              C54        C  \n","311      2  PC 17608  262.3750  B57 B59 B63 B66        C  \n","329      1    111361   57.9792              B18        C  \n","341      2     19950  263.0000      C23 C25 C27        S  \n","356      1    113505   55.0000              E33        S  \n","369      0  PC 17477   69.3000              B35        C  \n","373      0  PC 17760  135.6333              NaN        C  \n","393      0     35273  113.2750              D36        C  \n","435      2    113760  120.0000          B96 B98        S  \n","445      2     33638   81.8583              A34        S  \n","504      0    110152   86.5000              B79        S  \n","505      0  PC 17758  108.9000              C65        C  \n","539      2     13568   49.5000              B39        C  \n","550      2     17421  110.8833              C70        C  \n","585      2    110413   79.6500              E68        S  \n","627      0     13502   77.9583               D9        S  \n","641      0  PC 17477   69.3000              B35        C  \n","689      1     24160  211.3375               B5        S  \n","700      0  PC 17757  227.5250          C62 C64        C  \n","708      0    113781  151.5500              NaN        S  \n","710      0  PC 17482   49.5042              C90        C  \n","742      2  PC 17608  262.3750  B57 B59 B63 B66        C  \n","748      0    113773   53.1000              D30        S  \n","781      0     17474   57.0000              B20        S  \n","802      2    113760  120.0000          B96 B98        S  \n","853      1  PC 17592   39.4000              D28        S  \n","887      0    112053   30.0000              B42        S  "]},"metadata":{"tags":[]},"execution_count":8}]},{"cell_type":"markdown","metadata":{"id":"ZXJEOyilTeUD","colab_type":"text"},"source":["# Manipulando um dataset"]},{"cell_type":"markdown","metadata":{"id":"-SJ0cdmoTeUE","colab_type":"text"},"source":["Um dataset não é imutável, o pandas permite a manipulação completa dos dados e de suas características. \n","\n"," Por exemplo, o CSV utilizado já possui uma coluna de índice das linhas, mas o pandas criou uma outra própria. Para eliminar essa repetição, podemos transformar a coluna 'PassengerId' no índice do dataframe, por meio da função\n","**df.set_index('colname')**"]},{"cell_type":"code","metadata":{"id":"vfSHadufTeUG","colab_type":"code","outputId":"dc05b803-3f58-4eca-8303-1ffd1d459b89","colab":{}},"source":["df = df.set_index('PassengerId') # é possível utilizar a igualdade ou passar o parâmetro 'inplace = True' na função,\n","                                 # como será explorado depois, mas o efeito é igual\n","\n","df.head() #podemos ver que a 'PassengerId' está sendo utilizada como index agora "],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Name</th>\n","      <th>Sex</th>\n","      <th>Age</th>\n","      <th>SibSp</th>\n","      <th>Parch</th>\n","      <th>Ticket</th>\n","      <th>Fare</th>\n","      <th>Cabin</th>\n","      <th>Embarked</th>\n","    </tr>\n","    <tr>\n","      <th>PassengerId</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>1</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Braund, Mr. Owen Harris</td>\n","      <td>male</td>\n","      <td>22.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>A/5 21171</td>\n","      <td>7.2500</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n","      <td>female</td>\n","      <td>38.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>PC 17599</td>\n","      <td>71.2833</td>\n","      <td>C85</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>Heikkinen, Miss. Laina</td>\n","      <td>female</td>\n","      <td>26.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>STON/O2. 3101282</td>\n","      <td>7.9250</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n","      <td>female</td>\n","      <td>35.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>113803</td>\n","      <td>53.1000</td>\n","      <td>C123</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Allen, Mr. William Henry</td>\n","      <td>male</td>\n","      <td>35.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>373450</td>\n","      <td>8.0500</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["             Survived  Pclass  \\\n","PassengerId                     \n","1                   0       3   \n","2                   1       1   \n","3                   1       3   \n","4                   1       1   \n","5                   0       3   \n","\n","                                                          Name     Sex   Age  \\\n","PassengerId                                                                    \n","1                                      Braund, Mr. Owen Harris    male  22.0   \n","2            Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   \n","3                                       Heikkinen, Miss. Laina  female  26.0   \n","4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   \n","5                                     Allen, Mr. William Henry    male  35.0   \n","\n","             SibSp  Parch            Ticket     Fare Cabin Embarked  \n","PassengerId                                                          \n","1                1      0         A/5 21171   7.2500   NaN        S  \n","2                1      0          PC 17599  71.2833   C85        C  \n","3                0      0  STON/O2. 3101282   7.9250   NaN        S  \n","4                1      0            113803  53.1000  C123        S  \n","5                0      0            373450   8.0500   NaN        S  "]},"metadata":{"tags":[]},"execution_count":9}]},{"cell_type":"markdown","metadata":{"id":"3P58QvlgTeUL","colab_type":"text"},"source":["A maior parte dos algoritmos de IA não conseguem lidar com [NaN (not a number)](https://pt.wikipedia.org/wiki/NaN), portanto, é interessante substituí-los por um valor numérico ou categórico. Como um exemplo, vamos ver se existem dados faltantes na coluna 'Age':"]},{"cell_type":"code","metadata":{"id":"-ayxdDRwTeUN","colab_type":"code","outputId":"0d205a4e-756e-4d0d-e362-2d2a64a7727d","colab":{}},"source":["df['Age'].isnull().any()"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["True"]},"metadata":{"tags":[]},"execution_count":10}]},{"cell_type":"markdown","metadata":{"id":"91neqq8GTeUV","colab_type":"text"},"source":["Como podemos ver, existem sim dados faltantes na coluna de idade. Podemos preencher com algum valor, como por exemplo a mediana das idades. Para tal, utiliza-se a função **df['colname'].fillna('value')** (caso o nome da coluna não seja fornecido, os NaN do dataframe todo serão preenchidos)"]},{"cell_type":"code","metadata":{"id":"8OJqGrxETeUX","colab_type":"code","outputId":"48ef7dcc-236f-407a-ebcd-92ddf194c7ba","colab":{}},"source":["df['Age'].fillna(df['Age'].median(), inplace = True)\n","df['Age'].isnull().any() # verificando se todos os valores foram substituídos"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/plain":["False"]},"metadata":{"tags":[]},"execution_count":11}]},{"cell_type":"markdown","metadata":{"id":"0-ANcKH5TeUc","colab_type":"text"},"source":["Em alguns algortimos de IA, também muitas vezes só podem ser usados valores númericos, mesmo que para dados categóricos. Portanto, na coluna 'Sex', por exemplo, podemos trocar 'male' por 0 e 'female' por 1, utilizando-se da função **df.replace([antes],[depois])**, que se utiliza dos termos que serão trocados e por quais serão trocados, que podem ser passados por meio de listas. Com o código ficará mais intuitivo:"]},{"cell_type":"code","metadata":{"id":"tPP6rah-TeUe","colab_type":"code","outputId":"b1c8a2e5-d89e-47f9-c473-0e7734b17e72","colab":{}},"source":["df.replace(['male','female'],[0,1], inplace = True) \n","#neste caso, o n-ésimo item da primeira lista será substituído pelo n-ésimo valor da segunda lista\n","df.head() #somente para visualizar a mudança"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Name</th>\n","      <th>Sex</th>\n","      <th>Age</th>\n","      <th>SibSp</th>\n","      <th>Parch</th>\n","      <th>Ticket</th>\n","      <th>Fare</th>\n","      <th>Cabin</th>\n","      <th>Embarked</th>\n","    </tr>\n","    <tr>\n","      <th>PassengerId</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>1</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Braund, Mr. Owen Harris</td>\n","      <td>0</td>\n","      <td>22.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>A/5 21171</td>\n","      <td>7.2500</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n","      <td>1</td>\n","      <td>38.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>PC 17599</td>\n","      <td>71.2833</td>\n","      <td>C85</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>Heikkinen, Miss. Laina</td>\n","      <td>1</td>\n","      <td>26.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>STON/O2. 3101282</td>\n","      <td>7.9250</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n","      <td>1</td>\n","      <td>35.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>113803</td>\n","      <td>53.1000</td>\n","      <td>C123</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Allen, Mr. William Henry</td>\n","      <td>0</td>\n","      <td>35.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>373450</td>\n","      <td>8.0500</td>\n","      <td>NaN</td>\n","      <td>S</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["             Survived  Pclass  \\\n","PassengerId                     \n","1                   0       3   \n","2                   1       1   \n","3                   1       3   \n","4                   1       1   \n","5                   0       3   \n","\n","                                                          Name  Sex   Age  \\\n","PassengerId                                                                 \n","1                                      Braund, Mr. Owen Harris    0  22.0   \n","2            Cumings, Mrs. John Bradley (Florence Briggs Th...    1  38.0   \n","3                                       Heikkinen, Miss. Laina    1  26.0   \n","4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)    1  35.0   \n","5                                     Allen, Mr. William Henry    0  35.0   \n","\n","             SibSp  Parch            Ticket     Fare Cabin Embarked  \n","PassengerId                                                          \n","1                1      0         A/5 21171   7.2500   NaN        S  \n","2                1      0          PC 17599  71.2833   C85        C  \n","3                0      0  STON/O2. 3101282   7.9250   NaN        S  \n","4                1      0            113803  53.1000  C123        S  \n","5                0      0            373450   8.0500   NaN        S  "]},"metadata":{"tags":[]},"execution_count":12}]},{"cell_type":"markdown","metadata":{"id":"h0DnSO5UTeUn","colab_type":"text"},"source":["Outras alterações também são possíveis. É possível que se queira, por exemplo, alterar o nome de uma das colunas, o que será feito por meio da função '**df.rename(columns={dict})**, com um dicionário correspondende das alterações, ou se queira excluir uma coluna, por julgar desnecessária, por meio da função '**df.drop(list)**, onde uma lista das colunas pode ser passada.\n","\n","Por exemplo, podemos renomear a coluna 'Parch' para 'Parents_and_Children', para torná-la mais intuitiva, e podemos apagar a coluna 'Cabin', por julgá-la irrelevante para a situação:"]},{"cell_type":"code","metadata":{"scrolled":true,"id":"xAJSzycPTeUo","colab_type":"code","outputId":"30cd3e7d-98d9-412f-bf45-638c15b821d6","colab":{}},"source":["df.rename(columns={'Parch': 'Parents_and_Children'}, inplace = True)\n","\n","df.drop('Cabin', axis=1, inplace = True)\n","\n","df.head()"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Name</th>\n","      <th>Sex</th>\n","      <th>Age</th>\n","      <th>SibSp</th>\n","      <th>Parents_and_Children</th>\n","      <th>Ticket</th>\n","      <th>Fare</th>\n","      <th>Embarked</th>\n","    </tr>\n","    <tr>\n","      <th>PassengerId</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>1</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Braund, Mr. Owen Harris</td>\n","      <td>0</td>\n","      <td>22.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>A/5 21171</td>\n","      <td>7.2500</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n","      <td>1</td>\n","      <td>38.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>PC 17599</td>\n","      <td>71.2833</td>\n","      <td>C</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>Heikkinen, Miss. Laina</td>\n","      <td>1</td>\n","      <td>26.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>STON/O2. 3101282</td>\n","      <td>7.9250</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n","      <td>1</td>\n","      <td>35.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>113803</td>\n","      <td>53.1000</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Allen, Mr. William Henry</td>\n","      <td>0</td>\n","      <td>35.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>373450</td>\n","      <td>8.0500</td>\n","      <td>S</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["             Survived  Pclass  \\\n","PassengerId                     \n","1                   0       3   \n","2                   1       1   \n","3                   1       3   \n","4                   1       1   \n","5                   0       3   \n","\n","                                                          Name  Sex   Age  \\\n","PassengerId                                                                 \n","1                                      Braund, Mr. Owen Harris    0  22.0   \n","2            Cumings, Mrs. John Bradley (Florence Briggs Th...    1  38.0   \n","3                                       Heikkinen, Miss. Laina    1  26.0   \n","4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)    1  35.0   \n","5                                     Allen, Mr. William Henry    0  35.0   \n","\n","             SibSp  Parents_and_Children            Ticket     Fare Embarked  \n","PassengerId                                                                   \n","1                1                     0         A/5 21171   7.2500        S  \n","2                1                     0          PC 17599  71.2833        C  \n","3                0                     0  STON/O2. 3101282   7.9250        S  \n","4                1                     0            113803  53.1000        S  \n","5                0                     0            373450   8.0500        S  "]},"metadata":{"tags":[]},"execution_count":13}]},{"cell_type":"markdown","metadata":{"id":"f5m9eUWnTeUu","colab_type":"text"},"source":["# Separando e unindo dataframes"]},{"cell_type":"markdown","metadata":{"id":"FAmS8iwvTeUw","colab_type":"text"},"source":["É possível separar do dataframe em diversos outros. Por exemplo, vamos criar o dataframe df_jovens_1, que possui somente pessoas da 1a classe que possuíam menos de 25 anos, conforme foi feito anteriormente. Também vamos criar o dataframe df_homens, que possui somente os homens:"]},{"cell_type":"code","metadata":{"id":"XkWFXleDTeU0","colab_type":"code","colab":{}},"source":["df_jovens_1 = df[df['Age']< 25]\n","\n","df_homens = df[df['Sex'] == 0] #lembrando a alteração feita anteriormente que transformou 'male' para 0"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"b3i0iWPmTeU6","colab_type":"text"},"source":["Agora, vamos supor que a intenção seja de unir estes dataframes. Para a união de dataframes, existem as seguintes funções:\n","* **pd.concat()**: [concatena](https://pt.wikipedia.org/wiki/Concatena%C3%A7%C3%A3o) 2 dataframes, útil para dataframes sem colunas em comum\n","* **pd.merge()**: junta dataframes que possui colunas em comum \n","\n","Como os dataframes que queremos possui colunas em comum, usaremos do **pd.merge()** para criar um dataframe df2, união dos dois dataframes que criamos antes."]},{"cell_type":"code","metadata":{"id":"4ufdoKEjTeU7","colab_type":"code","outputId":"49ed0dc0-e2ea-43ab-e921-d86a66b825bc","colab":{}},"source":["df2 = pd.merge(df_jovens_1, df_homens)\n","\n","df2.head()"],"execution_count":0,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Survived</th>\n","      <th>Pclass</th>\n","      <th>Name</th>\n","      <th>Sex</th>\n","      <th>Age</th>\n","      <th>SibSp</th>\n","      <th>Parents_and_Children</th>\n","      <th>Ticket</th>\n","      <th>Fare</th>\n","      <th>Embarked</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Braund, Mr. Owen Harris</td>\n","      <td>0</td>\n","      <td>22.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>A/5 21171</td>\n","      <td>7.250</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Palsson, Master. Gosta Leonard</td>\n","      <td>0</td>\n","      <td>2.0</td>\n","      <td>3</td>\n","      <td>1</td>\n","      <td>349909</td>\n","      <td>21.075</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Saundercock, Mr. William Henry</td>\n","      <td>0</td>\n","      <td>20.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>A/5. 2151</td>\n","      <td>8.050</td>\n","      <td>S</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>Rice, Master. Eugene</td>\n","      <td>0</td>\n","      <td>2.0</td>\n","      <td>4</td>\n","      <td>1</td>\n","      <td>382652</td>\n","      <td>29.125</td>\n","      <td>Q</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>Fortune, Mr. Charles Alexander</td>\n","      <td>0</td>\n","      <td>19.0</td>\n","      <td>3</td>\n","      <td>2</td>\n","      <td>19950</td>\n","      <td>263.000</td>\n","      <td>S</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   Survived  Pclass                            Name  Sex   Age  SibSp  \\\n","0         0       3         Braund, Mr. Owen Harris    0  22.0      1   \n","1         0       3  Palsson, Master. Gosta Leonard    0   2.0      3   \n","2         0       3  Saundercock, Mr. William Henry    0  20.0      0   \n","3         0       3            Rice, Master. Eugene    0   2.0      4   \n","4         0       1  Fortune, Mr. Charles Alexander    0  19.0      3   \n","\n","   Parents_and_Children     Ticket     Fare Embarked  \n","0                     0  A/5 21171    7.250        S  \n","1                     1     349909   21.075        S  \n","2                     0  A/5. 2151    8.050        S  \n","3                     1     382652   29.125        Q  \n","4                     2      19950  263.000        S  "]},"metadata":{"tags":[]},"execution_count":15}]},{"cell_type":"markdown","metadata":{"id":"d0jsjWVfTeVJ","colab_type":"text"},"source":["# Salvando o dataframe"]},{"cell_type":"markdown","metadata":{"id":"M5ImXqTtTeVM","colab_type":"text"},"source":["Após as alterações, talvez seja interessante salvar o dataset atualizado, gerando um novo arquivo CSV (ou de outro tipo). Para tal, utiliza-se a função **df.to_csv()**. Esta função possui estes parâmetros como principais:\n","* O path até o caminho, como é feito no read_csv\n","* **sep**: simbolo usado para separar os valores (normalmente ',')\n","* **na_rep**: como os NaNs serão representados\n","* **header**: lista com nome pras colunas\n","* **columns**: colunas selecionadas pra serem colocadas no arquivo\n","* **index**: booleano, define se vai ter ou não id no arquivo"]},{"cell_type":"code","metadata":{"id":"NgUJU1gVTeVS","colab_type":"code","colab":{}},"source":["df.to_csv('titanic_novo.csv', index = True)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"5Xtt1xlOTeVd","colab_type":"text"},"source":["# E agora?"]},{"cell_type":"markdown","metadata":{"id":"NYZ7SR4STeVe","colab_type":"text"},"source":["Este notebook mostra algumas ferramentas básicas e mais essenciais para a criação e manipulação de dataframes por meio da biblioteca Pandas, de forma a gerar dataframes melhores e mais aplicados para utilização em visualização de dados, treinamento de algoritmos de IA, entre outras.\n","\n","Existe muito além do que foi apresentado aqui, visto que Pandas se trata de uma biblioteca extremamente completa. Para aprendizado de novos mecanismos, recomendo os seguintes mecanismos:\n","* Função 'help' do python: extremamente completa, detalha os parâmetros de cada função, ex: help(df.head())\n","* Documentação do Pandas: [disponível neste link](https://pandas.pydata.org/pandas-docs/stable/index.html), possui o mesmo detalhamento de parâmetros que a função 'help', mas a maior parte das entradas possui exemplos de aplicação para melhor entendimento\n","* Posts no Medium: existem diversos, alguns estão disponíveis no documento associado a esse notebook\n","* Stack Overflow: dispensa explicações"]},{"cell_type":"markdown","metadata":{"id":"fh8GnJjRXYdb","colab_type":"text"},"source":["<img src=\"https://www.politecnicos.com.br/img/075.jpg\" alt=\"Grupo Turing\" height=\"420\" width=\"420\">"]}]}