exploratory_data_analysis.Rmd

---
title: "Exploratory Data Analysis"
output: html_notebook
---

# Importing Required Libraries

```{r}

require(tidyverse)
```

# Loading in Data

```{r}
data <- read_csv("../data/diabetes_data_upload.csv")
data
```

```{r}
data %>% select(class) %>% filter(class == "Positive") %>% summarise(n())
```
```{r}
data %>% select(class) %>% filter(class == "Negative") %>% summarise(n())
```

```{r}
data %>% 
  ggplot(aes(x=class, fill=class)) +
  geom_bar(show.legend = FALSE) +
  theme_bw() +
    theme(plot.title = element_text(hjust = 0.5)) +
  theme(plot.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(panel.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  labs(title="Incidence of Diabetes", x="Diabetes Class", y="Count")
ggsave("../plots/Diabetes_Incidence.png", dpi = 1200, width = 4, height = 4, units = "in")
```


We have 320 positive and 200 negative patients.


```{r}
colnames(data)
```
The response variable is class, and all other variables are predictors. All predictors are categorical except for Age which is a ordinal variable.

# Correlation Plot

```{r}
# replcaing categorical labels to numeric ones
data_clean <- data.frame(lapply(data, function(x) {
                  gsub("Yes", "1", x)
              }))

data_clean <- data.frame(lapply(data_clean, function(x) {
                  gsub("No", "0", x)
              }))

data_clean <- data.frame(lapply(data_clean, function(x) {
                  gsub("Positive", "1", x)
              }))

data_clean <- data.frame(lapply(data_clean, function(x) {
                  gsub("Negative", "0", x)
              }))

data_clean <- data.frame(lapply(data_clean, function(x) {
                  gsub("Male", "0", x)
              }))

data_clean <- data.frame(lapply(data_clean, function(x) {
                  gsub("Female", "1", x)
              }))

# converting all columns to numeric
data_clean <- data.frame(sapply( data_clean, as.numeric ))
data_clean

write_csv(data_clean, path="../data/clean_numeric_data.csv")
```


```{r}
require(ggcorrplot)

corr <- round(cor(data_clean),2)
corr
```

```{r}
ggcorrplot(corr, method = "square")  +
  #theme_bw() +
  labs(title="Feature Correlation Matrix") +  
  theme(plot.title = element_text(hjust = 0.5)) +
  theme(plot.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(panel.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))+
  theme(legend.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))
ggsave("../plots/corrplot.png", dpi = 400, width = 7, height = 8, units = "in")
```

# Find Mean of Ages by Class

```{r}
Age_Mean <- data %>%
  group_by(class) %>%
  summarize(mean = mean(Age))
```

# Diabetes vs Age

```{r}
data %>% 
  ggplot(aes(x=Age, group=class)) +
  geom_density(aes(fill = class),alpha=0.25) + 
  geom_vline(data = Age_Mean, aes(xintercept = mean, color = class), size= 0.5, show.legend = FALSE) +
  theme_bw() +
  labs(title="Diabetes Incidence by Age", fill="Diabetes", x="Age", y="Density") +
  theme(plot.title = element_text(hjust = 0.5)) +
  theme(legend.position="bottom") +
  theme(plot.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(panel.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))+
  theme(legend.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))
ggsave("../plots/Diabetes_Incidence_by_Age.png", dpi = 1200, width = 4, height = 4, units = "in")
```


There does not appear to be a very large difference in diabetes status based on age.

# Diabetes vs Gender

```{r}
data %>% 
  ggplot(aes(x=Gender, fill=class)) +
  geom_bar() +
  theme_bw() +
  labs(title="Diabetes Incidence by Gender", fill="Diabetes", x="Gender", y="Frequency") +
  theme(plot.title = element_text(hjust = 0.5)) +
  theme(legend.position="bottom") +
  theme(plot.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(panel.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))+
  theme(legend.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))
ggsave("../plots/Diabetes_Incidence_by_Gender.png", dpi = 1200, width = 4, height = 4, units = "in")
```


## chi square test of independence

h0: Gender and diabetes status are independent

h1: not independent

```{r}
table(data$Gender, data$class)
```
```{r}
test <- chisq.test(table(data$Gender, data$class))
test
```

```{r}
test$expected
```
With a p-value < 2.2e-16, we reject our null hypothesis at all significance levels and conclude that gender and diabetes class are not independent.

# Diabetes vs Polyuria

```{r}
data %>% 
  ggplot(aes(x=Polyuria, fill=class)) +
  geom_bar() +
  theme_bw() +
  labs(title="Diabetes Incidence by Polyuria", fill="Diabetes", x="Polyuria", y="Frequency") +
  theme(plot.title = element_text(hjust = 0.5)) +
  theme(legend.position="bottom") +
  theme(plot.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(panel.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))+
  theme(legend.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))
ggsave("../plots/Diabetes_Incidence_by_Polyuria.png", dpi = 1200, width = 4, height = 4, units = "in")
```

## chi square test of independence

h0: Polyuria and diabetes status are independent

h1: not independent

```{r}
table(data$Polyuria, data$class)
```
```{r}
test <- chisq.test(table(data$Polyuria, data$class))
test
```

```{r}
test$expected
```
With a p-value < 2.2e-16, we reject our null hypothesis at all significance levels and conclude that polyuria and diabetes class are not independent.

# Diabetes vs Polydipsia

```{r}
data %>% 
  ggplot(aes(x=Polydipsia, fill=class)) +
  geom_bar() +
  theme_bw() +
  labs(title="Diabetes Incidence by Polydipsia", fill="Diabetes", x="Polydipsia", y="Frequency") +
  theme(plot.title = element_text(hjust = 0.5)) +
  theme(legend.position="bottom") +
  theme(plot.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(panel.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))+
  theme(legend.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))
ggsave("../plots/Diabetes_Incidence_by_Polydipsia.png", dpi = 1200, width = 4, height = 4, units = "in")
```

## chi square test of independence

h0: Polydipsia and diabetes status are independent

h1: not independent

```{r}
table(data$Polydipsia, data$class)
```

```{r}
test <- chisq.test(table(data$Polydipsia, data$class))
test
```

```{r}
test$expected
```
With a p-value < 2.2e-16, we reject our null hypothesis at all significance levels and conclude that polydipsia and diabetes class are not independent.

# Diabetes vs sudden.weight.loss 

```{r}
data %>% 
  ggplot(aes(x=`sudden weight loss`, fill=class)) +
  geom_bar() +
  theme_bw() +
  labs(title="Diabetes Incidence by Sudden Weight Loss", fill="Diabetes", x="Sudden Weight Loss", y="Frequency") +
  theme(plot.title = element_text(hjust = 0.5)) +
  theme(legend.position="bottom") +
  theme(plot.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(panel.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))+
  theme(legend.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))
ggsave("../plots/Diabetes_Incidence_by_Sudden_Weight_Loss.png", dpi = 1200, width = 4, height = 4, units = "in")
```

## chi square test of independence

h0: Sudden weight loss and diabetes status are independent

h1: not independent

```{r}
table(data$`sudden weight loss`, data$class)
```

```{r}
test <- chisq.test(table(data$`sudden weight loss`, data$class))
test
```

```{r}
test$expected
```
With a p-value < 2.2e-16, we reject our null hypothesis at all significance levels and conclude that sudden weight loss and diabetes class are not independent.

# Diabetes vs Partial Paresis

```{r}
data %>% 
  ggplot(aes(x=`partial paresis`, fill=class)) +
  geom_bar() +
  theme_bw() +
  labs(title="Diabetes Incidence by Partial Paresis", fill="Diabetes", x="Partial Paresis", y="Frequency") +
  theme(plot.title = element_text(hjust = 0.5)) +
  theme(legend.position="bottom") +
  theme(plot.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(panel.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2')) +
  theme(legend.background = element_rect(fill = '#f2f2f2', colour = '#f2f2f2'))
ggsave("../plots/Diabetes_Incidence_by_Partial_Paresis.png", dpi = 1200, width = 4, height = 4, units = "in")
```

## chi square test of independence

h0: Partial paresis and diabetes status are independent

h1: not independent

```{r}
table(data$`partial paresis`, data$class)
```

```{r}
test <- chisq.test(table(data$`partial paresis`, data$class))
test
```

```{r}
test$expected
```
With a p-value < 2.2e-16, we reject our null hypothesis at all significance levels and conclude that partial paresis and diabetes class are not independent.

```{r}
chi2 <- c()
variable <- c()
for (i in 2:(ncol(data)-1)){
  statistic <- chisq.test(table(data[[i]], data[[17]]))$statistic
  name <- colnames(data)[[i]]
  
  chi2 <- c(chi2, statistic)
  variable <- c(variable, name)
  
}

d <- data.frame(variable, chi2)

d %>% arrange(desc(chi2))
```