Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging files #24

Open
wants to merge 101 commits into
base: climate-data-analysis
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
fc261c8
Merge pull request #21 from uchicago-capp122-spring23/climate-data-an…
hvpachisia Mar 4, 2023
beab7eb
adding exploratory data analysis for counties in disaster data compar…
Mar 4, 2023
05a987f
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 4, 2023
4009c52
top 5 events per state by federal obligations
Mar 4, 2023
b255886
Merge pull request #22 from uchicago-capp122-spring23/top_5_public_as…
jackieglasheen Mar 4, 2023
158f99f
restructure of census api query
Mar 4, 2023
8928378
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 4, 2023
2820091
Delete Census_State_codes.txt
hvpachisia Mar 4, 2023
aa861d0
Delete PublicAssistanceFundedProjectsSummaries.csv
hvpachisia Mar 4, 2023
dcec196
Delete bring_it_all_together_test.py
hvpachisia Mar 4, 2023
944b1eb
Delete census_api_query.py
hvpachisia Mar 4, 2023
35f81d8
Delete census_demographic_data.csv
hvpachisia Mar 4, 2023
10cdede
Delete climate_data.ipynb
hvpachisia Mar 4, 2023
06c9f6b
Delete climate_datasets.py
hvpachisia Mar 4, 2023
e279674
Delete climate_disasters_2000_2023_clean.csv
hvpachisia Mar 4, 2023
4c8f0b3
Delete disaster_declarations.csv
hvpachisia Mar 4, 2023
63dd24c
Delete economic_impact.ipynb
hvpachisia Mar 4, 2023
560bffd
Delete economic_impact.py
hvpachisia Mar 4, 2023
421670e
Delete eda_census_disaster_data_merged.py
hvpachisia Mar 4, 2023
32ad12e
Delete exploratory_data_analysis.py
hvpachisia Mar 4, 2023
434807b
Delete state_pop_2000_2009.csv
hvpachisia Mar 4, 2023
6787c9a
Delete state_pop_2010_2019.csv
hvpachisia Mar 4, 2023
df886e6
Delete voting_record.py
hvpachisia Mar 4, 2023
f341800
Delete state_pop_2020_2022.csv
hvpachisia Mar 4, 2023
3ca864c
Merge pull request #23 from uchicago-capp122-spring23/climate-data-an…
hvpachisia Mar 4, 2023
b3f4b8f
edits to name variables in census datasets
Mar 4, 2023
5efdbd1
Merge pull request #25 from uchicago-capp122-spring23/climate-data-an…
hvpachisia Mar 5, 2023
12a38bf
Create __init__.py
hvpachisia Mar 5, 2023
8e57dbe
Create __init__.py
hvpachisia Mar 5, 2023
9e0a5c9
minor edits to census_api_query file
Mar 5, 2023
cc254bb
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 5, 2023
f3a1a69
updating repository withe forrect file
Mar 5, 2023
f316916
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 5, 2023
1393479
fix file overwrite
Mar 5, 2023
d1e1759
Added poetry files
Mar 5, 2023
ea9fae5
Merge branch 'main' of https://github.com/uchicago-capp122-spring23/3…
Mar 5, 2023
70d5c6e
Create .gitkeep
hvpachisia Mar 5, 2023
60ee36f
Create add_traces.py
hvpachisia Mar 5, 2023
a121dd4
Create bubble_map.py
hvpachisia Mar 5, 2023
29fb6aa
Create final_viz.py
hvpachisia Mar 5, 2023
e2bd4f5
add variabl to web scrape and clean up code
Mar 6, 2023
cb89070
Merge pull request #27 from uchicago-capp122-spring23/code_cleanup
jackieglasheen Mar 6, 2023
243f7eb
added alpha state codes to census
Mar 6, 2023
5382f71
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 6, 2023
5d1ba78
Create __init__.py
hvpachisia Mar 6, 2023
19a72ba
Push current changes to final viz
Mar 6, 2023
f3d5348
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 6, 2023
f448130
updated pyproject.toml
Mar 6, 2023
12b97a9
Run from command line
Mar 6, 2023
454d2df
Working interactive graphs
Mar 6, 2023
b493420
census graphs
Mar 6, 2023
22fc277
census graphs + titles added
Mar 6, 2023
c4877f3
added voting graph
Mar 6, 2023
a5d71d0
Updated readme
Mar 6, 2023
015624a
Updated readme
Mar 6, 2023
28c034e
Updated readme
Mar 6, 2023
db38354
Updated readme
Mar 6, 2023
36f42b3
Updated readme
Mar 6, 2023
91cd4a3
clean up economic impact and investigate duplicates
Mar 6, 2023
b3c996b
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 6, 2023
8f35b22
design stuff
Mar 6, 2023
cd01048
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 6, 2023
1335331
Changed EDA for climate
Mar 6, 2023
fdcbd9d
Merge branch 'main' of https://github.com/uchicago-capp122-spring23/3…
Mar 6, 2023
e0b225c
Changed EDA for climate
Mar 6, 2023
1a6e5c3
Changed EDA for climate
Mar 6, 2023
9003b6b
Text and titles added
Mar 6, 2023
153942e
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 6, 2023
2c692d9
Last style edits
Mar 6, 2023
6dd154e
edits
Mar 6, 2023
12c4cbf
Updated ReadMe
Mar 6, 2023
5cad3ca
Updated ReadMe
Mar 6, 2023
7103f96
Final edits
Mar 6, 2023
e930677
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 6, 2023
2cb775e
Update README.md
hvpachisia Mar 6, 2023
5a7f9fa
Updated ReadMe
Mar 6, 2023
a1343fc
Merge branch 'main' of https://github.com/uchicago-capp122-spring23/3…
Mar 6, 2023
2e79765
Updated ReadMe
Mar 6, 2023
1d592ce
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 7, 2023
4419969
rounding
Mar 7, 2023
2af3f22
rounding
Mar 7, 2023
440f07f
added text as a separate file
Mar 7, 2023
ee96823
Updated climate files
Mar 7, 2023
e412ea6
Minor changes to climate
Mar 7, 2023
91e23aa
Minor changes to climate
Mar 7, 2023
a7e4791
edits to make API key private
Mar 7, 2023
a3671a6
minor edit to api query file for making api key private
Mar 7, 2023
2724af5
adding an eda file that got lost in our big merge
Mar 7, 2023
0334f60
Final Data Viz Updates
Mar 7, 2023
64ac487
Final updates
Mar 7, 2023
80e15a3
Delete data_viz_text.py
shwetha-srinivasan Mar 7, 2023
e1581a7
Delete add_traces.py
shwetha-srinivasan Mar 7, 2023
e27a99f
Delete bubble_map.py
shwetha-srinivasan Mar 7, 2023
1909ef7
Formatting changes only
Mar 8, 2023
f427482
shwetha's edits to tables in final_viz code
Mar 8, 2023
d28d206
Merge branch 'main' of github.com:uchicago-capp122-spring23/30122-pro…
Mar 8, 2023
5738818
cleaned data_viz file
Mar 8, 2023
60fb87e
Final changes
Mar 8, 2023
f482bf8
Add files via upload
jkyeaton Mar 8, 2023
6ec4fbe
Delete proj-paper.pdf
jkyeaton Mar 8, 2023
0f446ba
Add files via upload
jkyeaton Mar 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 22 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,32 @@ Final Project for CAPP122

### Team

Jackie Glasheen: jglasheen
Jackie Glasheen, Harsh Vardhan Pachisia, Shwetha Srinivasan, and Jennifer Yeaton

Harsh Vardhan Pachisia: harshpachisia
### Description

Shwetha Srinivasan: shwetha
**42.63 %** of natural disasters in the United States since 1980 have taken place over the last 12 years. It is evident that climate change and global warming are causing extreme weather disasters to occur with more frequency. The increase in natural disaster declarations over the past several decades has led to the federal government spending more money on disaster relief. Due to the increase in frequency and cost of such disasters, a Congressional Research Service [paper](https://sgp.fas.org/crs/homesec/R45484.pdf) from January 2022 suggests that Congress may consider limiting federal disaster relief spending.

Jennifer Yeaton: jkyeaton
Disasters impact different geographic regions, affecting specific states and communities disproportionately. Through this dashboard, we hope to shed light on the increase in disaster frequency and cost nationwide. We highlight the specific situation in each state, where its political stance is on climate, and showcase the top ten counties that are at most risk based on socio-economic factors and require long term policy action.

### Description
- Why this problem matters. What did we do about it?
- Major insights.
- How can this be helpful?
This dashboard was built using Python and Dash, using [climate disaster](https://www.fema.gov/openfema-data-page/disaster-declarations-summaries-v2) and [public assistance](https://www.fema.gov/openfema-data-page/public-assistance-funded-project-summaries-v1) data from the Federal Emergency Management Agency, scraped data of the voting patterns on the Climate IRA bill, and the [Census API](https://www.census.gov/data/developers/data-sets.html) for county-level socio-economic breakdown.

### Runing the project
- Add steps that the person needes to follow to run the entire project.

### Example Insights (Add charts and screenshots of graphics).
-
This project can be run using Poetry. The steps are outlined below.

1. Make a clone of the project repository

2. Go to the project directory: `cd 30122-project-snow-lm`

3. From the directory install virtual environment and dependencies: `poetry install`

4. Activate the virtual environment: `poetry shell`

5. Create an environment variable for the U.S. Census API key: `export CENSUS_API_KEY=YOUR_KEY_HERE`

6. Run the project: `python3 -m snowlm`

It will ask you to open the dash application in the browser to see the interactive dashboard.

Pressing Control+C in your terminal will close the application.
1,139 changes: 1,102 additions & 37 deletions poetry.lock

Large diffs are not rendered by default.

Binary file added proj-paper.pdf
Binary file not shown.
12 changes: 11 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,24 @@ version = "0.1.0"
description = ""
authors = ["Harsh Vardhan Pachisia <[email protected]>"]
readme = "README.md"
packages = [{include = "30122_project_snow_lm"}]
packages = [{include = "snowlm"}]

[tool.poetry.dependencies]
python = "^3.8"
pandas = "^1.5.3"
requests = "^2.28.2"
lxml = "^4.9.2"
plotly-express = "^0.4.1"
dash = "^2.8.1"
plotly-geo = "^1.0.0"
geopandas = "^0.12.2"
pyshp = "^2.3.1"
dash-bootstrap-components = "^1.4.0"
cssselect = "^1.2.0"
dash-table = "^5.0.0"
dash-bootstrap-templates = "^1.0.8"
black = "^23.1.0"
pylint = "^2.16.4"


[build-system]
Expand Down
50 changes: 50 additions & 0 deletions snowlm/EDA/census_disaster_data_merged_eda.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
'''
CAPP 30122
Team: Snow Laughing Matter
Primary Author: Jennifer Yeaton

Purpose: Exploratory data analysis of county list with disaster data and census data.
Merging list of counties with disaster data into census dataset.
'''

import pandas as pd
import pprint
from snowlm.scrape_api.census_api_query import api_query
from snowlm.data.climate import get_cleaned_data, counties_affected_by_disasters


def merge_data_census_disaster():
'''
Merge list of counties with disaster data into census dataset.

Inputs:
None

Returns:
Pandas dataframe with census and disaster data merged together.
'''

full_census_df = api_query()
cleaned_data_df = get_cleaned_data("disaster_declarations.csv")

counties_with_disaster_list = counties_affected_by_disasters(get_cleaned_data("disaster_declarations.csv", only_2000_onwards = True))
column_name = ["state_and_county_code"]
counties_with_disaster_df = pd.DataFrame(columns = column_name, data = counties_with_disaster_list)

#Check data types for county_code:
print(full_census_df.dtypes)
print(counties_with_disaster_df.dtypes)

#Look at head of each dataset:
print(full_census_df.head)
print(counties_with_disaster_df.head)

#Add binary variable for whether disaster data exists:
counties_with_disaster_df["county_has_disaster_data"] = 1

#Merge datasets
census_data_w_disaster_counties_df = full_census_df.merge(counties_with_disaster_df,
how = "left", on = "state_and_county_code")

#Return counties_with_disaster_df, full_census_df
return census_data_w_disaster_counties_df
Loading