The goal of this project is to create a simple-to-use web application that makes monitoring the activity of state elected officials much easier for the average person.
This web application will allow users to identify state-level elected officials who represent them based on the user's address. Then users can follow links to web pages that have information on their state legislators. The web page includes basic information about the legislator in addition to other data a user may want to use to assess their behavior such as roll call votes, social media messages (Twitter, Facebook), campaign finance, etc.
You can visit an example of this application that focuses on Pennsylvania at StateRep.Me. We are currently working on cleaning up the project layout and making it easier to use in locales besides Pennsylvania.
The following instructions will set up and install a StateRep.Me instance on a VM. The project will not have data in the database, but will include at least serve the homepage. (We are currently working on automating data deployment as well). If you run into issues, check the troubleshooting instructions below or open an issue on the bug tracker.
The preferred installation also requires that you have Vagrant and Ansible. See the documentation for those tools for installation instructions.
Clone the repo (git clone https://github.com/staterepme/State-Gov-Tracker.git).
Install ansible and vagrant
sudo apt-get install vagrantpip install ansible
Copy and edit the example Ansible configuration file from the playbooks directory. cp playbooks/ansible_vars.example.yml playbooks/ansible_vars.yml
Open ansible_vars.yml in your favorite text editor and fill in the following settings.
django_secret_key: Your secret key - do not share this. See Django documentation.sunlight_key: StateRep.Me relies on some of the OpenStates APIs which require an API key. You can register here (don't worry it is free).cicero_keyandcicero_user: You will also need Cicero API keys through Azavea. This service powers the matching of state legislators to longitude and latitude coordinates. You can register for a free trial here.bing_key: We use the Bing Maps API to geocode addresses. To use this service requires registration. Instructions to obtain an API key go here.state: 2 letter postal abbreviation After specifing the settings you are ready to run thescripts/vagrant-ansible.shscript. This will create an Ubuntu 12.04 VM for StateRep.Me that will have a local instance of StateRep.Me running and available at http://localhost:6060/.
Lastly, you will need to SSH into the VM (e.g. vagrant ssh) and adjust some settings for the Postgresql database. Specifically, because Postgres does not compare ints and integers by default, this can cause some errors with the current set up. If you do the following, error messages should clear up:
- Sign into the DB as the superuser (postgres)
- Select the staterep database (
\c staterep) - Run the following command (
CREATE CAST (integer AS text) WITH INOUT AS IMPLICIT;)
We are working on making the initial loading and updates for data as easy as possible. Currently, loading legislators, loading their contact/office information, uploading twitter handles from a CSV, and downloading tweets are available through custom Django commands. Descriptions on how to get started for your state are below.
First you need to SSH into the VM (vagrant ssh), activate your virtualenv (source /usr/local/staterep/env/bin/activate), and then navigate to the app (cd /usr/local/staterep/app).
After you are in the correct directory you can begin loading data on the legislators for your state. To load a list of legislators and their offices into the database run python manage.py load_leg_data <2-letter state abbreviation>. The 2-letter abbreviation tells the command which state to query OpenStates for.
To set up twitter monitoring you need to create (or use the example file for PA) a CSV that has atleast 2 columns: legid and twitter that has the unique legislator id used in the Officials table and the Twitter handles for legislators. Each row in the CSV should be a legislator. Then run python manage.py load_twitter_handles <path/to/csv> where you specify the path to the CSV that you created in the previous step. This loads all twitter handles into the database. Lastly, to add actual tweets to the database you can run python manage.py update_tweets. This will download recent tweets for all the twitter handles you downloaded. Be aware - this is rate limited so if you do this too often you will hit the limit and have to wait to do it again.
That is currently all we have set up in an easy to use format. We will be adding similar functionality for press-releases, votes, and facebook posts as well.
Let us know if you run into any other issues!
There are some known issues with the Ansible and Vagrant scripts depending on OS.
When running scripts/vagrant-ansible.sh on some systems the location for the Vagrant SSH key is not correctly passed to the ansible-playbook script. To solve this issue you need to hard code the location of your vagrant insecure_private_key on line 10.
The Ansible script also installs the requirements for StateRep.Me. There is a known issue where the script hangs when running pip install -r requirements.txt for the project. If this happens, follow the following steps.
- SSH into your vagrant vm with
vagrant ssh - Activate the virtualenv for the project (should be located in your
staterepfolder.) - Change into the
staterep/app/directory - Run
pip install -r requirements.txtand wait for dependencies to install - Exit from the VM and try running the
scripts/vagrant-ansible.shfile again.
For geo-coding addresses and matching them to state legislative districts we use Cicero from Azavea. We also formed an initial list of Twitter and Facebook usernames for state legislators through Cicero.
The list of members, voting history, offices, and contact information is obtained through the Sunlight Foundation's OpenStates.
We gather press releases ourselves using webscraping scripts. Those files can be found in the /data_scripts subfolder.
This website would not be possible if not for a number of great packages written for Django.
secretballot is a package written by James Turk for Sunlight Labs. This version is slightly modified to work with our code. To see the original code, check it out on github.
StateRep.Me would not be possible without the help of many individuals and organizations who have provided advice, answered questions, and helped with some of the coding.
Jason Blanchard
Chris Brown
Joshua Darr
Lauren Gilchrist
Georgia Guthrie
Adam Hinz
Bennet Huber
Charlie Milner
Christopher Nies
Andrew Thompson
Nick Weingartner
State-Gov-Tracker is an open-source, free application, covered under the GPLv3 License. Please fork and experiment with it!