Kenya News Scrapper

It gets news from Tuko, Capital FM, The Standard, Daily Nation and The Star, then returns top news from each and a short summary of each article.

There is a cli version and a flask web api version.

Screenshots

Prerequisites

Python 3 (3.6.1 used in the project)

Installing

pip install requests beautifulsoup4 A step by step series of examples that tell you have to get a development env running in unix systems

1. Clone the repo

2. Setting up a virtual environment, Python 3 virtual env docs

python3 -m venv /path/to/new/virtual/environment

4. Activate the Virtual env

source /path/to/new/virtual/environment/bin/activate

5. cd to the cloned directory.(The one with requirements.txt)

6. Install the requirements from requirements.txt

pip install -r requirements.txt

There are two ways to run this script

Run the cli version

python scheduler_script.py get_news_cli

OR

First scrape the news sources to create and fill up the database before running the flask version

python scheduler_script.py scrape_news

then...

Run app.py to start web server (you can check end points at app.py)

python app.py

Note: you can run python scheduler_script.py delete_old_news to delete news more that 48 hours old

Built With

• Python HTTP Requests - HTTP for Humans
• Beautiful Soup 4 - html parser
• Me ¯\_(ツ)_/¯

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details