Skip to content

Latest commit

 

History

History
41 lines (37 loc) · 793 Bytes

README.md

File metadata and controls

41 lines (37 loc) · 793 Bytes
Raw

Shopee Products Scraper

Scrapes shopee.com products per category and saves it to a Postgres database

Requirements

  • Python 3
  • Google Chrome
  • Selenium Chrome Driver compatible with installed Google Chrome version. Download here

Technology

  • Python 3
  • Selenium
  • Multiprocessing
  • Docker
  • PostgreSQL

Setup

Create virtual environment
python3 -m venv venv
Install depedencies
pip install -r requirements.txt
Setup PostgreSQL database configuration and adapt to your needs .env
cp .env.example .env
Run the application
python main.py

Setup with dockerize PostgreSQL database (Alternative)

docker-compose up -d
Run the application
python main.py