web-scraping.md

Bookmarks tagged [web-scraping]

^{www.codever.land/bookmarks/t/web-scraping}

GitHub - jsdom/jsdom

^{https://github.com/jsdom/jsdom}

jsdom is a pure-JavaScript implementation of many web standards, notably the WHATWG DOM and HTML Standards, for use with Node...

• tags: web-scraping, tools, dom, node.js, javascript
• source code

---

Advanced web spidering with Puppeteer

^{https://blog.kowalczyk.info/article/ea07db1b9bff415ab180b0525f3898f6/advanced-web-spidering-with-pup...}

Puppeteer is a node.js library that makes it easy to do advanced web scraping and spidering. Older generation of web scraping and spidering tools would grab and analyze HTML pages as returned by a web...

• tags: node.js, puppeteer, web-scraping

---

cola

^{https://github.com/chineking/cola}

A distributed crawling framework.

• tags: python, web-crawling, web-scraping
• source code

---

feedparser

^{https://pythonhosted.org/feedparser/}

Universal feed parser.

• tags: python, web-crawling, web-scraping

---

grab

^{https://github.com/lorien/grab}

Site scraping framework.

• tags: python, web-crawling, web-scraping
• source code

---

MechanicalSoup

^{https://github.com/MechanicalSoup/MechanicalSoup}

A Python library for automating interaction with websites.

• tags: python, web-crawling, web-scraping
• source code

---

portia

^{https://github.com/scrapinghub/portia}

Visual scraping for Scrapy.

• tags: python, web-crawling, web-scraping
• source code

---

pyspider

^{https://github.com/binux/pyspider}

A powerful spider system.

• tags: python, web-crawling, web-scraping
• source code

---

robobrowser

^{https://github.com/jmcarp/robobrowser}

A simple, Pythonic library for browsing the web without a standalone web browser.

• tags: python, web-crawling, web-scraping
• source code

---

scrapy

^{https://scrapy.org/}

A fast high-level screen scraping and web crawling framework.

• tags: python, web-crawling, web-scraping
• source code

---