mixed-content-crawler

Find mixed content errors on a site by crawling it with Puppeteer

Installation

$ npm install mixed-content-crawler

CLI Demo

CLI Usage

$ mixed-content-crawler

  Find mixed content errors on a site by crawling it with Puppeteer

  Usage

    $ mixed-content-crawler <url>

  Example

    $ mixed-content-crawler https://buster.neocities.org/crwlr/

$ mixed-content-crawler https://buster.neocities.org/crwlr/
resolved: 200 https://buster.neocities.org/crwlr/
resolved: 200 https://buster.neocities.org/crwlr/other.html
blocked-mixed-content: http://http.badssl.com/icons/icon-red.png
resolved: 200 https://buster.neocities.org/crwlr/mixed-content.html
resolved: 404 https://buster.neocities.org/crwlr/missing.html
requestfailed: https://buster.neocities.org/crwlr/dummy.pdf {"errorText":"net::ERR_ABORTED"}

Module Usage

const mixedContentCrawler = require('mixed-content-crawler');

const results = await mixedContentCrawler('https://buster.neocities.org/crwlr/');
results.forEach((mixedRequests, page) => {
  console.log(page);
  for (let r of mixedRequests) {
    console.log('->', r);
  }
});
// https://buster.neocities.org/crwlr/mixed-content.html
// -> http://http.badssl.com/icons/icon-red.png

TODO

• [] Allow for ignoring location.hash when determinining unique URLs
• [] Better documentation

License

ISC © Buster Collings