This script provides an accurate report on the current Google indexation status for a given url. It displays Indexed or Not Indexed on a results.csv file.
The script is able to verify an unlimited number of URLs with any kind of problematic characters: parameters, encoding, reserved characters, unsafe characters, different alphabets – if Google has indexed it, our script will find it. To find read more read our article at Builtvisible | Scaling Google indexation checks with Node.js
Google does not allow automated queries according to their Terms of Service. So, if you use our script, please use it responsibly.
Comparing Google indexation checker with other tools available:
Download zip or clone repo:
git clone https://github.com/alvaro-escalante/google-index-checker.gitWith npm
npm installOr Yarn
yarn installThe tool uses ScraperAPI as a proxy to be able to make multiple request without being blocked.
Set up an account with scraperapi.com to get your api key.
Insert your API key on the .env_sample and rename this file .env this file is excluded from the repo for security reasons.
Depending on your plan you will have more or less concurrent request allowed, the script will automatcally make a request to ScraperAPI to check the max concurrent request for the account.
You can use this file for testing: urls.csv
Place the urls.csv file on the main folder.
Note: Make sure urls containing commas have double quotes around them
npm startyarn startIn the ternimal:
And finally a results.csv will be created with the indexation report.
| URL | Status |
|---|---|
| https://builtvisible.com/ | Indexed |
| https://www.megafaaaaaakeurl.com/no-way | Not Indexed |
| http://thisoneisanotherfakeurlfortesting.co.uk/ | Not Indexed |
| https://descubriendoelviaje.es/ | Indexed |
| http://www.gruppo.mps.it/ap_trasparenzaweb/Documenti%5C103029489.pdf | Indexed |
| https://www.swing-autovermietung.de/#!ueberuns | Indexed |
Any errors will be automatically recycled and run again
ScraperAPI will not consider errors as requests and won't count them as credits.
Note: Make sure the provided csv with the urls it's named
urls.csv
| Name | Description |
|---|---|
| axios | Promise based HTTP client for the browser and node.js |
| chalk | Terminal string styling done right |
| csv-parser | Streaming CSV parser that aims for maximum speed as well as compatibility with the csv-spectrum test suite |
| sanitize-html | Provides a simple HTML sanitizer with a clear API |
| dotenv | Dotenv is a zero-dependency module that loads environment variables |