-
Notifications
You must be signed in to change notification settings - Fork 1
Add more sites to the scrapy start_urls #6
Comments
@groovecoder i made a REALLY good list. ...and I thought I emailed it to you? but now i can't find it. :( |
Yeah, I thought I saw that somewhere too ... but I can't find it in email. 😢 Re-creating on GH issue will keep it public and permanent at least. |
nextcity.org cnt.org/blog |
I found the file to edit, and can easily do that if you want me to, this weekend. Lemme know. |
found another great one. |
Another one: |
With just nextcity.org and govtech.com, I was able to crawl and index 18.8k articles/pages. @chimchim237 knows the bigger list of real sites to crawl.
The text was updated successfully, but these errors were encountered: