Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zarthcode custom table scraper #193

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

zarthcode
Copy link

My router has a locally accessible webpage that presents the public ipv6 and ipv4 addresses in a table format. This patch allows for accessing an arbitrary url and looking for the element after given table headers using the config variables scraper_url, scraper_ipv4_label, and scraper_ipv6_label.

e.g. for an ATT BGW320-500 router, the following config.json will acquire the ipv4 and v6 addresses:

{
  "cloudflare": [
    {
      "authentication": {
        "api_token": "your secret token"
      },
      "zone_id": "your zone id",
      "subdomains": [
        {
          "name": "www",
          "proxied": true
        }
      ]
    }
  ],
  "scrape_url": "http://192.168.1.254/cgi-bin/broadbandstatistics.ha",
  "scrape_ipv4_label": "Broadband IPv4 Address",
  "scrape_ipv6_label": "Global Unicast IPv6 Address",
  "a": true,
  "aaaa": true,
  "purgeUnknownRecords": false,
  "ttl": 300
}

This is a much less intrusive way to acquire ip information - if your gateway/router provides this information readily.

zarthcode added 2 commits June 2, 2024 14:55
My router has a locally accessible webpage that presents the public ipv6 and ipv4 addresses in a table format.  This patch allows for accessing an arbitrary url and looking for the element after given table headers using the config variables scraper_url, scraper_ipv4_label, and scraper_ipv6_label.

e.g. for an ATT BGW320-500 router, the following config.json will acquire the ipv4 and v6 addresses:

{
  "cloudflare": [
    {
      "authentication": {
        "api_token": "your secret token"
      },
      "zone_id": "your zone id",
      "subdomains": [
        {
          "name": "www",
          "proxied": true
        }
      ]
    }
  ],
  "scrape_url": "http://192.168.1.254/cgi-bin/broadbandstatistics.ha",
  "scrape_ipv4_label": "Broadband IPv4 Address",
  "scrape_ipv6_label": "Global Unicast IPv6 Address",
  "a": true,
  "aaaa": true,
  "purgeUnknownRecords": false,
  "ttl": 300
}
Added beautifulsoup4 requirement for use in scraper
@zarthcode zarthcode marked this pull request as ready for review June 6, 2024 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant