-
Notifications
You must be signed in to change notification settings - Fork 0
sbcwaste
License
kolonuk/sbcwaste.top
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
sbcwaste What this does: Using chromdp, (reasons become apparent), this interrogates the Swindon Borough Council residential waste collection, and extracts the waste collectioon types, along with the next collection dates. There is no API for collections available directly from the site, so HTML parsing it is. There are a number of issues with the SBC site becuase it is very badly designed and uses IIS (probably?) ASP, wiht loads of overkill JS and CSS. Issues with the SBC site I countered: 1. The HTML is returned complete with the dates and no obvious queryable API, hence the HTML parsing. 2. The images of the waste types are only available after the javascript has been run, so the use of a compiled HTML/webpage is required, hence the use of chromedp rather than collyv2 3. The address lookup is required because, strangely, the address is just passed as a URL text parameter to the final HTML page, so, yes, you can have any address you like, or blank. Thats a bad design. 4. The output here is in one of two formats, JSON or iCalendar (ics). The format is controlled via the format section of the path parameter. 5. The ics RFC does not allow for the use of calendar entry icons, so the image is included as an attachment in case your calendar app does manage to somehow display it. Usage: 1. Find your UPRN (Unique Property Reference) by going to https://maps.swindon.gov.uk/ and searching for your address. The UPRN is the number that shows up on the left hand side once you have done a search. 2. Use the UPRN in the URL path, e.g. http://localhost:8080/[UPRN]/[ics|json] You should be able to just plug the URL with the ics as the format into your calendar app. The JSON option is if you want to do something else with the data. 3. I will provide a public URL for this at somepoint, but for now you can run it locally. I aim to get a caching service in place to avoid hitting the SBC website too much - I think a day is enough. Dev environment Install gcloud cli: https://cloud.google.com/sdk/docs/install-sdk (just follow the instructions for installation at this stage) Whilst the install is progressing, now create a google cloud project: https://console.cloud.google.com/projectcreate (You can see the main dashboard here: https://console.cloud.google.com/projectselector2/home/dashboard) And then enable the Artifact Registy API (make sure the project is selected from the drop down at the top): https://console.cloud.google.com/apis/library/artifactregistry.googleapis.com You can see the Cloud Run instance here: https://console.cloud.google.com/run You can see the Artifactory repo here: https://console.cloud.google.com/artifacts Deploy Github actions 1. Github actions workflow availabe in repo - Change params to suit. 2. Run deploy-githubactions.sh - Change params and run, making sure that gcloud sdk is installed and authenticated first (see the second part of the setup script). Make sure you also create the secrets. The permissions the script creates are most likelu a little overkill, but I don't have the time (or patience!) to strip it down. 3. Once you push this to your own repo, the github actions will run, and it should deploy using your google cloud resources.
About
sbcwaste
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published