blocky lists updater

Blocky lists updater updates lists without restarting blocky DNS.

Introduction

blocky is a DNS proxy that blocks DNS queries based on external lists. It reads blocking lists and allowing list from a configuration file. Every time we want to add a list or a domain to the blocking or allowing lists, we must edit the configuration file, then restart blocky. This may lead to DNS down.

Block lists updater runs as a companion service, allowing you to edit lists and domains in lists without restarting blocky.

Usage

You can run lists updater via Docker, see the example docker-compose.yml files.

It is a set of scripts, so you can also run it on the host as well .src/entrypoint.sh. It starts a infinite loop and watches the change of sources.

How it works

• Lists updater downloads and concatenates multiple lists into a single file based on a source file that contains URLs to tell where to download lists.
• Lists updater watches changes of the source files and/or runs downloading periodically.
• Lists updater also watches files of lists of domains. No downloads in this case.
• Lists updater ships with a static file server, allowing blocky to read the downloaded files.
• List updater sends refresh requests to blocky.

Lists updater is a set of shell scripts. The entrypoint is the main function inside entrypoint.sh. You can run Lists updater as a container as well, for example

docker run -d -p 8080:8080 --env "BLU_BLOCKY_URL=http://blocky:4000" -v ./my_sources:/sources -v ./my_domains:/web/watch shizunge/blocky-lists-updater

Without lists updater

Here is a snippet of a blocky configuration file. We use this as an example to demostarte how to translate an existing blocky configuration to lists updater.

blocking:
  denylists:
    group-one:
      - https://github.com/StevenBlack/hosts
      - https://github.com/Perflyst/PiHoleBlocklist
    group-two:
      - |
        blocked.example.com
        blocked2.example.com
  clientGroupsBlock:
    default:
      - group-one
      - group-two

Using lists updater to manage lists

Create a new source file for group-one, which contains the lists, for example /sources/group-one.txt.

# Comments are supported
https://github.com/StevenBlack/hosts
https://github.com/Perflyst/PiHoleBlocklist

Then You modify the blocky configuration file for group-one to read the aggregate list generated by Lists updater via the static file server from lists updater instead of two separated lists. The file name of the aggregate list will be same as the source file, but it goes to the destination folder e.g. /web/downloaded.

blocking:
  denylists:
    group-one:
      - http://lists-updater:8080/downloaded/group-one.txt

Start Lists updater, it reads the source file, downloads contents of these two lists, then concatenates them to a single file. The result file will have the same name group-one.txt, but it will go to the destination folder e.g. /web/downloaded. The static file server will serves files under /web for blocky to read.

For now on, if you want to add or remove lists for group-one, you no longer need to modify the blocky configuration file. Instead you edit the source file /sources/group-one.txt. Lists updater watches the changes of the source files, and starts new downloads following refresh requests to blocky. Lists updater can also download the lists periodically.

Using lists updater to manage domains

Lists updater also watches files that contain lists of domains besides watching the source files. In this case, you maintain local copies of the lists of domains, no download occurs.

Create a new file for lists of domains, e.g. /web/watch/group-two.txt

blocked.example.com
blocked2.example.com

Then You modify the blocky configuration file for group-two to read this file via the static file server.

blocking:
  denylists:
    group-two:
      - http://lists-updater:8080/watch/group-two.txt

When you add or remove domains from group-two.txt, lists updater watches the changes and sends a refresh request to blocky.

Configurations

You can configure the most behaviors of lists updater via environment variables.

Environment Variable	Default	Description
BLU_BLOCKY_URL		Enable refresh request to blocky. It should be the base URL of blocky. Use an empty string to disable refresh requests. Limitation: the updater will send requests to a single blocky instance today.
BLU_DESTINATION_FOLDER	/web/downloaded	The location of aggregate lists. This should be under BLU_WEB_FOLDER thus blocky can read the files via the static file server.
BLU_INITIAL_DELAY_SECONDS	0	Delay in seconds before the first download.
BLU_INTERVAL_SECONDS	86400	Interval between two downloads. Set to 0 disable periodically downloads, then downloads will occur only at source changes.
BLU_LOG_LEVEL	INFO	Control how many logs generated by Gantry. Valid values are NONE, ERROR, WARN, INFO, DEBUG.
BLU_NODE_NAME		Add node name to logs.
BLU_NOTIFICATION_APPRISE_URL		Enable notification after each refresh with apprise. This must point to the notification endpoint (e.g. http://apprise:8000/notify). Use an empty string to disable notification.
BLU_POST_DOWNLOAD_CMD		A command or function running after downloading a list. The first argument will be the path to the downloaded file, i.e. your command will be eval as your_command <file_path>. The command will apply to all downloaded lists. This can be used to fix problems in the lists before the upstream maintainer fixing it.
BLU_SOURCES_FOLDER	/sources	The location to read sources lists. The files contains URLs where to download the lists of domains. Use an empty string to disable watching.
BLU_WATCH_FOLDER	/web/watch	The location of user defined lists of domains. lists updater watches changes in this folder and sends refresh requests to blocky. This should be under BLU_WEB_FOLDER thus blocky can read the files via the static file server. Use an empty string to disable watching.
BLU_WEB_FOLDER	/web	The location that the static file server serves. Use an empty string to disable the file server.
BLU_WEB_PORT	8080	The port that static file server listens to.

Caveats

• Need a third party file editor.

  Blocky lists updater bases on a set of shell scripts. It does not provide any UI to edit lists. You need an additional service for lists editing. See the examples.

• No group management

  You still need to modify the blocky configuration file to edit groups.

Contacts

If you have any problems or questions, please contact me through a GitHub issue.

Related projects

You may also like my blocky Postgresql dashboard.