Skip to content

How do I configure a Qanary component using Docker?

Jump to bottom
Andreas Both edited this page Jan 5, 2023 · 4 revisions

tl;dr

In the simplest case you can start all relevant services (pipeline, components and triplestore) in the same Docker network (most likely host) and connect the services using default parameters only.

However, for most production use cases, the parameters used for networking need to be changed.

This Guide will go over different configurations that might be necessary for a component.

Required Prior Knowledge

How to configure the required parameters

There are three main properties which should be configured, depending on the context:

  • spring.boot.admin.client.url: the Qanary pipeline endpoint that the Qanary component connects to
  • server.port: the port on which the Qanary component will be available inside the Docker container
  • spring.boot.admin.client.instance.service-base-url: the address that is used to register the component at the pipeline

To override the default values set in application.properties the use of environment variables is encouraged. This is further described in this guide: How do I configure Qanary services using Docker containers

Remark: The specific configurations might differ in a development or production environment.

Development

The default Qanary pipeline endpoint to which a component connects is http://localhost:8080.

In order for localhost to be available for networking between components and the pipeline, both need to be started in the same Docker network. Otherwise, services will not be available on http://localhost inside the different docker containers.

The simplest way to achieve this is to start the Qanary pipeline and components in the host network:

docker run --net host <image>:<tag>

To avoid port conflicts when running multiple components, you might have to override the component port as well:

For example, to make a component available on port 8000 use the following command:

docker run --net host -e SERVER_PORT=8000 example-component:latest

Production

Port configuration

To register at the Qanary pipeline endpoint, a component uses the property server.port. Because of this, the published port must be the same as the exposed port! For example, if a Qanary component is supposed to be published on port 8000 (on the Internet) but the default port is 5555 (inside the Docker container) its server.port parameter has to be set to 8000, s.t., the Qanary pipeline can contact the component from the Internet.

I would not be enough to simply map port 5555 (internal) to 8000 (external)!

docker run -e SERVER_PORT=8000 -p 8000:8000 example-component:latest

Host configuration

In a production environment, it might not be possible to start all components and the pipeline in one host network. In such a case, the pipeline will not be available on http://localhost. For a component to connect to the Qanary pipeline, the property spring.boot.admin.client.url has to be set to the actual pipeline endpoint (host and port). Typically, that would be an IP address or public URL.

In turn, the pipeline also needs to be able to connect to the component. Therefore the component needs to be hosted on the Internet as well. This can be achieved by setting the parameter spring.boot.admin.client.instance.service-base-url to the IP address or public URL that your component should be available at. To make your component accessible from the Internet, you may want to use ngrok.

docker run \
-e SPRING_BOOT_ADMIN_CLIENT_URL=http://example.pipeline:8080 \
-e SPRING_BOOT_ADMIN_CLIENT_INSTANCE_SERVICE-BASE-URL=http://example.component:5555 \
-p 5555:5555 \
example-component:latest

Note: The use of ngrok is not described in this guide. Please see the official documentation: https://ngrok.com/docs

Docker-compose

The configurations shown above using the standard docker run command can easily be applied in a docker-compose.yml file. To start the Qanary component example-component (we assume here it was implemented using the standard Java Spring Boot component template provided in the Qanary repository) that is listening on port 5000 and is connecting to the pipeline endpoint at http://example.pipeline:8000. Additionally, it might be useful to define a custom name (e.g., if you run multiple components with different configurations). The configuration would look like this:

version: "3.5"
services:
    example-component:
      image: example-component:latest
      environment:
        - "SPRING_BOOT_ADMIN_CLIENT_URL=http://example.pipeline:8000"
        - "SERVER_PORT=5000"
        - "SPRING_BOOT_ADMIN_CLIENT_INSTANCE_SERVICE-BASE-URL=http://example.component:5000"
        - "SPRING_APPLICATION_NAME=My-Qanary-component"
      ports:
        - "5000:5000"

Notes:

  • The structure of the docker-compose.yml is very similar while using Qanary components implemented in different programming languages / frameworks (e.g., see this Python component).
  • If pipeline and components are started on the same host network, you might define a component similar to this example:
version: "3.5"
services:
    example-component:
      image: example-component:latest
      environment:
        - "SPRING_BOOT_ADMIN_CLIENT_URL=http://localhost:8000"
        - "SERVER_PORT=5000"
      network_mode: host

Summary

In this tutorial, you have learned how to run and configure Qanary components using Docker, and connect them to a Qanary pipeline instance.

Clone this wiki locally