Configuration items

[CplPunishment]

Does 'go-dnscollector' support specifying outbound interfaces? My systems have two redundant upstream connections that I aggregate in iptables to a loopback and I'd really like to specify that.

I'm trying to send send a lot of data from several dozen unbound-1.17.1 recursive DNS servers to a 'dnstap' collector that I'm using dnscollector to downsample ever 100th query, and at some point the buffer gets full:

processor=dnstap#2 logger [...] buffer is full, 2361 packet(s) dropped

Ideas?

[dmachard]

a lot of data

How much ?

Can you share your config ? Have you try to play with the following settings?

#   # Sets the socket receive buffer in bytes SO_RCVBUF, set to zero to use the default system value
#   sock-rcvbuf: 0
#   # Channel buffer size for incoming packets, number of packet before to drop it.
#   chan-buffer-size: 65535

[CplPunishment]

The system we're attempting this is mid-range at ~25,000q/s, the target ones will have 2-4x that amount of queries (Unbound is particularly useful in these cases)

I'll try and anonymize the config I'm using as much as possible. I did put the sock-rcvbuf:0 at the "dnstap" socket definition:

global:
   trace:
      verbose: true
      filename: "/data/dns/logs/dnscollector.log"
      log-malformed: false
      max-size: 10
      max-backups: 10

multiplexer:
   # Read DNSTap stream from a UNIX socket
   collectors:
      - name: tap_unix
        dnstap:
           sock-path: /var/run/dnstap.sock
           sock-rcvbuf: 0
        transforms:
          filtering:
            downsample: 100
          normalize:
            qname-lowercase: true
          suspicious:
            malformed-packet: false

   # Write DNS logs to a local file in TEXT format
   loggers:
      - name: file
        logfile:
           file-path: "/data/dns/logs/dnstap.log"
           max-size: 100
           max-files: 10
           mode: text

      - name: [remote-collector]
        dnstap:
          remote-address: [...]
          remote-port: 5001
          sock-path: null
          connect-timeout: 5
          retry-interval: 10
          flush-interval: 30
          tls-support: false
          tls-insecure: false
          tls-min-version: 1.2
          server-id: "myserver"
          overwrite-identity: false
          buffer-size: 256
          chan-buffer-size: 65535
          transforms:
            filtering:
              downsample: 100
            normalize:
              qname-lowercase: true

# Routes DNS messages from the tap 1 and 2 to the same file destination
   routes:
      - from: [ tap_unix ]
        to: [ remote-collector ]

What eventually happens is the packets to the "remote-collector" stop carrying data with a 0 length payload:

20:11:23.027520 IP [local].54622 > [remote].5001: Flags [redacted], length 2896
20:11:23.062771 IP [local].54622 > [remote].5001: Flags [redacted], length 2896
20:11:23.399099 IP [local].54622 > [remote].5001: Flags [redacted], length 344
20:11:23.692101 IP [local].54622 > [remote].5001: Flags [redacted], length 0
20:11:24.243100 IP [local].54622 > [remote].5001: Flags [redacted], length 0
20:11:25.307098 IP [local].54622 > [remote].5001: Flags [redacted], length 0

Additionally, is there a way for this to daemonize as a different user and specify the outbound interface?

[dmachard]

What is your remote collector ? if my understanding is correct, the remote logger is no more ready to receive logs

In your case, the downsampling on logger is not necessary because its already done in your dnstap collector.

[dmachard]

Additionally, is there a way for this to daemonize as a different user and specify the outbound interface?

Not supported right now, feel free to open an issue as feature request for that or push a pull request :)

[CplPunishment]

Tracking this issue a bit further, these are the types of errors I'm encountering:

ERROR: 2023/07/26 16:28:04.171933 [tap_unix] processor=dnstap#1 - logger[remote_collector] buffer is full, 4131 packet(s) dropped
ERROR: 2023/07/26 16:28:14.172616 [tap_unix] processor=dnstap#1 - logger[remote_collector] buffer is full, 4105 packet(s) dropped

Is there a possibility of detecting this buffer full condition and closing/reopening the TCP socket?

[dmachard]

Ys but no option available to do that.

Have you try to increase the 2 following buffers on your dnstap logger ?

• buffer-size: (integer) how many DNS messages will be buffered before being sent
• chan-buffer-size: (integer) channel buffer size used on incoming dns message, number of messages before to drop it.