recursor - refresh_on_ttl_perc with qtype ANY

[bjacquin]

• This is not a support question, I have read about opensource and will send support questions to the IRC channel, GitHub Discussions or the mailing list.

• I have read and understood the 'out in the open' support policy

• Program: Recursor

• Issue type: Maybe bug report

Short description

When pdns-recursor is configured with packetcache or recordcache enabled, when qtype ANY queries may or may not be answered from records in cache.

The behaviour I'm observing is that when a given record has multiple qtype defined with different TTL, recordcache.refresh_on_ttl_perc=15 does not refresh the record of the particular expiring qtype. I would assume that recordcache.refresh_on_ttl_perc would refresh the about to expire qtype.

When a qtype has expired from pdns-recursor cache, ANY qtype would not return that record, but as soon a query is received for the expiring qtype, a new request to authoritative is made, returning the expected result to the client, but also now returned for ANY qtype.

Should not pdns-recursor refresh each qtype when they are about to expire from ANY qtype ? This is the behaviour I'm observing with 8.8.8.8

Environment

• Operating system: Gentoo on aarch64
• Software version: 5.0.9
• Software source: Operating system repository

Steps to reproduce

Configuration:

incoming:
  reuseport: true
  tcp_fast_open: 65536
  proxy_protocol_from:
  - fc00::/7
  allow_from:
  - ::/0
  - 0.0.0.0/0
  use_incoming_edns_subnet: true

outgoing:
  source_address:
  - '::'
  - 0.0.0.0
  tcp_fast_open_connect: true

dnssec:
  validation: validate
  log_bogus: true

recordcache:
  refresh_on_ttl_perc: 15
  limit_qtype_any: false

1. watch -n 1 dig in ANY pants-on.net @<ip>
   1. On the first request, ANY return all record from upstream as expected, with a wide variate of qtype and TTL

;; ANSWER SECTION:
pants-on.net.           86397   IN	NS	ns.eu-west-1.pants-on.net.
pants-on.net.           57      IN	DNSKEY  257 3 13 uDE+OHR6H/J+1Ppdh/ouCWLw2AsPiS6WOecWH8dfHHtX6imRE3bku/7t vkvUcRJbPQin+qYmhXAXfkY9bw/pyw==
pants-on.net.           86397   IN	NS      ns.ap-northeast-2.pants-on.net.
pants-on.net.           57      IN	CDNSKEY 257 3 15 Ybbo1rn0YJLEwIYkEjAGMPyD+iGa2YyxyGk7lTF8a1I=
pants-on.net.           86397   IN	DS      35688 15 2 48836B5B2B21EFAB26BAB16475438F4F553EDC41D315B498ACC95CF8 1FC48F2C
pants-on.net.           57      IN	CDS     20476 13 2 F3947DDBEEADF5BC6154B9698FD4D35E7D938999B30A94457DA98788 1DDB2213
pants-on.net.           897     IN	SOA     ns.pants-on.net. nsmaster.pants-on.net. 1729889133 14400 3600 604800 60
pants-on.net.           297     IN	AAAA    2a05:d018:e2::29
pants-on.net.           57      IN	DNSKEY  257 3 15 Ybbo1rn0YJLEwIYkEjAGMPyD+iGa2YyxyGk7lTF8a1I=
pants-on.net.           86397   IN	DS	20476 13 2 F3947DDBEEADF5BC6154B9698FD4D35E7D938999B30A94457DA98788 1DDB2213
pants-on.net.           57      IN	NSEC3PARAM 1 0 0 -
pants-on.net.           86397   IN	NS      ns.us-east-1.pants-on.net.
pants-on.net.           57      IN	CDS     35688 15 2 48836B5B2B21EFAB26BAB16475438F4F553EDC41D315B498ACC95CF8 1FC48F2C
pants-on.net.           57      IN	A       54.73.98.29
pants-on.net.           57      IN	CDNSKEY 257 3 13 uDE+OHR6H/J+1Ppdh/ouCWLw2AsPiS6WOecWH8dfHHtX6imRE3bku/7t vkvUcRJbPQin+qYmhXAXfkY9bw/pyw==
pants-on.net.           86397   IN	TXT     "google-site-verification=1na8jxIY5Rzsc3dkar2QW3N6mvYn14KS7QAWP_FZ_d0"
pants-on.net.           86397   IN	CAA     0 issue "letsencrypt.org"
pants-on.net.           897     IN	HTTPS   1 . alpn="h3,h2,http/1.1" ipv4hint=54.73.98.29 ipv6hint=2a05:d018:e2::29
pants-on.net.           86397   IN	TXT     "v=spf1 mx -all"
pants-on.net.           897     IN	MX      0 mx.eu-west-1.pants-on.net.

2. Here the A record has a TTL for 60 seconds for debugging purpose
3. After 60 seconds, the A record TTL expire and no A record is returned from previous dig request
4. If separately, a A query is made to pants-on.net, the proper record is returned and ANY request also return the record with a fresh TTL

Expected behaviour

I would expect that when A record from ANY request expire, it would be refreshed. Assuming the A record non longer exist, then negative_ttl would apply and just stop being returned.

Actual behaviour

See above for details

[omoerbeek]

We only do an auth query for ANY requests if there are NO records available at all, otherwise we answer with what we happen have in cache. (But note the CNAME chains from the cache might trigger additional resolving even for ANY queries).
The refreshing heuristic does not change that.

In general there has been a tendency to move away form fully supporting ANY queries. See https://datatracker.ietf.org/doc/html/rfc8482, section 4.1 in particular. It is unwise to depend on any particular behaviour for ANY queries.

[omoerbeek]

Moving this to a discussion.