Need help with redistribute default routes

[Dando-Real-ITA]

Hi,
I am testing redistribution of default routes, using 2 routes and 1 client.
The idea is to have both routes installed on the client either with different metrics, or as 1 rule with 2 nexthops of different weights

Use case: 2 ISPs, with one preferred over the other, different public IPv4 and IPv6 ranges, routers distributing the default routes with custom metric ( or weigth ), the routers would be monitoring the external connection, and change the redistribution metric or retract the distribution if an ISP goes offline. Also source routing that can work for both IPv4 and IPv6.

These are the things I tried without success:

• Have the routes installed in the client with a pref-src.
  I tried a combo of the in filter, pref-src <IP> and no selector or selectors like table 2, ip ::/0 eq 0, ip 0.0.0.0/0 eq 0, id <RouterID> but the route gets always installed in the default table, with no source, also no metric on the IPv4:

default via 12.144.66.185 dev ens3 proto babel onlink
default via fe80::e6a:a3ff:fea7:0 dev ens3 proto babel metric 1024 onlink pref medium

• Have multiple routes installed
  What happens is that only one gets installed, the other stays "feasible". Also it is on first-come basis, so the lower metric one can end to be not the one installed. Example of a log output:

::/0 from ::/0 metric 1096 (1096) refmetric 1000 id 0e:08:1c:ff:fe:dd:00:00 seqno 13650 age 87 via ens3 neigh fe80::e08:1cff:fedd:0 (installed)
::/0 from ::/0 metric 106 (106) refmetric 10 id 0e:6a:a3:ff:fe:a7:00:00 seqno 38400 age 87 via ens3 neigh fe80::e6a:a3ff:fea7:0 (feasible)

0.0.0.0/0 from 0.0.0.0/0 metric 1096 (1096) refmetric 1000 id 0e:08:1c:ff:fe:dd:00:00 seqno 13650 age 87 via ens3 neigh fe80::e08:1cff:fedd:0 nexthop 12.144.66.185 (installed)
0.0.0.0/0 from 0.0.0.0/0 metric 106 (106) refmetric 10 id 0e:6a:a3:ff:fe:a7:00:00 seqno 38400 age 87 via ens3 neigh fe80::e6a:a3ff:fea7:0 nexthop 66.199.5.161 (feasible)

If both could be installed, the OS could choose the lower metric one. Other option could be to override the same route installed when a new lower metric is received. [Extra] nexthop merging of route + weight could be also an option

• [Extra] Have the same route installed twice, with different parameters by altering with the install filter. This would allow to add the same route in the default table and in a separate table to be used with source routing-policy.

The main settings are:
Router 1:

redistribute ip 0.0.0.0/0 eq 0 metric 1000
redistribute ip ::/0 eq 0 metric 1000

Router 2:

redistribute ip 0.0.0.0/0 eq 0 metric 10
redistribute ip ::/0 eq 0 metric 10

Client:
Ubuntu with kernel 5.15.0-92-generic

in id 0e:08:1c:ff:fe:dd:00:00 table 2

[jech]

I tried a combo of the in filter, pref-src

pref-src only makes sense in the install filter.

Have multiple routes installed

We don't do that. Babeld will always install at most one route for a given prefix. What issue are you trying to solve?

Also it is on first-come basis, so the lower metric one can end to be not the one installed.

That's strange, babeld should switch to the lower-metric one when it beacomes feasible. Could we please see your full configuration?

Have the same route installed twice, with different parameters by altering with the install filter.

We don't do that. Please describe the problem you're trying to solve.

[Dando-Real-ITA]

pref-src only makes sense in the install filt

Yes I am sorry for pref-src I was using the wrong filter

We don't do that. Babeld will always install at most one route for a given prefix. What issue are you trying to solve?

I am trying to dynamically create on the client the equivalent of this netplan config:

# 2024-02-21 19:27:51
network:
  version: 2
  renderer: networkd
  ethernets:
    lo:
      addresses:
        - "fd72:1fdc:5c49:b151:4072:6550:93eb:4d62/128"
    ens3:
      optional: false
      dhcp4: no
      dhcp6: no
      accept-ra: yes
      addresses:
        # ISP1
        - "12.144.66.186/29"
        - "2001:1890:1f76:4400:4072:6550:93eb:4d62/64"
        # ISP2
        - "66.199.5.162/28"
        - "2001:4870:24a:500:4072:6550:93eb:4d62/56"
      mtu: 1500
      routes:
        # ISP1, master table
        - from: 66.199.5.162
          to: default
          via: 66.199.5.161
          metric: 10
        - from: "2001:4870:24a:500:4072:6550:93eb:4d62"
          to: default
          via: "2001:4870:24a:500:68a3:dc3e:8ca7:320c"
          metric: 10
        # ISP1, table 1
        - from: 66.199.5.162
          to: default
          via: 66.199.5.161
          table: 1
        - from: "2001:4870:24a:500:4072:6550:93eb:4d62"
          to: default
          via: "2001:4870:24a:500:68a3:dc3e:8ca7:320c"
          table: 1
        # ISP2, master table
        - from: 12.144.66.186
          to: default
          via: 12.144.66.185
          metric: 1000
        - from: "2001:1890:1f76:4400:4072:6550:93eb:4d62"
          to: default
          via: "2001:1890:1f76:4400:cca7:c254:dcf2:1c28"
          metric: 1000
        # ISP2, table 2
        - from: 12.144.66.186
          to: default
          via: 12.144.66.185
          table: 2
        - from: "2001:1890:1f76:4400:4072:6550:93eb:4d62"
          to: default
          via: "2001:1890:1f76:4400:cca7:c254:dcf2:1c28"
          table: 2
      routing-policy:
        # ISP 1
        - from: 66.199.5.162
          table: 1
        - from: "2001:4870:24a:500:4072:6550:93eb:4d62"
          table: 1
        # ISP 2
        - from: 12.144.66.186
          table: 2
        - from: "2001:1890:1f76:4400:4072:6550:93eb:4d62"
          table: 2

Which allows to have double ISP connection. the same routes are installed in master table ( with different metric ) and in separate tables ( for src ip based same ISP response )
Then if an ISP is down and by external checks the master table default routes of the failing ISP are changed metric or removed, the client can keep connection with the other ISP on its IPs
If the default routes are distributed by the router with the ISP connection, the check can be in 1 place, update the live babeld config and be propagated to all clients

That's strange, babeld should switch to the lower-metric one when it beacomes feasible. Could we please see your full configuration?

This was the config, but I can retest better if the route with lower metric is selected

# 2024-02-21 20:17:10

# Global
local-path-readwrite        /var/run/babeld.sock
# kernel-check-interval       600
shutdown-delay-ms           5000
link-detect                 true
# key id id type {hmac-sha256|blake2s128} value value

# Interfaces
# {auto|wired|wireless|tunnel}
default type                wired
default update-interval     600
default v4-via-v6           true
# default key id

# Filters
# # Distribute ips on loopback
# redistribute local if lo allow
# Distribute ips on nebula
redistribute local if nebula0 allow
# # Distribute ips on main ethernets
# redistribute local if ens3 allow
# Do not distribute other local addresses
redistribute local deny

I am currently testing with a hybrid solution, by defining the more preferred routes with

0.0.0.0/1
128.0.0.0/1
::/1
8000::/1

and removing from the client network config the default routes from master table, but leaving the table routes and table policies

[Dando-Real-ITA]

I retested with the 2 metrics, started first the babeld with the higher metric, then the one with lower metric. The client installed the higher metric first, but after a few minutes it switched to the lower metric:

Feb 22 16:49:05 ubuntu-cloud babeld[1771]: ::/0 from ::/0 metric 1096 (1096) refmetric 1000 id 0e:08:1c:ff:fe:dd:00:00 seqno 11161 age 35 via ens3 neigh fe80::e08:1cff:fedd:0 (installed)
Feb 22 16:49:05 ubuntu-cloud babeld[1771]: ::/0 from ::/0 metric 106 (574) refmetric 10 id 0e:6a:a3:ff:fe:a7:00:00 seqno 32641 age 35 via ens3 neigh fe80::e6a:a3ff:fea7:0 (feasible)
Feb 22 16:49:05 ubuntu-cloud babeld[1771]: 0.0.0.0/0 from 0.0.0.0/0 metric 1096 (1096) refmetric 1000 id 0e:08:1c:ff:fe:dd:00:00 seqno 11161 age 35 via ens3 neigh fe80::e08:1cff:fedd:0 nexthop 12.144.66.185 (installed)
Feb 22 16:49:05 ubuntu-cloud babeld[1771]: 0.0.0.0/0 from 0.0.0.0/0 metric 106 (773) refmetric 10 id 0e:6a:a3:ff:fe:a7:00:00 seqno 32641 age 35 via ens3 neigh fe80::e6a:a3ff:fea7:0 nexthop 66.199.5.161 (feasible)

Feb 22 17:00:46 ubuntu-cloud babeld[1771]: ::/0 from ::/0 metric 106 (106) refmetric 10 id 0e:6a:a3:ff:fe:a7:00:00 seqno 32641 age 177 via ens3 neigh fe80::e6a:a3ff:fea7:0 (installed)
Feb 22 17:00:46 ubuntu-cloud babeld[1771]: ::/0 from ::/0 metric 1096 (1096) refmetric 1000 id 0e:08:1c:ff:fe:dd:00:00 seqno 11161 age 160 via ens3 neigh fe80::e08:1cff:fedd:0 (feasible)
Feb 22 17:00:46 ubuntu-cloud babeld[1771]: 0.0.0.0/0 from 0.0.0.0/0 metric 106 (106) refmetric 10 id 0e:6a:a3:ff:fe:a7:00:00 seqno 32641 age 177 via ens3 neigh fe80::e6a:a3ff:fea7:0 nexthop 66.199.5.161 (installed)
Feb 22 17:00:46 ubuntu-cloud babeld[1771]: 0.0.0.0/0 from 0.0.0.0/0 metric 1096 (1096) refmetric 1000 id 0e:08:1c:ff:fe:dd:00:00 seqno 11161 age 160 via ens3 neigh fe80::e08:1cff:fedd:0 nexthop 12.144.66.185 (feasible)

I did not wait long enough in my first test, thank you for poiting it out, I don't need the trick of splitting the routes anymore

Now I am trying to tackle the pref-src

# ISP1
install ip 0.0.0.0/0 le 1 id 0e:6a:a3:ff:fe:a7:00:00 pref-src 66.199.5.162
install ip ::/0      le 1 id 0e:6a:a3:ff:fe:a7:00:00 pref-src 2001:4870:24a:500:4072:6550:93eb:4d62
# ISP2
install ip 0.0.0.0/0 le 1 id 0e:08:1c:ff:fe:dd:00:00 pref-src 12.144.66.186
install ip ::/0      le 1 id 0e:08:1c:ff:fe:dd:00:00 pref-src 2001:1890:1f76:4400:4072:6550:93eb:4d62

For now it only works if I do not put the routers ids, like

install ip 0.0.0.0/0 le 1 pref-src 66.199.5.162

adds the pref-src to all the default routes

Edit:
This are the logs:

Received router-id 0e:6a:a3:ff:fe:a7:00:00 from fe80::e6a:a3ff:fea7:0 on ens3.
Received update for 0.0.0.0/0 from fe80::e6a:a3ff:fea7:0 on ens3.
Received update for dst 0.0.0.0/0 from fe80::e6a:a3ff:fea7:0 on ens3.
install_route(0.0.0.0/0 from 0.0.0.0/0)
kernel_route: add 0.0.0.0/0 from 0.0.0.0/0 table 254 metric 65535 dev 2 nexthop 66.199.5.161
Sending update to ens3 for 0.0.0.0/0 from 0.0.0.0/0.

The router id matches with the config, but the filter does not seem to apply. Am I writing it wrong and the filter does not apply?

[Dando-Real-ITA]

I found that the id was missing in the install_filter, I added it:

#113

Edit: actually I need to check it more, I did the test of starting a default route with high metric first then start the router with the lower metric, and when the route was updated it kept the src of the first route instead of updating the src