Tailscale integration - possible?

[btrepp]

Hi All,

Love the concept of being declarative driven for the OS, and am excited to use the project.

I'm wondering if it's possible to integrate Tailscale with Talos. On a normal machine, you would set this up before installing k8s/k3s or what have you, but with a talos setup there is no cli so you can't exactly ssh in and run some commands :).

This would be beneficial for a k8s/talos install, having the control plane and Talos APIs would simplify network planning, eg the cli nodes and any required attached resources (eg NAS/storage) would also be in the Tailscale network. It's probably less on the security aspect (talos seems secure with mtls) but more for a dns/network setup which would simplify network design.

I can see three possible ways

1. Extensions, perhaps including the Tailscale client, and configuring it with a secret to join the network per 'machine' seems more natural. I did experiment with trying to develop this on my laptop, but am still trying to get a good development setup. I would think an extension is the best fit, as it should really be an 'additional' capability and not folded into the core of talos.

2. Static pods, perhaps the Tailscale agents could run as the static pods, somehow with the routes feeding most traffic into the pod, this may depend on startup orders of things, but could be a solution

3. Just run subnet routers in pods, this makes sense for more ingress/egress style setups, but wouldn't have the talos components communicating (and also may/may not be easy for storage concerns)

Does anyone have any guidance for this? One and Two may be possible, and I'm keen to give it a try, though I don't know enough about how talos boots up and installs to even know if an extension could provide and extra network interface that the 'cluster' then communicates over.

Even if this is a terrible idea it would be good to know :).

[smira]

All three you described should be possible, I don't think there's anything specific already available right now, but Talos should work just fine with tailscale running. You might need to limit kubelet IPs/etcd IPs to 100.x/8 if you want to make sure it all goes over Tailscale.

There might be some interesting stuff related to Tailscale persisting data

[btrepp]

Thanks. I've gotten pretty close to this now. I opted for an extension as I wanted it to be close to 'machine-level' rather than depending on k8s.

So working at the moment in userspace networking.

In regards to state, I am currently saving that in /var/lib/tailscale volume mounted into the extension service.

The problem to crack is non-userspace. I'm close, but it seems Tailscale wants to run iptables commands. This seems a bit more tricky to figure out.

Either volume mounting /sbin, or including the iptables from the ghcr.io/siderolabs/iptables:v1.4.0 image both yields the below.

rpi1.localdomain: 2023/04/25 09:46:02 getLocalBackend error: createEngine: creating router: could not get iptables version: fork/exec /sbin/iptables: no such file or directory

I'm thinking iptables (or x-tables-legacy-multi, which seems to be the symlink) wants to maybe open something else?. Or symlinks aren't working quite as I'm expecting. I feel this is solvable though, just need to spend some more time with it :)

[frezbo]

fork/exec /sbin/iptables: no such file or directory this could also mean that the shared object files required for the binary are missing

[frezbo]

if you're mounting /sbin you'd also need to bind /lib eg: https://github.com/siderolabs/extensions/blob/main/storage/iscsi-tools/iscsid.yaml#L17-L30

[btrepp]

For those who find this. I wrote the extension and the PR is here
siderolabs/extensions#154