How to modify firewall rules?

[summatix]

By default it seems like Fedora CoreOS leaves all ports open. I’m trying to modify this behavior by adding some firewall rules. I can’t find any documentation on this topic but my assumption is that FCOS is using iptables.

In Container Linux you could add firewall rules via the /var/lib/iptables/rules-save file. That doesn't seem to be the case for FCOS, and I can't find out where else to do it.

Whenever I edit /etc/sysconfig/iptables using ignition, FCOS is unable to complete booting. Doesn’t matter what the contents of the file are. Instead it goes into emergency mode. For example, if I add this to ignition, it'll fail to boot:

storage:
  files:
    - path: /etc/sysconfig/iptables
      mode: 0600
      contents:
        inline: |
          *filter
          :INPUT ACCEPT [0:0]
          :FORWARD ACCEPT [0:0]
          :OUTPUT ACCEPT [0:0]
          COMMIT

Editing iptables manually after booting doesn’t seem to be an issue, it's only an issue if I write to that file via ignition.

What’s the correct way to update firewall rules for Fedora CoreOS?

[summatix]

I found the notes (and associated tickets) regarding the firewall here: https://github.com/coreos/fedora-coreos-tracker/blob/master/Design.md#firewall-management

It will be possible to set up static rules (i.e. meant to be valid and unchanged for the whole node lifetime) via Ignition.

I haven't been able to find out how to set up static rules via ignition.

I've managed to set up a firewall via my host. Perhaps that's the correct way to do this anyway.

[lucab]

@summatix you are looking for the discussion at #26.

TLDR outcome for that discussion is:

• we inherit the usual iptables-services from Fedora
• there is some friction and potential improvement in this RFE: https://bugzilla.redhat.com/show_bug.cgi?id=1667875
• we still need to document how to set it up via Ignition

[summatix]

Ok. Well this seems to be working:

systemd:
  units:
    - name: iptables-restore.service
      enabled: true
      contents: |
        [Unit]
        Description=Restore iptables firewall rules
        Before=iptables-store.service
        Before=network-pre.target
        Wants=network-pre.target

        [Service]
        Type=oneshot
        ExecStart=echo "*filter\
        ::INPUT ACCEPT [0:0]\
        ::FORWARD ACCEPT [0:0]\
        ::OUTPUT ACCEPT [0:0]\
        :COMMIT" | /usr/sbin/iptables-restore

        [Install]
        WantedBy=basic.target

[cgwalters]

By default it seems like Fedora CoreOS leaves all ports open.

IMO, that's not the right way to think about things. I'd phrase this more like: By default, Fedora CoreOS only listens for ssh. And in the age of containers we don't generally expect "unprivileged" to run in the host network namespace at all (which is where firewalls matter).

[summatix]

IMO, that's not the right way to think about things. I'd phrase this more like: By default, Fedora CoreOS only listens for ssh.

That's an incorrect assertion though based on what Fedora CoreOS currently does. The current default settings allow access to all ports. So once podman starts running containers with bound ports (e.g. podman run -p 80:80/tcp ...), then those ports are exposed to the outside world.

[cgwalters]

So once podman starts running containers with bound ports (e.g. podman run -p 80:80/tcp ...), then those ports are exposed to the outside world.

Yes, but you can also use e.g. -p 127.0.0.1:80:80 to only bind on localhost.

[summatix]

The above ignition I posted isn't working as expected. On boot it's loading the rules, but it appears that the rules are later reset.

[ratzrattillo]

Supporting firewall configuration via ignition is a very important feature. I do not want to expose my Docker Swarm interface for cluster management communications, container network discovery or container ingress network to the public when using swarm services.

Is there any progress regarding this task/bug?

[summatix]

Unfortunately I wasn't able to find a solution and eventually gave up

[m-yosefpor]

No updates/docs on this yet?
@lucab can you please at least post a simple example of how to do it? is the solution with systemd given by @summatix is the right/only way to do so?

[Davidnet]

Hi everyone! I also stumbled into this problem, I want to do a port forwarding, I have tried everything from iptables such as:

sudo iptables -t nat -A PREROUTING -i enp0s4 -p tcp --dport 80 -j REDIRECT --to-port 8006

but no avail to redirection, I also did not found any documentation apart from the links in here, @lucab I want to do a redirect from port 80 to 8006, any help?

[Davidnet]

I have solved my problem, I'm using nftables with the following file:

flush ruleset
table ip nat {
    chain prerouting {
        type nat hook prerouting priority 0;

        tcp dport 80 redirect to 9000
    }
    chain postrouting {
        type nat hook postrouting priority 0;
    }
    chain output {
        type nat hook output priority 0;

        tcp dport 80 redirect to 9000
    }
}

Redirecting 80 to 9000 on both loopback and non-loopback.

I think it will be cool to hear how this is suggest to do in Fedora CoreOS.

Thanks FCOS is awesome!

[LorbusChris]

It might be worth investigating to use ignition for setting up the more powerful nftables rules directly (instead of iptables rules). Once nftables lands in FCOS that is, of course (see #676) 🙂

[magnusviri]

I was able to get it to work using a combination of sources, including the information on this issue and the alternatives docs, and several hours of trail and error. This is the code I'm using in my ignition file.

I've tested it with Docker and it works great so far. I'm real happy with it. I haven't tested it with podman (and probably wont for awhile).

I have to use -d with fcct to specify the location of custom.nft. fcct -d . --pretty --strict < myfile.yml > myfile.ign

storage:
  files:
	- path: /etc/sysconfig/nftables.conf
	  append:
		- inline: include "/etc/nftables/custom.nft"
	- path: /etc/nftables/custom.nft
	  contents:
		local: custom.txt
	- path: /etc/alternatives/iptables
	  target: /usr/sbin/iptables-nft
	  overwrite: true
	  hard: false
	- path: /etc/alternatives/iptables-restore
	  target: /usr/sbin/iptables-nft-restore
	  overwrite: true
	  hard: false
	- path: /etc/alternatives/iptables-save
	  target: /usr/sbin/iptables-nft-save
	  overwrite: true
	  hard: false
	- path: /etc/alternatives/ip6tables
	  target: /usr/sbin/ip6tables-nft
	  overwrite: true
	  hard: false
	- path: /etc/alternatives/ip6tables-restore
	  target: /usr/sbin/ip6tables-nft-restore
	  overwrite: true
	  hard: false
	- path: /etc/alternatives/ip6tables-save
	  target: /usr/sbin/ip6tables-nft-save
	  overwrite: true
	  hard: false
systemd:
 units:
   - name: nftables.service
	 enabled: true
	 mask: false

Edit:

This is my custom.nft for docker (I haven't tried podman). The INPUT chain controls the system services like ssh and DOCKER-USER chain controls docker containers. To get a default deny you have to deny the wan interface after all of your accepts.

#!/sbin/nft -f

table ip filter {
	chain INPUT {
		type filter hook input priority filter; policy drop;
		ct state { established, related } counter accept
		iif "lo" accept
		ip saddr { 192.168.0.15 } accept
	}

	chain DOCKER-USER {
		tcp dport { http, https } ip saddr { 192.168.0.0/24 }  accept
		iif ens192 drop
	}
}

[ratzrattillo]

@magnusviri Great, that it works for you!

It would be good to have an opinion from the developers, if this is the go-to approach! Should it be done this way, or are you already working on a different solution?

[magnusviri]

EDIT:
So, Portainer doesn't work with nftables. (see comment below)

So I setup iptables. Here's what I did to get iptables to work. I don't promise that these are the best rules. But it works.

storage:
  files:
    - path: /etc/sysconfig/iptables-post-docker
      mode: 0755
      contents:
        inline: |
          #!/bin/sh
          /usr/sbin/iptables -D DOCKER-USER 1
          /usr/sbin/iptables -A DOCKER-USER -p tcp --dport 80 -j RETURN
          /usr/sbin/iptables -A DOCKER-USER -p tcp --dport 443 -j RETURN
          /usr/sbin/iptables -A DOCKER-USER -p tcp --dport 9000 -s 192.168.0.0/24 -j RETURN
          /usr/sbin/iptables -A DOCKER-USER -i ens192 -j DROP
          /usr/sbin/iptables -A INPUT -i lo -j ACCEPT
          /usr/sbin/iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
          /usr/sbin/iptables -A INPUT -p tcp --dport 22 -s 192.168.0.0/24 -j ACCEPT
          /usr/sbin/iptables -P INPUT DROP
systemd:
  units:
    - name: podman.service
      mask: true
    - name: docker.service
      enabled: true
    - name: iptables-post-docker.service
      enabled: true
      contents: |
        [Unit]
        Description=Post docker iptables firewall rules
        After=docker.service

        [Service]
        Type=oneshot
        ExecStart=/etc/sysconfig/iptables-post-docker

        [Install]
        WantedBy=basic.target

[magnusviri]

Turns out, Portainer wasn't working because of something stupid I was doing. Portainer does work with nftables.

I've got instructions for nftables and iptables now. I've got an itch to update https://github.com/coreos/fedora-coreos-docs. Maybe I'll get to it this week and then this issue can be closed.

[magnusviri]

FYI: I am able to get iptables to work. I've written a blog post describing how to do it. I wasn't able to get nftables to work correctly for containers. I also noticed that if I changed a port (docker -p 81:80) that the firewall blocks it. So there are still issues to be sorted out.

[adriangabura]

I installed k3s on fedora coreOS and then I used another machine to install helm on it and used the official rancher guide to do a rancher installation. But it is expected for port 443 to be open so I can access the link provided by rancher. How do I open this port on Fedora CoreOS ? I tried sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT but it does not work. Can you help please?

[dustymabe]

But it is expected for port 443 to be open so I can access the link provided by rancher. How do I open this port on Fedora CoreOS ?

by default all ports are open on Fedora CoreOS. Perhaps the issue is with the provisioning of k3s?