The difficulties of getting the combination of Linux KVM, host-side modern nftables packet filtering, and guest-side networking to work together without resorting to firewalld
on the host are fairly well published; for example, here. The recommended solution usually involves going back to iptables on the host, and sometimes to define libvirt-specific nwfilter rules. While that might be tolerable for dedicated virtualization hosts, it’s less than ideal for systems that also see other uses, especially uses where nftables’ expressive power and relative ease of use is desired.
Fortunately, it can be worked around without giving up on nftables.
I’m assuming that you have already set up a typical basic nftables client-style ruleset on the host, something along the lines of:
#!/usr/bin/nft -f
flush ruleset
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
ct state invalid drop
ct state established accept
ct state related accept
iifname "lo" accept
}
chain forward {
type filter hook forward priority 0; policy drop;
}
chain output {
type filter hook output priority 0; policy accept;
}
}
Start out by setting the KVM network to start automatically on boot. The network startup will also cause libvirt to create some NAT post-routing tables through iptables
, which through the magic of conversion tools get transformed into a corresponding nftables table ip nat
. This might cause an error to be displayed initially, but that’s OK for now. Reboot the host, run virsh net-list --all
to check that the network is active, and nft list table ip nat
to check to make sure that the table and chains were created. It should all look something like:
$ sudo virsh net-list --all
Name State Autostart Persistent
--------------------------------------------
default active yes yes
$ sudo nft list table ip nat
table ip nat {
chain LIBVIRT_PRT {
... a few moderately complex masquerading rules ...
}
chain POSTROUTING {
type nat hook postrouting priority srcnat; policy accept;
counter packets 0 bytes 0 jump LIBVIRT_PRT
}
}
$
Letting libvirt’s magic and the iptables-to-nftables conversion tools handle the insertion of the routing filters makes it less likely that issues will develop later on due to for example changes in what rules newer versions need. An alternative approach, which works currently for me but might not work for you or in the future, is to manually create a postrouting chain; the nftables magic incantation can be reduced to something similar to:
table ip nat {
chain postrouting {
type nat hook postrouting priority 100; policy accept;
ip saddr 192.168.122.0/24 masquerade
}
}
(In the above snippet, 192.168.122.0/24
maps to the details from the <ip>
node in the output of virsh net-dumpxml <name>
for each network listed by virsh net-list
earlier.)
You do, however, need to add some rules to the table inet filter
to allow incoming and forwarded packets to pass through to and from the physical network interface (eth0
here; substitute as appropriate, ip addr sh
will tell you the interface name):
table inet filter {
chain input {
# ... add at some appropriate location ...
iifname "virbr0" accept
}
chain forward {
# ... add at some appropriate location ...
iifname "virbr0" oifname "eth0" accept
iifname "eth0" oifname "virbr0" accept
}
}
The forward
chain rules probably aren’t necessary if your forward chain has the default accept
policy, but it’s generally better to have a drop
or reject
policy and only allow the traffic that is actually needed.
The finishing touch is to make sure that sysctl net.ipv4.ip_forward = 1
on the host; without it, IPv4 forwarding won’t work at all.
Unfortunately, as KVM still tries to use iptables
to create a NAT table when its network is started, and this can’t be done when a nftables NAT table exists, the table ip nat
portion, if manually configured, needs to go into a nftables script that is loaded after the KVM network is started thus replacing the automatically generated chain, whereas most distributions are set up to load the normal nftables rule set quite early during the boot process, likely and hopefully before basic networking is even fully up and running (to close the window of opportunity for traffic to sneak through). The easiest way to deal with this is very likely to just let the iptables
compatibility tools handle this for you when the KVM network is started and accept the need for a reboot during the early KVM configuration process. The most likely scenario in which this simple approach won’t work seems to be if you are already using nftables to do other IP forwarding magic as well; in that case, you may need to resort to a split nftables configuration and loading the post-routing NAT ruleset late during the boot process, such as perhaps through /etc/rc.local
(which is typically executed very late during boot). If so, then it’s probably worth the trouble to rewrite one or the other in terms of nft add
commands instead of a full-on, atomic nft -f
script.
With all this in place, KVM guests should now be able to access the outside world over IPv4, NATed through the host, including after a reboot of the host.
A huge tip of the proverbial hat to user regox on the Gentoo forums, who posted what I was able to transform into most of the above.