M-Labs/nix-scripts
9
0
Fork 9
Code Issues 6 Pull Requests Releases Wiki Activity

isolate wifi network #20

New Issue
Closed
opened 2019-10-11 13:19:51 +08:00 by sb10q · 10 comments
sb10q commented 2019-10-11 13:19:51 +08:00
Owner
Copy Link

As the wifi network is used by various guests at the lab (and most other machines are connected by wired Ethernet), it makes sense to isolate it from the LAN and only provide internet access.

Linux requires ipv4 forwarding to be enabled for NAT to work, so it currently happily routes packets between the WiFi and LAN even though they are on different interfaces and IP ranges.

This does not work for some reason (the rule is visible in iptables -L but has no effect):

--- a/nixbld-etc-nixos/configuration.nix
+++ b/nixbld-etc-nixos/configuration.nix
@@ -54,6 +54,10 @@ in
       enable = true;
       externalInterface = netifWan;
       internalInterfaces = [ netifLan netifWifi ];
+      extraCommands = ''
+        iptables -A FORWARD -i ${netifLan} -o ${netifWifi} -j DROP
+        iptables -A FORWARD -i ${netifWifi} -o ${netifLan} -j DROP
+      '';
     };
     sits."${netifSit}" = {
       dev = netifWan;

Note that the use of extraCommands would require a extraStopCommands counterpart, otherwise the rules accumulate every time nixos-rebuild switch is run.

As the wifi network is used by various guests at the lab (and most other machines are connected by wired Ethernet), it makes sense to isolate it from the LAN and only provide internet access. Linux requires ipv4 forwarding to be enabled for NAT to work, so it currently happily routes packets between the WiFi and LAN even though they are on different interfaces and IP ranges. This does not work for some reason (the rule is visible in ``iptables -L`` but has no effect): ```patch --- a/nixbld-etc-nixos/configuration.nix +++ b/nixbld-etc-nixos/configuration.nix @@ -54,6 +54,10 @@ in enable = true; externalInterface = netifWan; internalInterfaces = [ netifLan netifWifi ]; + extraCommands = '' + iptables -A FORWARD -i ${netifLan} -o ${netifWifi} -j DROP + iptables -A FORWARD -i ${netifWifi} -o ${netifLan} -j DROP + ''; }; sits."${netifSit}" = { dev = netifWan; ``` Note that the use of ``extraCommands`` would require a ``extraStopCommands`` counterpart, otherwise the rules accumulate every time ``nixos-rebuild switch`` is run.
astro commented 2019-10-14 05:09:35 +08:00
Contributor
Copy Link

Are there really no other rules in the FORWARD chain? Could you please provide iptables -S output, or even iptables-save with all the tables?

BTW, your approach is blacklisting. I recommend -P FORWARD DROP and then allowing internet access to LAN and WiFi. You'll notice that you need to update rules if for example your interfaces names change.

Note that the use of extraCommands would require a extraStopCommands counterpart, otherwise the rules accumulate every time nixos-rebuild switch is run.

Put the rules in a user-defined chain that you can delete/recreate at once. Flush the FORWARD chain before appending a -j rule to that user-defined chain. Precaution must be taken when adding extra rules in the future.

Alternatively, drop the nixos-fw and use iptables-save/iptables-restore, or even more modern: switch to nftables (still on my list).

Are there really no other rules in the FORWARD chain? Could you please provide `iptables -S` output, or even `iptables-save` with all the tables? BTW, your approach is blacklisting. I recommend `-P FORWARD DROP` and then allowing internet access to LAN and WiFi. You'll notice that you need to update rules if for example your interfaces names change. > Note that the use of extraCommands would require a extraStopCommands counterpart, otherwise the rules accumulate every time nixos-rebuild switch is run. Put the rules in a user-defined chain that you can delete/recreate at once. Flush the `FORWARD` chain before appending a `-j` rule to that user-defined chain. Precaution must be taken when adding extra rules in the future. Alternatively, drop the nixos-fw and use `iptables-save`/`iptables-restore`, or even more modern: switch to nftables (still on my list).
sb10q commented 2019-10-14 12:34:32 +08:00
Author
Owner
Copy Link
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N LIBVIRT_FWI
-N LIBVIRT_FWO
-N LIBVIRT_FWX
-N LIBVIRT_INP
-N LIBVIRT_OUT
-N nixos-drop
-N nixos-fw
-N nixos-fw-accept
-N nixos-fw-log-refuse
-N nixos-fw-refuse
-A INPUT -j LIBVIRT_INP
-A INPUT -j nixos-fw
-A FORWARD -i wlp4s0 -o enp3s0 -j DROP
-A FORWARD -i enp3s0 -o wlp4s0 -j DROP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A FORWARD -i enp3s0 -o wlp4s0 -j DROP
-A FORWARD -i wlp4s0 -o enp3s0 -j DROP
-A OUTPUT -j LIBVIRT_OUT
-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A nixos-drop -j DROP
-A nixos-fw -i lo -j nixos-fw-accept
-A nixos-fw -m conntrack --ctstate RELATED,ESTABLISHED -j nixos-fw-accept
-A nixos-fw -p tcp -m tcp --dport 22 -j nixos-fw-accept
-A nixos-fw -p tcp -m tcp --dport 25 -j nixos-fw-accept
-A nixos-fw -p tcp -m tcp --dport 80 -j nixos-fw-accept
-A nixos-fw -p tcp -m tcp --dport 143 -j nixos-fw-accept
-A nixos-fw -p tcp -m tcp --dport 443 -j nixos-fw-accept
-A nixos-fw -p tcp -m tcp --dport 587 -j nixos-fw-accept
-A nixos-fw -p tcp -m tcp --dport 631 -j nixos-fw-accept
-A nixos-fw -p udp -m udp --dport 53 -j nixos-fw-accept
-A nixos-fw -p udp -m udp --dport 67 -j nixos-fw-accept
-A nixos-fw -p udp -m udp --dport 631 -j nixos-fw-accept
-A nixos-fw -p udp -m udp --dport 5353 -j nixos-fw-accept
-A nixos-fw -p udp -m udp --dport 60000:61000 -j nixos-fw-accept
-A nixos-fw -p icmp -m icmp --icmp-type 8 -j nixos-fw-accept
-A nixos-fw -j nixos-fw-log-refuse
-A nixos-fw-accept -j ACCEPT
-A nixos-fw-log-refuse -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -j LOG --log-prefix "refused connection: " --log-level 6
-A nixos-fw-log-refuse -m pkttype ! --pkt-type unicast -j nixos-fw-refuse
-A nixos-fw-log-refuse -j nixos-fw-refuse
-A nixos-fw-refuse -j DROP

(Out of desperation, I had tried both -A FORWARD and -I FORWARD, which is why the rule appears twice)

We do use the nixos-fw and I would rather just have a quick and simple solution right now. And this says that nixos-fw isn't meant to affect packet forwarding: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/networking/firewall.nix#L329-L330

AFAIK we are not using the virbr networking and we can (should?) disable it completely.

```text -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N LIBVIRT_FWI -N LIBVIRT_FWO -N LIBVIRT_FWX -N LIBVIRT_INP -N LIBVIRT_OUT -N nixos-drop -N nixos-fw -N nixos-fw-accept -N nixos-fw-log-refuse -N nixos-fw-refuse -A INPUT -j LIBVIRT_INP -A INPUT -j nixos-fw -A FORWARD -i wlp4s0 -o enp3s0 -j DROP -A FORWARD -i enp3s0 -o wlp4s0 -j DROP -A FORWARD -j LIBVIRT_FWX -A FORWARD -j LIBVIRT_FWI -A FORWARD -j LIBVIRT_FWO -A FORWARD -i enp3s0 -o wlp4s0 -j DROP -A FORWARD -i wlp4s0 -o enp3s0 -j DROP -A OUTPUT -j LIBVIRT_OUT -A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT -A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT -A nixos-drop -j DROP -A nixos-fw -i lo -j nixos-fw-accept -A nixos-fw -m conntrack --ctstate RELATED,ESTABLISHED -j nixos-fw-accept -A nixos-fw -p tcp -m tcp --dport 22 -j nixos-fw-accept -A nixos-fw -p tcp -m tcp --dport 25 -j nixos-fw-accept -A nixos-fw -p tcp -m tcp --dport 80 -j nixos-fw-accept -A nixos-fw -p tcp -m tcp --dport 143 -j nixos-fw-accept -A nixos-fw -p tcp -m tcp --dport 443 -j nixos-fw-accept -A nixos-fw -p tcp -m tcp --dport 587 -j nixos-fw-accept -A nixos-fw -p tcp -m tcp --dport 631 -j nixos-fw-accept -A nixos-fw -p udp -m udp --dport 53 -j nixos-fw-accept -A nixos-fw -p udp -m udp --dport 67 -j nixos-fw-accept -A nixos-fw -p udp -m udp --dport 631 -j nixos-fw-accept -A nixos-fw -p udp -m udp --dport 5353 -j nixos-fw-accept -A nixos-fw -p udp -m udp --dport 60000:61000 -j nixos-fw-accept -A nixos-fw -p icmp -m icmp --icmp-type 8 -j nixos-fw-accept -A nixos-fw -j nixos-fw-log-refuse -A nixos-fw-accept -j ACCEPT -A nixos-fw-log-refuse -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -j LOG --log-prefix "refused connection: " --log-level 6 -A nixos-fw-log-refuse -m pkttype ! --pkt-type unicast -j nixos-fw-refuse -A nixos-fw-log-refuse -j nixos-fw-refuse -A nixos-fw-refuse -j DROP ``` (Out of desperation, I had tried both ``-A FORWARD`` and ``-I FORWARD``, which is why the rule appears twice) We do use the nixos-fw and I would rather just have a quick and simple solution right now. And this says that nixos-fw isn't meant to affect packet forwarding: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/networking/firewall.nix#L329-L330 AFAIK we are not using the virbr networking and we can (should?) disable it completely.
astro commented 2019-10-15 03:14:24 +08:00
Contributor
Copy Link

AFAIK we are not using the virbr networking and we can (should?) disable it completely.

If you really want virtd without the network, I found this:

virsh net-destroy default
virsh net-undefine default

Otherwise:

diff --git nixbld-etc-nixos/configuration.nix nixbld-etc-nixos/configuration.nix
index dda037e..9e192e4 100644
--- nixbld-etc-nixos/configuration.nix
+++ nixbld-etc-nixos/configuration.nix
@@ -254,1 +254,1 @@
-  virtualisation.libvirtd.enable = true;
+  virtualisation.libvirtd.enable = false;

You don't need libvirtd for just qemu/kvm/vbox. In fact, they're easier use without libvirt. :)

> AFAIK we are not using the virbr networking and we can (should?) disable it completely. If you really want virtd without the network, I found this: ``` virsh net-destroy default virsh net-undefine default ``` Otherwise: ```diff diff --git nixbld-etc-nixos/configuration.nix nixbld-etc-nixos/configuration.nix index dda037e..9e192e4 100644 --- nixbld-etc-nixos/configuration.nix +++ nixbld-etc-nixos/configuration.nix @@ -254,1 +254,1 @@ - virtualisation.libvirtd.enable = true; + virtualisation.libvirtd.enable = false; ``` You don't need libvirtd for just qemu/kvm/vbox. In fact, they're easier use without libvirt. :)
astro commented 2019-10-15 03:59:44 +08:00
Contributor
Copy Link

I can reproduce your rules and they do match here.

More to look into: raw and nat tables, -j LOG

I can reproduce your rules and they do match here. More to look into: raw and nat tables, `-j LOG`
sb10q commented 2019-10-15 17:21:38 +08:00
Author
Owner
Copy Link

Hmm, OK. I remembered QEMU+KVM wouldn't work unless libvirtd was enabled, but after checking again, this isn't the case.

Hmm, OK. I remembered QEMU+KVM wouldn't work unless libvirtd was enabled, but after checking again, this isn't the case.
sb10q commented 2019-10-15 18:17:53 +08:00
Author
Owner
Copy Link

Okay, so what is happening is that while access to the LAN 192.168.1.x with x > 1 gets indeed blocked from Wi-Fi, access to 192.168.1.1 (which is the IP address of the LAN interface on nixbld) is not.

This also happens with -P DROP.

iptables -w -N allow-nat
iptables -w -A allow-nat -i enp3s0 -o enp0s31f6 -j ACCEPT
iptables -w -A allow-nat -o enp3s0 -i enp0s31f6 -j ACCEPT
iptables -w -A allow-nat -i wlp4s0 -o enp0s31f6 -j ACCEPT
iptables -w -A allow-nat -o wlp4s0 -i enp0s31f6 -j ACCEPT
iptables -w -A FORWARD -j allow-nat
iptables -w -P FORWARD DROP
Okay, so what is happening is that while access to the LAN 192.168.1.x with x > 1 gets indeed blocked from Wi-Fi, access to 192.168.1.1 (which is the IP address of the LAN interface on nixbld) is not. This also happens with -P DROP. ```text iptables -w -N allow-nat iptables -w -A allow-nat -i enp3s0 -o enp0s31f6 -j ACCEPT iptables -w -A allow-nat -o enp3s0 -i enp0s31f6 -j ACCEPT iptables -w -A allow-nat -i wlp4s0 -o enp0s31f6 -j ACCEPT iptables -w -A allow-nat -o wlp4s0 -i enp0s31f6 -j ACCEPT iptables -w -A FORWARD -j allow-nat iptables -w -P FORWARD DROP ```
sb10q commented 2019-10-15 18:24:50 +08:00
Author
Owner
Copy Link

For some reason this also does not work:

iptables -w -N block-lan-from-wifi
iptables -w -A block-lan-from-wifi -i wlp4s0 -d 192.168.1.1/24 -j DROP
iptables -w -A FORWARD -j block-lan-from-wifi
iptables -w -A INPUT -j block-lan-from-wifi
For some reason this also does not work: ```text iptables -w -N block-lan-from-wifi iptables -w -A block-lan-from-wifi -i wlp4s0 -d 192.168.1.1/24 -j DROP iptables -w -A FORWARD -j block-lan-from-wifi iptables -w -A INPUT -j block-lan-from-wifi ```
sb10q commented 2019-10-15 19:39:52 +08:00
Author
Owner
Copy Link

dd490121b6 took care of this.

https://git.m-labs.hk/M-Labs/nix-scripts/commit/dd490121b60782d0570ff60e4c49e11043c6c37f took care of this.
sb10q closed this issue 2019-10-15 19:42:33 +08:00
astro commented 2019-10-15 20:27:11 +08:00
Contributor
Copy Link

Nice that it seems to work but test thoroughly because networking.firewall.trustedInterfaces affects only INPUT, not FORWARD.

Nice that it seems to work but test thoroughly because `networking.firewall.trustedInterfaces` affects only INPUT, not FORWARD.
sb10q commented 2019-10-16 13:14:28 +08:00
Author
Owner
Copy Link

Yes, it turns out that network A -> router -> router's address on network B goes through INPUT and not through FORWARD. This is what initially got me confused. And #20 (comment) is because the firewall rules on INPUT were matched first.

Yes, it turns out that network A -> router -> router's address on network B goes through INPUT and not through FORWARD. This is what initially got me confused. And https://git.m-labs.hk/M-Labs/nix-scripts/issues/20#issuecomment-92 is because the firewall rules on INPUT were matched first.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/nix-scripts#20
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?