After years of suffering a DSL internet connection I recently switched to a provider offering gigabit fiber. Unfortunately my building isn’t setup to get a strand dropped directly into my apartment so instead I have to live with an ethernet drop (this is typically referred to as a Fiber to the Building, or FTTB, setup).
After doing away with the awful CPE router I’d been stuck with for the last few years I realized I had an opportunity to replace it with something better. Without much time to spend working on something fancy I took the easy route and pulled an old 1U pfSense box out of storage that I had previously used when I had rackspace at a colo.
After a mostly painless upgrade and setup process I stuck the box in my closet and promptly forgot about it. That is until I started having problems. pfSense is a nice solution but it’s heavyweight, it does a lot of things, whether you really need them or not. While it’s quite plausible there were configuration and/or hardware tweaks I could’ve made, debugging network slowdowns and crashes is a significant time sink. This became somewhat of an annoyance, especially so when this isn’t my day job, I just want the internet to work, so I can watch videos of cats doing dumb things.
I decided that instead of trying to relearn all of the intricacies of pfSense and it’s many features I’d just go with something I already know: basic vanilla old fashioned Linux. At the same time I decided it’d be a good idea to slim down the hardware I was using. While the 1U I was already using wasn’t exactly a beast it was somewhat dated, using an older embedded Intel processor and a spinning disk, and drawing far more power than it really needed.
The feature set I decided on was pretty simple:
- NAT + basic filtering
- DHCP
And that’s it. The trade-offs of running local DNS are, for me at least, not worth it when Google’s or Cloudflare’s public offering are fast enough and I don’t really care about using internal naming (I have few enough machines that I can mostly just remember their addresses in my head). If I need to add a VPN server or something down the line I can, but there isn’t that much need for me to be able to access my machines remotely.
Given this sparse feature set I decided on using Alpine Linux, which has a nice lightweight system, some reasonable kernel hardening enabled by default, and can be run from RAM. For the hardware I went with a PC Engines apu2d4, which has a low-power AMD embedded processor, 4GB of memory (likely more than I’ll really need), and three ethernet interfaces with i210AT Intel controllers (4 TX/4 RX queues 😍). On top of this I’m using iptables for NAT and a few filtering rules, and dhcpd to handle… DHCP, and that’s about all.
I’ve been running this setup smoothly for a while now. Management has been an absolute breeze, even with the lack of any kind of GUI (or perhaps because of it). In terms of hardware I’ve also gone from running a system drawing somewhere north of 100W to around 10W, which while not a huge change to my power bill is still nice to see. Another somewhat unexpected side effect was that I’m now able to almost completely saturate my upstream link, going from a maximum throughput of around 500Mbps to around 900Mbps. Likely this is a combination of more modern hardware (and better ethernet controllers) and a lower system overhead. To be fair I’d likely see a similar speedup using pfSense on the same new hardware, but the switch to a simpler system with significantly fewer moving parts has been good for my remaining sanity.
‐‐‐‐‐‐
For those interested in my iptables rules (I’m sure there is at least one person out there…), here is the extremely basic set I’m running with for now (with annotations to explain what is actually happening, because iptables):
# eth0: WAN
# eth1: LAN
# NAT table
*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
# Rewrite source IP on packets leaving the firewal, can't use SNAT because of dynamic IP
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
# Filter table
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
# Allow packets for established connections
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# Allow loopback packets
-A INPUT -i lo -j ACCEPT
# Allow ICMP packets from the local network
-A INPUT -s 192.168.1.0/24 -i eth1 -p icmp -j ACCEPT
# Allow SSH, DHCP from the local network
-A INPUT -s 192.168.1.0/24 -i eth1 -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -s 192.168.1.0/24 -i eth1 -p udp -m state --state NEW -m udp --dport 67 -j ACCEPT
# Drop everything else from the local network nicely
-A INPUT -s 192.168.1.0/24 -i eth1 -j REJECT --reject-with icmp-host-prohibited
# Drop everything on the floor
-A INPUT -j DROP
# Don't forward packets that come from the LAN and are going to the LAN
-A FORWARD -d 192.168.1.0/24 -i eth1 -j DROP
# Allow packets to be forwarded from the LAN that are going to the WAN
-A FORWARD -s 192.168.1.0/24 -i eth1 -j ACCEPT
# Allow packets to be forwarded from the WAN that are going to the LAN
-A FORWARD -d 192.168.1.0/24 -i eth0 -j ACCEPT
# Drop everything else from the local network nicely
-A FORWARD -i eth1 -j REJECT --reject-with icmp-host-prohibited
# Drop everything else on the floor
-A FORWARD -j DROP
-A OUTPUT -j ACCEPT
COMMIT