This page describes my home server & networking setup.
The foremost targets of this venture are…
I luxuriate in three physical machines plugged into three switches, with all switches
linked to every assorted. I develop no longer luxuriate in a physical
router/gateway. As a substitute, a Linux digital machine handles the IPv4 NAT,
IPv6 announcements, DHCP, DNS, and heaps others, and that Linux VM floats between
the three machines as wanted, including are living migration all over repairs.
My four Wifi APs are PoE-powered from the 2 switches. I luxuriate in two ISPs.
I luxuriate in two UPSes and two PDUs powering separate halves of the gear,
and separate ISPs, giving me about 35-45 minutes of runtime (and thus
Web) all over a vitality outage. The total home would be unlit, however the
battery-powered wifi will work.
Better quality photos at https://photos.app.goo.gl/Y5Ah6AeGekVkf3tY9.
- 2 x UniFi Switch 24 PoE-250W: 24x Energy-over-Ethernet 1Gbps ports
- 1 x UniFi Switch sixteen XG: 10Gbps Aggregation Switch, basically for Ceph (however fragment of identical LAN). I most attention-grabbing luxuriate in one in all these, however if it fails the Linux bond fails over to the 1Gbps switches.
- UniFi Cloud Key to flee the Unifi controller. That is no longer critical as well the cluster. It perfect runs the most effective UI and is very essential so that you just can add modern devices. I would possibly perhaps per chance perhaps flee the utility on a VM too, I issue. However I had it from earlier, so I’m composed the employ of it.
- misc Raspberry Pis for monitoring
The total setup including all APs and switches attracts about 220 watts
sluggish. Energy is perfect low-price in Seattle. Washington Affirm (as of April
2018) has the most price-efficient electrical energy in the usa, at
- Proxmox VE is the Debian-basically based fully fully defective OS on the servers, and Proxmox is UI for managing qemu VMs and Ceph. I previously tried VMware for roughly a one year, both are annoying in assorted ways. Proxmox would be a minute tough in areas, however I put it.
- Ceph for storage. I love Ceph so great and discovering it makes this entire dawdle value it. Nonetheless great to learn, though.
- ISC DHCP for the DHCP server. I auto-generate its config from a Race program that has a design of most my main devices’ MAC addresses.
- CoreDNS for the DNS server on the gateway VM, which lets me encrypt all upstream DNS so ISPs can’t see or mess with it. (although they can see IPs and SNI)
- tcpproxy that Dave Anderson and I wrote. I employ it on an HA VM to route ingress traffic to a range of VMs & services and products.
- The LAN is
- Untrusted VLAN is
10.2.zero.zero/sixteen, which the LAN can connect to, however the untrusted machines can’t provoke connections reduction out to.
- Gateway, DHCP at
- DHCP fluctuate is
10.zero.one hundred-199.xso they’re straightforward to survey. Likewise for the untrusted VLAN.
- Networking gear luxuriate in static IPs
10.zero.6.x(6 is above the letter
Non the keyboard, which is how I design letters to numbers in overall)
- Ferm for simplifying writing iptables principles
- No longer sufficient yet. WIP. Realizing is to make employ of Prometheus extra.
- A Raspberry Pi has USB connections to the 2 UPSes.
TODO: link to program with dependency graph of all devices, services and products,
and connections, and to simulate failures to validate there are no
- I old to make employ of a Soekris net6501 as my home gateway, however its CPU maxes out NAT’ing about 300 Mbps, sadly, so I started having a learn at choices when I purchased Centurylink fiber.
- A truck once clipped the fiber working to our home. Or no longer it’s good having a 2d WAN link.
- I old to make employ of a UniFi Safety Gateway Pro however it failed someday and would no longer vitality on any extra. Dave had a backup for me to hand, however the Unifi controller utility wedged itself and would no longer let me determine the worn (insensible) one and thus I would possibly perhaps per chance perhaps now not add the modern replacement, since you potentially can most attention-grabbing luxuriate in one gateway able at a time. I used to be no longer amused, and that used to be the final straw that made me realize I wished a extremely-available setup.
- I old to make employ of VMware with extremely-available vCenter setup, however your whole element used to be felt bloated and leisurely and enterprisey, and I would possibly perhaps per chance perhaps now not stand the Flash UI, which used to be composed required for a range of operations. That’s increasingly going away and being modified with HTML5, however I also would possibly perhaps per chance perhaps now not stand the VMware venture-targeted documentation. And I wished to make employ of one thing Birth Source, too.
Mighty because of Dave Anderson for
serving to with fairly a range of this. He has a if truth be told identical setup at his home
and we ride staring at every assorted both be triumphant and fail at attempting modern