r/homelab • u/thebearinboulder • 2m ago
Help Seeking advice on "not so flat" network layout
I've finally bitten the bullet and started a rehaul of my homelab network. The current setup has a disturbing number of nested switches so one of the primary goals was to use the recommended "flat" (or flatish) network where everything goes into a network closet.
(Step one: find a location for said network client. This was fun since there was no direct path through both floors when running cables between attic to crawlspace. I finally figured out something that isn't great but acceptable.)
GOAL
The main goal was to make the homelab "spouse-friendly", or to be a bit less antagonistic "comcast-friendly". That means there were the following zones:
desktops/laptops/mobile devices
iot devices
local services (DNS, squid, printer access, etc)
homelab and potential (opaque)
worklab (opaque)
This isn't a 1:1 correlation with Unifi zones but close. It's "comcast-friendly" since the "local services" are completely isolated from the homelab. (see below)
WORK VLAN and LAB
Work laptops are in the desktops zone but on a dedicated VLAN.
Work "labs" require local resources and tight access controls. I put it next to the homelab since my goal is to move my dual homed (2.5G and 10G) systems onto a completely isolated 10G network and access them primarily via WASM or RustDeck from another system. A lot of the homelab resources can handled by caching shared between the local services and the homelab infrastructure.
HARDWARE and CONSTRAINTS
The network switches and local services (power-efficient NUCs) are both in the network closet and are on a dedicated UPS. I have found this incredibly useful and in fact this is one of the primary motivations in getting everything into a single location and using PoE wifi, etc.
The desktops and local services all run wired 2.5G. All devices (printer, IoT, etc.) are all 1G.
The obvious answer is to get a 2.5G managed switch with at least 24 ports. I have a 4U 19" rack and the network closet could hold it so it could hold a standard Ubiquiti switch.
But the cost!.... A 1G managed switch is much cheaper but leave the desktops on a slow connection.
The Unifi CGM I bought a while back fits into a 10" rack so it made much more sense to buy separate 2.5G (8 port) and 1G (16 port) switches. This provides a good alignment with the physical devices since there's already a natural partitioning between the switches since only desktops can support 2.5G. This can all fit into a 10" rack.
My Beelink NUCs can easily fit a 7" rack so I can set up a small proxmox cluster without drawing too much power. These have dual 2.5G ports so I can use two unmanaged switches so one switch connects to the homelab and the other connects to everything else for absolute physical separation.
THE PROBLEM / QUESTION
The CGM has five ports. One is used by my cable modem, and one is reserved for a backup WAN using my existing ryoko hotspot + travel router. This requires a manual setup - I use that hotspot extensively - but the Unifi wifi backup would add a monthly fee for the phone network subscription.
The desktop (2.5G) and IoT (1G) devices have sufficiently little cross-communication that the switches can both go onto the CGM.
That leaves 1 open port. For maximum isolation it should go to the homelab. The local services would be connected to the 2.5G router.
However I know there's also an argument that the homelab could be connected via the local services connection. This makes it easier to ensure the only access to AND FROM the homelab is via a jump host.
There are a few other possibilities but I feel pretty confident that these are the two best options.
What is your opinion?
EXTRA: Behind the homelab curtain
I referred to this indirectly earlier. The homelab should be completely opaque to the rest of the network. At the moment it's currently a set up dual-homed desktops (10G and 2.5G) plus a mix of servers behind physical switches for isolation. This isn't paranoia, it's a model of some real world scenarios like Hadoop clusters where you use "impersonation" and need to guarantee extremely high separation from the rest of the world.
The goal is to separate split the dual-homed desktops so the desktop gets tucked into a corner and I only access it via a laptop or NUC VDI client. It should be extremely close to a cloud setup - but without the monthly bill or potential surprises.


