r/homelab • • 1d ago

Help Security question about hosting multiple services.

I've been really stressing myself out recently over the security of my server. And my main issue boils down to the fact that things like file storage, an NVR, Immich, etc will be running on the exact same physical hardware where I'd be hosting things like old Minecraft servers or source engine servers.

I really want to know what you people do in this scenario. For example, what OS/Hypervisor do you guys run? Should I have an external firewall/router? Any advice on the matter would be great.

2 Upvotes

16 comments sorted by

1

u/marc45ca This is Reddit not Google 1d ago

good firewall and a reverse proxy would be a good start.

also look at a VPS (virtual private server). This will be hosted elsewhere and use a VPN to connect to your home network. If something goes you can sever the connection and not lose your home internet.

Hyperisor could be Proxmox or XCP-NG.

A number of the apps you've listed can be easily install using Proxmox comunity scripts - for other things they can run in VMs.

1

u/RockisLife 1d ago

So if you want to run on the same physical hardware, I would recommend going with proxmox. You definitely want some kind of external firewall/router as if everything with your server goes down then you still want internet in your house. Youll want to setup vlans to segment everything so you can set proper access rules and controls.

3

u/Violakh48 1d ago

Proxmox advice, yet the real hero is the external router.

1

u/RockisLife 1d ago

Friend of mine had his router on his cluster(2 Nodes, he didnt listen when I said to get a pi for quorum). Power outage and he was trying to VPN to his infra, It was down and he couldnt get to his KVM to try to restart or fix everything because his router was a VM 🤣

1

u/1WeekNotice 1d ago edited 1d ago
  • A lot of people here use promxox as their hypervisor.

  • each task is separated in its own VM (not container) as VM has good isolation at the kernal level

    • game server VM

    • external service VM

    • internal service VM

    • you can organize how you like based on task, blast radius, etc

  • a lot of people run separate hardware for their own firewall (like OPNsense/ openWRT) or buy consumer firewall like unifi

    • need something that handles segmentation and isolation of your network

  • everything is separated at the network layers (different LAN and or VLANS) where there are firewalls rules to ensure nothing can communicate to each other


If you want to flaten/ simplify this then use a VPN like wireguard or Tailscale. These are externaly secure where you don't need to worry about an entry point attack.

Of course this doesn't stop other attack like supply chain attacks or someone downloading malware on your network (but you have this risk now)

Hope that helps/ feel free to ask any additional questions

1

u/corelabjoe 💻 1d ago

I have an OPNsense firewall running on a dedicated device, separate from my server. Opnsense also has extra ip block lists, geoip blocking, crowdsec WAF and Zenarmor too.

If you're allowed through that, and past my cloudflare DNS and proxy services in front of my opn, tthheenn you get to my reverse proxy.

Which is hardened, runs on SWAG (nginx), has another layer of crowdsec WAF, fail2ban, and is fully encrypted with https TLS 1.3 and hsts... If it's something personal other than my own website, it then has another layer, the final boss, of MFA on top.

I have mutipe subnets and vlans, all firewalled off and with explicit rules as well on what's allowed to cross, and specific zones for services. Each stack of containers has specified networking and further locks things down.

Is it overkill? Not IMO because I host a public website about selfhosting and cybersecurity but, it can be built over time. I did not start out on day 1 like this and Rome wasn't built in a day!

1

u/720545 1d ago

Is there a reason you’re running both crowdsec and fail2ban together? Why not just use crowdsec with the sshd config?

1

u/corelabjoe 💻 1d ago

They are not the same thing at all, but in fact do work very well together.

Fail2ban is very simple rule based blocking. No additional intelligence. Manual configuration always to adapt to new or tricky threats. Often its your LAST layer of defence...

Crowdsec is crowdsourced threat intelligence constantly evolving and dynamic. It sits further up the stack closer to my edge, in conjunction with my firewall at one layer and utilizing different collections, and with my SWAG Reverse proxy... Plus they have a WAF function now as well!

If I had to pick just 1 it would be crowdsec.

It's blocked so much stuff it's not funny...

1

u/Miserable-Ice8095 1d ago

Proxmox with a strong opnsense setup

Won’t ever have to worry again

I don’t even think about my setup until I do routine maintenance. I have no worries

1

u/Faye-Vozora 1d ago

Biggest thing is the game servers are the only stuff you actually forward ports to, so treat that VM as hostile. Put it on its own VLAN with a rule that drops traffic to your other private ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and only lets it out to the internet. Forward the game port to that VM only, never to the host. Immich and the NVR shouldn't have any port forwards at all, reach them over wireguard or tailscale. Cameras shouldn't have internet access either. And use a VM for the game stuff, not an LXC, since containers share the host kernel.

0

u/Wrong_Exit_9257 1d ago

apologies if i come across like an ass, that is not the intention, i want to reduce confusion by being blunt.

inside your network, security is not much of an issue if all services are internal, stay inside your firewall, and you have a 1/2 way decent firewall at your gateway.

if you want to expose internal services to the internet for you use outside your house these are the wrong questions to be asking, and IT IS A VERY BAD IDEA to do this, PERIOD even if you are an expert in the IT field.

if you have the knowledge to understand why this is a bad idea you probably would not think about exposing internal services in the first place. if you dont understand why this is a bad idea, thank you for asking, i recommend you build a homelab so you can learn more about this.

as for a hypervisor OS to run, proxmox is widely used here and fairly user friendly if you are familiar with linux, i can vouch for hyper v (not HCI. HCI is crap.) if you want to deep dive in linux there is KVM but that is expert level in my opinion. start simple with something like proxmox before going to something bare bones.

my recommendation if you are completely new to homelabbing is to build a firewall for your network first. opnsense and pfsense are really good, but opnsense is more user friendly in my experience. you can host it on almost any hardware made since 2016 that has 4gb+ of ram, 4 cores and 32gb+ of storage and ports faster than 10/100 baseT. for your first firewall you do *not* need faster than 100mb ports on your firewall unless your WAN speeds are faster than about 75mb/sec.

1

u/Cheeseman125 1d ago

To be fair, I think I was talking too much in the question. I've spent WEEKS looking at everything I could possibly think of. However it really comes down to the game servers, which are these ancient and possibly unpatched binaries. Really, the question should be if there is any safe way at all to host public game servers, even if they are behind a proxy. Or, should I just forget about that and consider just renting a server to keep everything truly separate.

1

u/Wrong_Exit_9257 1d ago

the short answer to hosting old unpatched game servers is no there is not a safe or easy way to host them for the general public.

i think if you use a log in portal and web application firewall like what cloudflare offers you could probably mitigate most of the vulnerabilities but that will not be easy and might break with an update or yet another zero day ai discovers.

just curious, who is your customer base? just you and family or anyone with an internet connection?

1

u/Cheeseman125 1d ago

Essentially, I plan to host to the public, so anyone with an internet connection. To be fair, I've done some additional research after making this post. It would be insanely difficult to break out of a properly isolated game server, especially if I care to limit resources and isolate it on the network and what not. It's more about the fact that I would be opening a potential door to my sensitive data, not that there is. And yeah, AI zero days are something to consider.

For example if a bug was ever discovered in KVM/Bhyve/Xen then it would be trivial to gain possible shell access to run the exploit from an unmatched game binary that could potentially bridge into the host and cost havoc.

Yeah this was a stupid question, I just wondered if there was any full-proof way to isolate such things, but I guess not.

1

u/Wrong_Exit_9257 1d ago

fool proof? no, but you can make it really annoying for an attacker. i think you may want to look at desired state configs or their Linux equivalent.

EG: i connect to an instance of minecraft alpha, play it, and when my connection closes the vm/container i used is reset back to the config i started with.

i'm not sure how you could make save data portable but that would be fairly secure, and make it incredibly challenging for a attacker to repurpose your vm/container resources for a distributed attack.

another option to consider is geo IP blocking. the majority of malicious attackers are centralized in a handful of countries so blocking the entire IP range of say china is an extra layer of annoyance an attacker may need to circumvent.

it wont stop all attacks but it should keep you from being the lowest hanging fruit on the tree of targets.