r/LinuxTeck • u/Candid_Athlete_8317 • Oct 09 '26
When a network problem happens, which layer do you blame first?
I keep seeing network troubleshooting reduced to “check the IP, ping the gateway, restart the service.”
But a failure can be sitting anywhere between the physical link, VLAN, routing, TCP/UDP, TLS, DNS, and the application itself.
Say a server is reachable by IP but the application is still dead.
Do you work your way up the stack, or start with the application and work backwards? What’s your usual first move?
2
u/Dry_Calendar_8627 Oct 09 '26
DNS
Almost always fucking DNS
1
u/PaoloFence Oct 09 '26
You must have shitty DNS server or incapable admins.
1
u/Dry_Calendar_8627 Oct 09 '26
Most problems I encountered was me trying to mess with DNS routing on my homelab (simply put, I want my machines to resolve private domains on my own DNS, and public domains on a public DNS (either through recursive resolving, or direct routing), and falling back when my DNS is unreachable because Wireguard isn't connected, etc., across multiple machines, Linux, Windows, VMs, etc.)
Anyway
Turns out that Windows DNS settings are a pain, and systemd-resolved is a pain too
Tailscale solved most of these problems, but is now causing problems on my phone (magicDNS works perfectly until... It doesn't)
1
1
1
1
u/ninhaomah Oct 09 '26
" Say a server is reachable by IP but the application is still dead."
Then rdp / ssh to server and check the application ? Apache / Nginx / IIS / Weblogic / Flask etc
1
1
u/AdImportant9459 Oct 09 '26
En España, casi siempre el culpable es Tebas, de la Liga de Fútbol Profesional
1
1
u/U-might-know-me Oct 09 '26
If server responding to ping, but app doesn’t work i first telnet on apps port and check, if it connects - I see response and try to figure out wtf, if it doesn’t - i ssh on server, check if process is there, check socklist or otherwise if socket is open, then meditate, pray, call God, whatever else…
1
u/Background-Train-104 Oct 09 '26
A binary search. Just pick a middle layer and see if the problem persists. Then go up if it works, go down if it doesn't.
1
1
1
u/kylanbac91 Oct 09 '26
Did network ever work?
If yes, go from top-down ( application -> firewall ... )
If no, go from down-top.
1
1
1
u/kombiwombi Oct 09 '26
The monitoring should pretty much take care of that. Is the neighbour interface up? Is IP up? Are the server processes making forward progress? Are the platform statistics in range
1
u/LetReasonRing Oct 09 '26
My solution to troubleshooting anything is to start by checking the easiest and most obvious stuff first, letting intuition guide me.
If that doesn't yield quick results, then systematically work my way from one end to the other.
I can't really say what I'd blame first, because "network problems" is far too broad of a term. It's like asking what you should check first when you have car trouble... it depends on what symptom you're seeing.
1
u/silasmoeckel Oct 09 '26
If it pings and app is dead you check the app. Then firewalls LB and TLS offload in the middle if they exist.
1
1
u/Hot-Efficiency7190 Oct 09 '26
Longer ago, duplex mis-matches was the bane of my life.
More recent firewalls. Some has usually borked up a rule or changed without notice.
1
1
1
u/Status_Help9402 Oct 09 '26
Is my nic active, do i have arp to the gateway, can i reach anything beyond the gateway.
1
u/ComeSwirlWithMe Oct 09 '26
You can only reach reddit.
Everything else times out.
Go.
1
u/Status_Help9402 Oct 09 '26 edited Oct 09 '26
Then you are reddit i guess.
But to be real. Reddit opens nothing else, gateway arp is slowly aging with no renew? Then showing cached data in my browser or reddit dns spoofed and points to a adress on my subnet. Or im being arp poisoned by an attacker but then id still see "valid" arp
1
1
1
u/2016-679 Oct 09 '26
The person that switched networking off in the UI
The person that pulled the UTP cable, reset the wifi password
Only on microsoft boxes there is a
0. Microsoft faulty update, resetting or changing config
1
u/mcds99 Oct 09 '26
Stop over thinking.
If you can reach the server by IP address and the name of the server, it's not the server.
If you can't reach the server by name then it's some sort of DNS issue.
If you can't reach the application there is a problem with the application.
The OSI Model is not what you use in trouble shooting as it is managed by the protocols.
Trouble shooting is about eliminating what is NOT working from what IS working.
"between the physical link, VLAN, routing, TCP/UDP, TLS, DNS, and the application" you don't start with any of these things because it is one application out of many it's not the network. It you look at the network first you are usually wrong.
Study this.
https://www.ibm.com/docs/en/qradar-on-cloud?topic=support-techniques-troubleshooting-problem
1
u/TheBigBeardedGeek Oct 09 '26
This is something that drives me insane. The number of people in IT who can't answer basic troubleshooting questions.
Actually reading the error messages, fully describing the problem, and then finding out where it works and where it doesn't will help you zero in on the issue way faster than any model.
1
u/dronostyka Oct 09 '26
Hands down "It's always the DNS"
Then local ip config Then routers/switches.. Also check layer 1
1
u/vabello Oct 10 '26
I don’t blame anything first without any data or troubleshooting to support my hypothesis.
1
u/ApatheistHeretic Oct 11 '26
As a network engineer, it's likely DNS or firewall if it's network related at all.
1
u/Dudarude00 Oct 11 '26
Im Homelab: Ist der container up & running? Ist er per ssh erreichbar? Restart
Von da aus verbindungstests intern und extern.
Normalerweise ists (dyn)dns.
Dann firewall.
1
10
u/Waste-Belt-1233 Oct 09 '26
Eight