Every network fails in a handful of classic ways, and each one has a shape you can learn to recognise on sight. This series walks the most common failures — what the user sees, why it happens, an animated diagram of the mechanism, how to catch it in thirty seconds, and how to stop it coming back.
The architecture guides show how to build networks right; this series shows how they go wrong. Each post takes one failure mode that every network engineer meets eventually and dissects it: the symptom as the user reports it, the mechanism with an animated diagram, the field check to confirm it in seconds, and the fix. The companion series, Surgery on a Running Network, is how you change a live network without causing these.
They are filed under the broken-by-design category on the blog.
Layer 2
The broadcast domain is a shared fate: a loop or a lie on it hits every device at once.
01The Switching LoopA second cable “for redundancy” with no spanning tree: a broadcast circles, re-floods and multiplies until every CPU is pinned and the site is down. STP, BPDU guard and storm control, animated.7 min→ 02The Gateway That LiedARP has no authentication, so one device can claim the gateway IP and redirect the subnet through itself — or a duplicate IP does it by accident. Dynamic ARP Inspection stops both.6 min→Layer 3 & the WAN
Routing and firewalls that each work alone combine into one-way valves, black holes and flapping links.
03The Return Path That VanishedTraffic leaves one way and returns another, into a stateful firewall that never saw the connection start — so it drops the reply and the connection half-opens. Pin flows symmetric.7 min→ 04The Ping Works, the Transfer HangsA packet too big for a tunnel link, a blocked ICMP that would have said so, and a sender that keeps resending into the void. Why pings pass and transfers freeze — and why MSS clamping fixes it.7 min→ 05The Failover That FlapsA bad health probe makes the default route oscillate every few seconds, resetting every session — worse than the outage it was meant to cover. Make failover rare and sticky.6 min→Addressing & Services
DHCP and DNS are invisible until they fail, and then nothing new works — or everything goes slow.
06No One Can Get an AddressNew devices get a useless 169.254 address while everything already online keeps working. Starvation, an undersized pool, or a rogue server — three causes, told apart by the lease list.6 min→ 07The Resolver That Needed Itself“Everything is slow” on an idle link: a forwarder loop, a resolver that depends on what it resolves, or a dead first entry taxing every lookup five seconds. Break the loop, fail fast.6 min→Queueing
Bandwidth is not latency, and a full buffer trades one for the other.
08The Link Isn't Slow, the Queue Is FullLatency explodes under load while the speed test looks perfect: an oversized buffer fills and every urgent packet waits behind the backlog. Why more bandwidth does not help, and why AQM does.6 min→I audit networks for exactly these failure modes — loop exposure, asymmetric paths, MTU and ICMP handling, DHCP and DNS resilience, bufferbloat — and fix the ones you are one bad day away from meeting.
Book a Discovery Call →