by Laura Chappell [Released July 17, 2000]

Originally, I thought I'd write a book out of this topic --and maybe someday this will evolve into a book… ya never know… Anyway… here are the 10 Truths of Network Troubleshooting.

TRUTH #1
Perfection is a Myth; There is No Such Thing as a Perfect Network

Ah, yes… I remember going onsite to a company and really hoping that I could find some problem with their network - after all, that would justify the bucks they'd spent on bringing me out there. What if the network was 'clean' ? What would I do… just sit in front of the client and say … duh… ok… well…. Lookin' good here. Then write up an invoice and sneak out the back door? My mind reeled with the thought of working on a network that has no errors…. What a fool, eh?

Undoubtedly, some bozo is going to email me and say that his/her network is 'error-less'. (If that's you, just stop now and take a friggin' break, would ya?). Oh, come on, now! The only error-less network has one server and one workstation - no users, no applications, nothin'!

If you think you are working on a network that is completely without errors, you either have a lousy analyzer, blind management system, really strong drugs, or have not looked hard enough at the communications. Look all the way up to the application layer -- you'll certainly find some errors roaming about.

TRUTH #2
The Customer is Often Right

During an onsite analysis visit, I always ask the local IS staff, "What do you think the problem is?" It's amazing how often the local folks know exactly what is going on, but management has chosen not to believe them.

In many cases, I am hired as a 'validator' to simply regurgitate what the local IS folks said. In other situations, I am simply gathering packets and presenting the IS staff viewpoint in a graphical format.

TRUTH #3
Every Network has a Fred (User From Hell)

I believe that 1 out of every 10 users on a network is a 'Fred, User From Hell' (F,UFH). Fred is the user that (intentionally or not intentionally) screws everything up on the network. He typically collects and displays all the networking books, but doesn't read them.

Nowadays, we need to expand the definition of Fred to include the 'tinkering hacker' on the network. You know the one -- the guy that has ping running on his computer in the background all day long and he doesn't know it. In the case of high school interns, you have a little Fred, Jr. on your hands. Be alert!

Note: If you look around you and find that nine other people are NOT Fred…. consider writing up a new resume.

TRUTH #4
You Cannot Ignore the Damn OSI Model

No matter how much you try to live your networking life without the OSI model, you can't. So just buckle down and learn it. There are hundreds of resources around the Net and in textbooks so just go grab something and lock yourself away for an hour. Pay particular attention to the physical, datalink, network and transport layers -- since those are the layers that move data around the network and define the upper layer applications in use.

TRUTH #5
Packets Don't Lie

Not to imply that people do, but… It's always best to get solid proof of what is happening on your network. The best way to do this is with a protocol analyzer. Get the packets, print 'em out, build the charts and graphs.

TRUTH #6
Troubleshooting is like Tennis

In troubleshooting, like tennis, there'll always be someone better than you and there'll always be someone worse than you. You can't know it all - don't try. Build up solid resources; know how to research. Also, try to remember that troubleshooting is an art, not a science -- the good troubleshooter has strong people skills and deductive reasoning skills, and showers on a regular basis.

TRUTH #7
If It Feels Wrong, It Probably Is

Trust your intuition… Consider that if you wake up and go to the mirror to find that you have a vegetable growing out of forehead you've just gotta say to yourself, "No, that's just not right…. Something's wrong here." C'mon… it's really frustrating to see how many times people bypass their intuition.

TRUTH #8
Nothing is Automatic

Ok… say this one out loud -- Auto configuration is evil! Do you trust the vendors to pick and choose stuff for you? We let the vendors choose simple things (like the MAC address of a station) and that's about as far as that should go. Don't let the vendors choose your frame type or your IP addressing scheme please! The frame type problem is documented in "Why Auto Frame Typing is Evil" and the IP address assignment issue is defined further in the videotape lecture contained in the "Laura Chappell presents… Packet-Level DHCP" Certificate Course.

TRUTH #9
Every Network Protocol has a Personality

It's true…. you can tap into the cabling system and find that all types of personalities flying about - Consider the following personalities -- got any on your network?

Servers that SAP all the time -- they just sit there and broadcast their information (as if we all care what they have to say). Blah, blah, blah…. droning on.
Applications using UDP and IPX as their transport… let's face it folks, UDP and IPX don't give a (&*#$&* about your data. Your precious little packets could be headin' out into the ol' ether for all they care. They're connectionless and proud of it.
Applications using TCP, on the other hand, are quite militant. Salute when you say that, buddy! Formal handshaking and a need for the 'Yes, Sir!' acknowledgments make TCP the right transport for the mission (mission-critical data, that is).
Applications that use SPX are militant, but lame. Think of Don Knotts dressed in a sergeant's uniform. (Too young for Don Knotts? Ok… Picture Ricky Martin being a tough guy… oh, stop… I'm laughing too hard!).
Token Ring stations are all just a bunch of whining 5 year olds -- tattling on each other at the drop of a hat (or token). The problem with this type of network is that you just know these devices aren't ever going to grow up -- it's not just a phase .
Picturing your poor little pathetic devices and applications in this way will help in two ways -- first, you might begin to feel some pity for them -- they know not what they do! Second, you will realize that you are more than just a network troubleshooter… you are a shrink to these misguided annoying systems.

TRUTH #10
IS Stands for Inferno of Servitude

Yes… you must have done something really lousy in a past life to be handed a job in IS. Perhaps you were the one who invented high heel shoes or ties. Maybe you thought up the idea of panty hose or musical toys for 3 year olds (without a volume dial or headset jack). In this case, you really deserve your life -- so enjoy it.

Get the PDF version of this article.

Laura Chappell
Sr. Protocol Analyst
Copyright 2000 Protocol Analysis Institute, L.L.C.

Other Articles:
• Catching the Lovsan Worm in Action [PDF]
• Time is of the Essence
• The Wonderful Thing About Triggers... [PDF]
• The Pain of Gnutella
• About the 2301 Traffic
• 10 Cool Things You Can Do with the EtherPeek Demo [PDF]
• Basic Packet Filtering [PDF]
• Advanced Packet Filtering [PDF]
• Looking at the Sniffer Dashboard [PDF]
• TrenchTime: Ports to Watch
• Did Your Know: Wireless Networks are Not Immune to Sniffing? [PDF]
• The 10 Truths of Network Troubleshooting [PDF]
• Carnivore? [PDF]
• Sniffer: Using the Capture Panel [PDF]