This chapter is from the book
This chapter is from the book
This chapter is from the book
Documentation Dividends
Enough with the dire warnings and horror stories! Let's set aside all the negatives due to a lack of documentation. On the positive side, you can see tangible benefits to having an acceptable level of documentation. Perhaps the one benefit you'll appreciate most is that a documented network is a network from which you can walk away. You can take a vacation from a documented network without worrying about whether you're going to be called if something goes wrong. Because you're not the only person with access to information about how your network is set up, others can deal with it while you watch the sun rise on your beach vacation without your pager or cellular phone ringing. The work day is already occupied enough without having to drag somebody around to show her all the nuances of the network; your network documentation does this for you, leaving you time to do other work. Table 3.1 describes the major types of network documentation.
Table 3.1 Types of Documentation
|
Type |
Purpose |
|
Functional map |
The "org chart" of your network, this type of documentation gives an overview of how data flows in general. This is the "black box" part of your documentation. It likely leaves out individual workstations and wire runs and simply shows the important parts of the network (such as critical servers, routers, and network segments), along with how they connect. |
|
Physical documentation |
This "nuts and bolts" documentation shows very specific information about the network, including wires, hubs, switches, and workstations. After you have used high-level documentation to aid in troubleshooting the problem flow, and you suspect a physical problem, use this to guide your specific fix. |
|
Logical documentation |
Shows the nonphysical, or "virtual" parts of your network, servers, and so on. VLAN documentation is a good example of virtual documentation because VLANs aren't physical entities. (A wireless LAN's SID is another good example because there can be many different SIDs on the radio waves.) |
|
Device and cable labeling |
A type of physical documentation, it deserves its own category because it is highly important. Consists of physical labels that identify the devices or cables they're attached to in big, bold letters. This allows someone to find the equipment that has been documented. |
|
Descriptive (or detail) documentation |
Usually consists of an organized write-up of an application or system, lists, or an everyday log of what's done to a device or system. It includes anything that's not intuitive and not included in the manufacturer's documentation. For example, if you use static IP addresses, you really want to make sure that you document them in a list format to make sure that you don't issue duplicates. |
Why do you need both functional maps and detail documentation? Well, when applying black box techniques to documentation, keep the "encapsulation" principle in mind: It is best, initially, to have the details as hidden as possible, that is, "in the box." So, you don't always necessarily need the highest level of detail on each map; sometimes, the 20,000-foot view is the correct view, sometimes you need a street map.
For example, when initially planning a trip from Atlanta to Savannah, it makes sense to have a Georgia map, not a map of the United States because you are doing the "high level" planning: You want to know the approximate distance, the highways you'll use, and so on. Then, assuming that you know how to get to the Atlanta highways, you'd switch to a Savannah street map to see how to get to your final destination. Similarly, when you are documenting something, you want to make sure that you provide "detail" documentation, as well as the "high level" documentation.
Functional Maps
Functional maps are probably the type of documentation that you'll initially refer to most often. After all, this is what being a "black box" troubleshooter is all about: starting high level and then moving to the details. Functional maps are used as a sort of "org chart" of your network, and, appropriately, they indicate how data travels and what choke points exist. By looking at this map, you will be able to determine which devices depend on other devices in order to work. Details are not as important; again, flow is what you're looking for. You'll want to be able to determine, without being confused by unnecessary details, why department A can't talk to the server, but department B can.
Is there more than one type of logical map? Are there hybrid types of maps?
You can bet that in most organizations, there will be. For instance, perhaps you have a map of wire closets in a large building (see Figure 3.1). This might be a good thing to refer to when you are receiving calls about a total outage from certain departments; it is both a functional map and a physical map!
){kind=link}
Figure 3.1 A functional map of physical closets.
You might also have a map (shown in Figure 3.2) of the various VLANs (Virtual LANs) that exist in the building, which is separate and distinct from the physical map. Because VLANs aren't a physical construct, they are considered logical entities.
){kind=link}
NOTE
I touched on VLANs in Hour 1, "Understanding Networking: The Telephone Analogy," but don't worry if you don't understand what VLANs are at this point. You'll get more information on VLANs in Hour 10, "Ethernet and Switching."
Figure 3.2 A functional map of logical entities: VLAN layout.
Obviously, not every PC or printer needs to be on a functional chart, but every device that somebody else relies on (such as a mail gateway or a router) for data flow does. You'll need to make a per-case decision whether to include individual devices on this type of chart, based on whether these act as a group or are standalones. In the case of Figure 3.1, the documenter decided to combine all the switches in a closet into one object and represent them with one symbol because, for this map, showing all the switches would clutter what is a very simple "major flow" map. The person who drew the map decided that several facts were important:
How many ports were available in the closet
How many ports were in use in the closet
How the closet switches connected back to the data center
If you have a network that's complex or larger than one site, it's time to grab another piece of paper. (It's not worth it to try to cram everything into one map—that just makes your map hard to read.) Typically, an organization with multiple sites needs both a big-picture view, as well as separate site maps. In your big-picture view, each site will be represented by a symbol that refers to a separate map and indicates how that site links to other maps. See Figure 3.3 for a sample map of a multi-campus network map. (Certain software, like WhatsUp Gold, enables you to create a map that has other maps contained within it.)
){kind=link}
Figure 3.3 A more complex functional site map, showing the entire enterprise.
Physical Versus Logical Maps
Physical maps, of course, refer to the physical world—that is, how things really are. Logical maps refer to virtual constructions: how physical devices interact to create a virtual entity. The two are often related. For example, every modern operating system has the concept of "virtual memory," that is, a total amount of memory that programs can allocate. Although virtual memory is a logical construct, it is made up of two physical constructs managed by software: RAM memory and disk space. The OS (operating system) software manages the two physical constructs so that all a program has to do is ask for memory. The OS's memory manager will "swap" out the RAM to disk as it runs out of RAM. The point I'm trying to make is that every virtual construction has a physical representation.
Physical maps can be incredibly detailed, so you'll want to chew them off in bite-sized chunks. Some sites (usually those with fewer than 50 network runs) can get away with just one physical map. Most sites usually need a physical map for each floor of the building. This is usually a good breakdown for most sites because it shows each and every wire running along with each and every PC, printer, switch, and hub; this can get rather large. Accordingly, for simpler sites, it's really terrific to be able to lay your hands on the architectural floor plan of your building and add the network wiring and wire closet layout to it. If you can't get the actual floor plan, you can draw an approximation when you are labeling either wire runs or wall box locations (see Figure 3.4).
){kind=link}
Figure 3.4 A floor plan labeled with major landmarks and with wall box locations.
NOTE
The world of VLANs can make what used to be a straightforward documentation process somewhat more complex. VLAN assignments of a given switch can change quite often as users move. You will definitely want some sort of method for knowing which physical ports are assigned to which VLAN. (If you're using a Cisco switch, typing show vlan will quickly print out your VLAN assignments.)
Most folks keep some sort of physical documentation worksheet; if you have VLANs, this is one place where your logical and physical worlds collide. See Table 3.2 for sample headings for this worksheet.
Table 3.2 Physical Documentation Worksheet Sample
Frobozz Corporation Closet Documentation Worksheet
|
Switch # |
Port # |
Wall Box # |
Tel# |
VLAN |
Description |
|
1 |
1 |
A1 |
x7501 |
2 |
Jon |
|
1 |
2 |
A7 |
x7750 |
2 |
Scott |
|
1 |
3 |
A5 |
x7505 |
2 |
George |
|
1 |
4 |
A2 |
x7703 |
2 |
Matt |
|
1 |
5 |
A3 |
x7708 |
2 |
Waldo |
|
1 |
6 |
A4 |
x7710 |
3 |
Hakka |
|
1 |
7 |
A8 |
x7506 |
2 |
Leppitsch |
|
1 |
8 |
A11 |
x7700 |
2 |
Gabe |
|
1 |
9 |
A10 |
x7801 |
2 |
Shelly |
|
1 |
10 |
A9 |
x7508 |
2 |
Vlad |
|
1 |
11 |
A5 |
x7702 |
2 |
Olivier |
Should this worksheet be hand-written or typed into your computer? It depends on the size of your organization. You might discover that a pencil and paper works as well as an Excel spreadsheet for a small organization. However, the principle is the same whether you are using NetViz (http://www.netviz.com) or Visio (http://www.Microsoft.com) for 10,000 users or a #2 pencil for 20 users.
No matter what type of network you have, a physical documentation worksheet can act as a quick reference sheet for all data about that particular physical network. Need different headings? By all means, use them. Some folks used to record the physical (MAC) addresses of their computers on these worksheets.
In general, when in doubt, you'll want to get as detailed as you can. (I've never seen someone crying after a network outage because the maps were too detailed.)
Tips for Mapping
Here are some tips that will help you manage your mapping:
Update your maps. Things change, and your maps need to show that. Make this part of your SOP (Standard Operating Procedure) with your consultant, network provider, or PC guy—particularly if you add or change things on your own.
Date your maps! Because things do tend to change, you'll end up with multiple maps of the same general area. If you have a date on each map, it's easy to figure out which map is current.
Use symbols instead of color. Using color in your maps can make them easier to understand, but you might find it harder to share your maps with associates and vendors who are trying to help you. (Although color copiers are pretty pervasive now, color maps are hard to read via fax, for example.) An alternative to color is the use of differently shaped symbols for items that need to stand out.
Use simple software (or a pencil). Software for mapping or flowcharting can help you a great deal, but don't make the mistake of laying out your network map at the same time that you're learning the new software package. Draw your network map out by hand if you're unfamiliar with the software—this will make the layout go faster. Software is great because minor changes are easy to update. (Besides, having an "autoflow" feature for line drawing can really make your life easier!) However, complex software just isn't necessary—any simple drawing program will do. In other words, you don't need to make a career out of network mapping when you've got troubleshooting to do.
Some of the possibilities include
-
SmartDraw (http://www.smartdraw.com). Good flowcharting capability and great symbol library (clipart).
-
Visio (http://www.microsoft.com). The Microsoft Word of network diagramming.
-
NetViz (http://www.netviz.com). This is a drawing utility, but it also has the ability to interface with and draw information from various databases.
Keep it simple! Other people will have to read your document, and you want them to actually understand it. Accordingly, try to stay with straight, clean lines and avoid flow chart spaghetti (crossed lines). If you have more detail than you can really fit into one page, add another page.
Device and Cable Labeling
A map does you no good, of course, if you can't find a device that's shown on the map. Don't laugh—I've seen plenty of shops in which nobody knows where the router is. (Naturally, you'll want to include the physical locations on your maps.)
But if the router isn't labeled, it's hard to know which router it is. How can you reboot the router if you don't know which one it is? A good troubleshooter makes it a point to label each device clearly and concisely before trouble strikes; a lack of labeling can really increase your downtime by a remarkable margin.
NOTE
If you ever visit the USS North Carolina or a similar World War II battleship, you'll marvel at all the hoses and pipes that run throughout the ship—particularly in the machine rooms. Take a look at the meticulous way all these pipes are labeled; believe me, they didn't do this for fun. They needed to know which pipes were air, steam, fresh water, brine, gas, or oil—a delay in a repair could literally sink the ship.
If you weren't fortunate enough to have had the U.S. Navy meticulously label your network (and your installer hasn't done it either), you'll need to start today with "as you go" labeling. Each time you find out what a device is or where a cable is going, slap a label on it. Don't assume that you'll remember—you might have a great memory, but during a crisis, that memory might be on vacation. (Also, remember that you want to go on vacation yourself one day and let someone else troubleshoot all this stuff for you.)
Go out and buy a label maker. They only cost about $30 and are really useful. They pump out quick, clear, and neat-looking labels. Also, you can print a label twice, cut once, and have a two-sided cable label, as shown in Figure 3.5. I've used models by Brother and Casio with good results.
){kind=link}
Figure 3.5 Use a label maker to make two-sided cable labels.
Tips for Labeling
You'd think labeling is a fairly simple and straightforward project, but here are a few practical pointers that you might not have considered:
-
Test labels for adhesion before you buy. There's nothing worse than making 250 labels, only to see them all flutter to the ground after a month. As with anything else in networking, a simple pilot test can save you a big headache later.
-
Paper or plastic? Cable labels should be plastic rather than paper, to avoid their ripping as cables get handled.
-
Can you read it? Any cable label should be put a good distance from the end (anywhere from 12 to 18 inches, if you have the room). This way, you can read the label once there are 50 or so other cables plugged into the same switch or concentrator.
-
Keep them current. You really need to keep up with your labels when items change or are moved. Troubleshooting can take much, much longer if you think you know which cable is correct but find out later that it's not.
Let's say you have a router that connects your earthbound headquarters with your company's moon base. If the router's name is LOONIE, a good device label would be
LOONIE: Router for moon base, Lockheed Satellite transmitter
I've seen folks slap a label with an IP address on such a router; however, this is optional if you have good functional documentation clearly stating that LOONIE has two IP addresses (say, 167.195.160.1 on the earth side and 167.195.161.1 on the moon side). Still, as my grandmother would say, "It couldn't hurt." LOONIE's label including IP addresses could look like this:
LOONIE 167.195.160.1 (EARTH Interface) 167.195.161.1 (MOON Interface)
LOONIE has two connections in the back (one to your local area network and one to the satellite transmitter); you should label each connection with the device name and its endpoint. For example, the cable going to the satellite transmitter could be labeled:
LOONIE -> Satellite Transmitter
Then, the other label might be
LOONIE -> Data Closet #1, Hub #3
Why label the cable that's directly connected to LOONIE as "LOONIE"? Well, suppose that you have to disconnect the cable. It would be nice to know where to put it back. Perhaps even more importantly, suppose that someone else disconnects the cable and you (or anyone else) has to put it back. Those labels can be very important.
I like to label each end of a cable with both devices that will be connected with the cable so that I don't have to think about which end to put in the cable. I've been at sites where some poor, confused soul has labeled each end with the source rather than the destination, so all you know is that the LOONIE router has a cable that goes to, uh, LOONIE. Oops! It's best to look at cable labeling from both sides. (After all, you bought a $30 label maker, and making labels is easy.) It really does save aggravation.
Detail/Description Documentation
The difference between a well-documented site and a professionally documented site is in the details. Detail documentation—although the least practiced documentation type—can augment the other types of documentation very nicely.
Detail documentation in the form of a formal write-up of how a system was configured, along with site-specific standards or frequently used troubleshooting practices, can make it easier for others to retrace your footsteps. This type of documentation, which includes how-to's and cheat sheets, contributes greatly to your ability to actually take a vacation.
The Art of Logging
You're the captain, so keep a log. It's hard to find a block of time to sit down and write a formal description of previous troubleshooting techniques, but if you keep a logbook near each crucial device (such as main servers, firewalls, routers, and so on), you can document as you go. Make it part of your SOP to write down anything done to a server, switch, or router (and at what date and time this operation was performed)—you'll save yourself a bunch of time when it comes to retracing your steps. Logbooks are also great for providing backup documentation with a vendor. For example, an entry such as "The new server has crashed each week at 10:00 on Monday night" might lead a vendor to realize that the hard-drive cleanup program scheduled to run on the server at 9:50 on Monday nights might be causing a problem.
Do paper logbooks get lost? Oh, heck, yes—particularly by "nobody."
For those of you who are part of a large IT staff, it might be a good idea to use some sort of centralized change tracking system. I'd be willing to bet that your help desk system has some sort of change management function built into it. If not, this is a really simple kind of intranet application to build. I've seen quite a few custom-written Web log programs. (One hint: Web programmers like cookies and become quite receptive to suggestion after a sugar glow.)
Dealing with Undocumented Networks
Is it possible to reverse engineer an undocumented network? Sure, but this requires a good bit of knowledge. What's more, it's really a waste of time if the network is being installed or changed while the documentation is taking place. The sad part is that most networks aren't very well documented; most professional troubleshooters readily agree with this. It's no wonder that many networks have so much downtime: avoid it by having good documentation practices.
If you run into someone else's network nightmare, see Hour 24, "Reverse Engineering, Discovery Tools, and Other Black Magic."
Is this a hint that you should insist on good documentation when you contract out for someone to build a network for you? Yes! Any professional worth her salt will probably be labeling like crazy anyway, but just in case you run into someone who thinks that an undocumented network equals job security, you need to get tough. Insist on labels on all cables and ask for maps. If you have to pay more, either pay a reasonable amount, which should be nominal (plus you'll save time and money in the long run), or find a different vendor. A lot of vendors are out there, and, ultimately, you're the person who either suffers or benefits from the documentation of the network and cable plan.