Location, Location, Location!
The simplest way to provide good, solid physical security is to choose a secure location for your system's installation. What this means in simple terms is that if a system doesn't need to be physically available to the public, don't let it be. There are a few simple axioms that, when followed closely, can prevent nearly all physical attacks involving systems that do not need to be publicly accessible.
Keep your server and other machines in the back room
If you run a small business with enough floor space, it may seem impressive to display your Web server to your customers. However, if possible, keep all of your computer systems in a back room. When spatially possible, run monitors, keyboards, and mice on extended cabling to public areas. Keeping the processing units themselves in a back room and away from public hands solves nearly every problem discussed in this hour.
Secure the environment
Of course, no midnight thief is going to consider the back room to be off limits. Ensure that the environment in which your system runs is well secured day and night. Invest in high-quality locks as well as in an alarm system when possible, and be sure to lock up and enable the alarm whenever you leave the area, even if it's just for a 10-minute trip to the sandwich shop.
Secure the power controls
If your environment is locked tightly but your Web server is down due to loss of power, your customers are still cut off from your services. Know where your building's power sources are. Old buildings especially may have circuit breakers or fuses in odd or publicly accessible locations. A padlock for the breaker box is an inexpensive investment that can prevent all kinds of juvenile pranks, innocent accidents, or malicious attacks.
Invest in a continuous power system
When possible, invest in a continuous power system (or battery backup, as they are sometimes called) to mitigate the problem in the case of unexpected power failure. Even if you've locked your box tightly, you're helpless if the power company decides to work on your block's main line for an hour, cutting the power in the process. Having a solid, regularly tested continuous power system online can keep service up through any minor brownouts and can give you time to notify users, shut down, and prevent data loss in the event of a sudden brownout.
All of these steps should be combined with a holistic attitude toward your computing environment. Be mindful of the physical circumstances surrounding your equipment and of the public at large as a potential danger. It is time to think of security in terms of your computer's physical components, in addition to its cyber-existence.