Building a Secure MSP
The MSP (midframe service processor) is the gateway between general purpose internal networks and the private SC network. As such, it controls access between these networks. To effectively protect it against unauthorized access, harden it and implement encrypted access mechanisms.
Hardening is critical to the security of the SC because the default configuration of Solaris OE does not provide the required protection for the MSP.
The recommended Solaris OE installation for the MSP is the End User Cluster rather than the Developer, Entire Distribution, or OEM Installation Clusters. Using the End User Cluster significantly reduces the number of Solaris OE packages installed on the MSP.
Hardening the MSP consists of performing the following tasks:
"Configuring Network Topology"
"Installing Apache Web Server"
"Adding Security Software"
"Installing Downloaded Software and Implementing Modifications"
"Configuring the MSP SYSLOG"
In our example, we use the Solaris Security Toolkit software and the FixModes software to secure the MSP. The Solaris Security Toolkit implements recommendations made in the Sun BluePrints OnLine security articles. These recommendations are documented in the following articles:
Solaris Operating Environment Security: Updated for the Solaris 8 Operating Environment
Solaris Operating Environment Network Settings for Security: Updated for Solaris 8 Operating Environment
The Solaris Security Toolkit - Installation, Configuration, and Usage Guide: Updated for version 0.3
You can build the MSP either through an interactive CD-ROM-based or Solaris JumpStart installation. The Solaris Security Toolkit software can be used in either type of installation. Refer to the Sun BluePrints OnLine article The Solaris Security Toolkit - Quick Start: Updated for Version 0.3.
Configuring Network Topology
Configure the SC on a private SC network, using the MSP as a non-routing gateway to provide a secure access mechanism between general purpose networks and the private SC network.
In this section, we show a sample network topology containing one Sun Fire 6800 server, two SCs, and one MSP. You can extrapolate other architectures from this sample design. The systems in this topology are as follows:
FIGURE 1 shows a logical diagram and does not include all of the components required to make this sample environment function. Specifically, the network switches required are not addressed. We recommend that you use separate network switches for the private SC network instead of VLANs on a larger switch. Whichever switch you use for the private SC network, we recommend that the switch be managed and monitored the same way as other switches in the environment.
FIGURE 1 Sample Network Topology Configuration
The network diagram illustrates the separate networks we use to isolate the SC from general network traffic. The general network (192.168.0.0/24) is not routed to the private SC network (192.168.100.0/24), because IP Forwarding is disabled on the MSP.
Two access mechanisms are available to connect to the SC in this network architecture:
An administrator can SSH to the MSP (msp01 in the diagram), then Telnet from it to the SC.
An administrator can use the serial connection accessible from the network terminal server (nts01 in the diagram) as an alternative access mechanism to the SC. In this topology, even when the MSP is not available the SC is accessible through the network terminal server.
Installing Apache Web Server
In the configuration documented in this article, the MSP uses the Apache Web Server to perform Solaris Web Start Flash updates of the SC EPROMs and to provide restoreconfig with a transport mechanism to restore SC backups created with dumpconfig.
Other web servers can be used on the MSP, instead of the Apache Web server. However, only the recommended Apache configuration is described in this article.
The Apache distribution available in Solaris 8 OE is not installed with the End User Cluster, therefore, it may be necessary to manually install the three required Apache packages. If Apache is already installed on your MSP, some of the following steps may not be necessary.
To Install the Apache Web Server
Obtain the required packages from any Solaris 8 OE 2 of 2 CD-ROM, dated 4/01, in the following directory:
# pwd /cdrom/sol_8_401_sparc_2/Solaris_8/Product
The three required Solaris 8 OE Apache Web Server packages are as follows:
system SUNWapchd Apache Web Server Documentation system SUNWapchr Apache Web Server (root) system SUNWapchu Apache Web Server (usr)
Create a tar file containing these three packages in the following manner:
# tar -cvf /tmp/apache-pkgs.tar SUNWapchd SUNWapchr SUNWapchu
Move the tar file to the MSP, extract it, and install it using the following commands:
# tar -xf apache-pkgs.tar # pkgadd -d . SUNWapchd SUNWapchr SUNWapchu
Answer Yes to all the questions asked.
After the installation is completed, use the pkginfo | grep Apache command to verify that all three required Apache Web Server packages are present.
In the next steps, you'll create an appropriate user and group ID for Apache to run as.
Create a new group by adding the following line to the /etc/group file:
The example uses a group ID of 15 for mspstaff. If this group ID is already used in your environment, select a group ID that is not being used.
Create a user account for the Apache daemon.
The following example uses msp http:
# /usr/sbin/useradd -m -g mspstaff msphttp 11 blocks
For all administrators who need access to files shared by Apache, add their user IDs to the end of the mspstaff entry in the /etc/group file.
Before starting the Apache daemon, you must configure it. Only a few steps are required to do that.
Create an httpd.conf file using the following command:
# pwd /etc/apache # cp httpd.conf-example httpd.conf
Open the /etc/apache/httpd.conf file in an editor and search for the following line:
Add the following line immediately after it.
Where the IP address is the IP address of the MSP on the private SC network:
This step configures the Apache Web Server to respond only to connection requests from the private SC network and not to the general purpose network. This configuration is important because other systems must not be able to access the information that is made available over HTTP to the SC.
A few other Apache configuration modifications are still required. The Apache Web Server must be told what name to use. Because the name of the MSP on the private SC network may not be resolvable, this configuration uses the IP address of that interface.
Search for the following line in the /etc/apache/httpd.conf file:
Add the following line immediately after it.
Where the IP address is the IP address of the MSP on the private SC network:
The Apache Web Server must be told what directory structure to make available. This directory is called the DocumentRoot and should be the top-most directory of where the Flash archives and backup files are kept.
Search for the following line in the /etc/apache/httpd.conf file:
Replace it with the following line.
Where the directory is the top-most directory of what is available to the SC:
By default, the Apache Web Server runs as the user ID nobody and group ID nobody. On the MSP, this should be changed to a more restrictive configuration by creating a new user ID and group ID for the Apache Web server to better control access to the /msp directory. In this way, only administrators requiring access to the directory structure accessed by Apache can be added to the Apache group and, therefore, have access. Earlier in this section, you created a user ID and group ID for this purpose. They were msphttp and mspstaff, respectively.
Now that Apache Web Server is installed, make the following change in the httpd.conf file to configure the server to use the user ID and group ID you entered earlier:
User msphttp Group mspstaff
To allow this configuration to work, change the ownerships of the Apache Web Server log file directory using the following command:
# chown -R msphttp:mspstaff /var/apache/logs
Create the /msp directory on the MSP; use a partition with adequate free space.
In the following example, the directory is created on the /, or root, file system of msp01:
# mkdir /msp
Set the ownerships and permissions of the /msp directory to the msphttp user ID and mspstaff group ID with the following commands:
# chown msphttp:mspstaff /msp # chmod 770 /msp
Start the Apache Web Server with the following command:
# /etc/init.d/apache start httpd starting.
The Apache Web server is now ready to function as a restoreconfig server and can be used as a flashupdate server.
Adding Security Software
The next stage in hardening an MSP requires downloading and installing additional software security packages. This section covers the following tasks:
- "Install Solaris Security Toolkit Software"
- "Download Recommended Patch Cluster Software"
- "Download FixModes Software"
- "Download OpenSSH Software"
- "Download the MD5 Software"
Of the software described in this section, the Solaris Security Toolkit, Recommended and Security Patch Cluster, FixModes, and MD5 software are required. Instead of OpenSSH, you can substitute a commercial version of SSH, available from a variety of vendors. You must install an SSH product on the MSP.
Install Solaris Security Toolkit Software
The Solaris Security Toolkit software must be downloaded first, then installed on the MSP. Later, you'll use the Solaris Security Toolkit software to automate installing other security software and implementing the Solaris OE modifications for hardening the MSP.
The primary function of the Solaris Security Toolkit software is to automate and simplify building secured Solaris OE systems based on the recommendations contained in this and other security-related Sun BluePrints OnLine articles.
The following instructions use filenames that are correct only for version 0.3.6 and later of the Solaris Security Toolkit software.
To Download Solaris Security Toolkit Software
Download the latest version of the source file.
At the time of this publication, the version is SUNWjass-0.3.6.pkg.Z. The source file is located at:
Extract the source file into a directory on the server using the uncompress command:
# uncompress SUNWjass-0.3.6.pkg.Z
Install the Solaris Security Toolkit software onto the server using the pkgadd command:
# pkgadd -d SUNWjass-0.3.6.pkg SUNWjass
Executing this command creates the SUNWjass subdirectory in /opt. This subdirectory contains all Solaris Security Toolkit directories and associated files. The script make-pkgincluded in Solaris Security Toolkit software releases since version 0.3allows administrators to create custom packages using a different installation directory.
Download Recommended Patch Cluster Software
Patches are regularly released by Sun to provide Solaris OE fixes for performance, stability, functionality, and security. It is critical to the security of a system that the most up-to-date patch is installed. To ensure that the latest Solaris OE Recommended and Security Patch Cluster is installed on the MSP, this section describes how to download the latest patch cluster.
Downloading the latest patch cluster does not require a SUNSOLVE ONLINE program support contract.
Apply standard best practices to all patch installations. Before installing any patches, evaluate and test them on non-production systems or during scheduled maintenance windows.
To Download Recommended Patch Cluster Software
Download the latest patch from the SUNSOLVE ONLINE Web site at:
Click on the "Patches" link at the top of the left navigation bar.
Select the appropriate Solaris OE version in the "Recommended Solaris Patch Clusters" box.
In our example, we select Solaris 8 OE.
Select the best download option, either HTTP or FTP, with the associated radio button, then click "Go."
A "Save As" dialog box is displayed in your browser window.
Save the file locally.
Move the file securely to the MSP with the scp command, or ftp if scp is not available.
The scp command used should be similar to the following:
% scp 8_Recommended.zip msp01:/var/tmp
Move the file to the /opt/SUNWjass/Patches directory and uncompress it as follows:
# cd /opt/SUNWjass/Patches # mv /var/tmp/8_Recommended.zip . # unzip 8_Recommended.zip Archive: 8_Recommended.zip creating: 8_Recommended/ inflating: 8_Recommended/CLUSTER_README inflating: 8_Recommended/copyright inflating: 8_Recommended/install_cluster [. . .]
Later, using the Solaris Security Toolkit software, you'll install the patch after downloading all the other security packages.
If you do not place the Recommended and Security Patches software into the /opt/SUNWjass/Patches directory, a warning message displays when you execute the Solaris Security Toolkit software.
Download FixModes Software
FixModes is a software package that tightens the default Solaris OE directory and file permissions. Tightening these permissions can significantly improve overall security of the MSP. More restrictive permissions make it even more difficult for malicious users to gain privileges on a system.
To Download FixModes Software
Download the FixModes pre-compiled binaries from:
The FixModes software is distributed as a precompiled and compressed tar file formatted for systems based on SPARC. The file name is FixModes.tar.Z.
Once downloaded, move the file securely to the MSP with the scp command, or ftp if scp is not available.
The scp command used should be similar to the following
% scp FxiModes.tar.Z msp01:/var/tmp
Save the file, FixModes.tar.Z, in the Solaris Security Toolkit Packages directory in /opt/SUNWjass/Packages.
The following commands perform these tasks:
# cd /opt/SUNWjass/Packages # mv /var/tmp/FxiModes.tar.Z .
Leave the file in its compressed state.
Later, using the Solaris Security Toolkit software, you'll install the FixModes software after downloading all the other security packages.
Download OpenSSH Software
In any secured environment, the use of encryption in combination with strong authentication is required to protect user-interactive sessions. At a minimum, administrator access to the SC through Telnet sessions and platform/administrator shells must be encrypted. This requirement is one of the major reasons for the presence of the MSP.
The tool most commonly used to implement encryption is Secure Shell (SSH) software, whether a commercial or open source (freeware) version. To implement all the security modifications performed by the Solaris Security Toolkit software and recommended in this article, you must implement an SSH software product.
Information on where to obtain commercial versions of SSH is provided in "Related Resources" on page 58.
The Solaris Security Toolkit software disables all non-encrypted user-interactive services and daemons on the system, in particular daemons such as in.rshd, in.telnetd, and in.ftpd.
The in.ftpd daemon is re-enabled during dumpconfig. Refer to "Backing Up and Restoring Configurations" on page 45.
Access to the system can be gained with SSH similarly to what is provided by RSH, Telnet, and FTP.
If you choose to use an SSH product other than OpenSSH, install and configure it before or during the Solaris Security Toolkit software run.
To Download OpenSSH Software
Obtain the following Sun BluePrints OnLine article and use the instructions in the article for downloading the software.
A Sun BluePrints OnLine article about how to compile and deploy OpenSSH titled: Building and Deploying OpenSSH on the Solaris Operating Environment is available at:
Later, using the Solaris Security Toolkit software, you'll install the OpenSSH software after downloading all the other security packages.
Do not compile OpenSSH on the MSP and do not install the compilers on the SC. Use a separate Solaris OE systemrunning the same Solaris OE version, architecture, and mode (for example, Solaris 8 OE, Sun4U, and 64 bit)to compile OpenSSH. If you implement a commercial version of SSH, then no compiling is required.
Download the MD5 Software
The MD5 software validates MD5 digital fingerprints on the MSP. Validating the integrity of Solaris OE binaries provides a robust mechanism to detect system binaries that are altered or trojaned (hidden inside something that appears safe) by unauthorized users. By modifying system binaries, attackers provide themselves with back-door access onto a system; they hide their presence and cause systems to operate in unstable manners.
To Install the MD5 Software (Intel and SPARC)
Download the MD5 binaries from the following web site:
The MD5 programs are distributed as a compressed tar file.
Move the file md5.tar.Z securely to the MSP with the scp command, or ftp if scp is not available.
The scp command used should be similar to the following:
% scp md5.tar.Z msp01:/var/tmp
Copy the file, md5.tar.Z, to the Solaris Security Toolkit Packages directory in /opt/SUNWjass/Packages
Do not uncompress the tar archive.
After the MD5 software is saved to the /opt/SUNWjass/Packages directory, the execution of the Solaris Security Toolkit installs the software.
After the MD5 binaries are installed, you can use them to verify the integrity of executables on the system through the Solaris Fingerprint Database. More information on the Solaris Fingerprint Database is available in the Sun BluePrints OnLine article titled The Solaris™ Fingerprint Database - A Security Tool for Solaris Software and Files.
(Optional) Download and install Solaris Fingerprint Database Companion and Solaris Fingerprint Database Sidekick software from the SUNSOLVE ONLINE Web site at:
We strongly recommend that you install these optional tools and use them with the MD5 software. These tools simplify the process of validating system binaries against the database of MD5 checksums. Use these tools frequently to validate the integrity of the Solaris OE binaries and files on the cluster nodes.
These tools are described in the The Solaris™ Fingerprint Database - A Security Tool for Solaris Software and Files article.
Installing Downloaded Software and Implementing Modifications
The Solaris Security Toolkit version 0.3.6 and later provides a driver (sunfire_mf_msp-secure.driver) for automating the installation of security software and Solaris OE modifications. The driver performs the following tasks:
Installs and executes the FixModes software to tighten file system permission
Installs the MD5 software
Installs the Recommended and Security Patch Cluster software
Implements almost 100 Solaris OE security modifications
The actions performed by each of the scripts is described in the Sun BluePrints OnLine article "The Solaris Security Toolkit - Internals: Updated for Version 0.3." The hardening described is performed in standalone mode, not JumpStart mode, because the MSP was built using an interactive Solaris OE installation. For details on the differences between standalone mode and JumpStart mode, refer to the Solaris Security Toolkit documentation.
During the installation and modifications implemented in this section, all non-encrypted access mechanisms to the MSP such as Telnet, RSH, and FTPare disabled. The hardening steps do not disable console serial access over SC serial ports.
To Install Downloaded Software and Implement Changes
Execute the sunfire_mf_msp-secure.driver script as follows:
# pwd /opt/SUNWjass # ./jass-execute -d sunfire_mf_msp-secure.driver ./jass-execute: NOTICE: Executing driver, sunfire_mf_msp-secure.driver ============================================================= sunfire_mf_msp-secure.driver: Driver started. ============================================================= ============================================================= JASS Version: 0.3.6 Node name: mako-ss Host ID: 80c68774 Host address: 192.168.100.10 MAC address: 8:0:20:c6:87:74 Date: Wed Feb 20 09:47:43 PST 2002 ============================================================= [...]
To View the Contents of the Driver File
To view the contents of the driver file and obtain information about the Solaris OE modifications, refer to the Solaris Security Toolkit documentation available either in the /opt/SUNWjass/Documentation directory or through the web at:
For information about other scripts in the Solaris Security Toolkit software, refer to the Sun BluePrints OnLine article titled Solaris Security Toolkit Internals: Updated for Version 0.3.
To Undo a Solaris Security Toolkit Run
Each Solaris Security Toolkit run creates a run directory in /var/opt/SUNWjass/run. The names of these directories are based on the date and time the run is initiated. In addition to displaying the output to the console, the Solaris Security Toolkit software creates a log file in the /var/opt/SUNWjass/run directory.
Do not modify the contents of the /var/opt/SUNWjass/run directories under any circumstances. Modifying the files can corrupt the contents and cause unexpected errors when you use Solaris Security Toolkit software features such as undo.
The files stored in the /var/opt/SUNWjass/run directory track modifications performed on the system and enable the jass-execute undo feature.
To undo a run or series of runs, use the jass-execute -u command.
For example, on a system where two separate Solaris Security Toolkit runs are performed, you could undo them by using the following command and options:
# pwd /opt/SUNWjass # ./jass-execute -u Please select from one of these backups to restore to 1. September 25, 2001 at 06:28:12 (/var/opt/SUNWjass/run/20010925062812) 2. April 10, 2002 at 19:04:36 (/var/opt/SUNWjass/run/20020410190436) 3. Restore from all of them Choice? 3 ./jass-execute: NOTICE: Restoring to previous run //var/opt/SUNWjass/run/20020410190436 ============================================================ undo.driver: Driver started. ============================================================ [...]
Refer to the Solaris Security Toolkit documentation for details on the capabilities and options available in the jass-execute command.
Configuring the MSP SYSLOG
The MSP is configured to function as the SYSLOG repository for all SYSLOG traffic generated by the SC. The behavior of the SYSLOG daemon is controlled through the file /etc/syslog.conf; in this file, selectors and actions are specified.
Each SYSLOG selector specifies the facility (for example, kern, daemon, auth, and user) and level at which a message is logged. Five levels ranging from most serious (emerg) to least serious (debug) are available. The facility groups log messages together by subsystem. For instance, all kernel messages are grouped together through the facility kern. Some of the facilities available include:
For a complete listing of SYSLOG facilities, refer to the syslogd(1m) man page.
Also, it is possible to substitute a wildcard (*) for the facility name in the syslog.conf file. This approach is particularly useful when all messages (for example, *.debug), or all messages at one level or higher, must be logged (for example, *.kern).
Each SYSLOG message includes a level. This level specifies the type of message being generated. The most critical level is emerg, which is only used on messages of particular importance. Correspondingly, the log level debug indicates that a message contains debugging information and may not be particularly important. Some of the levels available in the syslog.conf include:
For a complete listing of SYSLOG levels, refer to the syslogd(1m) man page.
Although you can use a wildcard to define a facility, you cannot use it to define a level. Hence, the entry *.debug is acceptable; however, the corresponding entry of auth.* is incorrect and cannot be used.
In the MSP configuration, we recommend for the secured configuration that all SYSLOG messages be stored both in the /var/adm/messages file and in a separate file containing only Sun Fire Midframe SYSLOG traffic.
It is not recommended that the SYSLOG traffic be forwarded from the MSP to another SYSLOG server. If this were done, then a SYSLOG message after being forwarded from the MSP would identify itself as having been generated on the MSP and not the SC, as would actually be the case.
The recommended syslog.conf should be similar to the following:
*.debug /var/adm/messages local0.debug /var/adm/sc-messages-platform local1.debug /var/adm/sc-messages-domain-a local2.debug /var/adm/sc-messages-domain-b local3.debug /var/adm/sc-messages-domain-c local4.debug /var/adm/sc-messages-domain-d kern.crit console
This configuration logs all incoming messages to /var/adm/messages, all SC messages to /var/adm/sc-messages-<name>, and displays all critical kernel messages on the console.
If an automated log parsing tool such as logcheck or swatch is used, it may be appropriate to generate one file containing the SYSLOG messages from the platform and all the domains. If this consolidated file is required, then add the following lines to those listed previously:
local0.debug /var/adm/sc-messages local1.debug /var/adm/sc-messages local2.debug /var/adm/sc-messages local3.debug /var/adm/sc-messages local4.debug /var/adm/sc-messages
This configuration logs all incoming SYSLOG messages to /var/adm/sc-messages for reconciliation by an automated tool.
This configuration is relatively generic and should only be considered a starting point for configuring the SYSLOG daemon on the MSP for an organization.
It is critical the two columns be separated by tabs and not spaces. If spaces are used in an entry, the SYSLOG daemon will ignore that entry.