Home > Articles > Operating Systems, Server > Solaris

This chapter is from the book

This chapter is from the book

Patching and Upgrading Your Cluster

The approach you take to patching and upgrading your cluster will vary depending on your data center standards. Some data centers apply only the bare minimum of changes, such as critical security patches, operating on the principle of "if it ain't broke, don't fix it." Clearly, if a system is stable, this is a reasonable approach, especially if availability is your highest priority. Other data centers schedule regular planned outages to perform patching, often choosing to lag a little behind the most recent patches to avoid encountering bad patches. Still others take the "latest and greatest" approach, given that the patches are meant to fix all the key, known issues. Whichever strategy you use, minimizing the overall outage will clearly be your main priority.

You can also upgrade the Solaris Cluster software by patching it with the patches that are built into a later release of the Solaris Cluster software. However, you must follow all the instructions in the README file contained within each patch.

Upgrade Methods

Three main methods are available for patching or upgrading your operating system and Solaris Cluster software: standard upgrade, rolling upgrade, and dual-partition upgrade. Solaris Live Upgrade provides yet more options when used with these methods. Each method results in differing lengths of outage on your cluster, which further depends on the number and type of zones you have configured (see the section "Oracle Solaris Zones" in Chapter 3, "Combining Virtualization Technologies with Oracle Solaris Cluster Software"). The following sections describe these methods.

Before you begin making changes to the system, consider the following checklist:

  • Has the upgrade procedure been tested on a test cluster before being implemented on the production system? If not, is management aware of the potential risk of untested procedures?
  • Have you verified that the new configuration is supported by all your services? Some applications place greater restrictions on the operating system or other software versions they support.
  • Have the proposed changes been agreed upon and recorded in your change control system?
  • Do you have a full backup of the cluster prior to making any changes?
  • Have you tested that you can switch over all services between cluster nodes prior to your changes? This testing isolates any post-upgrade problems to the upgrade and patches.
  • Is there an agreed outage period of a set length?
  • Is there a procedure to reverse any changes if the upgrade does not go according to plan? Have you determined when you will need to start backing out any changes if that happens?
  • Have you tested that you can switch over all services between cluster nodes after the changes have been made? This testing ensures that if a failure occurs later, your system will continue to provide a set of highly available services.

Standard Upgrade

With standard upgrade, you schedule a complete cluster outage to perform the patch or upgrade process. This method might be considered for applying patches, such as kernel patches, that need to be installed in single-user mode. Major operating system upgrades also fall into this category. You cannot run a Solaris Cluster configuration with nodes running different major versions of the Oracle Solaris OS, for example, Solaris 9 OS and Solaris 10 OS.

A standard upgrade is by far the simplest method but necessitates the longest cluster outage. During this period, your services are unavailable. Consequently, this method is the least desirable from an availability standpoint.

A standard upgrade can be combined with Solaris Live Upgrade. This approach significantly reduces the cluster outage by no longer requiring the entire cluster to be shut down while you are upgrading the software. You first create an alternative boot environment with the upgraded software on each machine (see "Solaris Live Upgrade" for details). When you are ready to activate the new software, halt the entire cluster, and then boot the entire cluster using the alternative boot environment with the new software on each machine.

Rolling Upgrade

With rolling upgrade, the nodes in your cluster are upgraded one at a time. Any service hosted on a node to be upgraded is manually switched over to another cluster node prior to the target node being shut down for patching. After the target node has been patched, it is brought back into the cluster, and the process is repeated until all nodes have been upgraded. After all of the nodes have been upgraded, you execute the scversions -c command to commit the cluster to use the new software. The cluster software does not enable cluster-wide protocols until the commit happens. Prior to the commit, all nodes, including nodes with new software, continue to use the old cluster-wide protocols. There is no rolling downgrade after the commit happens. The outage incurred by a service is therefore just twice the time it takes to switch over from one cluster node to another. A switchover of a simple application can take about 30 seconds, though the time varies significantly depending on the application.

A rolling upgrade changes only the Solaris Cluster software or the Oracle Solaris OS. Rolling upgrades can be used as long as the major cluster release and major operating system remain the same, for example, the Solaris Cluster 3.2 software or the Solaris 10 OS. You cannot perform rolling upgrades between major cluster or major OS releases, for example, the Sun Cluster 3.1 software to the Solaris Cluster 3.2 software or the Solaris 9 OS to the Solaris 10 OS.

The rolling upgrade process updates the software on the local storage of a node when that node is not in cluster mode. The highly available (HA) container places the software on shared storage. Because the shared storage is never out of service, the rolling upgrade process has no opportunity to change the software residing on shared storage. Therefore, a rolling upgrade cannot be used to upgrade software on HA containers.

A rolling upgrade can also be combined with Solaris Live Upgrade. This combination does not reduce the service outage. A rolling upgrade without Solaris Live Upgrade reduces the service outage by performing software upgrades while the machines are not in the cluster. During this time period, the ability of the cluster to survive machine failures is reduced. The use of Solaris Live Upgrade can significantly reduce the time period of increased vulnerability. Solaris Live Upgrade also reduces the period of time in which nodes in the cluster have different software versions. You first create an alternative boot environment with the upgraded software on each machine (see "Solaris Live Upgrade"). You then serially halt each machine and boot that machine from the alternative boot environment with the upgraded software. After you have booted all nodes from the alternative boot environment with the upgraded software, you execute the scversions -c command to commit the cluster to use the new software.

Dual-Partition Upgrade

A dual-partition upgrade offers the greatest flexibility in terms of what changes you can make to the system. The change can be a patch, an update release, or a major release. Changes in multiple areas can be applied simultaneously. A dual-partition upgrade can even be used when all the changes are in areas other than the Solaris Cluster software, including the following:

  • The Oracle Solaris OS
  • A Solaris Cluster software release
  • Third-party volume managers and file systems
  • Application software (as long as the applications are not installed on shared storage)

A dual-partition upgrade divides your cluster into two parts, each of which is updated separately and sequentially. To invoke a dual-partition upgrade, you must use the scinstall command from the target media of the release to which you are upgrading. This command then prompts you with the options shown in Example 4.11. Choose option 3 for a dual-partition upgrade. You must divide the cluster machines into two sets of nodes, which are called partitions. You must partition the cluster such that each partition has for each storage device at least one node that can access that storage device. During the upgrade process, each partition will host the cluster applications, and the cluster applications must be able to access storage. If you are unsure of how to partition the cluster, you can choose an option whereby the system presents you with the various ways to group your cluster nodes for the purposes of the upgrade. After you have chosen your partitioning, you next select the option to initiate the actual upgrade.

Example 4.11. Main Menu of the scinstall Command Located on the Solaris Cluster Release Media

The main menu of the scinstall command provides you with the following options:

*** Main Menu ***

    Please select from one of the following (*) options:

        1) Create a new cluster or add a cluster node
        2) Configure a cluster to be JumpStarted from this install server
      * 3) Manage a dual-partition upgrade
      * 4) Upgrade this cluster node
      * 5) Print release information for this cluster node

      * ?) Help with menu options
      * q) Quit

    Option:  3

The system software makes changes to the quorum votes, migrates your services to the partition that is still running while the first partition is upgraded, and halts the nodes of the first partition. You boot the nodes of the first partition into non-cluster mode and upgrade the software. After you have completed the upgrade on the first partition, you choose to continue the dual-partition upgrade. The system software reboots the nodes of the first partition into cluster mode as a new cluster. Unlike what happens in an ordinary boot, the nodes of the first partition proceed through the boot process and stop just prior to the point where the cluster would activate resources used by applications (such as mounting file systems and plumbing IP addresses) and start applications. At this point, most of the boot process has completed. The system software halts the nodes of the second partition and waits for them to actually halt. Next, the system software imports volumes, mounts file systems, plumbs IP addresses, and starts applications. In effect, the system performs a switchover of all applications and their resources from the second partition to the new partition. Next, reboot each node of the second partition and upgrade the software. When done with that step, you choose to continue the upgrade. The system software reboots the nodes of the second partition into cluster mode. After all nodes are back in the cluster, the system software restores the quorum configuration and completes the dual-partition upgrade.

If the dual-partition upgrade fails, you boot all nodes in non-cluster mode and execute the scinstall -u recover command on each node to restore the initial copy of the Cluster Configuration Repository (CCR). The system is already completely shut down at this point, so a minimal downtime upgrade is not possible. You do not restart the dual-partition upgrade. Instead, you perform a standard upgrade.

The service outage when you perform a dual-partition upgrade on a system with failover application is approximately the same as for a rolling upgrade. Because a time comes when neither partition can host an instance of a scalable application, there is a service outage for the scalable application, and the service outage is approximately the same as that of a failover application. For the simplest scalable applications, the service outage can be about 30 seconds, though the time varies significantly based on the application.

You cannot upgrade any software or data on the shared storage because the services remain running throughout the upgrade process, except for the point when the services are switched over from the partition running the old software to the partition running the new software. Do not make any configuration changes during the dual-partition upgrade. Make configuration changes either before or after the dual-partition upgrade.

A dual-partition upgrade can also be combined with Solaris Live Upgrade. This combination does not reduce the service outage. A dual-partition upgrade without Solaris Live Upgrade reduces the service outage by performing software upgrades while the machines are not in the cluster. During this time period the ability of the cluster to survive machine failures is reduced. The use of Solaris Live Upgrade can significantly reduce the time period of increased vulnerability. You first create an alternative boot environment with the upgraded software on each machine (see "Solaris Live Upgrade"). You then initiate the actual dual-partition upgrade. The system software automatically manages all of the work of the dual-partition upgrade. There no longer is any need to boot into non-cluster mode, so the number of reboots is reduced by half. The net result is that the time required for upgrades is dramatically reduced.

Choosing Which Upgrade Method to Use

The choice of an upgrade method is governed by three factors: the simplicity of the upgrade process, the compatibility of the current and target software environments, and the level of availability your services must achieve during the upgrade process. Thus, if you require simplicity above all else, then the standard upgrade is the best method. If your current and target software releases do not span a major release, and you want to maintain service availability even for scalable services, then you can use a rolling upgrade. Finally, if changes to your cluster environment involve software spanning major releases and you need to minimize your service outage, then you must choose a dual-partition upgrade.

Solaris Live Upgrade

Solaris Live Upgrade is not a stand-alone technology for upgrading a Solaris Cluster system. Solaris Live Upgrade must be used as part of either a standard upgrade, a rolling upgrade, or a dual-partition upgrade. Solaris Live Upgrade makes it possible to create an alternate boot environment with the upgraded software while the machine continues to run unaffected using the current boot environment. See "Standard Upgrade," "Rolling Upgrade," or "Dual-Partition Upgrade" for details on when to bring the alternate boot environment with the updated software into operational status. The boot environment with the old software is not damaged when you boot the alternate boot environment with the new software. If you encounter a problem with the new software, you can recover by halting all nodes of the cluster and then booting the entire cluster from the boot environment with the old software and configuration information. Solaris Live Upgrade is available for all the operating system releases supported by the Solaris Cluster 3.2 software.

If you are using the Solaris 10 OS with a ZFS root (/) file system, then the effort involved in creating an alternate boot environment is virtually zero because the required new file systems can be created from the existing root zpool. However, if you are using a Solaris Volume Manager or a Veritas Volume Manager mirrored root disk, then the process is significantly more complicated because you must find additional partitions to match the existing root partition allocation. Consequently, using the Solaris 10 OS and ZFS for the root disk has significant advantages. However, do not upgrade the software version of the zpools in your cluster until you are satisfied that your new environment is working correctly. If you do upgrade the zpool version, you will not be able to fail back to your old boot environment, as the old environment might not be able to mount the zpools that already have a higher version number. Such zpools cannot be downgraded.

After you have updated the Oracle Solaris OS on your new boot environment, you can proceed with updating the Solaris Cluster software. When using Solaris Live Upgrade, you must follow the documentation on how to handle specific steps relating to the volume managers, Solaris Volume Manager and Veritas Volume Manager, and which version of the Java Runtime Environment (JRE) is required.

The process for updating the Solaris Cluster software on the alternate boot environment is outlined in the following steps:

  1. Create an installer program state file to drive the upgrade of the shared components on the alternate boot environment, but without actually performing the actions.
  2. Run the installer program again using this state file but targeted on the alternate boot environment.
  3. Update the Solaris Cluster framework by running the scinstall command from the new Solaris Cluster media, again targeting the alternate boot environment.
  4. Update the Solaris Cluster agents by running the scinstall command from the new Solaris Cluster media, again targeting the alternate boot environment.

Upgrading Nodes Using Oracle Solaris Zones

When upgrading a cluster node that uses the Solaris 10 OS, you need to consider the impact the upgrade process has on any Oracle Solaris Zones (also known as Oracle Solaris Containers) that are configured on the system. These containers represent additional, virtual Solaris instances that must also be upgraded.

To maintain service availability during an upgrade, a resource group must have a node list that contains at least one physical cluster node that will still be available to host the resource group while the rolling upgrade or dual-partition upgrade is performed. This is feasible both for resource groups that use global-cluster non-voting nodes and for resource groups existing in zone clusters. However, HA containers are, by their very nature, the sole place in which the service resides. Therefore, taking the cluster node into single-user mode to patch it results in the HA container, and consequently the service, being unavailable for the duration of the update process [ZonePatch].

One way to minimize HA container service outage is to use Solaris Live Upgrade. The procedure is not particularly practical if you have tens or hundreds of HA containers because it involves many steps.

The following procedure is meant for environments where each HA container has its zonepath, on its own Oracle Solaris ZFS file system, in its own zpool, for example, ora-pool/ora1 for ora1. Both ora-pool and ora-pool/ora1 must have appropriate mount points set.

Solaris Live Upgrade requires full control over the zpools when the Oracle Solaris OS reboots in order to perform the necessary renaming of the ZFS file systems and the changes to the zones' configuration files. Therefore, the ZFS file systems must not be under Solaris Cluster software control during the shutdown and boot process. This can be achieved by creating an alternate boot environment where each zone and its associated zonepath are under cluster control. Then, prior to shutting down the system, you can offline and unmanage the resource groups controlling the HA containers and import each zpool that holds a zonepath. Finally, you can copy all the affected configuration files in the CCR and the zpool.cache file to the alternate boot environment.

In general, this procedure requires you to perform the following steps:

  1. Alter the HA container's resource group node list to just the current node.
  2. Perform a Solaris Live Upgrade on the node.
  3. Disable the resources of the HA container's resource group, which takes the container offline and unmanages the resource groups.
  4. Import the zpool holding the zone's zonepath.
  5. Copy the zpool.cache file and corresponding CCR files for the resource group to the alternate boot environment.
  6. Shut down and reboot the node.
  7. After Solaris Live Upgrade has performed all of the necessary reconfigurations, manage and bring the resource group online.
  8. Detach the HA container from its other nodes, and then use Solaris Live Upgrade and reboot the other nodes in turn.
  9. Attach the zonepath of the HA container with the -F (force) option to bring the container's state back to installed.
  10. Reset the HA container's resource group node list.
  11. Check that switchover still works and finish by switching the HA container back to its primary machine.

Although this procedure requires a shorter outage, other factors might make it impractical. First, the HA container cannot be failed over while the upgrade is under way because the node list has been restricted to the single node. Additionally, the service-level agreements (SLAs) for other HA containers that are deployed on the system might limit when the outage can occur because any upgrade will affect them, too. If there isn't a single time period when groups of containers can be updated, then the restriction will prevent any of the changes from being made.

The final option for upgrading HA containers is to use the update on attach feature in the Solaris 10 10/08 release. This option can be used only to update the packages in a container and does not allow them to be downgraded. If you use this option, you must ensure that the process does not introduce any packages that are older than the current version the Solaris Cluster software requires. If it does, you must update them again using the installer program.

An outline of the steps for using the update on attach feature follows:

  1. Evacuate (or switch over) all the containers that will not be patched from the target node (nodeT) using clresourcegroup switch ..., leaving just the containers to be patched.
  2. On node nodeT, use zoneadm -z ... detach to detach each remaining container from nodeT so that the operating system will not attempt to patch them.
  3. On node nodeT, patch nodeT and reboot back into the cluster.
  4. Use clresource disable ... to disable the HA container resources that were switched over to the alternative node (nodeA), and switch their resource groups back to nodeT. This switches over the zonepaths to nodeT.
  5. On node nodeA, detach the container from nodeA so that the operating system will not attempt to patch the container during the patch process or reboot the container after the upgrade.
  6. On node nodeT, use zoneadm -z ... attach -u to reattach and patch the HA container on nodeT.
  7. Test the containers by booting and halting them manually.
  8. Place the containers back under cluster control by reenabling the HA container resource.
  9. Repeat the upgrade process with nodeA.

Ensure that HA containers are prevented from failing between the nodes while this procedure is being followed.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information

To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.


Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.


If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information

Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.


This site is not directed to children under the age of 13.


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information

If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information

Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents

California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure

Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact

Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice

We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020