Home > Articles > Operating Systems, Server > Solaris

  • Print
  • + Share This
Like this article? We recommend

Campus Cluster Topologies and Components

There are many considerations involved when planning a campus cluster topology, such as:

  • Number of cluster nodes

  • Type of interconnects

  • Type of servers and storage interconnects

  • Distance

  • Availability of a third site

Although the vast majority of clusters deployed today are two-node clusters, a more robust campus cluster consists of four nodes, two at each site, and two-way, host-based mirrors across sites. To protect against local storage failures, controller-based RAID (such as RAID-5) is used within the storage arrays.

Multipathing solutions protect against failures of the storage paths and make it very unlikely that data needs to be completely resynchronized under normal circumstances. This configuration helps ensure that in most failures, failover would only take place within the same site. This configuration would not cause any additional overhead for the administrative staff to move to another data center. It maintains full redundancy in case of a total site loss.

Some financial decision makers may question the expense involved in deploying more than a two-node campus cluster. However, using Solaris Resource Manager software, corporations can allocate some resources to nonclustered services on the remote systems, making use of otherwise idle resources while still reserving resources for failover in case of a disaster.

Quorum Devices in Campus Clusters

During cluster membership changes (for example, those caused by node outages), Sun Cluster 3.0 software uses a quorum mechanism to decide which nodes of the cluster are supposed to form the new cluster. Only a group of nodes with the majority of quorum votes may form a new cluster. This quorum mechanism helps ensure data integrity even in cases where cluster nodes cannot talk to each other because of broken interconnects. All other nodes not having majority are either shut down or prevented from accessing the data disks by means of reservation mechanisms in the storage layer (SCSI-2 and SCSI-3 persistent group reservations). Thus, only the nodes of a cluster with quorum have physical access to data.

Nodes and disks have quorum votes. By default, nodes have one vote. Dedicated quorum disks have as many votes as the sum of the nodes' votes attached to it, minus one. For example, dual-ported quorum disks have one vote, a four-node attached quorum disk has three votes. Because there must be a mechanism to break the tie in a two-node cluster, this configuration requires a quorum device. Sun Cluster 3.0 software enforces the configuration of a quorum device in these cases.

Quorum rules are not only valid for normal clusters, but for campus clusters as well. Because campus clusters are designed to protect against total site failures, it is important to understand the function of the quorum device in two- and three-site configurations.

For example, consider a typical two-site campus cluster setup. For the sake of simplicity, the quorum device (QD) is represented as a separate disk, which is not a requirement. It could be a disk in the data storage.

As shown in FIGURE 1, the quorum device (QD) is configured in site A. If site A fails completely, two out of three votes would be unavailable, leaving the node in site B with only one vote. The node in site B could not gain quorum and thus would shut down, leaving the whole cluster—even with a surviving node and a good copy of data in a local mirror—useless. When an operational node cannot communicate with its peer, it has no way of telling whether the communication paths or the peer itself are down. This situation is known as split brain. Shutting down node B in this scenario makes sense because it is unclear what has happened to A. If A is still alive, this action prevents data corruption. If it is down, the administrator has to decide how to proceed.

Figure 1FIGURE 1 Two-Site Campus Cluster Configuration

Without the quorum mechanism, each site could think it was the only survivor, then form a new cluster and start HA services. If both sites access the same data simultaneously, there is a high probability that they might cause data corruption.

In a configuration where one site is production and the other is either idle or running nonproduction work, the recommended practice is to configure the quorum device in the production site. If the remote site fails, the production site has enough votes to continue without interruption. If the production site fails, the problem cannot be overcome automatically with a two-site topology.

In this case, two options exist. Either an administrator must initiate a manual procedure to recover the quorum, or implement a third site for the quorum. Both methods are supported with Sun Cluster 3.0 software.

FIGURE 2 illustrates a three-site configuration. The quorum device is in a separate third site, C. In all scenarios where only one site is affected by a disaster, two remain in operation and provide one quorum vote each, so that the needed quorum of two votes is gained by the two surviving sites, that then form a new cluster. Therefore, a three-site configuration is highly recommended for enterprises that require fully automatic failover, even in case of a disaster affecting one site.

Figure 2FIGURE 2 Three-Site Campus Cluster Configuration

As previously noted, enterprises may choose to use a manual procedure to recover from a loss of quorum. In this situation, the node that lost quorum is unavailable and cannot boot into cluster mode. This situation makes it necessary to change the quorum device definition in the cluster configuration repository (CCR) of the other node in the surviving site to an available quorum device, then to reboot this node into cluster mode.

CAUTION

Experience has shown that this technique is very error prone and requires highly skilled personnel to implement correctly. This option should be considered only if the cause of loss of quorum is a total loss of a site and no system in that site is accessing any data.

A final possibility is to use a third server in the third location to serve as the quorum. This node would then serve as the third vote in a three-site configuration.

Cluster Interconnect Hardware Configurations

Typical NICs can only be used with either copper or multimode fiber cables. However, the maximum distance can be extended by converting the media to single-mode fiber.

Transceivers for Fast Ethernet adapters plug into the RJ-45 or the MII port of the NIC and convert to single-mode fiber cables that can then span more than 15 km (in this combination). This type of transceiver has been qualified for campus clusters based on Sun Cluster 2.2 software.

Similar converters exist for Gigabit Ethernet to convert from multimode fiber to single-mode fiber. Using single-mode fiber, the maximum distance can be extended at least to 5 km. However, because the public network is not part of the cluster, it is up to the administrator to extend the network appropriately.

Data and Volume Manager Configuration

Mirroring data across sites helps to ensure that a copy of data survives any disaster. For campus clusters, host-based mirroring using a volume manager is recommended. However, special care should be taken when configuring Solaris™ Volume Manager (formerly Solstice DiskSuite™) software as the volume manager, especially when distributing replicas. (Refer to the Sun product documentation for details.)

Newer releases of volume managers tend to be equipped with more intelligence regarding placement policies for mirrors. Therefore, it is even more important to have control over this placement process. It is highly recommended to use the appropriate controls provided by the volume managers to spread mirrors across sites.

The prolonged distance between sites may introduce latency problems in accessing data. Volume managers offer a property called "preferred plex," which directs read requests to the preferred local plex, thus avoiding the overhead of going to the remote storage.

Storage Configurations

Since the advent of Fibre Channel, extending the distance between servers and storage devices is no longer a problem. However, limitations in the maximum distance exist that may limit the usefulness of this technology in certain scenarios. Campus clusters using Sun Cluster 3.0 software today support the following:

  • Upgrades from Sun Cluster 2.2 software campus cluster configurations using Sun Enterprise™ servers, Fibre Channel host bus adapters (code named SOC+) and Fibre Channel arbitrated loop (FC-AL), long wave GBICs (LWGBICS), and Sun StorEdge A5x00 storage systems

  • Storage configurations using cascaded Fibre Channel switches with LWGBICs

Wave Division Multiplexers

In many areas, single-mode fiber is too costly or not available in sufficient quantities. A typical campus cluster configuration requires two wires for the storage, two for the cluster interconnect, and at least one for the public network. Additionally, many configurations have another network for backup purposes and one for administration that is connected to the terminal concentrator and other console ports. In total, a single campus cluster might need seven single-mode fiber connections.

WDMs use certain properties of the fiber to multiplex several streams onto a single fiber. Using WDMs over distances longer than 10 km has been successfully tested. This approach enables enterprises to deploy campus clusters in most geographic locations.

  • + Share This
  • 🔖 Save To Your Account

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020