Java

Clusters for Performance

Last updated Mar 25, 2005.

To ensure the availability of a J2EE application, it's typical for companies to implement clusters, in which two or more machines that service the same requests. Here's the basic concept: if an application server crashes, the load balancer (which is distributing load between the application servers in the cluster) simply omits that application server instance, and spreads the load across the remaining servers.

Figure 62 shows the distribution of load across all servers. If one of those servers has to be brought down, or if the server crashes, the load can be redistributed across only three of the four servers, as you see in Figure 63.

To further enhance high-availability in J2EE applications, each application server usually has one or more server with which it shares its stateful information. When an application server goes down, future requests are directed to the "backup" application server instance that retains the stateful components of the client/server communication — and as far as the end-user knows, nothing happened. The result is high-availability, through both scheduled upgrades/maintenance as well as during application server crashes.

Clustering is an imperative part of any application server, because down time affects the business of your company in numerous ways:

• In a business-to-consumer environment, downtime may force your users to turn to your competitors for service; this results in a direct loss of revenue.
• In a business-to-business environment, downtime can mean a loss of revenue, as well as a degradation of confidence in your corporate infrastructure. Working in the B2B e-commerce industry I learned that major financial institutions can measure that cost of delayed transactions sometimes in the millions of dollars per down hour.
• In an intranet application, downtime can mean a loss in productivity by your corporate staff, which results in a net loss to your company.

All major commercial vendors provide clustering technology, including BEA, IBM, Oracle, as well as in the open-source alternative Jboss. I do not know the roadmap for when Apache Geronimo will support clustering, but they do have a subproject to support it.

High-availability at all tiers of your application is essential to your business function, so you should have two or more instances of each of the following components, working together in a clustered environment:

• Web Tier
• Middleware (application server)
• Database

Turning our attention from high-availability to performance, clustering can have a significant impact on the performance of your application. Before looking at the details, it is important to identify the two types of clusters that can be created in a J2EE environment:

• Horizontal Clusters
• Vertical Clusters

Horizontal clustering means deploying application server instances (in a cluster) across multiple physical machines. Vertical clustering is deploying multiple application server instances on a single machine. Figure 64 graphically displays the differences between these.

The performance benefit of creating a horizontal cluster is fairly obvious: using more machines helps you solve your problem better than using a single machine. The concept of a cluster is that with one application server instance, you should be able to solve your problem (or in this case service your users), but with more application server instances, you should be able to service more users and faster!

The performance benefits of creating a vertical cluster is not so obvious. There are two main benefits:

1. When you bring down a logical application server instance running on a vertically clustered machine, you do not lose the full capabilities of that machine, just that particular instance.
2. Studies have successfully demonstrated that a single Java process cannot effective utilize all of the resources on a powerful machine; multiple processes have been shown to more effectively utilize system resources.

Figure 65 demonstrates the first point: if you are running a single application server instance on a machine and it goes down, then you have lost 50% of your capacity. Whereas if you are running two application server instances on a single machine and one of them goes down (or you bring it down), you have only lost 25% of your capacity. There is a point of diminishing returns here; you do not want to take this to an extreme and install 100 application server instances on a single machine (unless you have 150-200 CPUs, of course.)

Addressing the second point, here are a couple references for you:

1. BEA has found that in general, "optimal results are generally obtained using a ratio of one Weblogic Server instance for every two CPUs." If you have a four CPU machine, they recommend that you run two application server instances on it.
2. In IBM's "Performance, Scalability, and High Availability WebSphere Handbook Series" (sg246198), page 18, it states, "In vertical scaling ... multiple cluster members for an application server are defined on the same physical machine, or node, which may allow the machine's processing power to be more efficiently allocated."

It has been my experience, out in the field, that if we install several application server instances on a single machine to create a vertical cluster, that application throughput increases.

This is evidence that vertical clusters can help the performance of your application. But the next logical question is: considering the guidelines, what is the number of application server instances that is optimal for my environment? The answer depends completely on your application and the nature of your users interacting with your business processes.

But before determining the optimal application server-to-CPU ratio, you need to determine if your application is truly bound by CPU or if it is bound by network or disk I/O:

• Network Requirements: using performance monitoring tools, assess the quantity of network traffic generated by your application and compare that to your network capacity. If your application is bound by your network capacity, then modifying your network (faster cards, cabling, etc.) will have a more significant impact to the performance of your application than adding additional CPUs. This is partially the result of CPUs remaining idle while waiting for a network socket to respond.
• Disk I/O Requirements: again, using performance monitoring tools, you should assess the quantity of data being read from and written to disk. If you are spending significant time here, you will gain better performance impact by upgrading your disk I/O hardware than CPUs; this may be the result of CPUs sitting idle while waiting for data to be written to or read from disk.

After you determine that your application is truly CPU-bound, you can start defining multiple application server instances on a single box, and assess performance. Start by defining one application server instance to every two CPUs, load test your application, and observe the usage of each CPU. If the CPU usage is constantly at or below 50%, you can add additional application server instances to try to optimize CPU utilization. If the CPU usage is closer to 100%, you may want to back off and run fewer application server instances.

The ideal usage for each CPU on your machine is between 75% and 90% utilization while your application is under severe stress; you want to ensure that you are maximizing system resource utilization, but you also do not want to saturate your hardware and create contention!

Vertical clustering can significantly improve the performance of your J2EE environment if you start with best practice values, observe the behavior of your environment under stress, and adjust appropriately to the aforementioned indicators.

Summary

In this series I am relating information about engagements that I have been on over the past nine months tuning J2EE enterprise applications. We have covered the following subjects:

1. The three most common application server tuning problems that I have seen: misconfigured heaps, inappropriately sized and allocated thread pools, and connection pools.
2. My general approach to tuning: the wait-based tuning methodology. Wait-based tuning is nothing new and as a matter of fact Oracle 9i databases (and later) provide these metrics natively – in J2EE the waits still need to be inferred, but the approach is solid!
3. How to proactively architect your application object-lifecycles to reduce memory overhead.
4. How vertical clusters can improve the overall performance of your J2EE applications

Performance tuning is an iterative process of constantly trying to hone in to the optimal values and setting of performance criterion and I hope that by analyzing your applications and application servers in light of what we have discussed here will help you improve your application performance and enhance your end-user experience.

---