J2EE Performance Tuning, Part 3: Application Server Architecture

Date: Apr 4, 2003

Article is provided courtesy of Sams.

By understanding what a J2EE application server must do, you can understand how it will do it, which leads directly to how to tune it. Steven Haines delves deep into the internal workings of a J2EE application server by examining the J2EE specification in the context of leading application servers.

Thus far in this series on J2EE performance tuning, we have quantified what we mean by performance tuning (maximize concurrent users, throughput, and reliability), and we have defined a methodology that we will use when tuning our applications and application servers. Now we need to look under the hood of a generic application server and see what can be tuned and the impact of that component on our environment. In this article, we will look into the J2EE specifications and apply practical knowledge of application server implementation details to identify those tunable components.

Under the Hood: What Does an Application Server Have to Do?

At the time of this writing, most production application servers are J2EE 1.3-compliant, meaning that they satisfy all the requirements defined in the J2EE 1.3 specification. Although there is one J2EE specification, I intentionally referred to it as specifications because the specification refers to a set of other "Application Programming Interfaces" that have their own individual specifications.

Inside the J2EE Specifications

Let's take a look inside the J2EE 1.3 specification. It is available at the following URL: http://java.sun.com/j2ee/download.html.

Depending on when you visit this link, you will notice that the 1.4 specifications are in "Proposed Final Draft 2" (or later), but the reason I am focusing on 1.3 is because application servers do not yet support 1.4.

Chapter 6 of the J2EE specification defines the set and versions of component specifications that must be supported. Those are summarized in Table 1.

Table 1—J2EE Specification Components and Versions

Component	Version
JDBC	2.0
Enterprise JavaBeans(EJBs)	2.0
Servlet	2.3
JavaServer Pages (JSP)	1.2
Java Messaging Service(JMS)	1.0
Java Transaction API (JTA)	1.0
JavaMail	1.2
J2EE Connection Architecture (JCA)	1.0
Authentication and Authorization (JAAS)	1.0

You can find links to all the specifications for these technologies inside the J2EE Specification, so we will focus on the important aspects of each with respect to performance.

Because each application server must support the aforementioned APIs, we can look at an application server from a general perspective and understand what we might want to tune. Then, when we look at individual application servers, it is just a matter of finding the implementation of that conceptual technology.

JDBC

All application servers must provide a pooling mechanism for database connections. The creation of a database connection from within your application is an expensive operation and can take anywhere between 0.5 and 2 seconds. Thus, application servers pool database connections so applications and individual threads running inside applications can share a set of database connections.

The process is as follows: A thread of execution needs to access a database, so it requests a connection from the database connection pool, it uses the connection (some execution of a SELECT or UPDATE or DELETE statement), and then it returns the connection back to the connection pool for the next component that requests it. Because J2EE applications support many concurrent users, the size of your connection pools can greatly impact the performance of your application. If 90% of your requests need database connections and you want to be able to support 500 simultaneous users, you will need a substantial number of connections in your connection pool. Note that when factoring in think time, you will probably need far fewer than 500 connections, but you will need quite a few.

Each application uses the database differently, and thus tuning the number of connections in your connection pool is application-specific. It is important to keep in mind that in practice tuning, JDBC connection pool size is one of the factors with the highest impact on your application's overall performance.

Enterprise JavaBeans (EJBs)

Enterprise JavaBeans (EJBs) provide the middleware layer of your J2EE application. They come in four flavors:

Entity Beans
Stateful Session Beans
Stateless Session Beans
Message Driven Beans

Both Entity Beans and Stateful Session Beans maintain some kind of stateful information. An Entity Bean may represent a row in a database or the result of some complex query, but regardless, it is treated as an object in your object-oriented model. Stateful Session Beans, on the other hand, represent temporary storage that exists beyond the context of a single request, but is not stored permanently. Typically, in a web-based application, a Stateful Session Bean's lifetime will be associated with a user's HTTP session. Because their nature is to maintain state, the application server must provide some form of caching mechanism to support them. Simple application servers may maintain a single cache that stores all Entity Beans and Stateful Session Beans, whereas more-advanced application servers provide caches on a bean-by-bean basis.

A cache has a preset size and can hold a finite number of "things." When a request is made for an item, the cache is searched for the requested item: If it is found, it is returned to the caller directly from memory; otherwise, it must be loaded from some persistent store (for example, database or file system), put into the cache, and returned to the caller. Once the cache is full, it becomes more complicated: The cache manager must select something to remove from the cache (for example, the least-recently used object) to make room for the new object. The EJB term used for removing an item from the cache to make room for the new item is passivating, and the term used for loading an item into the cache is activating. If activation and passivation are performed excessively, the result is that the cache manager spends more time reading from and writing to persistent storage than serving requests; this is called thrashing. On the other hand, if the cache manager can locate an item in its cache and return it to the user, the performance is optimal; this is referred to as a cache hit.

When tuning a cache, the goal is to maximize the number of cache hits and minimize thrashing. This is accomplished after a thorough understanding of your application's object model and each object's usage.

Stateless Session Beans and Message Driven Beans are not allowed to maintain any stateful information; a single process may request a Stateless Session Bean for one operation and then request it again for another operation, and it has no guarantees that it will receive the same instance of that bean. This is a powerful paradigm because the bean manager does not have to manage the interactions between business processes and its bean; it exists simply to serve up beans as requested.

The size of bean pools must be large enough to service the requests from business processes; otherwise, a business process will have to wait for a bean before it can complete its work. If the pool size is too small, there are too many processes waiting for beans; if the pool size is too large, you are using more system resources than you actually need.

Another helpful effect of the fact that Stateless Session Beans and Message Driven Beans are stateless is that application servers can preload them to avoid the overhead of loading beans into a pool upon request; the request would have to wait for the bean to be loaded into memory before it could be used.

Two of the most influential tuning parameters of Stateless Session Bean and Message Driven Bean pools are the size of the pools that support them and the number of beans preloaded into the pools.

Servlets and JSPs

Although there are individual specifications for both Servlets and JavaServer Pages, the end result of both is a Servlet class loaded in memory; JSPs are translated from a JSP to a Servlet Java file, compiled to a class file, and finally loaded into memory. Servlets and JSPs do not maintain state between requests, so application servers pool them. So you can tune the pool size and the number of Servlets that are preloaded into the pools.

Because JSPs go through the translation and compilation step prior to being loaded into memory, most application servers provide a mechanism by which you can precompile your JSPs before you deploy them. This removes the delay that end-users would experience the first time a JSP is loaded.

Servlets (and JSPs) are required to maintain four different scopes, or areas of memory that data can be stored in:

Page: Data stored here exists for the context of a single page.
Request: Data stored here exists for the duration of a request (it is passed from Servlet to Servlet, JSP to JSP, until a response is sent back to the caller).
Session: Data stored here exists for the duration of a user's session (it exists through multiple requests until it is explicitly removed or it times out).
Application: Data stored here is global to all Servlets and JSPs in your application until it is explicitly removed or until the Servlet container is restarted.

As a programmer, the choice of where to store data is a very important one that will impact the overall memory footprint of your application. The greatest impact, however, is the session scope: The amount of data that you store in here is multiplied for each concurrent user. If you store 10 kilobytes of data in your session scope for each user and you have 500 users, the net impact is 5MB. 5MB might not kill your application, but consider all 500 users going away and 500 more come. If you do not "clean up" after the users that left, you now are using 10MB, and so on. HTTP is a stateless protocol, meaning that the client connects to the server, makes a request, the server responds, and the connection is terminated. The application server then cannot know when a user decides to leave its site and terminate the session. The mechanism that application servers employ, therefore, is a session timeout; this defines the amount of time that a session object will live without being accessed before it is reclaimed. The session timeout that you choose will be dependent on your application, your users, and the amount of memory you are willing to set aside to maintain these sessions. You do not want to interrupt a slow user and make him restart his transaction, but you do not want to drain your system resources with a timeout that is any longer than is necessary.

Java Messaging Service

JMS Servers facilitate asynchronous operations in your application. With the advent of EJB 2.0 came the introduction of Message Driven Beans: stateless beans representing business processes that are initiated by a JMS message. You put a message in a JMS destination, which is either a Topic or a Queue, and someone takes that message out of the destination and performs some business process based off of the contents of that message.

Because JMS Servers host messages, application servers usually define limits either to the number of messages that can be in the server at any given time or size of the messages in bytes. You can define these upper limits; the balance is between memory consumption and properly servicing your JMS subscribers. If your thresholds are too low, messages will be lost; if your thresholds are too high and the server is used to an excessive upper limit, it can degrade the performance of your entire system.

Along with total storage requirements, there are other aspects of JMS Servers that can be tuned, including the following:

Message Delivery Mode: persistent or non-persistent
Time-to-live: defines a expiration time on a message
Transaction States
Acknowledgments

Each of these aspects will be explored in detail in later articles, but the basic questions that you have to ask are these: How important are these messages to the business process? Does it matter if one or more messages are lost? Obviously, the less you care about the messages actually reaching their destination, the better the performance—but this will be dictated by your business requirements.

Java Transaction API (JTA)

Application server components can be transactional, meaning that if one part of a transaction fails, the entire operation can be rolled back; and all components participating in the transaction are also rolled back. J2EE however, defines different levels of transaction participation. The more your components participate in a transaction, the more overhead is required, but the more reliable your business processes are. EJBs define several levels of transactional participation on a method-by-method basis for each bean's methods:

Not Supported: The method does not support transactions and must be executed outside of any existing transaction.
Required: A transaction is required, so if one exists, this method will use it; otherwise, you have to create a new one for it.
Supports: A transaction is not required, but if one exists, this method will participate in it.
Requires New: A new transaction is required, so regardless if one exists or not, a new one must be created for this method.
Mandatory: A transaction is required and furthermore must be passed to this method; if a transaction does not exist when this method is invoked, it will throw an exception.
Never: A transaction is forbidden; if this method is called and a transaction exists, the method will throw an exception.

The implications of each should be apparent, and the performance impact is as you can guess: Supported is the most unintrusive and yields the best performance at the cost of possible loss of data; Required is safe, yet a little more costly; and Requires New is probably the most expensive.

General Considerations

We have touched on most of the major performance-related aspects that are found in the various J2EE specifications, but there are three things that dramatically affect performance that are the natural side effects of running an application server. Because application servers can service multiple simultaneous requests and because thread creation is expensive, application servers have to maintain a pool of threads that handle each request. Some application servers break this thread pool into two: one to handle the incoming requests and place those in a queue and one to take the threads from the queue and do the actual work requested by the caller. Regardless of the implementation, the size of the thread pool limits the amount of work your application server can do; the tradeoff is that there is a point at which the context-switching (giving the CPU to each of the threads in turn) becomes so costly that performance degrades.

The other performance consideration is the size of the heap that the application server is running in. We already saw that it needs to maintain caches, pools, sessions, threads, and your application code, so the amount of memory you allocate for your application server has a great impact on its overall performance. The rule-of-thumb is to give the application server all the memory that you can afford to give it on any particular machine.

Finally, you must consider tuning the garbage collection of the heap in which your application server is running. As you load things into memory (and unload them), your heap will soon fill up—requiring that garbage collection free memory for your application to continue. In the Sun JDK version 1.3.x, which ships with most production application servers, it defines two types of garbage collection: minor and major.

Minor collections are performed by a process called copying that is very efficient, whereas major collections are performed by a process called mark compact, which is very burdensome to the Virtual Machine. The heap is broken down into two partitions: the young generation, in which objects are created and hopefully destroyed; and the old generation, in which objects are retired to a more permanent place in memory. The young generation uses copying to perform minor collections, whereas the old generation uses mark compact to perform major collections, so your goal when tuning garbage collection is to size the generations to maximize minor collections and minimize major collections.

Summary

We have covered a whole lot of ground in this article. Namely, we have looked at the J2EE specification, observed the individual APIs that it requires, and analyzed them for performance-tuning parameters. At this point, you should be able to take this information and look more deeply into your application server.

What's Next?

This article, the third in the series, completes the introduction to J2EE performance tuning. This introduction to application server architecture is a great launching pad into the tuning of specific application servers. If there is enough interest and user response to continue this series, the first application server we will dive into is BEA WebLogic. So if you find this information useful, please post a response to this article accordingly. Thanks for reading!