Policy Limits and Queuing
The total number of concurrent policies an Action Server will run by default is 50 (ad hoc and monitored combined). This is known as the Action Server Policy Throttle (ASPT), which is set as part of the installation of OIS 6.2.2 and not modified with the 6.3 upgrade. When an Action Server reaches the maximum number of concurrent policies set by the ASPT, it will no longer run additional policies. If there are no other Action Servers available, the policy will queue and wait for resources to become available.
Consider an example using a single Action Server deployment. You have 49 policies running and your ASPT is set to the default of 50. You trigger two policies to start using the OOC. The first policy executes normally and becomes the 50th policy running on your Action Server. The second policy queues in the datastore and waits until one of the 50 running policies completes before it will run.
Action Server Policy Throttle
The throttle value of 50 is configurable by using aspt.exe found in %ProgramFiles(x86)%\Opalis Software\Opalis Integration Server\Management Service. The aspt executable allows you to change the value on all Action Servers or on a specific Action Server. The following is the usage for this executable:
aspt (ActionServerName or *) (MaxRunningPolicies 1-1000)
To set the policy throttle limit for all Action Servers to 300, enter the following:
aspt * 300
After you change the value for the ASPT, you must restart the Action Server service for any Action Server changed.
Maximum Number of Policies to Run
With the default ASPT of 50, a common question is how many policies can actually be run. There are a number of factors dictating how many policies can be run safely on a single Action Server. These factors include the following:
- Desktop Heap
- Operating System
- Policy size and complexity
- CPU and Memory resources
Refer to Chapter 7 for information on sizing.
Desktop Heap Limitations and Policies
The first resource typically fully consumed by an Action Server (especially those running Windows 2003) is not memory or CPU but the desktop heap. There are several heaps on a server, but when dealing with OIS, the heap being referred to is the desktop heap for the non-interactive desktops (desktop heap).
The default of 50 concurrent policies on a single Action Server is to help prevent exhaustion of the desktop heap. If a system runs out of desktop heap, it can experience unexpected runtime issues such as processes terminating and being unable to allocate proper resources to other processes.
OIS policies all use desktop heap; although they might use different amounts. The amount consumed varies depending on which objects the policy uses and the total number of objects in the policy. The most reliable way to determine the actual consumption of desktop heap by policies is to monitor the resource as a policy runs. If your implementation reaches a steady state, this will give you the best possible estimation of your needs.
Information Technology (IT) organizations often want to have estimates before reaching a steady state—generally when designing the OIS architecture. As a guideline, you can estimate that each policy (policymodule.exe) will consume 10KB of desktop heap.
The desktop heap for the noninteractive desktops is the third parameter of the SharedSection= segment of the following registry value:
Figure 3.2 shows the registry location of the desktop heap, highlighting the Shared Windows Section. The value of the desktop heap for the noninteractive desktops is 768 in Windows Server 2008 (Figure 3.2) and 512 on Windows Server 2003.
Figure 3.2 The registry location of the desktop heap on Windows 2008
Estimating the Maximum Policy Count and Desktop Heap Size
Before setting the desktop heap, estimate the maximum number of policies you expect an Action Server to run.
The desktop heap for the noninteractive desktops can be estimated by
(Maximum # of concurrent policies) * 10 = (Desktop Heaps)
As an example, if you want to run 100 concurrent policies, 100 * 10 = 1000, rounding that number to the next highest memory size gives you 1024. The new value for the desktop heap in the registry segment would look like this:
This example uses the estimate of 10K for a policy. After you have working examples of your policies, determine the actual value for your policies and revise the desktop heap size based on the actual value.
You will also want to consider that because the Service Control Manager creates a new desktop in the noninteractive window station for each service process running under a user account, increasing the desktop heap for non-interactive desktops reduces the number of user account services that can run on the system.
Increasing the Action Server's Desktop Heap
To increase the number of policies an Action Server can run, perform the following steps:
- Increase the size of the desktop heap for noninteractive window stations on your Action Servers.
- Modify the value of the ASPT on your Action Servers.
- Reboot the Action Servers. Increasing the desktop heap is a Windows systemwide change and requires a reboot. Altering the ASPT requires you restart the Action Server Service.
You can find additional information about desktop heap in Knowledge Base (KB) article 184802, available at http://support.microsoft.com/kb/184802.
Policy Maximums Based on Operating System
OIS 6.3 Action Servers can run on Windows Server 2003 or 2008; the total number of concurrent policies supported by each operating system (OS) will differ. As the maximum total heap size for a Windows 2003 server is limited to 48MB, Action Servers on Windows 2003 are not able to run as many policies as those on Windows 2008 (assuming the heap size is maximized for each). The exact number of policies you can run should be determined by testing; however, you can use the following guidelines when estimating the maximum number of concurrent policies per Action Server:
- Windows 2003—250 concurrent policies
- Windows 2008—500 concurrent policies
These are only suggested maximums. Your Action Servers might not be able to run the maximums listed given other constraints. (As an example, if you have large policies, you might not be able to run 250 concurrent policies on a Windows 2003 Action Server.) You might be able to run more policies than these maximums, but if you attempt to do so, be sure you understand the performance aspects and implications involved. Normally, you would want to add more Action Servers rather than risk over committing those Action Servers you have.
Policy Size and Complexity
The total number of objects and type of objects in a policy will change the memory footprint and resource consumption considerably. There are no good sizing guidelines, as policies can vary incredibly in size. A four-object policy that creates help desk incidents based on a SQL query easily consumes more resources than a user provisioning policy with 20 objects. Imagine if the SQL query produces 5,000 rows, which in turn creates 5,000 incidents in the help desk. That four-object policy is far more resource intensive than passing one set of user data through the 20-object policy.
CPU and Memory Resources Also Affect Policy Limits
CPU and memory resources and other typical performance metrics are more likely to apply to Windows 2008 Action Servers rather than those running on Windows 2003. The reason for this is Windows 2003 servers are limited to about 250 concurrent policies, and modern server hardware can generally handle that load quite easily. As Windows 2008 can run about twice as many policies, it is possible that normal performance resources might become strained.
All the performance aspects—heap size, operating system, policy size, and performance metrics—should be methodically tested using real-world data. This is the best and most reliable method to understand what impact your policies will have on your Action Servers. The estimates provided in this chapter are only the starting point for your calculations.