Overview

In some circumstances the default ASP.NET settings require tuning. The symptoms will be that CPU and memory are not exhausted but client performance is slow.

If CPU usage is high then this document is not applicable. This document describes increasing default ASP.NET values in the case where CPU is low but ASP.NET worker processes are bottle necked by the ASP.NET default performance values.

If CPU is high and the setting below have not been implemented than that is usually indicative of a system that requires more CPU cores. For example, either "scale up" the CentreStack virtual machine, or "scale out" to a second CentreStack virtual machine if it is not practical to "scale up".

If memory usage is high see this article: How to Control CentreStack's Memory Consumption

The changes to the system that are described in this article will increase CPU consumption on the CentreStack web server. Monitor CPU usage after these changes are implemented. 

Details

Disable IIS Logging

IIS logging is enabled during IIS installation/configuration. It may affect IIS performance. Suggest to disable the IIS logging in regular operation. Only need to enable it when need to check the incoming traffic directly.

To disable the IIS Logging:

• Open IIS Manager
• Select server on Connections. Double click Logging
  
• When logging is enabled, see 'Disable' link on Actions on the right pane. Click it to Disable the logging.
  
• Once IIS Logging is disabled, under Action, will see 'Enable' link. It confirms the logging is disable.

Initial Troubleshooting

In latest Centrestack, login as cluster admin on web portal. Go to Cluster Dashboard, Reports, System Diagnostic Report. Here, cluster admin can visit Performance Monitor

When client performance is noticeably slow but CPU is also low, you may see on the Dashboard of this page that the number in the Request card is high and either the Dir Queue(High) and/or Dir Queue(Low) cards show a high queue length or high drop rate:

You may also see in the Live Requests tab that the number values are very high. These number values reflect the number of milliseconds it takes for the request to complete.

Automated Configuration

The Set-CSPerfTuning.ps1 PowerShell script can be used to automate all of the configuration changes that are detailed below in the Manual Configuration sections. The script is the recommended method for implementing these changes because it is fast and less error prone than manual configuration.

Script Prerequisites

• The Set-CSPerfTuning.ps1 script requires PowerShell 5.1. Windows Server 2016 and later includes PowerShell 5.1. Older version of Windows can be upgraded to PowerShell 5.1 by installing the Windows Management Framework 5.1 update. To see what version of PowerShell is installed, start PowerShell and execute this command:

  Get-Host

• The Set-CSPerfTuning.ps1 requires that PowerShell be elevated ("run as Administrator").

Script Execution Process

Use the instructions to download and execute the PowerShell script:

1. In a web browser, download: Set-CSPerfTuning.ps1
2. Start an elevated PowerShell session (i.e. running as Administrator)
3. Make sure the system is configured to run PowerShell scripts by executing:
   

   Set-ExecutionPolicy RemoteSigned

4. Change the current directory to the directory where the Set-CSPerfTuning.ps1 is located, for instance:
   

   cd $env:UserProfile\Downloads

5. Since the script was downloaded from the Internet, it must be unblocked before it can be executed:
   

   Unblock-File .\Set-CSPerfTuning.ps1

6. To view basic command line syntax and comment based help execute:
   

   .\Set-CSPerfTuning.ps1 -?

7. The script has no command line parameters so execute:
   

   .\Set-CSPerfTuning.ps1

The script modifies each of these files but a backup of each file will be made each time the script executes. The backup file name will be the original filename appended with a timestamp in the format "yyyy-MM-dd_hh-mm-ss".

• "C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Aspnet.config"
•  "C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config"
• "C:\Windows\System32\inetsrv\config\applicationHost.config"
• "C:\Program Files (x86)\Gladinet Cloud Enterprise\root\web.config"

Manual Configuration

Manually configuring the system is not recommended unless the system administrator is unfamiliar with executing PowerShell script or the system cannot be upgraded to PowerShell 5.1. 

Tune Aspnet.config

Settings altered in this section are documented here: <applicationPool> Element (Web Settings) 

1. Backup "c:\windows\Microsoft.NET\Framework64\v4.0.30319\Aspnet.config"
2. Start a text editor "as Administrator"
3. Open "c:\windows\Microsoft.NET\Framework64\v4.0.30319\Aspnet.config"
4. This file is typically very small. By default there will not be a "<system.web>" section. Add this section above the line that reads: </configuration>

   <system.web>
    <applicationPool maxConcurrentRequestsPerCPU="5000" requestQueueLimit="20000"/>
   </system.web>

   The new section will look like this in your text editor:
   

5. Save the file.

Tune machine.config

Update the processModel settings

1. Backup "C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config"
2. Start a text editor "as Administrator"
3. Open "C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config"
4. Search for:

   <processModel autoConfig="true"

5. The default will typically look like this in your text editor:
6. Replace:

   <processModel autoConfig="true"/>

   With:

   <!--
   <processModel autoConfig="true"/>
   -->
   <processModel autoConfig="false" maxWorkerThreads="500" maxIoThreads="500" minWorkerThreads="2"/>

   Notice that the original default was commented out and replaced with custom settings.
7. Your text editor should now display something similar to this:
   
8. Save the machine.config file but keep the file open as another change will be made in the next section

Update the connectionManagement settings

The connectionManagement setting is documented here: <connectionManagement> Element (Network Settings)

1. Search machine.config for

   <system.net>

   By default the system.net section won't exist in machine config.
2. If the system.net section doesn't exist search for:

   </configuration>

   and add the following text before the </configuration>:

   <system.net>
       <connectionManagement>
           <add address="*" maxconnection="20000"/>
       </connectionManagement>
   </system.net>

3. Your text editor should now look similar to this:
   
4. Save the machine.config and close the file.

Set the appConcurrentRequestLimit

The appConcurrentRequestLimit is documented here: Server Runtime <serverRuntime>

1. Start the Command Prompt "as administrator"
2. Backup "C:\Windows\System32\inetsrv\config\applicationHost.config"
3. Execute this command:

   "C:\Windows\System32\inetsrv\appcmd.exe" set config "Default Web Site" -section:system.webServer/serverRuntime /enabled:"True" /appConcurrentRequestLimit:50000 /commit:apphost

This XML block will be added to "C:\Windows\System32\inetsrv\config\applicationHost.config":

 <location path="Default Web Site">
     <system.webServer>
         <serverRuntime enabled="true" appConcurrentRequestLimit="50000" />
     </system.webServer>
 </location>

Tune CentreStack web.config

In the case where either the Dir Queue(High) and/or Dir Queue(Low) cards show a high queue length or high drop rate and CPU is low:

1. Backup "C:\Program Files (x86)\Gladinet Cloud Enterprise\root\web.config"
2. Start a text editor "as Administrator"
3. Open "C:\Program Files (x86)\Gladinet Cloud Enterprise\root\web.config"
4. Locate:

   </appSettings>

5. Before the </appSettings> line add these lines:

   <add key="HighDirWorkerCount" value="20" />
   <add key="HighDirMaxQueueLength" value="35" />
   <add key="LowDirWorkerCount" value="10" />
   <add key="LowDirMaxQueueLength" value="15" />

6. Your text editor will look similar to this:
   
7. Save web.config.

Note: You may experiment with setting HighDirWorkerCount to a value higher than 20, but this will have a direct impact on CPU utilization. 

Final Steps

You will need to restart the IIS Web Server for the changes above to take affect. At an elevated command prompt execute:

iisreset

Once these changes have been implemented you must monitor the CPU usage. If the CPU is very high and you are unable to "scale up" the CentreStack server, then you will have no choice but to revert the changes made above.