Troubleshoot High CPU Issues on IIS Using DebugDiag

In this article, we will look into troubleshooting High CPU consumption by a web site hosted on IIS 7 and IIS 7.5. Sometimes, our ASP.NET application might consume high CPU on the w3wp.exe process. This means that the application is behaving abnormally due to a code bug or heavy load and so on. We need to find the root cause for this issue. We can troubleshoot this type of issue by collecting a memory dump of the w3wp.exe at the time of the issue. Let's download DebugDiag 1.2 ([available here) based on server bit-ness (32 or 64 bit).

Let's create a sample MVC application. Add Following code to simulate the CPU spike:

public class HomeController : Controller

{

public ActionResult Index()

{

string s = "";

while (true)

{

s += "AAA";

}

return View();

}

Now publish this app on IIS using the following procedure.

1. Go to the Project properties and click on Web tab.
2. Select "Local IIS Web Server" as shown below:
   
   
   

When we browse to the application (http://localhost/MvcApplication4), we will see a CPU spike on w3wp.exe.



Let's open DebugDiag and collect a full user memory dump of the spiking w3wp.exe as shown below:



This will create the following two files:

1. The *.dmp file havs the memory dump of the process containing information about the threads, stack trace, locks and so on.
2. The *.txt file contains a list of processes running at the time of dump collection.
   We need to collect at least 2-3 memory dumps with an interval 4-5 seconds to ensure that we are troubleshooting in the correct path. Once it is done, go to the Advanced Analysis tab and add the dump file with the analysis script Crash\hang Analyzers as shown below than click on "Start Analysis":
   
   
   

It will open up a .mht report in the browser:



Thread 22 has triggered GC and its stack trace is:



This analysis report provides the following details:

1. All running threads along with CPU time.
2. List of all running requests (under Client Connections).
3. Stack trace of each thread.
4. Process details like up time, process ID and so on.
5. Server and OS details.

We can further analyze the dump using the WinDbg tool and determine why GC has been triggered.

Let's complete the explanation with an overview:

1. High CPU utilization may occur due to an infinite loop in code or heavy load or by exhausting memory and in turn triggering GC, which spikes CPU.
2. To troubleshoot the issue, first restart IIS using the iisreset command and collect 3-4 sets of memory dumps at the time of the CPU spike.
3. Analyze each dump individually using Advanced Analysis of DebugDiag.
4. Look at the thread running for a long time and analyze its stack trace from bottom to top.
5. Dig into the method having the issue as per stack trace by looking into the code.
6. If the advanced analysis won't help much then we can use Debugging Tools for Windows specifically the WinDbg tool to analyze the dump (we will discuss this in the next article).
7. The WinDbg tool is best suited to troubleshoot .NET (Managed) applications, that have extensions to dump the CLR Stack, Heap objects and so on.