Check Application Performance Under Load Test Using Visual Studio

Background

Before starting work in performance optimization, first need to set a performance benchmark for your site. As I mentioned in my previous article, if you need to improve your application performance then first check existing performance and then work on optimization to meet performance benchmarks.

Introduction

Please read my first article Check .NET Application Performance Using Performance Optimization before, because it is their extension.

Before starting the Load test, you should be aware of the difference between the Load Test and the Performance Test.

Performance testing is the testing, that is performed, to ascertain how the components of a system are performing, given a particular situation as I explained in my previous article. There I explained admin users log into the site create some records and then check their performance against a single user. A load test is a container of performance test and it runs a performance test under a certain load till it reaches its threshold limit. So in the Load test same functionality (Admin user) will be run by multiple concurrent users.

Create a Load test step by step and verify the results

We will use the same project created in the last article.

Step 1. Create Load Test In Solution Explorer, right-click the existing Web Performance and Load Test project node, Add, and then choose Load Test.

Step 2. Load test wizard starts, provide Scenario name. A scenario consists of a set of multiple performance tests.

E.g. Scenario Name, Buy item.

• Test 1: Login into the site, and the Search for item.
• Test 2: Search Item, Add to the cart, and Place an order.

Here you can set Think time (artificial human delay times between Web requests).

Step 3. Set Load Pattern setting, two types of patterns can be provided.

1. Constant Load: Specify the same number of users constantly hitting your application for a specified time. But at the beginning of the load test, it is not reasonable and realistic demand for any site. So use it carefully.
2. Step Load: Specify a user load that increases with time up to a defined maximum user load. It is a good option to check, what maximum user load capacity that your application can handle at a particular server configuration. As the load increases, the server will eventually run out of resources.

Step 4. Set Search Item for load test, here if your application has multiple workflows like Search Items, Add to Cart and Place Order, Upload Master Data. So you can define load according to your work flow means how many virtual users will use which functionality more. In general, Search Items are most widely used by customers in e-commerce sites rather than Add to Cart and Upload Master Data.

There are four model types that you can provide.

1. Based on the total number of tests: In this case, you can select which Web Performance Test will run more when a virtual user starts hitting your application. After load test execution, it matches a number of test runs with the assigned test distribution. You will see this below under the result section. For example, I have the following 3 Test cases and during the load test if there will be 1000 test cycles runs (i.e. same test case runs multiple times), then
   • 600 Search Item test cases should be executed
   • 300 Add to Cart and Place Order tests case should be executed
   • 100 Upload Master Data tests case should be executed
2. This describes the maximum iterations that will occur for the Search Item test case.
3. Based on the number of virtual users: It provides % the age of virtual users that will run a particular test case. E.g. If there are 1000 active users on your site, then 600 users are using the Search Item functionality. It means continually 60% of the total user load will keep using the Search Item module without keeping track that how many cycles have been completed for the Search Item workflow.
4. Based on user pace: it provides frequency for particular tests that will run many times per user per hour.
5. Based on sequential order: Here you can specify the sequence for test cases that need to execute in some order. During the load test same order will follow through multiple loops until the load test is complete.

Generally, we use only the first two options to get more realistic results.

Step 5. Add Web Performance Test that needs to be run under load test. Here you can add multiple Web Performance Tests (created earlier in the previous article). One workflow can be one performance test.

You can also provide Load distribution, value for the Test Mix Model selected in step 4, across multiple Web Performance test cases and can provide more realistic load distribution based upon application usage. In the following image, 71% of applications are used by admin, and 29% are used by accountants.

So for any eCommerce Site, you can create the following type of Web Performance Test cases and distribute load accordingly. E.g.

• Customer Search Items – 60%
• Add items to Cart and Place Order – 25%
• Customer Registration – 5%
• Upload master data – 10%

Step 6. Specify which type of Network connection will be used by the client to connect your site.

Distribute load against different types of network connections e.g. LAN 60%, 3G 15%, CDMA 10%, etc. Keep LAN 100% to test applications hosted on your local machine.

Step 7. Specify which type of browser is used by the end user. Create a closer approximation of the Web browsers that will be used with your applications. A Web browser type is randomly associated with a virtual user, based on the browser mix.

Step 8. Specify the Counter Set for resources against which you want to collect data from the server where the Load test will run. There are three counter-categories: percentages, counts, and averages. E.g. % CPU usage, SQL Server lock counts, and IIS requests per second.

Click Add Computer and select resources against which you want to capture the counter set. E.g. ASP.Net, IIS, SQL in the above figure.

You can also specify remote server (website host machine) and threshold rules that are set on an individual performance counter to monitor system resource usage during a load test. It keeps monitoring what is threshold value and what is current value for the counter set. E.g. CPU % threshold value is 80%.

Step 9. Specify Run Settings to determine such properties as the duration of the test, warm-up duration, sampling rate, connection model (Web performance tests only), results storage type, validation level, and SQL tracing.

There are two options

1. Specify Load test duration,
   1. Specify the Warm-up duration (hh mm ss). Use the hour, minute, and second spin controls.
   2. Specify the Run duration (hh mm ss). Use the hour, minute, and second spin controls.
2. Specify the number of times to run the test. Use the Test iterations spin control.

Apart from that you can also specify the Sampling Rate and Validation level (validation that was mentioned during the web performance tests like Response Goal should be 1 second).

Step 10. Double-click the .loadtest file and click the Run test button to run the load test.

Click Manage Test Controller, here you can also provide a Database connection (The database name is LoadTest2010) where you want to store load test results.

Note. If you don’t have the LoadTest2010 database on the server. Then go to the IDE folder on your machine where Visual Studio installs search for the “loadtestresultsrepository.sql” file and run the same on the server to create the LoadTest2010 database. E.g. C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\loadtestresultsrepository.sql

Now you have done all the required settings for the Load test, so let us run the Load test and verify the results step by step.

Run Load Test and analyze the results

Step 1. Click the Run button, after running successfully different types of result reports are available like Summary, Graphical, Table, or Details. By default, Summary will open, and here the following points to be noted down.

Top five slowest pages, here you can get a list of pages which are taking more time for processing.

Total number of Web Performance Test cased execution distribution, as we specified distribution during load test creation admin 71% and accountant 29%.

Type of errors occurs, here ResponseGoal was set 1 second for all pages under admin web performance test but almost all the pages fail to achieve this target. On clicking of Error link, you can also get exact details.

Step 2. Click Graphs, here you can see multiple graphs at a time. On the left-hand side, Counter set values for all resources consumed. On the right-hand side, resource consumption details are in the form of Graphs and Range (Max, Min, and Avg) values.

Under the Counter Set panel, under the Computer Name node, all resource data is available. Some of them are highlighted below.

1. .NET CLR resources: It provides all resource consumption against load tests like how many bytes from the heap are consumed by your application. On the right-hand side under the Key Indicator graph, you can see .NET CLR memory from heap consumption is increasing during load. Here you can take decision if there is any memory leakage or not.
2. IIS (ASP.NET/ASP.Net Application): It provides resource consumption at the IIS level like Request/Sec, Session Active, Request Rejected (due to security or any reason), Request Queued, Output Cache Hit Ratio (how many pages are processed directly from cache), Cache Hit Ratio (how many times application cache used), etc.
3. Memory: It provides how much memory is consumed during load tests like % Committed Byte In Use, and Available Mbytes (amount of physical memory, in MB, available to running processes).
4. Network Interface: It provides network resource consumption like Bytes sent/sec, Bytes received/sec, and Current Bandwidth. Here you can make a decision if bandwidth consumption more, then try to make a request/repose size optimization by taking the following action for the same.
   • Use Ajax calls (lightweight requests)
   • Use JSON serialization instead of XML serialization (lightweight response).
   • Use client-side cache, if the same static data needs to be processed from the server multiple times (make sure not to cache secure or unnecessary data on the browser as it also has limited memory).
5. Process: It provides what are processes used by your application and what are their consumption like “% processor time”, Handle Count, and Thread Count taken by the SQL server, devenv (visual studio).
6. SQL Server resources: It provides all consumption at the SQL server level like Full scan/sec, Index search/sec, Transactions/sec, Lock Timeout/sec, and Number of Deadlock/sec. Here you can see what issues occur at heavy load on the database side (e.g. deadlock, timeout, table scan).

To see recourse usage details, double-click on the same and their respective details will be available on the right-hand side in the form of Graphs and Range values as in the above image.

In Graphs, the horizontal axis represents the time duration with a fraction of 10 seconds. I run the load test only for 3 minutes.

• Key Indicator graph: I selected the following resources from the Counter set.
• User Load: It increases every 10 sec, starts from 10, and reaches up to 180 (in 3 min).
• Transaction/sec: Transactions/sec increase with respect to user load.
• Bytes in all Heap: It increases with user load. But if it sharply rises then there can be memory leakage problems like resources do not release properly after consumption.
  
• Page Response Time: Click pages from Counter ⇒ Scenario ⇒ Web Performance Test ⇒ Pages Page Name, against which you want to check performance across load test. On the graph, you can see how it performs when the load increases. As the load increases, its performance goes down. There are multiple properties that you can check against any page like average page Time, Pages/Sec, etc.
  
• Controller and Agent: System resources like % CPU utilization, Memory utilization, Network I/O bytes sent and received per second, and .NET CLR threads can be checked here. Here CPU % usage exceeds the threshold value of 80%.
  

If you want to share your application results with peers then export test results to an Excel file. To export into Excel format, click the Create Excel Report icon under the Load Test result toolbar. On click, it will open an Excel file with the following window and provide the required input.

Note. Load Test Excel plug-in (as shown in the above image) might not load properly. To correct this, in Microsoft Excel 2010 or later, follow these steps.

1. In the Office ribbon, choose File.
2. Choose Options and then choose Add-Ins.
3. In the drop-down list under Manage, choose COM Add-Ins, and then select Go.
4. Select the checkbox for Load Test Report Addin.

Once the Excel plug-in is loaded properly then perform the following steps.

1. Select Create Report
2. Select Trend
3. Provide Report Name
4. Select Load Test results that need to be provided in the report (if the same Load Test runs multiple times)
5. Select the Resource name against which the Counter value needs to be captured in the Excel report.

It will generate an Excel report that contains a table of contents as the first worksheet and the remaining counter set values as per the workbook as in the following,

If you compare the results with the previous article, where all the web pages are working fine (except one stud_entry.aspx) but under load (where max user load is only 180 users) all the page's performance goes down as you can see below.

Note. If you already write unit test cases for your application, then you can also run a Load test against the same. No need to create a Web Performance Test (UI steps recording). Please refer to this link for more details: Creating and Running a Load Test Containing Unit Tests.

Lastly, Load test results also depend upon your current machine configuration and several processes running over the same. So make sure whenever you run a Load test on your local machine keep all the unnecessary processes closed to get more corrective results.

After running the load test you can get what are the pages that need improvements. Now need to identify which area of code is taking more time to execute. You can use Stopwatch under System.Diagnostics.Stopwatch to get methods that take more execution time, as you can see below.

Stopwatch watch = new Stopwatch();
watch.Start();

// Call your code here

watch.Stop();

// You can capture execution time in logs to verify the results later
Console.WriteLine("Measured time: " + watch.Elapsed.TotalMilliseconds + " ms.");

For more details about Stopwatch, you can refer here.

By using all these steps, as a developer you have scope and area of code that requires manipulation from a performance point of view. No need to wait for any performance testing team to achieve these results, it will take only 15-20 minutes for any developer to check their application performance.

Let’s summarize both articles again step by step through the following diagram. Now you can evaluate your work better than others.

In my next article, I will explain, performance optimization checkpoints that need to be taken care of in the following phases

• Architect
• Design
• Implementation

References

• Creating and Running a Load Test Containing Web Performance Tests