There are plenty of tools available in the market which supports performance testing of the web application. Many of them are licensed tools and few of them are open source as well. Few are supporting only web applications few can handle only web APIs. However, VSTS stands tall among them with having support from both of them.
Unlike the Load UI and Jmeter, it has lots of flexibility because of the support of C# as a backend scripting language. Of course, it has that UI support to enable record and playback functionality to novice users. Let’s have a high-level overview of different processes that can be performed by VSTS to achieve the complete goal of performance testing of a web application.
Just for easiness of understanding, we will take an example of a sample e-commerce website (http://demo.nopcommerce.com/) throughout the series of this article.
Performance Testing Requirement
The first phase of any performance testing plan is to identify the actual performance testing requirement and mold it into the format which can be better understood by the tool and the technical person going to use that tool. So here also let’s start identifying the requirement of performance testing of any standard e-commerce web application. Here, we will be using our sample application as a reference.
Nopcommerce is the e-commerce web application that provides users an easy interface to search for various products, login, add to cart and pay for the product, and all basic functionality specific for the e-commerce web application.
Performance testing is required to get the following information related to the application.
-
To validate application behavior from the end user’s perspective when a high no of concurrent users are accessing the application. Parameters to be captured for analysis will be……
- Page response time
- Transaction response time
- To find slowest responsive pages
-
To validate the application server, DB server’s behavior when high no of concurrent users accessing the application. Parameters to be captured for analysis will be……
- Memory usage (RAM)
- CPU consumption
- Disk space utilization
-
For capacity planning. My current hardware is supporting n number of concurrent users. Now the firm is anticipating a potential increase in user load due to some business requirements. So performance testing is required for capacity planning. Basically to identify…..
- X size of hardware is required to support Y no of concurrent users
Load Test and Workload mix creation with VSTS:
Nopcommerce is the web application and for performance testing of the application Web performance test functionality of the VSTS can be used. Along with the accurate recording of the requests VSTS also provides well-formed UI for generating the load test on top of that. It also provides various options for creating a real-life scenario by creating different workload mix.
For this performance test experiment, we have considered the below information for creating a workload mix.
-
Login -> Search Product(s)-> Log Out
- P % of total user will perform this action
-
Open Application -> Search Products ( As guest users )
- Q % of total user will perform this action
-
Login -> Search Product(s)->Add To Cart -> Check out
- R % of total user will perform this action
-
Search Product(s) -> Add To Cart -> Check out
- S % of total user will perform this action
* Numbers P, Q, R, and S will be derived from the historical usage of the application / or from the standard usage of similar existing application
To test the above-mentioned scenario first we need to create a web test for each of the scenarios. Once these web tests are ready you can add them to your load test. There are various options that can be used for the generating most realistic tests.
Multiple options for browser mix and network mix can be used along with various load patterns for creating various permutation combinations of load using the same set of tests.
The sample load test is generated using the above information. In the VSTS load test would look like ….
Performance Test Execution:
Based on the requirement of the performance test method of execution can be varied. Generally, instead of standalone execution of performance tests, it gets executed in some pattern to get some statistical data for analysis. This is called performance test experiments.
Here, we will execute the same test for a defined time duration with different concurrent user loads (10,50,100,175,250 and 500). We will get different statistical data for analysis out of all these execution cycles.
For example one of the data set for web applications is page response time…
The above table is extracted from the standard summary report generated by VSTS during one of the execution. This data will be captured for each execution cycle (with 10, 50, 100, 175, 250, and 500 users) for the further result analysis.
Performance Test Result Analysis and Reporting:
Performance test result analysis is a very generic term. The process for analyzing results will be varied based on the experiment we are carrying out. Successful completion of this task is really depending upon the accuracy of data collected during the earlier phase of performance test execution.
Here for our experiment, we have selected the data of page response time during the performance test execution. Based on data captured during each execution we can generate various graphs.
What we need to identify is that with an increase in the number of concurrent user page response time can be increased but it should be increased in a smooth way. If we notice any spike in this chart, then there is an issue that needs to be investigated.
Users Vs Time Chart
This graph shows the condition of page response time along with an increase in the number of concurrent users.
The current situation in the graph is the ideal condition where response time is not being hampered with increased load.
If the line in the graph inclines with an increase in the number of concurrent users than we need to identify the threshold value for acceptable response time for page and based on that value we will have information of a supported number of concurrent users with current set up of hardware.
Users Vs Failures Chart
This graph shows the number of failures with an increase in concurrent user load. Failures increases with an increase in a number of concurrent user load. However, the graph should be a straight line which is shown in the below chart.
If you see a spike here than that is the issue and that needs to be investigated with other experiments.
Users Vs Transactions
This graph shows the number of transactions with an increase in concurrent user load. Ideally, the transaction should increase in parallel with user load which is shown in the below graph.
If you see this as a straight line instead of inclining then there is an issue. Transactions are not increasing with users and that issue needs to be investigated with other experiments
*all these charts are generated with custom written code with excel based on data exported from VSTS summary report.
Conclusion:
So far we have seen a very high-level overview of web performance testing with VSTS. Now next article in the series would be taking you into details of each and every activity mentioned in this article so that you will be able to generate your first web performance test using your VSTS.
So basically this was all about what we can do with VSTS with respect to web performance testing and the next set of articles would be all about how we can use VSTS to do all these activities described above.
⇓ Subscribe Us ⇓
If you are not regular reader of this website then highly recommends you to Sign up for our free email newsletter!! Sign up just providing your email address below:
Happy Testing!!!