Starting Performance Testing: What Metrics To Use?

Today is an era of web (www: World Wide Web) where everything is available is now available on web.  Today every organization uses web not only to provide services to their client but also running their own business functions online.  These days web is in very high demand due to its hassle free availability. The users are really concerned about responsive time of the websites. If any page takes more than 3 sec to load, it is regarded as slow and leads to a large impact on business. In this article lets understand which Performance Testing Metrics to use.

Performance Testing is testing the quality assurance of the software. This testing is done to ensure that the software will perform well under the expected load. As stated above, the response time of a website matters a lot. Apart from features and functionalities the response time is a major concern. The testers find out bugs to eliminate the bottlenecks of the performance of the application.

Performance Testing is done to ensure that the software will perform as expected in the production environment. The outcome of the this testing helps the testers in the finding the gaps between the expected and actual result. To achieve maximum success in performance testing, some criteria should be defined which would measure and compare the actual output of performance testing. Different metrics are used to measure the output of performance testing. Information obtained from these metrics helps the companies to increase their productivity levels, decrease the error rate and provide excellent quality service level to achieve their business goals.

In an application there are many performance keys generated during performing performance testing. It is not possible to thoroughly study each of these perf testing counters and work on them. A performance tester should have a strong knowledge of the weak areas of the application under test and the right way to set these performance parameters. Now we will understand the performance metrics and the potential weak areas of web applications and the performance parameters of these areas which need to be kept under an eye during performance testing.

Performance Testing Metrics:

A metric is a unit of measurement which calculates the result. Performance metrics are used to calculate the performance parameters and find out the areas of the application which are weak and creating bottlenecks in the application.

Metrics are used to track the project progress. It is used to make a baseline for all the tests. Using metrics we can measure the main cause of the problem. Using metrics we can compare the result of the different tests and find out the impact of any changes made to the application. It helps in improvement of quality of product. It provides clear output of the activity and find out the areas which require attention.

There are numerous performance metrics generated during performance testing but we cannot evaluate all of the metrics. We select a few metrics on the basis of the following characteristics.

1. Performance metrics which are important for the application under test.
2. Metrics which are easy to calculate.
3. Performance metrics whose complete data can be found out and calculated.

Performance Testing Requirement: Gathering the metrics

First step is to understand the metrics required to gathering the complete set of performance requirements. It is the important step to understand the requirements properly and clearly. The requirements can be gathering by contacting different clients or it can be done using server log activity. We can consider some of the below server side metrics to achieve the performance goals.

• Average page Size
• Average Hit Size
• Total hits per week
• Total user sessions per week
• Total page views per week.
• Page request distribution

Total user sessions per weak: A page view is regarded as a request on server with all the objects.  It tells information about weekly traffic on application. It is common indicator of website traffic, it tells about page views or bytes transferred.

Total hits per week: A request received by web server from the client is called hit. A web server receives many hits for a web page. Web pages are made of files and images. Number of hits to web server for a web page is equal to the number of resources it contains.

Average Hit Size: Average hit size is measured in Kb. It is average amount of data transferred from a web server when there is a particular hit.

Client Side Gathering Requirements Metrics: Client is user who interacts with the application to complete their requirements. The response of the applications and usage of the application is different for all clients as it depends on client-side variables. Thus all the user requirements should be considered for design an effective performance test. Client-side metrics like Connection Speed, location, Interaction Speed and latency tolerance should be considered while designing a performance test.

User geographic location: This factor majorly affects the user experience of the application. Response time is directly proportional to the distance between the client and the server.

What Includes In Performance Testing Metrics:

Let us know discuss the performance metrics to be evaluated in performance tests. The performance metrics include:

Monitoring Transactions: Transactions are executed for users to test its performance. It is important to know the status and response time of each transaction happening through the application. The average response time of transactions is noticed for every single transaction to evaluate the online user’s impact on the system. The average response time should not be more than 3 seconds.

HTTPS response per second: The HTTP response graph tells us about all the HTTP status codes generated every seconds during the performance testing. It tells us about the status on web server on every second of performance Testing.

Connections Per Second: This performance test metric tells us about the number of TCP/IP connections opened and shut every second during performance Testing. The connections per second should always be lesser than the number of hit per seconds as it is very costly to open and shut TCP/IP connections.

CPU Usage: The CPU usage metric provides information on CPU utilization. Ideally the CPU utilization should not be more than 70 %.

Memory Usage and Disk Space Usage: This metric gives information on free disk space, memory used during execution of a scenario. Disk Space provides information of disk space used.

Pass failed transactions: It is important to have idea about how many transactions passed and how many transaction failed during performance Testing

Web Server Performance Metrics

Web Server metrics provide useful information on web server performance. It gives information on the usage of resources on web servers during the execution of performance tests. There are numerous web server metrics and these parameters depend on the webserver we are using. Common web server metrics like Apache, Microsoft IIS, Sun Java system, etc. The lists of performance testing parameters depend on the type of web server we are using. The web server performance metrics for Microsoft IIS are:

• Percent Disk Time
• Cache Hit length
• Request Queued
• Transactions per second.
• Bytes sent and received per second.
• Private bytes
• Error 404 (not found) bytes

Application Server Performance Metrics

Application server is the backbone of application in case of highly complex business applications. Some of the application servers are Web logic, Microsoft .Net, Apache Tom Cat, Web Sphere, J Boss. Most of the application activities are performed on the application server. The parameters for application server performance metrics are:

• Connection Time
• Connection Wait Time
• Total Threads
• Memory Use
• Total Memory
• Active Transactions
• Transactions suspended
• Rolled Back transactions
• Timeouts
• Servlet Errors

Network Configuration Metrics

Network is a vital part of any application. No application can perform well if its network is not designed properly. There are multiple segments in each application network and it is important to monitor each and every network segment delay time during performance testing. These days all the tools are configured with network monitoring feature during performance testing and to measure the impact of networking configuration on the performance of the application.

Some of the network configuration metrics are as follows:

• TCP Connections established
• TCP Connections reset
• TCP connections failures
• TCP segments per second
• TCP segment received by second

Client Side Performance Metrics

Since there is a need of better application performance and in any application the front end takes maximum of the time to load. Thus reducing the front end loading time of the application by 40% can increase the performance of the application. There are various performance Testing tools in market which tell us about the details of time taken to loading of images , JavaScript, CSS and HTML files. Some of the client side performance testing metrics is as follows:

• TCP connection time
• HTML resources load time
• CSS files load time
• Images load time
• Javascript file load time
• HTTP response time and HTTP response status.

Conclusion:

Before marketing your software product it is necessary to do Performance testing. It is most important to understand the complete performance requirements of product before start your testing. In this article we learned about what all performance testing metrics needs to be considered.

I Hope this article is useful to understand how to start with Performance Testing and what metrics to use.

Let us know your queries/comments.