Reliability testing is one of the keys to better software quality. In today’s class let’s first look at the definition of reliability testing according to the ANSI standards and then understand how and why it is performed.
According to ANSI, Software Reliability is defined as:
“The probability of failure-free software operation for a specified period of time in a specified environment. Software Reliability is not a direct function of time. Electronic and mechanical parts may become “old” and wear out with time and usage, but the software will not rust or wear-out during its life cycle. The software will not change over time unless intentionally changed or upgraded.”
From the above definition it is clear that it is a part of both functional and non-functional testing (like stress, load, network and security testing). For example, a mobile software. If the lifecycle of the software is 5 years then its reliability would be determined by analyzing the quality of the software, which is not affected even if the mobile hardware is broken or some part of it is now not functional. But software used in it should work in the same way as it was working on its first year of release till it completes its lifecycle.
There are three categories in which we can divide the study of reliability testing, namely Modeling, Measurement, and Improvement.
Measurement of reliability testing is done in terms of mean time between failures (MTBF). There are two units known as the mean time to failure (MTTF) and mean time to repair (MTTR). The time between two consecutive failures is called as MTTF whereas time taken to fix these failures is known as MTTR. Reliability is measured as a fractional number between 0 and 1. In simple mathematics, less number or error/bugs or failure more will be the reliability of the software.
Types of Reliability Testing:
It includes three types of testing.
1. Feature test: It is done in the following three steps.
- Each function in the software should be executed at least once.
- Interaction between two or more functions should be reduced.
- Each function should undergo proper execution or functional test.
2. Regression test: Whenever any new functionality is added or old functionalities are removed in an application, it undergoes a regression test to make sure with introduction of new functionality no new bugs are introduced in an application. Regression testing is always needed whenever there is version change for software build.
3. Load test: Load test is done to test whether the application is supporting the required load without getting Breakdown. In order to find the breakpoint of an application, the load is gradually increased until the application gets hung, breakdown, unavailable, etc. Just at this threshold the application marked as reliable and anything beyond this point will result in application failure or application unreliability mode. For example, a website is marked to support 1000 users at any time. These 1000 users at a time are the threshold where any number of users below 100, the application is considered as reliable and anything above this as unreliable and can break down.
4. Objectives of Reliability Testing: Restrictions like software behavior under given conditions, feasible objectives and time constraints are applicable before defining objectives of the reliability testing.
5. Software application undergoes reliability testing to identify its mean life for which it can be put to use safely.
- To evaluate the number of failures occurring at a given period of time.
- To observe perpetual structure of repeating failures.
- To identify main cause of software failure.
- To conduct the performance testing of various modules of software application after applying the bug fix or preventive actions.
Importance of Reliability Testing:
As we now software applications are in use in almost every sector of manufacturing, healthcare, defense, government, telecommunication, etc. Therefore the question is that how reliable the software application is? This question is very important because the reliable software is of great use and the organization working on it can feel the confidence in their business systems. To verify the software reliability by reliability testing below measures are required:
- Test cases should be designed covering all the required functionality of the software application and these test cases should be executed for sufficient amount of time to get a rough estimate of how long this software can execute fine without failure. Test should be executed for long duration to observe environment level failures like out of memory error, buffer overflows, database deadlocks, etc. which require time to show up.
- The test case execution sequence to run the overall functionality of the software application should match the real production operational plan of the application. More often a function is executed; more is the percentage of test cases that should allocate that function or module of the software application.
Conclusion: For an organization to conduct reliability testing is relatively costlier than the normal functional testing as it requires more tweak and turns to identify the application breakpoint and lots of technical skills and experience on the testing resource side.
Have you worked on Reliability Testing? Let us know your experience or queries in comments below.
⇓ Subscribe Us ⇓
If you are not regular reader of this website then highly recommends you to Sign up for our free email newsletter!! Sign up just providing your email address below:
Happy Testing!!!