Mutation testing is used in unit testing. The sole purpose of this type of testing is to check how efficient the test cases are? The mutation itself means to change or modify, so in mutation testing, testers change syntax in source code to verify if the test cases are able to find out bugs or not. This type of testing is generally done to check the efficiency and effectiveness of the test cases. This is also called fault-based testing as it includes creating faults in the program.
Mutation testing process:
- Create versions called mutants and introduce faults into the source code of the program. There should be a single fault in every version and goal is to fail the mutant version to show the effectiveness of test cases.
- Apply the test cases to both the programs, original and mutant.
- Results of the original and mutant program should be compared.
- Mutant is killed if the result of both the program is the same.
- Mutant is alive if the result of both programs is different.
Example:
A mutation is a single change in the syntax made to program statement. Mutant program is different from the original program by one mutation.
Original program:
if (Red > Green)
print "Red"
else
print "Green"
Mutant Program:
if (Red < Green)
print "Red"
else
print "Green"
Mutation testing is usually automated as it is a very complicated and time-consuming task to be done manually. Some tools for mutation are jester, Judy, Ninja Turtles, Mutagenesis.
Types of Mutation Testing:
- Value mutations: In value mutations, tester changes the value either to a much larger value or to a much smaller value to detect errors in the program.
- Decision mutations: In Decision mutations, arithmetic, relational and logical operators are changed to detect the errors in the program.
- Statements mutations: In Statement mutations, statements are changed either by deleting the statements or by duplicating the lines to detect errors in the program.
Though, there are few mutants which need to be avoided because all the mutants are not useful.
Mutants with the incorrect syntax:
Example:
Original Program:
Read numbers
if numbers < 10
Type = Single Digit()
End if
Mutant Program:
Read numbers
if numbers %% 10
Type = Single Digit()
End if
This will give the compile error. Mutants should have the correct syntax.
Equivalent mutants:
Example:
Read numbers
if numbers < 10 & numbers < 10
Type = Single Digit()
End if
This will give the same result as the original program. As numbers<10 & numbers <10 means number<10, which is equivalent to the original program.
Trivial Mutants:
Example:
Read numbers
if numbers = 10
Remove assignment statement
End if
In any case, this mutant will not do any programming. Any data set can kill this mutant.
If at the end of the test, some mutants are alive then it means it is an invalid mutant.
Advantages of Mutation Testing:
- Customers get the most stable and reliable system after this testing.
- Mutation testing has the ability to detect all faults in the source code
- High coverage of the source program is attained.
- Program mutants are tested thoroughly.
- Quality of software program is improved.
- Loopholes in test data can be identified.
Disadvantages of Mutation testing:
- Complex mutations are difficult to implement.
- Mutation testing is time-consuming and expensive.
- Testers need to have the programming knowledge to do mutation testing.
- Mutation testing is not applicable for black-box testing as involves a lot of source code changes
- Large mutant programs need to be tested against the original test suite.
- Automation is necessary for mutation testing as it is very time-consuming.
Mutation score
Mutation score is defined as the parentage of dead mutants with total mutants.
Mutation score = (Dead mutants/Total mutants)*100
If the mutation score is 100% then test cases are mutation adequate.
Over to you:
- Mutation testing is time-consuming, it requires automation.
- Mutation testing is the most comprehensive technique to test any program.
- Mutation testing is a unit testing method.
- It uses fault injection to generate mutants.
- Some tools used for automation of mutation testing are: Jester, pester or MuClipse.