Determining the number of tests required to test a method
- Interesting essay on something called pair-wise testing
&ObjectId=8581> and orthagonal arrays...
Something that had puzzled me recently is determining the number of
tests needed to validate a method when there are many arguments or many
different values to the arguments. The cross-product of all the
possible arguments and data can become very large very quickly. For
example, let's say the method1() could take arguments like this:
Arg1.... 8 possible options.
Arg2..... 12 possible options
Arg3.........yes or no...2 options
Arg4...... 4 possible options
So at first glance, to test this completely with all permutations of
data, you might think of the cross-product of the arguments ....8 x 12
x 2 x 4 .... = 768. Wow....that is a lot of tests to write!
So ...yes, there is code coverage for answering questions of
completeness ....but I just found maybe another approach as well.
The .pdf below the article in the link describes an approach for
determining the number of tests required for testing an application
that is apparently almost as complete and thorough as the
cross-product, but much more manageable from a testing perspective. The
article discusses 3 approaches. I actually tried the 2nd approach on
the example above...using a free download discussed in the essay called
Rather than 768 tests in the cross-product of all the arguments ...the
program defined a mere 58 tests using specific combinations of data .
Or so says this this theory.
[Non-text portions of this message have been removed]