In
computer science
Computer science is the study of computation, automation, and information. Computer science spans theoretical disciplines (such as algorithms, theory of computation, information theory, and automation) to practical disciplines (includi ...
, all-pairs testing or pairwise testing is a
combinatorial
Combinatorics is an area of mathematics primarily concerned with counting, both as a means and an end in obtaining results, and certain properties of finite structures. It is closely related to many other areas of mathematics and has many ap ...
method of
software testing
Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to apprecia ...
that, for ''each pair'' of input parameters to a system (typically, a
software
Software is a set of computer programs and associated software documentation, documentation and data (computing), data. This is in contrast to Computer hardware, hardware, from which the system is built and which actually performs the work.
...
algorithm
In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
), tests all possible discrete combinations of those parameters. Using carefully chosen
test vector In computer science and engineering, a test vector is a set of inputs provided to a system in order to test that system. In software development, test vectors are a methodology of software testing and software verification and validation.
Rationa ...
s, this can be done much faster than an exhaustive search of
all combinations of all parameters, by "parallelizing" the tests of parameter pairs.
Rationale
The most common bugs in a program are generally triggered by either a single input parameter or an interaction between pairs of parameters. Bugs involving interactions between three or more parameters are both progressively less common and also progressively more expensive to find---such testing has as its limit the testing of all possible inputs. Thus, a combinatorial technique for picking test cases like all-pairs testing is a useful cost-benefit compromise that enables a significant reduction in the number of test cases without drastically compromising functional coverage.
More rigorously, if we assume that a test case has
parameters given in a set
.
The range of the parameters are given by
.
Let's assume that
.
We note that the number of all possible test cases is a
. Imagining that the code deals with the conditions taking only two parameters at a time, might reduce the number of needed test cases.
To demonstrate, suppose there are X,Y,Z parameters.
We can use a
predicate
Predicate or predication may refer to:
* Predicate (grammar), in linguistics
* Predication (philosophy)
* several closely related uses in mathematics and formal logic:
**Predicate (mathematical logic)
**Propositional function
**Finitary relation, o ...
of the form
of order 3, which takes all 3 as input, or rather three different order 2 predicates of the form
.
can be written in an equivalent form of
where comma denotes any combination. If the code is written as conditions taking "pairs" of parameters,
then the set of choices of ranges
can be a
multiset
In mathematics, a multiset (or bag, or mset) is a modification of the concept of a set that, unlike a set, allows for multiple instances for each of its elements. The number of instances given for each element is called the multiplicity of that e ...
, because there can be multiple parameters having same number of choices.
is one of the maximum of the multiset
The number of pair-wise test cases on this test function would be:-
Therefore, if the
and
then the number of tests is typically O(''nm''), where ''n'' and ''m'' are the number of possibilities for each of the two parameters with the most choices, and it can be quite a lot less than the exhaustive
ยท
N-wise testing
N-wise testing can be considered the generalized form of pair-wise testing.
The idea is to apply
sorting to the set
so that
gets ordered too.
Let the sorted set be a
tuple :-
Now we can take the set
and call it the pairwise testing.
Generalizing further we can take the set
and call it the 3-wise testing.
Eventually, we can say
T-wise testing.
The N-wise testing then would just be, all possible combinations from the above formula.
Example
Consider the parameters shown in the table below.
'Enabled', 'Choice Type' and 'Category' have a choice range of 2, 3 and 4, respectively. An exhaustive test would involve 24 tests (2 x 3 x 4). Multiplying the two largest values (3 and 4) indicates that a pair-wise tests would involve 12 tests. The pairwise test cases, generated by Microsoft's "pict" tool, are shown below.
See also
*
Software testing
Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to apprecia ...
*
Orthogonal array testing
Orthogonal array testing is a black box testing technique that is a systematic, statistical way of software testing. It is used when the number of inputs to the system is relatively small, but too large to allow for exhaustive testing of every ...
Notes
External links
Pairwise testingPairwise and generalized t-way combinatorial testingPairwise Testing in the Real World: Practical Extensions to Test-Case Scenarios
{{DEFAULTSORT:All-Pairs Testing
Software testing
Combinatorics
Design of experiments