Mats Heimdahl

For decades, researchers have explored the topic of software testing. Early work in this area covered the formal foundations of program testing. By exploring the foundations of testing largely separate from any specific method of testing, these researchers provided a general discussion of the testing process, including the goals, the underlying problems, and the limitations of testing. Unfortunately, a common, rigorous foundation has not been widely adopted in empirical software testing research, making it difficult to generalize and compare empirical research. Furthermore, much of current testing research has lost track of the goal of testing (finding consequential faults) and has focused on furthering specific aspect of the testing problem without considering the impact on the testing process as a whole.

In this talk I will revisit this foundational work and discuss the crucial (and often overlooked) interrelationship between the program under test, the test data used in its testing, and the test oracle determining whether or not a test passes.  I will, with examples from our empirical work, discuss a new coverage criterion, Observable MC/DC, and a new technique to select which variables to observe when testing a program; both techniques attempting to leverage the interrelationship between programs, test data, and test oracles.