Structural Specification-Based Testing: Automated Support and Experimental Evaluation

TitleStructural Specification-Based Testing: Automated Support and Experimental Evaluation
Publication TypeConference Proceedings
Year of Publication1999
AuthorsChang, J., and D. J. Richardson
Conference Name7th ACM SIGSOFT Symposium on the Foundations of Software Engineering (SIGSoft '99)
Date Published6-10 Sept. 1999
PublisherACM Press
Conference LocationToulouse, France
KeywordsADLscope, Assertion Definition Language, Automated support, Automated testing tool, Coverage conditions, Experimental evaluation, formal specification, Program testing, Specification-based test criteria, Structural specification-based testing, Test coverage measurement, Test selection

In this paper, we describe a testing technique, called structural specification-based testing (SST), which utilizes the formal specification of a program unit an the basis for test selection and test coverage measurement. We also describe an automated testing tool, called ADLscope, which supports SST for program units specified in Sun Microsystems' Assertion Definition Language (ADL). ADLscope automatically generates coverage conditions from a program's ADL specification. While the program is tested, ADLscope determines which of these conditions are covered by the tests. An uncovered condition exhibits aspects of the specification inadequately exercised during testing. The tester uses this information to develop new test data to exercise the uncovered conditions. We provide an overview of SST's specification-based test criteria and describe the design and implementation of ADLscope. Specification-based testing is guided by a specification, whereby the testing activity is directly related to what a component under test is supposed to do, rather than what it actually does. Specification-based testing is a significant advance in testing, because it is often more straightforward to accomplish and it can reveal failures that are often missed by traditional code-based testing techniques. As an initial evaluation of the capabilities of specification-based testing, we conducted an experiment to measure defect detection capabilities, code coverage and usability of SST/ADLscope; we report here on the results.

Short TitleStructural Specification-based Testing