Improving the Quality of Software using Testing and Fault Prediction

Software will always have bugs, and as software continues to become a more and more pervasive part of our lives, software bugs will continue to affect more people than ever. If we don’t engineer for quality, over time software quality decreases. Prof. Iftekhar Ahmed’s research interests are fundamentally driven by a desire to build less error prone systems, not only by identifying bugs through effective testing, but also by helping developers write better code from the start.

Ahmed explains, “Collecting and analyzing software engineering data across hundreds of thousands of software projects is easier than ever thanks to advances in cloud computing and the availability of Open Source software repositories. We should seize this great opportunity to test old assumptions, learn about factors that impact code quality, and reason about software ecosystems at large. Thus I focus on performing empirical studies on large scale real world data to identify problems and propose solutions.

“Broadly, my research can be categorized under two thrusts: (1) Identifying factors that are associated with faults in software. For example, will the code be more bug prone in the future if it was something that multiple developers worked on in parallel? (2) Improving code quality by making testing more efficient. For example, can we scale testing techniques such as mutation analysis for complex high impact systems like Linux kernel.”

Elaborating on his first thrust of research, Ahmed explains that although there has been much research on predicting failures, those predictions usually concentrate either on the technical or the social side of software development. However, software development is not an isolated activity, it requires coordination between individuals and technology.

Development activity traces left behind in the artifacts involved in the software development lifecycle such as code base, version control systems, issue trackers, and discussion forums allow us to understand these complex interactions. Therefore, to attain the best possible predicative capability, we need to analyze the complex interactions between socio-technical factors. With this goal in mind, Ahmed has been focusing on the impact of collaborative work and developers’ interaction in the software development process on software quality. Ahmed has also investigated the evolution of design quality and its impact on the bug-proneness of software. Through these research endeavors, he has identified various factors related to code, developer experience and interests, and development processes that affect the quality of software, in terms of bugs and design issues.

The second thrust of Ahmed’s research is in testing. According to Ahmed, “our confidence in software systems depends on our confidence in the exhaustiveness of our testing.” His testing related work has helped to improve the quality of the Linux Kernel by efficiently applying mutation testing and identifying bugs. This work has so far identified four bugs and the associated fixes have been incorporated in the mainline Linux distribution, which has more than one billion instances running across the world, from data centers to mobile devices. Ahmed has successfully developed novel techniques to answer open questions such as how to measure the effectiveness of testing, and has successfully developed novel techniques to improve the efficiency of using model checkers in real-world settings. Ahmed strongly believes that software engineering research is most useful when it has real world applicability and industry relevance. He states, “I have tried to make sure that my research has quantifiable benefits for working developers. My collaboration with developers at IBM in identifying bugs in the Linux Kernel primarily used mutation analysis, a technique that has been around for decades. However, this technique hasn’t been adopted by industry due to the computational requirements and large number of false positives associated with it. By successfully applying this technique in the Linux kernel together with an industry partner, we have opened up opportunities for wider industry applications.” As a part of his effort to help developers write better code by providing tools that are scalable, Ahmed, using mutation analysis, developed novel techniques to improve the efficiency of model checkers. His techniques aid developers in understanding “successful” verification results and determining when a test harness is flawed. In correcting said harness, the harness is mutated in order to determine if there exist any similar harnesses having an equal (or better) mutation kill rate. Most importantly, his techniques also determine how to modify the model checker, the harness, and mutants to automatically produce successful highcoverage executions covering mutated code.

Ahmed believes that due to the everincreasing presence of cyber-physical systems (e.g., self-driving cars, Internet of Things) and the growing complexity of software, quality of software is more important than ever. Reliable software is essential and has compelling safety implications for both the users and the people surrounded by such cyber-physical systems. Using traditional testing methods for these systems is expensive, time consuming or, in some cases, infeasible. This is due to the systems’ continuous interactions with the environment and the deep intertwining of software with hardware. Ahmed plans to continue developing tools and techniques for identifying faults in software running in cyber-physical systems through testing and analysis to help developers build more reliable software.

To learn more about Prof. Ahmed and his research, visit his website: https://www.ics.uci.edu/~iftekha/