Fall/Winter 2019 – Technical Debt

An increasingly important concern for many software organizations is the accrual of technical debt in their software systems. Technical debt is a concept in software development that reflects the implied cost of additional rework caused by choosing an easy (limited) solution now instead of using a better approach that would take longer. Technical debt may manifest itself in any software engineering artifact, such as architectural models, source code, and tests. By accruing technical debt, a software system becomes increasingly difficult and costly to develop and maintain over time.

Technical debt is a symptom of refusing to follow certain basic software engineering principles. In fact, historically, a significant body of software engineering research has aimed to develop solutions to prevent the occurrence of technical debt. Take for example research on architecture-based software development, advocated by a number of prominent scholars in our field. Architecture-based development advocates spending time, particularly in the early stages of software development, to properly identify and evaluate alternative architectural choices for a software project. In this paradigm, software architecture further serves as a centerpiece that guides all other software development activities, including implementation, validation and verification, and maintenance.

Despite its promise, many prominent software organizations do not follow such an approach to software development. At these organizations, modeling and architecting are considered harmful, as they are deemed to slow down the developers, and do not help achieve what really matters: working code. The irony of this is that the same organizations that refuse to document the architecture of their systems soon find themselves in a situation in which they have no idea of what is the structure of their systems and why they behave in abnormal ways to smallest changes in code. Subsequently, significant effort, arguably more than would have been required to carefully document the architecture from the get-go, is spent on recovering the architecture of the software to determine its properties and identify ways in which it has decayed. In the end, however, considering the developer turnover and complexity of modern systems, most such efforts at recovering the architecture tend to fail. It is indeed extremely difficult to determine the intuition behind the architectural decisions in a system from its lines of code. The short-term gain of working code without proper consideration and documentation of architectural choices leads to unwieldy software that is exceptionally difficult to maintain.

This is also an area of research that does not receive the attention it deserves. Although there are a few specialized events targeting issues of technical debt in software (notably TechDebt, a conference collocated with ICSE), there are not many publications on this topic at the premier software engineering conferences. Yet practitioners have significant interest in effective solutions to understand, measure, and eliminate technical debt in their systems. In my own experiences, almost every large software organization that I visited in the past few years expressed interest in one form or another for solutions to tackle the challenges posed by technical debt. While the research community has many good preventive measures in the form of principles and methodological approaches to development, there are actually not many effective reactive measures to mitigate technical debt once it affects a system. Tools for calculating code quality metrics, for instance, provide a good starting point, but do not capture the more complex properties of the system, e.g., architectural degradation. This appears to be yet another industry-relevant, but grossly neglected topic, in the top-tier software engineering conferences, possibly explaining why industry participation at these events have been declining. Perhaps the research community is disheartened by the fact that our preventive solutions to this problem are not adopted by industry, and thus do not find value in pursuing research in this space any further. Or, maybe it is one of those areas that is exceptionally difficult to publish, since proper evaluation would require complex experiments with industrial collaborators over long periods of time.

Whatever the reasons may be, advancements in this area will require close collaborations between industry and academia. In fact, in addition to myself at ISR, we have a number of other faculty members, including Profs. Joshua Garcia, Iftekhar Ahmed, and Nenad Medvidović, that have ongoing collaborations with companies such as Google and SAP aimed at studying the emergence of technical debt and proper eradication methods. Stay tuned for updates on these collaborations in future editions of the ISR Connector.

For more about ISR Director Sam Malek, visit his website.