In Search of Better Tools

With all the advances they have made in designing new software systems for all kinds of different domains, purposes, and users, one would think that software engineers would pay as much attention to improving the software they use themselves. After all, why would they not want to work with the best possible development tools they could imagine?

To a degree, software engineers do exactly that: they reflect upon their experiences with their current tool set and, when they encounter problems or identify opportunities for further improvement, they will experiment with creating new or enhanced functionality. It is not an accident that tools such as Jenkins, Heroku, and Cloud9 were all born out of developers who recognized ways in which they could make their own lives easier.

At the same time, it sometimes takes an outsider looking in to see things software engineers themselves might not or to think of out-of-the-box opportunities that would help them work in a more effective and efficient manner. This is the specialty of ISR Professor and Informatics Department Chair André van der Hoek and his research group. They love thinking about, designing, and experimenting with new software development tools.

For van der Hoek, it all started during his Ph.D. work at the University of Colorado Boulder. “I have always had an interest in connecting to ‘the real world’,” van der Hoek says, “not wanting my research to just solve some artificial problem, but tackle issues faced by Jane and Joe developer in their day-to-day work.” At the time, van der Hoek dove into configuration management, developing distributed versioning systems that explored new ways of managing conflicts stemming from parallel software development. He also began work in software architecture, studying design environments for software product line architectures. In both cases, van der Hoek spent countless hours programming the new tools he envisioned and putting them to the test—sometimes by actually using the experimental tools across various real-world settings and other times by recreating industrial case studies in his tools to assess the relative strengths and weaknesses of his approaches.

Since joining UC Irvine, and ISR, in January 2000, van der Hoek has continued to focus on creating new collaboration and software design tools, but also expanded his research agenda in the process. “I like it when the students bring their own concerns and possible research ideas to the table. After all, they are software developers too, growing up working with the latest development tools and understanding the limitations they innately have.” An example is the work of van der Hoek’s graduate student Lee Martie, who has been designing new code search tools as part of his dissertation research. At the time Martie started his Ph.D. work, it had become abundantly clear that software developers programmed very differently than in the years before: they now routinely search for code examples on the internet, so much so that they can spend 20% or more of their time searching instead of actually writing code. With such an important and growing role for search, then, an opportunity existed to rethink the functionality of search tools.

Martie’s observation was that code search tends to be an iterative process, but that current search engines—whether Google as the default or a specialized code search engine such as SearchCode or Krugle—do little to help the programmer in iterating. That is, when the first query does not lead to the results the programmer would like to see, the programmer is left to their own devices in identifying a new set of keywords to be issued as the next search. “While we are used to searching this way,” van der Hoek says, “it need not be. Indeed, some search engines provide keyword recommendations that can be added to a query with a single click. But can we do more for the programmer?”

Martie answered this challenge by designing and implementing two experimental code search engines: CodeExchange and CodeLikeThis . In CodeExchange, the search tool is enriched with features through which it is possible to take aspects of the search results (e.g., the length of a code result, a method call inside a code result, a keyword common across multiple results) and add it to the query (e.g., shorter than this result, including this method call, including this common keyword) with one click.

CodeLikeThis, however, takes a more radical approach: once a first set of keywords returns a first set of results, any subsequent query is issued simply by selecting a result and issuing a directive: more like this, somewhat like this, or less like this. CodeLikeThis, then, delivers results that are similar to the chosen result, somewhat different, or divergent. In evaluating both CodeExchange and CodeLikeThis, Martie found that each can hold their own against Google (the gold standard), but that the two solutions are complementary: one works better when developers have specific results in mind, the other when their search is more openended. As such, a next step is to attempt to put the unique features of each together in a single, still easy to use code search engine.

Van der Hoek notes the critical role ISR alumni and corporate sponsors have played in the research. “We asked alumni and our friends in industry to take a look at our early prototypes, and were provided with a wealth of brutally honest feedback that we were able to parley into the current incarnation of the search engines,” he says. “Being part of a community like ISR’s is just fabulous for the research and a reason why I joined UCI in the first place.”

Martie, for his part, will soon join other alumni from van der Hoek’s research group—and indeed ISR—who have gone on to bring their expertise and ideas to the real world. Defending his dissertation in August, he has been recruited by Microsoft to join the Tools for Software Engineers group at Microsoft, loosely affiliated with Microsoft Research, thereby keeping with the tradition of engaging with research, but solving problems real developers experience every day.

For more information, contact Prof. van der Hoek or visit his research group website.