Query Expansion Based on Crowd Knowledge for Code Search

📝 Abstract

As code search is a frequent developer activity in software development practices, improving the performance of code search is a critical task. In the text retrieval based search techniques employed in the code search, the term mismatch problem is a critical language issue for retrieval effectiveness. By reformulating the queries, query expansion provides effective ways to solve the term mismatch problem. In this paper, we propose Query Expansion based on Crowd Knowledge (QECK), a novel technique to improve the performance of code search algorithms. QECK identifies software-specific expansion words from the high quality pseudo relevance feedback question and answer pairs on Stack Overflow to automatically generate the expansion queries. Furthermore, we incorporate QECK in the classic Rocchio’s model, and propose QECK based code search method QECKRocchio. We conduct three experiments to evaluate our QECK technique and investigate QECKRocchio in a large-scale corpus containing real-world code snippets and a question and answer pair collection. The results show that QECK improves the performance of three code search algorithms by up to 64 percent in Precision, and 35 percent in NDCG. Meanwhile, compared with the state-of-the-art query expansion method, the improvement of QECK Rocchio is 22 percent in Precision, and 16 percent in NDCG.

💡 Analysis

As code search is a frequent developer activity in software development practices, improving the performance of code search is a critical task. In the text retrieval based search techniques employed in the code search, the term mismatch problem is a critical language issue for retrieval effectiveness. By reformulating the queries, query expansion provides effective ways to solve the term mismatch problem. In this paper, we propose Query Expansion based on Crowd Knowledge (QECK), a novel technique to improve the performance of code search algorithms. QECK identifies software-specific expansion words from the high quality pseudo relevance feedback question and answer pairs on Stack Overflow to automatically generate the expansion queries. Furthermore, we incorporate QECK in the classic Rocchio’s model, and propose QECK based code search method QECKRocchio. We conduct three experiments to evaluate our QECK technique and investigate QECKRocchio in a large-scale corpus containing real-world code snippets and a question and answer pair collection. The results show that QECK improves the performance of three code search algorithms by up to 64 percent in Precision, and 35 percent in NDCG. Meanwhile, compared with the state-of-the-art query expansion method, the improvement of QECK Rocchio is 22 percent in Precision, and 16 percent in NDCG.

📄 Content

PAGE 1

Query Expansion Based on Crowd Knowledge for Code Search Liming Nie, He Jiang*, Zhilei Ren, Zeyi Sun, Xiaochen Li Abstract—As code search is a frequent developer activity in software development practices, improving the performance of code search is a critical task. In the text retrieval based search techniques employed in the code search, the term mismatch problem is a critical language issue for retrieval effectiveness. By reformulating the queries, query expansion provides effective ways to solve the term mismatch problem. In this paper, we propose Query Expansion based on Crowd Knowledge (QECK), a novel technique to improve the performance of code search algorithms. QECK identifies software-specific expansion words from the high quality pseudo relevance feedback question and answer pairs on Stack Overflow to automatically generate the expansion queries. Furthermore, we incorporate QECK in the classic Rocchio’s model, and propose QECK based code search method QECKRocchio. We conduct three experiments to evaluate our QECK technique and investigate QECKRocchio in a large- scale corpus containing real-world code snippets and a question and answer pair collection. The results show that QECK improves the performance of three code search algorithms by up to 64% in Precision, and 35% in NDCG. Meanwhile, compared with the state-of-the-art query expansion method, the improvement of QECKRocchio is 22% in Precision, and 16% in NDCG.
Index Terms—Code search, crowd knowledge, query expansion, information retrieval, question & answer pair.
——————————  —————————— 1 INTRODUCTION ODE search is a frequent developer activity in software development practices, which has been a part of software development for decades [45]. As repositories containing billions lines of code become available [1], [3], [6], [41], the search mechanisms have evolved to provide better recommendation for given queries. On Google Code Search, a developer composes 12 search queries per weekday on average [39]. Meanwhile, developers search for sample codes more than anything else, 34% queries are conducted to find sample codes, and almost a third of searches are incrementally performed through query reformulation [39].
The performance of text retrieval based search techniques used in code search strongly depends on the text contained in queries and the code snippets (a method is viewed as a code snippet [21]). The term mismatch problem, also known as the vocabulary problem [13], is a critical language issue for retrieval effectiveness, as the queries given by users and the code snippets do often not use the same words [10]. Meanwhile, the length of queries is usually short. Sadowski et al. report that the average number of words per query is 1.85 for the queries proposed to Google search for code [39]. Obviously, it is not an easy task to formulate a good query, which depends greatly on the experience of the developer and his/her knowledge of the software system [35]. To solve the vocabulary problem, the query expansion methods provide some effective ways by reformulating the queries [10], [34]. In recent years, some query expansion based code search approaches are presented. For example, Wang et al. [56] incorporate users’ opinions on the feedback code snippets returned by a code search engine to refine result lists. Hill et al. [38] suggest alternative query words by calculating the frequencies of co- occurring words with the words in the queries. Lu et al. [28] propose a query expansion method denoted as PWordNet by leveraging the Part-Of-Speech (POS) of each word in queries and WordNet [29] to expand queries. Lemos et al. [24] automatically expand test cases based on WordNet and a code-related thesaurus.
In this paper, we propose Query Expansion based on Crowd Knowledge (QECK) to improve the performance of code search. Specifically, QECK retrieves relevant Question & Answer (Q&A) pairs in a collection extracted from Stack Overflow as the
Pseudo Relevance Feedback (PRF) documents for a given free-form query, identifies the software-specific words from these documents, and generates an expansion query by adding words to the original query. The advantages of QECK are three fold. First, it automatically generates expansion queries without human intervention, as QECK employs PRF to automatically generate expansion queries. Second, it generates high quality PRF Q&A pairs by considering textual similarity and the quality of both questions and answers. Third, it identifies software-specific words from Q&A pairs by TF-IDF weighting function.
The underlying idea behind QECK is utilizing the software-specific words contained in Q&A pairs to xxxx-xxxx/0x/ $xx.00 © 200x IEEE Published by the IEEE Computer Society C ————————————————  L. Nie, H. Jiang, Z. Ren, Z. Sun, and X. Li are with School of Software, Dalian University of Technology, Dalian, China. E

This content is AI-processed based on ArXiv data.