[2503.20934] Leveraging LLMs, IDEs, and Semantic Embeddings for Automated Move Method Refactoring

View PDF of the article Leveraging LLMs, IDEs, and Semantic Embedding for Automated Transfer Method Refactoring, by Abhiram Bilur and 14 other authors

View PDF HTML (beta)

a summary:MOVEMETHOD is the hallmark refactoring. Although there are a large number of search tools that recommend transportation routes and where, these recommendations do not correspond to how expert developers perform MOVEMETHOD. Due to the extensive training of large language models and their dependence on the nature of the code, they must expertly recommend which methods are misplaced in a given class and which classes are a better host. Our 2016 LLM Recommendations Formative Study revealed that LLM holders provide expert suggestions, yet they are unreliable: up to 80% of suggestions are hallucinations. We present the first fully-supported LLM plugin for MOVEMETHOD refactoring that automates its entire lifecycle end-to-end, from recommendation to implementation. We have designed new solutions that automatically filter out LLM hallucinations using static analysis from IDEs and a new workflow that requires LLMs to be self-consistent and critical and rank restructuring suggestions. Since MOVEMETHOD refactoring requires global project-level thinking, we solve the limited context size of LLMs by using Refactoring-Aware Recall Augmented Generation (RAG). Our approach, MM-assist, synergistically combines the strengths of LLM, IDE, static analysis, and semantic salience. In our comprehensive, multi-methodological experimental evaluation, we compare MM assistance with previous state-of-the-art approaches. MM-assist significantly outperforms them: (1) according to a benchmark widely used by other researchers, Recall@1 and Recall@3 show a 1.7x improvement; (2) On a set of 210 recent refactorings of open source software, our recall rates improve by at least 2.4 times. Finally, we conducted a user study with 30 experienced participants who used MM-assist to refactor their code for one week. They evaluated 82.8% of MM assistance recommendations positively. This shows that MM help is effective and useful.