https://hdl.handle.net/10388/16429

Roy, P. R. (2025) An Exploratory Study on the Roles of LLMs in Code Similarity Detection and Generation. University of Saskatchewan, Harvest, Graduate Theses and Dissertations. https://hdl.handle.net/10388/16429

Code similarity, also referred to as cloned code, involves identifying redundant code fragments, enabling effective refactoring, and promoting code reuse. This, in turn, enhances both the maintainability and scalability of software systems. However, despite advancements in automated tools, detecting and generating clones, particularly across different programming languages and considering semantic variations—remains a significant challenge in software engineering.

With the rise of large language models (LLMs) such as GPT, Gemini, and LLaMA, there is growing interest in leveraging their capabilities to improve clone detection, code generation, and cross-language translation. Despite this interest, research has yet to thoroughly evaluate how these models can be systematically applied to address the challenges in these domains. This thesis aims to bridge that gap by investigating the role of LLMs in three studies: clone generation, code quality assessment of similar codes, and cross-language clone detection, with the goal of advancing software engineering practices.

The first study evaluates the performance of GPT-3 in generating semantic and cross-language code clones(similar codes) using the SemanticCloneBench dataset. Extensive experiments, validated by nine expert judges over 158 hours, show that GPT-3 achieved 62.14% accuracy and a 0.55 BLEU score in semantic clone generation, along with 91.25% accuracy in cross-language clone generation. These results underscore the potential of LLMs, particularly GPT-3, to support code reuse and facilitate migration across programming languages.

The second study assesses the quality of similar code generated through three approaches: human-generated solutions from Stack Overflow, LLM-based code search tools such as CodeBERT and GraphCodeBERT, and AI-powered models including GPT-4, Gemini Pro-1.5, and LLaMA-3-8B. Analyzing 800 programming questions (400 in Python and 400 in Java), the study demonstrates that GPT-4 consistently outperformed other tools in usability, accuracy, and overall code quality, while the code search tools underperformed. These findings offer valuable insights for developers, particularly in selecting reliable tools for high-quality code generation.

In the third study, I developed Lang2Clone, a novel tool that combines zero-shot prompting with GPT for code translation and NiCad for clone detection. Evaluations on the LeetCode and GPTCloneBench datasets showed that Lang2Clone achieved recall rates of 80% and 85%, respectively, with a precision of 95%. Notably, Lang2Clone outperformed state-of-the-art tools like GraphCodeBERT and CLCDSA in cross-language clone detection, particularly when translating between languages with similar programming paradigms.

Collectively, these studies highlight the potential of LLMs to enhance key software engineering tasks, particularly in clone detection and code reuse. The findings also suggest promising avenues for future research, including the application of LLMs to broader challenges such as software maintenance and cross-language code translation, paving the way for more sophisticated and efficient development tools.

https://hdl.handle.net/10388/16429