Publications

You can also find my articles on my Google Scholar profile.

Journal Articles

LlamaRestTest: Effective REST API Testing with Small Language Models

Published in Proceedings of the ACM on Software Engineering, Volume 2, FSE, 2025

This paper presents LlamaRestTest, a novel REST API testing approach that employs two custom fine-tuned small language models (Llama3-8B) to generate realistic test inputs and uncover inter-parameter dependencies during testing by analyzing server responses. The approach demonstrates that fine-tuning enables smaller models to outperform much larger models in REST API testing, balancing effectiveness and efficiency.

Recommended citation: Myeongsoo Kim, Saurabh Sinha, and Alessandro Orso. 2025. LlamaRestTest: Effective REST API Testing with Small Language Models. Proc. ACM Softw. Eng. 2, FSE, Article FSE022 (July 2025), 24 pages. https://doi.org/10.1145/3715737
Download Paper

Conference Papers

A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs

Published in Proceedings of the IEEE/ACM 47th International Conference on Software Engineering (ICSE 2025), 2025

This paper presents AutoRestTest, the first black-box REST API testing tool to adopt a dependency-embedded multi-agent approach that integrates multi-agent reinforcement learning (MARL) with a semantic property dependency graph (SPDG) and Large Language Models (LLMs). AutoRestTest treats REST API testing as a separable problem where four specialized agents collaborate to optimize API exploration, achieving superior code coverage and fault detection compared to state-of-the-art tools.

Recommended citation: Myeongsoo Kim, Tyler Stennett, Saurabh Sinha, and Alessandro Orso. 2025. A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs. In Proceedings of the IEEE/ACM 47th International Conference on Software Engineering (ICSE 2025). IEEE Press, 1409–1421.
Download Paper

ASTER: Natural and Multi-Language Unit Test Generation with LLMs

Published in 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 2025

This paper presents ASTER, an LLM-assisted test generation technique guided by program analysis that generates natural and high-coverage unit tests for Java and Python. ASTER achieves competitive coverage with state-of-the-art tools while producing significantly more natural test cases that developers prefer. Distinguished Paper Award at ICSE-SEIP 2025.

Recommended citation: R. Pan, M. Kim, R. Krishna, R. Pavuluri and S. Sinha, "ASTER: Natural and Multi-Language Unit Test Generation with LLMs," in 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), Ottawa, ON, Canada, 2025, pp. 413-424, doi: 10.1109/ICSE-SEIP66354.2025.00042.
Download Paper

AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL

Published in 2025 IEEE/ACM 47th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2025

This paper presents a tool demonstration of AutoRestTest, an automated REST API testing tool that combines multi-agent reinforcement learning (MARL) with semantic property dependency graphs (SPDG) and Large Language Models (LLMs) for enhanced test generation and fault detection.

Recommended citation: T. Stennett, M. Kim, S. Sinha and A. Orso, "AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL," in 2025 IEEE/ACM 47th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), Ottawa, ON, Canada, 2025, pp. 21-24, doi: 10.1109/ICSE-Companion66252.2025.00015.
Download Paper

Leveraging Large Language Models to Improve REST API Testing

Published in Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER 2024), 2024

This paper presents RESTGPT, an innovative approach that leverages Large Language Models to improve REST API testing by extracting machine-interpretable rules and generating example parameter values from natural-language descriptions in API specifications. RESTGPT achieves 97% precision in rule extraction and 73% accuracy in value generation, significantly outperforming existing techniques.

Recommended citation: Myeongsoo Kim, Tyler Stennett, Dhruv Shah, Saurabh Sinha, and Alessandro Orso. 2024. Leveraging Large Language Models to Improve REST API Testing. In Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER 2024). Association for Computing Machinery, New York, NY, USA, 37–41.
Download Paper

Improving Program Debloating with 1-DU Chain Minimality

Published in 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE-Poster), 2024

This paper introduces RLDebloatDU, a novel debloating technique that employs 1-DU chain minimality within abstract syntax trees to maintain essential program data dependencies. The approach strikes a balance between aggressive code reduction and preservation of program semantics, significantly reducing CVEs while maintaining soundness better than both aggressive and conservative baseline approaches.

Recommended citation: Myeongsoo Kim, Santosh Pande, and Alessandro Orso. 2024. Improving Program Debloating with 1-DU Chain Minimality. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion 24). Association for Computing Machinery, New York, NY, USA, 384–385. https://doi.org/10.1145/3639478.3643518
Download Paper

Adaptive REST API Testing with Reinforcement Learning

Published in Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering (ASE 2023), 2023

This paper presents ARAT-RL, an adaptive REST API testing technique that incorporates reinforcement learning to prioritize operations and parameters during exploration. The approach dynamically analyzes request and response data to inform dependent parameters and achieves superior code coverage and fault-detection capability compared to state-of-the-art tools.

Recommended citation: Myeongsoo Kim, Saurabh Sinha, and Alessandro Orso. 2024. Adaptive REST API Testing with Reinforcement Learning. In Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering (ASE 2023). IEEE Press, 446–458.
Download Paper

Enhancing REST API Testing with NLP Techniques

Published in Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2023), 2023

This paper presents NLPtoREST, an automated approach that applies natural language processing techniques to extract rules from the human-readable part of OpenAPI specifications. The technique generates enhanced specifications that significantly improve the performance of REST API testing tools, increasing coverage by up to 103% and successful request rates by 20%.

Recommended citation: Myeongsoo Kim, Davide Corradini, Saurabh Sinha, Alessandro Orso, Michele Pasqua, Rachel Tzoref-Brill, and Mariano Ceccato. 2023. Enhancing REST API Testing with NLP Techniques. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2023). Association for Computing Machinery, New York, NY, USA, 1232–1243.
Download Paper

Automated test generation for REST APIs: no time to rest yet

Published in Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2022), 2022

This paper presents a comprehensive empirical study of 10 state-of-the-art REST API testing tools, evaluating their performance on 20 benchmarks in terms of code coverage and fault-detection ability. The study identifies key strengths, weaknesses, and limitations of existing approaches and provides concrete directions for future research.

Recommended citation: Myeongsoo Kim, Qi Xin, Saurabh Sinha, and Alessandro Orso. 2022. Automated test generation for REST APIs: no time to rest yet. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2022). Association for Computing Machinery, New York, NY, USA, 289–301.
Download Paper

Subdomain-Based Generality-Aware Debloating

Published in 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2020

This paper proposes DomGad, a debloating technique that produces reduced programs guaranteed to work for subdomains rather than specific inputs. Using stochastic optimization, it generates debloated programs achieving close-to-optimal tradeoff between reduction and generality, with results showing 50% code reduction and 95% generality on average.

Recommended citation: Qi Xin, Myeongsoo Kim, Qirun Zhang, and Alessandro Orso. 2020. Subdomain-Based Generality-Aware Debloating. In 35th IEEE/ACM International Conference on Automated Software Engineering (ASE 20), September 21–25, 2020, Virtual Event, Australia. ACM, New York, NY, USA, 13 pages.
Download Paper

Program debloating via stochastic optimization

Published in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER 20), 2020

This paper proposes a general approach that formulates program debloating as a multi-objective optimization problem. Debop, a specific instance of this approach, uses Markov Chain Monte Carlo (MCMC) sampling to consider three objectives: size reduction, attack-surface reduction, and generality to generate debloated programs that achieve optimal trade-offs between different debloating goals.

Recommended citation: Qi Xin, Myeongsoo Kim, Qirun Zhang, and Alessandro Orso. 2020. Program debloating via stochastic optimization. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER 20). Association for Computing Machinery, New York, NY, USA, 65–68.
Download Paper

Scam Detection Assistant: Automated Protection from Scammers

Published in 2019 First International Conference on Societal Automation (SA), 2019

This paper presents a Scam Detection Assistant (SDA) that analyzes attack content through semantic analysis to detect social engineering attacks. The approach focuses on natural language content analysis, making it applicable to various attack vectors including phone, text, chat, and in-person communications.

Recommended citation: M. Kim et al., "Scam Detection Assistant: Automated Protection from Scammers," 2019 First International Conference on Societal Automation (SA), Krakow, Poland, 2019, doi: 10.1109/SA47457.2019.8938036.
Download Paper

A Chatbot for Conflict Detection and Resolution

Published in 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE), 2019

This paper presents a chatbot system for detecting and resolving code conflicts in collaborative software development, helping developers maintain awareness of configuration management issues.

Recommended citation: E. Paikari et al., "A Chatbot for Conflict Detection and Resolution," 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE), Montreal, QC, Canada, 2019, pp. 29-33, doi: 10.1109/BotSE.2019.00016.
Download Paper

Catch me, Yes we can! - Pwning Social Engineers using Natural Language Processing Techniques in Real-Time

Published in Black Hat USA 2018 (White Paper), 2018

White paper presenting an approach to detect social engineering attacks through natural language processing and semantic analysis. Applicable to multiple attack vectors including email, texting, chat applications, and phone/in-person communication.

Recommended citation: Myeongsoo Kim, Changheon Song, Hyeji Kim, Deahyun Park, Yeeji Kwon, Eun Namkung, Ian G Harris, Marcel Carlsson. (2018). "Catch me, Yes we can! - Pwning Social Engineers using Natural Language Processing Techniques in Real-Time." Black Hat USA 2018 White Paper.
Download Paper