Enhancing REST API Testing with NLP Techniques

RESTful services, also known as web APIs, are commonly documented using OpenAPI specifications. Although numerous automated testing techniques have been proposed that leverage the machine-readable part of these specifications to guide test generation, their human-readable part has been mostly neglected. This is a missed opportunity, as natural-language descriptions in the specifications often contain relevant information, including example values and inter-parameter dependencies, that can be used to improve test generation. In this spirit, we propose NLP2REST, an automated approach that applies natural language processing techniques to assist REST API testing. Given an API and its specification, NLP2REST extracts additional OpenAPI rules from the human-readable part of the specification. It then enhances the original specification by adding these rules to it. Testing tools can transparently use the enhanced specification to perform better test case generation. Because rule extraction can be inaccurate, due to either the intrinsic ambiguity of natural language or mismatches between documentation and implementation, NLP2REST also incorporates a validation step aimed at eliminating spurious rules. We performed studies to assess the effectiveness of our rule extraction and validation approach and the impact of enhanced specifications on the performance of a set of state-of-the-art REST API testing tools. Our results are encouraging and show that NLP2REST can extract many relevant rules with high accuracy, and significantly improve testing tools’ performance.