AI Test Cases Generation | Bondar Academy
Course: Mastering testRigor
Module: Generative AI
Instructor: Artem Bondar
Lesson Summary
This module discusses the generative AI features of the TestRigor framework , focusing on how to automatically generate test scenarios and test cases based on application descriptions. Key Concepts Generative AI : Used to create test scenarios and cases automatically. Application Example : The demonstration uses BestBuy.com , an online store for consumer electronics, to illustrate the process. Test Suite Creation : A new test suite named generative AI tests is created, and settings are adjusted for optimal performance. Generating Test Cases To generate test cases: Provide a detailed application description to help AI understand the context. Use the generate test case feature to create scenarios based on the description. For example, a well-structured description of Best Buy includes details about product navigation, search functionality, and the checkout process. Corner Cases Another feature allows for the generation of corner cases , which test negative scenarios, such as error messages for input fields. Test Case Execution After generating a test case, the AI executes it, simulating user interactions. Adjustments can be made post-execution to refine the test script. In summary, TestRigor's generative AI capabilities enable users to efficiently create and manage test cases, enhancing the testing process for applications.
Video Transcript
In this module, we're going to talk about some of the generative AI features of the TestRigor framework. So you can use the generative AI to automatically generate your test scenarios based on the description, or generate automatic test cases, or based on your written test cases, you can also create automated scenarios. So, and in this section, we're going to talk about it. And in this particular lesson, we will talk about how to generate the test cases automatically based on the description of your application. So let's get into it. So for this demonstration, I will take the different application, for example, this BestBuy.com. So why I'm taking this instead of our previous test application? Because AI in general likes more complete, functional, and broad applications. Just test application or demo application that we used before will not play really well. That's why I'm going to use BestBuy just for this demo. So this is online store where you can buy different consumer electronics. And using generative AI, we will try to run some scenarios over here. And let me create a new test suite. So I will call it generative AI tests. I will copy the URL, BestBuy.com. I will not have any credentials. And here for now, I put a zero. So we will automatically generate the test cases later inside of the test suite, because we need to make a certain configuration, which is easier to make when the test suite is created. And I click create test suite. All right, a new test suite is created. And first thing, we need to make a couple of adjustments in the settings. So I'm going to the settings. Let's click speed optimization first and remove this checkmark. So it will run a little bit faster. Click save. And after that, I go to this test generation section, where I need to change this settings. Instead of do not use Vision AI, which is selected by default, I want to use Vision AI and select this. The next step, we need to provide here our app description. So you need to provide as much details as possible and as much context as possible about your application. This way, it helps easier for AI to understand, okay, what is type of your application? What your application can do? What are the main flows? And when later, when you tell to generate test cases or the particular test cases, follow the steps, it will be easier for AI to identify the elements on the page if you provide enough context. So I can give you like a couple of examples. So for example, if you do something like this, here you can buy different products. You can search for products or select those products. From the menu, the selected products can be purchased. Usually you can select different color specifications and number of items you want to buy. So this is a bad description. It does not provide much of a description what Best Buy is about, what you can do, what are the typical user flows, the core scenarios, and so on. And this is example of, I would say, a better description. I would not say that this is good description as well because you definitely can provide more context about the Best Buy, but this is anyway better than it was before. And something like this. This is a big e-commerce online store, mostly selling electronic. There is a navigation hamburger menu that opens menu and users can select different departments, such as appliances, TV, computers, cell phones, et cetera. By navigating through those categories, users can choose the needed products category to shop for the products. All users can use search input field on the main page to search any product by name. Search product has information such as model, color, reviews, and price. To buy any product, user must select Add to Cart button to add the desired product to cart. Then user has to navigate to cart if user want to complete the checkout process and purchase the product. This is kind of a better, more context-driven description, which tells what the store is about, what it sells, and what user can do and how to overall navigate on the application. So let's take this description as an example and paste it over here. So this test description, test rig, we're going to use as a reference point for the future building of the test scripts to better understand our application. And we click Save. All right. Now we're going back here to the test cases, and we have several options. For example, we can absolutely randomly tell AI to generate for us a test case which AI may think is reasonable for our application based on the description that we have. And to do that, we have this generate test case based on the feature description. So you click on this button. Then you can select the number of test cases. I select one. And then the feature description is pulled from the settings of the test suite. And you can update this description if you want, but I would like to keep it as is. And I click Generate. It takes some time, and Test Rigger creates this kind of scenario. Verify end-to-end purchase flow from product search to checkout in e-commerce store. So what it's going to do, it will try to make end-to-end process of the purchase for the product because this is e-commerce store. This is what it got from the description. And let's just add and build and just see what is going to happen. So the new test case is added, and now let's see what it's going to do. All right. Test case started. We navigated to the BestBuy.com, and let's see. All right. So it is searching for the laptop into what can we help to find today. So why it decided to search for a laptop? I don't know. Probably because it decided and understand that this is a consumer electronics. That's why it's searching for the laptops. All right. The first laptop was selected. It's clicking on the cart. Click Add to cart. Now it looks like we are on the payment page. Now we are on checkout page. So here it decided to go, since we don't have account, it decided to go to continue as a guest. And it's done. So the test case completed at the checkout page. So it didn't try to enter any information. So it decided that it's enough. We selected the product. We went through the process of adding it to the cart, and we are on the checkout page. Be ready to check out. If we open this test case quick edit, this is what we can see, what test rigor is generating. So let's see. This is what we can see, what test rigor is generated. So it went through the description of our application and based on the description, pretty much generated the well-known for you commands that test rigor can understand. So if we run this test case one more time, then test rigor will not use the just gen AI. It will just follow these exact steps. So this is how it works. And let me show you one more example. So when you generated a particular case, then you can automatically generate a corner cases. So look here, we have a button generate a corner cases. So what this button essentially does, it will try to generate a negative scenarios for some of the input fields. So for example, test will try to execute the flow based on the previous test, but when it will interact with particular input field, where need to type something or something like this, it will test different negative scenarios. For example, error messages for this input field or something like this. And let's try to generate this kind of corner case. So I click the button and it automatically generating the test case and executing it. All right. It seems like it's completed, but I don't see it on the page. Let me refresh. And we have a corner case. So let me open up and see what happened. So test trigger executed the following scenario. So it's just click submit search and verify that the input field contains this and that page contains submit search. So in this example, when we submit the search, it did not really generate the error message. But usually when you try to submit something like this, this actually generates. So AI tried to simulate this scenario and validate the message of the input field. So this is how corner cases works. And sometimes it can generate a pretty reasonable corner cases. So let me show you one more example. So when you create, for example, a new test case, you can generate a new test case just based on description. And the description that you're going to provide in the test case description over here will be added to the context that you provided in the test suite settings. So let's create scenario something like this. Find iPhone 15 Pro. Add it to cart. Then navigate to cart and validate that iPhone is added to cart. Something like this. And then I have an option clicking either generate sample test using AI or generate actual test using AI. So if you want to generate the actual test, which is related to this context, we need to click this button, generate actual test using AI and click this button. So now AI, using this description of the test case and context that we provided at the test suite level, will try to automate this scenario, figuring out the flow on the application. All right, and the test case finally was executed. And I did a speed up a little bit because it took way more operations that we wanted it to complete, but this is what AI decided about the iPhone. So let's look real quick in this scenario what actually happened. So he searched for iPhone, then click submit search, that on this page, he decided to click on the reviews. But anyway, that way it was able to select the iPhone, but look what happened next. Best Buy unexpectedly had a pop-up and AI figured out, no, we want to continue and it automatically clicked no thanks. So then we are on the iPhone page, then we try to add it into cart. And then the next step, we are on the cart. Looks like, yeah, this is your cart page. And then it's going to the checkout, but it did not perform. Unfortunately, validation of that iPhone was selected and instead it was trying to continue checkout, continue as a guest, then continue as a payment, then trying to enter in credentials, phone number, and then validation of the error message. So it did some of the stuff that we didn't ask it to do. Yeah, unfortunately, the AI world is still not perfect. But if you go back after the test execution, you later can fix this and adjust the script, remove the steps that you don't need over here and adjust the assertions that you would like have over here. So when this example can be useful. So remember when we initially started our lectures, I showed you the test trigger Chrome extension that can record your steps on the Chrome browser. So this is a second example, the alternative. If you, for example, have a test cases that you want to try and see how test trigger gonna set up the test for you, it could probably work well for your test application. And also this AI feature is something that test trigger continuously working on improvement. So I'm sure it's gonna work only better in future. All right, that's it guys. This is how you can use generative AI features of test trigger based on the details description, how you can generate test cases, corner test cases, and test cases based on description of the test case. All right, that's it guys. See you in the next lesson.