Data-Driven Testing | Bondar Academy
Course: Playwright API Testing with TypeScript
Module: Test Management
Instructor: Artem Bondar
Lesson Summary
In this lesson, we explore the concept of data-driven testing , which involves executing the same test multiple times with different data sets. This approach is particularly useful in API testing to validate various error messages returned by the API when invalid data is submitted. Key Concepts Data-Driven Testing: Running the same test with different inputs to verify various outcomes. API Validation: Ensuring that the API returns appropriate error messages for invalid requests. Demonstration Using Conduit Application The instructor demonstrates data-driven testing using the Conduit application, focusing on the signup process: Invalid inputs trigger specific error messages, such as Email invalid or Username is too short . Using tools like Postman , different scenarios are tested by sending various request bodies to the API. Testing Framework Setup The lesson covers setting up a testing framework to automate the validation of error messages: Create a new spec file for negative tests. Define test cases for different username lengths (e.g., less than 3, exactly 3, 20, and more than 20 characters). Use an array to store test data and iterate through it using forEach . Assertions and Validations Assertions are added to check if the appropriate error messages are returned or if no error messages are displayed for valid inputs: Use conditional statements to validate the presence or absence of error messages based on input length. Run the tests to ensure all scenarios pass successfully. In conclusion, data-driven testing is a powerful method to efficiently validate API responses without duplicating test cases. This approach enhances testing coverage and maintains code clarity.
Video Transcript
Hey guys, welcome back. And in this lesson, we're gonna talk about data-driven testing. So first of all, what is that? Data-driven testing is when you run the same test, but with a different data set. So repeating the same scenario again and again. In API testing, this type of testing is usual when you want to validate, for example, error messages for your API when you send an invalid request body. And your API returns you a different errors related to what went wrong. So let me demonstrate in our Conduit application how this can be applied. So this is our Conduit app. And if we go to the signup, here is username, email, and password. And if we provide some invalid information during the signup, for example, let's do something like this. And I try to sign up, we see the error message. Email invalid, username is too short, and password is too short as well. And this validation actually happens on API level, not on the UI. I make right-click inspect, go into the networking tab, and let me run this one more time. You see, I show the response, and here's the error messages. Errors, email is invalid, username is too short. Or if I try to provide some crazy long characters, something like this, for the username and password, and try to sign up. Okay, I need to update here. And now we see a different message right here, that password is too long, and username is still showing too short. Because this is email, this is the username, here we go. And here we go, for the username, we have the same error message, that it should be maximum 20 characters long. So let's test this scenario, validating error messages for this API using data-driven testing approach. And for that, I have set up the postman that we can validate this approach. So here is the 20 characters. If I try to make it smaller, I send, we don't have any error message. If I put something like this, and send it again. And for the username, I think we need less than that. Yeah, for the username, it's required to have at least three characters, for the password, at least eight characters. For if I put for username, three characters, it's okay. So let's run the test and validate three cases, let's say only for the username. So when the password is less than three, so when we do two characters, we should see this message. If I have three characters and run the test, we should not see this error at all in the object. Or if I provide something crazy, big value, send it again. Sorry, it was a password, I provide a crazy big value. We should see this error, all right? So going back to our test framework, and for the negative scenarios, let's create a new spec file. New file, negative tests.spec.ts, right here. So let's copy some of the stuff that we need, test and expect imports, right here, and let's start a new test. Test, and I call it ErrorMessageValidations. Async, then we need to provide API fixture, and like this. Okay, this is our test body. Now let's create, write actually the test that we created in the postman, just with the hard-coded values so far. Const new userResponse equals to await API. And then we need to write the test. So we need to send the request body, and the request body, going back to the postman, and let's take this one as our starting point. And I will update it just to a single character each for the password and for the user name. What else? We need also provide the path. Need to provide the path for our API, which is users. Is it? Yeah, it's a user's endpoint. And also, since by default, you remember our framework authorizing all of our calls, we need to make a clear of to make sure that it will be just a regular call without authorization. And it will be a post request, and the expected status code will be 422, and I type it 422. Sounds good. Let's just bring this to the console, console.log, just to make sure that we see the new response object like this. And, okay, I don't see this run mark over here. Let's remove this test match, going back one more time, and we can run it. It works successfully, we can see the error messages are printed. So let's validate only the error messages scenario for the user name. Basically, we're gonna have four test cases, right? When the user name is less than three, when it is three, we should not see error message, when it is 20, we should not see error message as well, and when it's 21 and more, we should see the error message that user name is too long. You can create the data set bigger with more values than the user name, but just for this demonstration, I'm gonna use just a single value for the simplicity. So what are we gonna start with? First of all, let's create our data set. And we start with just an array of the values. So what we're gonna need? We need a user name, let's do it like this, user name, and the test value initial would be dd. And when we provide this, the user name error message have to be, and it have to be, Let me delete this, send it, a message should be this one. Going back and paste it right here, okay. This is the first data set. Now put the comma, the second data set. When we have three characters, then the message should be nothing. So it would be empty like this. Then the next data set, if it is more than that, so we need at least 20, so 1, 2, 3, 4, 5, 4, 5. And then Ctrl C, it would be 10, 15, 20, this should be a 20. We should not see any error message as well. And the final error message, if we have 21 character over here, we should see another error message, which is generated one more time. That user name is too long, that this guy, going here and pasting over here. All right, data set is ready. And by the way, this data set have to be created outside of the test. I have created inside of the test, so let me put it outside of the body of the test right here. So now how to use it? When we create a data set from the array, we call method forEach like this. Then inside of the forEach method, we create a callback function like usually, like this. And then inside of the callback function, we need to create object with these two properties, which are username. And the second property is usernameErrorMessage, that's it. And now inside of the forEach, we need to put our test. So let me make it a little bit compact. Command X, and I put the test inside of it. Now we have a little problem. So Playwright is complaining that duplicate test title, why is that? Because it understand that we want to run the same test four times, and this test will have the same title. So we need to change the title and to change it dynamically. So instead of string, I change it to a backticks. And then add this variable, which is errorMessageValidation. Let's do it like this, forUsername. So this name will be printed in the title of the test when the test will be executed. And that's it, right now, Playwright is not complaining anymore. So that part is done, and all we have to do is just to replace this variable inside of the request object. So for the username, instead of d, I put the username. This username will represent for every cycle this value from the object, from the data set on each cycle, on each iteration of the test execution. All right, this part is done. And the only final part left is to add validation right here. So since we have kind of two types of validation. First is when the error message is displayed, we need to validate this message and this message. And when we have three characters or 20 characters, the error message should not be displayed at all. So we need to create kind of conditional statement here that if we have a username length of three or length of 20, then the property of error for the username should not be displayed. All right, so let's do this. So I type if usernameLength equals to 3, or usernameLength is equals to 20, then the assertion that we're gonna do, expect newUserResponse.errors. So I am looking for, if I go in back to the postman, I'm looking for this one. So newUserResponse.errors should not have property username. So this property should disappear from the response object. And this is what validation I am adding. Should not have property, this one. And I paste this property as a string, because this is a property name. This is not a variable like this one, username. I put it as a string. Else, I add the second validation. And that, right now we validate the actual error message. So I expect that the same newUserErrors.userName, and we use our custom assertion, should equal, and the error message would be usernameErrorMessage, like this. Is it correct? So errors, username. So let's see one more time. Okay, not really, because username, you see, it is an array. So we need to point the first element of the array to correctly grab the message from it. So going back, and username, and I get the first value from the array. And I guess this is it. So let's run this test. We'll see if it's working. So running the test, and yeah, all tests passed. Look at that. So I can remove this console log. So it will not be printed right here. Running the test again, and everything works successfully. So all four error messages, all four cases were successfully validated for the DD, DDD, 20 characters, and more than 20 characters. We validated that this is it. And if just to check if it is working, look, let's check something, let's change something in our error message. Let's say, change 20 to 50, and I run this, and the test should fail. And yeah, the test expectedly failed, because we see error message that expected to loan maximum 50 characters, but received 20 characters. So our automation working as expected. So that's it, guys. That's how you run, let me run it one more time. Yeah, everything passed. All right, that's it, guys. This is how you run data-driven tests in Playwright. This is very useful when you want to validate error messages for your APIs. And instead of copy-pasting your test again and again, you can run the same test, but with a different data set, testing different error messages from your API. All right, see you in the next lesson.