Status Code Validator | Bondar Academy
Course: Playwright API Testing with TypeScript
Module: Building a Framework
Instructor: Artem Bondar
Lesson Summary
In this lesson, we enhance our testing framework by integrating a custom logger to capture logs when tests fail, improving the information presented in the report. Key Modifications We aim to improve the stack trace displayed in the report, which currently shows misleading line numbers from the utility functions instead of the actual test steps. We will modify the request handler to log request and response details for each API call. Implementation Steps Remove the dummy logger and integrate the new logger in the request handler. Create a new instance of the logger in the API fixture for each test run to ensure isolation. Modify the request handler constructor to accept the logger instance and store it in a private field. Log request details before making API calls and log response details after receiving the response, ensuring logs are captured before any assertions are made. Custom Status Code Validator We implemented a custom status code validator method that compares the actual and expected status codes. If they differ, it throws an error with detailed logs: if (actualStatus !== expectedStatus) { const logs = this.logger.getRecentLogs(); throw new Error(`Expected status ${expectedStatus} but got ${actualStatus}\nRecent API activity:\n${logs}`); } Results After implementing these changes, the test reports now correctly indicate the line number where the failure occurred, along with detailed logs of the API requests and responses. This significantly enhances debugging and understanding of test failures. In summary, we have successfully integrated a custom logger into our testing framework, improving the clarity and usefulness of our test reports.
Video Transcript
All right guys, in this lesson we will use a custom logger that we created in previous lesson to capture the logs of the test when the test fails and display those logs in the report. All right, so let's jump into it. And this is the report that we have so far and one more thing that we're gonna need to fix is the stack trace that it's printed in the report when test fails. So look, currently we have this error message expected 201 but got 422, different status code. Because of this assertion expect received to equal expected, right? The problem with this stack trace is which line of code is showing us where the error happened. So look, it's showing line 61 under the utils request handler. And this information is kind of misleading for us because if we would have a separate request inside of our test, this line does not tell us where exactly inside of our test this issue happened. Which test step inside of the test resulted in this error? What we would want to see in the report is the number of the line of the test step itself where the issue happened. It would be more informative for us. So in this lesson we will modify both of those things. We will use logger and we will modify the stack trace and error handling to point to the right line in the test case. All right, so let's jump into it. This is our test. First of all, let's delete this example of just dummy logger. We don't need this anymore. And let's integrate logger into the request handler. How are we going to do that? I navigate to the fixtures right here. And remember our API fixture is responsible for creating a new instance of the request handler for every test. So let's give this fixture also responsibility for creation of the new logger every time. So const logger equals to new API logger like this. You can see the import was added automatically. Logger is created. And now we will pass this instance of the logger inside of request handler. So we can use this logger to capture the logs. And also why do we put the instance creation right here? Remember I was showing you how the object-oriented instance creation works. So when we call an API fixture in the test, every time we call the API fixture, a new instance of the logger will be created for this particular test run. And this will make sure that we will not interfere with other loggers for other tests. So every test will collect its own logs even if we run tests in parallel. Okay, so logger is added. Now let's modify request handler. So we need to add in the constructor logger over here of type API logger. So new import is added right here. Then we need to create a new private field private logger of the type API logger. And we need to assign the value of the logger into this field. So this dot logger equals to logger. That's it. Now we can use this instance of the logger inside of the request handler. Where do we want to use it? We want to capture the logs for each of the requests. Get request, post, put, and delete. We want to capture the request details and the response details. So before we create a request, right here before the get, we call the logger this dot logger dot and log request. And we provide all details that we want to capture. First of all, this is a get request. Then we need to provide a URL. We provide a URL which is this one on the previous line. The next step we need to provide headers which is this API headers. And we need to provide body which is this API body. That's it. Logs are captured. After we made the request right here, we validate the status code. And then we extract the JSON response. So let me modify it a little bit. I will create a new constant actual status equals to response dot status like this. And we'll replace the status code right here. Then response JSON like this. So just modify a little bit the sequence. Okay. Expect actual status to equal the status code. Okay. And after we got the response, we make the logging again. So this dot logger dot log response. And we want to capture is the actual status code and response JSON like this. Okay. I hope it makes sense. All right. And after that, we make a validation and then return JSON object. The rest is everything the same. Okay. So let's just double check and repeat what we did. So we imported the logger into the constructor right here assigned to the field. And then we use the instance of the logger before the request. We capture the request details. And after we get the response, before the assertion, because we don't want to test fail before we capture this logs. Before the assertion, we call log response, capture the status code and response JSON object. Now let's modify the same thing for the other methods. So the next will be a POST request. And it will be a POST URL headers and API body. And by the way, GET request does not have API body. So we can remove it here. All right. URL API headers. Good. And then we let's modify this. Okay. Actual status. Put it right here. I remove this. Okay. Refactor this. And then log response before we make an assertion. Actual status code and response JSON. All right. Looks good. POST request. POST request. The same thing let's do for the next one, which is a PUT request, which is this guy. We change only PUT request. URL is the same. Headers and API body. Everything is good here. Now let's take this. Put it here. Actual status. Put here. And this logger before the assertion. Response. Actual status. Response JSON. And the last one is the DELETE request. For DELETE request, we don't have request body, so we can remove it. For here, we need only status code because we don't have a response body for the DELETE request. So status code right here. We also need to update DELETE over here. And I take this logger and before the assertion put logger right here. JSON response. We don't have it. Like this. Only status code. That's it. So we modified all methods inside of the request handler to capture the request and capture the response details before the assertion. It's important because if assertion will fail, it will not capture the details of the response. Now, how to use it? So we somehow need to read the details that we collected over here. And the best way to handle this is just to create a custom status code validator. Something like this. So think about it. All we do is just comparing the status code. Again, the actual status code. That's it. So if the code is the same, then we're good to go. But if they are not equal, then we want to throw the error message and print the details about the request and response. So let's create this simple private method that will be responsible just for this. And I add it right here below in the bottom of the class. Private. And let me call it status code validator. The status code will have two arguments. Actual status, which is a number, and second will be expected status, which is also number as well. So now let's write some simple logic. If actual status code is not equal to expected status code, then what we want to do? First of all, we want to get the logs that we collected before during the test. So const logs equals to this logger dot get recent logs. The next step. We want to throw a new error message with the details about these logs. How can we do that? I type new error and new error is the object that technically fails the test. And the message that we provide inside over here will be printed in the console and added to a reporter automatically with error message that we want. So let's type some custom error message. Expected status. Let's add this. Expect status but got actual status. And after that, we want to add the log details that we captured. So let's do this on the next line. So dash n dash n. That will be two spaces in the log. Then message that we want to print. For example, recent API activity. And then dash n. Running out of space. Let me do like this. Recent API activity dash n. So we want to move to the next line one more time. And after that, we would like to print the logs right here. So the logs that we collected over here. And after we did this, we need to throw this error. Something like this. And we're almost done. So if we keep this method like this for the validation of the status code, when the status code will not be equal, the test will still fail at this line of code. So in the stack trace, in the reporter, we will see the similar error which will point to our request handler class. This is not something what we want. What we would like to do is a kind of throw error up to the test level, to the place where the method was called that resulted into this error. So how can we do this? Right here, we can call this method object, sorry, error and capture stack trace right here. Then we provide the error message. And then the second argument, we need to provide a function that was called to trigger this error message. And then the error will be printed in the console on the level of the test. So we need to add one more argument over here. I will call it calling method, which is a function. And then I provide the second argument right here, calling method. Okay, I think the method is ready. Now let's replace the assertions inside of the methods, get request, post request, and so on with our new method. So this, how did we call it? Status code validator. So this status code validator, then actual status is actual status, expected status is status code. And the method that was called was this get request. So I use exactly the same method that was used in the test. So the method that I call should match over here with the argument that I pass inside of this method. So that's it. And let's now just copy paste this. We remove this assertion. We don't need it anymore. And I replace the assertions everywhere here, post request. Let's make it a little bit more compact. Status code logger, then put request, replace this, put request. And the last one, this one. And this will be a delete request. All right, I believe we're done. Let's try to run the same test and see what's going to happen. Will we see a new picture with a custom logger? So here we don't need this anymore. And the test that previously failed was this one. Remember it was status code 422. Now let's run it and see what's going to happen. All right, test failed. Yeah, it's a good result. And actually where it failed? It failed in the line 32, exactly what we wanted. Look, the test right now failed inside of the test that we wanted. So this step resulted into the error. Now let's open the report. Open in default browser, test failed. And look at this, we have a stack trace attached to the request. Method post, this is the URL, headers authorization token, and this is a body. And this is a response detail, status code 422. And here we go. This is why the test failed. Errors title must be unique. Now we have understanding why we have code 422. It's because the title is not unique. We're trying to create the article that already exists right here. So if I just remove this article, and run this test one more time, run it. Now we have failed again, but right now it failed over here with this assertion. And if we open the report, open in default browser, and unfortunately currently a report does not have a log information yet. We currently configured it only to be able to run against the status code. So going back over here and let's test it for example 204, something like this. Run it one more time. Okay, test failed. Now it works correctly. It failed in the... No, it failed again in the same line. So let me delete it. Hold on a second. So refresh. I delete it. Run it again. Okay, you see, now it failed at this line of code because it expected status 204, but the actual status was open in default browser. The actual status was 200. And look, we collected all the information about the requests and response during this test run. So first we created the article, was this one. This was response for creation of the article. Then we got a GET request with all the details. You see articles, limit, offset, everything. We have everything. Now the response details. We have huge response details with all articles from our API. And then at the end we have the number of line. It's line 40 inside of the test where the issue happens. And now we can see that, okay, the response was 204 but got 200. You can look in the response. Yeah, it's actually status code was 200. So all we need is probably just update the expected status code. All right guys, so that's pretty much it. Let's finalize what we did in this lesson. So we created a logger in the previous lesson and here inside of the fixture we added this logger inside of the API fixture, creating a new instance of the logger every time we run the test and passing this instance inside of the request handler. Inside of the request handler we created a new field of the logger and assigned this value of a logger inside of the field of request handler. And then we use this logger inside of the get request, put request, post and delete methods to collect the request details and collect the response details for every API call. And then we replaced the standard assertion with our custom status code validator to be able to throw a custom error right here, custom error message to the report. And then we used a capture stack trace method to throw the error up to the test level so we will know in the report where our test failed. All right, that's it guys and see you in the next lesson.