Assertions Enhancement | Bondar Academy
Course: Playwright API Testing with TypeScript
Module: Building a Framework
Instructor: Artem Bondar
Lesson Summary
In this lesson, we focus on enhancing our testing framework by creating custom assertions in Playwright. The goal is to improve the default assertions to include log details, which are crucial for effective debugging. Key Improvements Created a new file custom-expect.ts under the utils folder. Extended the base expect functionality to include custom matchers. Developed a should equal assertion that replaces the default toEqual assertion. Implementation Steps Imported the necessary modules and set up the logger instance. Defined the setCustomExpectLogger function to pass the logger instance into custom-expect.ts . Created the shouldEqual function to handle assertions and log details. Implemented a try-catch block to manage assertion results and log errors when assertions fail. Formatted error messages to include logs for better clarity during test failures. Testing the Custom Assertions After implementing the custom assertion, we tested it by replacing toEqual with shouldEqual in our test cases. The tests successfully passed, and when they failed, detailed logs were displayed, enhancing our debugging process. In conclusion, we successfully created a logging-enhanced custom assertion framework, allowing us to capture and report detailed logs during test failures. For further clarification, feel free to reach out via Slack.
Video Transcript
Hey guys, I know that the previous two lectures were quite intense. So we created a custom logger and we created a method to validate status code and attach the log details to the report if the validation of status code fails. So we need to make one more improvement to the framework and update the assertions. Currently assertions just provide us the received and expected result and that's it. But to effectively debug the test, we also need log details, what went wrong during the test so we can see the history of execution. And in this lesson, we will create a custom assertions, improving the default assertions of the playwright to adding the log details to it. All right, so let's jump into it. So this is our smoketest.spec.ts. And to create a new assertions, I will create a new file under the utils folder and I will call it custom-expect.ts. All right, so what we're gonna do here and what we put here. So I open playwright documentation, playwright assertions, just Google for that, open playwright assertions, and here add custom matchers using expect extend. So playwright has a capability when you can extend the default functionality of the assertions, adding your own functionality, whatever you need. And this is what we're gonna do. So let's copy the first line of code and we're gonna use this example, this assertion as the example. But we will remove anything what we don't need and keep only what is relevant for our API testing. So I copy the first line and paste it over here inside of our code. And look, the syntax is very similar to how we did with the fixtures, remember? So we import expect as base expect and then we can extend base expect with the functionality that we need. All right, moving on. So we don't need the second line, page locator. We don't use page, we don't use locator. It's irrelevant for us. Test, it's also not needed. And this is the body of the actual expect assertion. So copying this and pasting it right here. Curly brace, brace right here. And this is the body of the expect function. Now, we need to add a logger to this class. So let's create this first. How are we gonna do this? So since this is not a class, we cannot create the instance of the class and use a constructor. I will create a function that will be responsible for pulling the logger from the fixture. So here in the fixture, remember, we create the instance of a logger. And I will create a function that will pass the instance of the logger into custom expect.ts file. So first of all, I will create the variable API logger of the type API logger. Import is created, then I create export const set custom expect. Logger and assign the result of the function. The argument will be logger of type of API logger. And the function will do a very simple single step. API logger equals to logger, that's it. So now we can call this function inside of the fixture and that way we will pass the instance of the logger inside of this file. So over here, I call set custom expect logger like this. Import edit automatically. And as the argument, I take the instance of the logger and pass it inside. That's it, so now custom expect has the instance of a logger. So let's continue. Going back to PlayWrite documentation and look, this is actual function, the custom expect function that is created. They call it to have amount. And essentially, what this to have amount does is some other modifications using to have attribute built in assertion into the PlayWrite. And they use async and await because this is a locator type of the assertion. We're gonna use just a simple one. So what assertion do we use right now? Currently, we use just to, to equal and to be less than or equal. This is a generic assertions. And instead of using the same wording like to equal, let's create own naming convention. For example, if we want to use our custom assertions, let's call it, for example, should equal instead of to equal. By using a different name, we will be able to use same expect object inside of the test. But when we want it, we can call, for example, to equal assertion. But when we want to use a custom assertion, we can call expect should equal, and it will work both ways, okay? So let's for the to equal, we will use should equal. It's gonna be our assertion name. So should equal, okay? And should equal will have just two arguments, expected of type any and expected of type any as well. So these are two values that we're going to compare, okay, moving on. So what else do we need? Here we need absolutely to get this Boolean flag, which is pass or fail. So the result of test execution, is it pass or fail will be assigned to this Boolean flag. So we need this guy. The next thing, this is the main body where the magic actually happens. Let me copy this try-catch block. So what we are doing here, instead of based expect, so I will remove await. We use a regular assertion. Instead of locator, I provide received this. And then we replace it to equal assertion. And to equal will be the expected. So we technically execute inside of the try block. We execute exactly the same assertion as we run inside of our test. Then what happened next? When this assertion pass, then we assign result of true to the pass flag. And the catch block is simply skipped. If this assertion fails, then error will be created. This error will be caught by this catch block. And then false value will be assigned to the pass flag. And this pass flag will be returned to the playwright. And that way, playwright will know, should I fail this test or should I pass this test? This is how it works. Matcher result, we don't need this. We're gonna keep it super simple. We're not gonna use matcher. And this is the place where we can introduce our logger. So when the test fails, we want to get the log details and attach it to the message. So I will create a new variable logs of type string and assign a default value of the empty string. And here I call logs equals to API logger.getRecentLogs. Here we go. When the test fails in the catch block, we read the logger and read log details. Perfect, the next step. The next step, we need to return the details with a message. How our test fails, including the log details. Let's look into the next section. This is where the message is printed. Let me copy this entire code, paste it temporarily over here. And let me explain it. What's going on here? So according to the requirements, the message have to be a function. And what logic is demonstrated here. So when the pass is true, then this section of the code is executed. When the pass flips to false, then this section is executed. And this is a function that technically return just a message. This is a message and this is also a string. And if you look at both of those blocks, look at this. They are pretty much the same. The only little tiny difference between them is this text not. That's it. So expected, not expected result. So this text not is used when we trigger this type of the assertion like this. For example, not to equal. And how Playwright know what to do about this dot not to equal, how it works. So Playwright has an object over here, this is not. So when dot not is used in the test, then the flag of this is not will change the value to true. If dot not is not used in the test, then this value is a false. This flag is technically just define how the message will be printed and then the logic later on. I will explain the logic. So instead of for us copy pasting this thing two times, we can use this flag to drive the message to use not or to not. It will be just a simpler for us. So I like simple code, so I remove all this stuff, okay? Also, I will remove the function, we will create function later. And it will be just a message. We assign the message to the constant message. Now let's replace this not using the flag, this is not. Const hint equals to and this is not. So if is not is true, it means that not was used, then we want to print not. Otherwise, we will print just an empty string like this. And then this not, we can replace with this flag hint. Okay, so this is how we optimized this stuff. Now let's fix the red squigglies over here. Assertion name. Assertion name should match the name of the assertion that we're gonna use. So this is the name that will be printed in the error message, should equal, okay? Then we have two horizontal spacers. Then locator, we run API test, we don't need locator, I remove it. Then expected, this is the message for the expected value. And then message for received value, they use matcher result. So matcher result related to locator type of the assertion when you dynamically need to pull the value from the web page and then use it as the actual result in the assertion message. For us, we know both received and expected. We don't need to pull anything from the page. We know all the values, so we don't use this, so I remove it as well. And matcher result actual, I replace with a received like this. Okay, so we simplified a lot. So that's the message that's gonna be printed. And the last final step, we need to return this message at the end. So let's look into documentation. That's the block of code that is used for the return. Copy it here and paste it right here, quick formatting. So what we need? The last three things we just don't need, remove it. All we need is just message and the result is it pass or fail. For the message I mentioned before, it is required to be a function. So let's convert it to a function. A message function that returns a message like this, so simple. Yeah, I believe that's it, and we can try this custom assertion. Should equal, let me copy the name, going back. And let's say I will use it for this test. I replace to equal to should equal, and let's run the test. Running the test, and test pass successfully. Now, let's try to fail it. For example, let's put 9 over here, and test failed. Look at this, and okay, something didn't work. The response was not printed for some reason. So let's check it out one more time over here. Okay, we forgot to add the most important part, which is a logger right here. So the message was printed, but not with the logs. I add –n –n to add a couple more spaces. Then plus, and let's add actual log to the message. Recent API activity, colon –n. And the log message will be logs like this. Okay, let's run it one more time. Okay, test failed, and now we see the logs. Let's open the report, open in default browser. Here we go, and everything is working as expected. So look, this is the error message. Let me show you just line by line. So this error message in the code is responsible for this line of code, okay? Then expected and received are these two line, expected and received. And then the last one, recent API activity, which is this one. Recent API activity, request details, and response details. If we scroll all the way down, we see the line of code where the issue actually happened. Now let's test the negative scenario using a .not. So I put .not right here, run, okay, test pass successfully. And when I add 10, it should fail. Okay, test failed, but we also don't see the logs. Why we don't see the logs? Going back to the custom expect. So we don't see the logs because this assertion actually passed. You see, it's actually running as to equal. And the initial value of flag was true. Since this assertion passed, it never went to the catch blog. So we never get the recent logs. And what Playwright actually does, so it returned the result of pass into the Playwright, but then somewhere internally in the Playwright, it looks into this, this is not flag. And if this is not flag was used, if it is true, then it just flips the result of the test and then fail the test. So what we can do in this case, we can add additional logic over here, something like this. If the this is not flag is true, it means that .not was used inside of the test. Then we want to add logs right here, like this. So quick reformat, let's try to run the test one more time. Okay, test failed. And right now logs, you can see logs were added successfully. Open in default browser and correct error message. Expected received not should equal expected. Expected not 10, received 10. So everything is working. And the log also was printed. All right, so moving on. We need to fix one more thing. So look, when we call should equal or should not equal or not should equal, well, I know it sounds weird, but we use the wording like this. So not should equal. It's highlighted in red squiggly. So why is that? Because the should not equal or should equal does not exist on match type. We need to add this custom namespace to the custom expect just to fix this issue. How to do that? I need to declare the global namespace. And the namespace name is PlayWriteTest. And within this namespace, there is an interface called Matchers. This interface requires two types. So first type is the type of the returned value. And let's name this type as R or return or you can actually put any name over here. You can put bananas and it's going to work as well. So R and then the type of the value under test. Let me put it T. And as like I said, it can be absolutely different type or different word. We just need to make a TypeScript happy. That's all it is about. Then we call this method, our custom should equal assertion. And then use this type, expected return type of T. Sorry, the value under test is type of T and return type of R, like this. So going back to the test and you see red squigglies is gone. And also when we declared this namespace, now the IntelliSense also would work. So look, if I remove the assertion, expect dot. And now for example, to, to be, to be attached and so on. So this is a standard PlayWrite assertion. But if I type should, now we have, here we go, this is our should equal guy. You call should equal and for example, should equal 10, like this. So run the test and test working successfully. Yeah, by the way, one more thing. Currently we're importing expect from the PlayWrite test. I don't know why it's working, maybe it's caching or something. But the correct way would be to import the expect from the custom expect. So I type utils and then custom expect. So that's the correct way of the import for the expect. Let's double check it. Okay, it's working, it's working as expected. So now what we can do is just simply take and replace all those to equal with should equal. Let's VS Code help us with this. So I replace to equal to should equal and I hit refresh. And I want to replace only within the small test that's back that TS. So let me hit this one. And yeah, I believe everything is replaced. Yeah, to equal was replaced into should equal. Let's run the small test real quick. Okay, let's run all the tests. One, two, three, four. Okay, all assertions passed, everything is working. And the same way now we can just update this assertion to be a custom assertion as well, right? So how can you do that? So we take just this assertion, should equal. I copy the entire thing as our template, paste it right here, and just rename. The name should be less than or equal. Should be less than or equal, then replace the name over here. And this assertion should be this one instead of to equal. Also, we need to add this assertion to the namespace. So IntelliSense will work right here. Expected T of the type R. And yeah, I believe that's it. Let's try if it's gonna work. Should be less than equal. Yeah, this assertion is available. And let's try to run this test. And new assertion is working. Let's try to run four and see if we see the logs printed. Okay, test failed, and here we go, we see the logs. Everything working, guys. So I believe this is it. Yeah, you see, it's quite intense, but we created custom assertions. And now we have a full logging if the test will fail. How cool is that? So let's make a quick review and summary what we did in this lesson. So inside of the fixtures, we added the setCustomExpectLogger method and passed the instance of the logger inside of the CustomExpect.ts file right here. Then inside of the CustomExpect.ts file, we extended the base expect object with a new capability. We created a new shouldEqual assertion that expect received and expected two values coming into. And then inside of the try-catch block, this is where the actual validation is happening. So we're calling the toEqual assertion. If the assertion pass, the pass flag is assigned, if assertion is failed, it triggers the error message. It goes into the catch block, and then logs are generated. Then we create a custom error message over here with all the standard details about the assertion, but at the end, we add the logs as well. And also, this assertion handles .not scenario. When .not is used in this kind of case, we also create the logs. And in both scenarios, now we have log-enhanced reporting. All right, guys, so if something was not clear, I know this is quite intense and complex topic, please reach me out in Slack. Other than that, see you in the next lecture.