Test Retries | Bondar Academy
Course: Playwright UI Testing with TypeScript
Module: Advanced Tricks and Techniques
Instructor: Artem Bondar
Lesson Summary
In this lesson, we explore the retry feature of Playwright , which automatically retries failed tests to enhance the stability of test suites. This feature is particularly useful for failures caused by temporary issues, such as elements not being found. High-Level Behavior of Playwright When the retry feature is: Off: Playwright runs tests in a new worker (a clean browser session). If a test fails, a new worker is created for subsequent tests. On: If a test fails, Playwright retries the failed test in a new worker, ensuring a clean environment for the retry. Configuring the Retry Feature To configure retries in your Playwright framework: Open the playwright.config.ts file. Set the retries option. For example, to enable one retry locally, set: retries: 1 To run tests sequentially, set parallel to false . Overriding Retry Settings You can override global retry settings for specific tests using: test.describe.configure({ retries: 2 }) This allows you to specify different retry counts for flaky tests. Additional Control During Retries To perform actions before a retry (e.g., cleaning the database), use: if (testInfo.retry) { /* cleanup code */ } Summary The retry mechanism in Playwright enhances test reliability by isolating retries in new workers. Configuration is done in playwright.config.ts , and overrides can be applied at the test level. Additionally, you can manage preconditions for retries using the testInfo object.
Video Transcript
In this lesson, we will talk about retry feature of the playwright. So this feature will automatically retry a failed test and will make the execution of the entire test suite more stable that way. If your test, for example, failed because of the element was not found, playwright will make a retry one more time to make sure that this failure is not related to actual failure of the test, that it's related to some temporary glitch or something like that. And in this lesson, we will talk about it in details how to configure a retry inside of your framework. So let's get into it. So let me show you first overall high level behavior of the playwright when the retry is on and when the retry is off. So when the retry is off, the worker is initiated and playwright run the test within this worker. Run test number one. If it pass, run test number two. If it pass, run test number three. So what is worker? Worker is in a high level is a new instance of the browser running in the incognito mode. So it's a clean session of the web browser to make sure that tests run in clean and fresh session. But playwright by default try to reuse the existing worker just to save the time and save the resources on the machine. But let's see what happens if the test fails. So worker is initiated. First test is pass. Second test is fail. And when the second test is fail, playwright creates a second worker and the third test will be executed in a new worker to make sure that test will be executed in the completely clean session. So this is when the retry is off. So let's see what playwright gonna do when retry is on. Worker is initiated. First test pass. Second test fail. And when the second test fail, playwright create a second worker and retry execution of the test number two that failed in the worker number one. So it tries to run this test again. And if in this case test pass, then the next test executed and so on. So retry mechanism is pretty simple. It retries the test that fails. And to make sure that the test will be executed in the clean environment, when the test fails, playwright initiate a new worker to run the test in the clean browser session. So now let me show you how the retry actually works and how to configure it in the framework. So this is our PW practice app. And let's say we will use this UIComponents.spec.ts. Here we have many tests that we created before. And let's take the first one. The first test describe section forms layout page. And inside of this test describe, we have two tests, input fields and radio buttons. So let me run these tests, just these two tests. And I put test describe only because I want to run only these tests. And I intentionally fail the assertion for this test. For example, here input value to expect com 1. So this assertion should fail. And I want that playwright retry this test because the assertion failed. And I want to give a second attempt. Hey, try one more time. Maybe something went wrong. How to configure the retry? If we go to playwright config file over here, already default configuration defined right here. Retries column. And then we have process in VCI and some unknown 2 and 1. So what does it mean? This is a condition. When we want to run in the continuous integration, when this expression is true, we want retry two times. Otherwise, zero retries. So if we run on our local computer, we don't want to make retries at all. So in order to activate retries running locally, we need to change this number to one. Or we can just completely remove the entire expression and put just one. But I will leave the default settings because it can be useful when integration with CI gonna happen. So I put one over here. It means that when we run the test and if the test fail, we want to retry it just once. And in order to use just a single worker, I temporarily disable this parallel to false to make sure test will be executed one by one using the same worker. And now what we need to do is trigger the test using the command line. Just for the speed up, I will copy the command from the package.json from the previous lesson. And all I need to do here is just replace the name of the file. UIComponents.spec.ts. All right, and let's run it. So two tests are running. We can see it. We have a delay here of half seconds. So we see retry number one. It's trying to run forms layout page input fields test second time. And it failed. It's trying to run the second test and the second test path. So we see in the test result over here, one test failed and one test passed. And we know that this test was retried here. So we see the test result. This is the first result. Assertion failed. Then it made another retry. And this is the result of the second retry. So this worked perfectly fine. Test retried successfully. If you need to override the retry for some tests or group of tests, you can do it inside of the test body itself. For example, in the playwright config, we configured the global retry is just one. We want every failed test to be retried just once. But let's say that this block of tests, these two tests, are more flaky than the other ones. So we want to retry them two times. So we can override the settings here. So we type test.describe.configure. And we type retries and put here two. And we type retries and put here two. This entire describe block and all the tests that are located inside of the describe block, test number one and test number two, will be retried two times in case tests will fail. So let's run it one more time just to demonstrate. Retry number one. Retry number two. And test failed. The second test is executed and the same result for the third test. And test failed. The second test is executed and the same result. First test failed, second test passed. So we were able to override default retries with this configuration. And also sometimes if test failed and you need to retry it, you may need to run certain precondition or clean the database before the second retry or something like that. You also can do this. So here inside of the async page, you need to add a second argument called test info. And then inside of the test, you can create a condition, something like this. If testinfo.retry, then do something. For example, before the next retry, you want to clean the database. And here in this section, you can do this. And before the next retry, you want to clean the database. And here in this section, you can put the code that is responsible for database cleanup before the next retry. So this is also an option how you can control the retry inside of the runtime. All right, so let's quickly summarize what we did in this lesson. Retry is the mechanism in the Playwright that can retry a failed test. Every time test is retried, Playwright will create a new worker to make sure that test runs in complete isolation. The way to configure retry is inside of playwright.config.ts. You add retries option and add the counter how many retries you want to do if the test failed. Also, you can override this counter additionally inside of the test, providing the settings testDescribeConfigure and provide retries. In this case, the test inside of this describe block will use more retries compared to what's defined in the config.ts. And also, you can control retry during the test run. If you need to clean up the database before the next retry, you can use condition testInfor.retry. All right, that's it, guys, and see you in the next lesson.