My first thoughts on Playwright

I love automated testing.

When deciding which tests to prioritize, the infamous Twitter post from Guillermo Rauch at Vercel comes to mind:

Write tests. Not too many. Mostly integration.

— Guillermo Rauch (@rauchg) December 10, 2016

While I agree with this for the most part and write most tests using Vitest and React Testing Library, there is a huge benefit to having strong end to end tests. Especially for the mission-critical parts of your application. Like login, signing up for a new account, etc.

My past experience

I've used Selenium in the past. Mostly inherited from a legacy codebase with tests that no longer work consistently and are commented out to prevent flakiness in CI/CD.

Then Cypress came along and was a breath of fresh air. It just worked. It has a beautiful documentation site with loads of examples and plugins to do anything you want. It has some weirdness with how you define variables and have to reference them later with syntax like cy.get('@myInput') . But you could avoid that for most scenarios.

I didn't think I needed another end to end framework. But as part of my due diligence before setting up a test suite from scratch for my team at Sigma Squared, I checked out Playwright to see what the latest hotness could offer.

Using Playwright for the first time

Long story short: it was incredible.

Remember how I mentioned earlier that Cypress had plugins for everything you needed? Well Playwright has it built-in. And the API is trivial. And there's very little configuration needed, or packages to install for accomplishing the same thing in Cypress.

The test runner when running in UI mode is not as pretty as Cypress. But it has all of the same features and you get to see the test code running the automation in real-time. Which comes in handy when debugging your tests.

Setting up visual regression testing was as trivial as snapshot testing with Jest.

Setting up a GitHub Action to run the tests in CI/CD was trivial. Mostly just copy and paste. And they have an official Docker image if you want to have reproducible builds locally. Which I can see being valuable for teams using different platforms.

Setting up accessibility testing was pretty simple too. Just install a package from Axe and copy/paste some code to give you a method that you can call from your tests. They refer to this pattern as a "fixture".

The weirdness

The only thing I found thus far that was a little confusing was the fixture approach.

When I hear of a test fixture, I think of a factory object that I can use to mock data with. Playwright refers to a fixture as a method added to the test function's parameters that you wrap your tests in. You can see some examples in their fixture documentation.

Fixtures seem like the only approach to write reusable logic. So instead of writing a function to set up a test for a particular page, you'd extend the test function's params with a fixture in that test file and then call the method from the param as they show in the documentation. I say seem since I'm still new to this framework though. I still have a lot to learn on how they handle common use cases like this.

What's next

I still have to finish my diligence with a quick attempt to get the same thing going with Cypress. It's been about 6 months since I've last seen what they've been up to. Which is a very long time in the JavaScript ecosystem. Maybe they have the same features to compete!

Happy coding! SL