From Cypress to Playwright - cost, migration steps, timeline and AI tools

Playwright has recently surpassed Cypress in NPM weekly downloads. After collaborating with dozens of teams, we compiled this guide and share what worked best for migration from Cypress to Playwright.

The cost of migration to Playwright

Engineers working on greenfield projects are already choosing Playwright, as evidenced by the npm trends.

But what about the existing projects that are already using Cypress? Many teams are intrigued by Playwright, but their testing suite is already written in Cypress. The transition seems scary and expensive - the costs include:

• Rewriting the "tests infrastructure": custom commands, shared utilities like authentication, and generation of test data
• Updating CI/CD pipelines
• Updating custom integrations and reporting
• Rewriting the existing tests
• Training the team on the new framework and concepts

The challenge is to estimate the cost of migration (including potential savings and benefits) and compare it with the status quo.

As an example, one of the teams we worked with used as a baseline the 8-month cost of running the tests in Cypress, including vendor fees and CI expenses, and then projected the costs of running the tests using Playwright. The estimation was based on a 60% reduction in CI resources usage (some teams reported a 70% reduction in test times).

Based on the number of tests, and complexity of the existing customizations, the team estimated that the migration would take ~3 months, with an initial 1-month high-intensity spike that included training, rewriting the core components and gradual migration of existing tests.

The "cost of ownership" in the charts below includes the costs of running the tests in the CI, the vendor fees, and the team's time translated to dollars.

Based on the estimation, it was clear to the team that migration would be benefitial in the long run and would pay off in 5 months.

The exact cost of migration depends on the size of the test suite, the complexity of the application, the team's experience and the available resources.

Apart from the cost, the team considered the non-monetary benefits like DX, ability to test variety of scenarios, simple syntax and industry trends.

After starting the migration process, the team was able to see the benefits of Playwright in the first weeks. The team used the opportunity to refactor the testing suite and get rid of redundant tests and technical debt. The actual migration took a few weeks longer than planned - the team realized that there was no immediate need to migrate all the tests. Low priority tests were postponed, and the team focused on the high-value scenarios.

You don't have to migrate all the tests at once but rather gradually. Once you invested in building the necessary infrastructure, new tests are written in Playwright, while the existing tests are migrated based on the available resources.

Migration Steps

After working with dozens of teams that went through the migration process, we've compilted the suggested steps to effectively manage the migration.

Step 1: Prepare the testing infrastructure

Most teams were able to initiate the migration by allocating a dedicated team of experienced engineers and doing a 2-4-week spike. The goal is to allow the rest of the team to start writing new tests in Playwright, including in the CI.

The activities include:

• Creating fixtures for often-used components like authentication, sessions management and populating test data
• Rewriting custom commands and integrations
• Migrating a few critical path tests using Playwright
• Setting up the CI to run the new tests

Step 2: write new tests in Playwright, migrate high-value tests

After the initial spike, the rest of the team can start writing new tests in Playwright. This approach allows the migration of the existing tests based on the available resources gradually. Embracing the need to run and maintain two testing suites implemented in different frameworks is cumbersome; you should make it clear to the team that this is a temporary state.

Playwright syntax is "just Javascript" with no special syntax - it has great compatibility with Typescript, IDEs, and AI assistants. As the team starts gaining more experience with Playwright it becomes more efficient (as mentioned by Business Insider and Firstbase) and productive within weeks because of convenience of troubleshooting and running tests right in IDE (VSCode plugin, UI Mode) and using Playwright Traces for troubleshooting CI failures.

Step 3: Fine tune the CI setup and reporting

As the amount of Playwright tests grows, you can start fine-tuning the CI setup and reporting. Playwright has built-in support for parallelization via sharding, and plenty of solutions for reporting and analytics.

Playwright ecosystem offers a variety of solutions for all kind of testing needs:

Feature	Cypress	Playwright
Parallelization	🔒 Vendor locked workaround cypress-split	Included: Test Sharding
Smart orchestration	🔒 Vendor Locked	Available with Orchestration for Playwright
Debugging	🔒 Vendor locked	Included: Screenshots, Videos, Traces ⚡️ with Currents Dashboard
Reporting & Analytics	🔒 Vendor locked Only high-level reports are available	3rd party providers available ⚡️ with Currents Dashboard
Accessibility Testing & Reports	🔒 Vendor locked	Included: Accessibility Testing
Flakiness analysis	🔒 Vendor locked	Included: Flakiness ⚡️ with Currents Dashboard
Code, CSS Coverage	Possible with a plugin	Included: Coverage
Visual regression testing	Possible with a 3rd party plugin	Included: Visual Comparison
Lighthouse reports	Possible with @cypress-audit/lighthouse	Possible with playwright-lighthouse
Component testing	Included Component Testing	Possible with Storybook Included in experimental mode
Tags	Run level only not supported for individual tests	Included: Tag Tests
BDD	Possible with cypress-cucumber-preprocessor	Possible with playwright-bdd
UI mode	Included Cypress App	Included: UI Mode
API testing	Included Cypress API Testing	Included: API Testing

Our team at Currents specializes in making the CI setup and reporting as efficient as possible; keep reading for details.

Step 4: Migrate the existing tests

The final step is to migrate the existing tests. In reality, teams often postpone migrating non-critical scenarios or "freezing" certain projects until more resources are available. Also, teams use the opportunity to refactor the testing suite and get rid of redundant tests and technical debt.

Team training and concepts

While there are plenty of online materials (and excellent documentation) on Playwright best practices, we wanted to highlight a couple of important concepts that the team needs to get familiar with for organizing Playwright testing suite or that are different from Cypress.

• Timing and DOM elements visibility - Playwright is much faster and is also strict about the visibility of the elements, tests that rely on the timing and were passing in Cypress might fail in Playwright. The solution is to use waits, network responses and assertions to ensure the elements are visible.
• Fixtures - Fixtures in Playwright is a powerful concept that you will need to become well familiar with - it can help you to manage the test data and the state of the application. There are some intricacies that you need to be aware of, like the scope and execution order of the fixtures.
• Custom commands - Playwright has a different API and syntax, so custom commands need to be rewritten using plain Javascript functions and async/await syntax. The team can use the opportunity to refactor the custom commands and make them more efficient.
• Concurrency - Playwright is faster and can run tests from the same specs in parallel, but it requires careful test isolation - ensure that tests are do not depend on each other. Running tests in parallel on multiple Workers required familiarity with the Playwright API and deciding on the best approach to storing the shared state (like login credentials or test data). See Auth Strategies for more details.
• DX and IDE support - Playwright has great support for IDEs and has an UI mode that allows running tests interactively. While the learning curve is comfortable, the team needs to get used to using the new tools and also to the ability to run and debug tests right from an IDE.
• CI optimizations - Playwright provides out-of-the-box support for parallelization of test in CI via sharding. However, the result can be suboptmimal, so you can use Playwright Orchestration to speed up the CI test execution.
• Extracting and merging test results - Playwight doesn't have NodeJS API (as of July 2024), so you need to use the Playwright CLI to run the tests and use custom reporters to extract the results for postprocessing. Currents can help you to get the results in a structured JSON format or can merged reports from parallel runs via REST API and to analyze the flakiness and performance of the tests.
• Tags - Playwright has a first-class support for tagging tests - it allows you to group tests by various criteria and run them selectively. It is recommended to use the opportunity of the migration to add tags. For example, tags can be used to mark tests reliability e.g. @new or @unstable, to mark webapp functionality like @onboarding or @checkout, or to designate ownership @team:growth or @team:platform (also see test annotations).
• Runtime - while Cypress runs in the same browser context as the application under test, Playwright is a standalone NodeJS process that communicates with browsers using developer protocol. While it is an implementation detail, remember that running any kind of custom code in browser requires sending serialized commands via WebSocket, see Evaluating Javascript for details.

Migration Timeline

We've seen that after the initial spike of preparing the infrastructure, teams were able to switch to low-intensity mode when all the new tests are written in Playwright, and high-value and important tests are being gradually rewritten within 2-3 months.

Depending on the size of the test suite, the complexity of the application and custom functionality attached the your tests, the team's experience and the available resources the timeline can vary.

AI tools for migration from Cypress to Playwright

AI agents are the perfect tools for facilitating the migration of your testing suite from Cypress to Playwright, however, we've seens that the AI agents are less successfull in migrating the custom commands and shared utilities.

Once you have the utilities, core components and a few example tests migrated, AI becomes much more efficient in migrating the rest of the tests. Generic popular AI agents like ChatGPT, LLAMA or GitHub Copilot were quite successful in converting the tests with minimal supervision.

A more comprehensive solution like Grit.io integrates with your codebase and gradually starts AI-driven refactoring process via a series of automatically created PRs.

Here's an example of how Grit helped a team to migrate from Cypress to Playwright:

There are a few free community tools that can help with automating the migration, however, we've seen limited success and adoption of those:

• Cypress to Playwright Converter by Ray
• Cypress to Playwright by Jose Cabrera
• Convert Cypress Syntax to Playwright by Playwright Team

How Currents can help?

Currents collaborated with dozens of teams of different sizes to facilitate the migration.

Our team at Currents has been working on providing the best-in-class cloud-based reporter for Playwright. Currents is not just a reporter - it is a platform that integrates with your testing suite and workflows, improving your testing suite's performance, reliability and efficiency.

• We help reduce CI execution and cost via improved orchestration for Playwright, unlocking the usage of cloud spot instances for E2E tests
• We provide a near-realtime streaming of step-level test results, together with screenshots, videos and traces for better debugging and troubleshooting
• We help improve the quality of your tests by proactively identifying the flaky tests, analyzing trends and providing actionable insights
• We integrate with your existing tools and workflows, like Slack, GitHub, GitLab, etc.

For companies migrating from Cypress to Playwright - you can run both Cypress (up to version 12.17.4) and Playwright, using Currents. We offer dedicated plans adjusted to facilitate the migration. Please reach out for more information.

Join the growing community of teams using Currents to run their Cypress and Playwright tests

Trademarks and logos mentioned in this text belong to their respective owners.