From Cypress to Playwright - cost, migration steps, timeline and AI tools
We collaborated wih dozen of teams that migrated from Cypress to Playwright. We discuss the cost of migration, migration steps and timeline, tips for smooth migration and adoption, use of AI tools to automate the migration.
Playwright has recently surpassed Cypress in NPM weekly downloads. After collaborating with dozens of teams, we compiled this guide and share what worked best for migration from Cypress to Playwright.
The cost of migration to Playwright
Engineers working on greenfield projects are already choosing Playwright, as evidenced by the npm trends.
playwright-core
is the core package of Playwright testing framework - it is a good indicator of the adoption and it exceeded cypress
in the number of NPM downloads in early 2024.
![Cypress vs playwright-core. Source: https://npmtrends.com/cypress-vs-playwright-core](/img/posts/migration-cypress-to-playwright/cypress-playwright-core-trends.png)
But what about the existing projects that are already using Cypress? Many teams are intrigued by Playwright, but their testing suite is already written in Cypress. The transition seems scary and expensive - the costs include:
- Rewriting the "tests infrastructure": custom commands, shared utilities like authentication, and generation of test data
- Updating CI/CD pipelines
- Updating custom integrations and reporting
- Rewriting the existing tests
- Training the team on the new framework and concepts
The challenge is to estimate the cost of migration (including potential savings and benefits) and compare it with the status quo.
As an example, one of the teams we worked with used as a baseline the 8-month cost of running the tests in Cypress, including vendor fees and CI expenses, and then projected the costs of running the tests using Playwright. The estimation was based on a 60% reduction in CI resources usage (some teams reported a 70% reduction in test times).
Based on the number of tests, and complexity of the existing customizations, the team estimated that the migration would take ~3 months, with an initial 1-month high-intensity spike that included training, rewriting the core components and gradual migration of existing tests.
The "cost of ownership" in the charts below includes the costs of running the tests in the CI, the vendor fees, and the team's time translated to dollars.
Based on the estimation, it was clear to the team that migration would be benefitial in the long run and would pay off in 5 months.
![Comparing the cost of a 3-month migration plan from Cypress to Playwright vs the status quo](/img/posts/migration-cypress-to-playwright/migration-cypress-to-playwright-cost.png)
The exact cost of migration depends on the size of the test suite, the complexity of the application, the team's experience and the available resources.
Apart from the cost, the team considered the non-monetary benefits like DX, ability to test variety of scenarios, simple syntax and industry trends.
After starting the migration process, the team was able to see the benefits of Playwright in the first weeks. The team used the opportunity to refactor the testing suite and get rid of redundant tests and technical debt. The actual migration took a few weeks longer than planned - the team realized that there was no immediate need to migrate all the tests. Low priority tests were postponed, and the team focused on the high-value scenarios.
You don't have to migrate all the tests at once but rather gradually. Once you invested in building the necessary infrastructure, new tests are written in Playwright, while the existing tests are migrated based on the available resources.
Migration Steps
After working with dozens of teams that went through the migration process, we've compilted the suggested steps to effectively manage the migration.
![Migration steps from Cypress to Playwright](/img/posts/migration-cypress-to-playwright/migration-steps.png)
Step 1: Prepare the testing infrastructure
Most teams were able to initiate the migration by allocating a dedicated team of experienced engineers and doing a 2-4-week spike. The goal is to allow the rest of the team to start writing new tests in Playwright, including in the CI.
The activities include:
- Creating fixtures for often-used components like authentication, sessions management and populating test data
- Rewriting custom commands and integrations
- Migrating a few critical path tests using Playwright
- Setting up the CI to run the new tests
Step 2: write new tests in Playwright, migrate high-value tests
After the initial spike, the rest of the team can start writing new tests in Playwright. This approach allows the migration of the existing tests based on the available resources gradually. Embracing the need to run and maintain two testing suites implemented in different frameworks is cumbersome; you should make it clear to the team that this is a temporary state.
The key to success is to keep improving the testing infrastructure, like custom commands and shared utilities, and to make it easy to write new tests.
Playwright syntax is "just Javascript" with no special syntax - it has great compatibility with Typescript, IDEs, and AI assistants. As the team starts gaining more experience with Playwright it becomes more efficient (as mentioned by Business Insider and Firstbase) and productive within weeks because of convenience of troubleshooting and running tests right in IDE (VSCode plugin, UI Mode) and using Playwright Traces for troubleshooting CI failures.
Step 3: Fine tune the CI setup and reporting
As the amount of Playwright tests grows, you can start fine-tuning the CI setup and reporting. Playwright has built-in support for parallelization via sharding, and plenty of solutions for reporting and analytics.
Playwright ecosystem offers a variety of solutions for all kind of testing needs:
Feature | Cypress | Playwright |
---|---|---|
Parallelization | đź”’Â Vendor locked workaround cypress-split | Included: Test Sharding |
Smart orchestration | 🔒 Vendor Locked | Available with Orchestration for Playwright |
Debugging | 🔒 Vendor locked | Included: Screenshots, Videos, Traces ⚡️ with Currents Dashboard |
Reporting & Analytics | 🔒 Vendor locked Only high-level reports are available | 3rd party providers available ⚡️ with Currents Dashboard |
Accessibility Testing & Reports | 🔒 Vendor locked | Included: Accessibility Testing |
Flakiness analysis | 🔒 Vendor locked | Included: Flakiness ⚡️ with Currents Dashboard |
Code, CSS Coverage | Possible with a plugin | Included: Coverage |
Visual regression testing | Possible with a 3rd party plugin | Included: Visual Comparison |
Lighthouse reports | Possible with @cypress-audit/lighthouse | Possible with playwright-lighthouse |
Component testing | Included Component Testing | Possible with Storybook Included in experimental mode |
Tags | Run level only not supported for individual tests | Included: Tag Tests |
BDD | Possible with cypress-cucumber-preprocessor | Possible with playwright-bdd |
UIÂ mode | Included Cypress App | Included: UI Mode |
APIÂ testing | Included Cypress API Testing | Included: API Testing |
Our team at Currents specializes in making the CI setup and reporting as efficient as possible; keep reading for details.
Step 4: Migrate the existing tests
The final step is to migrate the existing tests. In reality, teams often postpone migrating non-critical scenarios or "freezing" certain projects until more resources are available. Also, teams use the opportunity to refactor the testing suite and get rid of redundant tests and technical debt.
Team training and concepts
While there are plenty of online materials (and excellent documentation) on Playwright best practices, we wanted to highlight a couple of important concepts that the team needs to get familiar with for organizing Playwright testing suite or that are different from Cypress.
- Timing and DOM elements visibility - Playwright is much faster and is also strict about the visibility of the elements, tests that rely on the timing and were passing in Cypress might fail in Playwright. The solution is to use waits, network responses and assertions to ensure the elements are visible.
- Fixtures - Fixtures in Playwright is a powerful concept that you will need to become well familiar with - it can help you to manage the test data and the state of the application. There are some intricacies that you need to be aware of, like the scope and execution order of the fixtures.
- Custom commands - Playwright has a different API and syntax, so custom commands need to be rewritten using plain Javascript functions and
async/await
syntax. The team can use the opportunity to refactor the custom commands and make them more efficient. - Concurrency - Playwright is faster and can run tests from the same specs in parallel, but it requires careful test isolation - ensure that tests are do not depend on each other. Running tests in parallel on multiple Workers required familiarity with the Playwright API and deciding on the best approach to storing the shared state (like login credentials or test data). See Auth Strategies for more details.
- DX and IDE support - Playwright has great support for IDEs and has an UI mode that allows running tests interactively. While the learning curve is comfortable, the team needs to get used to using the new tools and also to the ability to run and debug tests right from an IDE.
- CI optimizations - Playwright provides out-of-the-box support for parallelization of test in CI via sharding. However, the result can be suboptmimal, so you can use Playwright Orchestration to speed up the CI test execution.
- Extracting and merging test results - Playwight doesn't have NodeJS API (as of July 2024), so you need to use the Playwright CLI to run the tests and use custom reporters to extract the results for postprocessing. Currents can help you to get the results in a structured JSON format or can merged reports from parallel runs via REST API and to analyze the flakiness and performance of the tests.
- Tags - Playwright has a first-class support for tagging tests - it allows you to group tests by various criteria and run them selectively. It is recommended to use the opportunity of the migration to add tags. For example, tags can be used to mark tests reliability e.g.
@new
or@unstable
, to mark webapp functionality like@onboarding
or@checkout
, or to designate ownership@team:growth
or@team:platform
(also see test annotations). - Runtime - while Cypress runs in the same browser context as the application under test, Playwright is a standalone NodeJS process that communicates with browsers using developer protocol. While it is an implementation detail, remember that running any kind of custom code in browser requires sending serialized commands via WebSocket, see Evaluating Javascript for details.
Migration Timeline
We've seen that after the initial spike of preparing the infrastructure, teams were able to switch to low-intensity mode when all the new tests are written in Playwright, and high-value and important tests are being gradually rewritten within 2-3 months.
![Migration plan from Cypress to Playwright](/img/posts/migration-cypress-to-playwright/migration-timeline.png)
Depending on the size of the test suite, the complexity of the application and custom functionality attached the your tests, the team's experience and the available resources the timeline can vary.
AI tools for migration from Cypress to Playwright
AI agents are the perfect tools for facilitating the migration of your testing suite from Cypress to Playwright, however, we've seens that the AI agents are less successfull in migrating the custom commands and shared utilities.
Once you have the utilities, core components and a few example tests migrated, AI becomes much more efficient in migrating the rest of the tests. Generic popular AI agents like ChatGPT, LLAMA or GitHub Copilot were quite successful in converting the tests with minimal supervision.
A more comprehensive solution like Grit.io integrates with your codebase and gradually starts AI-driven refactoring process via a series of automatically created PRs.
Here's an example of how Grit helped a team to migrate from Cypress to Playwright:
-
A team had 2,900 tests across 354 spec files, all tested with Cypress.
-
Instead of internally migrating these tests, their CTO set up the initial Playwright test environment and had Grit generate additional PRs to handle migrating tests and associated helpers.
-
Assuming scaffolding and configuration took 2 days and reviewing each PR took 30 minutes, they spent < 30 hours total on the project.
-
The actual work of migrating code, across 30 different PRs, would have taken an estimated 240 hours to do by hand.
![Using AI to convert Cypress to Playwright](/img/posts/migration-cypress-to-playwright/cypress-to-playwright-migration-ai.png)
There are a few free community tools that can help with automating the migration, however, we've seen limited success and adoption of those:
- Cypress to Playwright Converter by Ray
- Cypress to Playwright by Jose Cabrera
- Convert Cypress Syntax to Playwright by Playwright Team
How Currents can help?
Currents collaborated with dozens of teams of different sizes to facilitate the migration.
Our team at Currents has been working on providing the best-in-class cloud-based reporter for Playwright. Currents is not just a reporter - it is a platform that integrates with your testing suite and workflows, improving your testing suite's performance, reliability and efficiency.
- We help reduce CI execution and cost via improved orchestration for Playwright, unlocking the usage of cloud spot instances for E2E tests
- We provide a near-realtime streaming of step-level test results, together with screenshots, videos and traces for better debugging and troubleshooting
- We help improve the quality of your tests by proactively identifying the flaky tests, analyzing trends and providing actionable insights
- We integrate with your existing tools and workflows, like Slack, GitHub, GitLab, etc.
For companies migrating from Cypress to Playwright - you can run both Cypress (up to version 12.17.4
) and Playwright, using Currents. We offer dedicated plans adjusted to facilitate the migration. Please reach out for more information.
Join the growing community of teams using Currents to run their Cypress and Playwright tests
Trademarks and logos mentioned in this text belong to their respective owners.