TL;DR

This article covers the Playwright practices that matter once a suite grows to hundreds or thousands of tests across parallel workers and CI machines, ordered by what breaks first. It goes from the CI block in the Playwright config and per-role auth through per-worker test data, balanced shards, merged reports, split PR and nightly runs, and tracking flaky tests.

Set forbidOnly on CI, and always set a global timeout in CI. Turn retries on in CI only, so a failure on your machine stays a failure while you debug it.

  1. Set forbidOnly on CI, and always set a global timeout in CI.
  2. Give each worker its own data before you raise workers.
  3. Turn on fullyParallel before you add shards, and merge blob reports from the day you add them.
  4. Treat flaky as a status you track and fix by impact, never one that retries clear.

At a few dozen tests, a checklist of good locators and web-first assertions carries a Playwright suite a long way. Once a suite has a few hundred tests spread across parallel workers and CI machines, the red builds stop coming from selectors. Take sharding: without fullyParallel, Playwright assigns whole test files to shards. In that case, if test files are unevenly sized, some shards may run significantly more tests than others, and some may run few or none.

Part of the problem is Playwright's defaults. Failing tests are not retried by default. A run also has no upper time bound by default, so a suite that hangs in CI gets killed by the runner mid-run and leaves no report at all, as the CI guide warns. Sharding splits the suite into smaller parts called shards, and in a CI pipeline each shard can run as a separate job.

Each of those failures has a config or workflow fix. A suite needs the fixes in the same order the failures show up.

Where the checklists stop

Most of what makes a single test reliable is settled advice, and Playwright's own best-practices page is the place to start. Locate elements the way a user finds them, by role, label or text, rather than with CSS tied to the DOM structure. Assert with web-first assertions instead of reading a visibility check once. Don't sleep for a fixed time either: Playwright's docs call tests that wait for time inherently flaky. Mock the third-party APIs you don't own, so an outage on their side doesn't fail your build.

Page objects are a code-organization choice, and a fine one. At scale, the fixture scope a page object is wired through matters more than the class itself, because the scope decides which tests share its state.

All of that makes one test trustworthy. None of it decides what happens when thousands of tests share one backend, one login and a handful of CI machines.

The CI block in playwright.config

A config that works on a laptop fails in CI in quiet ways: slower runs, a stray focused test nobody catches, a killed job with nothing to read.

Anti-pattern

Laptop config in CI

The config that grew up on a developer's machine runs unchanged in the pipeline. It traces every test because that helped while debugging, uses the same retries and workers everywhere, and sets nothing that bounds the run.

// Anti-pattern: one config for laptop and CI, a trace for every test,
// no guard against test.only() and no upper bound on the run
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  retries: 2,
  workers: 4,
  use: {
    trace: 'on',
  },
});

Playwright's best-practices page advises against tracing every test because it is very performance heavy. Nothing stops a stray test.only() from reaching the pipeline. Local failures get retried while you try to debug them, and a hung suite runs until the CI job kills it.

Fix

Put every CI-specific setting behind process.env.CI, the way the configuration docs set retries: process.env.CI ? 2 : 0.

import { defineConfig } from '@playwright/test';

const isCI = !!process.env.CI;

export default defineConfig({
  testDir: './tests',
  forbidOnly: isCI,
  retries: isCI ? 2 : 0,
  workers: isCI ? 1 : undefined,
  // Stop the run before the CI job's own time limit does
  globalTimeout: isCI ? 60 * 60 * 1000 : undefined,
  use: {
    baseURL: process.env.BASE_URL,
    trace: 'on-first-retry',
  },
});

forbidOnly makes the run exit with an error when any test or group is marked with test.only() or test.describe.only(). A forgotten focus fails the build where everyone sees it.

Playwright doesn't retry failing tests by default. Turn retries on in CI only, so a failure on your machine stays a failure while you debug it.

The 'on-first-retry' trace mode records a trace and keeps it only for a test's first retry. With retries on, you pay for tracing only on tests that already failed once.

workers defaults to half the logical CPU cores, so a laptop and a CI runner with different core counts run different numbers of workers from the same config. Playwright's CI guide recommends 1 worker in CI for stability and reproducibility. Start there, and raise it only after each worker has its own test data.

The CI guide also says to always set a global timeout in CI. Without one, a run has no upper bound, and a suite that hangs or slowly outgrows the job limit is killed mid-run without producing a report. Keep globalTimeout below the job's limit so Playwright's limit is the one that fires.

Per-role auth with setup projects

With the config settled, the next cost shows up in every test file: login, paid over and over.

Anti-pattern

UI login in beforeEach

Every test signs in through the login form before it does anything else. At twenty tests nobody notices.

// Anti-pattern: the slowest flow in the app runs once per test
import { test, expect } from '@playwright/test';

test.beforeEach(async ({ page }) => {
  await page.goto('/login');
  await page.getByLabel('Email').fill(process.env.ADMIN_EMAIL!);
  await page.getByLabel('Password').fill(process.env.ADMIN_PASSWORD!);
  await page.getByRole('button', { name: 'Sign in' }).click();
  await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
});

test('admin can open billing settings', async ({ page }) => {
  await page.goto('/settings/billing');
  await expect(page.getByRole('heading', { name: 'Billing' })).toBeVisible();
});

Across a few hundred tests, the login form runs a few hundred times per build. Every slow response or glitch on that page fails a test that was about something else.

Fix

Log in once per role in a setup project and save each session to a file. The auth guide recommends a setup project declared as a dependency of every testing project; that setup project always runs and authenticates before all the tests.

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  use: {
    baseURL: process.env.BASE_URL,
  },
  projects: [
    { name: 'setup', testMatch: /.*\.setup\.ts/ },
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
      dependencies: ['setup'],
    },
  ],
});

The setup file signs in as each role and writes one state file per role.

// tests/auth.setup.ts
import { test as setup, expect, type Page } from '@playwright/test';

async function signIn(page: Page, email: string, password: string, file: string) {
  await page.goto('/login');
  await page.getByLabel('Email').fill(email);
  await page.getByLabel('Password').fill(password);
  await page.getByRole('button', { name: 'Sign in' }).click();
  await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
  await page.context().storageState({ path: file });
}

setup('authenticate as admin', async ({ page }) => {
  await signIn(page, process.env.ADMIN_EMAIL!, process.env.ADMIN_PASSWORD!, 'playwright/.auth/admin.json');
});

setup('authenticate as user', async ({ page }) => {
  await signIn(page, process.env.USER_EMAIL!, process.env.USER_PASSWORD!, 'playwright/.auth/user.json');
});

Keep the state files in playwright/.auth and add that directory to .gitignore, as the auth guide recommends. A state file is a signed-in session, and a session doesn't belong in the repo.

With several roles, set storageState per test file or group instead of in the config. Each file then picks the role it tests:

// tests/billing.spec.ts
import { test, expect } from '@playwright/test';

test.use({ storageState: 'playwright/.auth/admin.json' });

test('admin can open billing settings', async ({ page }) => {
  await page.goto('/settings/billing');
  await expect(page.getByRole('heading', { name: 'Billing' })).toBeVisible();
});

One shared login per role stops holding up once tests change server-side state, because every worker signed in as the admin edits the same admin's settings. For those tests, Playwright recommends authenticating each parallel worker once. The tool for that is a worker-scoped fixture, the same kind that gives each worker its own data.

Per-worker test data

Shared data is where parallel runs start to look random.

Anti-pattern

One seeded user for the suite

The suite seeds a single test user, and every test that touches a cart or a profile edits that one user's data. Each test passes on its own. Across parallel workers, two tests that create or edit the same record race with each other. The failing test changes from run to run, so the failure gets filed as flaky.

Fix

Give each worker its own user. Playwright exposes the worker index as testInfo.workerIndex and process.env.TEST_WORKER_INDEX, and the parallelism docs suggest using it to isolate user data in the database between workers, with every test in a worker reusing the same user.

Pick the fixture scope by cost. Playwright tears down test-scoped fixtures after each test, and worker-scoped fixtures only when their worker process is torn down. Put expensive setup, like a seeded account, in worker scope and per-test state, like a cart, in test scope.

In the fixtures file below, ./test-data stands for your own helper module that talks to the database.

// tests/fixtures.ts
import { test as base } from '@playwright/test';
import { createCart, deleteCart, upsertUser, type Cart, type User } from './test-data';

export const test = base.extend<{ cart: Cart }, { workerUser: User }>({
  // Worker scope: one user per worker, reused by every test that worker runs
  workerUser: [
    async ({}, use) => {
      // Parallel index, not worker index: a replacement worker keeps the same parallel index
      const index = process.env.TEST_PARALLEL_INDEX;
      // Upsert, not insert: this runs again whenever Playwright replaces the worker
      const user = await upsertUser(`e2e-worker-${index}@example.com`);
      await use(user);
    },
    { scope: 'worker' },
  ],

  // Test scope: a fresh cart per test, deleted when the test ends
  cart: async ({ workerUser }, use) => {
    const cart = await createCart(workerUser.id);
    await use(cart);
    await deleteCart(cart.id);
  },
});

export { expect } from '@playwright/test';

Tests import test from the fixtures file and ask for what they need:

// tests/cart.spec.ts
// Relative paths need use.baseURL, which the CI block's config sets
import { test, expect } from './fixtures';

test('adds a product to the cart', async ({ page, cart }) => {
  await page.goto(`/cart/${cart.id}`);
  await page.getByRole('button', { name: 'Add sample product' }).click();
  await expect(page.getByTestId('cart-count')).toHaveText('1');
});

Worker restarts

Worker-scoped setup doesn't run once per run. When a test fails, Playwright discards the whole worker process along with its browser, starts a new worker and continues with the next test. The new worker sets up its worker-scoped fixtures again, mid-run, against a database the old worker already wrote to. Make that setup create-or-reuse, as the upsert above does, and never assume an empty table.

Assertions stay in tests

An assertion buried in a fixture fails during setup, far from the test that depended on it, and the report points at the wrong place. Keep fixtures to supplying state, and put every assertion in the test that needs it.

Environment config

Decide how each CI job sets backend URLs and credentials before parallel workers depend on them, and use one mechanism in every job. Don't wait for an official convention. When a user asked for guidance on handling .env files, a maintainer replied that managing the environment falls outside Playwright's scope and pointed to the dotenv package (#34393).

Balancing shards

Once one machine is too slow, adding machines is the obvious move. Without the fullyParallel setting, Playwright assigns entire test files to shards.

Playwright can only shard tests that can run in parallel, as the sharding docs put it. By default, test files run in parallel while the tests in one file run in order in the same worker, so by default Playwright shards whole files. Without fullyParallel, each shard gets whole files, not individual tests.

Anti-pattern

Four shards over a few big files

The workflow gets a four-job matrix and a --shard flag, over a suite whose tests sit in a handful of large spec files, with fullyParallel off in the config.

# Anti-pattern: four shards over a few large spec files with fullyParallel off.
# Whole files are assigned to shards, so with uneven files some shards may run far more tests than others.
name: E2E
on: pull_request

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test --shard=${{ matrix.shard }}/4

When test files are unevenly sized, some shards may end up running significantly more tests than others, while others may run fewer or even none. The run lasts as long as its slowest shard, so the extra machines buy little.

Fix

Turn on fullyParallel. Sharding then works at test-level granularity, and each shard tries to balance the number of individual tests it runs. The matrix stays as it is.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  // Shard by test, not by file
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  use: {
    trace: 'on-first-retry',
  },
});

What fullyParallel costs

Each shard balances how many tests it runs, not how long they take, so a cluster of slow tests can still land on one shard. Tests from one file can also end up on different shards, where they no longer run in order in one worker. Any test that relies on an earlier test in its file turns into a flake. Make every test independent first, then turn fullyParallel on.

If count-based balancing still leaves one shard running long, Currents Test Orchestration replaces the up-front split with a live queue that hands pending tests to each CI machine as it becomes free, and the orchestrator runs Playwright with the same configuration.

Traces from every shard

A sharded run that fails is only as debuggable as the artifacts it kept. On a laptop you can rerun a failing test with the debugger attached. In CI, the sharding docs' GitHub Actions example uploads each blob report to the GitHub Actions Artifacts, which makes it available to other jobs in the workflow. For CI failures, Playwright recommends the trace viewer over videos and screenshots, so the trace is the artifact to keep.

Anti-pattern

Per-shard HTML reports

Each shard writes its own HTML report, and the upload step runs only when the job succeeds. The team gets up to four partial reports to click through. GitHub Actions applies a default status check of success() to a step unless it includes one of the status check functions, so the shard that failed skips its upload.

Fix

Write a blob report on every shard. A blob report holds every test that ran, its results and all its attachments, traces and screenshot diffs included. Upload it whether the shard passed or not, then merge all the blob files in a final job. The merge job puts them in one directory, and the sharding docs note that blob report names contain the shard number, so they don't clash.

name: E2E
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    # Above the config's 60-minute globalTimeout, so Playwright stops first
    timeout-minutes: 75
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --reporter=blob
        env:
          BASE_URL: ${{ secrets.BASE_URL }}
      # Upload even when tests failed: the failing shard's traces matter most
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report
          retention-days: 7

  merge-reports:
    if: ${{ !cancelled() }}
    needs: [test]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - uses: actions/download-artifact@v4
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - run: npx playwright merge-reports --reporter html ./all-blob-reports
      - uses: actions/upload-artifact@v4
        with:
          name: html-report
          path: playwright-report
          retention-days: 14

The merge job turns the shards' blob files into one HTML report for the whole run. Start every CI failure from the failing test's trace. Keep the job's timeout-minutes above the config's globalTimeout, so a hung shard hits Playwright's limit before the CI job's.

Splitting PR and nightly runs

Once the full suite takes longer than anyone will wait on a pull request, it shouldn't run on every pull request.

Tag tests so each run can pick its set. Tag a test with a details object when you declare it, or with an @-token in its title; either way, every tag must start with @.

// tests/checkout.spec.ts
// Relative paths need use.baseURL, which the CI block's config sets
import { test, expect } from '@playwright/test';

test('checkout with a saved card', { tag: '@critical' }, async ({ page }) => {
  await page.goto('/checkout');
  await page.getByRole('button', { name: 'Pay now' }).click();
  await expect(page.getByRole('heading', { name: 'Order confirmed' })).toBeVisible();
});

test('export order history as CSV @reports', async ({ page }) => {
  await page.goto('/orders');
  await page.getByRole('button', { name: 'Export CSV' }).click();
  await expect(page.getByText('Export ready')).toBeVisible();
});

Tags are where E2E coverage priorities become concrete. Tag the paths that lose money or users when they break as critical, run those on every pull request, and run everything on a nightly schedule. For a tag set that holds up across thousands of tests, see how to organize tags in a large CI suite.

# Pull requests: critical paths only, with the failure cap on
PR_RUN=1 npx playwright test --grep @critical
# Nightly schedule: the full suite
npx playwright test

Install only the browsers your projects use: npx playwright install chromium installs only Chromium.

Cap failures on the PR run. With max failures set, Playwright stops after that many failed tests and skips the tests that haven't run yet, so a build the change broke stops spending CI minutes.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  forbidOnly: !!process.env.CI,
  // Stop a pull-request run after 10 failures; nightly runs report everything
  maxFailures: process.env.PR_RUN ? 10 : undefined,
});

Tags aren't quarantine

Don't tag a flaky test out of the PR run unless something tracks it. Moving it to the nightly set, or marking it skipped or fixme, takes it out of sight, and a test nobody watches stays out for good.

Tracking flaky tests

Retries turn a red build green, and that is exactly where flakes hide.

With retries on, Playwright reports a test that failed on its first run and passed on a retry as flaky rather than failed, as the retries docs describe. By default, Playwright does not fail the run when a test is flagged as flaky, as the CLI options table shows for --fail-on-flaky-tests.

Anti-pattern

Retries until green

Each time a flake turns the build red, someone raises retries by one. The build goes green and nobody reads the flaky label. Every retry adds run time, and a real race condition in the app passes on its second or third attempt and ships.

Fix

Keep retries on in CI, and add a scheduled job that fails on flakes. failOnFlakyTests makes the run exit with an error when any test is marked flaky. Turn it on through an environment variable that only the flake-check job sets.

import { defineConfig } from '@playwright/test';

const isCI = !!process.env.CI;

export default defineConfig({
  testDir: './tests',
  retries: isCI ? 2 : 0,
  // Only the scheduled flake-check job sets FLAKE_CHECK
  failOnFlakyTests: !!process.env.FLAKE_CHECK,
  use: {
    trace: 'on-first-retry',
  },
});

Run the flake-check job nightly against the full suite and treat its failures as the fix list.

Fix order

Fix flakes by impact, meaning how often each one flakes and how often it runs, not by which one annoyed someone last. A slightly flaky test that runs often can be a bigger problem than a very flaky test that rarely runs. A test on the pull-request path runs far more often than one in the nightly set, so it usually goes first. Currents Test Explorer records each test's flakiness rate, the share of its executions with a non-deterministic outcome, and shows how its behavior changes over time, so the fix list comes from data instead of memory.

App-side flakes

Some flakes are bugs in the app, and no wait or retry fixes them. Tests against one prerendered Blazor app failed because Playwright interacted with the page before the app was interactive (#33156). A maintainer replied that this is a bug in the page, and the reporter later fixed it by adding a loading indicator to the app. When a flake traces back to the page, fix the page rather than lengthen the wait.

When these practices don't pay

If the whole suite finishes inside one CI job's time budget on one machine, the CI block and a setup project are all it needs. Sharding would add a merge job and artifact plumbing for little gain.

One worker in CI trades speed for reproducibility, which is why the CI guide recommends it. If one worker is fast enough, stay there until run time, not habit, forces more workers or more shards.

A test that depends on another test's side effects works when tests run in order, but breaks the moment they run in parallel or in a different order. Turning on fullyParallel there swaps an uneven-shard problem for a flake problem. Fix the dependencies first.

failOnFlakyTests exits with an error on any flaky test, including one the change didn't cause, so a pull-request gate with it on blocks merges for reasons outside the change. Keep it on the schedule.

A CI-ready baseline

Every suite that runs in a pipeline should start with two things: the CI block (forbidOnly, retries on CI only, traces on first retry, a global timeout) and a setup project per role. Neither costs much on a small suite, and both stop the first failures a growing suite hits.

The rest comes in a fixed order. Give each worker its own data before you raise workers. Turn on fullyParallel before you add shards, and merge blob reports from the day you add them. Treat flaky as a status you track and fix by impact, never one that retries clear.

Frequently asked questions

What are the best practices for running Playwright tests in CI?

Put every CI-specific setting behind process.env.CI, the way the configuration docs set retries: process.env.CI ? 2 : 0. Playwright's CI guide recommends 1 worker in CI for stability and reproducibility. The CI guide also says to always set a global timeout in CI, as the CI block in playwright.config explains.

Should Playwright retry failed tests in CI?

Yes, but only in CI. Playwright doesn't retry failing tests by default, so turn retries on in CI only, and a failure on your machine stays a failure while you debug it. The config is in the CI block in playwright.config.

How many workers should Playwright use in CI?

Playwright's CI guide recommends 1 worker in CI for stability and reproducibility. Start there, and raise it only after each worker has its own test data, as the CI block in playwright.config explains.

How do I avoid logging in before every Playwright test?

Use a setup project: Playwright's auth guide recommends a setup project declared as a dependency of every testing project, and that setup project always runs and authenticates before all the tests. With several roles, set storageState per test file or group instead of in the config, as per-role auth with setup projects shows.

How do I keep test data from clashing across Playwright workers?

Give each worker its own user. Put expensive setup, like a seeded account, in worker scope and per-test state, like a cart, in test scope. The fixtures are in per-worker test data.

Why are my Playwright shards unbalanced?

Without the fullyParallel setting, Playwright assigns entire test files to shards. When test files are unevenly sized, some shards may end up running significantly more tests than others, while others may run fewer or even none. The fix is in balancing shards.

How do I get one report from sharded Playwright runs?

Write a blob report on every shard, then merge all the blob files in a final job. The merge job turns the shards' blob files into one HTML report for the whole run, as the workflow in traces from every shard shows.

How do I stop retries from hiding flaky Playwright tests?

Keep retries on in CI, and add a scheduled job that fails on flakes. failOnFlakyTests makes the run exit with an error when any test is marked flaky, so turn it on through an environment variable that only the flake-check job sets. The config is in tracking flaky tests.