Run tests in parallel

Retest runs test files at the same time, each in a process of its own. The tests of one file run one after another, and every test gets a fresh browser context.

Workers #Link to Workers

$ npx retest run --workers 4# one file after another$ npx retest run --workers 1

• --workers <n> sets how many files run at once. The default is half the machine's cores, at least one.
• Setups run first, one after another, so every saved state exists before a test starts from it.
• Workers share each target's browser. Each test opens its own context in it, so tests side by side keep their cookies apart.

Browsers #Link to Browsers

--browsers <n> sets how many browsers a target's tests are spread over, each worker keeping to one. The default is one browser for every three workers that have a file to run.

One browser serves all its pages from a single process, which many workers can fill. The report says how many each target used, as in started web=chromium Chrome 154 · 3 browsers.

Locks #Link to Locks

Tests in different files run at the same time. Two that share something outside the page, such as one inbox or one counter on a server, can disturb each other. A lock keeps them apart.

// retest.config.tslocks: ['inbox'],// a test filetest('reads the code from the inbox', { locks: ['inbox'] }, async ({ page }) => {})

• Two tests that hold a common lock never run at the same time, across workers and browsers.
• A test takes all its locks at once, or waits and takes none, so two tests never wait on each other.
• Waiting tests are served in the run's order. The wait counts against none of the test's budgets.
• test.describe passes its locks down, and a test's own add to them.
• A lock lasts one run. Two runs at once do not see each other's locks.

The report adds a line such as holds lock inbox, after waiting 2.1s under the test, and names who held it.

Desktops, simulators and data folders #Link to Desktops, simulators and data folders

A test holds everything it needs before it acts on any app:

• each lock it names;
• the Mac's desktop, for a macOS app;
• its simulator, for an iOS app;
• the data folder, for an Electron app with userDataDir.

• Every test takes these in the same order, all of one kind at once. So no two tests can each hold what the other waits for.
• Nothing launches for a test until it holds all of them.
• A test that does not get what it needs ends setup_failed. The failure names what it waited for and who held it.
• A desktop or a simulator comes back only once its app has ended. One that does not come free in time stays held, so no other test gets it while it may still be in use.

Sessions across runs #Link to Sessions across runs

A program that starts several runs can share one limit on browser sessions between them. A session is one browser context and its page.

import { runFiles, SessionBudget } from '@rehearsal-labs/retest/runner'const budget = new SessionBudget({ perOwner: 4, host: 8 })await Promise.all([  runFiles({ ...options, outputDir: first, sessions: { owner: 'worker-1', budget } }, []),  runFiles({ ...options, outputDir: second, sessions: { owner: 'worker-2', budget } }, []),])

No owner ever holds more than perOwner sessions, and all owners together never more than host. A test asks for a session for each of its apps at once, and never holds some while it waits for the rest.