Checks on commits

Read the check Rehearsal posts on each commit, run it again from GitHub and require it before merging.

Every run a rule starts appears as a check on the commit in GitHub, beside your other checks. The check follows the run from queued to finished, says how each test did and links to the run in Rehearsal. Once you require it, a pull request cannot merge until its tests pass.

The check's name Link to The check's name

The check takes its name from the rule, so you can require it by that name in GitHub. A pull request rule reports as Rehearsal. A branch or pattern rule adds its branch, such as Rehearsal · main or Rehearsal · release/*, and a branch rule moved to another branch reports under the new name from then on.

What a check says Link to What a check says

The check's title says how the run ended, and its conclusion is what GitHub reads to decide whether the check passed:

The runThe check's titleConclusion
Every test passed3 tests passedsuccess
A test failed1 of 3 tests failed: Check out a cartfailure
A page changed under a testThe page changed. Someone needs to accept it.failure
Tests could not run1 of 3 tests could not run, or the reason, such as a rule that does not say where to test an appaction_required
The run was cancelledThis run was cancelledcancelled
Rehearsal could not run itCould not run these tests. You were not charged.action_required

Only a run in which every test passed ends as success. A run whose tests could not run never counts as a pass, so a required check keeps the pull request from merging until someone looks.

Under the title, the check names the browser and the address it tested, then lists each test with its browser, result and time. It looks like this on GitHub:

Output
1 of 3 tests failed: Check out a cartOpen the run to see where each test stopped.Browser: ChromiumTested https://parcel-git-fix-cart.vercel.appTest                Browser   Result   TimeCheck out a cart    Chromium  Failed   22sSave a card         Chromium  Passed   14sInvite a teammate   Chromium  Passed   17sSee the run
ResultWhat it means
PassedThe test passed.
FailedThe test stopped at a step. The run shows which one and what the page showed.
Page changedSomething the test uses is no longer where it was. Someone decides whether the app changed on purpose, as Read a run explains.
Could not runSomething kept Rehearsal from checking the test, so the result says nothing about your app. The run in Rehearsal shows what it was.
CancelledThe run stopped before it reached this test.
Not readyThe test was not ready when the run started, so it was not checked.

A test marked Could not run is the one the run in Rehearsal marks Blocked, as A blocked test explains. It is not the same as a check titled "Could not run these tests. You were not charged.", which means Rehearsal itself failed to finish the run, as Runs we could not finish below explains.

See the run at the foot of the check, and the check's details link in GitHub, open the run in Rehearsal with its recording and steps. Opening it needs a Rehearsal sign-in with access to the project, and the check carries no other link. A rule runs its tests in Chromium only. When its suite runs on more than one screen, the rule starts one run for each screen, and one check reports them all with the worst result among them. Each test's line then names its browser and screen, such as "Check out a cart on Chromium phone".

A check of a fixed address adds that its deployed commit was not verified. A check can also end before any test runs, and When no tests run lists those endings.

Runs we could not finish Link to Runs we could not finish

Sometimes a run stops because of a problem on our side. Its check ends as action_required, says you were not charged, and counts towards nothing on your plan. Re-run the check when you are ready, and Runs we could not finish explains the rest.

Run a check again Link to Run a check again

Re-run a Rehearsal check in GitHub, and Rehearsal runs the same test versions again on the same commit, at the same address. The new run posts a check of its own, because GitHub never reopens a finished check.

  • The check has to have reported a run, and that run has to have finished. A check still waiting for a deployment has nothing to run again yet.
  • A check that closed without a run has nothing to repeat, so re-running it does nothing. Examples are a check that waited too long for a deployment, one whose deployment failed, one whose rule does not say where to test an app, and one that ended because the workspace had used its runs or reached its spending limit. A new commit starts a new check, so push one once the cause is fixed.
  • Re-running every check on a commit runs each Rehearsal check on it again.
  • The new run counts towards your plan, like any other run.

Good to know

Run again on the run in Rehearsal runs the same tests too, but that run posts no check. Re-run from GitHub when the pull request should see the new result.

Require the check before merging Link to Require the check before merging

GitHub is what blocks a merge, so requiring the check is a setting in GitHub. Rehearsal shows you the steps and then reads GitHub's rulesets back, so the rule can say whether merging is really blocked.

A pull request rule that tests The pull request's Vercel preview posts its check on every pull request. Its check, Rehearsal, appears as soon as a pull request opens or changes, so GitHub always has a check to wait for. A pull request rule with an address from GitHub Actions posts its check when your workflow runs the Rehearsal GitHub Action, so it holds a merge only when that workflow runs on every pull request. Branch and pattern rules post their checks on the commits pushed to their branches, under names of their own, and they have no switch.

Require Rehearsal only for a pull request rule on the Vercel preview, or one with an address from GitHub Actions whose workflow runs the Action on every pull request. A pull request rule set to The same address every time never starts a run, and an Action rule posts its check only when the Action runs. If GitHub requires a check that never comes, every pull request into that branch waits.

  1. Step 1: Ask for it in the rule

    In a pull request rule that tests The pull request's Vercel preview, or one with an address from GitHub Actions, turn on Require passing tests before merging and save the rule. Only a pull request rule has the switch, because only a pull request has a merge to block. Under the switch, Set this up in GitHub lists the next steps.

  2. Step 2: Require the check in GitHub

    • Open the repository's settings in GitHub.
    • Open Rules and choose the ruleset for your target branch, such as main, or add one.
    • Require status checks and add Rehearsal.
  3. Step 3: Let Rehearsal read it back

    Back in Rehearsal, open the rule from Rules with Show the steps or Edit, and press Check now. Check now reads the rulesets that apply to the branch, and when one of them requires the check the rule says GitHub requires the Rehearsal check. with when it last looked. Its sentence then ends "and require passing tests before merging."

Until GitHub confirms it, the rule says Merging is not blocked yet. Make "Rehearsal" required in GitHub., on the rule's page and on its card in Rules. Rehearsal reads the rulesets only when you press Check now or Check again, so press it again after you change them. A ruleset that ties the check to another app does not count, because that app could pass it instead of Rehearsal.

Every check in Rehearsal Link to Every check in Rehearsal

Pull requests in the sidebar lists the runs your rules started, newest first, grouped by pull request and by branch. Each group starts with its newest run's result, its number and branch, and how many runs it holds, so the left edge of the list says which pull requests passed. A pull request still waiting for its preview has a quiet line reading Waiting for a deployment, and each run's menu has Open the check on GitHub.

The search box finds a commit, the counts above the list narrow it by how the runs ended, and Filter narrows it by pull request or branch, by rule and by when the run started.

Next steps Link to Next steps