Rules for pull requests and deployments
Choose when Rehearsal runs your tests by itself, which tests it runs and which address it tests.
A rule tells Rehearsal to run tests by itself when something happens in your repository: a pull request opens, a branch changes or a deployment is ready. Each rule answers three questions: when to run, which tests to run and which address to test. It reports every run it starts as a check on the commit. With one rule, every pull request into main runs Parcel's release suite against its own preview before anyone merges it.
Before you start Link to Before you start
- The project has a connected repository.
- You have Manage integrations, which admins and owners have. Only people with it see Rules on the Pull requests page.
- The tests you want are ready, in a suite or among the app's tests. Create tests shows how they are made.
Add a rule Link to Add a rule
Step 1: Open the rules
Open Pull requests in the sidebar, press Rules at the top right, then Add a rule. The page asks three questions in turn, and a sentence at its top fills in with what the rule will do as you answer them.
Step 2: Choose when tests run
Under When should tests run?, pick one of three kinds of rule and write the branch it is about:
Option You write It covers When a pull request is opened or updated The target branch, such as mainPull requests into that branch When a branch is updated A branch, such as mainPushes and merges to that branch When a branch matching a pattern is updated A pattern, such as release/*Every branch the pattern matches In a pattern,
*stands for any characters, and the pattern has to match the whole name, sorelease/*matchesrelease/2.1but nothotfix/release/2.1. Only a pattern may hold a*. A pull request rule also offers Require passing tests before merging. Turn it on only when the rule tests The pull request's Vercel preview, or takes its address from GitHub Actions in a workflow that runs on every pull request, because only those pull request rules post their check, as Require the check before merging explains.Step 3: Choose the tests
Under Which tests should run?, pick a suite, or All approved tests, which is every test of the app that Rehearsal has checked and made ready. If the project has more than one address, first choose the one this rule is about under Address to test, and the suites on offer follow it.
Step 4: Choose the address
Under Which address should Rehearsal test?, choose where the tests open your app:
Option Use it with What Rehearsal tests The pull request's Vercel preview Pull request rules The preview Vercel deploys for the pull request's newest commit Vercel's deployment of the branch Branch and pattern rules Each deployment Vercel makes of a matching branch An address from GitHub Actions Pull request, branch and pattern rules The address your workflow sends once it has deployed, as Send an address from GitHub Actions shows. The same address every time Branch and pattern rules The address you write under Web app address The first two are one option, named for the kind of rule you picked.
Step 5: Save the rule
Press Save rule. The rule applies to what happens from then on, so a pull request that is already open gets its check the next time it changes.
The sentence at the top now reads the whole rule, for example "When a pull request into main opens or changes, run release on its Vercel preview and report the result in GitHub." Under it is the name the rule's check takes in GitHub: Rehearsal for a pull request rule, and the branch added for the other two kinds, such as Rehearsal · main or Rehearsal · release/*.
When tests start Link to When tests start
| Rule | Tests start |
|---|---|
| A pull request rule on the Vercel preview | When Vercel reports the preview of the newest commit ready. The check appears before that, as soon as the pull request opens or changes, and waits. If the preview was ready before Rehearsal heard of the pull request, as when you push a branch and open the pull request later, the tests start about 30 minutes after it opened, when Rehearsal looks for the preview. |
| A branch or pattern rule on Vercel's deployment | When Vercel reports a deployment of a matching branch ready. |
| A branch or pattern rule at the same address | As soon as someone pushes to a matching branch. |
| Any rule with an address from GitHub Actions | When your workflow runs the Rehearsal GitHub Action, after your deployment step. |
Preview deployments follows one pull request from its first push to the result.
A fixed address Link to A fixed address
With The same address every time, Rehearsal starts testing that address as soon as the push arrives. That is usually before your staging site has deployed the new commit, so the check may test the previous deploy. Rehearsal cannot tell which commit the address is serving, and the check says so: "Its deployed commit was not verified." Use a fixed address to keep an eye on a shared staging site that every merge to main updates, not to prove one commit.
Good to know
A fixed address runs from branch and pattern rules only. A pull request rule set to The same address every time never starts a run, so give pull requests a Vercel preview or an address from GitHub Actions.
Send an address from GitHub Actions Link to Send an address from GitHub Actions
When your app deploys somewhere other than Vercel, your own workflow knows the new address. The Rehearsal GitHub Action takes that address after your deployment step, starts the rule's tests against it and posts the check. It signs in with the identity GitHub gives each workflow run through OpenID Connect, so the repository holds no Rehearsal secret.
Step 1: Save the rule
Choose An address from GitHub Actions as the rule's address and press Save rule. The page stays open and shows the step to add under the question, with the rule's ID in it.
Step 2: Add the step to your deployment job
.github/workflows/deploy.yml permissions: contents: read id-token: writesteps: # Your deployment steps come first. - uses: rehearsal-labs/run@v1 with: url: ${{ steps.deploy.outputs.url }} sha: ${{ github.event.pull_request.head.sha || github.sha }} wait: true timeout: 30Replace
steps.deploy.outputs.urlwith the output of your own deployment step.shanames the commit that address serves, which is the pull request's newest commit or the commit that was pushed.id-token: writelets the step ask GitHub for the workflow's identity, andcontents: readkeeps checkout working, because a job that names any permission loses the rest. Keep any other permission your deployment needs.The step names no rule. Rehearsal uses the one Action rule that matches the push or pull request, and a rule that names a workflow file or a GitHub environment matches only that one. When none matches, the step fails and says so. When more than one does, such as a branch rule and a pattern rule that both cover
main, the step asks for one: addrule:underwith:, with the ID the page shows.Step 3: Allow the address
The address must use HTTPS and carry no sign-in, query or fragment. It may be on the host of the address the rule tests or one of its subdomains, so a rule about
https://parcel.exampletakeshttps://pr-12.parcel.example. For a preview on another host, add it to the rule under Extra preview hosts, optional, such asparcel-*.vercel.app. A*may stand only in the first part of a host name, and it never crosses a dot.
With wait: true, the default, the step waits for the run and succeeds only when every test passed, so later steps can depend on it. A failed, blocked or cancelled run fails the step, and so does reaching timeout, in minutes from 1 to 60, while the run is still going. With wait: false the step succeeds once the run has started, and the check carries the result. Either way the job summary links to the run, and the step sets run-id, run-url, status and conclusion for the steps after it.
- Run the step after your deployment answers, because Rehearsal starts testing the address at once.
- Skipping the step leaves no check on the commit, so a required check keeps the pull request waiting.
- Workflow file, optional and GitHub environment, optional hold the rule to one workflow, such as
.github/workflows/deploy.yml, or one GitHub environment. Left empty, any workflow whose event and branch match the rule may use it. - The Action works in
pushandpull_requestworkflows of the repository itself. Pull requests from forks, merge queues andpull_request_targetworkflows are refused. - A retry within one workflow attempt finds the run it already started. Re-running the workflow starts a new run and cancels the older one if it is still going.
Which tests a run checks Link to Which tests a run checks
- A suite runs its tests, and All approved tests runs every test of the app that Rehearsal has checked.
- The run fixes the version of each test when it starts, so a test that changes later changes nothing in that run. A test of the suite that is not ready yet is listed as Not ready on the check and keeps it from passing.
- A rule runs its tests in Chromium only, even when the suite names other browsers too. A suite set to more than one screen starts one run for each screen, and one check reports them all with the worst result among them. Each test then names its browser and screen on the check, such as "Check out a cart on Chromium phone".
- A suite can hold tests of another app in the project. When that app has more than one address, the rule does not say which one to test, so the check stops before any test runs and says it does not know where to test that app.
- Runs that rules start count towards your plan like any other run. When the workspace has used its runs or reached its spending limit, the check says so and no tests run, as When runs are used up explains.
One rule per situation Link to One rule per situation
Two rules cannot describe the same situation, which is the same kind of rule on the same branch for the same address. Rehearsal refuses the second one with A rule already covers that branch at this address., so change the rule you already have instead.
Pause, change or remove a rule Link to Pause, change or remove a rule
Each rule on the Rules page reads as its sentence, with its check's name, its last runs and three buttons:
- Pause stops the rule starting runs. It keeps its history and shows Paused until you press Resume.
- Edit opens the same three questions. A branch or pattern rule that moves to another branch reports under a new name from then on, such as
Rehearsal · staging. - Remove asks first, then stops Rehearsal checking that branch. Runs that already finished keep their results.