Reports and evidence

Open the HTML report, read the events as JSON lines, and know what a run captures and keeps.

Everything a run saw stays in its run folder: events, the result, logs, screenshots, recordings and records of each page. Reports are built from those files.

The HTML report Link to The HTML report

Terminal
npx retest run --reporter htmlnpx retest report .retest/runs/<time>

Both write report.html into the run folder. --reporter html prints the terminal report as well. retest report builds one for a folder that already exists, finished or not.

  • Each failure comes first: the check, the locator, the page, both values, the wait, the screenshot and the lines of the test.
  • Every test shows its outcome and, as a separate fact, whether its evidence is complete, partial or unavailable.
  • The steps of each app are in time order. In a run that records, a click on a step moves the video to that moment.
  • The report is one file. It loads nothing from the network and opens from its file address, so you can zip the folder and send it.

Events as JSON lines Link to Events as JSON lines

Every run writes events.jsonl, one event per line, as things happen. --reporter jsonl prints the same lines to the terminal and nothing else.

Terminal
npx retest run --reporter jsonl
  • Every event carries schemaVersion: 1. New fields are optional, so a newer reader reads older run folders.
  • Each event says who reported it: origin: 'parent' for what Retest's own process saw, origin: 'child' for what the test file's process claimed.
  • The JSON Schemas for events and results ship in the package. eventSchemaUrl and resultSchemaUrl from @rehearsal-labs/retest/protocol point at them.
  • result.json is written once, at the end. A folder without it is a run that did not finish.

Screenshots Link to Screenshots

A failed test gets one screenshot of each app's page, after its checks. Each screenshot's record names its app, its session, when it was taken and which driver took it.

Recordings Link to Recordings

Recording is off by default. Turn it on in the config:

retest.config.ts
recording: {  record: true,  keep: 'failures',  fps: 10,  size: { width: 1280, height: 720 },},
  • record records every app. apps: { web: false } leaves one out.
  • keep: 'failures' removes the recordings of a test that passed. The default keeps all of them.
  • required: true makes an incomplete recording end the run as evidence_incomplete.
  • A recording that fails never turns a passing test into a failing one. Its gap is reported apart from the test's outcome.

Recording needs Retest's media process and ffmpeg on the machine. retest install media builds the media process from the source in the package, with Rust 1.88 or later. retest doctor checks both when the config asks for recording.

Frames are samples. A stretch with no frame never proves that nothing appeared on the screen.

Console and network Link to Console and network

On Chromium and WebKit pages, Retest records each test's console messages, uncaught errors and requests, in passing and failing runs alike. On Firefox it records requests only. The records are saved under diagnostics/ and counted in the reports.

  • No header and no body of a request or a response is kept. Queries and fragments are cut from every address.
  • Nothing in them fails a test unless the config asks. diagnostics: { strict: { runtimeErrors: true, httpErrors: true } } fails a test that otherwise passed.
  • A strict rule never passes on a capture that was not complete.

What a run keeps Link to What a run keeps

Screenshots, recordings and frames go under artifacts/, by attempt and by app. A file name holds no test title, no page text and no secret. Retest keeps everything, with two exceptions:

  • With keep: 'failures', a passed test loses its recordings, unless an AI check judged them.
  • A partial file a lost recording left behind is removed when the run ends.

What may be captured Link to What may be captured

Text redaction hides a secret in every text Retest writes, but it cannot reach pixels. So each app has two rules, screenshots and recordings, each allowed or never.

retest.config.ts
pixels: { web: { screenshots: 'allowed', recordings: 'never' } },
  • A capture the rules forbid is not taken, and the run says so where it would have been.
  • While a secret is typed into a field that shows its text, Retest stops capturing that app. It starts again once the field is gone, empty or masked, or the page opens another document.
  • An app can still show a secret on its own, such as on the next page. Keep run folders where the secrets they may hold can be kept.