# Results and exit codes

> Interpret test statuses, failure classes, result fields and process exit codes.

Source: https://rehearsal.dev/retest/docs/reference/results

A test result says what Retest observed; the run result says whether the selected work finished reliably. Read both when deciding whether a release check passed, especially when a browser stopped or evidence could not be saved.

```sh
npx retest inspect .retest/runs/<time> --json
```

This prints the saved run result without opening a browser. Its top-level `status`, `exitCode` and `complete` describe the run. Individual results are under `files[].tests[]`. The inspect command itself exits 0 when it reads the folder successfully, even when the saved run failed.

## Test statuses

| Status | Meaning |
| --- | --- |
| `passed` | The test and its required checks passed, and cleanup introduced no failure. |
| `failed` | A check, action, timeout or test-code rule failed. Read failure.class for the cause. |
| `error` | Retest could not obtain or finish a reliable result, such as a lost session or cleanup failure. |
| `not_run` | The selected test never ran, for example because its setup failed. |
| `inconclusive` | A required AI check returned a valid answer that could not decide the requirement. |
| `skipped` | test.skip or a skipped group deliberately kept the test from running. It is never counted as passed. |

`outcome_unknown` is a failure class, not a separate test status. It produces `error` because an action was dispatched but Retest could not establish its effect. Retest does not repeat that action to obtain a clearer answer.

## Run statuses and completeness

| Run status | Exit code | Meaning |
| --- | --- | --- |
| `passed` | 0 | Selected work passed and finished cleanly. Declared skips may be present. |
| `failed` | 1 | At least one test failed, subject to the precedence rules below. |
| `error` | 2 | The run could not check everything or keep a trustworthy output. |
| `interrupted` | 130 or 143 | A signal or a programmatic cancellation stopped the run. |

`complete: true` means selected tests settled as passed, failed or skipped, files ended cleanly and required output was kept. A completed run can fail its checks. `complete: false` means some work or required evidence remains uncertain.

A run narrowed by `test.only` can pass while checking only part of the suite. Its `narrowed` field names the focused declarations and reports how many tests they kept. A baseline-update run also records `baselinesWritten`, so you can distinguish it from an ordinary comparison run.

## Exit codes

| Code | Meaning | What to do |
| --- | --- | --- |
| 0 | At least one test ran and the selected work passed. Declared skips do not prevent it. | Check narrowed and baselinesWritten before treating this as a full suite result. |
| 1 | At least one test failed. | Read its failure and evidence before changing the app or test. |
| 2 | Nothing ran, something could not be checked, or run-level output or required work failed. | Read the run, file and test failures; fix the reported problem. |
| 130 | Stopped by SIGINT, usually Ctrl+C, or by a host-provided Failure. | Inspect the saved partial work if needed. |
| 143 | Stopped by SIGTERM. | Inspect the saved partial work if needed. |

Precedence matters when results are mixed. Interruption wins first. An invalid run request, no tests running, lost output or a required host check waived by skipping produces 2 before test failures are considered.

Otherwise, any `failed` test gives 1, even beside errors, inconclusive tests, unfinished files or missing recordings. With no failed test, those problems give 2. A selection containing only skipped tests also gives 2 because nothing ran.

SIGINT and SIGTERM let Retest write its result and attempt cleanup. A second signal stops it immediately. SIGKILL cannot produce a final result; inspection reconstructs the events and marks the run incomplete.

## Failure classes

Failures have `class` and `message`, with optional `location` and `details`. The class identifies the problem; its position in a run or test and any other failures determine the final exit code.

| Class | Meaning |
| --- | --- |
| `check_failed` | An ordinary assertion failed. |
| `host_check_failed` | A page check required by the host or a strict diagnostics policy failed. |
| `not_found` | A required locator match did not appear. |
| `ambiguous` | A call needing one element found several. |
| `not_actionable` | The element could not receive the requested input. |
| `timeout` | A test or operation exceeded its budget. |
| `session_lost` | The browser or session disappeared. |
| `outcome_unknown` | Input may have reached the app, but its effect could not be established. |
| `setup_failed` | A target, server, session or required preparation could not be set up. |
| `cleanup_failed` | Required cleanup did not finish successfully. |
| `collection_failed` | A file could not declare its tests successfully. |
| `test_error` | Test code threw or violated the execution protocol. |
| `no_assertions` | A test ended without making an assertion. |
| `not_awaited` | A call was left unawaited or unfinished. |
| `concurrent_commands` | Two operations tried to use the same app incompatibly. |
| `unsupported` | The selected platform cannot perform the requested operation. |
| `usage` | An argument, configuration or selection was invalid. |
| `interrupted` | The run or operation was stopped from outside. |
| `reporting_failed` | A reporter or required output failed, requireComplete found incomplete diagnostics, or a strict diagnostics rule could not judge. |
| `evaluation_failed` | A required AI check judged a criterion false. |
| `evaluation_inconclusive` | A required AI check could not decide from the evidence. |
| `evaluation_error` | An AI check could not obtain a valid judgement. |
| `evidence_incomplete` | Required capture or recording evidence was incomplete. |

The primary failure appears in `failure`. Additional failures may appear in `failure.details.also`, while cleanup failures have their own array. A later cleanup problem does not replace an earlier check failure.

## Attempt endings

`ending.kind` identifies what ended an attempt. It separates a failed intended check from an action or setup that stopped the test before that check ran. `checkId` names a required check when available; `notRun` lists required host checks that never ran.

| Kind | What ended the attempt |
| --- | --- |
| `passed` | Checks passed. |
| `skipped` | A declaration skipped the test. |
| `not_run` | The attempt never started. |
| `assertion_failed` | A check in the test failed. |
| `required_check_failed` | A host-required check failed. |
| `check_error` | The parent could not complete a required check. |
| `inconclusive` | A required AI judgement was undecided. |
| `evaluation_error` | The AI evaluation could not complete correctly. |
| `setup_failed` | Setup or preparation failed. |
| `action_failed` | An action failed before a check settled the result. |
| `timed_out` | The test ran out of time. |
| `test_error` | Test code or its protocol failed. |
| `crashed` | The browser was lost or the test process ended on its own. |
| `cancelled` | The run stopped the attempt. |
| `outcome_unknown` | A dispatched action never produced a known outcome. |
| `cleanup_failed` | Cleanup alone prevented a clean ending. |

## AI check verdicts

| Verdict | Meaning |
| --- | --- |
| `pass` | Every criterion passed. |
| `fail` | At least one criterion failed. |
| `inconclusive` | None failed, but at least one remained undecided. |
| `error` | The check could not obtain a valid judgement. |
| `cancelled` | The check was stopped. |
| `not_run` | The required check never ran. |

Individual criteria use only `pass`, `fail` and `inconclusive`. A required check affects the test even if its promise is caught. An advisory check records its result without making the test fail. See [AI checks](https://rehearsal.dev/retest/docs/guides/ai-checks.md) for evidence rules.

## Evidence and capture states

Recording evidence has its own `evidenceStatus.state`, separate from the observed test status. Missing recordings do not rewrite a passed test as failed. When recordings are required, their absence prevents the run from passing.

| Evidence state | Meaning |
| --- | --- |
| `complete` | Requested evidence was kept without a reported gap. |
| `partial` | Some evidence was kept, with losses described in gaps. |
| `unavailable` | Requested evidence could not be obtained. |
| `not_requested` | No evidence of this kind was requested. The field may instead be absent. |

Console and network capture each use `complete`, `partial`, `unavailable` or `disabled`. Complete capture with zero records means nothing occurred within the declared scope. Unavailable capture does not establish that the app was quiet.

## Result fields

| Run field | Contents |
| --- | --- |
| `schemaVersion`, `runId`, `retestVersion` | Schema version 1, run identity and writer version. |
| `startedAt`, `finishedAt`, `durationMs` | Start and finish timestamps, and elapsed duration. |
| `status`, `exitCode`, `complete` | Overall outcome and completeness. |
| `counts` | passed, failed, error, notRun and inconclusive counts, plus skipped when present. |
| `browser`, `browsers`, `natives` | First browser or null, and configured browser or native execution identities when present. |
| `failure` | A run-level problem that no one test explains. |
| `narrowed`, `baselinesWritten` | Focused selection and baseline writes when applicable. |
| `evidenceStatus` | Run-wide recording evidence state, requirement and attempt counts. |
| `files` | File results with file, collection status, optional failure and tests. |

| Test field | Contents |
| --- | --- |
| `testId`, `name`, `file`, `location`, `describePath` | Stable test identity and source, with optional group path. |
| `variant`, `variantKey` | App-to-target choices and their printable identity when configured. |
| `setup` | true for a setup result. |
| `attemptId`, `status`, `durationMs`, `assertionCount` | This attempt's identity and observed outcome. |
| `failure`, `cleanupFailures` | Primary failure and separate cleanup problems. |
| `hostChecks`, `evaluations` | Host page checks and AI check records. |
| `execution` | Bundle, configuration, runtime, sessions, starting state and available host identities. |
| `preparations`, `cleanups`, `ending` | Host preparation and cleanup records, and the attempt's ending. |
| `diagnostics` | Capture summaries and artefact paths by session. |
| `recordings`, `evidenceStatus` | Recordings and their separate evidence state. |
| `evidence` | Failure screenshot references with portable paths and available capture identities. |

Optional fields can be absent when they do not apply or an older Retest version did not record them. Use [Run folder and events](https://rehearsal.dev/retest/docs/reference/run-folder.md) for schemas and event identities, or [Use Retest from code](https://rehearsal.dev/retest/docs/scaling-up/from-code.md) to consume results directly.

## Next steps

- [Debugging a failure](https://rehearsal.dev/retest/docs/essentials/debugging.md): Follow a failed result to its steps and evidence.
- [Run folder and events](https://rehearsal.dev/retest/docs/reference/run-folder.md): Read the durable records behind a result.
- [Continuous integration](https://rehearsal.dev/retest/docs/scaling-up/continuous-integration.md): Use run outcomes in a release check.