Results and exit codes
Interpret test statuses, failure classes, result fields and process exit codes.
A test result says what Retest observed; the run result says whether the selected work finished reliably. Read both when deciding whether a release check passed, especially when a browser stopped or evidence could not be saved.
npx retest inspect .retest/runs/<time> --jsonThis prints the saved run result without opening a browser. Its top-level status, exitCode and complete describe the run. Individual results are under files[].tests[]. The inspect command itself exits 0 when it reads the folder successfully, even when the saved run failed.
Test statuses Link to Test statuses
| Status | Meaning |
|---|---|
passed | The test and its required checks passed, and cleanup introduced no failure. |
failed | A check, action, timeout or test-code rule failed. Read failure.class for the cause. |
error | Retest could not obtain or finish a reliable result, such as a lost session or cleanup failure. |
not_run | The selected test never ran, for example because its setup failed. |
inconclusive | A required AI check returned a valid answer that could not decide the requirement. |
skipped | test.skip or a skipped group deliberately kept the test from running. It is never counted as passed. |
outcome_unknown is a failure class, not a separate test status. It produces error because an action was dispatched but Retest could not establish its effect. Retest does not repeat that action to obtain a clearer answer.
Run statuses and completeness Link to Run statuses and completeness
| Run status | Exit code | Meaning |
|---|---|---|
passed | 0 | Selected work passed and finished cleanly. Declared skips may be present. |
failed | 1 | At least one test failed, subject to the precedence rules below. |
error | 2 | The run could not check everything or keep a trustworthy output. |
interrupted | 130 or 143 | A signal or a programmatic cancellation stopped the run. |
complete: true means selected tests settled as passed, failed or skipped, files ended cleanly and required output was kept. A completed run can fail its checks. complete: false means some work or required evidence remains uncertain.
A run narrowed by test.only can pass while checking only part of the suite. Its narrowed field names the focused declarations and reports how many tests they kept. A baseline-update run also records baselinesWritten, so you can distinguish it from an ordinary comparison run.
Exit codes Link to Exit codes
| Code | Meaning | What to do |
|---|---|---|
| 0 | At least one test ran and the selected work passed. Declared skips do not prevent it. | Check narrowed and baselinesWritten before treating this as a full suite result. |
| 1 | At least one test failed. | Read its failure and evidence before changing the app or test. |
| 2 | Nothing ran, something could not be checked, or run-level output or required work failed. | Read the run, file and test failures; fix the reported problem. |
| 130 | Stopped by SIGINT, usually Ctrl+C, or by a host-provided Failure. | Inspect the saved partial work if needed. |
| 143 | Stopped by SIGTERM. | Inspect the saved partial work if needed. |
Precedence matters when results are mixed. Interruption wins first. An invalid run request, no tests running, lost output or a required host check waived by skipping produces 2 before test failures are considered.
Otherwise, any failed test gives 1, even beside errors, inconclusive tests, unfinished files or missing recordings. With no failed test, those problems give 2. A selection containing only skipped tests also gives 2 because nothing ran.
SIGINT and SIGTERM let Retest write its result and attempt cleanup. A second signal stops it immediately. SIGKILL cannot produce a final result; inspection reconstructs the events and marks the run incomplete.
Failure classes Link to Failure classes
Failures have class and message, with optional location and details. The class identifies the problem; its position in a run or test and any other failures determine the final exit code.
| Class | Meaning |
|---|---|
check_failed | An ordinary assertion failed. |
host_check_failed | A page check required by the host or a strict diagnostics policy failed. |
not_found | A required locator match did not appear. |
ambiguous | A call needing one element found several. |
not_actionable | The element could not receive the requested input. |
timeout | A test or operation exceeded its budget. |
session_lost | The browser or session disappeared. |
outcome_unknown | Input may have reached the app, but its effect could not be established. |
setup_failed | A target, server, session or required preparation could not be set up. |
cleanup_failed | Required cleanup did not finish successfully. |
collection_failed | A file could not declare its tests successfully. |
test_error | Test code threw or violated the execution protocol. |
no_assertions | A test ended without making an assertion. |
not_awaited | A call was left unawaited or unfinished. |
concurrent_commands | Two operations tried to use the same app incompatibly. |
unsupported | The selected platform cannot perform the requested operation. |
usage | An argument, configuration or selection was invalid. |
interrupted | The run or operation was stopped from outside. |
reporting_failed | A reporter or required output failed, requireComplete found incomplete diagnostics, or a strict diagnostics rule could not judge. |
evaluation_failed | A required AI check judged a criterion false. |
evaluation_inconclusive | A required AI check could not decide from the evidence. |
evaluation_error | An AI check could not obtain a valid judgement. |
evidence_incomplete | Required capture or recording evidence was incomplete. |
The primary failure appears in failure. Additional failures may appear in failure.details.also, while cleanup failures have their own array. A later cleanup problem does not replace an earlier check failure.
Attempt endings Link to Attempt endings
ending.kind identifies what ended an attempt. It separates a failed intended check from an action or setup that stopped the test before that check ran. checkId names a required check when available; notRun lists required host checks that never ran.
| Kind | What ended the attempt |
|---|---|
passed | Checks passed. |
skipped | A declaration skipped the test. |
not_run | The attempt never started. |
assertion_failed | A check in the test failed. |
required_check_failed | A host-required check failed. |
check_error | The parent could not complete a required check. |
inconclusive | A required AI judgement was undecided. |
evaluation_error | The AI evaluation could not complete correctly. |
setup_failed | Setup or preparation failed. |
action_failed | An action failed before a check settled the result. |
timed_out | The test ran out of time. |
test_error | Test code or its protocol failed. |
crashed | The browser was lost or the test process ended on its own. |
cancelled | The run stopped the attempt. |
outcome_unknown | A dispatched action never produced a known outcome. |
cleanup_failed | Cleanup alone prevented a clean ending. |
AI check verdicts Link to AI check verdicts
| Verdict | Meaning |
|---|---|
pass | Every criterion passed. |
fail | At least one criterion failed. |
inconclusive | None failed, but at least one remained undecided. |
error | The check could not obtain a valid judgement. |
cancelled | The check was stopped. |
not_run | The required check never ran. |
Individual criteria use only pass, fail and inconclusive. A required check affects the test even if its promise is caught. An advisory check records its result without making the test fail. See AI checks for evidence rules.
Evidence and capture states Link to Evidence and capture states
Recording evidence has its own evidenceStatus.state, separate from the observed test status. Missing recordings do not rewrite a passed test as failed. When recordings are required, their absence prevents the run from passing.
| Evidence state | Meaning |
|---|---|
complete | Requested evidence was kept without a reported gap. |
partial | Some evidence was kept, with losses described in gaps. |
unavailable | Requested evidence could not be obtained. |
not_requested | No evidence of this kind was requested. The field may instead be absent. |
Console and network capture each use complete, partial, unavailable or disabled. Complete capture with zero records means nothing occurred within the declared scope. Unavailable capture does not establish that the app was quiet.
Result fields Link to Result fields
| Run field | Contents |
|---|---|
schemaVersion, runId, retestVersion | Schema version 1, run identity and writer version. |
startedAt, finishedAt, durationMs | Start and finish timestamps, and elapsed duration. |
status, exitCode, complete | Overall outcome and completeness. |
counts | passed, failed, error, notRun and inconclusive counts, plus skipped when present. |
browser, browsers, natives | First browser or null, and configured browser or native execution identities when present. |
failure | A run-level problem that no one test explains. |
narrowed, baselinesWritten | Focused selection and baseline writes when applicable. |
evidenceStatus | Run-wide recording evidence state, requirement and attempt counts. |
files | File results with file, collection status, optional failure and tests. |
| Test field | Contents |
|---|---|
testId, name, file, location, describePath | Stable test identity and source, with optional group path. |
variant, variantKey | App-to-target choices and their printable identity when configured. |
setup | true for a setup result. |
attemptId, status, durationMs, assertionCount | This attempt's identity and observed outcome. |
failure, cleanupFailures | Primary failure and separate cleanup problems. |
hostChecks, evaluations | Host page checks and AI check records. |
execution | Bundle, configuration, runtime, sessions, starting state and available host identities. |
preparations, cleanups, ending | Host preparation and cleanup records, and the attempt's ending. |
diagnostics | Capture summaries and artefact paths by session. |
recordings, evidenceStatus | Recordings and their separate evidence state. |
evidence | Failure screenshot references with portable paths and available capture identities. |
Optional fields can be absent when they do not apply or an older Retest version did not record them. Use Run folder and events for schemas and event identities, or Use Retest from code to consume results directly.