Yard Docs
Sign in
On this page

Test progress

How far a slow test run is while it goes: tests done of the total, the failing ones by name, and whether it still moves, for any test tool.

Updated November 3, 2026 · 6 min read

A slow test run (end-to-end, browser, integration) can take many minutes. While one goes in a project, Yard shows how far it is: tests finished out of the total, how many failed and which, what runs now, and whether it still moves.

It works in every repository, whatever its test tool, and needs no setup: the tool is recognised from what it prints, not from a config.

What you see

  • The project's status line (its row in the sidebar and on the Projects page, and the working bar on its page) reads e2e: 42 of 118, 2 failed, 6 min instead of the command and its minutes. From the first failing test on the line takes the failing colour. With several runs at once it names the oldest and says +1 more.
  • The chat lists each test run its own turns started as a row: a bar when the total is known, the counts, the test running now, and Show failing tests, which opens to their files and names while the run still goes. Stop is where it is for any background command. Runs started from another chat of the project show there, not here; the project's status line counts them all.
  • A run that went quiet: after 3 minutes without any output the row says No output for 4 min. The mark goes when output comes again.
  • A run that ended keeps its result in the list: 118 passed in 14 min, 3 of 118 failed in 14 min, Stopped after 6 min, Timed out after 30 min, Killed for memory after 6 min (only when the container's memory was the cause) or Killed after 6 min with what killed it (a signal from inside the container, its task limit), until the agent's next reply, which reports it. A run Yard stopped says why (Stopped after 6 min: stopped by Yard: …). The list then leaves the bottom of the chat.
  • A failed run and the base branch: once yard tests base ran it again on the base branch (test baselines), the row says what it found ("All 14 fail on main@a1b2c3d too", "2 new, 12 also fail on main") and marks each failing test that fails on main too. A failed run nobody checked reads "Not checked on the base branch" and offers Check on main, which runs that check from the chat.
  • A run Yard lost track of: yard run reports that it is alive every few seconds. When it stops doing so for a minute without saying how the command ended (it was killed itself), the run reads Stopped with the line "Yard lost track of this run", not as stopped by a signal.
  • Baselines and comparisons (test baselines): the repository block names the command that runs and its counts, and yard tests baseline and yard tests compare print them while they wait.

Running tests so that Yard can read them

Shell
yard run -- bun run e2e
yard run --timeout 30 -- npx playwright test

yard run -- <command> runs the command as it is and keeps its output where Yard reads it. The output and the exit code stay the command's own. Agents are told to start slow tests this way, in the foreground or in the background. --timeout <minutes> stops a run that hangs (exit code 124), and --memory borrows memory for it (agents).

The command does not run under the yard run the agent started: it runs on its own, and yard run follows its output and ends with its exit code. When the agent's shell is ended (its turn ended, or its tool timed out), yard run goes with it but the check goes on, and Yard shows how it ends; a new turn does not stop it. Ctrl-C, a stop from the chat or --timeout ends it. In a terminal, yard run runs the command in the foreground as before.

A command under yard run does not see the markers an agent CLI sets (CLAUDECODE, AI_AGENT and the like). bun, Jest and Vitest print only failing tests when they see one, which would leave nothing to count.

A test command started without yard run is still shown as a test run, but its output cannot be read: the row says that it runs and for how long, with no counts.

What is counted

Yard reads the output as it is written. Recognised from the start:

Tool Total known up front Counted as
Playwright (list, line and dot reporters) Yes: Running N tests tests
pytest (default, -v, -q, xdist) Yes: collected N items tests
TAP (node --test, prove, bats and others) When the plan 1..N comes first tests
bun test No tests
Vitest (default and verbose reporters) No tests
Cypress No tests
Jest No files, and tests with --verbose
go test No packages, and tests with -v or -json

When the tool does not announce a total, Yard uses the total of the last finished run of the same command in the same repository and says so: 42 of about 118. Without an earlier run only the counts are shown, with no bar. A percent is never made up.

Output that no reader recognises gives no counts at all. The row then shows the last line of output and how long ago it came, so you still see whether the run is alive.

Reporting progress yourself

A tool that is not recognised, or a script that wraps your tests, can report its own progress. Under yard run (and for the commands of a baseline or a comparison) the environment variable YARD_TEST_PROGRESS names a file. Append one JSON object per line to it:

Shell
echo '{"total": 40}' >> "$YARD_TEST_PROGRESS"
echo '{"passed": 12, "failed": 1, "current": "checkout pays by card"}' >> "$YARD_TEST_PROGRESS"
echo '{"failure": {"file": "checkout.feature", "name": "refunds an order"}}' >> "$YARD_TEST_PROGRESS"
Field Meaning
total How many tests there are to run.
passed, failed, skipped How many so far: the totals, not what changed since the last line.
current The test file or test running now; null clears it.
failure One failing test: { "file": "…", "name": "…" }. Each one is added to the list.
result Instead of the counts: one finished test, "passed", "failed" or "skipped". With "failed", the line's name and file name it.

Every field is optional, and a line may hold several. Lines that are not a JSON object are skipped. From the first readable line on, the file wins over what Yard reads from the output; the last line of output is still shown.

Limits

  • Yard reads what the tool prints. A reporter that prints nothing until the end (a JSON report, for example) shows no counts until then: use a reporter that prints as it goes, or the progress file.
  • A run's counts are kept for the project while it goes; the result of the last five runs that ended stays in the chat's list until the agent replies (for 12 hours at most).
  • Progress of GitHub's checks on a pull request is not shown here.
Was this page useful?
Edit this page