Test runs

A test run is one run of a product's Playwright tests against its deployment in an environment, numbered per workspace, on one machine or split into shards across several. Start a shard, upload each test's files, then finish the shard with every test's first attempt; the run finishes with its last shard. A retry runs the failed tests again on the same commit, and a failed test that passes every retry attempt counts as flaky.

The TestRun object

A test run with its shards, its retries and every test with its attempts.

Attributes

idstringRequired

Unique ID of the test run.

numbernumberRequired

The run's number, counting up from 1 in each workspace, such as 1289.

urlstringRequired

The run's page on the website.

statestringRequired

running until its last shard finishes, then finished, or interrupted when any shard was. One still running after 6 hours becomes interrupted.

One ofrunningfinishedinterrupted

resultnullable stringRequired

passed when the run has no error and no failed test, otherwise failed. Flaky tests count as passed. Null while running and for interrupted runs.

One ofpassedfailed

countsobjectRequired

How many tests have each status.

passRatenullable numberRequired

(passed + flaky) ÷ (tests − skipped), from 0 to 1. Null while running or when no test counts.

firstAttemptPassRatenullable numberRequired

Tests that passed on attempt 1 ÷ (tests − skipped), from 0 to 1. Retries never change it. Null while running or when no test counts.

errornullable stringRequired

What failed outside the tests, such as a web server that did not start, or No tests ran. In a run with several shards, each shard's error starts with Shard 2 of 4:. A run with an error is failed and cannot be retried.

productobjectRequired

The product the run tested.

environmentobjectRequired

The environment the run tested.

baseUrlstringRequired

The URL the tests ran against.

deploymentBaseUrlnullable stringRequired

The deployment's URL when the run started, or null. It differs from baseUrl when clanker run --base-url overrode it.

deploymentVersionnullable stringRequired

The deployment's version when the run started, or null.

gitobjectRequired

The commit the run tested.

coveragenullable stringRequired

The coverage level the run selected, or null when it selected none.

One ofcriticalacceptanceregressionexhaustive

quarantinebooleanRequired

True when the run ran only the cases tagged @quarantine, with clanker run --quarantine; every other run leaves them out.

cibooleanRequired

True for a CI run: published by an account API key against a site that is not on the machine running it. Only CI runs file failure tickets and count in the flaky cases report.

shardCountnumberRequired

How many shards the run is split into, 1 for a run started without one.

finishedShardsnumberRequired

How many of its shards have finished or been interrupted. The run finishes with its last.

cleanupnullable objectRequired

What deleting every shard's scope reported, or null when the deployment had no Fixture API.

cpunullable objectRequired

What the run's shards cost the machines that ran them, summed, or null until a shard reports it.

failuresByKindobjectRequired

How many tests failed their first attempt, by failure kind. Kinds with none are left out.

publishedByobjectRequired

Who published a run or a retry: a person, an account API key, or one since removed.

startedAtstringRequired

When the run started, as an ISO 8601 timestamp.

finishedAtnullable stringRequired

When the run finished or was marked interrupted, as an ISO 8601 timestamp, or null while running.

durationMsnumberRequired

How long the run took, in milliseconds: its longest shard's duration, as reported. 0 while running.

shardsarray of objectsRequired

The shards that have started, by index.

retriesarray of objectsRequired

The run's retries, oldest first.

testsarray of objectsRequired

Every test, failed first, then flaky, passed and skipped, by title.

The TestRun object

{  "id": "Tr8KcW3nPx5Q",  "number": 1289,  "url": "https://clankership.com/w/acme/test-runs/1289",  "state": "finished",  "result": "passed",  "counts": {    "total": 2,    "passed": 1,    "flaky": 1,    "failed": 0,    "skipped": 0  },  "passRate": 1,  "firstAttemptPassRate": 0.5,  "error": null,  "product": {    "slug": "storefront",    "name": "Storefront"  },  "environment": {    "slug": "staging",    "name": "Staging",    "class": "non-production"  },  "baseUrl": "https://staging.acme.dev",  "deploymentBaseUrl": "https://staging.acme.dev",  "deploymentVersion": "9f2c41e",  "git": {    "branch": "web-31-docs-search",    "sha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",    "dirty": false,    "subject": "Add docs search"  },  "coverage": null,  "quarantine": false,  "ci": true,  "shardCount": 1,  "finishedShards": 1,  "cleanup": {    "scopes": [      "scp_7h2k9q2m4x1a"    ],    "expiresAt": "2026-10-14T15:40:00.000Z",    "status": "complete",    "deleted": 214,    "remaining": 0  },  "cpu": {    "shards": 1,    "cores": 18,    "model": "Apple M5 Max",    "durationMs": 151230,    "cpuMs": 1149551,    "mean": 0.42,    "peak": 0.97,    "runCpuMs": 486120  },  "failuresByKind": {    "assertion": 1  },  "publishedBy": {    "kind": "user",    "name": "Maya Patel",    "email": "maya@acme.dev"  },  "startedAt": "2026-10-14T09:40:00.000Z",  "finishedAt": "2026-10-14T09:42:31.000Z",  "durationMs": 151230,  "shards": [    {      "index": 1,      "state": "finished",      "machine": {        "hostname": "maya-mbp",        "platform": "darwin"      },      "testCount": 2,      "error": null,      "cleanup": {        "scope": "scp_7h2k9q2m4x1a",        "expiresAt": "2026-10-14T15:40:00.000Z",        "status": "complete",        "deleted": 214,        "remaining": 0      },      "cpu": {        "cores": 18,        "model": "Apple M5 Max",        "durationMs": 151230,        "cpuMs": 1149551,        "mean": 0.42,        "peak": 0.97,        "runCpuMs": 486120      },      "durationMs": 151230,      "startedAt": "2026-10-14T09:40:00.000Z",      "finishedAt": "2026-10-14T09:42:31.000Z"    }  ],  "retries": [    {      "number": 1,      "state": "finished",      "retries": 2,      "testCount": 1,      "flakyCount": 1,      "baseUrl": "https://staging.acme.dev",      "gitSha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",      "cleanup": {        "scope": "scp_q8w3e5r7t9y2",        "expiresAt": "2026-10-14T15:40:00.000Z",        "status": "complete",        "deleted": 12,        "remaining": 0      },      "publishedBy": {        "kind": "user",        "name": "Maya Patel",        "email": "maya@acme.dev"      },      "startedAt": "2026-10-14T09:44:05.000Z",      "finishedAt": "2026-10-14T09:44:22.000Z"    }  ],  "tests": [    {      "caseId": "checkout-applies-coupon",      "title": "checkout › applies a coupon",      "file": "checkout.spec.ts",      "coverage": "critical",      "testKind": "behavior",      "priority": null,      "status": "flaky",      "durationMs": 4105,      "errorMessage": "Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")",      "files": [        "case.md",        "spec.ts"      ],      "ticket": {        "identifier": "WEB-212",        "title": "Flaky case: checkout › applies a coupon",        "status": "Backlog",        "open": true,        "url": "https://clankership.com/w/acme/tickets/WEB-212"      },      "attempts": [        {          "number": 1,          "shard": 1,          "retry": null,          "status": "failed",          "durationMs": 30412,          "startedAt": "2026-10-14T09:40:12.000Z",          "errorMessage": "Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")",          "errorStack": "Error: Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")\n    at checkout.spec.ts:24:41",          "failureKind": "assertion",          "files": [            "screenshot-1.png"          ]        },        {          "number": 2,          "shard": null,          "retry": 1,          "status": "passed",          "durationMs": 4210,          "startedAt": "2026-10-14T09:44:07.000Z",          "errorMessage": null,          "errorStack": null,          "failureKind": null,          "files": [            "video.webm",            "trace.zip"          ]        },        {          "number": 3,          "shard": null,          "retry": 1,          "status": "passed",          "durationMs": 4105,          "startedAt": "2026-10-14T09:44:13.000Z",          "errorMessage": null,          "errorStack": null,          "failureKind": null,          "files": [            "video.webm",            "trace.zip"          ]        }      ]    },    {      "caseId": "docs-search-finds-guides",      "title": "docs › search finds guides",      "file": "docs.spec.ts",      "coverage": "acceptance",      "testKind": "journey",      "priority": null,      "status": "passed",      "durationMs": 6820,      "errorMessage": null,      "files": [        "case.md",        "spec.ts"      ],      "ticket": null,      "attempts": [        {          "number": 1,          "shard": 1,          "retry": null,          "status": "passed",          "durationMs": 6820,          "startedAt": "2026-10-14T09:40:12.000Z",          "errorMessage": null,          "errorStack": null,          "failureKind": null,          "files": []        }      ]    }  ]}

List test runs

GET/v1/workspaces/{slug}/test-runs

Returns the workspace's runs, newest first, 50 per page. Send the response's nextCursor as cursor for the next page.

Path parameters

slugstringRequired

The workspace's slug or ID, such as acme.

Query parameters

productstring

Only runs of this product, by slug.

environmentstring

Only runs in this environment, by slug.

resultstring

Only runs that passed or failed, or flaky for runs with at least one flaky test.

One ofpassedfailedflaky

qstring

Matches part of the branch or the commit subject, or the start of a commit SHA.

commitstring

Only runs of this full commit SHA.

minestring

true for only the runs you published.

Always true

cursorstring

The nextCursor of the previous page.

Returns

Returns an object with these fields. Status 200.

runsarray of objectsRequired

Up to 50 runs, newest first.

nextCursornullable numberRequired

The cursor for the next page, or null on the last page.

Errors

400: InvalidInputThe request is malformed, or a value is not allowed.
401: UnauthorizedThe request has no valid API key.
403: ForbiddenThe key's user or role may not do this.
404: NotFoundThe workspace or object does not exist, or the caller cannot see it.
409: ConflictThe change conflicts with current data, such as a slug that is already in use.
500: InternalClanker Ship failed to handle the request. Retry, or report the response's x-request-id.

GET/v1/workspaces/{slug}/test-runs

1curl https://api.clankership.com/v1/workspaces/acme/test-runs \2  -H "Authorization: Bearer $CLANKER_API_KEY"

Response200

{  "runs": [    {      "id": "Tr8KcW3nPx5Q",      "number": 1289,      "url": "https://clankership.com/w/acme/test-runs/1289",      "state": "finished",      "result": "passed",      "counts": {        "total": 2,        "passed": 1,        "flaky": 1,        "failed": 0,        "skipped": 0      },      "passRate": 1,      "firstAttemptPassRate": 0.5,      "error": null,      "product": {        "slug": "storefront",        "name": "Storefront"      },      "environment": {        "slug": "staging",        "name": "Staging",        "class": "non-production"      },      "baseUrl": "https://staging.acme.dev",      "deploymentBaseUrl": "https://staging.acme.dev",      "deploymentVersion": "9f2c41e",      "git": {        "branch": "web-31-docs-search",        "sha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",        "dirty": false,        "subject": "Add docs search"      },      "coverage": null,      "quarantine": false,      "ci": true,      "shardCount": 1,      "finishedShards": 1,      "cleanup": {        "scopes": [          "scp_7h2k9q2m4x1a"        ],        "expiresAt": "2026-10-14T15:40:00.000Z",        "status": "complete",        "deleted": 214,        "remaining": 0      },      "cpu": {        "shards": 1,        "cores": 18,        "model": "Apple M5 Max",        "durationMs": 151230,        "cpuMs": 1149551,        "mean": 0.42,        "peak": 0.97,        "runCpuMs": 486120      },      "failuresByKind": {        "assertion": 1      },      "publishedBy": {        "kind": "user",        "name": "Maya Patel",        "email": "maya@acme.dev"      },      "startedAt": "2026-10-14T09:40:00.000Z",      "finishedAt": "2026-10-14T09:42:31.000Z",      "durationMs": 151230    }  ],  "nextCursor": null}

Start a test run

POST/v1/workspaces/{slug}/test-runs

Starts a shard of a running run of a product's deployment in an environment and records the deployment's URL and version. Without shard it starts a new run of one shard. With shard, the first request with its key starts a run of count shards and later ones join it; a shard that already started, another count or commit, another publisher or a run that is no longer running fails the request. When the deployment has a Fixture API, each shard gets a scope of its own and a fixture token to create its test data with. While it runs, its publisher uploads each file as the raw body of PUT /v1/workspaces/{slug}/test-runs/{run}/files/{path}, with Content-Length and at most 100 MB, deletes the scope, then finishes the shard. Viewers cannot publish runs, and an archived product or environment or a missing deployment fails the request.

Path parameters

slugstringRequired

The workspace's slug or ID, such as acme.

Body parameters

productstringRequired

The product to test, by slug, such as storefront.

environmentstringRequired

The environment it is deployed to, by slug, such as staging.

baseUrlstringRequired

The URL the tests run against, usually the deployment's baseUrl.

fixtureUrlnullable string

Use this Fixture API instead of the deployment's, such as a local server on another port. The fixture token is only valid there.

gitobjectRequired

The commit being tested.

machineobjectRequired

The computer running the tests.

selectionobjectRequired

Which tests the run selects.

shardnullable object

This machine's share of a run split across machines. The first shard with a key starts the run; the others join it. Omit it for a run of one shard.

Returns

Returns an object with these fields. Status 201.

numbernumberRequired

The run's number in the workspace. Later calls name the run by it.

urlstringRequired

The run's page on the website.

shardobjectRequired

The shard this machine started; 1 of 1 without one.

fixturenullable FixtureGrant objectRequired

The shard's scope and fixture token, or null when the deployment has no Fixture API.

semaphoresobjectRequired

Where a shard or retry takes semaphore leases, and the run token it takes them with.

Errors

400: InvalidInputThe request is malformed, or a value is not allowed.
401: UnauthorizedThe request has no valid API key.
403: ForbiddenThe key's user or role may not do this.
404: NotFoundThe workspace or object does not exist, or the caller cannot see it.
409: ConflictThe change conflicts with current data, such as a slug that is already in use.
500: InternalClanker Ship failed to handle the request. Retry, or report the response's x-request-id.

POST/v1/workspaces/{slug}/test-runs

1curl -X POST https://api.clankership.com/v1/workspaces/acme/test-runs \2  -H "Authorization: Bearer $CLANKER_API_KEY" \3  -H "Content-Type: application/json" \4  -d '{5    "product": "storefront",6    "environment": "staging",7    "baseUrl": "https://staging.acme.dev",8    "git": {9      "branch": "web-31-docs-search",10      "sha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",11      "dirty": false,12      "subject": "Add docs search"13    },14    "machine": {15      "hostname": "maya-mbp",16      "platform": "darwin"17    },18    "selection": {19      "coverage": null,20      "folders": [],21      "caseIds": [],22      "quarantine": false23    },24    "shard": {25      "index": 2,26      "count": 4,27      "key": "github-3905-1"28    }29  }'

Response201

{  "number": 1289,  "url": "https://clankership.com/w/acme/test-runs/1289",  "shard": {    "index": 2,    "count": 4  },  "fixture": {    "scope": "scp_7h2k9q2m4x1a",    "token": "eyJhbGciOiJFZERTQSIsImtpZCI6ImZ4XzEifQ.eyJzY29wZSI6InNjcF83aDJrOXEybTR4MWEifQ.c2lnbmF0dXJl",    "url": "https://staging.acme.dev/fixtures",    "expiresAt": "2026-10-14T15:40:00.000Z"  },  "semaphores": {    "url": "https://api.clankership.com/v1/workspaces/Wk7Qm2RtXa9L/semaphores",    "token": "eyJhbGciOiJFZERTQSIsImtpZCI6ImZ4XzEifQ.eyJraW5kIjoicnVuIiwicnVuIjoxMjg5fQ.c2lnbmF0dXJl",    "expiresAt": "2026-10-14T15:40:00.000Z"  }}

List flaky cases

GET/v1/workspaces/{slug}/test-runs/flaky-cases

Returns a product's flaky cases by lane, an environment and a coverage level: for each, its latest finished CI runs, how many passed every first attempt, and each case that failed its first attempt in them, with how often, the failures' kinds and the latest one's run. CI runs are those an account API key published against a site not on the machine running it; runs of quarantined cases only are left out.

Path parameters

slugstringRequired

The workspace's slug or ID, such as acme.

Query parameters

productstringRequired

The product to report on, by slug.

environmentstring

Only this environment's lanes, by slug.

coveragestring

Only lanes that ran this coverage level.

One ofcriticalacceptanceregressionexhaustive

runsstring

How many of each lane's latest CI runs to count, 1 to 200. Default 20.

Returns

Returns an object with these fields. Status 200.

A product's cases that failed their first attempt in its latest CI runs, by lane.

productobjectRequired

The product reported on.

runsnumberRequired

How many of each lane's latest runs count.

lanesarray of objectsRequired

Each lane, by environment name and coverage level.

Errors

400: InvalidInputThe request is malformed, or a value is not allowed.
401: UnauthorizedThe request has no valid API key.
403: ForbiddenThe key's user or role may not do this.
404: NotFoundThe workspace or object does not exist, or the caller cannot see it.
409: ConflictThe change conflicts with current data, such as a slug that is already in use.
500: InternalClanker Ship failed to handle the request. Retry, or report the response's x-request-id.

GET/v1/workspaces/{slug}/test-runs/flaky-cases

1curl "https://api.clankership.com/v1/workspaces/acme/test-runs/flaky-cases?product=storefront" \2  -H "Authorization: Bearer $CLANKER_API_KEY"

Response200

{  "product": {    "slug": "storefront",    "name": "Storefront"  },  "runs": 20,  "lanes": [    {      "environment": {        "slug": "staging",        "name": "Staging"      },      "coverage": null,      "runs": 20,      "cleanRuns": 17,      "cases": [        {          "caseId": "checkout-applies-coupon",          "title": "checkout › applies a coupon",          "file": "checkout.spec.ts",          "failures": 3,          "flaky": 3,          "kinds": {            "infrastructure": 1,            "assertion": 2          },          "latest": {            "run": 1289,            "url": "https://clankership.com/w/acme/test-runs/1289",            "kind": "assertion",            "flaky": true,            "error": "Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")",            "at": "2026-10-14T09:40:00.000Z"          },          "ticket": {            "identifier": "WEB-212",            "title": "Flaky case: checkout › applies a coupon",            "status": "Backlog",            "open": true,            "url": "https://clankership.com/w/acme/tickets/WEB-212"          }        }      ]    }  ]}

Get a test run

GET/v1/workspaces/{slug}/test-runs/{run}

Returns one run with its shards, its retries and every test with its attempts. Download a stored file with GET /v1/workspaces/{slug}/test-runs/{run}/files/{path}, which accepts a Range header.

Path parameters

slugstringRequired

The workspace's slug or ID, such as acme.

runintegerRequired

The run's number in the workspace, such as 1289.

Returns

Returns the TestRun object. Status 200.

Errors

400: InvalidInputThe request is malformed, or a value is not allowed.
401: UnauthorizedThe request has no valid API key.
403: ForbiddenThe key's user or role may not do this.
404: NotFoundThe workspace or object does not exist, or the caller cannot see it.
409: ConflictThe change conflicts with current data, such as a slug that is already in use.
500: InternalClanker Ship failed to handle the request. Retry, or report the response's x-request-id.

GET/v1/workspaces/{slug}/test-runs/{run}

1curl https://api.clankership.com/v1/workspaces/acme/test-runs/{run} \2  -H "Authorization: Bearer $CLANKER_API_KEY"

Response200

{  "id": "Tr8KcW3nPx5Q",  "number": 1289,  "url": "https://clankership.com/w/acme/test-runs/1289",  "state": "finished",  "result": "passed",  "counts": {    "total": 2,    "passed": 1,    "flaky": 1,    "failed": 0,    "skipped": 0  },  "passRate": 1,  "firstAttemptPassRate": 0.5,  "error": null,  "product": {    "slug": "storefront",    "name": "Storefront"  },  "environment": {    "slug": "staging",    "name": "Staging",    "class": "non-production"  },  "baseUrl": "https://staging.acme.dev",  "deploymentBaseUrl": "https://staging.acme.dev",  "deploymentVersion": "9f2c41e",  "git": {    "branch": "web-31-docs-search",    "sha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",    "dirty": false,    "subject": "Add docs search"  },  "coverage": null,  "quarantine": false,  "ci": true,  "shardCount": 1,  "finishedShards": 1,  "cleanup": {    "scopes": [      "scp_7h2k9q2m4x1a"    ],    "expiresAt": "2026-10-14T15:40:00.000Z",    "status": "complete",    "deleted": 214,    "remaining": 0  },  "cpu": {    "shards": 1,    "cores": 18,    "model": "Apple M5 Max",    "durationMs": 151230,    "cpuMs": 1149551,    "mean": 0.42,    "peak": 0.97,    "runCpuMs": 486120  },  "failuresByKind": {    "assertion": 1  },  "publishedBy": {    "kind": "user",    "name": "Maya Patel",    "email": "maya@acme.dev"  },  "startedAt": "2026-10-14T09:40:00.000Z",  "finishedAt": "2026-10-14T09:42:31.000Z",  "durationMs": 151230,  "shards": [    {      "index": 1,      "state": "finished",      "machine": {        "hostname": "maya-mbp",        "platform": "darwin"      },      "testCount": 2,      "error": null,      "cleanup": {        "scope": "scp_7h2k9q2m4x1a",        "expiresAt": "2026-10-14T15:40:00.000Z",        "status": "complete",        "deleted": 214,        "remaining": 0      },      "cpu": {        "cores": 18,        "model": "Apple M5 Max",        "durationMs": 151230,        "cpuMs": 1149551,        "mean": 0.42,        "peak": 0.97,        "runCpuMs": 486120      },      "durationMs": 151230,      "startedAt": "2026-10-14T09:40:00.000Z",      "finishedAt": "2026-10-14T09:42:31.000Z"    }  ],  "retries": [    {      "number": 1,      "state": "finished",      "retries": 2,      "testCount": 1,      "flakyCount": 1,      "baseUrl": "https://staging.acme.dev",      "gitSha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",      "cleanup": {        "scope": "scp_q8w3e5r7t9y2",        "expiresAt": "2026-10-14T15:40:00.000Z",        "status": "complete",        "deleted": 12,        "remaining": 0      },      "publishedBy": {        "kind": "user",        "name": "Maya Patel",        "email": "maya@acme.dev"      },      "startedAt": "2026-10-14T09:44:05.000Z",      "finishedAt": "2026-10-14T09:44:22.000Z"    }  ],  "tests": [    {      "caseId": "checkout-applies-coupon",      "title": "checkout › applies a coupon",      "file": "checkout.spec.ts",      "coverage": "critical",      "testKind": "behavior",      "priority": null,      "status": "flaky",      "durationMs": 4105,      "errorMessage": "Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")",      "files": [        "case.md",        "spec.ts"      ],      "ticket": {        "identifier": "WEB-212",        "title": "Flaky case: checkout › applies a coupon",        "status": "Backlog",        "open": true,        "url": "https://clankership.com/w/acme/tickets/WEB-212"      },      "attempts": [        {          "number": 1,          "shard": 1,          "retry": null,          "status": "failed",          "durationMs": 30412,          "startedAt": "2026-10-14T09:40:12.000Z",          "errorMessage": "Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")",          "errorStack": "Error: Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")\n    at checkout.spec.ts:24:41",          "failureKind": "assertion",          "files": [            "screenshot-1.png"          ]        },        {          "number": 2,          "shard": null,          "retry": 1,          "status": "passed",          "durationMs": 4210,          "startedAt": "2026-10-14T09:44:07.000Z",          "errorMessage": null,          "errorStack": null,          "failureKind": null,          "files": [            "video.webm",            "trace.zip"          ]        },        {          "number": 3,          "shard": null,          "retry": 1,          "status": "passed",          "durationMs": 4105,          "startedAt": "2026-10-14T09:44:13.000Z",          "errorMessage": null,          "errorStack": null,          "failureKind": null,          "files": [            "video.webm",            "trace.zip"          ]        }      ]    },    {      "caseId": "docs-search-finds-guides",      "title": "docs › search finds guides",      "file": "docs.spec.ts",      "coverage": "acceptance",      "testKind": "journey",      "priority": null,      "status": "passed",      "durationMs": 6820,      "errorMessage": null,      "files": [        "case.md",        "spec.ts"      ],      "ticket": null,      "attempts": [        {          "number": 1,          "shard": 1,          "retry": null,          "status": "passed",          "durationMs": 6820,          "startedAt": "2026-10-14T09:40:12.000Z",          "errorMessage": null,          "errorStack": null,          "failureKind": null,          "files": []        }      ]    }  ]}

Finish a shard

POST/v1/workspaces/{slug}/test-runs/{run}/shards/{shard}/finish

Records the shard's tests with their first attempts and closes the shard. The last shard to finish closes the run: it counts every shard's tests and ends the run finished, or interrupted when any shard was. Until then the run stays running and finishedShards counts up. Only the run's publisher can finish a shard, and only while it is running, except that the same finish sent again with its Idempotency-Key answers as the first did. A finished run with no tests gets the error No tests ran. and fails. When the run finishes and no automatic retry follows, its first-attempt failures are kept for the flaky cases report; in a CI run each goes to its case's ticket on the product's failure team, and a CI run of quarantined cases counts each case's clean first attempts instead.

Path parameters

slugstringRequired

The workspace's slug or ID, such as acme.

runintegerRequired

The run's number in the workspace, such as 1289.

shardintegerRequired

The shard's index in the run, such as 2; 1 for a run started without a shard.

Header parameters

idempotency-keystring

Up to 100 letters, digits, dashes and underscores, such as a UUID. Send the finish again with the same key when its answer never came: a shard this key already finished answers as before and records nothing twice.

Body parameters

interruptedbooleanRequired

Whether the shard stopped early, such as on Ctrl-C. Its tests count, and the run ends interrupted with no result.

errornullable string

What Playwright reported outside the tests, such as a web server that did not start. It fails the run. Ignored when interrupted is true.

durationMsnumberRequired

How long the shard took, in milliseconds.

cleanupnullable object

What deleting the shard's scope reported. Required for a shard with a scope; a scope with no report counts as incomplete.

cpunullable object

What the shard cost the machine that ran it.

retriesnumber

How many times the run's automatic retry runs each failed test once the run finishes, 0 to 5; 0 or left out when no retry follows. The run's failures go to their cases' tickets after that retry, or with the last shard when no retry follows: when it is 0, the run has an error, or more than 10 tests failed.

testsarray of objectsRequired

Every test the shard ran, each with a caseId no other shard of the run reported. A run holds up to 5,000 tests.

Returns

Returns an object with these fields. Status 200.

The run after a shard or retry finished, and the quarantined cases whose ticket is closed.

idstringRequired

Unique ID of the test run.

numbernumberRequired

The run's number, counting up from 1 in each workspace, such as 1289.

urlstringRequired

The run's page on the website.

statestringRequired

running until its last shard finishes, then finished, or interrupted when any shard was. One still running after 6 hours becomes interrupted.

One ofrunningfinishedinterrupted

resultnullable stringRequired

passed when the run has no error and no failed test, otherwise failed. Flaky tests count as passed. Null while running and for interrupted runs.

One ofpassedfailed

countsobjectRequired

How many tests have each status.

passRatenullable numberRequired

(passed + flaky) ÷ (tests − skipped), from 0 to 1. Null while running or when no test counts.

firstAttemptPassRatenullable numberRequired

Tests that passed on attempt 1 ÷ (tests − skipped), from 0 to 1. Retries never change it. Null while running or when no test counts.

errornullable stringRequired

productobjectRequired

The product the run tested.

environmentobjectRequired

The environment the run tested.

baseUrlstringRequired

The URL the tests ran against.

deploymentBaseUrlnullable stringRequired

The deployment's URL when the run started, or null. It differs from baseUrl when clanker run --base-url overrode it.

deploymentVersionnullable stringRequired

The deployment's version when the run started, or null.

gitobjectRequired

The commit the run tested.

coveragenullable stringRequired

The coverage level the run selected, or null when it selected none.

One ofcriticalacceptanceregressionexhaustive

quarantinebooleanRequired

True when the run ran only the cases tagged @quarantine, with clanker run --quarantine; every other run leaves them out.

cibooleanRequired

True for a CI run: published by an account API key against a site that is not on the machine running it. Only CI runs file failure tickets and count in the flaky cases report.

shardCountnumberRequired

How many shards the run is split into, 1 for a run started without one.

finishedShardsnumberRequired

How many of its shards have finished or been interrupted. The run finishes with its last.

cleanupnullable objectRequired

What deleting every shard's scope reported, or null when the deployment had no Fixture API.

cpunullable objectRequired

What the run's shards cost the machines that ran them, summed, or null until a shard reports it.

failuresByKindobjectRequired

How many tests failed their first attempt, by failure kind. Kinds with none are left out.

publishedByobjectRequired

Who published a run or a retry: a person, an account API key, or one since removed.

startedAtstringRequired

When the run started, as an ISO 8601 timestamp.

finishedAtnullable stringRequired

When the run finished or was marked interrupted, as an ISO 8601 timestamp, or null while running.

durationMsnumberRequired

How long the run took, in milliseconds: its longest shard's duration, as reported. 0 while running.

closedQuarantinearray of objectsRequired

In a finished run of quarantined cases, each case whose ticket is done or canceled while the case is still quarantined; clanker run --quarantine fails on them. Empty for any other run.

Errors

400: InvalidInputThe request is malformed, or a value is not allowed.
401: UnauthorizedThe request has no valid API key.
403: ForbiddenThe key's user or role may not do this.
404: NotFoundThe workspace or object does not exist, or the caller cannot see it.
409: ConflictThe change conflicts with current data, such as a slug that is already in use.
500: InternalClanker Ship failed to handle the request. Retry, or report the response's x-request-id.

POST/v1/workspaces/{slug}/test-runs/{run}/shards/{shard}/finish

1curl -X POST https://api.clankership.com/v1/workspaces/acme/test-runs/{run}/shards/{shard}/finish \2  -H "Authorization: Bearer $CLANKER_API_KEY" \3  -H "Content-Type: application/json" \4  -d '{5    "interrupted": false,6    "error": null,7    "durationMs": 151230,8    "cleanup": {9      "status": "complete",10      "deleted": 214,11      "remaining": 0,12      "error": null13    },14    "cpu": {15      "cores": 18,16      "model": "Apple M5 Max",17      "durationMs": 151230,18      "cpuMs": 1149551,19      "mean": 0.42,20      "peak": 0.97,21      "runCpuMs": 48612022    },23    "retries": 2,24    "tests": [25      {26        "caseId": "checkout-applies-coupon",27        "title": "checkout › applies a coupon",28        "file": "checkout.spec.ts",29        "coverage": "critical",30        "testKind": "behavior",31        "priority": null,32        "files": [33          "case.md",34          "spec.ts"35        ],36        "attempt": {37          "status": "failed",38          "durationMs": 30412,39          "startedAt": "2026-10-14T09:40:12.000Z",40          "errorMessage": "Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")",41          "errorStack": "Error: Timed out 5000ms waiting for expect(locator).toHaveText(\"$45.00\")\n    at checkout.spec.ts:24:41",42          "files": [43            "screenshot-1.png"44          ]45        }46      },47      {48        "caseId": "docs-search-finds-guides",49        "title": "docs › search finds guides",50        "file": "docs.spec.ts",51        "coverage": "acceptance",52        "testKind": "journey",53        "priority": null,54        "files": [55          "case.md",56          "spec.ts"57        ],58        "attempt": {59          "status": "passed",60          "durationMs": 6820,61          "startedAt": "2026-10-14T09:40:12.000Z",62          "errorMessage": null,63          "errorStack": null,64          "files": []65        }66      }67    ]68  }'

Response200

{  "id": "Tr8KcW3nPx5Q",  "number": 1289,  "url": "https://clankership.com/w/acme/test-runs/1289",  "state": "finished",  "result": "passed",  "counts": {    "total": 2,    "passed": 1,    "flaky": 1,    "failed": 0,    "skipped": 0  },  "passRate": 1,  "firstAttemptPassRate": 0.5,  "error": null,  "product": {    "slug": "storefront",    "name": "Storefront"  },  "environment": {    "slug": "staging",    "name": "Staging",    "class": "non-production"  },  "baseUrl": "https://staging.acme.dev",  "deploymentBaseUrl": "https://staging.acme.dev",  "deploymentVersion": "9f2c41e",  "git": {    "branch": "web-31-docs-search",    "sha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",    "dirty": false,    "subject": "Add docs search"  },  "coverage": null,  "quarantine": false,  "ci": true,  "shardCount": 1,  "finishedShards": 1,  "cleanup": {    "scopes": [      "scp_7h2k9q2m4x1a"    ],    "expiresAt": "2026-10-14T15:40:00.000Z",    "status": "complete",    "deleted": 214,    "remaining": 0  },  "cpu": {    "shards": 1,    "cores": 18,    "model": "Apple M5 Max",    "durationMs": 151230,    "cpuMs": 1149551,    "mean": 0.42,    "peak": 0.97,    "runCpuMs": 486120  },  "failuresByKind": {    "assertion": 1  },  "publishedBy": {    "kind": "user",    "name": "Maya Patel",    "email": "maya@acme.dev"  },  "startedAt": "2026-10-14T09:40:00.000Z",  "finishedAt": "2026-10-14T09:42:31.000Z",  "durationMs": 151230,  "closedQuarantine": []}

Start a retry

POST/v1/workspaces/{slug}/test-runs/{run}/retries

Starts a retry of a finished run's failed tests and returns them with the URL to test. The run must have no error, git.sha must match its commit and only one retry runs at a time, or the request fails with Conflict; viewers cannot retry runs. Its publisher uploads the retry's files like the run's, then finishes the retry.

Path parameters

slugstringRequired

The workspace's slug or ID, such as acme.

runintegerRequired

The run's number in the workspace, such as 1289.

Body parameters

retriesnumberRequired

How many times to run each failed test, 1 to 5. A test that passes every attempt counts as flaky.

baseUrlnullable stringRequired

The URL to test instead of the run's, or null for the run's.

fixtureUrlnullable string

Use this Fixture API instead of the deployment's, such as a local server on another port. The fixture token is only valid there.

gitobjectRequired

The commit being tested. sha must be the commit the run tested.

machineobjectRequired

The computer running the tests.

Returns

Returns an object with these fields. Status 201.

retrynumberRequired

The retry's number within the run.

retriesnumberRequired

How many times to run each failed test.

baseUrlstringRequired

The URL to test.

testsarray of objectsRequired

The run's failed tests, to run again.

environmentobjectRequired

The environment the run tested.

fixturenullable FixtureGrant objectRequired

A new scope and fixture token for the retry, or null when the run had no Fixture API.

semaphoresobjectRequired

Where a shard or retry takes semaphore leases, and the run token it takes them with.

Errors

400: InvalidInputThe request is malformed, or a value is not allowed.
401: UnauthorizedThe request has no valid API key.
403: ForbiddenThe key's user or role may not do this.
404: NotFoundThe workspace or object does not exist, or the caller cannot see it.
409: ConflictThe change conflicts with current data, such as a slug that is already in use.
500: InternalClanker Ship failed to handle the request. Retry, or report the response's x-request-id.

POST/v1/workspaces/{slug}/test-runs/{run}/retries

1curl -X POST https://api.clankership.com/v1/workspaces/acme/test-runs/{run}/retries \2  -H "Authorization: Bearer $CLANKER_API_KEY" \3  -H "Content-Type: application/json" \4  -d '{5    "retries": 2,6    "baseUrl": null,7    "git": {8      "branch": "web-31-docs-search",9      "sha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",10      "dirty": false,11      "subject": "Add docs search"12    },13    "machine": {14      "hostname": "maya-mbp",15      "platform": "darwin"16    }17  }'

Response201

{  "retry": 1,  "retries": 2,  "baseUrl": "https://staging.acme.dev",  "tests": [    {      "caseId": "checkout-applies-coupon",      "title": "checkout › applies a coupon",      "file": "checkout.spec.ts",      "nextAttempt": 2    }  ],  "environment": {    "slug": "staging",    "class": "non-production"  },  "fixture": {    "scope": "scp_7h2k9q2m4x1a",    "token": "eyJhbGciOiJFZERTQSIsImtpZCI6ImZ4XzEifQ.eyJzY29wZSI6InNjcF83aDJrOXEybTR4MWEifQ.c2lnbmF0dXJl",    "url": "https://staging.acme.dev/fixtures",    "expiresAt": "2026-10-14T15:40:00.000Z"  },  "semaphores": {    "url": "https://api.clankership.com/v1/workspaces/Wk7Qm2RtXa9L/semaphores",    "token": "eyJhbGciOiJFZERTQSIsImtpZCI6ImZ4XzEifQ.eyJraW5kIjoicnVuIiwicnVuIjoxMjg5fQ.c2lnbmF0dXJl",    "expiresAt": "2026-10-14T15:40:00.000Z"  }}

Finish a retry

POST/v1/workspaces/{slug}/test-runs/{run}/retries/{retry}/finish

Adds the retry's attempts to the run and recounts it. A failed test that ran at least retries times and passed every time becomes flaky. The run's first retry is its last pass: its first-attempt failures then go to their cases' tickets as with a shard's finish. Only the retry's publisher can finish it, and only while it is running, except that the same finish sent again with its Idempotency-Key answers as the first did.

Path parameters

slugstringRequired

The workspace's slug or ID, such as acme.

runintegerRequired

The run's number in the workspace, such as 1289.

retryintegerRequired

The retry's number within the run, such as 1.

Header parameters

idempotency-keystring

Up to 100 letters, digits, dashes and underscores, such as a UUID. Send the finish again with the same key when its answer never came: a retry this key already finished answers as before and records nothing twice.

Body parameters

interruptedbooleanRequired

Whether the retry stopped early. The attempts sent still count.

cleanupnullable object

What deleting the retry's scope reported. Required for a retry with a scope.

testsarray of objectsRequired

The tests this retry ran.

Returns

Returns an object with these fields. Status 200.

The run after a shard or retry finished, and the quarantined cases whose ticket is closed.

idstringRequired

Unique ID of the test run.

numbernumberRequired

The run's number, counting up from 1 in each workspace, such as 1289.

urlstringRequired

The run's page on the website.

statestringRequired

running until its last shard finishes, then finished, or interrupted when any shard was. One still running after 6 hours becomes interrupted.

One ofrunningfinishedinterrupted

resultnullable stringRequired

passed when the run has no error and no failed test, otherwise failed. Flaky tests count as passed. Null while running and for interrupted runs.

One ofpassedfailed

countsobjectRequired

How many tests have each status.

passRatenullable numberRequired

(passed + flaky) ÷ (tests − skipped), from 0 to 1. Null while running or when no test counts.

firstAttemptPassRatenullable numberRequired

Tests that passed on attempt 1 ÷ (tests − skipped), from 0 to 1. Retries never change it. Null while running or when no test counts.

errornullable stringRequired

productobjectRequired

The product the run tested.

environmentobjectRequired

The environment the run tested.

baseUrlstringRequired

The URL the tests ran against.

deploymentBaseUrlnullable stringRequired

The deployment's URL when the run started, or null. It differs from baseUrl when clanker run --base-url overrode it.

deploymentVersionnullable stringRequired

The deployment's version when the run started, or null.

gitobjectRequired

The commit the run tested.

coveragenullable stringRequired

The coverage level the run selected, or null when it selected none.

One ofcriticalacceptanceregressionexhaustive

quarantinebooleanRequired

True when the run ran only the cases tagged @quarantine, with clanker run --quarantine; every other run leaves them out.

cibooleanRequired

True for a CI run: published by an account API key against a site that is not on the machine running it. Only CI runs file failure tickets and count in the flaky cases report.

shardCountnumberRequired

How many shards the run is split into, 1 for a run started without one.

finishedShardsnumberRequired

How many of its shards have finished or been interrupted. The run finishes with its last.

cleanupnullable objectRequired

What deleting every shard's scope reported, or null when the deployment had no Fixture API.

cpunullable objectRequired

What the run's shards cost the machines that ran them, summed, or null until a shard reports it.

failuresByKindobjectRequired

How many tests failed their first attempt, by failure kind. Kinds with none are left out.

publishedByobjectRequired

Who published a run or a retry: a person, an account API key, or one since removed.

startedAtstringRequired

When the run started, as an ISO 8601 timestamp.

finishedAtnullable stringRequired

When the run finished or was marked interrupted, as an ISO 8601 timestamp, or null while running.

durationMsnumberRequired

How long the run took, in milliseconds: its longest shard's duration, as reported. 0 while running.

closedQuarantinearray of objectsRequired

In a finished run of quarantined cases, each case whose ticket is done or canceled while the case is still quarantined; clanker run --quarantine fails on them. Empty for any other run.

Errors

400: InvalidInputThe request is malformed, or a value is not allowed.
401: UnauthorizedThe request has no valid API key.
403: ForbiddenThe key's user or role may not do this.
404: NotFoundThe workspace or object does not exist, or the caller cannot see it.
409: ConflictThe change conflicts with current data, such as a slug that is already in use.
500: InternalClanker Ship failed to handle the request. Retry, or report the response's x-request-id.

POST/v1/workspaces/{slug}/test-runs/{run}/retries/{retry}/finish

1curl -X POST https://api.clankership.com/v1/workspaces/acme/test-runs/{run}/retries/{retry}/finish \2  -H "Authorization: Bearer $CLANKER_API_KEY" \3  -H "Content-Type: application/json" \4  -d '{5    "interrupted": false,6    "cleanup": {7      "status": "complete",8      "deleted": 12,9      "remaining": 0,10      "error": null11    },12    "tests": [13      {14        "caseId": "checkout-applies-coupon",15        "attempts": [16          {17            "status": "passed",18            "durationMs": 4210,19            "startedAt": "2026-10-14T09:44:07.000Z",20            "errorMessage": null,21            "errorStack": null,22            "files": [23              "video.webm",24              "trace.zip"25            ]26          },27          {28            "status": "passed",29            "durationMs": 4105,30            "startedAt": "2026-10-14T09:44:13.000Z",31            "errorMessage": null,32            "errorStack": null,33            "files": [34              "video.webm",35              "trace.zip"36            ]37          }38        ]39      }40    ]41  }'

Response200

{  "id": "Tr8KcW3nPx5Q",  "number": 1289,  "url": "https://clankership.com/w/acme/test-runs/1289",  "state": "finished",  "result": "passed",  "counts": {    "total": 2,    "passed": 1,    "flaky": 1,    "failed": 0,    "skipped": 0  },  "passRate": 1,  "firstAttemptPassRate": 0.5,  "error": null,  "product": {    "slug": "storefront",    "name": "Storefront"  },  "environment": {    "slug": "staging",    "name": "Staging",    "class": "non-production"  },  "baseUrl": "https://staging.acme.dev",  "deploymentBaseUrl": "https://staging.acme.dev",  "deploymentVersion": "9f2c41e",  "git": {    "branch": "web-31-docs-search",    "sha": "9f2c41e7b8a3d5c6f1e0b2a4d6c8e0f1a3b5c7d9",    "dirty": false,    "subject": "Add docs search"  },  "coverage": null,  "quarantine": false,  "ci": true,  "shardCount": 1,  "finishedShards": 1,  "cleanup": {    "scopes": [      "scp_7h2k9q2m4x1a"    ],    "expiresAt": "2026-10-14T15:40:00.000Z",    "status": "complete",    "deleted": 214,    "remaining": 0  },  "cpu": {    "shards": 1,    "cores": 18,    "model": "Apple M5 Max",    "durationMs": 151230,    "cpuMs": 1149551,    "mean": 0.42,    "peak": 0.97,    "runCpuMs": 486120  },  "failuresByKind": {    "assertion": 1  },  "publishedBy": {    "kind": "user",    "name": "Maya Patel",    "email": "maya@acme.dev"  },  "startedAt": "2026-10-14T09:40:00.000Z",  "finishedAt": "2026-10-14T09:42:31.000Z",  "durationMs": 151230,  "closedQuarantine": []}

NextSemaphores