TestSprite Files

Added

• schedule — read and manage recurring test runs from the CLI. schedule list / get / create / update / delete, plus schedule run list <id> for a schedule's past runs. update carries --pause / --resume; passing both is refused rather than silently resolved. create and update --cron both take a standard 5-field cron and, before the request, state how often that works out to in plain words, so a */5 typo is visible before it bills — retiming an existing schedule is the same order-of-magnitude mistake as creating one badly. When the API prices a run, the cost is reported per run and multiplied out to a monthly total at the schedule's own frequency — both under --dry-run too. A schedule the platform paused after repeated failures reads AUTO_PAUSED, distinct from the PAUSED that update --pause produces. delete requires --confirm, matching every other destructive verb. Creates, updates and deletes carry an idempotency key, so a retried command cannot double-apply.
• testlist — saved, named test collections, runnable as one batch. testlist list / get / create / update / add / remove / delete --confirm manage a list that can span multiple projects, with each project optionally pinned to an environment (--project-env <projectId>:<envName>; --clear-project-env removes all pins, distinct from omitting the flag). testlist run <list-id> dispatches every case through its pinned environment — or a --case <test-id> subset — and with --wait polls every run to a verdict, writing an optional JUnit sidecar (--report junit --report-file <path>). With --wait, --gh-output / --summary-file emit the same GitHub-native CI output as test run / test rerun — ::error:: annotations, a job-summary table, and the reduced machine summary JSON — auto-enabled under GITHUB_ACTIONS=true, rejected with a clear VALIDATION_ERROR without --wait, with non-dispatched members folded in as non-passed rows and each row carrying the run's dashboard link when the server provides one. The exit code is CI-honest end to end: a partial --case miss exits 4 (a mistyped id can't pass green), an all-conflict dispatch exits 6 with each cause named, and rate-deferred cases exit 7 on both the --wait and fire-and-return paths. Test lists are V3-only; a non-V3 account gets UNSUPPORTED (exit 7).
• --target-url is now probed before anything dispatches or bills. A confirmed-dead target — DNS NXDOMAIN, connection refused, or a 502/503/504 gateway error (the signature of an expired tunnel) — is refused with exit 5 before the trigger call, instead of dispatching a run that can only fail and still charge. A resolved address landing in private/loopback/link-local space is always refused; ambiguous signals (timeouts, TLS oddities) only warn, and behind a configured HTTP(S) proxy a local DNS failure downgrades to a warning since resolution happens at the proxy. --skip-preflight on test run / test create / test create-batch opts out entirely, with zero extra network calls. The V3 "--target-url ignored" advisory is now response-driven — it reads the run's actual trigger response instead of guessing from account flags, so it stays quiet on backends that apply the override to fresh frontend runs.

Changed

• One exit-code precedence table for every --wait fan-out. test run --all --wait, batch test rerun --wait, and testlist run --wait now resolve a mixed outcome through the same shared table — auth (3) and client-too-old (14) first, then insufficient-credits (12) / feature-gated (13), then not-found/validation/conflict (4/5/6), then rate-limited/unavailable (11/10), then timeout (7), then the generic failure (1). A member whose poll failed with a typed error now surfaces that real code instead of folding into 7 or 1, so a CI job can tell "credential died" from "test failed" without parsing stderr. An out-of-contract code folds into the generic bucket rather than leaking through.
• test run --all --wait rides out capacity instead of failing on it. A run_in_flight conflict that carries the in-flight run's id is auto-resumed — the CLI polls that run to its verdict rather than exiting 6 — and only unresumable conflicts (snapshot mid-mutation, body-hash mismatch) still exit 6. Batch conflicts are rendered by cause (already in flight, view-only mirror project, un-runnable environment, unknown id, dispatch error) instead of a blanket "already in flight". Rate-deferred tests are retried on a time budget — up to --timeout minus a reserved final poll window (60 s, or a third of a short timeout) — instead of a fixed three attempts, so a large batch under load drains instead of giving up early.

Fixed

• A batch --wait run that dispatched nothing now surfaces its verdict in CI instead of failing silently. test run --all, test rerun, and testlist run threw their exit code (6 when every targeted test was already in flight, 7 when every test was rate-deferred) before the GitHub-native output step, so a CI job saw a red exit with no ::error:: annotation and no --summary-file — indistinguishable from the tool crashing. All three now emit the annotations and summary (the non-dispatched tests fold in as non-passed rows) before exiting. The four copies of that emit block — test run <id> / --all, test rerun, testlist run — are also consolidated into one shared helper so a fix or regression can no longer reach only some of them.
• agent status no longer reports a clean Windsurf or Copilot install as stale. Both targets have to keep their rule file small — Windsurf truncates past a hard size cap, and Copilot's file is injected into every request — so agent install writes the trimmed verify skill rather than the full one — but status re-derived the expected content per skill without regard for the target, always assuming the full body. The fingerprint it computed could therefore never match the one the installer had stamped into the file, so testsprite-verify read stale immediately after a clean install and stayed that way: re-running agent install changed nothing, because the file was already correct. Since agent status exits 1 when anything needs attention, any CI job gating on skill freshness for those targets failed permanently. Install and status now resolve the expected content through one shared per-target rule, so they cannot disagree — and a genuinely outdated file on a capped target still reports stale.
• JUnit reports from test run --all and test rerun name their tests and carry real durations. Those sidecars rendered every <testcase> with an id-only name and a zero time, which most CI dashboards collapse into an unreadable wall; test name and wall-clock duration now come through, so the report is scannable where it is actually consumed. testlist run --report junit is not yet enriched — its rows still carry the test id as the name and no duration.
• An unreadable credentials lock no longer reads as infinitely stale. When the lock file's body could not be parsed, the staleness check treated it as older-than-everything and stole the lock immediately — two concurrent CLI processes could then both write the credentials file. An unreadable body now counts as fresh until the real staleness deadline passes.
• A partial stdout envelope on a timed-out single-frontend test rerun --wait. The command previously exited 7 with nothing on stdout when the poll timed out, so a redirected file was zero-byte and the runId was lost; it now emits the same partial {runId, status: "running"} envelope with a re-attach hint that every other --wait path already produced. (Community contribution — #219, thanks @Awad-de.)
• project create / project update guard their --password-file reads. A missing, unreadable, or directory path is refused as a local validation error before the network call instead of surfacing as a raw ENOENT stack trace. (Community contribution — #302, thanks @naufalfx805-source.)
• Buffered JSON responses are size-bounded. A misbehaving endpoint returning an unbounded body could balloon memory before the parse failed; the HTTP layer now caps what it will buffer. (Community contribution — #281, thanks @Yazan-O.)

• --dry-run sample coverage for a failed run-scoped step. The canned test steps --run-id response now includes a failed step, so agents learning the surface offline see the failure shape too. (Community contribution — #218, thanks @JerryNee.)

• Unsupported Node.js releases now fail fast. The CLI exits at startup on Node 20.0–20.18, 21.x, 22.0–22.12, and 23.x instead of continuing on an unsupported runtime.