e2e Files

Minor Changes

• #779 6639bfa Thanks @fecolinhares! - copilot() reaches the models Copilot serves only over its Responses API, such as gpt-6-luna, gpt-5.3-codex, and grok-4.5. On the first call it reads the plan's model listing and picks chat completions or the Responses API from the model's supported_endpoints; a model served over chat completions, or one the listing does not place, stays on chat completions. A listing that cannot be read sends that call over chat completions without remembering it, and a revoked login fails on the listing with the sign-in command. Responses models run through @ai-sdk/openai, which e2e init now adds for Copilot; without it the call names the package to install. Responses turns carry the same initiator and vision headers as chat turns. e2e models github-copilot marks what copilot() cannot call with no chat or responses in place of no chat completions, and a disabled model as not enabled only.

• #794 c66176b Thanks @neriousy! - Sign in with OpenCode Console to use OpenCode Zen and OpenCode Go models for agent steps. e2e login opencode-console runs Console's device flow: approve the code, pick the workspace, and the login is stored and refreshed like the other subscriptions. opencodeConsole('<id>') from e2e/oauth/opencode-console calls a Zen model for a bare id and a Go model for a go/ id (go/deepseek-v4.1-flash); on its first call it reads the workspace config to pick the model's API (OpenAI chat completions, OpenAI Responses, Anthropic Messages, or Google) and loads only that AI SDK package, so @ai-sdk/anthropic and @ai-sdk/google join the optional peers. Once the workspace config is read, a model it does not serve, or a Go model without the subscription, fails with MISCONFIGURED naming the fix. Each model sends one x-opencode-session, so a worker's calls stay on one upstream cache, and prompts cache the way e2e caches them for OpenAI and Anthropic directly: models served over Anthropic Messages get a breakpoint on the system prompt and, in agent steps, on the newest message, and Responses models get the system prompt's cache key. Reports name the model opencode/<id> or opencode.go/<id>, and chat models of both plans read provider options under opencode. e2e models opencode-console lists the enabled models tagged Zen or Go, e2e init offers OpenCode Console among the subscriptions, and OPENCODE_API_KEY, the variable OpenCode reads, takes a Console service account key in place of the stored login.

• #741 bfa58c7 Thanks @Marve10s! - Keep soft assertion failures in the report when a test skips itself, and show them in the CLI under Skipped After Failure. A failed attempt followed by a skip on retry also appears there with its original error.

Add failOnSkippedFailure: true to fail the run with exit code 1 for these tests. The default is false, preserving the existing exit code. Tests retain their skipped status and reason, teardown still runs, and a skip does not trigger another retry.

• #713 7ab80bc Thanks @okwasniewski! - Breaking: an interrupted test is no longer counted as failed. report.json gains run.summary.interrupted, and run.summary.failed counts only failed and timed-out tests. run.summary.skipped now counts only selected tests, so passed + failed + interrupted + flaky + skipped equals selected; the tests a filter left out are discovered - selected. The list reporter prints 3 interrupted in its own column, summary.md shows them with ⏹️ and gives them no failure block or page, and junit.xml writes each one as a <skipped> whose message starts interrupted:. A test that failed and whose retry an interrupt cut short stays failed, and its failure is the one reported. A --repeat-each run the interrupt stopped is listed as interrupted, not as a flake. The telemetry event gains tests_interrupted. In @e2e-dev/github, a test that was interrupted and then passed on a --last-failed rerun shows as passed, not flaky.

Patch Changes

• #711 a3da00d Thanks @okwasniewski! - device.installApp() no longer pins the build's file path as the app when agent-device reports no bundle id or package for it, which on Android made app.open() fail with "Android runtime hints require an installed package name". It fails at the install with ENGINE_FAILURE naming app.bundleId or installApp's app option; the mobile docs' Troubleshooting covers the Android cause (agent-device 0.21.18 reads an aapt2-built APK's package only through the SDK's aapt, which it does not find in the macOS default SDK location, or when the install added the package). app.open() on a device now keeps the engine's own reason it cannot launch, such as a build not installed yet with device.installApp(), instead of a generic "pin one with app.bundleId or app.appPath".

• #732 3bf295e Thanks @okwasniewski! - The replay cache no longer passes a step whose effect did not happen. Entries are keyed by the agent and a hash of its redacted context, so one persona never replays another's steps and a changed agentContext records again. A route now includes the origin and the query (ids, tokens, and timestamps aside), and the end route must match exactly. A replay checks what the step made appear, with checked or selected state, and what it made disappear, needs at least one of those changes to happen during the replay, and stops on an alert the recording never saw. A count the step made appear is part of the effect, and an unnamed control with twins needs a named row or group to replay. Every existing entry is re-recorded on the first run after the upgrade.

• #713 7ab80bc Thanks @okwasniewski! - A forced interrupt (a second Ctrl-C, or a CI cancel that sends SIGINT and then SIGTERM) still writes junit.xml and summary.md. The runner used to abandon them along with every other reporter, which left the previous run's files beside the new report.json. Custom reporters are still abandoned on force.

• #799 96ff11b Thanks @okwasniewski! - A recorded step that removes a control while something else on screen keeps its text, such as a radio labelled "Express" replaced by a status reading "Express", now replays. Its recording used to fail its own end check on every replay (end-mismatch, or REPLAY_STALE under --strict-cache). Re-record such a step once to pick up the fix.

• #726 f7c0756 Thanks @okwasniewski! - locator.waitFor adds Playwright's attached and detached states and refuses any other state, or an option key besides state and timeout, with INVALID_ARGUMENT. Before, every state but visible ran as hidden, so waitFor({ state: 'attached' }) and a misspelled state passed at once on an absent node. toBeHidden, toBeVisible({ visible: false }), toBeAttached({ attached: false }), and waitFor detached and hidden read a locator under a frame missing from the document as zero matches instead of timing out waiting for the frame. toHaveProperty reads a primitive on the path through its wrapper, as Jest does, so toHaveProperty('label.length', 3) passes. expect(browser).toHaveURL takes ignoreCase, and urlMatches in e2e/engine takes it as an optional fourth argument.

• #784 cc54c6b Thanks @okwasniewski! - Every model call now identifies e2e, whichever provider serves it: the User-Agent starts with e2e/<version> (<platform>; <arch>) ahead of the AI SDK's own, and HTTP-Referer: https://tester.army/e2e with X-Title: e2e attribute the traffic on the Vercel AI Gateway and OpenRouter. These replace an HTTP-Referer or X-Title set on the provider instance. Before, only subscription logins sent the e2e user agent.

• #722 f0f9c8d Thanks @okwasniewski! - Explicit navigation now uses an allowlist instead of a denylist: app.open, browser.goto, and the agent's navigate verb admit http:, https:, and the exact about:blank, and every other scheme (chrome:, blob:, about:srcdoc, ...) is POLICY_DENIED. A wrapped scheme such as view-source:file:///... no longer loads a local file; it is POLICY_DENIED like file: itself. device.openLink and device.openApp also refuse view-source:, blob:, and filesystem: links. browser.setCookies refuses an about:blank cookie URL with POLICY_DENIED.

• #729 4ce01b2 Thanks @okwasniewski! - A registered secret no longer leaks through the observed screen: a test id, an iframe name in a frame path, a selector, or an attribute holding one is masked in the model's text, an executor's tree, and cache entries, and a secret with a line break, tab, CRLF, no-break space, or repeated spaces that an engine collapsed and then cut at the name or text limit no longer leaves its leading part in observations or end anchors.

• #798 02ad48a Thanks @okwasniewski! - Breaking: @e2e-dev/web depends on playwright-core pinned to an exact version instead of peering on playwright. Projects no longer install Playwright themselves, and the engine always runs the Playwright it was tested against. Remove playwright from your dependencies unless your app uses it for its own tests:

bash npm uninstall playwright

Install browsers in CI with the package's new command, which runs the engine's Playwright: npx @e2e-dev/web install chromium --with-deps (pnpm exec e2e-web install chromium --with-deps under pnpm) replaces npx playwright install chromium --with-deps. Open traces with npx playwright-core@1.63.0 show-trace <file> (pnpm dlx in a pnpm project). An app that also depends on @playwright/test keeps its own copy; the two share a browser cache only when their versions match. e2e init no longer adds playwright to a new project.

• #725 9b93575 Thanks @okwasniewski! - An expect.poll that the test body, an afterEach hook, or a fixture teardown returns without awaiting is now cancelled and fails that phase with STEP_NOT_AWAITED at the line of the call; in a beforeAll or afterAll hook it fails the hook with HOOK_FAILED. Before, the test passed and the poll's timeout failed whichever test ran next, or nothing at all.

The negation window of a negated expect(locator) or expect(browser) matcher now starts when the first read that saw the negation was issued, not when it returned, so a slow read counts toward it. With a timeout under a second, a negation that held for the whole budget now passes at the deadline; before, a slow first read made a true negation time out.

• #777 79931d0 Thanks @DeryFerd! - The replay cache abstracts short prefixed record ids as minted segments. A route segment of letters, a dash, and a digit tail of two or more (PROJ-016, INV-2041) now reduces to :id, so a step recorded on one record replays on the next instead of missing with wrong-context and running live every time. A single trailing digit (page-2) stays a route word.

• #724 3a9667c Thanks @okwasniewski! - --last-failed no longer goes green on tests it never ran. It reruns tests --max-failures skipped, and a test or failed beforeAll/afterAll another filter leaves out stays owed in the report's new run.carried until a rerun runs it. A rerun also keeps the artifacts of the run it reruns and writes its own under artifacts/rerun-<n>/ in the output directory (.e2e by default), so the folded pull request comment keeps its evidence and stays red while anything is owed.

• #721 94ddbfe Thanks @okwasniewski! - Two credentials or two secrets whose names map to the same override variable (api-key and api_key both read E2E_SECRET_API_KEY) now fail the config load with INVALID_CONFIG instead of both silently taking one value.

• #800 b5dc0f8 Thanks @okwasniewski! - --strict-cache now fails a step with REPLAY_STALE when the cache directory holds its recording under another key, for example after an e2e or engine upgrade or a change to the agent's context. Before, such a step ran live and spent model calls with no error. The message names the old entry file. A step with no recording still runs live.