Recent changes to Running Tests

Running Tests modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:53 -0000

--- v3
+++ v4
@@ -1,6 +1,6 @@
 # Running Tests

-Lastest supports three execution modes for running tests.
+Lastest supports two execution modes. **Local Playwright execution on the host is no longer supported** -- every test runs inside an Embedded Browser pod, even in local development.

 ---

@@ -8,11 +8,10 @@

 | Mode | How | When |
 |------|-----|------|
-| **Local** | Playwright runs on the same machine as Lastest | Development, debugging |
-| **Remote Runner** | Tests dispatched to remote machines via WebSocket | Distributed execution, different OS/browsers |
-| **Embedded Browser** | Browser runs in a container with live streaming back to the UI | Cloud deployments, recording/running without local Playwright |
+| **Embedded Browser** (default) | Browser runs in a containerized pod with CDP live streaming back to the UI. Provisioned dynamically into k3d locally, or into your cluster in production -- one EB per test. | Default for all dev and prod runs |
+| **Remote Runner** | Tests dispatched to remote machines via the `@lastest/runner` npm package over HTTP polling | Distributed execution, different OS/browsers, CI/CD |

-All three modes support both **running** and **recording** tests.
+Both modes support both **running** and **recording** tests. EB pods are short-lived (one job, one EB) -- the provisioner pre-warms a small pool to keep latency low.

 ---

@@ -77,11 +76,30 @@

 ---

+## Headed Playback
+
+Replay any test (or test version) with a visible browser window for debugging. Useful when a failure isn't reproducible in headless or you want to watch selectors resolve in real time. Available from the test detail page; honours the same setup/teardown sequence as a normal run.
+
+---
+
+## Recording Verification
+
+After a recording finishes, Lastest runs a sanity-check pass that resolves every captured selector on the page before saving the test. Bad selectors are surfaced immediately rather than at the next test run, dramatically reducing brittle-on-day-one failures.
+
+---
+
+## Watermarked Share Links
+
+Any build, run, or visual diff can be shared via a public link without requiring login. The shared view embeds a watermark and read-only controls so external reviewers (designers, PMs, customers) can see screenshots and approve/comment without an account on the instance.
+
+---
+
 ## Parallel Execution

 Configure the maximum number of tests that run simultaneously:

 - Set in [Settings Reference](Settings > Playwright)
-- Applies to both local and remote runners
+- Applies to the embedded-browser pool and remote runners
+- The EB pool size and warm-pool keep-alive are tuned via env (`EB_LAUNCH_INTERVAL_MS` etc.)
 - Each remote runner has its own parallel limit

 ---

Running Tests modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:52 -0000

--- v2
+++ v3
@@ -18,7 +18,7 @@

 ## Triggering Builds

-Builds can be triggered in three ways:
+Builds can be triggered in multiple ways:

 ### Manual
 Click **Run** in the UI to trigger a build immediately. You can:
@@ -36,6 +36,10 @@
 ### CI/CD

 - **GitHub Action** -- use the reusable action (see [CI CD Integration])
 - **CLI Runner** -- run `pnpm test:visual` in any CI pipeline
+
+### Schedule
+- **Cron-based** -- automate builds on preset schedules (daily, weekly, hourly) or custom cron expressions
+- See [Scheduled Runs] for details

 ---

Running Tests modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:51 -0000

--- v1
+++ v2
@@ -1,6 +1,6 @@
 # Running Tests

-Lastest2 supports three execution modes for running tests.
+Lastest supports three execution modes for running tests.

 ---

@@ -8,7 +8,7 @@

 | Mode | How | When |
 |------|-----|------|
-| **Local** | Playwright runs on the same machine as Lastest2 | Development, debugging |
+| **Local** | Playwright runs on the same machine as Lastest | Development, debugging |
 | **Remote Runner** | Tests dispatched to remote machines via WebSocket | Distributed execution, different OS/browsers |
 | **Embedded Browser** | Browser runs in a container with live streaming back to the UI | Cloud deployments, recording/running without local Playwright |

@@ -56,7 +56,7 @@
 Run only tests affected by your code changes:


 1. Select a feature branch (not main/master)
-2. Lastest2 compares against the default branch via GitHub/GitLab API
+2. Lastest compares against the default branch via GitHub/GitLab API
 3. Tests are matched to changed files by URL patterns and code references
 4. Only affected tests run, skipping unchanged areas

@@ -94,7 +94,7 @@

 ## Branch Baseline Management

-Lastest2 manages baselines per branch:
+Lastest manages baselines per branch:

 - **Fork**: When a branch is created, baselines are forked from the parent branch
 - **Merge**: When a PR is merged, baselines merge back to the target branch
 - **Promote**: Promote test versions across branches

Running Tests modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:50 -0000

Running Tests

Lastest2 supports three execution modes for running tests.

Execution Modes

Mode	How	When
Local	Playwright runs on the same machine as Lastest2	Development, debugging
Remote Runner	Tests dispatched to remote machines via WebSocket	Distributed execution, different OS/browsers
Embedded Browser	Browser runs in a container with live streaming back to the UI	Cloud deployments, recording/running without local Playwright

All three modes support both running and recording tests.

Triggering Builds

Builds can be triggered in three ways:

Manual

Click Run in the UI to trigger a build immediately. You can:

Run all tests in a repository
Run a specific test suite
Run individual tests
Use the Compose page to cherry-pick specific tests and versions

Webhook

When a PR is opened or updated:

GitHub webhooks trigger a build automatically
GitLab MR webhooks trigger a build automatically
Results are posted back as PR/MR comments

CI/CD

GitHub Action -- use the reusable action (see [CI CD Integration])
CLI Runner -- run pnpm test:visual in any CI pipeline

Build Statuses

Status	Meaning
`passed`	All tests passed, no visual changes
`safe_to_merge`	Tests passed, only approved/insignificant changes
`review_required`	Visual changes detected that need review
`blocked`	Unapproved meaningful changes present
`failed`	One or more tests failed to execute

Smart Run

Run only tests affected by your code changes:

Select a feature branch (not main/master)
Lastest2 compares against the default branch via GitHub/GitLab API
Tests are matched to changed files by URL patterns and code references
Only affected tests run, skipping unchanged areas

This dramatically reduces test time for large suites while maintaining coverage.

Debug Mode

Step through test execution with live feedback:

See each step as it executes
View screenshots at each step
Identify exactly where failures occur

Parallel Execution

Configure the maximum number of tests that run simultaneously:

Set in Settings Reference
Applies to both local and remote runners
Each remote runner has its own parallel limit

Test Composition

Cherry-pick tests and pin specific test versions per build:

Go to the Compose page
Select which tests to include
Optionally pin specific versions (override the latest version)
Run the composed build

Branch Baseline Management

Lastest2 manages baselines per branch:

Fork: When a branch is created, baselines are forked from the parent branch
Merge: When a PR is merged, baselines merge back to the target branch
Promote: Promote test versions across branches
SHA256 carry-forward: Matching hashes mean instant pass without pixel comparison

Setup & Teardown

Setup Scripts

Run preparation steps before tests execute:

Playwright scripts -- browser-based setup (login, seed data via UI)
API scripts -- HTTP requests to seed data programmatically
Test-as-setup -- use another test as a setup step

Teardown Scripts

Clean up after tests complete:

Repository-default teardown sequence
Per-test overrides (skip default steps or add extra ones)
Teardown errors are non-blocking -- they won't fail your build