Running Tests

Running Tests

Lastest supports two execution modes. Local Playwright execution on the host is no longer supported -- every test runs inside an Embedded Browser pod, even in local development.

---

Execution Modes

Mode	How	When
Embedded Browser (default)	Browser runs in a containerized pod with CDP live streaming back to the UI. Provisioned dynamically into k3d locally, or into your cluster in production -- one EB per test.	Default for all dev and prod runs
Remote Runner	Tests dispatched to remote machines via the @lastest/runner npm package over HTTP polling	Distributed execution, different OS/browsers, CI/CD

Both modes support both running and recording tests. EB pods are short-lived (one job, one EB) -- the provisioner pre-warms a small pool to keep latency low.

---

Triggering Builds

Builds can be triggered in multiple ways:

Manual

Click Run in the UI to trigger a build immediately. You can:

• Run all tests in a repository
• Run a specific test suite
• Run individual tests
• Use the Compose page to cherry-pick specific tests and versions

Webhook

When a PR is opened or updated:

• GitHub webhooks trigger a build automatically
• GitLab MR webhooks trigger a build automatically
• Results are posted back as PR/MR comments

CI/CD

• GitHub Action -- use the reusable action (see [CI CD Integration])
• CLI Runner -- run pnpm test:visual in any CI pipeline

Schedule

• Cron-based -- automate builds on preset schedules (daily, weekly, hourly) or custom cron expressions
• See [Scheduled Runs] for details

---

Build Statuses

Status	Meaning
passed	All tests passed, no visual changes
safe_to_merge	Tests passed, only approved/insignificant changes
review_required	Visual changes detected that need review
blocked	Unapproved meaningful changes present
failed	One or more tests failed to execute

---

Smart Run

Run only tests affected by your code changes:

1. Select a feature branch (not main/master)
2. Lastest compares against the default branch via GitHub/GitLab API
3. Tests are matched to changed files by URL patterns and code references
4. Only affected tests run, skipping unchanged areas

This dramatically reduces test time for large suites while maintaining coverage.

---

Debug Mode

Step through test execution with live feedback:

• See each step as it executes
• View screenshots at each step
• Identify exactly where failures occur

---

Headed Playback

Replay any test (or test version) with a visible browser window for debugging. Useful when a failure isn't reproducible in headless or you want to watch selectors resolve in real time. Available from the test detail page; honours the same setup/teardown sequence as a normal run.

---

Recording Verification

After a recording finishes, Lastest runs a sanity-check pass that resolves every captured selector on the page before saving the test. Bad selectors are surfaced immediately rather than at the next test run, dramatically reducing brittle-on-day-one failures.

---

Watermarked Share Links

Any build, run, or visual diff can be shared via a public link without requiring login. The shared view embeds a watermark and read-only controls so external reviewers (designers, PMs, customers) can see screenshots and approve/comment without an account on the instance.

---

Parallel Execution

Configure the maximum number of tests that run simultaneously:

• Set in Settings Reference
• Applies to the embedded-browser pool and remote runners
• The EB pool size and warm-pool keep-alive are tuned via env (EB_LAUNCH_INTERVAL_MS etc.)
• Each remote runner has its own parallel limit

---

Test Composition

Cherry-pick tests and pin specific test versions per build:

1. Go to the Compose page
2. Select which tests to include
3. Optionally pin specific versions (override the latest version)
4. Run the composed build

---

Branch Baseline Management

Lastest manages baselines per branch:

• Fork: When a branch is created, baselines are forked from the parent branch
• Merge: When a PR is merged, baselines merge back to the target branch
• Promote: Promote test versions across branches
• SHA256 carry-forward: Matching hashes mean instant pass without pixel comparison

---

Setup & Teardown

Setup Scripts

Run preparation steps before tests execute:

• Playwright scripts -- browser-based setup (login, seed data via UI)
• API scripts -- HTTP requests to seed data programmatically
• Test-as-setup -- use another test as a setup step

Teardown Scripts

Clean up after tests complete:

• Repository-default teardown sequence
• Per-test overrides (skip default steps or add extra ones)
• Teardown errors are non-blocking -- they won't fail your build