Recent changes to Creating Tests

Creating Tests modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:53 -0000

--- v3
+++ v4
@@ -36,6 +36,10 @@

 2. Use "Generate with AI" and provide a URL or description
 3. Review the generated test code
 4. Edit if needed, then save
+
+### Enhance with AI (live MCP wiring)
+
+The "Enhance with AI" flow now streams progress through the Lastest MCP server: as AI plans new test cases the real test names appear in the UI before the code is generated, and selectors are validated against the live page via MCP tooling rather than guessed. This drastically reduces "test renamed in code but still labelled `untitled-3` in the UI" drift and surfaces selector failures while AI is still iterating.

 **Best for:** Day-to-day development, iterating on tests, fixing breakages fast.

@@ -140,3 +144,20 @@

 - **Playwright Inspector** -- The official Playwright recording tool

 Configure the default engine in [Settings Reference](Settings > Playwright).
+
+---
+
+## Recording Verification
+
+After you stop a recording, Lastest replays the captured steps against a fresh page and verifies every selector resolves before saving the test. Selectors that resolved during the original recording but fail on replay (because the page rebuilt the DOM, an element was conditionally rendered, etc.) are surfaced immediately so you can pick a sturdier selector or add an explicit wait. This sanity-check step prevents the classic "the test passed when I recorded it, then failed on the very next run" failure mode.
+
+---
+
+## Test Specs & Agent Plans
+
+The freeform `description` columns on tests and functional areas have been replaced with two structured fields:
+
+- **`test_specs`** -- the user-facing description of what a test should do, used by AI when re-generating or healing the test
+- **`agent_plan`** -- the Play Agent's planning output (functional area decomposition, test case rationale)
+
+Existing descriptions were backfilled into `test_specs` when the v1.12 schema migration ran. The Compose page and AI prompts read from these structured fields directly.

Creating Tests modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:52 -0000

--- v2
+++ v3
@@ -43,17 +43,21 @@

 ## 3. Full Autonomous (Play Agent)

-One click kicks off a 9-step pipeline:
+One click kicks off an 11-step pipeline:

-1. Scan your repo for routes
-2. Classify your app type
-3. Generate tests for discovered routes
-4. Run the tests
-5. Fix failures (up to 3 attempts per test)
-6. Re-run fixed tests
-7. Report results
+1. Check settings and AI provider
+2. Select repository
+3. Environment setup
+4. Scan routes and apply testing template
+5. Plan functional areas
+6. Review plan (human approval checkpoint)
+7. Generate tests
+8. Run the tests
+9. Fix failures (up to 3 attempts per test)
+10. Re-run fixed tests
+11. Report results

-The agent pauses and asks for help only when it hits something it can't resolve on its own (missing settings, server offline). You resume and it picks up where it left off.
+Uses specialized sub-agents: Orchestrator, Planner, Scout, Diver, Generator, and Healer. The agent pauses and asks for help only when it hits something it can't resolve on its own (missing settings, server offline). You can pause/resume, approve plans, and skip steps. Monitor progress in real-time via the [Agent Monitoring](activity feed).

 **How to use:**

 1. Configure an AI provider in Settings

Creating Tests modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:51 -0000

--- v1
+++ v2
@@ -1,12 +1,12 @@
 # Creating Tests

-Lastest2 offers three ways to create tests, from fully manual to fully autonomous.
+Lastest offers three ways to create tests, from fully manual to fully autonomous.

 ---

 ## 1. AI-Free (Manual Recording)

-Open the recorder, click through your app, hit stop. Lastest2 captures every interaction and generates deterministic Playwright code -- no AI involved, no API keys needed. You own the test code and can edit it by hand.
+Open the recorder, click through your app, hit stop. Lastest captures every interaction and generates deterministic Playwright code -- no AI involved, no API keys needed. You own the test code and can edit it by hand.

 **How to use:**

 1. Navigate to your repository
@@ -119,7 +119,7 @@

 ## Auto-Detect Capabilities

-When recording tests, Lastest2 automatically detects required browser capabilities:
+When recording tests, Lastest automatically detects required browser capabilities:

 - File upload
 - Clipboard access
 - Downloads
@@ -131,8 +131,8 @@

 ## Recording Engines

-Lastest2 supports two recording engines:
-- **Custom Recorder** -- Lastest2's built-in recorder with enhanced capabilities
+Lastest supports two recording engines:
+- **Custom Recorder** -- Lastest's built-in recorder with enhanced capabilities

 - **Playwright Inspector** -- The official Playwright recording tool

 Configure the default engine in [Settings Reference](Settings > Playwright).

Creating Tests modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:48 -0000

Creating Tests

Lastest2 offers three ways to create tests, from fully manual to fully autonomous.

1. AI-Free (Manual Recording)

Open the recorder, click through your app, hit stop. Lastest2 captures every interaction and generates deterministic Playwright code -- no AI involved, no API keys needed. You own the test code and can edit it by hand.

How to use:

Navigate to your repository
Click Record
Enter the URL of your app
Interact with your app -- click, type, scroll, navigate
Click Stop when done
Review the generated Playwright code
Save the test

Best for: Teams that don't want AI, air-gapped environments, simple flows.

2. AI-Assisted (Human-in-the-Loop)

AI generates, fixes, or enhances tests -- but you review and approve before anything is saved.

Capabilities:

Feed it a URL and get a test back
Import OpenAPI specs, user stories, or markdown files -- AI extracts test cases
When a test breaks, AI proposes a fix and you decide whether to accept it
Enhance existing tests with additional coverage

How to use:

Make sure you have an AI provider configured in AI Configuration
Use "Generate with AI" and provide a URL or description
Review the generated test code
Edit if needed, then save

Best for: Day-to-day development, iterating on tests, fixing breakages fast.

3. Full Autonomous (Play Agent)

One click kicks off a 9-step pipeline:

Scan your repo for routes
Classify your app type
Generate tests for discovered routes
Run the tests
Fix failures (up to 3 attempts per test)
Re-run fixed tests
Report results

The agent pauses and asks for help only when it hits something it can't resolve on its own (missing settings, server offline). You resume and it picks up where it left off.

How to use:

Configure an AI provider in Settings
Navigate to your repository
Click Play Agent
Let it run -- monitor progress in the UI
Review results when complete

Best for: Onboarding a new project, generating full coverage from scratch, CI bootstrapping.

Spec-Driven Test Generation

Import structured specifications and let AI generate tests from them:

OpenAPI specs -- import your API spec and generate tests for each endpoint
User stories -- paste or import user stories in any format
Markdown files -- import documentation or requirements

How to use:

Navigate to your repository's test list
Click Import Spec
Paste or upload your spec/stories
AI extracts test cases and generates Playwright code
Review and save individual tests

Test Organization

Functional Areas

Organize tests into a nested hierarchy of functional areas:

Create parent and child areas (e.g., "Auth" > "Login", "Auth" > "Registration")
Drag and drop to reorder
Tests within an area run together

Test Suites

Group tests into ordered suites for structured execution:

Create named suites
Add tests in a specific order
Run entire suites at once

Test Versioning

Every change to a test creates a new version with a change reason:

Manual edit
AI fix
AI enhance
Restored (from a previous version)

You can view the full history and restore any previous version.

Multi-Step Screenshots

Capture multiple labeled screenshots within a single test for multi-page flow testing. Each screenshot is compared independently against its own baseline.

Auto-Detect Capabilities

When recording tests, Lastest2 automatically detects required browser capabilities:

File upload
Clipboard access
Downloads
Network interception

These are automatically enabled in the corresponding Playwright settings.

Recording Engines

Lastest2 supports two recording engines:

Custom Recorder -- Lastest2's built-in recorder with enhanced capabilities
Playwright Inspector -- The official Playwright recording tool

Configure the default engine in Settings Reference.