Recent changes to Visual Diffing

Visual Diffing modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:54 -0000

--- v2
+++ v3
@@ -66,13 +66,22 @@

 ---

-## Ignore Regions
+## Ignore Regions (per-step)

 Mask dynamic areas from comparison:

 - Draw ignore regions directly on the screenshot
-- Regions are excluded from diff calculation
+- Regions are now scoped **per step** (per-screenshot), not per-test -- a region you draw on the "Dashboard" screenshot won't bleed into the "Settings" screenshot of the same test
+- Excluded from diff calculation
 - Useful for timestamps, ads, counters, avatars, etc.
 - Configurable mask styles: solid-color or placeholder-text
+
+> Migration note: existing per-test ignore regions are migrated to a `step_label` of `null` (apply-to-all). New regions are bound to the screenshot they were drawn on.
+
+---
+
+## DOM-Based Diff Fallback
+
+When the pixel diff is inconclusive (e.g. very small differences clustered around inserted/removed nodes), Lastest can run a structural DOM diff in addition to the pixel comparison. The DOM diff is computed from the captured DOM snapshot at screenshot time and surfaces the affected node tree alongside the visual diff. Toggle in **Settings > Diff Sensitivity**.

 ---

Visual Diffing modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:52 -0000

--- v1
+++ v2
@@ -1,6 +1,6 @@
 # Visual Diffing

-Lastest2 offers three comparison engines and extensive configuration for reducing false positives.
+Lastest offers three comparison engines and extensive configuration for reducing false positives.

 ---

Visual Diffing modified by Viktor Fási

Viktor Fási — Wed, 06 May 2026 09:06:51 -0000

Visual Diffing

Lastest2 offers three comparison engines and extensive configuration for reducing false positives.

Diff Engines

pixelmatch (default)

Pixel-perfect comparison. Fast and deterministic. Best for most use cases.

SSIM (Structural Similarity Index)

Measures structural similarity between images. More tolerant of minor rendering differences while catching meaningful layout changes.

Butteraugli

Human-perception-aligned comparison. Models how the human visual system perceives differences. Best for catching changes that users would actually notice.

Configure the default engine in Settings > Diff Sensitivity.

Text-Region-Aware Diffing

OCR-based two-pass comparison with separate thresholds for text vs non-text regions:

First pass: OCR detects text regions in the screenshot
Second pass: Text regions are compared with a more tolerant threshold (cross-OS font rendering varies)
Non-text regions use the standard threshold

This dramatically reduces false positives from dynamic text and font rendering differences across operating systems.

Enable in Settings > Diff Sensitivity.

Diff Sensitivity Configuration

Setting	Description
Diff engine	pixelmatch, SSIM, or Butteraugli
Text-region-aware	Enable OCR-based two-pass comparison
Pixel threshold	Number of different pixels to trigger a change
Percentage threshold	Percentage of different pixels to trigger a change
Classification	Unchanged / Flaky / Changed thresholds

Diff Classifications

Classification	Meaning
Unchanged	No meaningful difference detected
Flaky	Minor differences within the flaky threshold (likely noise)
Changed	Meaningful visual difference that needs review

AI Diff Analysis

When configured, AI can automatically classify visual diffs:

Insignificant -- rendering noise, minor font differences
Meaningful -- actual UI changes that need review
With confidence scores and change category labels

Configure a separate AI provider for diff analysis in AI Configuration.

Ignore Regions

Mask dynamic areas from comparison:

Draw ignore regions directly on the screenshot
Regions are excluded from diff calculation
Useful for timestamps, ads, counters, avatars, etc.
Configurable mask styles: solid-color or placeholder-text

Planned Screenshots

Compare test results against design files:

Upload a design file (Figma export, PNG, etc.)
Run the test
See a separate planned vs actual diff alongside the baseline diff

Page Shift Detection

Detect vertical content shifts (inserted/deleted rows):

Fuzzy row matching identifies shifted content
Distinguishes between actual changes and simple insertions/deletions
Avoids flagging every pixel below an inserted row as "changed"

Diff Engine Benchmarks

Built-in benchmark framework to compare all three engines:

Synthetic test scenarios with known differences
Timing and accuracy metrics
Helps you choose the best engine for your use case