#1615 Feature patch: Ignore hyphens, quotes and line breaks in search

[Alex Jpn]

Hi

When searching technical PDFs, drafts, and books in Skim, exact string matching frequently misses intended text because of typesetting artifacts:

1. End-of-line justification hyphens (e.g. "dis-" at the end of a line and "cussion" on the next line).
2. Compound words written interchangeably with hyphens or spaces (e.g. "ultra-high" vs "ultra high").
3. Typography variations: smart double quotes (“ ”), smart apostrophes/single quotes (‘ ’), and dashes (– —) versus standard ASCII (" ' -).
4. American quotation style, where punctuation is placed inside quotation marks (e.g., searching for "in the clear" misses “in the clear,” due to the comma inside the quotes).
5. Multi-word phrases that wrap across line breaks.
   This patch adds an optional "Ignore Hyphens, Quotes & Line Breaks" search mode (enabled by default, fully toggleable via both the Find Bar and Sidebar search menus under the key SKSmartSearch). When unchecked, Skim falls back to standard literal search with zero overhead.

Technical Implementation

1. PDFDocument_SKExtensions:
2. Implements SKSmartPageText: Lazily caches a normalized text representation for each PDFPage.
3. Quotation marks are ignored during matching so quotation styles and interior punctuation (e.g., commas before closing quotes) do not block phrase matches.
4. Hyphens, dashes, and whitespace sequences are treated as interchangeable word separators (matching [- ]+).
5. Line-broken hyphenated words preserve the hyphen in normalized text while skipping newline and margin indentation, allowing both compound queries ("ultra-high", "ultra high") and single-word queries ("discussion") to match.
6. Reconstructs accurate character selections using an indexMap mapping back to the page's original character offsets.
7. Provides synchronous smartFindString:fromSelection:withOptions:forward: for the Find Bar and background multi-string search beginSmartFindStrings:withOptions:delegate: with cooperative cancellation.
8. Defensive Handling for Apple PDFKit Multi-Line / Table Layout (macOS 15 Tahoe):
9. Apple's PDFKit on macOS 15 has documented edge cases where calling [PDFPage selectionForRange:] or character bounds inside tables and complex layout blocks returns empty rects, infinite coordinates, or non-contiguous cells (partially addressed in Skim commits r16391, r16397–r16400).
10. Because smart search regularly matches across wrapped lines and table structures, it triggers this upstream PDFKit issue more frequently than literal single-line search.
11. PDFSelection_SKExtensions adds safeSelectionByFlattenting and safe bounds accessors to ensure multi-line highlights and viewport scroll-to-rect operations remain deterministic without crashing or misaligning.
12. UI & Preferences:
13. Added menu item "Ignore Hyphens, Quotes & Line Breaks" to both FindBar and LeftSideView search fields.
14. Persisted in NSUserDefaults under SKSmartSearchKey (@"SKSmartSearch"), defaulting to YES in InitialUserDefaults.plist.
15. Synchronized dynamically between the Find Bar and Sidebar search controls.
    Tested against large documents (including a 700+ page IEEE draft standard). Search speed across 700+ pages takes under 50ms with no UI stutter.
    The attached patch is relative to the codebase root (skim-app-code/) and applies cleanly against trunk (revision 16412).

[Christiaan Hofman]

The search is done by APple's PDFKit. We are not going to change that. Working behind PDFKit's back has always been a nightmare to maintain, so tghat is not an option. We also don't have access to the internals to get the text when needed most efficiently, as well as layout control of the text (e.g. multi columns).

[Christiaan Hofman]

Sorry, I did not understand this was meant as an addition, rather than a replacement. But then this is not a bug report, but rather a feature request.

I am still very hesitant to add it though, as it is a large overhead (also in memory), and just overlaps considerably with an already present feature, so the enhancement is somewhat limited.

[Alex Jpn]

Thanks for the feedback, understood. Yes it would be another search option, rather than a replacement.

Regarding memory optimization, I made some improvements in the new attached diff

• Removed all associated object retention on PDFPage.
• Implemented an NSCache bounded to 30 pages maximum (countLimit = 30). Even during long asynchronous searches over thousands of pages, at most 30 pages of character maps exist in RAM, and NSCache automatically evicts under system memory pressure.
• Added [SKSmartPageText clearCache], which is called immediately as soon as a search completes (documentDidEndDocumentFind:) or is cancelled (cancelSmartFind), completely freeing all temporary buffers.

Testing the actual memory profile on a 7,000+ page PDF doc, macOS's footprint tool says:

• Upstream Trunk Release (standard beginFindStrings:):
  Physical Footprint: 3,778 MB | MALLOC_SMALL: 3,543 MB (999 regions)
• With Smart Search Enabled (Debug build):
  Physical Footprint: 3,787 MB | MALLOC_SMALL: 3,552 MB (1,003 regions)
  The large ~3.7 GB footprint on a 7,000-page document is almost entirely Apple PDFKit inflating its internal glyph metrics and text layout caches across all pages.

I agree it's theoretically possible that searching the [page string] provided by PDFkit could differ from PDFkit's native search due to any differences in the reading order it uses. But at least in my testing I have not experienced differences in search results caused by that, whereas I do frequently miss search results with the current search function due to hyphens at the end of lines etc.

Anyway, if you feel that maintaining any search logic alongside PDFKit is outside Skim's architectural goals, I completely understand and respect your decision. Thank you for your time and continued maintenance of Skim!