offroadtrip

Great, thanks. I never realized how complicated text parsing is until I started trying to script some pgn stuff and looked at Scid (vs PC)'s and pgn-extract's source. I realized the way I phrased correct handling wasn't exactly correct since technically \" could be \\" (escaped backslash and non escaped quote). I don't think Lumbra's Gigabase (which I was trying to clean with Scid/vs PC/pgn-extract) has any escaped backslashes, but grep counted 6600 lines with escaped quotes.

Great, thanks. I never realized how complicated text parsing is until I started trying to script some pgn stuff and looked at Scid (vs PC)'s and pgn-extract's source. I realized the way I phrased correct handling wasn't exactly correct since technically \" could be \\" (escaped backslash and non escaped quote). I don't think Lumbra's Gigabase (which I was trying to clean with Scid/vs PC/pgn-extract) has any escaped backslashes, but grep counted about 6600 lines with escaped quotes.

Great, thanks. I never realized how complicated text parsing is until I started trying to script some pgn stuff and looked at Scid (vs PC)'s and pgn-extract's source. I realized the way I phrased correct handling wasn't exactly correct since technically \" could be \" (escaped backslash and non escaped quote). I don't think Lumbra's Gigabase (which I was trying to clean with Scid/vs PC/pgn-extract) has any escaped backslashes, but grep counted about 6600 lines with escaped quotes.

The PGN standard allows quotation marks and backslashes in strings and specifies that they should be escaped by a backslash (cf. 4.7 Tokens, par. 2). I suppose the standard-correct approach is to treat a quotation mark as closing iff it is not preceded by a backslash (which pgn-extract, for example, does). The issue I encountered was importing correctly escaped quotation marks terminated tag strings at the backslash, so the tag becomes truncated and exports with no end-quote (since the new string...

Escaped quotation marks in PGN header tag strings do not import and export as an unescaped backslash

PGN export does not escape quotes (and backslashes?) in header tag strings