#88 Escaped quotation marks in PGN header tag strings do not import and export as an unescaped backslash

[offroadtrip]

If a PGN header tag string (most commonly Events or TimeControl) contains a backslash escaped quotation mark, the tag string terminates at the quotation mark. So, e.g., [Event "an \" example"] imports with just 'an \' in the Event field. I made it work for my PGN by changing the while loop at line 344 of pgnparse.cpp to look ahead for a closing tag when a quote is found to decide whether to treat it as a character or an end quote, but I guess that would break if there's white space.

[Steven]

The solution is simple - rewrite the PGN standard... or stop using it :)

[offroadtrip]

The PGN standard allows quotation marks and backslashes in strings and specifies that they should be escaped by a backslash (cf. 4.7 Tokens, par. 2). I suppose the standard-correct approach is to treat a quotation mark as closing iff it is not preceded by a backslash (which pgn-extract, for example, does). The issue I encountered was importing correctly escaped quotation marks terminated tag strings at the backslash, so the tag becomes truncated and exports with no end-quote (since the new string ends in an unescaped backslash and makes the new end-quote an escaped-quote). Anyway, I fixed it for my purposes, so I'm not personally concerned if it's not considered an issue.

[Steven]

Thanks for that info. I'll look at it.

[Steven]

Yeah, looks like we should be handling \"
I see we're ignoring line comments too (semicolons).

[offroadtrip]

Great, thanks. I never realized how complicated text parsing is until I started trying to script some pgn stuff and looked at Scid (vs PC)'s and pgn-extract's source. I realized the way I phrased correct handling wasn't exactly correct since technically \" could be \\" (escaped backslash and non escaped quote). I don't think Lumbra's Gigabase (which I was trying to clean with Scid/vs PC/pgn-extract) has any escaped backslashes, but grep counted 6600 lines with escaped quotes.