#2245 Assert on preprocessor parsing in released version

[Isiledhel]

Scite version 4.4.6 on Mac OS.
Just hiting # key make the editor crash.

Assertion failed: (sc.state == (SCE_C_PREPROCESSOR|inactiveFlag)), function Lex, file /Users/neil/merc/scintilla/lexers/LexCPP.cxx, line 1320.

Assert are not supposed to be in release version; still this is the sequence with which I can reproduce, to improve the assert test:

With attached file

• go to line 835
• hit enter to get a new line
• hit # to start a preprocessor block : crash.
  alternate:
• go to line 836
• hit # to start a preprocessor block
• hit enter to get a new line: crash

[Neil Hodgson]

I can't manage to make the failure occur. LexCPP.cxx, line 1320 is the '#else' handler (with current if not taken) so the assertion likely failing after lexing the example file line mentioned and more lines up to 890 where the next #else is.
Line 835 is

        if(fd >= 0) {

so with a '#' at the start

#        if(fd >= 0) {

looks like a preprocessor expression. Now 'fd' is normally undefined unless it has been added to keywords5 (preprocessor values) like so

keywords5.$(file.patterns.cpp)=GTK_MAJOR_VERSION=2 fd=4

The expression (with fd undefined) tokenizes to

[(,fd,>=,0,),{]

then evaluates variables to

[(,0,>=,0,),{]

removes brackets to

[0,>=,0]

then evaluates to

[1]

Which is true so that #if is active and the following code remains active. So then the line 890 #else should be switching the code to inactive but that won't hit the assertion.

Trying to define fd=-1 doesn't deactivate the section since the expression evaluator doesn't handle unary - correctly. fd=(0-1) is understood better but still doesn't deactivate the code due to the stray '{'. Changing the expression to (fd > 0) and defining fd as 0 then the conditions to reach the assert are set up but the assert still doesn't fail.

Its possible you have other settings that could affect this code and I'd like to work out what is really happening.

Assertions will be turned off in the next release 5.0.2. Today's 5.0.1 release still has assertions turned on.

[Isiledhel]

Hi Neil,

I was not clear enough in my description:
First step is to go at the end of line 835, so it would look like:
835 if(fd >= 0) {
(++) #if 0
836 while( ri ) {

Here are the potentially relevant options I have in my SciTEUser.properties file:
save.recent=1
save.session=1
session.bookmarks=1
session.folds=1
save.position=1
save.find=1

strip.trailing.spaces=1
ensure.final.line.end=1
ensure.consistent.line.ends=1

wrap=0

fold.on.open=1

I also defined user keywords (keywords2) and added custom definitions (extra types) in keywords but neither of these are used in this file (external source) and neither feature # character either.
I don’t know what’s the state of my session because of the repeated tries and the crashes (sometimes the session felt preserved, sometimes not) didn’t restore always the same state.

Lastly, I the the fold.on.open option active, which didn’t behave the same in all cases (maybe some interaction with the session): sometimes all was really folded, sometimes it was a partial folding only. (I remember because there was cases I had to locate the line 835 and cases where the editor would open directly in the area of the line).

Happy hunting ;)

Stéphane

On 11 Apr 2021, at 01:56, Neil Hodgson nyamatongwe@users.sourceforge.net wrote:

I can't manage to make the failure occur. LexCPP.cxx, line 1320 is the '#else' handler (with current if not taken) so the assertion likely failing after lexing the example file line mentioned and more lines up to 890 where the next #else is.
Line 835 is

    if(fd >= 0) {

so with a '#' at the start

if(fd >= 0) {

looks like a preprocessor expression. Now 'fd' is normally undefined unless it has been added to keywords5 (preprocessor values) like so

keywords5.$(file.patterns.cpp)=GTK_MAJOR_VERSION=2 fd=4
The expression (with fd undefined) tokenizes to

[(,fd,>=,0,),{]
then evaluates variables to

[(,0,>=,0,),{]
removes brackets to

[0,>=,0]
then evaluates to

[1]
Which is true so that #if is active and the following code remains active. So then the line 890 #else should be switching the code to inactive but that won't hit the assertion.

Trying to define fd=-1 doesn't deactivate the section since the expression evaluator doesn't handle unary - correctly. fd=(0-1) is understood better but still doesn't deactivate the code due to the stray '{'. Changing the expression to (fd > 0) and defining fd as 0 then the conditions to reach the assert are set up but the assert still doesn't fail.

Its possible you have other settings that could affect this code and I'd like to work out what is really happening.

Assertions will be turned off in the next release 5.0.2. Today's 5.0.1 release still has assertions turned on.

[bugs:#2245] https://sourceforge.net/p/scintilla/bugs/2245/ Assert on preprocessor parsing in released version

Status: open
Group: Bug
Created: Sat Apr 10, 2021 01:17 PM UTC by Isiledhel
Last Updated: Sat Apr 10, 2021 01:17 PM UTC
Owner: Neil Hodgson
Attachments:

jffs2extract.c https://sourceforge.net/p/scintilla/bugs/2245/attachment/jffs2extract.c (24.6 kB; application/octet-stream)
Scite version 4.4.6 on Mac OS.
Just hiting # key make the editor crash.

Assertion failed: (sc.state == (SCE_C_PREPROCESSOR|inactiveFlag)), function Lex, file /Users/neil/merc/scintilla/lexers/LexCPP.cxx, line 1320.

Assert are not supposed to be in release version; still this is the sequence with which I can reproduce, to improve the assert test:

With attached file
go to line 835
hit enter to get a new line
hit # to start a preprocessor block : crash.
alternate:
go to line 836
hit # to start a preprocessor block
hit enter to get a new line: crash

Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/scintilla/bugs/2245/ https://sourceforge.net/p/scintilla/bugs/2245/
To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/ https://sourceforge.net/auth/subscriptions/

[Neil Hodgson]

A bare '#' or '#' followed only by whitespace caused the preprocessor history to be attached to the wrong lines which then caused the '#else' to be treated as not having a corresponding '#if' so changing the active state failed. There were also assertions if there was an actual dangling '#else'.

Fixed with
https://github.com/ScintillaOrg/lexilla/commit/d4ee34f50264a75727e465ac55653ff39278d0b4

Turned off assertions in release builds on macOS with
https://github.com/ScintillaOrg/lexilla/commit/7a35d25bb5ae00e1af66dcd50367f08ecef3a640