Tcl syntax highlighting

[Rick]

I have this proc in some Tcl

proc AddType {type} {
    set map [GetMap]
    if {[info exists map("$type")]} {    
        set frame $map("$type")
    } else {
        set frame [MakeFrame]
    }
}

which works correctly. Unfortunately, something in the middle of it "breaks" Notepad++'s syntax highlighting, such that it shows like this:

If I change $map("$type") to $map( "$type") then all is happy in terms of the syntax highlighting:

however the Tcl then fails to load. This is some legacy Tcl, and this proc is near the start of a pretty large file and all of the syntax highlighting is wrong from this point to the end of the file, so it's pretty irritating.

Is this a Notepad++ bug? Could it be fixed?

[cchris]

Could you attach the possibly shortened file for us to examine it?

CChris

[Rick]

The Tcl extract at the top of my message, placed in a file on its own, is sufficient to reproduce it here.

[cchris]

Ok, I was able to trace the problem down.
This is a bug in the TCL lexer that is supplied with Scintilla, and should be reported on <www.scintilla.org>.
In a nutshell:
- at line 4, the characters "map" are in a style called "substitution" - I don't know TCL, so I just assume the $ symbol requst to substitute text, which is probably a varuable name, with the content of the variable -.
- when the ( is encountered, ait is properly styled as operator, but the immediatelly following double quote is set to "substitution" style. Here the trouble starts.
- As a result, the next double quote is seen as starting an "in quote" segment.
- Actually the lexer could still recover at this point, but then it comes to line 136:

        default :
            // maybe spaces should be allowed ???
            if (!IsAWordChar(sc.ch)) { // probably the code is wrong
                sc.SetState(SCE_TCL_DEFAULT);
                subParen = 0;
            }

not only spaces should be allowed, but the double quote should be spotted here, and the style should go to SCE_TCL_IN_QUOTE, not SCE_TCL_DEFAULT.
- in contrast, line 3 is not a substitution and everything looks ok.

Apart from integrating a revised lexer when it decides to upgrade Scintilla, there is nothing N++ can do about it. In the meantime, follow open parenthesis in substitution with a space, that gets the lexer back to reason.

CChris

[Rick]

Thanks for tracking that one down, Chris. Is there anything else I should do in terms of "reporting" such that it gets tracked, reported as a bug in Scintilla, identified as and when it gets fixed there, incorporated and finally released, or may I assume that that's already in hand?

To my immediate needs, I also asked on StackOverflow from the Tcl perspective, and now have the workaround that I can re-code the offending fragment like this:

if {[info exists map($type)]} {    
    set frame $map($type)

i.e. "simply" remove the quotes. I do of course need to make other adjustments elsewhere to accommodate that, and make sure that the quotes don't cause knock-on problems, but it's looking good, so that's another suggestion you can hang on to if anyone else comes up against the same issue in the future.

Rick

[cchris]

I think posting my report and your picture on the Scintilla mailing list should be ok. The problem is not complex by itself, it's just that I have no idea whether anyone maintains it currently, nor do I know whether what you type is deemed standard Tc or some dialect of it.

After that, there is not much you can do. Scintilla will some day upgrade with a fixed lexer, and sometimes later N++ will upgrade the Scintilla version it builds upon, at which point every other fix will be brought into N++.

CChris