Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

Tcl syntax highlighting

2. Help
Rick
2014-05-23
2014-06-07
  • Rick
    Rick
    2014-05-23

    I have this proc in some Tcl

    proc AddType {type} {
        set map [GetMap]
        if {[info exists map("$type")]} {    
            set frame $map("$type")
        } else {
            set frame [MakeFrame]
        }
    }
    

    which works correctly. Unfortunately, something in the middle of it "breaks" Notepad++'s syntax highlighting, such that it shows like this:

    screenshot showing syntax highlighting error

    If I change $map("$type") to $map( "$type") then all is happy in terms of the syntax highlighting:

    screenshot showing correct syntax highlighting

    however the Tcl then fails to load. This is some legacy Tcl, and this proc is near the start of a pretty large file and all of the syntax highlighting is wrong from this point to the end of the file, so it's pretty irritating.

    Is this a Notepad++ bug? Could it be fixed?

     
  • cchris
    cchris
    2014-05-24

    Could you attach the possibly shortened file for us to examine it?

    CChris

     
  • Rick
    Rick
    2014-05-27

    The Tcl extract at the top of my message, placed in a file on its own, is sufficient to reproduce it here.

     
  • cchris
    cchris
    2014-05-31

    Ok, I was able to trace the problem down.
    This is a bug in the TCL lexer that is supplied with Scintilla, and should be reported on <www.scintilla.org>.
    In a nutshell:
    - at line 4, the characters "map" are in a style called "substitution" - I don't know TCL, so I just assume the $ symbol requst to substitute text, which is probably a varuable name, with the content of the variable -.
    - when the ( is encountered, ait is properly styled as operator, but the immediatelly following double quote is set to "substitution" style. Here the trouble starts.
    - As a result, the next double quote is seen as starting an "in quote" segment.
    - Actually the lexer could still recover at this point, but then it comes to line 136:

            default :
                // maybe spaces should be allowed ???
                if (!IsAWordChar(sc.ch)) { // probably the code is wrong
                    sc.SetState(SCE_TCL_DEFAULT);
                    subParen = 0;
                }
    

    not only spaces should be allowed, but the double quote should be spotted here, and the style should go to SCE_TCL_IN_QUOTE, not SCE_TCL_DEFAULT.
    - in contrast, line 3 is not a substitution and everything looks ok.

    Apart from integrating a revised lexer when it decides to upgrade Scintilla, there is nothing N++ can do about it. In the meantime, follow open parenthesis in substitution with a space, that gets the lexer back to reason.

    CChris

     
  • Rick
    Rick
    2014-06-02

    Thanks for tracking that one down, Chris. Is there anything else I should do in terms of "reporting" such that it gets tracked, reported as a bug in Scintilla, identified as and when it gets fixed there, incorporated and finally released, or may I assume that that's already in hand?

    To my immediate needs, I also asked on StackOverflow from the Tcl perspective, and now have the workaround that I can re-code the offending fragment like this:

    if {[info exists map($type)]} {    
        set frame $map($type)
    

    i.e. "simply" remove the quotes. I do of course need to make other adjustments elsewhere to accommodate that, and make sure that the quotes don't cause knock-on problems, but it's looking good, so that's another suggestion you can hang on to if anyone else comes up against the same issue in the future.

    Rick

     
  • cchris
    cchris
    2014-06-07

    I think posting my report and your picture on the Scintilla mailing list should be ok. The problem is not complex by itself, it's just that I have no idea whether anyone maintains it currently, nor do I know whether what you type is deemed standard Tc or some dialect of it.

    After that, there is not much you can do. Scintilla will some day upgrade with a fixed lexer, and sometimes later N++ will upgrade the Scintilla version it builds upon, at which point every other fix will be brought into N++.

    CChris