#541 New completely rewritten LexPascal lexer

Completed
closed
Scintilla (358)
5
2009-04-28
2008-10-23
No

Attached you'll find new completely rewritten LexPascal lexer. I've spent some time to fix all noticed problems with syntax highlighting and folding in current lexer and also to port it to the new StyleContext framework. Besides this, I also implemented much better folding support that is able to fold more types of code blocks (i.e. folding of groups of consecutive comment lines, folding of interface declarations, proper folding of class declarations, etc.) and also properly handle various special situations when fold points should not be created.

On top of my head some of the problems with current lexer are: wrong highlighting of preprocessor directives that use alternative syntax (*$...*), wrong highlighting of double dots, last keyword in document is not highlighted if there's no additional EOL character, number highlighting is sometimes wrong, there's no line continuation in Pascal inside preprocessor directives (this is probably a leftover from CPP lexer that was used as a base for original LexPascal), folding code has a lot of leftovers from CPP lexer that are not applicable to Pascal - folding of //{ //} blocks has no meaning in Pascal, folding of preprocessor directives won't always work since Pascal is not case sensitive and also there's no such preprocessor directive entitled "region" (this is probably "#pragma region" leftover from CPP lexer), and last but not least folding on special keywords doesn't handle a lot of special cases regarding Object Pascal class declarations.

All this has been fixed in my rewritten version of LexPascal, which also has one additional feature regarding highlighting of some keywords only in appropriate context (as implemented those are keywords related to property and DLL exports declarations, similar to how Delphi IDE works). This has replaced the feature in current lexer that highlighted so-called classwords inside class declarations, which also didn't work properly, because it didn't handle cases when class keyword was used outside actual class declaration. The new lexer also has the ability to highlight non-terminated strings.

More information can be found in comments inside LexPascal.cxx file.

I also included modified pascal.propertis file to match the new LexPascal lexer and also updated keywords to match Delphi 2009 (including .NET specific features). Besides this, I also fixed comment.box.* properties in order to actually work, since comment.box.middle.pascal was empty, and also updated properties related to auto-indentation on keywords.

I'm also going to post a message to Scintilla mailing list to ask for any feedback regarding the new lexer, so that it can be included with Scintilla.

Discussion

  • Marko Njezic

    Marko Njezic - 2008-10-23

    New completely rewritten LexPascal lexer

     
  • Marko Njezic

    Marko Njezic - 2008-11-23

    New version with support for folding region directives

     
  • Marko Njezic

    Marko Njezic - 2008-11-23

    I've uploaded new version of LexPascal lexer with support for folding region directives.

     
  • Neil Hodgson

    Neil Hodgson - 2008-11-27

    Committed to CVS.

     
  • Neil Hodgson

    Neil Hodgson - 2008-11-27
    • assigned_to: nobody --> nyamatongwe
     
  • Marko Njezic

    Marko Njezic - 2008-12-23

    I've uploaded a patch that should fix two warnings exposed by the GCC 4.3 compiler that were mentioned on the mailing list.

    The patch fixes parentheses in two while statements so that the right hand side is only processed if counter is inside document constrains.
    File Added: LexPascal.cxx.gcc43warnings

     
  • Neil Hodgson

    Neil Hodgson - 2009-04-24
    • milestone: --> Completed
     
  • Neil Hodgson

    Neil Hodgson - 2009-04-24

    Warning patch committed.

     
  • Neil Hodgson

    Neil Hodgson - 2009-04-28

    Released in 1.78.

     
  • Neil Hodgson

    Neil Hodgson - 2009-04-28
    • status: open --> closed
     

Log in to post a comment.