#1012 Submit an improved makefile lexer

Completed
open
nobody
lexer (35)
5
2013-09-23
2013-08-26
Gur Stavi
No

An improved makefile lexer for GNU makefiles.

1 Attachments

Discussion

  • Gur Stavi
    Gur Stavi
    2013-08-26

    Fixes compilation errors with MinGW gcc
    Also compiles with Windows SDK

     
    Last edit: Gur Stavi 2013-08-26
    Attachments
  • Neil Hodgson
    Neil Hodgson
    2013-08-26

    • labels: --> lexer
     
  • Neil Hodgson
    Neil Hodgson
    2013-08-26

    This will be examined once 3.3.5 is released in about one week.

     
  • Gur Stavi
    Gur Stavi
    2013-09-01

    Added folding.
    Lots of additional corner cases are handled and overall improvements.

     
    Attachments
  • Neil Hodgson
    Neil Hodgson
    2013-09-02

    There appear to be two modules here: a generic framework and a specific lexer. Any additional framework should be discussed on the mailing list first. If the generic code is not reused in other lexers then it is maintenance-expensive dead weight.

    There are many conventions used in the current lexers and builds rely on this for meta-data and documentation. Not following the conventions for exposing properties, for example, leads to scite/scripts/RegenerateSource.py not picking up the properties documentation. lexer.makefile.hlUnterminatedIdent does not follow the (dot.separated.words) property naming convention. Helper code for lexers goes in lexlib. Headers should not contain #include statements.

     
  • Gur Stavi
    Gur Stavi
    2013-09-02

    Moved LexCommon to lexlib.
    Removed includes from header. (why?)
    Modified property mechanism to be compatible with RegenerateSource.py.
    Modified property names to be dot.seperated.words.

    Joined the mailing list to discuss the benefits of LexerCommon.

     
    Attachments
    • Neil Hodgson
      Neil Hodgson
      2013-09-04

      Using #include in headers makes it difficult to understand and minimize dependencies which then blows out build times. Scintilla often builds in less than 30 seconds.

       
  • Neil Hodgson
    Neil Hodgson
    2013-09-05

    There are some warnings from various compilers and static checkers. I haven't looked closely at these and some may be false positives or unjustified stylistic issues.

    ..\lexlib\LexCommon.cxx(59) : warning C4267: '+=' : conversion from 'size_t' to 'unsigned int', possible loss of data
    ..\lexlib\LexCommon.cxx(68) : warning C4267: '=' : conversion from 'size_t' to 'unsigned int', possible loss of data

    ../lexers/LexMake.cxx:456:2: warning: extra ';' [-pedantic]

    scintilla\lexlib\LexCommon.h:31: warning: Member variable 'DoLexContext::lexer' is not initialized in the constructor.
    scintilla\lexlib\LexCommon.h:31: warning: Member variable 'DoLexContext::startPos' is not initialized in the constructor.
    scintilla\lexlib\LexCommon.h:31: warning: Member variable 'DoLexContext::length' is not initialized in the constructor.
    scintilla\lexlib\LexCommon.h:31: warning: Member variable 'DoLexContext::doneStyling' is not initialized in the constructor.
    scintilla\lexlib\LexCommon.cxx:306: style: The scope of the variable 'tmp' can be reduced.
    scintilla\lexers\LexMake.cxx:290: style: The scope of the variable 'i' can be reduced.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::tok' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::tokSize' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::lineNumber' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::prevFoldLevel' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::foldLevel' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::ContLineCount' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::type' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::pType' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::isContinueLine' is not initialized in the constructor.
    scintilla\lexers\LexMake.cxx:703: warning: Member variable 'MkTokener::ch' is not initialized in the constructor.

    Since there doesn't yet appear to be interest in using the lexer framework, the code should be collapsed into LexMake.cxx which will make it easier to integrate into existing builds.

     
  • Neil Hodgson
    Neil Hodgson
    2013-09-05

    Some of the style names (SCE_MAKE_*) are a little too short to be easily understood by client developers. UNT_ is the least obvious but INCL_ and PRE_ could also be a little longer.

     
  • Gur Stavi
    Gur Stavi
    2013-09-22

    Fixed relevant issues.
    Added a new lexer for MIB files with folding.The new lexer is implemented on top of LexCommon. Basically it should replace the ASN.1 lexer but they may also co-exist if someone prefers the current one.

     
    Attachments
  • Gur Stavi
    Gur Stavi
    2013-09-22

    Fix bug in Make folding

     
    Attachments
  • Neil Hodgson
    Neil Hodgson
    2013-09-23

    As mentioned before, LexCommon should be merged into the lexer files. In this case modularity trumps code duplication.

    Too many type names pollute the global namespace without being all that unique such as PosBank. The unnamed namespace can be used to cage those names.

    Are the MIB keywords really so unchanging that they should be hardcoded in the lexer instead of being set by the app?

    scintilla\lexlib\LexCommon.h:40: warning: Member variable 'DoLexContext::lexer' is not initialized in the constructor.
    scintilla\lexlib\LexCommon.h:40: warning: Member variable 'DoLexContext::userPtr' is not initialized in the constructor.
    scintilla\lexlib\LexCommon.h:40: warning: Member variable 'DoLexContext::startPos' is not initialized in the constructor.
    scintilla\lexlib\LexCommon.h:40: warning: Member variable 'DoLexContext::length' is not initialized in the constructor.
    scintilla\lexlib\LexCommon.h:40: warning: Member variable 'DoLexContext::doneStyling' is not initialized in the constructor.
    scintilla\lexers\LexMib.cxx:346: style: Unused variable: tmpStr

    Spelling: ketwords

    *.iface should not be in file.patterns.make since that implies it can be run with make.