#1459 Improvements to Haskell Lexer

Bug
closed-fixed
Neil Hodgson
3
2013-07-21
2013-04-03
kudah
No
  • Added support for MagicHash extension (lexer.haskell.allow.hash)
  • $ and # are now colored as operators.
  • .0 and -0 are now properly colored as operator and a number, not as just number.
  • Operators starting with double dash (e.g ---->) are properly colored as operators, not comments.
  • Added pragma highlighting.
  • Added basic C-preprocessor highlighting.
  • Qualified names (e.g. ABC.xyz) are now properly highlighted as identifiers, not types.
  • Qualified operators (e.g. ABC.<$>) are now properly highlighted as operators.
  • Operator . is now properly highlighted as an operator, not part of the identifier, when applied to a qualified and an unqualified value (e.g. Abc.xyz.yzx <- last one is an operator)
  • Operators starting with ':' are now properly highlighted as type constructors, not operators.
  • "family" after "data" is highlighted, as per TypeFamilies.
1 Attachments

Discussion

<< < 1 2 3 4 > >> (Page 3 of 4)
  • kudah
    kudah
    2013-05-21

    With Unicode classification, the patch is bulked out with a subset of the case conversion table and can be made shorter by dropping that data

    At this point, we might as well do case folding too: editors do it anyway, and that's one less table for them to worry about.

     
    • Neil Hodgson
      Neil Hodgson
      2013-05-21

      Scintilla already contains code for case folding. If lexers need to case-fold strings then the current methods should be packaged in a way that they can be used by lexers.

      The case-conversion data in the patch is only a subset and is exposed as 3 character->character maps (upper, lower, title), none of which is folding. Unicode case folding can produce a folded string that is a different length from the input string. The full data is in CaseFolding.txt file in the Unicode character database available from the Unicode web site.

       
  • kudah
    kudah
    2013-05-08

    • Fixed a bug when --:> was highlighted as a constructor
    • Fixed a bug when X wasn't highlighted as a module in import {-# SOURCE #-} X
    • Fixed a bug when preprocessor directives could mess up context keywords
     
  • kudah
    kudah
    2013-05-09

    • Literate Haskell support
    • CPP highlighting is disableable
    • CPP is now correctly colored when inside comments and strings
    • Some bugs with incremental lexing fixed
    • Indented imports folding is now undisableable
     
  • Neil Hodgson
    Neil Hodgson
    2013-05-11

    Removing a property "fold.haskell.imports.indented" is worrying as the interface to a lexer should be stable. If any of the properties or lexical states are only temporary then this code should not be committed to main.

    Some warnings:
    ..\lexers\LexHaskell.cxx(374): warning C4512: 'LexerHaskell' : assignment operator could not be generated [C:\u\haskell\scintilla\win32\SciLexer.vcxproj]
    ..\lexers\LexHaskell.cxx(224) : see declaration of 'LexerHaskell'
    ..\lexers\LexHaskell.cxx(498): warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data [C:\u\haskell\scintilla\win32\SciLexer.vcxproj]
    ..\lexers\LexHaskell.cxx(519): warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data [C:\u\haskell\scintilla\win32\SciLexer.vcxproj]
    Message: The member function 'LexerHaskell::skipMagicHash' can be made a const function.
    Message: The member function 'LexerHaskell::LineContainsImport' can be made a const function.

     
    • kudah
      kudah
      2013-05-11

      • Neil Hodgson
        Neil Hodgson
        2013-05-13

        Sorry, I wasn't trying to get fold.haskell.imports.indented back. It was added after 3.3.1 was released so has not yet been seen by clients. If fold.haskell.imports.indented is a good feature then it should stay in. If no one is going to use it then it shouldn't be included.

        Since there will be a release in a week or two, I want to avoid releasing any features that may not work out. So, please reexamine the properties and lexical states added since 3.3.1 and determine if they will ever be used and whether they are worth maintaining for years.

         
        • kudah
          kudah
          2013-05-13

          If fold.haskell.imports.indented is a good feature then it should stay in. If no one is going to use it then it shouldn't be included.

          This option doesn't make sense with literate haskell, i.e. all imports in bird-style literate source will be indented.

           
          • Neil Hodgson
            Neil Hodgson
            2013-05-15

            OK.

             
          • Neil Hodgson
            Neil Hodgson
            2013-05-15

            3.3.2 will be early next week so any bugs/fixes should be uploaded soon.

            This tracker item is long which makes tracing history more difficult. New features should go into additional tracker items.

             
            • kudah
              kudah
              2013-05-15

              3.3.2 will be early next week so any bugs/fixes should be uploaded soon.
              This tracker item is long which makes tracing history more difficult. New features should go into additional tracker items.

              I take it that I should post any remaining fixes to this bug, but file any new features to a new ticket?

               
              • Neil Hodgson
                Neil Hodgson
                2013-05-15

                Yes, please.

                 
  • kudah
    kudah
    2013-05-18

    Fixed import folding in literate haskell.

     
    • Neil Hodgson
      Neil Hodgson
      2013-05-18

      Is this a bug fix or a new feature? Scintilla is in feature-freeze for 3.3.2. fold.haskell.imports.indented looks like a new feature. If its a bug, should I revert back to the 3.3.1 version of LexHaskell?

       
      • kudah
        kudah
        2013-05-18

        Is this a bug fix or a new feature?

        A minor fix.

        fold.haskell.imports.indented looks like a new feature.

        My working tree was inconsistent.

         
        • Neil Hodgson
          Neil Hodgson
          2013-05-19

          Committed as [e5cb15].

           

          Related

          Commit: [e5cb15]

  • kudah
    kudah
    2013-05-27

    • Removed extraneous LineState write which caused serious performance issues.
    • Fixed incoherent CPP highlighting with styling.within.preprocessor.
     
    • kudah
      kudah
      2013-05-27

      Can you close the ticket after this is merged?

       
      • Neil Hodgson
        Neil Hodgson
        2013-05-28

        There's a bug fix release 3.3.3 scheduled for June 2 so this won't be merged until then.

         
        • kudah
          kudah
          2013-05-28

          Huh? What does the release schedule have to do with merging to the main repo?

           
          • Neil Hodgson
            Neil Hodgson
            2013-05-28

            Why does it matter to you when the patch is applied? Its not going into 3.3.3 because it arrived after 3.3.3 was announced and the first RC was distributed.

             
            • kudah
              kudah
              2013-05-28

              Why does it matter to you when the patch is applied?

              And why does it matter to you? If 3.3.3 is freezed, like you say, then it should be on its own branch or tag, master branch's progress shouldn't be obstructed. But whatever, it doesn't matter except I'd rather have performance degradation in 3.3.2 reversed sooner.

               
              Last edit: kudah 2013-05-28
              • Neil Hodgson
                Neil Hodgson
                2013-05-28

                It matters to me because it is my time that is being used. I'm not seeing sufficient reason to perform any extra work on this issue.

                The performance degradation took somewhere between 1 and 5 weeks for you to notice so can't be all that significant.

                 
                • kudah
                  kudah
                  2013-05-28

                  The performance degradation took somewhere between 1 and 5 weeks for you to notice so can't be all that significant.

                  As I've said, it causes a full redraw on almost every input, that's as much performance impact as a lexer can have, aside of looping forever.

                  It matters to me because it is my time that is being used. I'm not seeing sufficient reason to perform any extra work on this issue.

                  You're being too dramatic over a one line change. Merge whenever, I see no benefit in further discussion.

                   
                  Last edit: kudah 2013-05-28
<< < 1 2 3 4 > >> (Page 3 of 4)