#1459 Improvements to Haskell Lexer

Bug
closed-fixed
3
2013-07-21
2013-04-03
kudah
No
  • Added support for MagicHash extension (lexer.haskell.allow.hash)
  • $ and # are now colored as operators.
  • .0 and -0 are now properly colored as operator and a number, not as just number.
  • Operators starting with double dash (e.g ---->) are properly colored as operators, not comments.
  • Added pragma highlighting.
  • Added basic C-preprocessor highlighting.
  • Qualified names (e.g. ABC.xyz) are now properly highlighted as identifiers, not types.
  • Qualified operators (e.g. ABC.<$>) are now properly highlighted as operators.
  • Operator . is now properly highlighted as an operator, not part of the identifier, when applied to a qualified and an unqualified value (e.g. Abc.xyz.yzx <- last one is an operator)
  • Operators starting with ':' are now properly highlighted as type constructors, not operators.
  • "family" after "data" is highlighted, as per TypeFamilies.
1 Attachments

Discussion

<< < 1 2 3 4 > >> (Page 3 of 4)
  • kudah

    kudah - 2013-05-21

    With Unicode classification, the patch is bulked out with a subset of the case conversion table and can be made shorter by dropping that data

    At this point, we might as well do case folding too: editors do it anyway, and that's one less table for them to worry about.

     
    • Neil Hodgson

      Neil Hodgson - 2013-05-21

      Scintilla already contains code for case folding. If lexers need to case-fold strings then the current methods should be packaged in a way that they can be used by lexers.

      The case-conversion data in the patch is only a subset and is exposed as 3 character->character maps (upper, lower, title), none of which is folding. Unicode case folding can produce a folded string that is a different length from the input string. The full data is in CaseFolding.txt file in the Unicode character database available from the Unicode web site.

       
  • kudah

    kudah - 2013-05-08
    • Fixed a bug when --:> was highlighted as a constructor
    • Fixed a bug when X wasn't highlighted as a module in import {-# SOURCE #-} X
    • Fixed a bug when preprocessor directives could mess up context keywords
     
  • kudah

    kudah - 2013-05-09
    • Literate Haskell support
    • CPP highlighting is disableable
    • CPP is now correctly colored when inside comments and strings
    • Some bugs with incremental lexing fixed
    • Indented imports folding is now undisableable
     
  • Neil Hodgson

    Neil Hodgson - 2013-05-11

    Removing a property "fold.haskell.imports.indented" is worrying as the interface to a lexer should be stable. If any of the properties or lexical states are only temporary then this code should not be committed to main.

    Some warnings:
    ..\lexers\LexHaskell.cxx(374): warning C4512: 'LexerHaskell' : assignment operator could not be generated [C:\u\haskell\scintilla\win32\SciLexer.vcxproj]
    ..\lexers\LexHaskell.cxx(224) : see declaration of 'LexerHaskell'
    ..\lexers\LexHaskell.cxx(498): warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data [C:\u\haskell\scintilla\win32\SciLexer.vcxproj]
    ..\lexers\LexHaskell.cxx(519): warning C4267: 'argument' : conversion from 'size_t' to 'int', possible loss of data [C:\u\haskell\scintilla\win32\SciLexer.vcxproj]
    Message: The member function 'LexerHaskell::skipMagicHash' can be made a const function.
    Message: The member function 'LexerHaskell::LineContainsImport' can be made a const function.

     
    • kudah

      kudah - 2013-05-11
       
      • Neil Hodgson

        Neil Hodgson - 2013-05-13

        Sorry, I wasn't trying to get fold.haskell.imports.indented back. It was added after 3.3.1 was released so has not yet been seen by clients. If fold.haskell.imports.indented is a good feature then it should stay in. If no one is going to use it then it shouldn't be included.

        Since there will be a release in a week or two, I want to avoid releasing any features that may not work out. So, please reexamine the properties and lexical states added since 3.3.1 and determine if they will ever be used and whether they are worth maintaining for years.

         
        • kudah

          kudah - 2013-05-13

          If fold.haskell.imports.indented is a good feature then it should stay in. If no one is going to use it then it shouldn't be included.

          This option doesn't make sense with literate haskell, i.e. all imports in bird-style literate source will be indented.

           
          • Neil Hodgson

            Neil Hodgson - 2013-05-15

            OK.

             
          • Neil Hodgson

            Neil Hodgson - 2013-05-15

            3.3.2 will be early next week so any bugs/fixes should be uploaded soon.

            This tracker item is long which makes tracing history more difficult. New features should go into additional tracker items.

             
            • kudah

              kudah - 2013-05-15

              3.3.2 will be early next week so any bugs/fixes should be uploaded soon.
              This tracker item is long which makes tracing history more difficult. New features should go into additional tracker items.

              I take it that I should post any remaining fixes to this bug, but file any new features to a new ticket?

               
              • Neil Hodgson

                Neil Hodgson - 2013-05-15

                Yes, please.

                 
  • kudah

    kudah - 2013-05-18

    Fixed import folding in literate haskell.

     
    • Neil Hodgson

      Neil Hodgson - 2013-05-18

      Is this a bug fix or a new feature? Scintilla is in feature-freeze for 3.3.2. fold.haskell.imports.indented looks like a new feature. If its a bug, should I revert back to the 3.3.1 version of LexHaskell?

       
  • kudah

    kudah - 2013-05-27
    • Removed extraneous LineState write which caused serious performance issues.
    • Fixed incoherent CPP highlighting with styling.within.preprocessor.
     
    • kudah

      kudah - 2013-05-27

      Can you close the ticket after this is merged?

       
      • Neil Hodgson

        Neil Hodgson - 2013-05-28

        There's a bug fix release 3.3.3 scheduled for June 2 so this won't be merged until then.

         
        • kudah

          kudah - 2013-05-28

          Huh? What does the release schedule have to do with merging to the main repo?

           
          • Neil Hodgson

            Neil Hodgson - 2013-05-28

            Why does it matter to you when the patch is applied? Its not going into 3.3.3 because it arrived after 3.3.3 was announced and the first RC was distributed.

             
            • kudah

              kudah - 2013-05-28

              Why does it matter to you when the patch is applied?

              And why does it matter to you? If 3.3.3 is freezed, like you say, then it should be on its own branch or tag, master branch's progress shouldn't be obstructed. But whatever, it doesn't matter except I'd rather have performance degradation in 3.3.2 reversed sooner.

               
              Last edit: kudah 2013-05-28
              • Neil Hodgson

                Neil Hodgson - 2013-05-28

                It matters to me because it is my time that is being used. I'm not seeing sufficient reason to perform any extra work on this issue.

                The performance degradation took somewhere between 1 and 5 weeks for you to notice so can't be all that significant.

                 
                • kudah

                  kudah - 2013-05-28

                  The performance degradation took somewhere between 1 and 5 weeks for you to notice so can't be all that significant.

                  As I've said, it causes a full redraw on almost every input, that's as much performance impact as a lexer can have, aside of looping forever.

                  It matters to me because it is my time that is being used. I'm not seeing sufficient reason to perform any extra work on this issue.

                  You're being too dramatic over a one line change. Merge whenever, I see no benefit in further discussion.

                   
                  Last edit: kudah 2013-05-28
<< < 1 2 3 4 > >> (Page 3 of 4)

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks