New UserDefinedLanguage module

Loreia2
2011-07-30
2015-04-30
<< < 1 2 3 4 5 .. 27 > >> (Page 3 of 27)
  • Loreia2

    Loreia2 - 2011-09-03

    Hi blakmatrix ,

    you can partially expand your definition in new UDL by defining DELIMITER3 as open: / /// and close: / ///
    This can cover RE case. Please test it, and report if you find any thing wrong/malfunctioning with it.

    Unfortunately, embedded delimiters are not supported. They were mentioned on this thread already but I won't work on them for time being. I have very limited time on my disposal and I need to focus on smaller number of new features in order to release anything at all.

    But it is an interesting idea, and I might work on it after I finish working on current feature set. It only depends on my free time.

    P.S.
    I just did a quick test in official build and it seems Perl coloring doesn't support comments within RE's. I guess that makes this feature even more desirable, but some concessions have to be made. So, maybe latter…

    Thank you and best regards,
    Loreia

     
  • Loreia2

    Loreia2 - 2011-09-06

    Hi Chris,

    I am working on GUI right now, and I just realized that Folding is separated in two different sets of settings (Open and Close). It never hit me before, (I guess I would just mechanically create two identical coloring schemes), but what was the reasoning behind using different coloring for something that is essentially the same thing.

    For example, you can't define different colors for comment (of delimiter) open and close sets. The same way it doesn't make sense (to me!) to have different settings for OpenFold and CloseFold.

    I ask because I'd like to change GUI to keep just one one coloring for both sets of Folding keywords. Is that a big no-no, or something that would be considered as improvement?
    Basically, Folding would look the same as Comment block in current official implementation.

    Thank you and best regards,
    Loreia

     
  • Loreia2

    Loreia2 - 2011-09-10

    Hi

    I have finally finished my work on comments and comment folding. Here is what folding looks like now:


    If user selects option "Allow folding keywords in comments" (in Folder & Default tab), he can achieve folding by using special comments. The only rule is: fold keyword must follow directly after comment sign (and rest of the comment is ignored).
    To demonstrate this, lines 17-23 show comments that contain fold keywords preceded by a space. Such lines do not allow folding.

    Also, comment lines that act as folding lines cannot be part of comment groups. This is demonstrated by last example.
    The same trick works with regular comments too:


    You can even mix them, as demonstrated in first example (lines 2-4)

    This is just the first working version of the code, it still requires further optimization and cleanup, but important thing is that I've finished putting new features in. This version will be the base for merge with Chris' stuff.

    So, for the time being, no nested delimiters will be implemented, but I'll keep that as a "next feature" (if I put anything else in, it will be the nested delimiters).

    Improvements in handling of comments will be demonstrated tomorrow, because I still need to update GUI (some small changes) and better test serialization to file, but I believe I can finish that part by tomorrow.

    General code optimization is going well too.
    Test file contained  examples of all coloring groups, and was expanded to 100.000 lines. It took roughly  45%-50% of time needed by 5.9.3 to do the coloring.

    Until next update…

    Thank you and best regards,
    Loreia

     
  • cchris

    cchris - 2011-09-11

    I'd email Don directly about this.

    From my viewpoint, not only using a diferent style for fold close keywords isn't too logical, like you aid. It also prevents Scintilla to apply its brace matching mechanism when the fold boundaries are { and } - not too uncommon.
    I had planned to split the close elimiters section between regular closrs - color them diferently if it serves any purpose - and closers to highlight like openers. Lack of time, and decreasing motivation given the sort of keen interest Don is showing about improving UDLs. If I need it I'"ll implement it, if you add it I'll like it.

    CChris

     
  • cchris

    cchris - 2011-09-11

    I hadn't seen your last post with screenshots: I like this indeed.

    CChris

     
  • Loreia2

    Loreia2 - 2011-09-11

    Hi,

    here is a small demonstration of new features in comments and delimiters.

    1. Lets start with comments.
    The old implementation allowed user to enter comment open and close strings, but it couldn't match correct comment open and close strings.
    e.g. if user entered "/* /+" as comment open, and "*/ +/" as comment close, then the obvious intent is to use these two comment values:
    /* first comment */
    /+ second comment +/

    But old implementation was simply looking for first string it could match, and that led to stuff like this:

    Notice how comment starts with "/+" and ends with "*/"

    New implementation, uses indexes to match correct string. So, if "/+" comment is found, new UDL records position (index) of string "/+", and then it looks only for the same index in close comment string.


    So, user just needs to define comment strings in correct order, and everything will work as intended:

     
  • Loreia2

    Loreia2 - 2011-09-11

    2. Lets continue with line comments:


    Now you can define comment line open string, comment continue and in those rare cases when line comments can end before end of line, you can define close string too.
    I guess 99,9% percent of time, third option will not be used, but some older languages have this "feature". E.g. in my company we have some old scripting language (from 1980s or even older) that uses comments like these (it is a strange mix of regular comments and line comments).
    Anyway, this is what it looks like in action:


     
  • Loreia2

    Loreia2 - 2011-09-11

    3. And finally delimiters:


    Definition in image covers C++ strings, but there is a problem: C/C++ have something called trigraphs, so line continuation can be '\\' or "??/". New UDL solves this problem with special grouping strings "((" and "))"


    In this example, Delimter1 can be
    " delimiter1 \""
    " delimiter1 ??/""
    ' delimiter1 /''

    So, vertical indexing still works, but anything inside (( )) is interchangeable.

    You may ask how to use "((" and "))" as delimiter strings, that's easy just put them in a group of their own:
    (( (( ))
    (( ))))

    White space is generally irrelevant, and you can have one or ten spaces as separators, but there is one exception: if you put "))" inside a group, it must be "glued" to group close string.
    So, you can write:
    ((   ((      ))
    But, you must write:
    ((     ))))

    In the background grouping works like this:
    When " ((" (space, brace, brace) is found group starts, when next "))" (brace, brace) is found group ends, but anything up to next space is considered a part of the group. That's why,  entire "))))" string is read as a part of group sub-string. And when a group is further processed, it works like this:
    Leading "((" is removed, closing "))" is also removed, and anything that is left is a list of operators within a group.

    One way to use this is to use Delmiter1 to define Regex.
    e.g. (I am making this stuff up for demonstration, not providing real world example)
    Open: (( m/ s/ /))
    Close (( /x /g / ))

    This will try to match strings left to right (in new UDL order is preserved as written in definition), that's why, in close group, I am putting longer strings first, so letters "m", "s", "x" and "g" would be highlighted too.

     
  • Loreia2

    Loreia2 - 2011-09-11

    Ok,

    I think I've demonstrated all I wanted.

    You can also see new GUI mockups, it is very similar to the old one, a bit wider, to save vertical space, and input boxes are smaller for comments and delimiters (but not keywords !)

    If you have any comments/critiques, just post them here.

    Until next update…
    Thank you and best regards,
    Loreia

     
  • Loreia2

    Loreia2 - 2011-09-11

    I'd email Don directly about this

    I want to conduct a small investigation of my own before proposing this to him, I'll download all definitions from official wiki, and make a small python script to check them all to see if any have different coloring for open and close set of delimiters.
    I bet I won't find any.

    It also prevents Scintilla to apply its brace matching mechanism when the fold boundaries are { and } - not too uncommon

    I didn't even realize this.

    Lack of time, and decreasing motivation given the sort of keen interest Don is showing about improving UDLs

    I can understand that.

    If I need it I'"ll implement it, if you add it I'll like it

    LOL, I'll see what happens. I am interested to see if any language has use fort his thing.

    I hadn't seen your last post with screenshots: I like this indeed.

    Thanks, I believe that new folding mechanism in UDL will improve working with UDL languages dramatically (in my company we use so much commenting it gets in my way at times, and having ability to fold it will be great).

    Nothing is wrong with "hide text" option, but it is not nearly as easy to use it, as having automatic folding in UDL:

    Thank you and best regards,
    Loreia

     
  • Jonas

    Jonas - 2011-09-11

    I've been trying to make notepad read the vs wizard files with style

    I'd like it to use closures for the if / else / endif
    What i want is very similar to the built-in "MS INI file"

    Doesn't seem like the user defined language supports this :(

    Here's an example:

    readme.txt
    [!if LIB_APP]
            [!if PRE_COMPILED_HEADER]
    stdafx.h
    targetver.h
    stdafx.cpp
            [!endif]
    [!else]
    stdafx.h
    targetver.h
    stdafx.cpp
    OpenFile | root.cpp
            [!if WIN_APP || SUPPORT_MFC]
    resource.h
    root.rc
            [!endif]
            [!if WIN_APP || SUPPORT_MFC]
    root.h
            [!else]
                    [!if DLL_APP && EXPORT_SYMBOLS]
    root.h
                    [!endif]
            [!endif]
            [!if WIN_APP]
    CopyOnly | small.ico
    CopyOnly | root.ico
            [!endif]
    [!endif]
    [!if DLL_APP && !SUPPORT_MFC]
    dllmain.cpp
    [!endif]
    
     
  • Vera

    Vera - 2011-09-11

    Hello loreia2

    that really looks and sounds great :-)

    Will the public be allowed to test it live as well or do we now wait until it's eventually implemented in any future release ?

    greetings ~ Vera

     
  • Loreia2

    Loreia2 - 2011-09-11

    Hi jonasno,

    what exactly do you want to do? Do you need to fold those sections?
    Anyway, you raised an interesting question, because current implementation does not support "else if" case. Just start and end of folding.

    I'll have to see how to include that option too.

    Thank you and best regards,
    Loreia

     
  • Loreia2

    Loreia2 - 2011-09-11

    Hi Vera,

    I'll release it as soon as I clean bugs, no need to release something that I know isn't working (in some cases).

    Also, after running few more tests today, I came to realize that I need full support for nested comments and delimiters in order to cover indexing 100%. That's why I need to expand current implementation with nested delimiters (comments are already supported), before proceeding with finalization of GUI.

    Also, when I add nested delimiters, I need to add a lot of new options in dialog, so I'll have to play around with it more that I originally anticipated. We'll see how it goes in a week or so. (My daughter came home after ten days, and I'll spend most of my free time in next few days playing with her, not UDL :-))

    Thank you and best regards,
    Loreia

     
  • Jonas

    Jonas - 2011-09-12

    Hello loreia2

    > Do you need to fold those sections?
    Yes.
    > Anyway, you raised an interesting question, because current implementation does not support "else if" case. Just start and end of folding.
    Indeed

    But not only folding.
    -Full row background coloring. See this (using "(MS) INI file" language style)

    -Fold tags (and other tags as well) should also be able to handle custom text in the tags Ex. ""
    The built-in xml style can handle custom text in tags and also color them but you can't seem to be able to do it in a user defined language

    You should be able to do all the things the built-in styles can do.
    Do you agree ?

     
  • Loreia2

    Loreia2 - 2011-09-12

    Hi jonasno ,

    You should be able to do all the things the built-in styles can do.
    Do you agree ?

    That is a tough question. UDL will always be a compromise between rich feature set and speed of execution. You just can't work in every feature that there is out there. But UDL should be able to cover most common cases, I agree with that.
    Besides someone must implement it, and that requires a lot of time and effort, which also means there won't be many volunteers for the job.

    Back to your request. One way to achieve what you want (in new UDL!) is to define extra pair of comment characters (""), then to check "Allow folding keywords in comments" option, and finally to define these fold keywords: "!if" and "!endif".
    Obviously, you cannot use "!else" or "!elseif", so I'll have to add another input box to cover that option too.

    XML example will be covered when I implement nested delimiters. I wanted to avoid working on this in this phase, but I need that functionality to make sure re-coloring works properly. So, you'll have to wait for next update to see this option at work.

    Thank you and best regards,
    Loreia

     
  • Loreia2

    Loreia2 - 2011-09-12

    Hi,

    Here is a picture of what I talked about in previous post:


    Is this what you had in mind?

    Thank you and best regards,
    Loreia

     
  • Vera

    Vera - 2011-09-12

    Dear Loreia,

    thanks for your quick reply and unweary task.
    I was a bit puzzled by repeatedly reading 'Now you can do…' without finding a note towards the current update. Don't worry -

    rather enjoy that precious time with your daughter & family.
    (regarding Japan - one never knows how much halcyon days remain)

    take care ~ Vera

     
  • Loreia2

    Loreia2 - 2011-09-12

    Hi  Vera,

    English is my second language (I'm Croatian actually), so I may have problems expressing my exact thought at times. Especially when it is late and I am tired. So, if anything is shady/unclear or simply not grammatically correct, just ask for clarification.

    And yes, you are right, it is precious time and we never know much we have left of it.

    Thank you and best regards,
    Loreia

     
  • Jonas

    Jonas - 2011-09-16


    The first case yes. The endif doesn't have anything after it, it should be just ""
    The full line background color for it is missing though but i take what i can get :)
    I'm not a fan of the font underline.

    The second case isn't valid for visual studio's wizard macro language.

    Here's some examples of  valid cases

    [!if FLAG_ONE || FLAG_TWo && FLAG_THREE]
    // something here
    [!else] (There's no elseif)
    // something else here
    [!endif]
    [!if SUPPORT_MFC && !GENERATE_ATL_DOCUMENT]
    // something
    [!endif]
    
    #include "[!output PROJECT_NAME].h"
    // [!output PROJECT_NAME].cpp : main project file.
    using namespace [!output SAFE_NAMESPACE_NAME];
    [!output SAFE_ATL_MODULE_NAME] _AtlModule;
    

    It doesn't support comments at all
    It only reads the commands within the

    If you  want some files to test with i can upload them to you

     
  • Loreia2

    Loreia2 - 2011-09-17

    Hi jonasno,

    colorization is something user picks through GUI. So, if you don't like underlined fonts, just don't select underline option ;)
    I made that example to demonstrate what can be done with current implementation of UDL2 (for the lack of better name).

    Both lines aim to demonstrate that text after the keywords (!if and !endif) is ignored. That means that you can, but don't have to write anything after the keyword and it will be highlighted as the comment.

    Think of it in this way:
    1. anything within  will be colored as comment (remember YOU are the one selection colorization scheme, the one I demonstrated in that picture is irrelevant)
    2. if opening bracket "[" is immediately followed by a "!if" or "!endif" than you get the folding ability (plus and minus sign in front of that line)

    Third option ("!else" case) will be added in the future.

    It doesn't support comments at all It only reads the commands within the

    Don't be confused by phrase "comment". In UDL a comment is simply a part of text that is defined in comment section of UDL. It doesn't have to be an actual comment in source code. In your case, it is irrelevant if you define "" as delimiters or comments, it simply tells UDL that anything within  "" should be colored as defined for comments or delimiters (whichever you selected).
    But if you define it as comment you can embed folding keywords in it. In that way you can achieve both colorization and folding.

    If all this sounds confusing, just wait until I post binaries here, so you'll be able to test it yourself and it will be clear what I mean.

    If you want some files to test with i can upload them to you

    No. When I post binaries here, I will ask you to test it and report any problems you might encounter.
    After all, you are the one that understands the syntax, not me.

    Thank you and best regards,
    Loreia

     
  • Loreia2

    Loreia2 - 2011-09-26

    Hi All,

    I am slowly finishing my work on new folding features. I am short on free time these days, but things are slowly moving forward. As mentioned last time, folding is now expanded with three sets of keywords: OPEN, MIDDLE and CLOSE. (Btw, I'd like to rename phrase "middle" into something more appropriate. Does anyone have a suggestion?)


    As you can see, now you can fold both sections: open-middle and middle-close
    But it even goes one step further. If your language does not have "else" keyword, you can achieve the same efect by placing CLOSE followed by OPEN on the same line:


    The idea is to support something like C++ statement (where folding keywords are just '{' and '}':

    if (var){
    //some code
    } else {
    //more code
    }

    Also, I checked all language definitions from official wiki, and out of 180+ language definitions, 8 used different stylers for OPEN and CLOSE fold keywords.
    One language had no maintainer listed, so I e-mailed rest of them. Three e-mails were non functional, and out of remaining four, I got two answers (thanks Evan and Gerd). Both guys were OK with my suggestion to use just one styler for all fold keywords, in fact Evan offered to update his definition and to remove different CLOSE fold styler.

    I guess this is a good start, I'll wait few more days for the other two guys to respond. I'd love to hear their take on this. Then, I'll e-mail Don to see what he thinks about it.

    Fun fact: a number of definitions had different coloring for OPEN and CLOSE fold keywords, but the had no fold keywords defined !! I guess the didn't pay much attention to this option, or simply didn't understand it.

    I guess that is all for this time.

    Until next update…
    Thank you and best regards,
    Loreia

     
  • Anonymous - 2011-09-29

    Nice work Lorea!
    I program in a language where the "comment open" and "comment close" are the same character "!". This is currently a problem in 5.9.3. Do you think this will work in your implementation?
    The workaround now is that I use "boundary open" "boundary close" instead and that works with the same character.

     
  • Loreia2

    Loreia2 - 2011-09-29

    Hi,

    in my company we use the same open/close characters. Wouldn't it be funny if we worked for the same company  :-) (tip: North of Europe, company logo is the fifth letter in English alphabet ;)

    Anyway, it is not problem at all

    Thank you and best regards,
    Loreia

     
  • Loreia2

    Loreia2 - 2011-09-29

    I forgot to add,

    if we do work in the same company, you'll need this functionality too:


    Thank you and best regards,
    Loreia

     
<< < 1 2 3 4 5 .. 27 > >> (Page 3 of 27)

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks