Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#1024 Rust lexer and SciTE support

Completed
closed
Neil Hodgson
5
2013-10-19
2013-10-06
SiegeLord
No

I've created a Rust lexer and added basic SciTE support. The lexer is written from scratch, and is patterned after the official Rust compiler's lexer. The C family lexer could not have been used due to some syntax peculiarities of Rust (e.g. raw string syntax) The fold support is taken virtually without changes from the C family lexer.

2 Attachments

Discussion

1 2 > >> (Page 1 of 2)
  • Neil Hodgson
    Neil Hodgson
    2013-10-06

    The implementation of block doc comments is a bit flaky, possibly dependent on line end type (\r\n or \n) and other context. Sometimes a /! comment appears as SCE_RUST_COMMENTBLOCKDOC and sometimes as SCE_RUST_COMMENTBLOCK.
    This 3 line block comment with \r\n line ends reverted to SCE_RUST_COMMENTBLOCK by typing a space before the 'B'.
    /
    ! Block comment
    x
    */

    Some compilers don't like "while (true)". "for (;;)" will avoid that warning.

    scintilla\lexers\LexRust.cxx:323: style: Same expression on both sides of '||'.
    scintilla\lexers\LexRust.cxx:322: style: Same expression on both sides of '||'.

    To avoid expanding UI elements like the language menu, new languages are added but disabled in SciTE. The language menu entry will be commented out as #Rust|rs||\ and "rust" will be added to imports.exclude. To prevent problems on old versions of Windows, source.files can't be made longer.

     
  • SiegeLord
    SiegeLord
    2013-10-06

    Ok, I will address those issues.

    One thing that came up while fixing block comments was that I realized that Rust (intentionally) only supports LF as the line ending for its line-based constructs (in this case line comments are the most disruptive). For example, this program:

    //
    fn main() {}

    Will not compile with CR line endings (it'll be treated as a big comment). It's easy to handle in the Scintilla lexer, but it might be confusing to observe line comments spanning multiple "lines". Alternatively, it'll be confusing to see a correctly highlighted file that will refuse to compile. I think neither solution is great, any preferences to which I should implement?

     
  • Neil Hodgson
    Neil Hodgson
    2013-10-06

    May be better to show as an error.

     
  • SiegeLord
    SiegeLord
    2013-10-06

    I thought about it some more, and I decided that just highlighting it correctly is best, as the bizzare highlighting for for single line comments will only occur in rare circumstances of CR-only line endings. CRLF endings are treated just fine (CR is highlighted, but that's invisible so it doesn't matter). Attached are the updated patches.

     
  • Neil Hodgson
    Neil Hodgson
    2013-10-06

    The SCLEX_CPPNOCASE lexer exists because some implement small configuration or settings languages to look like case-insensitive C. LexerFactoryRustInsensitive isn't exposed so the remnants of case-insensitivity should be removed from the Rust lexer.

     
  • SiegeLord
    SiegeLord
    2013-10-07

    Ah yes, my bad... a tad too many things copied over without sufficient review. Here's another set of patches (I noticed a bug with how character literals were parsed, so I fixed it).

     
  • Neil Hodgson
    Neil Hodgson
    2013-10-07

    • labels: --> lexer, rust
    • assigned_to: Neil Hodgson
     
  • Neil Hodgson
    Neil Hodgson
    2013-10-15

    • status: open --> closed
     
  • Neil Hodgson
    Neil Hodgson
    2013-10-18

    Visual C++ 2013 shows another couple of warnings. Not bugs but could be confusing:
    c:\u\hg\scintilla\lexers\lexrust.cxx(274): warning C6246: Local declaration of 'n' hides declaration of the same name in outer scope. For additional information, see previous declaration at line '259' of 'c:\u\hg\scintilla\lexers\lexrust.cxx'.: Lines: 259 [C:\u\hg\scite\win32\SciTE.vcxproj]
    c:\u\hg\scintilla\lexers\lexrust.cxx(293): warning C6246: Local declaration of 'n' hides declaration of the same name in outer scope. For additional information, see previous declaration at line '259' of 'c:\u\hg\scintilla\lexers\lexrust.cxx'.: Lines: 259 [C:\u\hg\scite\win32\SciTE.vcxproj]

     
1 2 > >> (Page 1 of 2)