#869 LexBash: Add support for M4 "dnl" comments

Scintilla (358)

M4 "dnl" command works essentially like a comment (apart that it isn't printed at all).
While M4 does not necessarily embed Shell, it is most common use nowadays is in Autoconf that does embed Shell, so having support for this in LexBash makes sense (to me).

Joined is a patch that implements this. Maybe it should have a lexer property to enable/disable the feature, but unfortunately I didn't find how to add properties to this lexer.


  • Colomban Wendling

    • labels: --> Scintilla
  • Kein-Hong Man

    Kein-Hong Man - 2011-12-06

    I am a LexBash contributor. Checking some m4 files using shell highlighting, single-quote delimiters seem to have different escape behaviour, and *nix-style usage of the backtick in sentences breaks a lot of things. Adding 'dnl' recognition alone still leaves a lot of delimiter breakage around.

    I'll leave this to Neil. I am a bit leery about active support of m4 highlighting in this manner. Things like m4 and awk are currently "sort of" usable with Shell and Perl highlighting, respectively, if delimiter breakage is tolerated. I'm not sure a long-term solution in this direction is best.

  • Neil Hodgson

    Neil Hodgson - 2011-12-06

    Perhaps having an 'm4' mode for the bash lexer would allow this to evolve towards being more comprehensive.

  • Kein-Hong Man

    Kein-Hong Man - 2011-12-07

    I've no objection to an m4 mode...

  • Colomban Wendling

    @khman: yes, I'm well aware it doesn't bring perfect highlighting for M4, but I think it better than nothing; and dnl comment support is one of the most visible and easy-to-fix thing IMO.

    More generally, I don't know how to support M4 correctly while keeping Shell highlighting in it:

    1) "raw" M4 uses `' (backtick open and single quote close) quotes, and allow to change them (using changequote() IIRC); and Autoconf implicitly sets the quotes to be [] (square brackets). So, it's hard to know what quotes to use when highlighting m4.
    I don't think Scintilla would do much to choose the appropriate quoting (changing the quotes when seeing "AC_*" macro would somewhat work in some situations, but would be ugly, unreliable and sometimes plain wrong), so it would have to be a lexer property that the app deals with.

    2) "raw" M4 has not much to highlight: basically macro calls and quoted strings, most of the script content being what M4 pre-processes. M4 is a generic pre-processor, and is "often" used with SH (Autoconf), C, raw text, etc. So generic M4 highlighting would not really be better than Bash highlighting for Autoconf M4 macros for example.

    3) At least in Autoconf, almost everything is quoted (e.g. the macro arguments, which may be the body of a new macro in case of AC_DEFUN(), define(), etc.), so it'd be useful to highlight quoted content, but it'd be easy to get confused by Shell/non-shell quoting inside them.

    Maybe I'm pessimistic, but while an M4 parser would probably not be that hard to write, an "Autoconf" one would be. So I think adding basic improvements for Autoconf support to the Bash lexer seems useful to me, at least until we get a proper Autoconf lexer.

  • Kein-Hong Man

    Kein-Hong Man - 2011-12-07

    My long-term thinking leans more to a separate lexer. A tweak or two may be easy, but a whole bunch of m4 additions lurking in LexBash is significantly more difficult to maintain. Personally, I wouldn't want to go down that path.

    I just looked at gvim 7.2, it breaks all over the place as well for *.m4 files. I wonder that language they are using for highlighting...

  • Kein-Hong Man

    Kein-Hong Man - 2011-12-08

    On the subject of a separate lexer, I've just spent some time reading up on m4 and parts of it is a nightmare. But if common m4 scripts normally avoid those bits, then I am in cautious agreement that a lexer for a sane subset of m4 can be written in a not-too-complicated way. :-)

  • Neil Hodgson

    Neil Hodgson - 2011-12-08

    If this is going to be for m4 processed shell scripts for autoconf then it may need to inherit much of the current bash lexer unless autoconf uses a shell subset.

  • Kein-Hong Man

    Kein-Hong Man - 2011-12-08

    If there is a really strong case for detailed bash highlighting for things like aclocal.m4 etc, then I suppose it will have to go into LexBash. Your call... But inside those shell script snippets are a lot of m4 macro stuff (or hacks) including pesky delimiters that break normal lexer delimiter processing. Good luck to anyone who wants to fix those delimiter breakage in m4 mode. :-)

    Try the Python lexer on aclocal.m4; the results are pretty reasonable too. So something that highlights main m4 structures, have autoconf stuff as keyword classes, have basic typed things like numbers, operators, etc, and deals reasonably with delimiters should look okay as well, with the added benefit that it will generally work for non-shell m4 scripts too.


Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.

No, thanks