#1033 Syntax highlighting for command substitution on Shell

[Sworddragon]

In SciTE 3.3.6 command substitution will be simply drawn as block/string as seen on the screenshot in the attachments. Maybe it would be possible to use syntax highlighting there too.

[Kein-Hong Man]

Sure, it's one of the remaining missing bits for the shell lexer. If anyone wants to do this, please feel free, this can be discussed on the list too. We can at least keep this feature ticket for now.

It is necessarily a very low priority item for me because strings in actual bash scripts are mostly pretty uncomplicated.

[Lars Paulsen]

The following patch fixes the issue (also see https://sourceforge.net/p/scintilla/code/merge-requests/22/):

--- a/lexers/LexBash.cxx        Sat Aug 17 16:32:13 2019 +0200
+++ b/lexers/LexBash.cxx        Wed Jul 17 15:49:13 2019 +1000
@@ -295,7 +295,7 @@

 void SCI_METHOD LexerBash::Lex(Sci_PositionU startPos, Sci_Position length, int initStyle, IDocument *pAccess) {
        WordList cmdDelimiter, bashStruct, bashStruct_in;
-       cmdDelimiter.Set("| || |& & && ; ;; ( ) { } $(");
+       cmdDelimiter.Set("| || |& & && ; ;; ( ) { }");
        bashStruct.Set("if elif fi while until else then do done esac eval");
        bashStruct_in.Set("for case select");

@@ -873,7 +873,7 @@
                        } else if (sc.ch == '`') {
                                sc.SetState(SCE_SH_BACKTICKS);
                                QuoteStack.Start(sc.ch, BASH_DELIM_BACKTICK);
-                       } else if (sc.ch == '$' && sc.chNext != '(') {
+                       } else if (sc.ch == '$') {
                                if (sc.Match("$((")) {
                                        sc.SetState(SCE_SH_OPERATOR);   // handle '((' later
                                        continue;
@@ -889,6 +889,9 @@
                                } else if (sc.ch == '"') {
                                        sc.ChangeState(SCE_SH_STRING);
                                        QuoteStack.Start(sc.ch, BASH_DELIM_LSTRING);
+                               } else if (sc.ch == '(') {
+                                       sc.ChangeState(SCE_SH_BACKTICKS);
+                                       QuoteStack.Start(sc.ch, BASH_DELIM_COMMAND);
                                } else if (sc.ch == '`') {      // $` seen in a configure script, valid?
                                        sc.ChangeState(SCE_SH_BACKTICKS);
                                        QuoteStack.Start(sc.ch, BASH_DELIM_BACKTICK);

[Neil Hodgson]

From the linked merge request, it looks like Kein-Hong Man doesn't approve of this change. Since I am just a novice when it comes to bash / sh I will commit the proposed if Kein-Hong Man thinks it should be committed.

[Neil Hodgson]

I don't think this was committed - there are few changes to LexBash.cxx since this 2019 patch and the only substantial change was to line end processing.

Trying the example shows the same styling as the image.

[Zufu Liu]

Misled by above code diff (which is opposite of the change from the merge request).
Opened for https://github.com/ScintillaOrg/lexilla/issues/153 this issue.

[Kein-Hong Man]

Hi Zufu, Neil. If anyone wants to update the bash/shell lexer, feel free to do so. I can't realistically work on the lexers I hack anymore... unless I find an enormous chunk of free time. Non-coding things get in the way. C++ used to be small(ish) and easier to code, but these days it's hard to keep abreast of things. I read one of Stroustrup's very long, talky books and thought -- I can only keep up with this if coding C++ is my only pasttime. Not gonna happen.

The problem with bash is that its syntax evolved piecemeal and folks nest whole scripts inside command strings or HERE docs and then expect everything to be highlighted nicely in some fashion. The actual parser in the bash sources is a fantastic kludge with some kind of nesting.

I think basic $var string highlighting is a reasonable request. But for general highlighting of whole scripts inside a command string, differentiating nested code is a problem. So you'd need (A) nested highlighting colors, well, if someone can code that, be my guest. And HERE docs will have whole programs that are not shell code. Maybe avoid all that HERE doc stuff, but they will and have asked for highlighting it all, haha. Or (B) use the current set of colors and you end up treating the delimiters as more like structural delimiters like {} and not string delimiters. But treating $() like structural {} would be semantically wrong. (A) is 'more correct' while (B) is the quickie solution. 737MAX was a quickie solution too.

Or one can just follow the general trend, e.g. what the other major editors or web-based lexers do. It's what the crowd wants, I suppose. If anyone has the time to hack the lexer, it's their baby. :-P

[Zufu Liu]

Change back to original status as this needs further discussion and can't be full implemented in short term.

[Neil Hodgson]

There is now an issue for part of this on the Lexilla project.
https://github.com/ScintillaOrg/lexilla/issues/153