#1761 Powershell lexer $() sub-expression expansion

[Alexandre Laroche]

Powershell highlighting have a bug. The problem is with '"'. See below example :

Get-WmiObject -Computer $computer -query "select * from Win32_GroupUser where GroupComponent=""Win32_Group.Domain='$computer',Name='$($group.Name)'""" | % {$_.partcomponent –match ".+Domain\=(.+)\,Name\=(.+)$"; if($matches[1].trim('"') -eq $computer) { $groupMembersCompare[$group.Name].Add("$($matches[2].trim('"'))", $False) } else { $groupMembersCompare[$group.Name].Add("$($matches[1].trim('"'))\$($matches[2].trim('"'))", $False) } } > $nul

[Neil Hodgson]

Is there any documentation that describes the rules that are supposed to be followed here?

[Alexandre Laroche]

Bruce Payette is a co-designer of the PowerShell language and the development lead for its implementation. From that, you might guess that his book: Windows PowerShell in Action would provide a great explanation of the language and how to use it – you'd be right.

Appendix C: The PowerShell grammar => https://manning-content.s3.amazonaws.com/download/2/4aee82f-9192-4e50-941f-4bfde960b998/AppCexcerpt.pdf

Reference : https://www.manning.com/books/windows-powershell-in-action#downloads

[Neil Hodgson]

That appears to confirm that the current lexing is correct. The relevant productions are:

    <ExpandableStringToken> = ".\*"
    <StringToken> = '.\*'

In Scintilla choose different colours for SCE_POWERSHELL_STRING (2:ExpandableStringToken) and SCE_POWERSHELL_CHARACTER (3:StringToken). For example, in SciTE:

style.powershell.2=fore:#008000
style.powershell.3=fore:#A0A000

You will see alternating styles. At the first location of "$ it switches to SCE_POWERSHELL_STRING until the next " is found. The next character is ' so it switches to SCE_POWERSHELL_CHARACTER, then again back to SCE_POWERSHELL_STRING then SCE_POWERSHELL_CHARACTER.

Inside ExpandableStringToken there may be VariableToken instances which are $ followed either by alphanumeric sequences or sequences surrounded by {}.

    <VariableToken> = \$[:alnum:]+ | \${.+}

The example code uses a $( expression which does not appear to be allowed based upon the referenced document.

[Alexandre Laroche]

Powershell ISE correctly display the line mentionned.

Github also display the line correctly. (see here: https://github.com/notepad-plus-plus/notepad-plus-plus/issues/882)

I really think the problem is in the scintilla powershell lexer.
I think the problem is just here : if($matches[1].trim('"') -eq $computer)
I think scintilla don't parse ' " ' correctly.

[Neil Hodgson]

Its unclear what the allowed syntax of strings is since the referenced document does not appear to allow the example. If there is a more accurate grammar for PowerShell then a link should be included on this bug report.

I will not be working on this issue myself. A well-written patch that fixes this issue will be accepted.

[Colomban Wendling]

Its unclear what the allowed syntax of strings is since the referenced document does not appear to allow the example. If there is a more accurate grammar for PowerShell then a link should be included on this bug report.

The linked document doesn't seem really trustworthy as it wouldn't even allow for e.g. escape sequences in double-quoted strings, which seem to be referenced everywhere on the web ("" or `", which should stand for ").

Not sure how trusted it can be, but these contain some information supporting the report:

• http://ss64.com/ps/syntax-esc.html
• http://ss64.com/ps/syntax-operators.html

It's not as clear as it should be, but at least it references escapes and placeholders.

[Zufu Liu]

The fail part is $() sub-expression expansion
https://learn.microsoft.com/en-us/powershell/scripting/lang-spec/chapter-02?view=powershell-7.3#2352-string-literals

Add("$($matches[2].trim('"'))", $False)