#1672 Few mishandlings of shell here-documents

[Colomban Wendling]

LexBash mishandles 3 here-document cases:

<<- is one single token

<<- is an operator by itself, it's not << followed by - as the lexer handles it. This breaks handling of delimiters that start with -, like this:

cat << -EOF
hello
-EOF
# outside heredoc

Indented delimiters can only be tab-indented

The <<- operator starts an indented here-document, but the indentation can only be tabulations, not spaces. This results on the lexer to possibly exit too early.

cat <<- EOF
  this isn't the end, only tabulations are stripped, not spaces
  EOF
next is the end, it's supposed to start with a tabulation but SF makes it spaces
    EOF
# outside heredoc

Empty delimiters can be indented like any other delimiter

A small bug in the special case for empty delimiters fails to allow indented delimiters, which are allowed.

# SF seems to eat the tab, but there should be one in the empty line
cat << ''

# outside heredoc

Attached are three patches, each fixing one of the issue.

[Neil Hodgson]

The third example isn't appearing like the issue tracker's highlighting with the comment appearing as part of the heredoc. Is there supposed to be two white lines before the final comment with the first containing a single tab character?

That is,
cat << ''\n\t\n#x
shows #x in heredoc style but
cat << ''\n\t\n\n#x
shows #x in comment style.

This is with all the patches applied.

[Colomban Wendling]

Oops, my bad, the 3rd example is incorrect, it should use <<- (e.g. allow indentation): cat <<- ''\n\t\n#x should have the comment in comment style.

Apparently I can't edit my first post to fix up the example though :/

[Colomban Wendling]

Here is a more sensible set of examples, that I used to fix CTag's handling and lead me to discover the bugs discussed here: https://github.com/fishman/ctags/blob/master/Units/sh-heredoc-checks.d/input.sh (don't trust GitHub highlight as it fails to handle most non-obvious cases). This script can be run by Bash so it's relatively easy to make sure the test is correct.

[Kein-Hong Man]

I have applied and tested the patches. Looks great to me. Thanks!

[Kein-Hong Man]

To say it unambiguously, I applied them on my own machine and they tested okay...

[Neil Hodgson]

Committed as [6a4dc8], [bb9fef], [e0def7].