#772 Todo highlighting breaks on curly braces in options

[Danio]

I'm using the package todonotes to add todo notes to my documents.
The syntax is simple: \todo[options]{note}.
Possible options are to specify if the notes should appear inline or in the document margin.

However when I need to use curly braces in the options the todo highlighting breaks and stops after that option. Presumably the first curly brace encountered is regarded as the content of the todo note, but this is clearly wrong when it is used in an option. Here is an example of a todo note that demonstrates the issue:

\todo[inline,caption={My Caption}]{My Note}

Please let me know if you need more information.

[Tim Hoffmann]

This is a known limitation. See http://sourceforge.net/p/texstudio/feature-requests/339/#de6f. Therefore I'll move this to the feature requests.

[Tim Hoffmann]

Ticket moved from /p/texstudio/bugs/1107/

[MaPePeR]

Sorry for digging up this ancient ticket, but i just encountered this problem.

I come from https://sourceforge.net/p/texstudio/bugs/1789/ and i agree: it is a known limitation, but its no limitation that is not fixable.

If i understand the QNFA-Documentation correctly it should be possible to use matching braces with it. I thought that was the whole point of that file format? (I don't really understand it and the documentation about it is really sparse)

Another aproach would be to use a simple regex that is not perfect, but provides good result for a finite depth of nested braces:

2 Pairs of nested Braces:

\\todo[^{]*{[^{}]*({[^}]*}[^{}]*)*}

matches

\todo{Some { Matching } Brackets and even {asd} }

One level deeper:

\\todo[^{]*{[^{}]*({[^{}]*({[^}]*}[^{}]*)*}[^{}]*)*}

matches

\todo{ }
\todo{ { } { } }
\todo{ { { } } { { }  }  }
\todo{ { {   } { } } { { } } } 

So you can increase the depth by replacing [^}]* with [^{}]*({[^}]*}[^{}]*)* repeatedly.

I don't think you need a depth of more than 4-8 and you can prevent the regex of highlighting wrong by replacing the last [^}]* with [^{}]*. Then it will not match bracket pairs that are deeper than it was designed for at all.

Edit:

And there is also this \\todo[^{]*({(?>[^}{]+|(?1))*}) regex, which i adapted from this Stackoverflow Response, which uses Recursive Regex, but i don't know if your regex engine supports that.

[Tim Hoffmann]

If i understand the QNFA-Documentation correctly it should be possible to use matching braces with it. I thought that was the whole point of that file format? (I don't really understand it and the documentation about it is really sparse)

The documentation is from the editor component we use (QCodeEdit). Sadly, its further development virtually stopped. I don't understand the format completely either.

Another aproach would be to use a simple regex that is not perfect, but provides good result for a finite depth of nested braces

Defining the complete \todo as a single sequence regex does fail if there is a linebreak inside the todo.

I've written a varaiant that now does correct highlighting up to an inner nesting level of 2: hg 6075 (f08956f3edf3).

[MaPePeR]

The regex i provided also fails, because according to the documentation the QCodeEdit "RegEx" implementation does not support round braces/grouping or alternations. :(

Your solution is great. Thanks for looking into it!

I tried it by placing the new tex.qnfa file in .config/texstudio/languages.
It comes with one minor drawback, though: commands that start with todo like \todoin are not highlighted anymore.

I was able to fix this myself by changing the

<start parenthesis="todo:open" prenthesisweight="12">\\todo\{</start>

line to

<start parenthesis="todo:open" prenthesisweight="12">\\todo[^\{]*\{</start>

I was surprised that the "RegEx" implementation supports the negative matching group [^...].

Maybe you can add this as well.

[Tim Hoffmann]

I'm a bit hesitant to add such a general regexp. If you are not careful and specific with regexps, the tend to find matches in unexpected and unwanted situations. Can you be more specific which expressions you want to cover with the wildcard match?

[MaPePeR]

I want it to also highlight \todoin{...}, which is my macro for inline todos.

Its not a wildcard match its "in between of \todo and { there can be 0 or more chars that are not a {".

So this matches

\todo{
\todoinline{
\tododgsdhsdsdad{
\todoEverythingThatsNotAnOpeningCurlyBrace{

but also

\todo     {
\todo___{
\todo\{
\todo<>[]}}}}+-={

But all of these where also highlighted by the previous version(before your fix), which started the highlight at \todo and ended it on the first }. (It was basically a regex with the same wildcard: \todo[^}]*})

But only allowing letters between \todo and { would totally work for my use case.

And if you don't add it i can still use my modified version in the config directory, which is great.

[Tim Hoffmann]

But all of these where also highlighted by the previous version(before your fix)

Yes, this was a bit sloppy and shouldn't be.

But only allowing letters between \todo and { would totally work for my use case.

Done.

[Tim Hoffmann]

Note

todo highlighting is now based on cwl information (not on reg-exp-based language definition). This fixes all the problems with braces.

hg 6261 (b7ff4d056e3e)