Just Launched: You can now import projects and releases from Google Code onto SourceForge

We are excited to release new functionality to enable a 1-click import from Google Code onto the Allura platform on SourceForge. You can import tickets, wikis, source, releases, and more with a few simple steps.

#574 Wrong syntax highlighting in dtx files

None
open
nobody
None
5*
2014-01-20
2013-10-08
mrpiggi
No

In .dtx files, matching braces are not recognized, if there's a source code segment like

%    \end{macrocode}
% Explanation
%    \begin{macrocode}


in between. It results in a wrong syntax highlighting, shown in the attached screenshot.

1 Attachments

Discussion

1 2 > >> (Page 1 of 2)

• mrpiggi
2013-10-08

Same with tags...

Attachments

• Tim Hoffmann
2013-10-08

This is because the bracing of \foo{} and \begin \end is interweaved in your example:

%   \begin{macromode}
\def\foo{
%   \end{macromode}
}


In simple braces the equivalent would be ( { ) } which is also marked as an error.

TXS assumes a strictly hirachial brace sequence: the first closing tag matches the last opening tag. It is considered an error if they are of different type. This assumption is correct in most languages. I'm not an expert in dtx, so does this not hold for dtx?

Last edit: Tim Hoffmann 2013-10-08

• mrpiggi
2013-10-09

In dtx files the concept of literate programming is used. This means, source code and it's documentation (maybe even the user documentation) is given in the same file. So when the dtx file is processed by docstrip, everything within the macrocode environment will be exported to a style or class file. Everything outside the macrocode environment is used for the source code documentation. So the segment

% \begin{macro}{\foo}
% Explanation of the macro
%    \begin{macrocode}
\def\foo{%
%    \end{macrocode}
% Explanation of a segment within the macro
%    \begin{macrocode}
foo%
}
%    \end{macrocode}
% \end{macro}^^A \foo


would result in this source code

\def\foo{%
foo%
}


and the attached documentation. You can find an introduction to dtx files here

Attachments

• Tim Hoffmann
2013-10-09

Ok, we cannot fix this. One would need two parallel brace stacks, one for the dtx-related braces and one for tex-related braces. This is beyond the capabilities of the language model of the editor.

There's one thing that I could offer, but I'm not sure if that's better. Since it's a speciality of \begin{macrocode} \end{macrocode} I can remove the parenthesis property from these commands in the language definition. Then, all brace-related functionality is deactivated for these commands (also including code folding, but that doesn't work correctly in your example code either). What do you think?

Of course, you may alternatively switch off brace matching in the options completely.

Last edit: Tim Hoffmann 2013-10-09

• mrpiggi
2013-10-09

Removing the parenthesis property from macrocode would be fine. That would be enough to me (almost).

But, since \begin{macrocode} and \end{macrocode} must be intended the same way, maybe there's another possibility to find \begin{macrocode} and matching \end{macrocode}. Both commands always have to be set with a leading percent sign followed by exactly four spaces. So maybe it is possible to use this characteristic, even for a different color markup?

Silly code example

% \begin{macro}{\foo}
% Explanation of the macro
%    \begin{macrocode}
\def\foo{%
%    \end{macrocode}
% Explanation before table
%    \begin{macrocode}
\begin{table}
%    \end{macrocode}
% Explanation befor minipage
%    \begin{macrocode}
\begin{minipage}{3cm}
%    \end{macrocode}
% Explanation before tabular
%    \begin{macrocode}
\begin{tabular}{ccc}
%    \end{macrocode}
% Explanation within tabular
%    \begin{macrocode}
f & o & o
\end{tabular}
\end{minipage}
\end{table}
}
%    \end{macrocode}
% \end{macro}^^A \foo


• Tim Hoffmann
2013-10-09

Detecting the commands is not an issue. It's the non-stack-like sequence of opening and closing tags which poses the problem. Therefore, the %-and-space characteristic unfortunately doesn't help.

On the bright side, different color markup is very easy. Should also other commands like \begin{macro} be highlighted differently or only macrocode? I could imagine a dtx-environment format for that.

• mrpiggi
2013-10-10

Typically, there are only \DescribeMacro{\cmd} and \DescribeEnv{env} for the user documentation together with \begin{macro}{\cmd} and \begin{environment}{env} for the inline documentation. But with package dox, a class or package author can also create environments and the corresponding API for options, length and other user-defined items. So, if you really want to create a special dtx color markup, it should be customizable.

Since theres are a documentation part and a code part in literate programming, a different background color for both parts would be fine. Here's an example. And maybe it should be possible to control the parenthesis property for macrocode and tags (<*tag>...</tag>) optionally.

Last edit: mrpiggi 2013-10-10

• Tim Hoffmann
2013-10-10

I've added the four commands you specified to highlighting as dtx:commands. Customizable commands are not possible because the the language definitions are currently not user-editable. But I think basic highlighting is still better than no highlighting :). And of course, you can set the format back to the values of the original format if you don't need extra highlighting for these.

• mrpiggi
2013-10-11

Where can I find a beta version for testing? And what about the different background colors for the both segments delimited by macrocode?

• Tim Hoffmann
2013-10-11

The code is in the version control system (mercurial) here on sf. If you want to have it right now, you'll have to compile it yourself. But we'll release a new version in a few days.

The macrocode delimited highlighting suffers from the same problem: As long as the matching of these tags doesn't work correctly, we cannot highlight the lines in between (well at least not without the additional overhead of a separate dtx parser).

1 2 > >> (Page 1 of 2)