#422 Improving table width calculations in LaTeX export

[Clément Pit-Claudel]

Table widths are approximate when generating LaTeX, and this causes overfull hbox warnings, with tables bleeding into the margins. For example:

.. list-table::

   -
     * 1
     * 2
   -
     * A
     * B

This generates the following LaTeX:

\setlength{\DUtablewidth}{\linewidth}
\begin{longtable*}[c]{|p{0.470\DUtablewidth}|p{0.470\DUtablewidth}|}
\hline

1
 & 
2
 \\
\hline

A
 & 
B
 \\
\hline
\end{longtable*}

This produces this warning:

Overfull \hbox (4.50082pt too wide) in alignment at lines 32--46
 [] [] 

The docstring of get_colspecs says "Table width is hairy.", which is true, but things can be improved. The problem is \setlength{\DUtablewidth}{\linewidth}, which doesn't account for the space between columns and the thickness of the vertical rules. The correct calculation in this case is:

\setlength{\DUtablewidth}{\dimexpr\linewidth-4\tabcolsep-3\arrayrulewidth\relax}

Which means:

\setlength{\DUtablewidth}
  {\dimexpr% Start a calculation
   \linewidth% Start with the user-requested width
   -4\tabcolsep% Subtract the column padding: one on each side of central dividers + one on right of leftmost divider + one on left or rightmost divider
   -3\arrayrulewidth%subtract the width of the dividers ("rules")
   \relax% Finish the calculation
  }

The general formula is <requested width> - <2 * ncols> * \tabcolsep - <ncols + 1> * \arrayrulewidth

And in fact with that formula most of the "wiggling room" in the implementation can be removed.

[Günter Milde]

Thank you for the report and analysis.
Unfortunately, the problem is even more tricky, as we also have to take into account the variable number of columns and the width of multi-column cells.
The attached patch (against repository revision 8810) is a suggested fix for the reported problem and more. Have a look at the output of the "latex_cornercases" functional test to see the outcome.

Open questions:

• The patch results in incompatible table layout changes: Apply in version 0.18 or warn in 0.18 and apply in 1.0.
• Column width percentages differ from the ratio of specified or grid-derived :widths: due to the allowance for padding. This is different to HTML output (which has its own problems): use exact ratio
  * when table option :width: is specified,
  * when column :widths: are specified,
  * when both are specified,
  * always?

[Clément Pit-Claudel]

Thanks Günter for the quick reply. I have tested the patch locally on my document and it worked beautifully.

Quick question on this part:

Column width percentages differ from the ratio of specified or grid-derived :widths: due to the allowance for padding.

I don't understand this part fully. Is the problem here is that very narrow columns might not be wide enough to accomodate tabcolsep, so that \dimexpr#1\DUtablewidth-2\tabcolsep\relax might be a negative number if #1 is too small?

I think the allowance solution is OK. Would the alternative of measuring the "available width" (line width - all padding and rules) and then dividing according to user-provided ratios not work? I'm guessing the issue with that is multi-column cells again?

I any case, the deviation from the user-specified ratio is a very minor issue. Still, an alternative would be to issue a warning if any(node['colwidth'] < allowance for node in self._col_specs) (and either let the table overflow in that case, or take corrective action by adding padding only to that narrow column).

Thanks a lot!

[Günter Milde]

On 2021-08-16, Clément Pit-Claudel via Docutils-develop wrote in gmane.text.docutils.devel:

Quick question on this part:

Column width percentages differ from the ratio of specified or
grid-derived :widths: due to the allowance for padding.

I don't understand this part fully. Is the problem here is that very
narrow columns might not be wide enough to accomodate tabcolsep, so
that \dimexpr#1\DUtablewidth-2\tabcolsep\relax might be a negative
number if #1 is too small?

In HTML, the column widths are increased if the content is too wide (unless a
table width is specified). In LaTeX, content flows into the next column.

Even small positive values are problematic with wide characters:
Characters/line with LaTeX article, A4, Times, default margins depends on
character: M: 40, A: 50, x: 70, i: 120.
This is a handicap when determining "nice" column widths from the rST source.
The difference between simple tables and grid tables
described in test/functional/expeced/latex_cornercases.txt makes the
task even harder.

Adding a fixed "allowance" makes most simple tables work "out of the
box".¹ It is an offset for the \tabcolsep and takes into account that
wider columns tend to have a mix of characters leveling things out. The
value of 3.5 turned out to give the best compromise (tested with
"latex_cornercases").

¹Especially for tables with many narrow columns, padding them in the source
is cumbersome.

I think the allowance solution is OK. Would the alternative of
measuring the "available width" (line width - all padding and rules)
and then dividing according to user-provided ratios not work?

It would further deviate from the HTML output, where column ratios are
based on widths including the padding. See the attached
"table-column-widths-test".

To allow fine tuning with similar results in HTML and LaTeX, patch2 does not
add the allowance if the table "width" option is set. (The LaTeX writer
input (the doctree) does not contain information whether column widths
are determined from the grid or from the "widths" option.)

I hope this will suffice to enable both: tables "out of the box" for
standard cases or drafts and a way to set exact table column ratios for
polished output.

[Clément Pit-Claudel]

Yep, that looks excellent. Thanks a lot!

[Günter Milde]

Additional test document. Compare HTML and LaTeX output.

Second proposal.

[Clément Pit-Claudel]

Looks good to me, though I didn't have time for a very thorough check.

[Günter Milde]

A third attempt, this time without much "blackbox magic". Always use the exact ratio like in HTML.
Will require padding or explicite width/widths for some tables in the source.
Could be implemented together with a backwards-compatibility option for the old algorithm.

[Günter Milde]

Fixed in rev. 8835.
Keep allowance to prevent small, unpadded columns becoming too narrow.
New setting "legacy_column_widths" for backwards compatibility.

Thank you for report, analysis, and suggestions.

[Clément Pit-Claudel]

Thanks so much! I'm really happy with the results and with the discussion. Sorry that I was less responsive about the last patch.

[Günter Milde]

Fixed in Docutils 0.18. Thanks again for report and analysis.