docutils-users — General discussions, questions, help requests.

Re: [Docutils-users] latex + pygments

[Guenter M. <mi...@us...>]

On 2023-01-15, Alan G. Isaac wrote:
> What is the current status of pygments support
> for LaTeX writers?

Syntax highlighting with Pygments is implemented in Docutils since
version 0.9. 

Code is parsed by the Pygments syntax highlighter and tokens are
stored in nested inline elements with class arguments according to their
syntactic category. 

As inline elements are supported by all writers, no change is required on
the writer side.

The actual highlighting requires a style-sheet (e.g. one generated by
Pygments, see the sandbox/stylesheets for examples).

See 
https://docutils.sourceforge.io/docs/ref/rst/directives.html#code
https://docutils.sourceforge.io/docs/user/config.html#syntax-highlight

and just try it out.

Günter Milde

[Docutils-users] latex + pygments

[Alan G. I. <ala...@gm...>]

What is the current status of pygments support
for LaTeX writers? I couldn't find much in the
history, and search made it seem that this
https://docutils.sourceforge.io/sandbox/code-block-directive/tools/pygments-enhanced-front-ends/rst2latex-pygments
(in the sandbox) is the basic answer.

Note that I am not advocating for or against support.
I am just trying to understand whether it is dormant
or not.

Thanks! Alan

[Docutils-users] latex write docinfo

[Alan G. I. <ala...@gm...>]

In the latex writer, in
	LaTeXTranslator.depart_document
might it be considered to move the docinfo section
to a separate method? Say `depart_document_docinfo`
or something like that?

Motivation: it is the only part of the method that
rst2beamer changes, but there has been churn in
the other parts of `depart_document`. (So there
was really no need for the rst2beamer method to
get out of sync as it has.)

Thanks for considering,
Alan Isaac

Re: [Docutils-users] rst2beamer problem

[Guenter M. <mi...@us...>]

On 2023-01-14, Alan G. Isaac wrote:
> I'm using current docutils.
> rst2beamer uses the LaTeX writer.

> Has a ``_use_latex_citations`` attribute
> been removed from the docutils LaTeXTranslator?

The name has been normalized with the other "use_latex_..." attributes
in r8880 from 05.11.21

Günter

Re: [Docutils-users] latex writer style sheets

[Alan G. I. <ala...@gm...>]

This is fixed by changing the return of LaTeXTranslator.stylesheet_call to
	return cmd % path.as_posix()
(I.e., by converting the path to a posix representation.)
Alan Isaac


On 1/14/2023 12:39 PM, Alan G. Isaac wrote:
> With current docutils on Windows:
>    --stylesheet=C:/Users/aisaac/svn/aisaac/mydocs/372/372slides
> is producing
>    %%% User specified packages and stylesheets
>    \usepackage{C:\Users\aisaac\svn\aisaac\mydocs\372\372slides}
> 
> The forward slashes need to be retained, of course.
> 
> This might (?) be related to this in the history:
> 
> docutils/utils/__init__.py
> 
>      find_file_in_dirs() now returns a POSIX path also on Windows; get_stylesheet_list() no longer converts "" to "/".
> 
> If so, that is a breaking change.
> 
> Alan Isaac

[Docutils-users] latex writer style sheets

[Alan G. I. <ala...@gm...>]

With current docutils on Windows:
   --stylesheet=C:/Users/aisaac/svn/aisaac/mydocs/372/372slides
is producing
   %%% User specified packages and stylesheets
   \usepackage{C:\Users\aisaac\svn\aisaac\mydocs\372\372slides}

The forward slashes need to be retained, of course.

This might (?) be related to this in the history:

docutils/utils/__init__.py

     find_file_in_dirs() now returns a POSIX path also on Windows; get_stylesheet_list() no longer converts "" to "/".

If so, that is a breaking change.

Alan Isaac

Re: [Docutils-users] rst2beamer problem

[Alan G. I. <ala...@gm...>]

I'm using current docutils.
rst2beamer uses the LaTeX writer.

Has a ``_use_latex_citations`` attribute
been removed from the docutils LaTeXTranslator?
This missing attribute looks to be causing the problem.

Thanks, Alan


On 1/14/2023 3:42 AM, Guenter Milde via Docutils-users wrote:
> On 2023-01-14, Alan G. Isaac wrote:
>> I can no longer compile some documents with rst2beamer.
>> (They compiled a year ago.)
> 
> Which Docutils version are you using now, which a year ago?
> Are you using the LaTeX or the XeTeX writer?
> 
>> Did something change with --use-latex-citations?
>> (I'm not setting it.)
> 
> Not yet. (See RELEASE-NOTES)
> 
> Suggested diagnosis:
> 
> Look out for changes between the relevant versions in the RELEASE-NOTES and
> the HISTORY (there were some default changes recently).
> 
>> Suggested fix?
> 
> Adapt rst2beamer.py to the new behaviour.
> 
> As a stop gap measure: try the various "--legacy..." and "--use-latex..."
> settings.
> 
> 
> Günter
> 
> 
> 
> _______________________________________________
> Docutils-users mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-users
> 
> Please use "Reply All" to reply to the list.

Re: [Docutils-users] rst2beamer problem

[Guenter M. <mi...@us...>]

On 2023-01-14, Alan G. Isaac wrote:
> I can no longer compile some documents with rst2beamer.
> (They compiled a year ago.)

Which Docutils version are you using now, which a year ago?
Are you using the LaTeX or the XeTeX writer?

> Did something change with --use-latex-citations?
> (I'm not setting it.)

Not yet. (See RELEASE-NOTES)

Suggested diagnosis:

Look out for changes between the relevant versions in the RELEASE-NOTES and
the HISTORY (there were some default changes recently).

> Suggested fix?

Adapt rst2beamer.py to the new behaviour.

As a stop gap measure: try the various "--legacy..." and "--use-latex..."
settings.


Günter

[Docutils-users] rst2beamer problem

[Alan G. I. <ala...@gm...>]

I can no longer compile some documents with rst2beamer.
(They compiled a year ago.)
Did something change with --use-latex-citations?
(I'm not setting it.)
Suggested fix?

Alan Isaac

Traceback (most recent call last):
   File "C:\Program Files\Python38\Scripts\rst2beamer-script.py", line 11, in <module>
     load_entry_point('rst2beamer3k==0.9.1', 'console_scripts', 'rst2beamer')()
   File "C:\Program Files\Python38\lib\site-packages\rst2beamer.py", line 1304, in main
     publish_cmdline(
   File "C:\Program Files\Python38\lib\site-packages\docutils-0.20b0.dev0-py3.8.egg\docutils\core.py", line 391, in publish_cmdline
     output = pub.publish(
   File "C:\Program Files\Python38\lib\site-packages\docutils-0.20b0.dev0-py3.8.egg\docutils\core.py", line 227, in publish
     output = self.writer.write(self.document, self.destination)
   File "C:\Program Files\Python38\lib\site-packages\docutils-0.20b0.dev0-py3.8.egg\docutils\writers\__init__.py", line 76, in write
     self.translate()
   File "C:\Program Files\Python38\lib\site-packages\docutils-0.20b0.dev0-py3.8.egg\docutils\writers\latex2e\__init__.py", line 267, in translate
     self.document.walkabout(visitor)
   File "C:\Program Files\Python38\lib\site-packages\docutils-0.20b0.dev0-py3.8.egg\docutils\nodes.py", line 199, in walkabout
     visitor.dispatch_departure(self)
   File "C:\Program Files\Python38\lib\site-packages\docutils-0.20b0.dev0-py3.8.egg\docutils\nodes.py", line 2019, in dispatch_departure
     return method(node)
   File "C:\Program Files\Python38\lib\site-packages\rst2beamer.py", line 930, in depart_document
     if self._use_latex_citations and len(self._bibitems) > 0:
AttributeError: 'BeamerTranslator' object has no attribute '_use_latex_citations'

[Docutils-users] Request for a patch release 0.19.1

[Matthias G. <mat...@gm...>]

Dear docutils maintainers.

A few months ago, I've reported a problem with sphinxcontrib-bibtex
(https://github.com/mcmtroffaes/sphinxcontrib-bibtex/issues/309),
which turned out to actually be a problem with docutils.
There is also a related Sphinx issue
(https://github.com/sphinx-doc/sphinx/issues/10784) which shows that
the problem can also appear without using sphinxcontrib-bibtex.

The maintainer of sphinxcontrib-bibtex has kindly provided a patch
(https://sourceforge.net/p/docutils/patches/195/), which has already
been merged in the meantime.

I don't know the release procedure nor the roadmap of docutils, but
would it be possible to create a new docutils release that contains
this fix (and maybe other improvements)?

Thanks in advance!

In case you are wondering how the problem looks in practice, here is
an example: https://nbsphinx.readthedocs.io/en/0.8.11/a-normal-rst-file.html#citations

cheers,
Matthias

Re: [Docutils-users] Dollar sign disappears

[Satish B.D. <bds...@gm...>]

Thanks Günter,

> I know of a "math extension" for sphinx that designs a special meaning
> (start/stop mathematical mode) to $. There may be other extensions or
> pre-/post-processing changing the outcome...

Indeed it was a post-processing bug, where the dollar had to be
escaped. Sorry for the noise.

Regards,
Satish

Re: [Docutils-users] Dollar sign disappears

[Guenter M. <mi...@us...>]

On 2022-09-09, Satish B.D. wrote:
> Hi,

> Does the dollar sign have any meaning inside double asterisks in rst?
>  I want the following in bold:

> **US $25**

> The HTML is rendered as:

><p><strong>US 5<p><strong>

Here, converting your example line with both,
`rst2html` as well as `rst2html5` results in::

  <p><strong>US $25</strong></p>
  
Docutils version 0.19.1dev
  
> Where did the $2 disappear? Am I missing something?

I can only guess: In standard rST, the character "$" has no special meaning.

I know of a "math extension" for sphinx that designs a special meaning
(start/stop mathematical mode) to $. There may be other extensions or
pre-/post-processing changing the outcome...

Günter

[Docutils-users] Dollar sign disappears

[Satish B.D. <bds...@gm...>]

Hi,

Does the dollar sign have any meaning inside double asterisks in rst?
 I want the following in bold:

**US $25**

The HTML is rendered as:

<p><strong>US 5<p><strong>

Where did the $2 disappear? Am I missing something?

Thanks and regards,
Satish

[Docutils-users] release 0.19 up on pypi

[engelbert g. <eng...@gm...>]

Release 0.19 (2022-07-05)
=========================


(Release 0.19b1 (2022-06-21))


* Drop support for Python 2.7, 3.5, and 3.6.


* Output changes:


  HTML5:

    Wrap groups of footnotes in an ``<aside>`` for easier styling.


    The CSS rule ``.footnote-list { display: contents; }`` can be used to
    restore the behaviour of custom CSS styles.


* After package installation, the CLI commands ``python -m docutils`` and
  ``docutils`` start the `generic command line front end tool`__.


  __ docs/user/tools.html#generic-command-line-front-end


* Support parsing "Markdown" input with 3rd party parsers
  myst_, pycmark_, or recommonmark_.


* The default values for the "pep-references", "rfc-base-url",
  and "python-home" `configuration settings`_ now use the "https:" scheme.
  The PEP-writer template's header is updated to fix links and
  resemble the header of official PEPs.


* Various bugfixes and improvements (see HISTORY_).


.. _myst: https://pypi.org/project/myst-docutils
.. _pycmark: https://pypi.org/project/pycmark/
.. _recommonmark: https://pypi.org/project/recommonmark/
.. _configuration settings: docs/user/config.html

[Docutils-users] docutils prerelease 0.19b1

[engelbert g. <eng...@gm...>]

Thanks to Günter and Adam several changes happened that made a release
necessary

RELEASE NOTES 0.19

* Drop support for Python 2.7, 3.5, and 3.6.
* Output changes:

  HTML5:

    Wrap groups of footnotes in an ``<aside>`` for easier styling.

    The CSS rule ``.footnote-list { display: contents; }`` can be used to

    restore the behaviour of custom CSS styles.

* After package installation, the CLI commands ``python -m docutils`` and

  ``docutils`` start the `generic command line front end tool`__.

  __ docs/user/tools.html#generic-command-line-front-end

* Support parsing "Markdown" input with 3rd party parsers

  myst_, pycmark_, or recommonmark_.

* The default values for the "pep-references", "rfc-base-url",

  and "python-home" `configuration settings`_ now use the "https:" scheme.

  The PEP-writer template's header is updated to fix links and

  resemble the header of official PEPs.

* Various bugfixes and improvements (see HISTORY_).


install with : pip install docutils --pre

Release 0.19 is planned for july 5th

cheers and all the best
 engelbert

[Docutils-users] docutils 0.19 timeline

[engelbert g. <eng...@gm...>]

Hello

due to the work of Günter and Adam a release is necessary

current plan is

0.19.0b1 on tuesday june 21
0.19 on july 5

cheers and thanks for the work
 e

[Docutils-users] I/O uses default encoding argument

[Adam T. <aat...@ou...>]

Using Python 3.10's ``-X warn_default_encoding`` argument to Python, we can see a large number of places where the default encoding is used. On posix systems this is now UTF-8 following PEP 538 [1], but on Windows a non-unicode codepage can be used.

The attached patch fixes the majority of these instances.

A

[1]: https://peps.python.org/pep-0538/

[Docutils-users] Python 3.7 test failure -- `test_CLI`

[Adam T. <aat...@ou...>]

test_CLI fails on Python 3.7 as "utf-8" is capitalised in the actual output.

Attached is a patch to fix the failure.

A

Re: [Docutils-users] rst2html5.py --template from file is not replacing {head} and {body} placeholders in output

[Sorin P. <sor...@gm...>]

Hi Engelbert!

I found that there is a separate pip3 package, rst2html that works.
The one included in docutils-0.18.1 isn't working. Also, this rst2html
package is installing the rst2html command in $PATH, while the one
included in docutils is installing only rst2html.py in $PATH.
I will currently stay with the separate package.

Best regards,
Sorin.


În dum., 8 mai 2022 la 13:06, engelbert gruber
<eng...@gm...> a scris:
>
> hi sorin
>
> sorry for the slow reply.
> did you manage to succeed ?
>
>
> i tried successfully ... rst2html5.py --template=afile.txt
>
> and the template :
>
> %(head_prefix)s
> %(head)s
> %(stylesheet)s
> %(body_prefix)s
> <div id="eins">
> %(body_pre_docinfo)s
> %(docinfo)s
> %(body)s
> </div>
> %(body_suffix)s
>
> On Thu, 28 Apr 2022 at 20:44, Sorin Pânca <sor...@gm...> wrote:
>>
>> Hello!
>> (I am not a subscriber of the mailing list and this is my first message.)
>>
>> I have an issue using --template from file: the {head} and {body}
>> placeholders are not replaced. I installed docutils 0.18.1 using pip3
>> install docutils on Majaro Linux. The template I'm using does not
>> contain something different than the example template, only the URLs
>> inside are changed and a <div> added around {body}.
>> When I'm not using --template, the html is generated properly. When
>> using --template from file, the generated document is the same as the
>> template and it contains the placeholders, just like the template was
>> copied over as output.
>>
>> Am I doing something wrong or is this a bug?
>> Command: rst2html5.py --template='site-root/htdocs/template.html'
>> --no-source-link --report=${ERROR_LEVEL} --no-generator
>> --toc-entry-backlinks --no-section-numbering --strip-comments --strict
>> --tab-width=4 doc.rst >> doc.html
>>
>> Thank you!
>>
>> --
>> Sorin Pânca.
>>
>>
>> _______________________________________________
>> Docutils-users mailing list
>> Doc...@li...
>> https://lists.sourceforge.net/lists/listinfo/docutils-users
>>
>> Please use "Reply All" to reply to the list.



-- 
Sorin Pânca.

Re: [Docutils-users] literal backslash in csv-table data

[Guenter M. <mi...@us...>]

On 2022-05-06, Guenter Milde via Docutils-users wrote:

> Testing with the minimal example (provided in private mail) revealed that
> a backslash must be quadrupled to appear in the output (rsp. the doctree).

> This is a common "feature" if the same escape character is used two times...
> (The first escape-round is the generic rST escape handling.)

Some more experiments showed, that the CSV escape handling is first and
rST escape handling second::

  .. csv-table::
     :header: "testa\" \\*testb*", "testa \*testb*", 2\\\\a, 3\\b
     :stub-columns: 1
   
     1,"(1\\2)","(2,\ *3*)","(3,4)"
     2,"(1,3)","(2,4)","(3,5)"
     3,"(1,4)","(2,5)","(3,6)"


> Looking at the definitions in docutils/parsers/rst/directives/tables.py
> reveals that headers from the argument list and table content are parsed
> using different "CSV dialects": HeaderDialect vs. DocutilsDialect.
> Only the first defines an escape character.

>> If so, it this intentional? (In the csv-table data,
>> they just need to be doubled.)

> Given that this two different csv dialects are defined in tables.py
> since at least 2004, it may be intentional or a side-effect of some
> intention.

> OTOH, the documentation
> https://docutils.sourceforge.io/docs/ref/rst/directives.html#csv-table-1
> says regarding the "header" option:

>   Must use the same CSV format as the main CSV data.

> Either implementation or documentation should be changed. Now we need to
> find out / decide which.

I favour a common dialect for content and header, because
using "\" as escapechar means "\\\\" is required for a literal
backslash in the header (but not the content cells) 
because "\" is also used by the rST escape mechanism :(.

However, I may be just missing the motivation for 
a separate HeaderDialect since the initial implementation
(r2309 "table-related directives" goodger 2004-06-18).
  
Günter  

Re: [Docutils-users] literal backslash in csv-table data

[Guenter M. <mi...@us...>]

On 2022-05-04, Alan G. Isaac wrote:
> Are backslashes stripped out of the csv-table header?

Testing with the minimal example (provided in private mail) revealed that
a backslash must be quadrupled to appear in the output (rsp. the doctree).

This is a common "feature" if the same escape character is used two times...
(The first escape-round is the generic rST escape handling.)

Looking at the definitions in docutils/parsers/rst/directives/tables.py
reveals that headers from the argument list and table content are parsed
using different "CSV dialects": HeaderDialect vs. DocutilsDialect.
Only the first defines an escape character.

> If so, it this intentional? (In the csv-table data,
> they just need to be doubled.)

Given that this two different csv dialects are defined in tables.py
since at least 2004, it may be intentional or a side-effect of some intention.

OTOH, the documentation
https://docutils.sourceforge.io/docs/ref/rst/directives.html#csv-table-1
says regarding the "header" option:

  Must use the same CSV format as the main CSV data.
  
Either implementation or documentation should be changed. Now we need to
find out / decide which.

Günter  

Re: [Docutils-users] <img> instead of <video> tag inserted for image:: foo.mp4

[Guenter M. <mi...@us...>]

Dear Agathe,

On 2022-05-03, Agathe Porte wrote:

...

> As I dig further and further into html5_polyglot, I am sad to see that
> <video> has no first class support in reStructuredText. 

Video is a relatively new addition and only rarely used up to now.
This is also related to poor support in the target document formats 
(just HTML5, no ODF, LaTeX, or troff (manpage)).

> I did a local patch to add more attributes to the renderer, then
> another patch for allowing alternative source format, 

thank you for sharing

> but I think that a new video class should be added instead of hacking
> the image class so deeply. This has been done in a plugin for sphinx
> [2]: full support of a distinct video node that can support extra
> attributes. 

I suppose "class" here means a new document node class and maybe matching
directive? 
This is one way to do it, it would mean a change to the Docutils document
model https://docutils.sourceforge.io/docs/ref/doctree.html as well as
the reStructuredText specification
https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html.

What would be the advantage over a common <image> node for both still and
moving images?

> One quirk of using "classes" for detecting {loop,controls,autoplay} is
> that the video keeps the "loop" CSS style class, which seems not useful
> and can conflict with CSS classes.

This may be solved by popping the relevant values from the "classes" list
(as is done with "language-.." values and, in `html5_polyglot` for the
classes corresponding to semantic HTML5 tags (cf. the
`html5-text-level-tags.txt` functional test input and resulting
`standalone_rst_html5.html`).
(CSS stylesheet rules can also use the "loop" attribute as selector.)

> I was not able to see prior discussion for proper video support in 
> docutils using a search engine. Please enlighten my light if anything 
> seems wrong with this proposal. 

Alternative sources are also useful (and supported for) still images in HTML5.
cf. https://sourceforge.net/p/docutils/mailman/docutils-develop/thread/a461cd5a-6222-45bc-ac66-0be0117e6002%40Spark/#msg37264494

Another use case is providing for alternative formats in the rst source 
for different output formats (e.g. SVG for HTML, PDF for LaTeX).
Different writers would filter out non-supported formats.

Instead of the HTML "srcset" syntax, I consider adding support for
optional nested <source> nodes (similar to the ones in HTML5 <video>).
This would, IMO, give a clearer way to specify image resolution and sizes
than the "srcset" and "sizes" attributes.

We should/could start a dedicated thread in docutils-develop or
feature-enhancement ticket for this.

Thanks,

Günter

Re: [Docutils-users] literal backslash in csv-table data

[Guenter M. <mi...@us...>]

On 2022-05-04, Alan G. Isaac wrote:
> Are backslashes stripped out of the csv-table header?
> If so, it this intentional? (In the csv-table data,
> they just need to be doubled.)

Doubling the backslash should also work in the csv table header. Can you
provide a *minimal* example (just the rST and description of how you
convert it)?

Günter

[Docutils-users] literal backslash in csv-table data

[Alan G. I. <ala...@gm...>]

Are backslashes stripped out of the csv-table header?
If so, it this intentional? (In the csv-table data,
they just need to be doubled.)

Thank you, Alan Isaac

Re: [Docutils-users] <img> instead of <video> tag inserted for image:: foo.mp4

[Agathe P. <mic...@mi...>]

Hi,

29/04/2022 10:54, Guenter Milde :
>> However, according to the documentation [1], this should export a
>> <video> tag instead of a <img> tag given that the file extension is ".mp4".
> It seems that Pelican (at least with your configuration) uses the
> traditional HTML4 writer for HTML export.
>
> Only the `HTML5 writer`_ supports video. Finding out how/whether Pelican
> can be configured to use the HTML5 writer is left as an exercise for the
> reader.
>
I was able to configure pelican to use html5_polyglot using 
rst_with_html5 Pelican plugin [1]. As I dig further and further into 
html5_polyglot, I am sad to see that <video> has no first class support 
in reStructuredText. I did a local patch to add more attributes to the 
renderer, then another patch for allowing alternative source format, but 
I think that a new video class should be added instead of hacking the 
image class so deeply.

This has been done in a plugin for sphinx [2]: full support of a 
distinct video node that can support extra attributes. One quirk of 
using "classes" for detecting {loop,controls,autoplay} is that the video 
keeps the "loop" CSS style class, which seems not useful and can 
conflict with CSS classes.

I was not able to see prior discussion for proper video support in 
docutils using a search engine. Please enlighten my light if anything 
seems wrong with this proposal. I can volonteer to implement this 
feature and document it for docutils. I think it would be better than 
the current two alternatives: using out-of-tree plugins or hacking image 
nodes.

Bests,

Agata

[1] https://github.com/bekcpear/pelican-rst_with_html5

[2] 
https://github.com/sphinx-contrib/video/blob/master/sphinxcontrib/video.py