docutils-develop — For developer discussions of the implementation.

[Docutils-develop] [docutils:bugs] Re: #450 0.18: New HTML markup for footnotes is difficult to stylise

[Günter M. <mi...@us...>]

> Must the role be part of the element definition? 

Current praxis is to use the `<footnote>` element for both:
a) individual notes that occur within the body of a work,
b) collections of notes that occur at the end of a section.
The *Digital Publishing WAI-ARIA Module 1.0*  explicitely says that [doc-footnote](https://www.w3.org/TR/dpub-aria-1.0/#doc-footnote) 
is only for usage case a).

Despite this, I announced the change to "doc-footnote", assuming that
1. this is the more frequent use and a better match to the element's name,
2. a possible future "select footnotes" directive (see TODO.txt) may serve for use case b).




---

** [bugs:#450] 0.18: New HTML markup for footnotes is difficult to stylise**

**Status:** open
**Created:** Sun Jun 05, 2022 08:28 PM UTC by Pradyun Gedam
**Last Updated:** Tue Jun 14, 2022 04:01 PM UTC
**Owner:** nobody


Docutils 0.18 changed the HTML markup for footnotes/citations generated for rst documents. This changed markup is significantly more difficult to stylise with CSS, in ways that was possible with Docutils 0.17. This has negatively affected HTML themes for Sphinx, since the Sphinx 5 release allows using docutils 0.18, which is preferentially installed by `pip`.

The reasons for this boils down to an inconsistent number of elements inside each `aside`.

1. It's not possible use CSS grid layouts sanely, with this setup.
2. Even if you added multiple rules based on number of elements in the section, it's not possible to know what the 2nd element might be -- which balloons the complexity+size of the stylesheet, if it tries to accomodate for this.

It is theoretically possible to stylise this but it would be on the order of 100s of lines of CSS to get this right, compared to ~20 with 0.17.

Would it be possible to change this markup, to wrap the label & backrefs in a `div` and to wrap the content paragraphs in a separate `div` as well? This would make it possible to stylise this content in ways that were feasible with 0.17, with significantly less complexity in the stylesheets.

---

Sources:

```
[some content that references these footnotes]

.. [1] A footnote contains body elements, consistently indented by at
   least 3 spaces.

   This is the footnote's second paragraph.

.. [#label] Footnotes may be numbered, either manually (as in [1]_) or
   automatically using a "#"-prefixed label.  This footnote has a
   label so it can be referred to from multiple places, both as a
   footnote reference ([#label]_) and as a hyperlink reference
   (label_).

```

0.17 output HTML:

```
<dl class="footnote brackets">
  <dt class="label" id="id6"><span class="brackets">1</span><span class="fn-backref">(<a href="#id1">1</a>,<a
        href="#id7">2</a>)</span></dt>
  <dd>
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </dd>
  <dt class="label" id="label"><span class="brackets">2</span><span class="fn-backref">(<a href="#id3">1</a>,<a
        href="#id8">2</a>)</span></dt>
  <dd>
    <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6"
        id="id7">1</a>) or
      automatically using a “#”-prefixed label. This footnote has a
      label so it can be referred to from multiple places, both as a
      footnote reference (<a class="footnote-reference brackets" href="#label" id="id8">2</a>) and as a hyperlink
      reference
      (<a class="reference internal" href="#label">label</a>).</p>
  </dd>
</dl>
```

0.18 output HTML:

```
<aside class="footnote brackets" id="id6" role="note">
  <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  <p>A footnote contains body elements, consistently indented by at
    least 3 spaces.</p>
  <p>This is the footnote’s second paragraph.</p>
</aside>
<aside class="footnote brackets" id="label" role="note">
  <span class="label"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id3" role="doc-backlink">1</a>,<a href="#id8" role="doc-backlink">2</a>)</span>
  <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6" id="id7"
      role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>) or
    automatically using a “#”-prefixed label. This footnote has a
    label so it can be referred to from multiple places, both as a
    footnote reference (<a class="footnote-reference brackets" href="#label" id="id8" role="doc-noteref"><span
        class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>) and as a hyperlink reference
    (<a class="reference internal" href="#label">label</a>).</p>
</aside>
```

The suggested change to the markup is to generate something like (using only the first aside for this demo):

```
<aside class="footnote brackets" id="id6" role="note">
  <div class="footnote-label">
    <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
    <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  </div>
  <div class="footnote-content">
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </div>
</aside>
```


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] docutils 0.19 timeline

[engelbert g. <eng...@gm...>]

Hello

due to the work of Günter and Adam a release is necessary

current plan is

0.19.0b1 on tuesday june 21
0.19 on july 5

cheers and thanks for the work
 e

[Docutils-develop] Participation in Docutils: Documentation Utilities project.

[DENNISSE P. Z. M. <den...@ut...>]

Dear Leah Wiemann.
We are a group of students from the State Technical University of Quevedo
and we want to participate in the Docutils: Documentation Utilities project.
We are interested in working on usability issues. We carry out usability
tests to find aspects to improve in the Docutils: Documentation Utilities
application.
The final result of the usability tests will be shared with the community.
We would like to know if it is possible to participate in the project.
Cheers,
Paola Zambrano

-- 
Universidad Técnica Estatal de QuevedoLa Primera Universidad Agropecuaria 
del País

Re: [Docutils-develop] I/O uses default encoding argument

[Guenter M. <mi...@us...>]

On 2022-06-16, Adam Turner wrote:

> Attached is a set of five patches rebased on current master

Thanks. I had a look at the first 4 and took them into account in commits
[r9075] to [r9078].

Günter

Re: [Docutils-develop] I/O uses default encoding argument

[Guenter M. <mi...@us...>]

Dear Adam,

On 2022-06-15, Adam Turner wrote:

> Unify naming of the "utf-8" codec
> ---------------------------------

>> I'd prefer 'utf-8' (lowercase, in quotes) also in documentation, if it
>> refers to the Python codec and UTF-8 for the abstract encoding
>> algorithm.

> [...] I couldn't find anywhere in my patch set that I would change [...]

Sorry, this was replying to an earlier statement ("UTF-8 in documentation"). 
Patch https://github.com/AA-Turner/docutils/pull/15/commits/f7f45addbd8cc728ef03c28d62b6ea981d0fc8ac
states it very well:

  - Use UTF-8 in prose text, error messages, and documentation
  - Use utf-8 in code or when referring to code
  - Use utf8 for LaTeX

I did not apply the changes in the sample SVG images
(generated with Inkscape), though.


> Add encoding arguments
> ----------------------

>> Don't add encoding when the locale encoding is OK.
>>  (We may switch to "locale" after implementing it in `docutils.io`.)

> Outwith ``FileInput``, where would you want to use 'locale' for the encoding?

"quicktest.py" is an old developer diagnostics tool without an option to
select the input/output encodings.  
I suggest keeping the encoding unspecified here, so Python's default is
used and the user can change the encoding via either a locale setting or
starting Python in UTF-8 mode.

...


> Handle encoding='locale' for docutils.io.Output
> -----------------------------------------------

Which encoding is used with ``open('foo', encoding='locale')``
if Python is in UTF-8 mode?


> I don't mind about putting support for ``encoding='locale'`` on just
> FileInput/FileOutput -- what would your preference be here?

We want to drop our 'locale' support when dropping support for Py<3.10.
Does Python support 'locale' also with str.encode()?

Maybe we don't even need backporting "locale" (see below).


> Deprecations
> ------------

>> Why do you want to deprecate ``io.locale_encoding``?

> Because after introducing ``encoding='locale'`` there's no use for
``io.locale_encoding`` in Docutils anymore, and to reduce API surface.

OK. We do not need special deprecation, as `io.locale_encoding` is new in
Docutils 0.19.dev (moved from `utils.error_reporting`).


>> Why do you want to deprecate auto-detection of the input encoding?
>> * ``encoding='locale'`` does not help if my input files are a mix of
>>   UTF-8 and latin-1.

> "auto-guessing" is a poor term -- basically I meant deprecating using
> the locale encoding as default (as it will change to UTF-8). 

> I'm not sure I understand the example you gave as Docutils works on a
> single file basis. Could you add more context please?

What I want to keep/restore is the "auto-detect" default behaviour for
reading/decoding input on Python2 (when opening files under Python 3,
this only kicks in when the first try rises an UnicodeError):

With unspecified `input_encoding` setting, `io.Input.decode` does:

a) Check the BOM mark and top 2 lines of data for an encoding specification
   and use it, else

b) try UTF-8.

c) If this fails, try the locale encoding (if valid).

d) Try latin-1.

e) Give up, report the error.

This allows decoding most input without the need to configure an encoding.


Whether the future default "input-encoding" should be "auto-detect" or
"utf-8" may be decided later. 

In any case I would keep "auto-detect" as an option.

Future (incompatible) changes:

* use `locale.getpreferredencoding()` in c):
  If a user starts Python in UTF-8 mode, we should report decoding errors
  instead of trying a locale encoding.

* maybe drop d)

* warn/info when input encoding is not UTF-8.


Günter

Re: [Docutils-develop] I/O uses default encoding argument

[Adam T. <aat...@ou...>]

Attached is a set of five patches rebased on current master -- I have updated the language in the deprecation warnings, used the encoding='locale' backport only for 3.7-3.9 (as 3.10 ``builtins.open`` knows about encoding='locale' natively), and updated the ``io.locale_encoding`` detection mechanism to ignore ``-X utf8``, as the system locale encoding doesn't change for the Python UTF-8 mode.

A

Re: [Docutils-develop] I/O uses default encoding argument

[Adam T. <aat...@ou...>]

> Parts of the patch-set that (IMO) do not require further discussion are now
> committed to master.

Thank you.


Unify naming of the "utf-8" codec
---------------------------------

> I'd prefer 'utf-8' (lowercase, in quotes) also in documentation, if it
> refers to the Python codec and UTF-8 for the abstract encoding
> algorithm.

This makes sense, although for specific references to the stdlib implementation of UTF-8 as in the ``encodings.utf_8`` module we could be explicit. I couldn't find anywhere in my patch set that I would change, but I may have missed something -- were there any specific instances you were thinking of?


Add encoding arguments
----------------------

Changes:

> Don't add encoding when the locale encoding is OK.
>  (We may switch to "locale" after implementing it in `docutils.io`.)

Outwith ``FileInput``, where would you want to use 'locale' for the encoding?

This diverges with the custom and practise in the general Python ecosystem (and as far as I can tell encodings in general) -- I would strongly suggest using UTF-8, as it eliminates an entire class of locale/encoding related bugs.

> Document changes that may affect users.
> Use 'ascii' in "tools/dev/unicode2rstsubs.py". 

Makes sense, thanks.

> Break too long lines.

Sorry, I thought I'd done a formatting pass but seemingly not.


Ensure locale_encoding is lower case
------------------------------------

> We can use locale.getpreferredencoding() after dropping Python versions
   where this was problematic.

Great, thanks.


Handle encoding='locale' for docutils.io.Output
-----------------------------------------------

> Is uppercase ``encoding='LOCALE'`` supported in the standard
> function open() in Python >= 3.10?

Good question, I tested and only the exact literal ``locale`` is accepted, so we can drop the ``.lower()`` call.

> IMO, we need ``encoding='locale'`` support in both, input and output.
> Should ``encoding='locale'`` be supported in all Input/Output classes or
> only in FileInput/FileOutput?

The patch set I set last time does, via the default encoding helper method I added.

I don't mind about putting support for ``encoding='locale'`` on just FileInput/FileOutput -- what would your preference be here?


Deprecations
------------

> Why do you want to deprecate ``io.locale_encoding``?

Because after introducing ``encoding='locale'`` there's no use for ``io.locale_encoding`` in Docutils anymore, and to reduce API surface.

> Why do you want to deprecate auto-detection of the input encoding?
> * ``encoding='locale'`` does not help if my input files are a mix of
>   UTF-8 and latin-1.

"auto-guessing" is a poor term -- basically I meant deprecating using the locale encoding as default (as it will change to UTF-8).
I'm not sure I understand the example you gave as Docutils works on a single file basis. Could you add more context please?

> Using Python 3.10's ``-X warn_default_encoding`` argument to Python,
> we can see a large number of places where the default encoding is
> used. On posix systems this is now UTF-8 following PEP 538 [1], but on
> Windows a non-unicode codepage can be used.

> Also on POSIX, the locale encoding is kept unless the locale is "C".

Yes, sorry, I wasn't precise enough.

Thanks,
Adam

Re: [Docutils-develop] I/O uses default encoding argument

[Guenter M. <mi...@us...>]

Dear Adam,

thank you for the update patches.

Parts of the patch-set that (IMO) do not require further discussion are now
committed to master.

Unify naming of the "utf-8" codec
---------------------------------

> I propose using UTF-8 (uppercase) in documentation and prose text and
> utf-8 (lowercase) in code

I'd prefer 'utf-8' (lowercase, in quotes) also in documentation, if it
refers to the Python codec and UTF-8 for the abstract encoding
algorithm.

r9068


Add encoding arguments
----------------------

Changes:

* Don't add encoding when the locale encoding is OK.
  (We may switch to "locale" after implementing it in `docutils.io`.)

* Document changes that may affect users.

* Use 'ascii' in "tools/dev/unicode2rstsubs.py". 
  Its a developer tool. The generated files should be usable with any
  ASCII-compatible encoding.

* Break too long lines.

r9072


Ensure locale_encoding is lower case
------------------------------------

Some simplifications:

* We can use locale.getpreferredencoding() after dropping Python versions
  where this was problematic.

* We can append ``.lower()`` as there is a catchall ``except`` later.

TODO: check whether io.locale_encoding is set correctly with every OS and
      Python version or whether front-end tools would need to call
      `locale.setlocale()` before importing this module.


Handle encoding='locale' for docutils.io.Output 
-----------------------------------------------

Is uppercase ``encoding='LOCALE'`` supported in the standard
function open() in Python >= 3.10?

IMO, we need ``encoding='locale'`` support in both, input and output.

Should ``encoding='locale' be supported in all Input/Output classes or
only in FileInput/FileOutput?



Deprecations 
------------

Why do you want to deprecate ``io.locale_encoding``?

Why do you want to deprecate auto-detection of the input encoding?

* ``encoding='locale'`` does not help if my input files are a mix of
  UTF-8 and latin-1.


> Using Python 3.10's ``-X warn_default_encoding`` argument to Python,
> we can see a large number of places where the default encoding is
> used. On posix systems this is now UTF-8 following PEP 538 [1], but on
> Windows a non-unicode codepage can be used.

Also on POSIX, the locale encoding is kept unless the locale is "C".

Test:

After setting up locales de_DE-UTF-8 and de_DE-ISO-8859-1 on my
Debian/stable system, I get::

  milde@heinz:~ > export LC_ALL=de_DE
  milde@heinz:~ > python3
  Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
  [GCC 10.2.1 20210110] on linux
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import locale
  >>> locale.getpreferredencoding()
  'ISO-8859-1'

Reading a latin-1 encoded file works::

  >>> f = open('/tmp/moff.txt')
  >>> f.read()
  'Grüße\n'

while reading the same file with utf-8 fails::

  >>> f = open('/tmp/moff.txt', encoding='utf-8')
  >>> f.read()
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/usr/lib/python3.9/codecs.py", line 322, in decode
      (result, consumed) = self._buffer_decode(data, self.errors, final)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 2: invalid start byte


Günter

[Docutils-develop] [docutils:bugs] Re: #450 0.18: New HTML markup for footnotes is difficult to stylise

[Günter M. <mi...@us...>]

Specifying the "footnote-reference" value on the wrapper
element would also allow to get rid of some redundancy in the HTML source
by not replicating the "brackets" or "superscript" class value on every
individual footnote element in future (after preparing the style-sheets and
an advance warning).




---

** [bugs:#450] 0.18: New HTML markup for footnotes is difficult to stylise**

**Status:** open
**Created:** Sun Jun 05, 2022 08:28 PM UTC by Pradyun Gedam
**Last Updated:** Tue Jun 14, 2022 04:01 PM UTC
**Owner:** nobody


Docutils 0.18 changed the HTML markup for footnotes/citations generated for rst documents. This changed markup is significantly more difficult to stylise with CSS, in ways that was possible with Docutils 0.17. This has negatively affected HTML themes for Sphinx, since the Sphinx 5 release allows using docutils 0.18, which is preferentially installed by `pip`.

The reasons for this boils down to an inconsistent number of elements inside each `aside`.

1. It's not possible use CSS grid layouts sanely, with this setup.
2. Even if you added multiple rules based on number of elements in the section, it's not possible to know what the 2nd element might be -- which balloons the complexity+size of the stylesheet, if it tries to accomodate for this.

It is theoretically possible to stylise this but it would be on the order of 100s of lines of CSS to get this right, compared to ~20 with 0.17.

Would it be possible to change this markup, to wrap the label & backrefs in a `div` and to wrap the content paragraphs in a separate `div` as well? This would make it possible to stylise this content in ways that were feasible with 0.17, with significantly less complexity in the stylesheets.

---

Sources:

```
[some content that references these footnotes]

.. [1] A footnote contains body elements, consistently indented by at
   least 3 spaces.

   This is the footnote's second paragraph.

.. [#label] Footnotes may be numbered, either manually (as in [1]_) or
   automatically using a "#"-prefixed label.  This footnote has a
   label so it can be referred to from multiple places, both as a
   footnote reference ([#label]_) and as a hyperlink reference
   (label_).

```

0.17 output HTML:

```
<dl class="footnote brackets">
  <dt class="label" id="id6"><span class="brackets">1</span><span class="fn-backref">(<a href="#id1">1</a>,<a
        href="#id7">2</a>)</span></dt>
  <dd>
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </dd>
  <dt class="label" id="label"><span class="brackets">2</span><span class="fn-backref">(<a href="#id3">1</a>,<a
        href="#id8">2</a>)</span></dt>
  <dd>
    <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6"
        id="id7">1</a>) or
      automatically using a “#”-prefixed label. This footnote has a
      label so it can be referred to from multiple places, both as a
      footnote reference (<a class="footnote-reference brackets" href="#label" id="id8">2</a>) and as a hyperlink
      reference
      (<a class="reference internal" href="#label">label</a>).</p>
  </dd>
</dl>
```

0.18 output HTML:

```
<aside class="footnote brackets" id="id6" role="note">
  <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  <p>A footnote contains body elements, consistently indented by at
    least 3 spaces.</p>
  <p>This is the footnote’s second paragraph.</p>
</aside>
<aside class="footnote brackets" id="label" role="note">
  <span class="label"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id3" role="doc-backlink">1</a>,<a href="#id8" role="doc-backlink">2</a>)</span>
  <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6" id="id7"
      role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>) or
    automatically using a “#”-prefixed label. This footnote has a
    label so it can be referred to from multiple places, both as a
    footnote reference (<a class="footnote-reference brackets" href="#label" id="id8" role="doc-noteref"><span
        class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>) and as a hyperlink reference
    (<a class="reference internal" href="#label">label</a>).</p>
</aside>
```

The suggested change to the markup is to generate something like (using only the first aside for this demo):

```
<aside class="footnote brackets" id="id6" role="note">
  <div class="footnote-label">
    <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
    <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  </div>
  <div class="footnote-content">
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </div>
</aside>
```


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

Re: [Docutils-develop] Docutils 0.19 Release

[Guenter M. <mi...@us...>]

On 2022-06-12, engelbert gruber wrote:

> yes I am for rolling 0.19 out

> maybe I am able to ship 0.19.0b1 on tuesday (tuesday is release day) june 14
> or waiting for 450 on 21.

I suggest June 21th, so we can agree on the optimal approach regarding
HTML5 footnotes and give it some road testing.

Thanks,
Günter

[Docutils-develop] [docutils:bugs] Re: #450 0.18: New HTML markup for footnotes is difficult to stylise

[Adam T. <aa-...@us...>]

+1, makes sense. The CSS Selectors Level 4 standard introduces a `:has` selector [1]_ that would allow doing `.footnote-list:has(> .brackets)` but it is only supported  by Safari according to MDN -- so the approach in your patch is needed.

_[1]: https://developer.mozilla.org/en-US/docs/Web/CSS/:has

A


---

** [bugs:#450] 0.18: New HTML markup for footnotes is difficult to stylise**

**Status:** open
**Created:** Sun Jun 05, 2022 08:28 PM UTC by Pradyun Gedam
**Last Updated:** Tue Jun 14, 2022 02:03 PM UTC
**Owner:** nobody


Docutils 0.18 changed the HTML markup for footnotes/citations generated for rst documents. This changed markup is significantly more difficult to stylise with CSS, in ways that was possible with Docutils 0.17. This has negatively affected HTML themes for Sphinx, since the Sphinx 5 release allows using docutils 0.18, which is preferentially installed by `pip`.

The reasons for this boils down to an inconsistent number of elements inside each `aside`.

1. It's not possible use CSS grid layouts sanely, with this setup.
2. Even if you added multiple rules based on number of elements in the section, it's not possible to know what the 2nd element might be -- which balloons the complexity+size of the stylesheet, if it tries to accomodate for this.

It is theoretically possible to stylise this but it would be on the order of 100s of lines of CSS to get this right, compared to ~20 with 0.17.

Would it be possible to change this markup, to wrap the label & backrefs in a `div` and to wrap the content paragraphs in a separate `div` as well? This would make it possible to stylise this content in ways that were feasible with 0.17, with significantly less complexity in the stylesheets.

---

Sources:

```
[some content that references these footnotes]

.. [1] A footnote contains body elements, consistently indented by at
   least 3 spaces.

   This is the footnote's second paragraph.

.. [#label] Footnotes may be numbered, either manually (as in [1]_) or
   automatically using a "#"-prefixed label.  This footnote has a
   label so it can be referred to from multiple places, both as a
   footnote reference ([#label]_) and as a hyperlink reference
   (label_).

```

0.17 output HTML:

```
<dl class="footnote brackets">
  <dt class="label" id="id6"><span class="brackets">1</span><span class="fn-backref">(<a href="#id1">1</a>,<a
        href="#id7">2</a>)</span></dt>
  <dd>
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </dd>
  <dt class="label" id="label"><span class="brackets">2</span><span class="fn-backref">(<a href="#id3">1</a>,<a
        href="#id8">2</a>)</span></dt>
  <dd>
    <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6"
        id="id7">1</a>) or
      automatically using a “#”-prefixed label. This footnote has a
      label so it can be referred to from multiple places, both as a
      footnote reference (<a class="footnote-reference brackets" href="#label" id="id8">2</a>) and as a hyperlink
      reference
      (<a class="reference internal" href="#label">label</a>).</p>
  </dd>
</dl>
```

0.18 output HTML:

```
<aside class="footnote brackets" id="id6" role="note">
  <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  <p>A footnote contains body elements, consistently indented by at
    least 3 spaces.</p>
  <p>This is the footnote’s second paragraph.</p>
</aside>
<aside class="footnote brackets" id="label" role="note">
  <span class="label"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id3" role="doc-backlink">1</a>,<a href="#id8" role="doc-backlink">2</a>)</span>
  <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6" id="id7"
      role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>) or
    automatically using a “#”-prefixed label. This footnote has a
    label so it can be referred to from multiple places, both as a
    footnote reference (<a class="footnote-reference brackets" href="#label" id="id8" role="doc-noteref"><span
        class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>) and as a hyperlink reference
    (<a class="reference internal" href="#label">label</a>).</p>
</aside>
```

The suggested change to the markup is to generate something like (using only the first aside for this demo):

```
<aside class="footnote brackets" id="id6" role="note">
  <div class="footnote-label">
    <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
    <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  </div>
  <div class="footnote-content">
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </div>
</aside>
```


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #450 0.18: New HTML markup for footnotes is difficult to stylise

[Günter M. <mi...@us...>]

One more variation: add the bracket/superscript class value on the wrapper, too.



Attachments:

- [0001-Add-footnote-references-value-to-the-wrappers-classes.patch](https://sourceforge.net/p/docutils/bugs/_discuss/thread/af593fcd69/937a/460a/81bc/e203/ecc3/c6b8/afff/c45c/attachment/0001-Add-footnote-references-value-to-the-wrappers-classes.patch) (10.7 kB; text/x-patch)


---

** [bugs:#450] 0.18: New HTML markup for footnotes is difficult to stylise**

**Status:** open
**Created:** Sun Jun 05, 2022 08:28 PM UTC by Pradyun Gedam
**Last Updated:** Mon Jun 13, 2022 08:55 PM UTC
**Owner:** nobody


Docutils 0.18 changed the HTML markup for footnotes/citations generated for rst documents. This changed markup is significantly more difficult to stylise with CSS, in ways that was possible with Docutils 0.17. This has negatively affected HTML themes for Sphinx, since the Sphinx 5 release allows using docutils 0.18, which is preferentially installed by `pip`.

The reasons for this boils down to an inconsistent number of elements inside each `aside`.

1. It's not possible use CSS grid layouts sanely, with this setup.
2. Even if you added multiple rules based on number of elements in the section, it's not possible to know what the 2nd element might be -- which balloons the complexity+size of the stylesheet, if it tries to accomodate for this.

It is theoretically possible to stylise this but it would be on the order of 100s of lines of CSS to get this right, compared to ~20 with 0.17.

Would it be possible to change this markup, to wrap the label & backrefs in a `div` and to wrap the content paragraphs in a separate `div` as well? This would make it possible to stylise this content in ways that were feasible with 0.17, with significantly less complexity in the stylesheets.

---

Sources:

```
[some content that references these footnotes]

.. [1] A footnote contains body elements, consistently indented by at
   least 3 spaces.

   This is the footnote's second paragraph.

.. [#label] Footnotes may be numbered, either manually (as in [1]_) or
   automatically using a "#"-prefixed label.  This footnote has a
   label so it can be referred to from multiple places, both as a
   footnote reference ([#label]_) and as a hyperlink reference
   (label_).

```

0.17 output HTML:

```
<dl class="footnote brackets">
  <dt class="label" id="id6"><span class="brackets">1</span><span class="fn-backref">(<a href="#id1">1</a>,<a
        href="#id7">2</a>)</span></dt>
  <dd>
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </dd>
  <dt class="label" id="label"><span class="brackets">2</span><span class="fn-backref">(<a href="#id3">1</a>,<a
        href="#id8">2</a>)</span></dt>
  <dd>
    <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6"
        id="id7">1</a>) or
      automatically using a “#”-prefixed label. This footnote has a
      label so it can be referred to from multiple places, both as a
      footnote reference (<a class="footnote-reference brackets" href="#label" id="id8">2</a>) and as a hyperlink
      reference
      (<a class="reference internal" href="#label">label</a>).</p>
  </dd>
</dl>
```

0.18 output HTML:

```
<aside class="footnote brackets" id="id6" role="note">
  <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  <p>A footnote contains body elements, consistently indented by at
    least 3 spaces.</p>
  <p>This is the footnote’s second paragraph.</p>
</aside>
<aside class="footnote brackets" id="label" role="note">
  <span class="label"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id3" role="doc-backlink">1</a>,<a href="#id8" role="doc-backlink">2</a>)</span>
  <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6" id="id7"
      role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>) or
    automatically using a “#”-prefixed label. This footnote has a
    label so it can be referred to from multiple places, both as a
    footnote reference (<a class="footnote-reference brackets" href="#label" id="id8" role="doc-noteref"><span
        class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>) and as a hyperlink reference
    (<a class="reference internal" href="#label">label</a>).</p>
</aside>
```

The suggested change to the markup is to generate something like (using only the first aside for this demo):

```
<aside class="footnote brackets" id="id6" role="note">
  <div class="footnote-label">
    <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
    <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  </div>
  <div class="footnote-content">
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </div>
</aside>
```


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] #194 Deprecate PEP 263 coding slugs support

[Günter M. <mi...@us...>]

Ticket moved from /p/docutils/bugs/451/


---

** [patches:#194] Deprecate PEP 263 coding slugs support**

**Status:** open
**Group:** None
**Created:** Thu Jun 09, 2022 10:48 PM UTC by Adam  Turner
**Last Updated:** Tue Jun 14, 2022 10:15 AM UTC
**Owner:** nobody


Python 3 uses utf-8 as the encoding for Python source files, there is no longer a compelling use-case for the support, which adds complexity to the IO implementation.

I propose deprecating support for removal in 1.0, but 2.0 might be a better option.

Support was added in [r4506].

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

Re: [Docutils-develop] Docutils 0.19 Release

[Adam T. <aat...@ou...>]

> The last patch for the problem described in bug #450 is "minimally invasive"
> but still an incompatible API change.
> https://sourceforge.net/p/docutils/bugs/450/#937a/460a/81bc/e203/ecc3/c6b8

> I am still unsure whether curing an announced incompatible API change
> with an unannounced one is the right way.
> I would prefer announcing in 0.19 and changing in 0.20 but can also live
> with an unannounced fix, if this is considered the lesser evil.

I would argue the latter, that it is better to have the fix released. For users who just use the default stylesheets, the fix will be automatic as we've fixed the stylesheets. For users who use a downstream framework (e.g. Sphinx) there will be no immediate effect as those frameworks put an upper bound on their Docutils version.

The class of affected users I can think of is those users who use Docutils directly with custom stylesheets -- and I think for this class of users the benefit of better control of styling outweighs the minor annoyance of (potentially) updating some CSS -- as we see from the latest patch the changes required are small.

Whilst not ideal therefore, I vote for including the patch in 0.19.0b0.

> (We still don't know, whether the fix would satisfy the OP.)

I have asked Pradyun on the issue to respond.

> > with 0.19.0 final then released on 05/07.

> We have done some large changes and some removals,
> so maybe a longer beta period is appropriate...

We do beta releases for Sphinx and very few people ever test them. We got one bug report for the 5.0.0 beta release and then over ten for when we released 5.0.0 final. Pragmatically, I think we should stick to the standard schedule, and just be prepared to deal with bug fixes as and when needed.

A

[Docutils-develop] [docutils:bugs] Re: #450 0.18: New HTML markup for footnotes is difficult to stylise

[Adam T. <aa-...@us...>]

@pradyun -- please may you test this patch & see if it allows you parity styling in furo with 0.17 Docutils?

A


---

** [bugs:#450] 0.18: New HTML markup for footnotes is difficult to stylise**

**Status:** open
**Created:** Sun Jun 05, 2022 08:28 PM UTC by Pradyun Gedam
**Last Updated:** Mon Jun 13, 2022 08:19 PM UTC
**Owner:** nobody


Docutils 0.18 changed the HTML markup for footnotes/citations generated for rst documents. This changed markup is significantly more difficult to stylise with CSS, in ways that was possible with Docutils 0.17. This has negatively affected HTML themes for Sphinx, since the Sphinx 5 release allows using docutils 0.18, which is preferentially installed by `pip`.

The reasons for this boils down to an inconsistent number of elements inside each `aside`.

1. It's not possible use CSS grid layouts sanely, with this setup.
2. Even if you added multiple rules based on number of elements in the section, it's not possible to know what the 2nd element might be -- which balloons the complexity+size of the stylesheet, if it tries to accomodate for this.

It is theoretically possible to stylise this but it would be on the order of 100s of lines of CSS to get this right, compared to ~20 with 0.17.

Would it be possible to change this markup, to wrap the label & backrefs in a `div` and to wrap the content paragraphs in a separate `div` as well? This would make it possible to stylise this content in ways that were feasible with 0.17, with significantly less complexity in the stylesheets.

---

Sources:

```
[some content that references these footnotes]

.. [1] A footnote contains body elements, consistently indented by at
   least 3 spaces.

   This is the footnote's second paragraph.

.. [#label] Footnotes may be numbered, either manually (as in [1]_) or
   automatically using a "#"-prefixed label.  This footnote has a
   label so it can be referred to from multiple places, both as a
   footnote reference ([#label]_) and as a hyperlink reference
   (label_).

```

0.17 output HTML:

```
<dl class="footnote brackets">
  <dt class="label" id="id6"><span class="brackets">1</span><span class="fn-backref">(<a href="#id1">1</a>,<a
        href="#id7">2</a>)</span></dt>
  <dd>
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </dd>
  <dt class="label" id="label"><span class="brackets">2</span><span class="fn-backref">(<a href="#id3">1</a>,<a
        href="#id8">2</a>)</span></dt>
  <dd>
    <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6"
        id="id7">1</a>) or
      automatically using a “#”-prefixed label. This footnote has a
      label so it can be referred to from multiple places, both as a
      footnote reference (<a class="footnote-reference brackets" href="#label" id="id8">2</a>) and as a hyperlink
      reference
      (<a class="reference internal" href="#label">label</a>).</p>
  </dd>
</dl>
```

0.18 output HTML:

```
<aside class="footnote brackets" id="id6" role="note">
  <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  <p>A footnote contains body elements, consistently indented by at
    least 3 spaces.</p>
  <p>This is the footnote’s second paragraph.</p>
</aside>
<aside class="footnote brackets" id="label" role="note">
  <span class="label"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id3" role="doc-backlink">1</a>,<a href="#id8" role="doc-backlink">2</a>)</span>
  <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6" id="id7"
      role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>) or
    automatically using a “#”-prefixed label. This footnote has a
    label so it can be referred to from multiple places, both as a
    footnote reference (<a class="footnote-reference brackets" href="#label" id="id8" role="doc-noteref"><span
        class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>) and as a hyperlink reference
    (<a class="reference internal" href="#label">label</a>).</p>
</aside>
```

The suggested change to the markup is to generate something like (using only the first aside for this demo):

```
<aside class="footnote brackets" id="id6" role="note">
  <div class="footnote-label">
    <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
    <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  </div>
  <div class="footnote-content">
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </div>
</aside>
```


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

Re: [Docutils-develop] Docutils 0.19 Release

[Guenter M. <mi...@us...>]

On 2022-06-12, Adam Turner wrote:

>> maybe I am able to ship 0.19.0b1 on tuesday (tuesday is release day)

The first in a series of beta releases should be 
release level = "beta", serial = 0, release = True.
https://docutils.sourceforge.io/docs/dev/policies.html#docutils-version

The upcoming release would then have ``__version__ = '0.19b'`` (or "0.19.0b0")
rsp. ``__version_info__ = (0, 19, 0, 'beta', 0, True)``.

>> june 14 or waiting for 450 on 21.

> Brilliant, thanks Engelbert! I will wait for Günter's response on
> bugs#450, but probably the 21st is the best option, 

The last patch for the problem described in bug #450 is "minimally invasive"
but still an incompatible API change.
https://sourceforge.net/p/docutils/bugs/450/#937a/460a/81bc/e203/ecc3/c6b8

I am still unsure whether curing an announced incompatible API change
with an unannounced one is the right way.
I would prefer announcing in 0.19 and changing in 0.20 but can also live
with an unannounced fix, if this is considered the lesser evil.
(We still don't know, whether the fix would satisfy the OP.)


> with 0.19.0 final then released on 05/07.

We have done some large changes and some removals,
so maybe a longer beta period is appropriate...


Günter

[Docutils-develop] [docutils:bugs] Re: #450 0.18: New HTML markup for footnotes is difficult to stylise

[Adam T. <aa-...@us...>]

+1 on the attached patch.

My only thought would be using `<section class="footnote-list">`  for the outer container, but I'm not sure it is a good fit:

> The section element represents a generic section of a document or application. A section, in this context, is a thematic grouping of content, typically with a heading.

So the `aside.footnote-list > aside.footnotes.(brackets|superscript)` seems the best option. If we commit today, we may be able to make a beta release tomorrow even.

A


---

** [bugs:#450] 0.18: New HTML markup for footnotes is difficult to stylise**

**Status:** open
**Created:** Sun Jun 05, 2022 08:28 PM UTC by Pradyun Gedam
**Last Updated:** Mon Jun 13, 2022 08:08 PM UTC
**Owner:** nobody


Docutils 0.18 changed the HTML markup for footnotes/citations generated for rst documents. This changed markup is significantly more difficult to stylise with CSS, in ways that was possible with Docutils 0.17. This has negatively affected HTML themes for Sphinx, since the Sphinx 5 release allows using docutils 0.18, which is preferentially installed by `pip`.

The reasons for this boils down to an inconsistent number of elements inside each `aside`.

1. It's not possible use CSS grid layouts sanely, with this setup.
2. Even if you added multiple rules based on number of elements in the section, it's not possible to know what the 2nd element might be -- which balloons the complexity+size of the stylesheet, if it tries to accomodate for this.

It is theoretically possible to stylise this but it would be on the order of 100s of lines of CSS to get this right, compared to ~20 with 0.17.

Would it be possible to change this markup, to wrap the label & backrefs in a `div` and to wrap the content paragraphs in a separate `div` as well? This would make it possible to stylise this content in ways that were feasible with 0.17, with significantly less complexity in the stylesheets.

---

Sources:

```
[some content that references these footnotes]

.. [1] A footnote contains body elements, consistently indented by at
   least 3 spaces.

   This is the footnote's second paragraph.

.. [#label] Footnotes may be numbered, either manually (as in [1]_) or
   automatically using a "#"-prefixed label.  This footnote has a
   label so it can be referred to from multiple places, both as a
   footnote reference ([#label]_) and as a hyperlink reference
   (label_).

```

0.17 output HTML:

```
<dl class="footnote brackets">
  <dt class="label" id="id6"><span class="brackets">1</span><span class="fn-backref">(<a href="#id1">1</a>,<a
        href="#id7">2</a>)</span></dt>
  <dd>
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </dd>
  <dt class="label" id="label"><span class="brackets">2</span><span class="fn-backref">(<a href="#id3">1</a>,<a
        href="#id8">2</a>)</span></dt>
  <dd>
    <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6"
        id="id7">1</a>) or
      automatically using a “#”-prefixed label. This footnote has a
      label so it can be referred to from multiple places, both as a
      footnote reference (<a class="footnote-reference brackets" href="#label" id="id8">2</a>) and as a hyperlink
      reference
      (<a class="reference internal" href="#label">label</a>).</p>
  </dd>
</dl>
```

0.18 output HTML:

```
<aside class="footnote brackets" id="id6" role="note">
  <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  <p>A footnote contains body elements, consistently indented by at
    least 3 spaces.</p>
  <p>This is the footnote’s second paragraph.</p>
</aside>
<aside class="footnote brackets" id="label" role="note">
  <span class="label"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id3" role="doc-backlink">1</a>,<a href="#id8" role="doc-backlink">2</a>)</span>
  <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6" id="id7"
      role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>) or
    automatically using a “#”-prefixed label. This footnote has a
    label so it can be referred to from multiple places, both as a
    footnote reference (<a class="footnote-reference brackets" href="#label" id="id8" role="doc-noteref"><span
        class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>) and as a hyperlink reference
    (<a class="reference internal" href="#label">label</a>).</p>
</aside>
```

The suggested change to the markup is to generate something like (using only the first aside for this demo):

```
<aside class="footnote brackets" id="id6" role="note">
  <div class="footnote-label">
    <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
    <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  </div>
  <div class="footnote-content">
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </div>
</aside>
```


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #450 0.18: New HTML markup for footnotes is difficult to stylise

[Günter M. <mi...@us...>]

Updated patch based on r9070

- keep using `<aside>` for footnotes. Both, a group of footnotes (footnotes list) and  the  individual footnotes "could be considered separate from the content [...] around the aside element".
- Adapt tuftig.css, make use of the wrapper to simplify plain.css.
- Document change in HISTORY and RELEASE_NOTES.
- Use ARIA role "doc-footnote" instead of "note". This could also be done later, in a separate fix.


Attachments:

- [0001-Minimal-fix-for-bug-450-footnote-representation-in-HTML5.patch](https://sourceforge.net/p/docutils/bugs/_discuss/thread/af593fcd69/937a/460a/81bc/e203/ecc3/c6b8/attachment/0001-Minimal-fix-for-bug-450-footnote-representation-in-HTML5.patch) (31.5 kB; text/x-patch)


---

** [bugs:#450] 0.18: New HTML markup for footnotes is difficult to stylise**

**Status:** open
**Created:** Sun Jun 05, 2022 08:28 PM UTC by Pradyun Gedam
**Last Updated:** Sun Jun 12, 2022 08:15 PM UTC
**Owner:** nobody


Docutils 0.18 changed the HTML markup for footnotes/citations generated for rst documents. This changed markup is significantly more difficult to stylise with CSS, in ways that was possible with Docutils 0.17. This has negatively affected HTML themes for Sphinx, since the Sphinx 5 release allows using docutils 0.18, which is preferentially installed by `pip`.

The reasons for this boils down to an inconsistent number of elements inside each `aside`.

1. It's not possible use CSS grid layouts sanely, with this setup.
2. Even if you added multiple rules based on number of elements in the section, it's not possible to know what the 2nd element might be -- which balloons the complexity+size of the stylesheet, if it tries to accomodate for this.

It is theoretically possible to stylise this but it would be on the order of 100s of lines of CSS to get this right, compared to ~20 with 0.17.

Would it be possible to change this markup, to wrap the label & backrefs in a `div` and to wrap the content paragraphs in a separate `div` as well? This would make it possible to stylise this content in ways that were feasible with 0.17, with significantly less complexity in the stylesheets.

---

Sources:

```
[some content that references these footnotes]

.. [1] A footnote contains body elements, consistently indented by at
   least 3 spaces.

   This is the footnote's second paragraph.

.. [#label] Footnotes may be numbered, either manually (as in [1]_) or
   automatically using a "#"-prefixed label.  This footnote has a
   label so it can be referred to from multiple places, both as a
   footnote reference ([#label]_) and as a hyperlink reference
   (label_).

```

0.17 output HTML:

```
<dl class="footnote brackets">
  <dt class="label" id="id6"><span class="brackets">1</span><span class="fn-backref">(<a href="#id1">1</a>,<a
        href="#id7">2</a>)</span></dt>
  <dd>
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </dd>
  <dt class="label" id="label"><span class="brackets">2</span><span class="fn-backref">(<a href="#id3">1</a>,<a
        href="#id8">2</a>)</span></dt>
  <dd>
    <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6"
        id="id7">1</a>) or
      automatically using a “#”-prefixed label. This footnote has a
      label so it can be referred to from multiple places, both as a
      footnote reference (<a class="footnote-reference brackets" href="#label" id="id8">2</a>) and as a hyperlink
      reference
      (<a class="reference internal" href="#label">label</a>).</p>
  </dd>
</dl>
```

0.18 output HTML:

```
<aside class="footnote brackets" id="id6" role="note">
  <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  <p>A footnote contains body elements, consistently indented by at
    least 3 spaces.</p>
  <p>This is the footnote’s second paragraph.</p>
</aside>
<aside class="footnote brackets" id="label" role="note">
  <span class="label"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id3" role="doc-backlink">1</a>,<a href="#id8" role="doc-backlink">2</a>)</span>
  <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6" id="id7"
      role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>) or
    automatically using a “#”-prefixed label. This footnote has a
    label so it can be referred to from multiple places, both as a
    footnote reference (<a class="footnote-reference brackets" href="#label" id="id8" role="doc-noteref"><span
        class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>) and as a hyperlink reference
    (<a class="reference internal" href="#label">label</a>).</p>
</aside>
```

The suggested change to the markup is to generate something like (using only the first aside for this demo):

```
<aside class="footnote brackets" id="id6" role="note">
  <div class="footnote-label">
    <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
    <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  </div>
  <div class="footnote-content">
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </div>
</aside>
```


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] #450 0.18: New HTML markup for footnotes is difficult to stylise

[Alan <ai...@us...>]

Well ... I guess I would throw this back at you. How much support for technical reports and ebooks do the docutils developers wish to provide? I consider those outputs natural for docutils. And I'd love to be able to turn on an `epub` option and get `<aside epub:type="footnote" role="doc-footnote">` (assuming such pairing is still allowed).  Anyway, I find the `doc` roles particularly appropriate for the output of docutils.


---

** [bugs:#450] 0.18: New HTML markup for footnotes is difficult to stylise**

**Status:** open
**Created:** Sun Jun 05, 2022 08:28 PM UTC by Pradyun Gedam
**Last Updated:** Sat Jun 11, 2022 12:23 AM UTC
**Owner:** nobody


Docutils 0.18 changed the HTML markup for footnotes/citations generated for rst documents. This changed markup is significantly more difficult to stylise with CSS, in ways that was possible with Docutils 0.17. This has negatively affected HTML themes for Sphinx, since the Sphinx 5 release allows using docutils 0.18, which is preferentially installed by `pip`.

The reasons for this boils down to an inconsistent number of elements inside each `aside`.

1. It's not possible use CSS grid layouts sanely, with this setup.
2. Even if you added multiple rules based on number of elements in the section, it's not possible to know what the 2nd element might be -- which balloons the complexity+size of the stylesheet, if it tries to accomodate for this.

It is theoretically possible to stylise this but it would be on the order of 100s of lines of CSS to get this right, compared to ~20 with 0.17.

Would it be possible to change this markup, to wrap the label & backrefs in a `div` and to wrap the content paragraphs in a separate `div` as well? This would make it possible to stylise this content in ways that were feasible with 0.17, with significantly less complexity in the stylesheets.

---

Sources:

```
[some content that references these footnotes]

.. [1] A footnote contains body elements, consistently indented by at
   least 3 spaces.

   This is the footnote's second paragraph.

.. [#label] Footnotes may be numbered, either manually (as in [1]_) or
   automatically using a "#"-prefixed label.  This footnote has a
   label so it can be referred to from multiple places, both as a
   footnote reference ([#label]_) and as a hyperlink reference
   (label_).

```

0.17 output HTML:

```
<dl class="footnote brackets">
  <dt class="label" id="id6"><span class="brackets">1</span><span class="fn-backref">(<a href="#id1">1</a>,<a
        href="#id7">2</a>)</span></dt>
  <dd>
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </dd>
  <dt class="label" id="label"><span class="brackets">2</span><span class="fn-backref">(<a href="#id3">1</a>,<a
        href="#id8">2</a>)</span></dt>
  <dd>
    <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6"
        id="id7">1</a>) or
      automatically using a “#”-prefixed label. This footnote has a
      label so it can be referred to from multiple places, both as a
      footnote reference (<a class="footnote-reference brackets" href="#label" id="id8">2</a>) and as a hyperlink
      reference
      (<a class="reference internal" href="#label">label</a>).</p>
  </dd>
</dl>
```

0.18 output HTML:

```
<aside class="footnote brackets" id="id6" role="note">
  <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  <p>A footnote contains body elements, consistently indented by at
    least 3 spaces.</p>
  <p>This is the footnote’s second paragraph.</p>
</aside>
<aside class="footnote brackets" id="label" role="note">
  <span class="label"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></span>
  <span class="backrefs">(<a href="#id3" role="doc-backlink">1</a>,<a href="#id8" role="doc-backlink">2</a>)</span>
  <p>Footnotes may be numbered, either manually (as in <a class="footnote-reference brackets" href="#id6" id="id7"
      role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>) or
    automatically using a “#”-prefixed label. This footnote has a
    label so it can be referred to from multiple places, both as a
    footnote reference (<a class="footnote-reference brackets" href="#label" id="id8" role="doc-noteref"><span
        class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>) and as a hyperlink reference
    (<a class="reference internal" href="#label">label</a>).</p>
</aside>
```

The suggested change to the markup is to generate something like (using only the first aside for this demo):

```
<aside class="footnote brackets" id="id6" role="note">
  <div class="footnote-label">
    <span class="label"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></span>
    <span class="backrefs">(<a href="#id1" role="doc-backlink">1</a>,<a href="#id7" role="doc-backlink">2</a>)</span>
  </div>
  <div class="footnote-content">
    <p>A footnote contains body elements, consistently indented by at
      least 3 spaces.</p>
    <p>This is the footnote’s second paragraph.</p>
  </div>
</aside>
```


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #451 Deprecate PEP 263 coding slugs support

[Adam T. <aa-...@us...>]

Fair enough, I will put this on hold for now. [bugs:#450] is more important to resolve at the moment before 0.19.0b1 release.

A


---

** [bugs:#451] Deprecate PEP 263 coding slugs support**

**Status:** open
**Created:** Thu Jun 09, 2022 10:48 PM UTC by Adam  Turner
**Last Updated:** Sat Jun 11, 2022 01:13 AM UTC
**Owner:** nobody
**Attachments:**

- [0001-Deprecate-PEP-263-coding-slugs.patch](https://sourceforge.net/p/docutils/bugs/451/attachment/0001-Deprecate-PEP-263-coding-slugs.patch) (5.5 kB; application/octet-stream)


Python 3 uses utf-8 as the encoding for Python source files, there is no longer a compelling use-case for the support, which adds complexity to the IO implementation.

I propose deprecating support for removal in 1.0, but 2.0 might be a better option.

Support was added in [r4506].

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #451 Deprecate PEP 263 coding slugs support

[Günter M. <mi...@us...>]

> The underlying thrust of my argument is that this is very fragile --

True, this method only works with ASCII compatible encodings.
(This is one of the reasons why Docutils as well as PEP 263 complement it
with BOM mark recognition.)

...

> If the need arises [...], we would accept a feature request for
> `buildhtml.py` to have some enumeration of files and their input
> encodings.

IMO, it is more safe keep "source code encoding both visible and
changeable on a per-source file basis". [PEP 263]

Python3 still supports the encoding slug. 
I vote to keep this option as well.


---

** [bugs:#451] Deprecate PEP 263 coding slugs support**

**Status:** open
**Created:** Thu Jun 09, 2022 10:48 PM UTC by Adam  Turner
**Last Updated:** Sat Jun 11, 2022 01:13 AM UTC
**Owner:** nobody
**Attachments:**

- [0001-Deprecate-PEP-263-coding-slugs.patch](https://sourceforge.net/p/docutils/bugs/451/attachment/0001-Deprecate-PEP-263-coding-slugs.patch) (5.5 kB; application/octet-stream)


Python 3 uses utf-8 as the encoding for Python source files, there is no longer a compelling use-case for the support, which adds complexity to the IO implementation.

I propose deprecating support for removal in 1.0, but 2.0 might be a better option.

Support was added in [r4506].

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

Re: [Docutils-develop] Docutils 0.19 Release

[Adam T. <aat...@ou...>]

> maybe I am able to ship 0.19.0b1 on tuesday (tuesday is release day) june 14
> or waiting for 450 on 21.

Brilliant, thanks Engelbert! I will wait for Günter's response on bugs#450, but probably the 21st is the best option, with 0.19.0 final then released on 05/07.

> the switch to html5 with 2.0 IMHO is good ... i see no reason to rush it

OK, I have prepared a patch (attached). No-one has spoken against the change in principle as far as I've been able to tell, and 2.0 should be a good amount of headroom. Sphinx and Nikola are unaffected as they both already use HTML 5 by default.

A

Re: [Docutils-develop] Docutils 0.19 Release

[engelbert g. <eng...@gm...>]

Hello Adam,

yes I am for rolling 0.19 out

maybe I am able to ship 0.19.0b1 on tuesday (tuesday is release day) june 14
or waiting for 450 on 21.

two weeks later 0.19

the switch to html5 with 2.0 IMHO is good ... i see no reason to rush it

thanks and cheers
 e

On Sat, 11 Jun 2022 at 02:33, Adam Turner <aat...@ou...>
wrote:

> Dear Engelbert,
>
> [Günter]
> > The default "html" writer change is still open/undecided
> > (no problem in 0.19 if envisaged for 2.0).
>
> > This means we should be clear for release 0.19b.
>
> Are you in favour of making a 0.19.0b1 release? The only potential
> unresolved items are:
>
> * announcing a date for the switch to HTML5-by-default (Günter has
> abstained, I am in favour of announcing the date as 2.0)
> * bugs#450 [1]_
>
> On the latter, Günter and I have converged towards a single similar patch,
> so that will hopefully be resolved soon. On the former, I think you would
> be the casting vote!
>
> Please let me know your thoughts,
> Thanks,
> Adam
>
> _[1]: https://sourceforge.net/p/docutils/bugs/450/
>

[Docutils-develop] [docutils:bugs] Re: #451 Deprecate PEP 263 coding slugs support

[Adam T. <aa-...@us...>]

> I see still a reason to keep (and properly document) a way to specify the
> encoding of an rST source in the document itself.

The underlying thrust of my argument is that this is very fragile -- for any encoding that is not compatible with ASCII (e.g. UTF-16) the current coding slug test fails::

```pycon
>>> m = re.search(br"coding[:=]\s*([-\w.]+)", "coding: utf-16".encode("utf-16-le"))
>>> m is None
True
>>> m = re.search(br"coding[:=]\s*([-\w.]+)", "coding: latin-1".encode("latin-1"))
>>> m.group(1).decode("ascii")
'latin-1'
```

Similarly any in-document metadata would suffer the same fate -- Unicode codepoints (which make up `str` objects) cannot be assumed to have a correspondence to bytes on disk. Better to fail loudly than have silent data corruption.

> A collection of files, where one file for whatever reason must be in a different encoding. Compilation with "buildhtml.py".

If the need arises for this, we would accept a feature request for `buildhtml.py` to have some enumeration of files and their input encodings.

> Documents in an 8-bit or 16-bit encoding intended for compilation anywhere. Avoids shipping a separate configuration file.

Not sure I understand this one fully, but such a file would likely come with compilation instructions that included the input encoding.

Annecdotally, I looked through the ~70 results for the following search [1]_ on "grep.app" 
`coding[:=]( \t)*(([^u\W]|u[^t\W]|ut[^f\W]|utf-?[^8\W])[-\w.]+)`
(lookahead/lookbehinds aren't supported) and no file had a coding slug that occured in the first two lines. Whilst obviously only a fraction of extant reST files are indexed by that provider, if it was a pattern in common usage I would expect to see more than 0.

One of my longer-term goals is to simplify `docutils.io` quite a lot, as I think there is a lot of duplicated code that the current (Python 3) stdlib provides automatically for us. Making our file parsing more vanilla/standard is a step towards this larger goal, although I do believe this change stands alone on its merits.

A

_[1]: `https://grep.app/search?current=7&q=coding%5B%3A%3D%5D%28%20%5Ct%29%2A%28%28%5B%5Eu%5CW%5D%7Cu%5B%5Et%5CW%5D%7Cut%5B%5Ef%5CW%5D%7Cutf-%3F%5B%5E8%5CW%5D%29%5B-%5Cw.%5D%2B%29®exp=true&filter[lang][0]=reStructuredText`


---

** [bugs:#451] Deprecate PEP 263 coding slugs support**

**Status:** open
**Created:** Thu Jun 09, 2022 10:48 PM UTC by Adam  Turner
**Last Updated:** Thu Jun 09, 2022 10:48 PM UTC
**Owner:** nobody
**Attachments:**

- [0001-Deprecate-PEP-263-coding-slugs.patch](https://sourceforge.net/p/docutils/bugs/451/attachment/0001-Deprecate-PEP-263-coding-slugs.patch) (5.5 kB; application/octet-stream)


Python 3 uses utf-8 as the encoding for Python source files, there is no longer a compelling use-case for the support, which adds complexity to the IO implementation.

I propose deprecating support for removal in 1.0, but 2.0 might be a better option.

Support was added in [r4506].

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

Re: [Docutils-develop] Docutils 0.19 Release

[Adam T. <aat...@ou...>]

Dear Engelbert,

[Günter]
> The default "html" writer change is still open/undecided
> (no problem in 0.19 if envisaged for 2.0).

> This means we should be clear for release 0.19b.

Are you in favour of making a 0.19.0b1 release? The only potential unresolved items are:

* announcing a date for the switch to HTML5-by-default (Günter has abstained, I am in favour of announcing the date as 2.0)
* bugs#450 [1]_

On the latter, Günter and I have converged towards a single similar patch, so that will hopefully be resolved soon. On the former, I think you would be the casting vote!

Please let me know your thoughts,
Thanks,
Adam

_[1]: https://sourceforge.net/p/docutils/bugs/450/