docutils-develop — For developer discussions of the implementation.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Adam T. <aa-...@us...>]

These seem fine, I'd go ahead and apply them. For reference I opened https://github.com/sphinx-doc/sphinx/pull/10162 to update `sphinx.writers.html` to use `settings_default_overrides`.

A


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Thu Feb 03, 2022 02:59 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] #435 invalid references in "problematic" nodes with report_level=4

[Günter M. <mi...@us...>]

- **status**: open --> open-fixed
- **Comment**:

The patch was implemented in [r8997]. 
With `report_level== 4`, inline markup that leads to an error is ignored and the raw source is inserted into the document as Text node.
Thank you for the report



---

** [bugs:#435] invalid references in "problematic" nodes with report_level=4**

**Status:** open-fixed
**Created:** Tue Nov 16, 2021 04:50 PM UTC by Dmitry Shachnev
**Last Updated:** Sat Dec 04, 2021 11:22 AM UTC
**Owner:** nobody


I don’t know if you will consider it a bug, but I decided to report it nevertheless.

When input reStructuredText has errors and you silence them with report_level=4, the following HTML will be generated:
```python
>>> from docutils.core import publish_parts
>>> parts = publish_parts('`', settings_overrides={'report_level': 4}, writer_name='html5')
>>> print(parts['html_body'])
<main>
<p><a href="#system-message-1"><span class="problematic" id="problematic-1">`</span></a></p>
</main>
```

The link is pointing to `#system-message-1`, however there is no element with `id="system-message-id"` because the warning itself was not generated.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Günter M. <mi...@us...>]

I am sorry for the delay and the additional work. The change turned out to be far more complicated than expected when I first proposed it.
In order to have informed choices, I had to study the current state in detail first. (See [r8996] for a by-product.)
I propose to start with clean up and deprecation steps to smoothe the actual OptionParser -> ArgumentParser change. (A proposal for the two first steps is attached.)



Attachments:

- [0001-Small-fixes-mostly-documentation-Test-more-list-settings.patch](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/45c6/9106/2d9c/5575/attachment/0001-Small-fixes-mostly-documentation-Test-more-list-settings.patch) (8.2 kB; text/x-patch)
- [0002-Clean-up-docutilsfrontend-before-the-switch-to-arparse.patch](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/45c6/9106/2d9c/5575/attachment/0002-Clean-up-docutilsfrontend-before-the-switch-to-arparse.patch) (10.5 kB; text/x-patch)


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Thu Feb 03, 2022 02:45 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] #441 Move from "optparse" to "argparse".

[Günter M. <mi...@us...>]

First draft on the OptionParser/ConfigParser separation:
Clear separation of duties for OptionParser and ConfigParser.

The ConfigParser:

* Reads and parses the sequence of configuration files passed to its
  `read()` method.
* Stores the extracted setting values as stings.
* Collects settings from "active sections" in an "ACTIVE" section.

The OptionParser:

* Validates the settings and converts the values to the expected data type.
* Eventually updates the settings from command line arguments.

TODO: Validate configuration file settings in the ConfigParser?

+1 Collecting "appending_settings" becomes easier.
 -1 Requires inheriting configparser.RawConfigParser to allow storing  values other than strings. 
       (Alternative: collect in a ``dict`` instance instead of ACTIVE section.)
 -1 Requires knowledge of all validation functions and converters in the ConfigParser.
 
 The attached files provide a working implementation and minimal test script for this approach.


Attachments:

- [cf_parser.py](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/6a1c/attachment/cf_parser.py) (7.0 kB; text/x-python)
- [config_list.txt](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/6a1c/attachment/config_list.txt) (342 Bytes; text/plain)
- [config_list_2.txt](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/6a1c/attachment/config_list_2.txt) (359 Bytes; text/plain)
- [test.conf](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/6a1c/attachment/test.conf) (2.2 kB; application/octet-stream)
- [test2.conf](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/6a1c/attachment/test2.conf) (281 Bytes; application/octet-stream)
- [test3.conf](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/6a1c/attachment/test3.conf) (190 Bytes; application/octet-stream)
- [test_cf_parser.py](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/6a1c/attachment/test_cf_parser.py) (8.8 kB; text/x-python)


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Mon Jan 31, 2022 11:28 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:feature-requests] #90 Create a warning instead of an error for a missing "include" file?

[Günter M. <mi...@us...>]

---

** [feature-requests:#90] Create a warning instead of an error for a missing "include" file?**

**Status:** open
**Group:** Default
**Created:** Tue Feb 01, 2022 09:25 AM UTC by Günter Milde
**Last Updated:** Tue Feb 01, 2022 09:25 AM UTC
**Owner:** nobody


The Python Developers Guide documents the Sphinx directive "literalinclude" and adds in a footnote
> There is a standard `include` directive, but it raises errors if the file is not found. This one only emits a warning.
> -- https://devguide.python.org/documenting/#id5

Should Docutils follow Sphinx and let the ["include" directive](https://docutils.sourceforge.io/docs/ref/rst/directives.html#including-an-external-document-fragment) only warn?

* No, an error is the right [report level](https://docutils.sourceforge.io/docs/user/config.html#report-level).
* For literal inclusions (options "literal", "code", "raw", "parser"):  they have only local effect.
* For all inclusions: an included reST file may have effects later in the document (defining a role or substitution, setting the default role, establishing a new section level, ...).

It boils down to an assesment of the "seriosity" of a missing include file:
Both, *warning* and *error* indicate a  problem but do not normally  [halt](https://docutils.sourceforge.io/docs/user/config.html#halt-level) processing.
A *warning* is easier to silence (other *errors* will still pop up).




---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/feature-requests/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/feature-requests/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Adam T. <aa-...@us...>]

Perfect, thanks!
A


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Mon Jan 31, 2022 10:35 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Günter M. <mi...@us...>]

On 2022-01-31, Adam Turner wrote:

> From the comment below "Use of frontend.OptionParser will drop
> significantly with just one helper function" are you happy to keep
> `frontend.get_default_settings()`?

I am fine with this one. It needs a docstring and a mentioning in the module
docstring, though.

Thanks, Günter Milde


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Mon Jan 31, 2022 10:35 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Günter M. <mi...@us...>]

On 2022-01-25, Adam Turner wrote:

>> I am just working on a way to disentangle frontend.OptionParser and frontend.ConfigParser but this is a topic for bug:#441

> Sounds good -- happy to review a proposed design. 

My major sticking point is the cross-referencing: OptionParser uses a
ConfigParser instance and passes "self" to config_parser.read() because
ConfigParser requires an instance of OptionParser!

The reason is that some settings have side-effects (overwriting other
settings) that need to be resolved for each config file before reading the
next to keep the settings priority as expected. (cf. the attached
documentation)

My idea is to scale back `frontend.ConfigParser` to provide backwards
compatibility and the custom `optionxform()` while moving the
setting_validation() to `frontend.OptionParser`.

The `OptionParser` would then loop over the config_files,
read them one-by-one with `config_parser(read(file))`, 
select settings from the "active" config file sections, and
"validate" them (including the overwriting of affected settings).

If you agree that this is a step in the right direction, I would try to
prepare a patch.

> I'd propose making the `--config` flag simply store the path to the
> file, and then add it to the list of config files to be read.

Mind, that the list of config files to read is processed before 
parsing the command line. 
An option parser based on "argparse" can use the `parse_known_args()` method
to pre-parse the command line, append additional config files and proceed.
...

> Ideally, we would construct a single `ArgumentParser` object and add
> arguments "normally" through `parser.add_argument`. 
> The problem here is  that subclasses can filter settings_spec (through  filter_settings_spec). 
...
> if settings_spec tuples were treated as immutable, then things would be much simpler. 

Actually, both `SettingsSpec` instances and `SettingsSpec.settings_spec`
tuples do not "mutate" in Docutils. 
`filter_settings_spec()` is only used in *creating* a settings spec tuple 
from the settings specification of a base or "sister" class in the class definitions of the "xetex", "html4" and "html5" writers.
Other components ('pep_html", "s5_html")  overwrite just some default values with `SettingsSpec.setting_default_overrides` and keep the  parent's  `setting_spec`.)

The Sphix "html" writer loops over the base class' settings_spec tuple
 in order to set a default value in writers/html.py:59
https://github.com/sphinx-doc/sphinx/blob/4.x/sphinx/writers/html.py
This should be changed (before retiring the old format)
to use `settings_default_overrides` instead.

> We could also move to a model (a la stdlib `dataclasses` where we
> simply specify the setting name, type and default within the Component
> itself.

This sounds like what I have in mind with a new data format to replace the
tuple `SettingsSpec.settings_spec` (mind, that `Component` inherits
`SettingsSpec`).

> The front-end tools could then use that information (or a
> mapping of settings to command line options or similar) to create the
> CLI arguments. 

I would keep the processing of settings specifications in the "frontend"
module with front-ends deploying them via the `core.Publisher` class.

> I think this would be cleanest, but this cannot be done overnight.

But, maybe we should discuss more about a clean end state before
(or in parallel) with a step-by step review of the optparse-argparse switch.



Attachments:

- [runtime-settings-detailled.txt](https://sourceforge.net/p/docutils/bugs/_discuss/thread/c76f9a2618/18f5/7f0c/attachment/runtime-settings-detailled.txt) (13.2 kB; text/plain)


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Mon Jan 31, 2022 09:43 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Adam T. <aa-...@us...>]

> For concrete use-cases where this interface is insufficient, we can work on enhancement and improvement, but I don't want to start a parallel, undocumented framework of "access functions".

Fair enough. From the comment below "Use of frontend.OptionParser will drop significantly with just one helper function" are you happy to keep `frontend.get_default_settings()`?

I'll review the patch you attached, thanks!

A


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Mon Jan 31, 2022 09:28 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Günter M. <mi...@us...>]

Sorry for the long delay. Here my update on patch 1/6. 

On 2022-01-20, Adam Turner wrote:

> The functions are of most use to downstream developers, I'd argue. It
> provides a higher level abstraction which means you don't need to know
> about the internals of how `OptionParser` &c work.

You don't need to know these details, when using the intended entry points:

* the framework provided in `docutils.core` for front-end tools and
  applications deploying docutils

* the `readers.Reader`, `parsers.Parser`, `writers.Writer`, and
  `transforms.Transforms` base classes for drop-in components.
  
For concrete use-cases where this interface is insufficient, we can work on
enhancement and improvement, but I don't want to start a parallel,
undocumented framework of "access functions".

> ... I was trying to move to a point where the `OptionParser` and
> `ConfigParser` classes are only used in `docutils.frontend` --
> providing a cleaner seperation of concerns and making any future
> refactorings easier.

The `frontend.ConfigParser` is only used in `frontend.OptionParser` already.
Use of `frontend.OptionParser` will drop significantly with just one helper
function: `frontend.get_default_settings()`. Remaining cases can be handled
later (before deprecating the old OptionParser).

...

> I think that is probably as minimal as it gets -- it was difficult
> enough to rewrite everything keeping 100% backwards compatability --
> and I can't make any assumptions about which names are "safe" to remove

We can check for usage in Sphinx and myst, this gives at least an indication.

I'll attach my proposal for a pre-conversion cleanup.



---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Sat Jan 29, 2022 11:29 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] Re: #190 Move to native `unittest`

[Adam T. <aa-...@us...>]

(sorry, misclicked!)

All tests pass on each commit, and they should be fairly easy to re-order if need be.

A


---

** [patches:#190] Move to native `unittest`**

**Status:** open
**Group:** None
**Created:** Thu Jan 20, 2022 01:10 AM UTC by Adam  Turner
**Last Updated:** Mon Jan 31, 2022 06:37 PM UTC
**Owner:** nobody


From FR81 https://sourceforge.net/p/docutils/feature-requests/81/#fc55/3d8d by Günter Milde

> I had a quick look and the work looks promising, so yes, I would like to
> give it a try.
> 
> What is the effect on the test time and on the output if there are, e.g.,
> differences between output and expected in functional tests?
> 
> To avoid wasted work (and the sunken-cost fallacy), I propose to split
> the project into smaller changesets (of 1 to ca. 7 commits) which we can
> discuss and adapt before pushing to origin/master and then move to the
> next step.
> I expect some rounds of discussion per step, so there is no
> need to invest heavily in polishing in the first round.
> A link to a patch that I can apply with git am -3 to a local branch
> would be fine.
> The discussion could move to docutils-develop or a new ticket,
> whatever you prefer.

This is the proposed tracking issue for moving to native unittest

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] #190 Move to native `unittest`

[Adam T. <aa-...@us...>]

> I propose to split the project into smaller changesets 

The linked PR now contains 3 commits -- the first removes `quicktest.py`, the second removes path hacking in `__init.py`, and the third is an update to the functional tests.

Hopefully this is a good place to start, I can work on the next bits whilst the changes here are reviewed.

https://github.com/AA-Turner/docutils/pull/9 // https://github.com/AA-Turner/docutils/pull/9.patch


---

** [patches:#190] Move to native `unittest`**

**Status:** open
**Group:** None
**Created:** Thu Jan 20, 2022 01:10 AM UTC by Adam  Turner
**Last Updated:** Thu Jan 20, 2022 01:11 AM UTC
**Owner:** nobody


From FR81 https://sourceforge.net/p/docutils/feature-requests/81/#fc55/3d8d by Günter Milde

> I had a quick look and the work looks promising, so yes, I would like to
> give it a try.
> 
> What is the effect on the test time and on the output if there are, e.g.,
> differences between output and expected in functional tests?
> 
> To avoid wasted work (and the sunken-cost fallacy), I propose to split
> the project into smaller changesets (of 1 to ca. 7 commits) which we can
> discuss and adapt before pushing to origin/master and then move to the
> next step.
> I expect some rounds of discussion per step, so there is no
> need to invest heavily in polishing in the first round.
> A link to a patch that I can apply with git am -3 to a local branch
> would be fine.
> The discussion could move to docutils-develop or a new ticket,
> whatever you prefer.

This is the proposed tracking issue for moving to native unittest

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Adam T. <aa-...@us...>]

Re-re-rebased. 

It is a not-insignficant amount of work each time to rebase the changes, so please may I ask for a review, or to know what is blocking a review? ( @milde @grubert )

I would like to progress these changes -- I've been holding off on other potential suggestions so as not to have merge conflicts with myself.

A


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Thu Jan 27, 2022 05:43 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:feature-requests] #89 Public API, versioning, and deprecation

[engelbert g. <gr...@us...>]

my2c

* the "principle of least surprise" results in semantic versioning 
* shouldn't we mention css/html-dom and tex-macros in api


---

** [feature-requests:#89] Public API, versioning, and deprecation**

**Status:** open
**Group:** Default
**Created:** Sun Jan 16, 2022 01:39 AM UTC by Adam  Turner
**Last Updated:** Tue Jan 25, 2022 07:36 PM UTC
**Owner:** nobody


As requested by @milde in https://sourceforge.net/p/docutils/bugs/441/#7043/cdb8/8742/6e7f I'm opening this issue to allow for discussion on Docutils' public API, versioning policy, and deprecation.

This also relates to FR 87 on type annotations. 

From Günter, 

> The idea is to reconcile the reality (Docutils is used as if it were mature) and the version number (<1) once the API sufficiently defined a deprecation policy is agreed.

The only text I can find on library versioning is at https://docutils.sourceforge.io/docs/dev/policies.html#version-identification, excerpt below:

> **Major releases** (x.0, e.g. 1.0) will be rare, and will
represent major changes in API, functionality, or commitment.  The
major number will be bumped to 1 when the project is
feature-complete, and may be incremented later if there is a major
change in the design or API.  When Docutils reaches version 1.0,
the major APIs will be considered frozen.
For details, see the `backwards compatibility policy`_.

> Releases that change the minor number (x.y, e.g. 0.5) will be
**feature releases**; new features from the `Docutils core`_ will
be included.

> Releases that change the micro number (x.y.z, e.g. 0.4.1) will be
**bug-fix releases**.  No new features will be introduced in these
releases; only bug fixes will be included.

The proposed backwards compatability policy reads:

> Docutils' backwards compatibility policy follows the rules for Python in
PEP 387. ... The scope of the public API is laid out at the start of the backwards
  compatibility rules

I propose two modifications to the policies, which will make future changes to Docutils easier to review and reason about, for project members as well as outside contributors. 

Firstly, I propose adopting a formal versioning "system" such as Sematic Versioning ([SemVer](https://semver.org/)) or Calendar Versioning ([CalVer](https://calver.org/)). 

Semantic versioning is nearly identical to the current versioning policy, and has the benefit of being a known quantity, reducing misunderstandings. A potential phrasing would be:

**"Docutils follows SemVer. All changes must also follow the backwards compatability policy."**

What this does change is that the API is never considered complete, as in the original phrasing. Docutils [isn't slated for inclusion](https://www.python.org/dev/peps/pep-0258/#rejection-notice) in the standard library any more, and there will always be potential improvements and changes -- new parsers, new node types, changes in the HTML specification, et cetera. 

The 1.0 release then does not need to be a "big deal", and future breaking changes can go to 2.0, 3.0, etc -- setuptools is now on `60.5.0`!

Semantic versioning does have some drawbacks (https://snarky.ca/why-i-dont-like-semver/), so projects like pip have adopted calendar based versioning -- the major version is the year (22), and the minor version is either the current month or an increasing number.  This relies on strong documentation and changelogs, so that when users upgrade it is obvious what changed (the idea being that every change breaks someone, so there is no contract that version numbers within a certain range mean no breakage). Luckily, Docutils has a good culture around changelogs and histories etc, so this would be easy to adopt.

______


I also suggest enumerating all modules, classes, and functions that form the public API. The current backwards compatability policy references PEP 387, which is specifically for the Python project itself. Checking through the documentation on every change to identify if a name is public or private is error prone (we are all human, and can miss things with no malintent) and time consuming.

At the very least I would suggest, `docutils.nodes`, the reader/writer/parser aliases, `docutils.core`,  and the front end tools. (I'm happy to write out the full list if you give me modules/etc that should be part of the public API). It would also be useful to identify what parts of Docutils large downstream consumers use, and either create higher level abstractions or mark those as public API. (I would suggest Sphinx and MyST-parser). 

It is also the [established practice](https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles) to mark names as private by prefixing them with an underscore. I suggest adopting this, as it gives strong guidance to downstream library authors what Docutils considers private and public. Making a private name public is far easier than going through a deprecation cycle for the reverse.

I'd suggest adding the ability to have exceptions to the policy, with removal after one minor version. My full suggested text is:

**"Removal or significant alteration of any members of the public API will only take place after the behaviour has been marked as deprecated for two minor releases. Certain changes may have a shorter deprecation period of one minor release. This requires at least two project members to be in favour of doing so, and no members against.**

**Changes that may affect end-users (e.g. by requiring changes to the configuration file or potentially breaking custom style sheets) should be announced with a FutureWarning."**

This post is intentionally opinionated so as to provide something to talk over / edit -- I'm not massively attached to any one thing!

A

----
for reference, amongst others, I took inspiration from:
https://setuptools.pypa.io/en/latest/development/releases.html
https://pip.pypa.io/en/latest/development/release-process/
https://numpy.org/neps/nep-0023-backwards-compatibility.html



---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/feature-requests/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/feature-requests/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:feature-requests] Re: #58 Migration Docutils from SourceForge to Github

[Chris S. <chr...@us...>]

Again you haven’t supplied any reference to support your claims.

> you have not given a single reason why not GitLab

You wouldn’t be able to have integrated referencing (in issues and PRs) between docutils and sphinx, readthedocs, myst, and the majority of the python ecosystem that are on GitHub. Also first-class integration with VS Code, the most used development tool,...

Anyhow I’m not going to continue a flame war. I just wanted to make sure that your views wasn’t the only one represented here.

Cheers,
Chris

On 23 Jan 2022, at 12:49, Jeffrey C. Jacobs <tim...@st...> wrote:




On Fri, Jan 21, 2022 at 02:39 Juan Luis Cano Rodríguez <jua...@re...<mailto:jua...@re...>> wrote:
GitHub is an American company, and GitLab is hosted on Google Cloud Platform, an American company, and SourceForge is owned by Slashdot Media, an American company. American companies are subject to American laws, in particular regarding trade restrictions and sanctions to other countries[1].

Nonetheless it is blocked by US companies and federal agencies and the reason stated is ties to China. I’m thinking fighting the US government on its opinions of this is a battle of logic you’re not going to win. GitHub is a barrier, GitLab is not. I vote GitLab.

Also, defending GitHub you have not given a single reason why not GitLab, so, if they are equivalent, they why not GitLab?

Personally, I don’t like leaving the office and reading GitHub on my phone just to do something.

Jeffrey
--
"Scan not a friend with a microscopic glass; you know his faults so let his foibles pass." -- Victorian Proverb, likely Sir Frank Crisp, 1st Baronet of Bungay

"Dans le silence on ne sait pas, il faut continuer, je ne peux pas continuer, je vais continuer." -- L'Innommable, Samuel Beckett

"and thus the whirligig of time brings in his revenges" -- The Clown
  from 'Twelfth Night' v, , Act 5, Scene 1 by William Shakespeare

"Le mieux est l'ennemi du bien." -- François-Marie Arouet (Voltare), La Bégueule (1772)

~,-;`    The TimeHorse



---

** [feature-requests:#58] Migration Docutils from SourceForge to Github**

**Status:** open
**Group:** Default
**Created:** Fri Feb 16, 2018 03:23 PM UTC by Yves Chevallier
**Last Updated:** Sat Jan 22, 2022 07:45 AM UTC
**Owner:** nobody


Sourceforge is not really user friendly to report issues, propose pull-request and contribute to the project. I would like to know if it is possible to migrate Docutils to GitHub. 


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/feature-requests/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/feature-requests/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] Re: #189 Remove 'prest'

[Günter M. <mi...@us...>]

On 2022-01-28, Adam Turner via Docutils-develop wrote in gmane.text.docutils.devel:

> It's been a fortnight with no response from Mark / others -- just
> checking here if you'd want to wait any further? 

I'd wait for 2 more weeks. Remember, Docutils development is slow...


---

** [patches:#189] Remove 'prest'**

**Status:** pending-remind
**Group:** None
**Created:** Thu Jan 06, 2022 02:23 PM UTC by Adam  Turner
**Last Updated:** Fri Jan 28, 2022 06:12 PM UTC
**Owner:** nobody


xref sourceforge.net/p/docutils/mailman/message/36748625 -- an earlier attempt.

It has now been 11 years (08/12/2010), and no changes. The code will remain in the souce control history should anyone want to revive it.

The first commit removes all the files under /prest and the second removes a final reference in the docs.

This code was quite confusing to me when I was first looking throuh the Docutils source tree - I hestitated to contribute for a while as I thought I might need to know Perl or need to translate any contributions. 

As a current contributor, when searching through the full codebase, there are spurious results from prest that are annoying -- not insurmountable, but a common annoyance, and I think dropping this would solve both issues of clarity (Docutils is purely a Python project) and developer experience.

A

https://github.com/AA-Turner/docutils/pull/5 // https://github.com/AA-Turner/docutils/pull/5.patch


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] #189 Remove 'prest'

[Adam T. <aa-...@us...>]

It's been a fortnight with no response from Mark / others -- just checking here if you'd want to wait any further? 

(I saw the status changed to "pending-remind", but I'm not sure when the "remind" part kicks in!)

A


---

** [patches:#189] Remove 'prest'**

**Status:** pending-remind
**Group:** None
**Created:** Thu Jan 06, 2022 02:23 PM UTC by Adam  Turner
**Last Updated:** Fri Jan 14, 2022 07:07 PM UTC
**Owner:** nobody


xref sourceforge.net/p/docutils/mailman/message/36748625 -- an earlier attempt.

It has now been 11 years (08/12/2010), and no changes. The code will remain in the souce control history should anyone want to revive it.

The first commit removes all the files under /prest and the second removes a final reference in the docs.

This code was quite confusing to me when I was first looking throuh the Docutils source tree - I hestitated to contribute for a while as I thought I might need to know Perl or need to translate any contributions. 

As a current contributor, when searching through the full codebase, there are spurious results from prest that are annoying -- not insurmountable, but a common annoyance, and I think dropping this would solve both issues of clarity (Docutils is purely a Python project) and developer experience.

A

https://github.com/AA-Turner/docutils/pull/5 // https://github.com/AA-Turner/docutils/pull/5.patch


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] Re: #441 Move from "optparse" to "argparse".

[Adam T. <aa-...@us...>]

Re-rebased
A


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Thu Jan 27, 2022 12:11 AM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] Re: #192 Codebase modernisation

[Adam T. <aa-...@us...>]

> avoid hard to read "parenthesis clusters"

Thanks! I tried to be as minimal as possible in the patch, agree with this change though.

A


---

** [patches:#192] Codebase modernisation**

**Status:** open
**Group:** None
**Created:** Wed Jan 26, 2022 12:37 AM UTC by Adam  Turner
**Last Updated:** Thu Jan 27, 2022 05:15 PM UTC
**Owner:** nobody


This follows on from the Python 2.x cleanup work.

The total diff looks quite large, but the marjority of these are mechanical search / replace changes -- viewing by commit or in groups of commits may be best. 

https://github.com/AA-Turner/docutils/pull/12 // https://github.com/AA-Turner/docutils/pull/12.patch

I kept things seperate so that it would be easier to extract and rebase changes that you don't want to merge (hopefully there aren't any of these!).

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] Re: #192 Codebase modernisation

[Adam T. <aa-...@us...>]

> Perhaps just remove?

Sorry, I meant remove the change I proposed!

A


---

** [patches:#192] Codebase modernisation**

**Status:** open
**Group:** None
**Created:** Wed Jan 26, 2022 12:37 AM UTC by Adam  Turner
**Last Updated:** Wed Jan 26, 2022 11:35 PM UTC
**Owner:** nobody


This follows on from the Python 2.x cleanup work.

The total diff looks quite large, but the marjority of these are mechanical search / replace changes -- viewing by commit or in groups of commits may be best. 

https://github.com/AA-Turner/docutils/pull/12 // https://github.com/AA-Turner/docutils/pull/12.patch

I kept things seperate so that it would be easier to extract and rebase changes that you don't want to merge (hopefully there aren't any of these!).

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] Re: #192 Codebase modernisation

[Günter M. <mi...@us...>]

On 2022-01-26, Adam Turner wrote:

>> error catching: here we should check for the most appropriate
>> (sub)class to use,

Done in [r8989].

> (I don't have recommonmark set up for docutils on my laptop, which is
> probably why I missed that tests fail -- sorry!) Perhaps just remove?

The recommonmark test are skipped unless a matching version is installed.
They test the wrapper that normalises/fixes the doctree returned from
recommonmark. IMO, we should keep the test as long as we are supporting
"recommonmark" (which currently is the only compatible Markdown parser
packed in Debian/stable).


---

** [patches:#192] Codebase modernisation**

**Status:** open
**Group:** None
**Created:** Wed Jan 26, 2022 12:37 AM UTC by Adam  Turner
**Last Updated:** Wed Jan 26, 2022 11:35 PM UTC
**Owner:** nobody


This follows on from the Python 2.x cleanup work.

The total diff looks quite large, but the marjority of these are mechanical search / replace changes -- viewing by commit or in groups of commits may be best. 

https://github.com/AA-Turner/docutils/pull/12 // https://github.com/AA-Turner/docutils/pull/12.patch

I kept things seperate so that it would be easier to extract and rebase changes that you don't want to merge (hopefully there aren't any of these!).

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] Re: #192 Codebase modernisation

[Günter M. <mi...@us...>]

On 2022-01-26, Adam Turner wrote:

> I noticed you removed some extra `u` characters and some
> combining marks compared to my original patch.
> Patch to fix (along with some other things)

Thank you for the fixup.

> The some other things are some str.join generator expression changes,

Here, I did also some refactoring to avoid hard to read 
"parenthesis clusters" like
~~~
...
+  % (self.name, ' < '.join((path, *(pth for pth, opt
+                           in reversed(include_log))))))
~~~

> removing one more set of parens

In the context of keyword arguments, I prefer the parentheses over a
space because of operator precedence
cf https://www.python.org/dev/peps/pep-0008/#other-recommendations

Thanks again.


---

** [patches:#192] Codebase modernisation**

**Status:** open
**Group:** None
**Created:** Wed Jan 26, 2022 12:37 AM UTC by Adam  Turner
**Last Updated:** Wed Jan 26, 2022 11:35 PM UTC
**Owner:** nobody


This follows on from the Python 2.x cleanup work.

The total diff looks quite large, but the marjority of these are mechanical search / replace changes -- viewing by commit or in groups of commits may be best. 

https://github.com/AA-Turner/docutils/pull/12 // https://github.com/AA-Turner/docutils/pull/12.patch

I kept things seperate so that it would be easier to extract and rebase changes that you don't want to merge (hopefully there aren't any of these!).

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:bugs] #441 Move from "optparse" to "argparse".

[Adam T. <aa-...@us...>]

I just rebased the branch onto the latest master
https://github.com/AA-Turner/docutils/pull/8

A


---

** [bugs:#441] Move from "optparse" to "argparse".**

**Status:** open
**Created:** Thu Jan 06, 2022 03:02 PM UTC by Günter Milde
**Last Updated:** Tue Jan 25, 2022 11:50 PM UTC
**Owner:** Günter Milde


The optparse documentation says:
> Deprecated since version 3.2: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.

We are currently suppressing related deprecation warnings in the test suite.
After raising the Python dependency to >=3.7, now may be the right time to make the move.


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/bugs/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] Re: #192 Codebase modernisation

[Adam T. <aa-...@us...>]

(The some other things are some str.join generator expression changes, removing one more set of parens, and the copy change above. In the str.join part, there is a redundant code block removed -- `nested_tags` is never used)


---

** [patches:#192] Codebase modernisation**

**Status:** open
**Group:** None
**Created:** Wed Jan 26, 2022 12:37 AM UTC by Adam  Turner
**Last Updated:** Wed Jan 26, 2022 11:29 PM UTC
**Owner:** nobody


This follows on from the Python 2.x cleanup work.

The total diff looks quite large, but the marjority of these are mechanical search / replace changes -- viewing by commit or in groups of commits may be best. 

https://github.com/AA-Turner/docutils/pull/12 // https://github.com/AA-Turner/docutils/pull/12.patch

I kept things seperate so that it would be easier to extract and rebase changes that you don't want to merge (hopefully there aren't any of these!).

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[Docutils-develop] [docutils:patches] #192 Codebase modernisation

[Adam T. <aa-...@us...>]

@milde I noticed you removed some extra `u` characters and some combining marks compared to my original patch.

Patch to fix (along with some other things): https://github.com/AA-Turner/docutils/pull/13 // https://github.com/AA-Turner/docutils/pull/13.patch

A


---

** [patches:#192] Codebase modernisation**

**Status:** open
**Group:** None
**Created:** Wed Jan 26, 2022 12:37 AM UTC by Adam  Turner
**Last Updated:** Wed Jan 26, 2022 10:42 PM UTC
**Owner:** nobody


This follows on from the Python 2.x cleanup work.

The total diff looks quite large, but the marjority of these are mechanical search / replace changes -- viewing by commit or in groups of commits may be best. 

https://github.com/AA-Turner/docutils/pull/12 // https://github.com/AA-Turner/docutils/pull/12.patch

I kept things seperate so that it would be easier to extract and rebase changes that you don't want to merge (hopefully there aren't any of these!).

A


---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/patches/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/patches/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.