[Docutils-develop] [docutils:feature-requests] Re: #89 Public API, versioning, and deprecation

[Adam T. <aa-...@us...>]

> I don't think we need to include the overview of docutils use(r)s in the
"Policies" document.

Removed

> Do you mean the provided output templates 

Yes, I updated to include the templates and stylesheets

> Why `__public__` and not `__all__`?

`__all__` has special meaning to the interpreter for `from module import *`. Perhaps it doesn't matter, but I didn't want to conflate the two usages.

> Do you want this ... in addition to [or] instead of SemVer

I'd prefer to adopt SemVer, but I dropped that from the proposal for later conversation. Would you be happy with adopting SemVer and updating the language in this section to "should" rather than "must"?

________

Fair enough, perhaps "DEP" is a reasonable way forwards.

A


---

** [feature-requests:#89] Public API, versioning, and deprecation**

**Status:** open
**Group:** Default
**Created:** Sun Jan 16, 2022 01:39 AM UTC by Adam  Turner
**Last Updated:** Wed Feb 16, 2022 02:32 AM UTC
**Owner:** nobody


As requested by @milde in https://sourceforge.net/p/docutils/bugs/441/#7043/cdb8/8742/6e7f I'm opening this issue to allow for discussion on Docutils' public API, versioning policy, and deprecation.

This also relates to FR 87 on type annotations. 

From Günter, 

> The idea is to reconcile the reality (Docutils is used as if it were mature) and the version number (<1) once the API sufficiently defined a deprecation policy is agreed.

The only text I can find on library versioning is at https://docutils.sourceforge.io/docs/dev/policies.html#version-identification, excerpt below:

> **Major releases** (x.0, e.g. 1.0) will be rare, and will
represent major changes in API, functionality, or commitment.  The
major number will be bumped to 1 when the project is
feature-complete, and may be incremented later if there is a major
change in the design or API.  When Docutils reaches version 1.0,
the major APIs will be considered frozen.
For details, see the `backwards compatibility policy`_.

> Releases that change the minor number (x.y, e.g. 0.5) will be
**feature releases**; new features from the `Docutils core`_ will
be included.

> Releases that change the micro number (x.y.z, e.g. 0.4.1) will be
**bug-fix releases**.  No new features will be introduced in these
releases; only bug fixes will be included.

The proposed backwards compatability policy reads:

> Docutils' backwards compatibility policy follows the rules for Python in
PEP 387. ... The scope of the public API is laid out at the start of the backwards
  compatibility rules

I propose two modifications to the policies, which will make future changes to Docutils easier to review and reason about, for project members as well as outside contributors. 

Firstly, I propose adopting a formal versioning "system" such as Sematic Versioning ([SemVer](https://semver.org/)) or Calendar Versioning ([CalVer](https://calver.org/)). 

Semantic versioning is nearly identical to the current versioning policy, and has the benefit of being a known quantity, reducing misunderstandings. A potential phrasing would be:

**"Docutils follows SemVer. All changes must also follow the backwards compatability policy."**

What this does change is that the API is never considered complete, as in the original phrasing. Docutils [isn't slated for inclusion](https://www.python.org/dev/peps/pep-0258/#rejection-notice) in the standard library any more, and there will always be potential improvements and changes -- new parsers, new node types, changes in the HTML specification, et cetera. 

The 1.0 release then does not need to be a "big deal", and future breaking changes can go to 2.0, 3.0, etc -- setuptools is now on `60.5.0`!

Semantic versioning does have some drawbacks (https://snarky.ca/why-i-dont-like-semver/), so projects like pip have adopted calendar based versioning -- the major version is the year (22), and the minor version is either the current month or an increasing number.  This relies on strong documentation and changelogs, so that when users upgrade it is obvious what changed (the idea being that every change breaks someone, so there is no contract that version numbers within a certain range mean no breakage). Luckily, Docutils has a good culture around changelogs and histories etc, so this would be easy to adopt.

______


I also suggest enumerating all modules, classes, and functions that form the public API. The current backwards compatability policy references PEP 387, which is specifically for the Python project itself. Checking through the documentation on every change to identify if a name is public or private is error prone (we are all human, and can miss things with no malintent) and time consuming.

At the very least I would suggest, `docutils.nodes`, the reader/writer/parser aliases, `docutils.core`,  and the front end tools. (I'm happy to write out the full list if you give me modules/etc that should be part of the public API). It would also be useful to identify what parts of Docutils large downstream consumers use, and either create higher level abstractions or mark those as public API. (I would suggest Sphinx and MyST-parser). 

It is also the [established practice](https://www.python.org/dev/peps/pep-0008/#descriptive-naming-styles) to mark names as private by prefixing them with an underscore. I suggest adopting this, as it gives strong guidance to downstream library authors what Docutils considers private and public. Making a private name public is far easier than going through a deprecation cycle for the reverse.

I'd suggest adding the ability to have exceptions to the policy, with removal after one minor version. My full suggested text is:

**"Removal or significant alteration of any members of the public API will only take place after the behaviour has been marked as deprecated for two minor releases. Certain changes may have a shorter deprecation period of one minor release. This requires at least two project members to be in favour of doing so, and no members against.**

**Changes that may affect end-users (e.g. by requiring changes to the configuration file or potentially breaking custom style sheets) should be announced with a FutureWarning."**

This post is intentionally opinionated so as to provide something to talk over / edit -- I'm not massively attached to any one thing!

A

----
for reference, amongst others, I took inspiration from:
https://setuptools.pypa.io/en/latest/development/releases.html
https://pip.pypa.io/en/latest/development/release-process/
https://numpy.org/neps/nep-0023-backwards-compatibility.html



---

Sent from sourceforge.net because doc...@li... is subscribed to https://sourceforge.net/p/docutils/feature-requests/

To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/docutils/admin/feature-requests/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.