#496 OptionParser deprecated without replacement makes Sphinx extensions crash in strict mode

[Sviatoslav Sydorenko]

Per best practice, many people run Sphinx with -n -W --keep-going. Plus the interpreter itself may be called with -Werror. This turns any warnings into errors, making sure that no problems or future deprecations will go unnoticed.
The typical situation is when a dependency updates, it emits a deprecation warning and we can address said warning by doing what it says and starting to use some new API. This is happening together with the version bump so it never breaks CI.
Things are different with docutils, though. When I was integrating sphinx-autodoc-typehints into well-established codebase, I started getting a traceback, breaking the docs builds.

I wasn't the first one to hit it: https://github.com/tox-dev/sphinx-autodoc-typehints/issues/454.

When I saw the deprecation message, I was expecting the usual process. I thought that I'd contribute a bit of compatibility code into sphinx-autodoc-typehints and be done with it.

I tracked it down to https://github.com/docutils/docutils/commit/6548b56d9ea9a3e101cd62cfcd727b6e9e8b7ab6#diff-d1f841500d12a20b000229849c98cd38d8fe4d0ab5a149119244931969ee82e3R659 and what I saw caught me by surprise — the was no replacement. The deprecation warning is emitted but in a way that it's impossible to react to it in any reasonable manner. It says that docutils.frontend.OptionParser is not supposed to be used anymore but does not offer any way forward. It's a stalemate.

So here I am, writing this bug report in hopes that the depecation warning would be dropped or the replacement added. I argue that this use of deprecation warnings is incorrect. It's not even a FutureDeprecationWarning. It's something that is stated to be deprecated already, right now, today, two years ago even.

[Günter Milde]

The docutils OptionParser class is not intended for direct use -- code using Docutils as a library is supposed to work with the provided API functions instead.

Could you be more specific about the situation that led to the warning? With a minimal working example showing the problem and a traceback, we may help finding the right replacement code.

[Sviatoslav Sydorenko]

The exact place in said plugin is linked in the issue I mentioned in the original post: https://github.com/tox-dev/sphinx-autodoc-typehints/issues/454#issuecomment-2489588907 .

It's here: https://github.com/tox-dev/sphinx-autodoc-typehints/blob/ffea355/src/sphinx_autodoc_typehints/init.py#L850-L854

P. S. While it's obviously the case of https://www.hyrumslaw.com/, may I suggest keeping private API in underscored modules? Regular module names are typically perceived as public API.

[Günter Milde]

Thanks for the info. Please try replacing

- from docutils.frontend import OptionParser
+ from docutils.frontend import get_default_settings

and

- settings = OptionParser(components=(RSTParser,)).get_default_values()
+ settings = get_default_settings(RSTParser)

[r9990] adds a replacement hint to the deprecation message of frontend.OptionParser.

may I suggest keeping private API in underscored modules?

The "frontend" module and frontend.OptionParser are from 2002, predating the inclusion of "optparse" into the Python standard library in Python 2.3 and 9 years before active development switched to "argparse".
In order to prepare for the transition to "argparse", docutils.frontend became classed as "provisional" in Docutils 0.19. Renaming it now to add an underscore would cause significantly more disruption than a deprecation warning.

[Sviatoslav Sydorenko]

I understand that breaking long-standing APIs is not an option. I was merely suggesting doing this for new things.

[Günter Milde]

It seems the proposed change did the trick.

So for the relevant use case there is a replacement and it seems rather like an "incomplete documentation" problem -- which is hopefully fixed by [r9990].

Thanks for reporting.