Re: [Docutils-develop] Recent commit activity

[Guenter M. <mi...@us...>]

Dear Adam,

thank you for the fast reply.

On 2022-11-01, Adam Turner wrote:
...

> The overall aim with the test changes is to be able to use the standard
> library's ``unittest`` runner, which will alleviate some of the points
> you raise.

I understand and support the overall aim to use standard tools for the tests.
I would have preferred a "feature branch" (as mandated by the "policies")
for this change set, though. (I agree that with git this would have been
easier and have to apologise for the delay of my respective reviews.)


>> r9135
>>   tex2unichar.py is a generated file, changes will be lost with an
>>   update from the database.
>       
>>   (I updated the generating script and prepared a fixup patch.)

> I saw the note that it is generated, but I could not find the
> generating script in the repository (even in ``sandbox/``) -- please
> may you let me know where it is?    

It is at the place indicated in the script,
http://milde.users.sourceforge.net/LUCR/Math/ in the "tools" subdir.
The latest version is not published yet, though, and I plan moving the
project to codeberg.org.

Generally, if a file is created or maintained by an active member of the
Docutils developer team, I'd prefer an "ask before change" (except for
emergencies and small typos).


>> r9140, r9141: Import ``DocutilsTestSupport`` from ``test``

>>   This commit breaks the ability to run individual test scripts from
>>   their respective directories. This is (was) an important help when
>>   working on the test scripts.

> The unittest runner resolves this, though we are not quite at that
> point. We could add an option to ``alltests.py`` to run only tests in a
> specific directory?

When editing a test script, I regularly run it from my editor (it is
just one click). This is a feature I don't want to miss, the actual
implementation is less important (but an option to alltests.py would not
help, it should work from the scrip's "shebang line"). Is there a way
to call the "unittest runner" in the shebang line of the individual
scripts?

...

>> r9146
>>  DocutilsTestSupport must be imported before docutils
>>  so that imports from docutils use the test-dirs parent directory
>>  (in case we want to test a not installed Docutils version)

> If alltests.py is used to run tests, this is still the case.
> We could add an import of DTS to those modules again, though my aim
> eventually is to remove DTS. Is testing an uninstalled Docutils a
> usecase that is widely relied upon?

At least for me, it is a "natural" thing to expect (as I got used to it over
the last decade).

...


>> r9167
Introduces a backwards incompatible API change! (see below)

>>   IMV, an encoded string ("bytes" object) is no "binary" output.
...

> Python 3 bytes objects are sequences of 8-bit bytes, so I would say
> that it is binary output, but perhaps we could improve here.

The Python 3.10 documentation for `bytes` calls it a "binary sequence
type", so I may have to adapt.

The naming of the `core.publish_string()` API function pre-dates Python3
and uses the term "string" as a superordinate for "Unicode strings"
(`str`/`unicode`) and "encoded strings" (`bytes`/`str`).
The documentation is explicit about the return value
(defaulting to 'utf-8' encoded `bytes` instances).

The function is part of the "first level API": intended for use by
3rd-party code. Changing the return value has the potential of many
breaks. 
Please revert the changes to core.py and io.py.

Unicode `str` output can be achieved passing
``settings_overwrites={'output_encoding': 'unicode'}`` to
publish_string().

(In the long run, I'd like to replace the "homegrown" docutils.io classes
with standard objects, but this is a separate topic.)


>> r9168 Use ``.rstrip()`` instead

>>   This fails with multi-lined output if line endings in output and extended
>>   differ.

> For tests, I believe we should ensure that we have as little 'magic' as
> possible -- what we say we expect should be bit-for-bit identical to
> what we generate. In this sense I fixed the newline errors to ensure
> correctness.

This depends on the context: over-specified tests contribute to noise due to
false positives. Test results should not depend on the "native" line endings
of the system running the tests.


>> r9178
8 lines of doctest code seems overly verbose in the context of an
"overview of the Docutils architecture".

>>   In case we don't want to ship "quicktest.py" with the Docutils
>>   distribution, we can still keep it in the Docutils repo and simply change:
>  
>>   -can be found in the ``tools/`` directory of the Docutils distribution)
>>   +can be found in the ``tools/`` directory of the Docutils repository)

> I was actually going to suggest deleting "quicktest.py"
> entirely--finding the version can now be done with "docutils --version"
> and the last substantive change in functionality was 2006.
> "EasyDialogs" was removed in Python 2.7.

It can still be useful for development: testing the reST parser (including
the option to generate input/output pairs for parser test scripts).
I'd move it to tools/dev and remove the EasyDialogs part.


Günter