docutils-develop

[Docutils-develop] Adding type hints directly to docutils?

[Stephen F. <st...@th...>]

o/

I recently had to contribute a fix to the stubs type hints package for docutils
in typeshed [1]. Is there any reason *not* to add these directly to docutils? I
would be willing to tackle this effort if it would be welcomed (I previously
tackled the "make docutils support Python 3 natively" effort, along with some
smaller quality-of-life fixes).

As a follow-up to the above, are there any significant concerns with using a
tool like 'ruff format' or 'black' to auto-format code before adding the
annotations? It's small thing but it would make the process of folding in hints
somewhat easier if I can rely on tooling to handle formatting for me.

Cheers,
Stephen

[1] https://github.com/python/typeshed/tree/main/stubs/docutils

Re: [Docutils-develop] Adding type hints directly to docutils?

[Guenter M. <mi...@us...>]

Dear Stephen,

On 2024-04-15, Stephen Finucane wrote:

> I recently had to contribute a fix to the stubs type hints package for
> docutils in typeshed [1]. Is there any reason *not* to add these
> directly to docutils? 

* Historically, support for legacy versions prevented directly adding type
  hints.
  
* Currently, the main reason is: "it must be done by someone" and it is just
  one of may open tasks.

* The main author is no longer active and sometimes the correct type is not
  known (e.g. there are still cases with 0 and 1 for booleans from the
  days Python did not have False/True). Sometimes no hint may be better than
  a misleading hint... I don't know about the quality of the existing stubs. 

* One more reason is the idea to wait until we drop support for
  Python 3.9 (once it reaches end of life...) so we can use the union
  operator (:PEP:`604`).
  
> I would be willing to tackle this effort if it would be welcomed (I
> previously tackled the "make docutils support Python 3 natively"
> effort, along with some smaller quality-of-life fixes).

Contributions are welcome.

> As a follow-up to the above, are there any significant concerns with
> using a tool like 'ruff format' or 'black' to auto-format code before
> adding the annotations? It's small thing but it would make the process
> of folding in hints somewhat easier if I can rely on tooling to handle
> formatting for me.

There is support for automatic checks with https://pre-commit.com for
Git-based repos (a ".pre-commit-config.yaml" file). It includes checks
for trailing-whitespace and flake8 (with ".flake8" configuration files
for the numerous exceptions due to legacy code or practicability).

"black" seems too rigid to me (practicability beats purity), especially as
we have a large codebase with long legacy and must rely on "git blame" 
in many cases to find out whether some legacy code may be dropped or has a
reason...


Günter

Re: [Docutils-develop] Adding type hints directly to docutils?

[Dave K. <dku...@da...>]

On 2024-04-16 13:45, Guenter Milde via Docutils-develop wrote:
> 
>> I would be willing to tackle this effort if it would be welcomed (I
>> previously tackled the "make docutils support Python 3 natively"
>> effort, along with some smaller quality-of-life fixes).
> 
> Contributions are welcome.
> 

I wrote the original code for rst2odt.
I'm willing to work on that module.
Any suggestions or guidance would be welcomed.

If we do attempt this task, it'd be good if we do
it consistently across all the module we add type hints to.
Is there something we should look at for a style guide?
I can't find anything that is specific to type hints in
PEP 8 (the style guide).  PEP 484 ("Type hints") has some
guidance.  Is there anything else I should read?

Dave

Re: [Docutils-develop] Adding type hints directly to docutils?

[Stephen F. <st...@th...>]

On Tue, 2024-04-16 at 16:52 -0600, Dave Kuhlman wrote:
> On 2024-04-16 13:45, Guenter Milde via Docutils-develop wrote:
> > 
> > > I would be willing to tackle this effort if it would be welcomed (I
> > > previously tackled the "make docutils support Python 3 natively"
> > > effort, along with some smaller quality-of-life fixes).
> > 
> > Contributions are welcome.
> > 
> 
> I wrote the original code for rst2odt.
> I'm willing to work on that module.
> Any suggestions or guidance would be welcomed.
> 
> If we do attempt this task, it'd be good if we do
> it consistently across all the module we add type hints to.
> Is there something we should look at for a style guide?
> I can't find anything that is specific to type hints in
> PEP 8 (the style guide).  PEP 484 ("Type hints") has some
> guidance.  Is there anything else I should read?

That would be great. I would recommend checking out the docutils stub package in
the typedshed repo to see if hints are present there first. If not, you'd be
starting from scratch.

Regarding styling these, I'm not aware of anything as concrete as PEP 8 so I
think we'll be making it up as we go. I would suggest we mainly follow the style
of typeshed itself, in so far as there is a style. I would also like us to fold
the hints into the code itself, rather than presenting separate '.pyi' files.

Is there anything you've seen done elsewhere that you think we should duplicate
here?

Stephen

> 
> Dave
> 
> 
> _______________________________________________
> Docutils-develop mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-develop
> 
> Please use "Reply All" to reply to the list.

Re: [Docutils-develop] Adding type hints directly to docutils?

[Stephen F. <st...@th...>]

On Tue, 2024-04-16 at 19:45 +0000, Guenter Milde via Docutils-develop wrote:
> Dear Stephen,
> 
> On 2024-04-15, Stephen Finucane wrote:
> 
> > I recently had to contribute a fix to the stubs type hints package for
> > docutils in typeshed [1]. Is there any reason *not* to add these
> > directly to docutils? 
> 
> * Historically, support for legacy versions prevented directly adding type
>   hints.
>   
> * Currently, the main reason is: "it must be done by someone" and it is just
>   one of may open tasks.
> 
> * The main author is no longer active and sometimes the correct type is not
>   known (e.g. there are still cases with 0 and 1 for booleans from the
>   days Python did not have False/True). Sometimes no hint may be better than
>   a misleading hint... I don't know about the quality of the existing stubs. 
> 
> * One more reason is the idea to wait until we drop support for
>   Python 3.9 (once it reaches end of life...) so we can use the union
>   operator (:PEP:`604`).
>   
> > I would be willing to tackle this effort if it would be welcomed (I
> > previously tackled the "make docutils support Python 3 natively"
> > effort, along with some smaller quality-of-life fixes).
> 
> Contributions are welcome.

Excellent. I just wanted to make sure it wouldn't be wasted effort.

> > As a follow-up to the above, are there any significant concerns with
> > using a tool like 'ruff format' or 'black' to auto-format code before
> > adding the annotations? It's small thing but it would make the process
> > of folding in hints somewhat easier if I can rely on tooling to handle
> > formatting for me.
> 
> There is support for automatic checks with https://pre-commit.com for
> Git-based repos (a ".pre-commit-config.yaml" file). It includes checks
> for trailing-whitespace and flake8 (with ".flake8" configuration files
> for the numerous exceptions due to legacy code or practicability).

I'm a big fan of pre-commit and use it in many of my projects. However, as the
project is hosted in SVN I had assumed any git-specific tooling would be out the
window. More than happy to add this.

> "black" seems too rigid to me (practicability beats purity), especially as
> we have a large codebase with long legacy and must rely on "git blame" 
> in many cases to find out whether some legacy code may be dropped or has a
> reason...

I too have some concerns with the style of black-formatted code - as I do with
the style of gofmt-formatted Go code - however, it _is_ robustly maintained,
requires little to no configuration on our end, and effectively allows
formatting to be ignored by developers. It is also possible to add a '.git-
blame-ignore-revs' file that will allow git-blame to ignore certain revisions.
This file is also respected by some software forges such as GitHub, though I
think it's fair to assume it is ignored for SVN repos on Sourceforge :)

If it's okay with you, I will propose adding hints to a single module _with_
black to see how it works. If the output is truly awful, we can change tack and
try to find a formatter than is slightly closer to what we already have but
still allows me to avoid fixing formatting in some many 100s of files by hand.

Cheers,
Stephen

> 
> 
> Günter
> 
> 
> 
> 
> _______________________________________________
> Docutils-develop mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-develop
> 
> Please use "Reply All" to reply to the list.

Re: [Docutils-develop] Adding type hints directly to docutils?

[Karl O. P. <ko...@ka...>]

On Wed, 17 Apr 2024 11:02:11 +0100
Stephen Finucane <st...@th...ru> wrote:

> On Tue, 2024-04-16 at 19:45 +0000, Guenter Milde via Docutils-develop
> wrote:

> > "black" seems too rigid to me (practicability beats purity),
> > especially as we have a large codebase with long legacy and must
> > rely on "git blame" in many cases to find out whether some legacy
> > code may be dropped or has a reason...  
> 
> I too have some concerns with the style of black-formatted code - as
> I do with the style of gofmt-formatted Go code - however, it _is_
> robustly maintained, requires little to no configuration on our end,
> and effectively allows formatting to be ignored by developers.

Unsolicited input.  It seems now would be the time to comment,
even though I am unlikely to ever be involved.

I like (the traditional) 79 char line length limit.
black --line-length 79 ...
isort --line-length 79 ...

(Black defaults to 88.)

There's a reason newspapers are written in columns.
The eye must track back from the end of one line
to the beginning of the next.  This is disturbed by
over-long lines.

Regards,

Karl <ko...@ka...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: [Docutils-develop] Adding type hints directly to docutils?

[Stephen F. <st...@th...>]

On Wed, 2024-04-17 at 09:03 -0500, Karl O. Pinc wrote:
> On Wed, 17 Apr 2024 11:02:11 +0100
> Stephen Finucane <st...@th...ru> wrote:
> 
> > On Tue, 2024-04-16 at 19:45 +0000, Guenter Milde via Docutils-develop
> > wrote:
> 
> > > "black" seems too rigid to me (practicability beats purity),
> > > especially as we have a large codebase with long legacy and must
> > > rely on "git blame" in many cases to find out whether some legacy
> > > code may be dropped or has a reason...  
> > 
> > I too have some concerns with the style of black-formatted code - as
> > I do with the style of gofmt-formatted Go code - however, it _is_
> > robustly maintained, requires little to no configuration on our end,
> > and effectively allows formatting to be ignored by developers.
> 
> Unsolicited input.  It seems now would be the time to comment,
> even though I am unlikely to ever be involved.
> 
> I like (the traditional) 79 char line length limit.
> black --line-length 79 ...
> isort --line-length 79 ...
> 
> (Black defaults to 88.)
> 
> There's a reason newspapers are written in columns.
> The eye must track back from the end of one line
> to the beginning of the next.  This is disturbed by
> over-long lines.

I agree.

> 
> Regards,
> 
> Karl <ko...@ka...>
> Free Software:  "You don't pay back, you pay forward."
>                  -- Robert A. Heinlein

[Docutils-develop] ODT writer (was: Adding type hints directly to docutils?)

[Guenter M. <mi...@us...>]

Dear Dave,

glad to hear that you are still around!

On 2024-04-16, Dave Kuhlman wrote:

> I wrote the original code for rst2odt.
> I'm willing to work on that module.
> Any suggestions or guidance would be welcomed.

> If we do attempt this task, it'd be good if we do
> it consistently across all the module we add type hints to.
> Is there something we should look at for a style guide?
...

My suggestion would be that we start with another module (core.py or
utils/__init__.py, say) and work out some style rules for our policies
from this example. If we agree to do the writers last (and leave the ODT
to you), there should be plenty of code to look at for guidance.

There is also a number of "Tickets" (bug reports, feature requests, patches)
tagged as "ODT writer", e.g. 
https://sourceforge.net/p/docutils/bugs/search/?q=labels%3AODT+AND+status%3Aopen+OR+status+pending-remind

Günter

Re: [Docutils-develop] Adding type hints directly to docutils?

[Guenter M. <mi...@us...>]

On 2024-04-17, Stephen Finucane wrote:
> On Tue, 2024-04-16 at 19:45 +0000, Guenter Milde via Docutils-develop wrote:
>> On 2024-04-15, Stephen Finucane wrote:

Dear Stephen,

>> Contributions are welcome.

> Excellent. I just wanted to make sure it wouldn't be wasted effort.

>> > As a follow-up to the above, are there any significant concerns with
>> > using a tool like 'ruff format' or 'black' to auto-format code before
>> > adding the annotations? It's small thing but it would make the process
>> > of folding in hints somewhat easier if I can rely on tooling to handle
>> > formatting for me.

>> There is support for automatic checks with https://pre-commit.com for
>> Git-based repos [...]
...

> I'm a big fan of pre-commit and use it in many of my projects. However,
> as the project is hosted in SVN I had assumed any git-specific tooling
> would be out the window. More than happy to add this.

It works with `git svn` because the check is at commit stage, not when
commiting with dcommit :)

>> "black" seems too rigid to me (practicability beats purity),
>> especially as we have a large codebase with long legacy and must rely
>> on "git blame" in many cases to find out whether some legacy code may
>> be dropped or has a reason...

> I too have some concerns with the style of black-formatted code - as I
> do with the style of gofmt-formatted Go code - however, it _is_
> robustly maintained, requires little to no configuration on our end,
> and effectively allows formatting to be ignored by developers. It is
> also possible to add a '.git- blame-ignore-revs' file that will allow
> git-blame to ignore certain revisions. This file is also respected by
> some software forges such as GitHub, though I think it's fair to assume
> it is ignored for SVN repos on Sourceforge :)

Would be nice, if it worked for a local git-svn checkout...

> If it's okay with you, I will propose adding hints to a single module
> _with_ black to see how it works. If the output is truly awful, we can
> change tack and try to find a formatter than is slightly closer to what
> we already have but still allows me to avoid fixing formatting in some
> many 100s of files by hand.

I am still sceptical but we may just test.
Is there some tool what only applies auto-formatting to changes?


Mind, that we are on course to the 0.21.2 maintenance release, so no 
changes to the repository master now and only fixes for a week after the
release (until we know if everything went fine).

Thanks Günter