#42 Permission to upload type hints to the Typeshed

[Anthon van der Neut]

As per PEP 484 I am requesting permission to submit Python type stubs to https://github.com/python/typeshed

A preview is at https://github.com/common-workflow-language/schema_salad/tree/master/typeshed/2.7/ruamel/yaml

(originally posted on 2016-08-10 at 09:43:56 by Michael Crusoe <mr-c@bitbucket>)

[Anthon van der Neut]

Michael, first of all thanks for your effort and contacting me up front.
I am trying to read up on what this implies, both to find out how I check that what you submit is correct, as well as how this affects changes that I make to the library in the future (I know for sure I will change at least the internals of some classes in comments.py)

I haven't looked at typing info in any detail yet, so I don't understand all the implications. One thing I am not sure of if it isn't better to keep that information in sync with ruamel.yaml by including it in the project and test-and-generate it from that information.

Are you under any time pressure to get this permission (or alternative)?
Is it on purpose that the stubs in the preview seem to be a subset of the interface?
Any recommendations, apart from reading PEP484, to get up to speed with this?

(originally posted on 2016-08-10 at 10:16:59)

[Anthon van der Neut]

Howdy Anthon,

This is just documenting the public interface in a single place. Right now I have multiple copies in my projects that I manually have to keep in sync. Here's a slightly different version https://github.com/common-workflow-language/cwl-upgrader/tree/master/typeshed/2or3/ruamel/yaml as an example.

Internal changes shouldn't affect the public interface. You might want to have more complicated types defined and testing that aren't public. It is up to you.

It would be nice to get them checked in as it would allow others to benefit ASAP, it would get the types reviewed by the Typeshed community, and it would allow others including yourself to improve upon them. I would enjoy not having to synchronize my copies across my projects.

Here are slides from the presentation at PyCon 2016: http://mypy-lang.blogspot.ro/2016/05/slides-for-mypy-talk-at-pycon.html

(originally posted on 2016-08-10 at 10:32:55 by Michael Crusoe <mr-c@bitbucket>)

[Anthon van der Neut]

Hi Michael,

I got some time today to look further into things. I made a small program (that just imports ruamel.yaml and call ruamel.yaml.load) and started running mypy against it (automatically on every save of any ruamel/yaml/*.py) until I got rid of all the errors (that is by generously adding Any and some # type: ignore stuff.

I then tried to run tox once more and had to decide that this is a good moment to drop support for 2.6. The other targets (2.7, 3.5, 3,4, 3.3, pypy, pep8/flake8) all passed.

Once I commit that, that would take away the need to maintain stubs on typeshed separately, wouldn't it?

(originally posted on 2016-08-15 at 15:21:44)

[Anthon van der Neut]

Hey Anthon,

Can you make the branch public so I can test it? For redistribution the type hints need to be in stub files .pyi as compiled python doesn't contain them.

(originally posted on 2016-08-15 at 16:19:16 by Michael Crusoe <mr-c@bitbucket>)

[Anthon van der Neut]

Nevermind they can indeed be in the main source. But it would still be nice to test them.

(originally posted on 2016-08-15 at 16:22:33 by Michael Crusoe <mr-c@bitbucket>)

[Anthon van der Neut]

I've just commited 0.12.0. Please test with that and let me know if there are additional issues with the typing that need resolving.

(originally posted on 2016-08-15 at 20:49:39)

[Anthon van der Neut]

• status set to resolved

I checked out the latest version of schema_salad, linked in my ruamel directory (replacing typeshed/2.7/ruamel ) and ran make mypy. That resulted in a few more additions and version 0.12.1

I consider this resolved now, please close if you can confirm this.

(originally posted on 2016-08-16 at 08:16:09)

[Anthon van der Neut]

Thanks for looking at this. Alas the PEP 484/mypy people haven't settled on a way to incorporate inline type annotations for installed packages without type checking all installed packages so I'm going to keep my stubs until they figure that out.

https://github.com/python/mypy/issues/1190

(originally posted on 2016-08-16 at 13:24:31 by Michael Crusoe <mr-c@bitbucket>)

[Anthon van der Neut]

I had not thought about that.

If you are on a filesystem that supports symbolic links, would finding the ruamel/yaml directory and setting up a link from typeshed/2.7/ruamel/yaml as part of the Makefile target be a solution? I can make that for you, should come in handy for others packages as well.

I have tried to generate the stubs with

mkdir -p out; for i in *.py; do stubgen ruamel.yaml.${i%.py}; done

(also with --py2), but stubgen is not smart enough to incorporate the available type information from the comments....

(originally posted on 2016-08-16 at 15:16:51)

[Anthon van der Neut]

Great idea, that worked

       rm -Rf typeshed/2.7/ruamel/yaml
       ln -s $(shell python -c 'import ruamel.yaml; import os.path; print os.path.dirname(ruamel.yaml.__file__)') \
               typeshed/2.7/ruamel/

Except that you have an error in your types :-)

(env) michael@dan:~/schema_salad$ make mypy
rm -Rf typeshed/2.7/ruamel/yaml
ln -s /home/michael/schema_salad/env/local/lib/python2.7/site-packages/ruamel/yaml \
    typeshed/2.7/ruamel/
MYPYPATH=typeshed/2.7 mypy --py2 --disallow-untyped-calls \
     --fast-parser --warn-redundant-casts --warn-unused-ignores \
     schema_salad
typeshed/2.7/ruamel/yaml/main.py:16: note: In module imported here,
typeshed/2.7/ruamel/yaml/__init__.py:81: note: ... from here,
schema_salad/tests/test_examples.py:8: note: ... from here:
typeshed/2.7/ruamel/yaml/compat.py:79: error: syntax error in type comment
Makefile:173: recipe for target 'mypy' failed
make: *** [mypy] Error 1
(env) michael@dan:~/schema_salad$ mypy --version
mypy 0.4.3

The offending line is

    unichr = unichr  # type: ignore   to allow importing

You'll need a second # after the embedded type

After fixing that you will find two unused ignores:

typeshed/2.7/ruamel/yaml/main.py:376: note: unused 'type: ignore' comment
typeshed/2.7/ruamel/yaml/compat.py:67: note: unused 'type: ignore' comment

Then all is well.

(originally posted on 2016-08-16 at 16:04:17 by Michael Crusoe <mr-c@bitbucket>)

[Anthon van der Neut]

Aha, you added some arguments to make mypy pickier ;-)

0.12.2 will have the above fixes, it is in the process of being generated and pushed

(originally posted on 2016-08-16 at 17:26:08)

[Anthon van der Neut]

Thanks!

(originally posted on 2016-08-19 at 13:03:26 by Michael Crusoe <mr-c@bitbucket>)

[Anthon van der Neut]

• status set to closed

(originally posted on 2016-08-20 at 10:58:01)