Menu

#365 New 0.15 release not Python-2 compatible

closed-fixed
python-2 (1)
5
2020-03-03
2019-07-21
Shai Berger
No

Hello,

At Matific, we are using docutils as a 2nd-order dependency, and this morning our CI failed with these messages:

[2019-07-21T09:23:34.566Z] File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
[2019-07-21T09:23:34.566Z] print('\n::: Runtime settings:', file=self._stderr)
[2019-07-21T09:23:34.566Z] ^
[2019-07-21T09:23:34.566Z] SyntaxError: invalid syntax. 

I believe a from __future__ import print_function is missing at the top of that file.

Related

Bugs: #365

Discussion

  • (inactive account)

    Copy that, our CI builds (which for reasons outside our control uses Python 2 software) also failed this morning. Could this be fixed ASAP?

     
  • Gurel Kaynak

    Gurel Kaynak - 2019-07-21

    Our production site was down because of this on a sunday. Please fix ASAP...

    Another library that we use requires docutils and they probably put it in requirements.txt as "docutils" or "docutils==0.x". So when our servers autoscaled today they started with docutils 0.15 and failed all requests.

     

    Last edit: Gurel Kaynak 2019-07-21
  • Andreas Maier

    Andreas Maier - 2019-07-21

    We (pywbem package) also ran into the same kind of issue on Python 2.7 (2.7.16 on local MacOS and 2.7.15 on Travis xenial), although on a different line of docutils.

    I verified that the issue occurs with docutils 0.15, and does not occur with docutils 0.14. We will pin docutils to 0.14 until this issue is fixed.

    Travis log of the issue: https://travis-ci.org/pywbem/pywbem/jobs/561714372
    pywbem issue documenting this: https://github.com/pywbem/pywbem/issues/1788

    End of the traceback:

     File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/docutils/io.py", line 348
        (self.destination.mode, mode)), file=self._stderr)
                                            ^
    SyntaxError: invalid syntax
    
     

    Last edit: Andreas Maier 2019-07-21
  • Dmitry Shachnev

    Dmitry Shachnev - 2019-07-21

    Dear docutils developers,

    It looks like only the docutils-0.15-py2-none-any.whl wheel is broken, the docutils-0.15.tar.gz tarball is correct.

    So as a temporary measure you can remove the wheel from PyPI, then pip will just use the tarball. I think that you can also generate a new wheel without making new release, just name it docutils-0.15-1-py2-none-any.whl.

     
  • David Alphus

    David Alphus - 2019-07-21

    I can confirm the docutils-0.15-py2-none-any.whl contains the contents of the the -py3 whl. Source, however, is totally fine.

     
  • Octavio Martin

    Octavio Martin - 2019-07-21

    Our EMRs cluster are not working because the boostraping are coming with the new version

    Traceback (most recent call last):
    File "/usr/bin/aws", line 19, in <module>
    import awscli.clidriver
    File "/usr/lib/python2.7/dist-packages/awscli/clidriver.py", line 35, in <module>
    from awscli.help import ProviderHelpCommand
    File "/usr/lib/python2.7/dist-packages/awscli/help.py", line 20, in <module>
    from docutils.core import publish_string
    File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
    print('\n::: Runtime settings:', file=self._stderr)
    ^
    SyntaxError: invalid syntax
    Command exiting with ret '1'</module></module></module>

     
  • Vinay Hiremath

    Vinay Hiremath - 2019-07-21

    This error is making it so our ec2 instance cannot start up. I have documented my investigation in an AWS forum thread:

    https://forums.aws.amazon.com/thread.jspa?messageID=909018󝻚

    This is the trace pulled from our EC2 system logs

    Traceback (most recent call last):
      File "/usr/bin/aws", line 19, in <module>
        import awscli.clidriver
      File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 36, in <module>
        from awscli.help import ProviderHelpCommand
      File "/usr/local/lib/python2.7/site-packages/awscli/help.py", line 20, in <module>
        from docutils.core import publish_string
      File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
        print('\n::: Runtime settings:', file=self._stderr)
                                             ^
    SyntaxError: invalid syntax
    Traceback (most recent call last):
      File "/usr/bin/aws", line 19, in <module>
        import awscli.clidriver
      File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 36, in <module>
        from awscli.help import ProviderHelpCommand
      File "/usr/local/lib/python2.7/site-packages/awscli/help.py", line 20, in <module>
        from docutils.core import publish_string
      File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
        print('\n::: Runtime settings:', file=self._stderr)
    

    This is from our Docker build via Travis CI/CD:

    Step 26/28 : RUN /root/.local/bin/aws s3 cp s3://xxxxxxxxxxx.pem /usr/local/loom/dist/server/config
    
     ---> Running in 854f779de63f
    
    Traceback (most recent call last):
      File "/root/.local/bin/aws", line 19, in <module>
        import awscli.clidriver
      File "/root/.local/lib/python2.7/site-packages/awscli/clidriver.py", line 36, in <module>
        from awscli.help import ProviderHelpCommand
      File "/root/.local/lib/python2.7/site-packages/awscli/help.py", line 20, in <module>
        from docutils.core import publish_string
      File "/root/.local/lib/python2.7/site-packages/docutils/core.py", line 246
        print('\n::: Runtime settings:', file=self._stderr)
                                    
    

    This is leading to a very large transcoder backup on our end. Is there any way I can help commandeer a change with someone/code review to accelerate a fix here?

     

    Last edit: Vinay Hiremath 2019-07-21
  • Vinay Hiremath

    Vinay Hiremath - 2019-07-22

    Seems like Opsworks has pined to >10 <15 to fix for now

     
  • engelbert gruber

    dear all
    I am very sorry I removed the py2 wheel
    and build a py2 one today

    CAUTION
    it seams that one should not::

    python2 setup.py bdist_wheel

    after python3 bdist_wheel
    it will build the py3 one with py2 name

     
  • engelbert gruber

    • assigned_to: engelbert gruber
     
  • engelbert gruber

    • status: open --> closed-fixed
     
  • engelbert gruber

    docutils-0.15.post1-py2-none-any.whl is on pypi

     
    • Shai Berger

      Shai Berger - 2019-07-23

      Thank you for the quick fix (and for all of your work on this package).

       
      • engelbert gruber

        you are very welcome
        and i am sorry for the hassle

        On Tue, 23 Jul 2019 at 14:43, Shai Berger shai-matific@users.sourceforge.net wrote:

        Thank you for the quick fix (and for all of your work on this package).

        Status: closed-fixed
        Group: Default
        Labels: python-2
        Created: Sun Jul 21, 2019 09:56 AM UTC by Shai Berger
        Last Updated: Mon Jul 22, 2019 09:03 PM UTC
        Owner: engelbert gruber

        Hello,

        At Matific, we are using docutils as a 2nd-order dependency, and this
        morning our CI failed with these messages:

        [2019-07-21T09:23:34.566Z] File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
        [2019-07-21T09:23:34.566Z] print('\n::: Runtime settings:', file=self._stderr)
        [2019-07-21T09:23:34.566Z] ^
        [2019-07-21T09:23:34.566Z] SyntaxError: invalid syntax.

        I believe a from future import print_function is missing at the top
        of that file.


        Sent from sourceforge.net because you indicated interest in
        https://sourceforge.net/p/docutils/bugs/365/

        To unsubscribe from further messages, please visit
        https://sourceforge.net/auth/subscriptions/

         

        Related

        Bugs: #365

  • Andreas Maier

    Andreas Maier - 2019-07-22

    The latest version I can see on Pypi is the 0.15 version released on 7/21.

    However, when I do a pip install docutils --upgrade it downloads a docutils-0.15.post1-py2-none-any.whl, wich after installation reports 0.15 (without .post1).

    That is very strange...

    I can confirm that the new version solves the original issue, but the strange thing is that it is called version 0.15. So in any given Python environment, I have no way to find out whether it is the broken 0.15 or the fixed 0.15??

     

    Last edit: Andreas Maier 2019-07-22
  • Lawrence Finn

    Lawrence Finn - 2019-07-22

    I am still having an issue with this release

    Lawrences-MacBook-Pro-2:aiq lawrencefinn$ wget "https://files.pythonhosted.org/packages/d2/d8/06e83efbfafab5d342df8835f3aa81c1dbfadedbea39cc2d2f93742938a4/docutils-0.15.post1-py2-none-any.whl"
    --2019-07-22 15:23:14-- https://files.pythonhosted.org/packages/d2/d8/06e83efbfafab5d342df8835f3aa81c1dbfadedbea39cc2d2f93742938a4/docutils-0.15.post1-py2-none-any.whl
    Resolving files.pythonhosted.org... 151.101.249.63, 2a04:4e42:2f::319
    Connecting to files.pythonhosted.org|151.101.249.63|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 546809 (534K) [binary/octet-stream]
    Saving to: ‘docutils-0.15.post1-py2-none-any.whl’

    docutils-0.15.post1-py2-none-any.whl 100%[===================================================================================================================>] 533.99K --.-KB/s in 0.08s

    2019-07-22 15:23:14 (6.81 MB/s) - ‘docutils-0.15.post1-py2-none-any.whl’ saved [546809/546809]

    Lawrences-MacBook-Pro-2:aiq lawrencefinn$ pex docutils-0.15.post1-py2-none-any.whl
    Traceback (most recent call last):
    File "/usr/local/bin/pex", line 10, in <module>
    sys.exit(main())
    File "/usr/local/lib/python2.7/site-packages/pex/bin/pex.py", line 697, in main
    pex_builder = build_pex(reqs, options, resolver_options_builder)
    File "/usr/local/lib/python2.7/site-packages/pex/bin/pex.py", line 627, in build_pex
    for resolved_dist in resolveds:
    File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 571, in resolve_multi
    use_manylinux=use_manylinux):
    File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 507, in resolve
    return resolver.resolve(resolvables_from_iterable(requirements, builder, interpreter=interpreter))
    File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 304, in resolve
    dist = self.build(package, resolvable.options)
    File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 384, in build
    dist = super(CachingResolver, self).build(package, options)
    File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 262, in build
    raise Untranslateable('Package %s is not translateable by %s' % (package, translator))
    pex.resolver.Untranslateable: Package WheelPackage('file:///Users/lawrencefinn/Documents/aiq/docutils-0.15.post1-py2-none-any.whl') is not translateable by ChainedTranslator(WheelTranslator, EggTranslator, SourceTranslator)</module>

     
    • engelbert gruber

      and what should be the result of making an executable from the
      docutils-wheel
      docutils is not one app but many rst2html.py rst2latex ...

      but I dont know pex ?

      On Mon, 22 Jul 2019 at 21:27, Lawrence Finn laf2019@users.sourceforge.net
      wrote:

      I am still having an issue with this release

      Lawrences-MacBook-Pro-2:aiq lawrencefinn$ wget "
      https://files.pythonhosted.org/packages/d2/d8/06e83efbfafab5d342df8835f3aa81c1dbfadedbea39cc2d2f93742938a4/docutils-0.15.post1-py2-none-any.whl
      "
      --2019-07-22 15:23:14--
      https://files.pythonhosted.org/packages/d2/d8/06e83efbfafab5d342df8835f3aa81c1dbfadedbea39cc2d2f93742938a4/docutils-0.15.post1-py2-none-any.whl
      Resolving files.pythonhosted.org... 151.101.249.63, 2a04:4e42:2f::319
      Connecting to files.pythonhosted.org|151.101.249.63|:443... connected.
      HTTP request sent, awaiting response... 200 OK
      Length: 546809 (534K) [binary/octet-stream]
      Saving to: ‘docutils-0.15.post1-py2-none-any.whl’

      docutils-0.15.post1-py2-none-any.whl 100%
      [===================================================================================================================>]
      533.99K --.-KB/s in 0.08s

      2019-07-22 15:23:14 (6.81 MB/s) - ‘docutils-0.15.post1-py2-none-any.whl’
      saved [546809/546809]

      Lawrences-MacBook-Pro-2:aiq lawrencefinn$ pex
      docutils-0.15.post1-py2-none-any.whl
      Traceback (most recent call last):
      File "/usr/local/bin/pex", line 10, in <module>
      sys.exit(main())
      File "/usr/local/lib/python2.7/site-packages/pex/bin/pex.py", line 697, in
      main
      pex_builder = build_pex(reqs, options, resolver_options_builder)
      File "/usr/local/lib/python2.7/site-packages/pex/bin/pex.py", line 627, in
      build_pex
      for resolved_dist in resolveds:
      File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 571,
      in resolve_multi
      use_manylinux=use_manylinux):
      File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 507,
      in resolve
      return resolver.resolve(resolvables_from_iterable(requirements, builder,
      interpreter=interpreter))
      File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 304,
      in resolve
      dist = self.build(package, resolvable.options)
      File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 384,
      in build
      dist = super(CachingResolver, self).build(package, options)
      File "/usr/local/lib/python2.7/site-packages/pex/resolver.py", line 262,
      in build
      raise Untranslateable('Package %s is not translateable by %s' % (package,
      translator))
      pex.resolver.Untranslateable: Package
      WheelPackage('file:///Users/lawrencefinn/Documents/aiq/docutils-0.15.post1-py2-none-any.whl')
      is not translateable by ChainedTranslator(WheelTranslator, EggTranslator,
      SourceTranslator)</module>


      Status: closed-fixed
      Group: Default
      Labels: python-2
      Created: Sun Jul 21, 2019 09:56 AM UTC by Shai Berger
      Last Updated: Mon Jul 22, 2019 03:36 PM UTC
      Owner: engelbert gruber

      Hello,

      At Matific, we are using docutils as a 2nd-order dependency, and this
      morning our CI failed with these messages:

      [2019-07-21T09:23:34.566Z] File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
      [2019-07-21T09:23:34.566Z] print('\n::: Runtime settings:', file=self._stderr)
      [2019-07-21T09:23:34.566Z] ^
      [2019-07-21T09:23:34.566Z] SyntaxError: invalid syntax.

      I believe a from future import print_function is missing at the top
      of that file.


      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/docutils/bugs/365/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

       

      Related

      Bugs: #365

  • Lawrence Finn

    Lawrence Finn - 2019-07-22

    We use pants as our build tool, and it uses pex to build one self contained python (executable i think). Anyway, I compiled docutils from the source and get

    pex ~/Documents/docutils-0.15/dist/docutils-0.15-py2-none-any.whl 
    Python 2.7.16 (default, Jun 19 2019, 07:42:24) 
    [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    (InteractiveConsole)
    >>> 
    

    Which is expected. I am not sure why I get an error for the version on pypi.

     
    • engelbert gruber

      On Mon, 22 Jul 2019 at 23:03, Lawrence Finn laf2019@users.sourceforge.net
      wrote:

      We use pants as our build tool, and it uses pex to build one self
      contained python (executable i think). Anyway, I compiled docutils from the
      source and get

      pex ~/Documents/docutils-0.15/dist/docutils-0.15-py2-none-any.whl
      Python 2.7.16 (default, Jun 19 2019, 07:42:24)
      [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
      Type "help", "copyright", "credits" or "license" for more information.
      (InteractiveConsole)

      Which is expected. I am not sure why I get an error for the version on
      pypi.

      it seams pex can't handle the .post1 version part in the filename
      i build a py2-wheel (like you) and pex is able to handle it
      but not if i rename it


      Status: closed-fixed
      Group: Default
      Labels: python-2
      Created: Sun Jul 21, 2019 09:56 AM UTC by Shai Berger
      Last Updated: Mon Jul 22, 2019 07:27 PM UTC
      Owner: engelbert gruber

      Hello,

      At Matific, we are using docutils as a 2nd-order dependency, and this
      morning our CI failed with these messages:

      [2019-07-21T09:23:34.566Z] File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
      [2019-07-21T09:23:34.566Z] print('\n::: Runtime settings:', file=self._stderr)
      [2019-07-21T09:23:34.566Z] ^
      [2019-07-21T09:23:34.566Z] SyntaxError: invalid syntax.

      I believe a from future import print_function is missing at the top
      of that file.


      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/docutils/bugs/365/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

       

      Related

      Bugs: #365

      • Lawrence Finn

        Lawrence Finn - 2019-07-23

        Weird, so it is a pex only issue? I ended up pinning .14.

         
        • engelbert gruber

          On Tue, 23 Jul 2019 at 18:23, Lawrence Finn laf2019@users.sourceforge.net
          wrote:

          Weird, so it is a pex only issue? I ended up pinning .14.

          maybe you could pin != 0.15
          0.15.1 is coming soon du eto bugs#366

          pex's problem might be that the version is 0.15 only the wheel's filename
          contains .post1


          Status: closed-fixed
          Group: Default
          Labels: python-2
          Created: Sun Jul 21, 2019 09:56 AM UTC by Shai Berger
          Last Updated: Tue Jul 23, 2019 12:43 PM UTC
          Owner: engelbert gruber

          Hello,

          At Matific, we are using docutils as a 2nd-order dependency, and this
          morning our CI failed with these messages:

          [2019-07-21T09:23:34.566Z] File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
          [2019-07-21T09:23:34.566Z] print('\n::: Runtime settings:', file=self._stderr)
          [2019-07-21T09:23:34.566Z] ^
          [2019-07-21T09:23:34.566Z] SyntaxError: invalid syntax.

          I believe a from future import print_function is missing at the top
          of that file.


          Sent from sourceforge.net because you indicated interest in
          https://sourceforge.net/p/docutils/bugs/365/

          To unsubscribe from further messages, please visit
          https://sourceforge.net/auth/subscriptions/

           

          Related

          Bugs: #365

        • engelbert gruber

          On Tue, 23 Jul 2019 at 18:23, Lawrence Finn laf2019@users.sourceforge.net
          wrote:

          Weird, so it is a pex only issue? I ended up pinning .14.

          it is an issue of ... i dont know.
          pep 440 says

          Post releases allow a ., -, or _ separator as well as omitting the
          separator all together.
          The normal form of this is with the . separator.

          Post release (IMHO) is here for this case, no change in code but something
          else

          BUT when using the "normal form" [pep440] pex might be looking for
          directories named
          docutils-0.15.post1.dist-info which do not exist.

          pex accepts the same wheel named docutils-0.15-post1-py2-none-any.whl .


          Status: closed-fixed
          Group: Default
          Labels: python-2
          Created: Sun Jul 21, 2019 09:56 AM UTC by Shai Berger
          Last Updated: Tue Jul 23, 2019 12:43 PM UTC
          Owner: engelbert gruber

          Hello,

          At Matific, we are using docutils as a 2nd-order dependency, and this
          morning our CI failed with these messages:

          [2019-07-21T09:23:34.566Z] File "/usr/local/lib/python2.7/site-packages/docutils/core.py", line 246
          [2019-07-21T09:23:34.566Z] print('\n::: Runtime settings:', file=self._stderr)
          [2019-07-21T09:23:34.566Z] ^
          [2019-07-21T09:23:34.566Z] SyntaxError: invalid syntax.

          I believe a from future import print_function is missing at the top
          of that file.


          Sent from sourceforge.net because you indicated interest in
          https://sourceforge.net/p/docutils/bugs/365/

          To unsubscribe from further messages, please visit
          https://sourceforge.net/auth/subscriptions/

           

          Related

          Bugs: #365


Log in to post a comment.