#13 Tests fail for python3

[Hans-Peter Jansen]

Hi Ralf,

first of all, I wanted to say thank you very much for this fine module. I'm packaging it since 11 years now, and updated it to 2.0 today: https://build.opensuse.org/package/show/home:frispete:python/python-ooopy

Yes, I know, packaging isn't that popular amongst some Python developers, but that has some nice properties, e.g. we build for a significant range of python versions on the different targets.

In that course, I tried to enforce the tests, you provided, but faced a couple of issues.
Most of them, I was able to fix, though. Have a look at the patch file ooopy-fix-setup.patch in the above link.

If you really want to make life of us packagers easier, please have a look into the %check section of spec file, but this is the gist of it:

# ooopy has some "very specific" requirements for the testing environment
pushd %{buildroot}%{_datadir}/ooopy
ln -s %{_builddir}/%{tarname}/bin .
ln -s %{_builddir}/%{tarname}/ooopy .
ls -Rla . bin/
%{python_expand 
PYTHONPATH=%{buildroot}%{$python_sitelib} $python run_doctest.py %{buildroot}%{$python_sitelib}/ooopy/Transformer.py || exit 1
PYTHONPATH=%{buildroot}%{$python_sitelib} $python run_doctest.py %{buildroot}%{$python_sitelib}/ooopy/OOoPy.py || exit 1
rm -f *.odt *stw *.sxw _current_flavor
}
rm -f bin ooopy

As you hopefully can see, I'm setting up the required environment in the datadir (/usr/share/ooopy) in order to fulfill assumptions of the test code, and have to clean that up again afterwards.

The down sides of this setup is, that the user needs to redo similar changes (as root) to reproduce this.

With such interventions in place, all tests succeed, but one build (Python 3.8 for Tumbleweed) triggers:

[    4s] + PYTHONPATH=/home/abuild/rpmbuild/BUILDROOT/python-ooopy-2.0-1.x86_64/usr/lib/python3.8/site-packages
[    4s] + python3.8 run_doctest.py /home/abuild/rpmbuild/BUILDROOT/python-ooopy-2.0-1.x86_64/usr/lib/python3.8/site-packages/ooopy/Transformer.py
[    5s] /home/abuild/rpmbuild/BUILDROOT/python-ooopy-2.0-1.x86_64/usr/lib/python3.8/site-packages/ooopy/Transformer.py fails 0 of 223 doc-tests
[    5s] Exception ignored in: <function OOoPy.close at 0x7fb62b9f5280>
[    5s] Traceback (most recent call last):
[    5s]   File "/home/abuild/rpmbuild/BUILDROOT/python-ooopy-2.0-1.x86_64/usr/share/ooopy/./ooopy/OOoPy.py", line 315, in close
[    5s]     if i : i.close ()
[    5s]   File "/usr/lib64/python3.8/zipfile.py", line 1838, in close
[    5s]     self.fp.seek(self.start_dir)
[    5s] ValueError: I/O operation on closed file.

The reason seems to be a race between close and del in the zipfile module, but I don't have a sparkling idea, how to prevent this other than rewriting the code to use a context manager, but that would be a major change in this central class.

Something with the autosuper stuff seems somewhat wrong:

>>> from ooopy.OOoPy import OOoPy
>>> for i in dir(OOoPy):
...   print('%s: %s' % (i, getattr(ooopy, i)))
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
AttributeError: module 'ooopy' has no attribute '_OOoPy__super'

Anyway, thank you again,

Best regards,
Pete

[Ralf Schlatterbeck]

Hi Pete,

On Mon, Dec 20, 2021 at 04:23:58PM -0000, Hans-Peter Jansen wrote:

first of all, I wanted to say thank you very much for this fine
module. I'm packaging it since 11 years now, and updated it to 2.0
today:
https://build.opensuse.org/package/show/home:frispete:python/python-ooopy

Thanks for packaging OOoPy!

Yes, I know, packaging isn't that popular amongst some Python
developers, but that has some nice properties, e.g. we build for a
significant range of python versions on the different targets.

Hmm it's always good when someone tests things.

In that course, I tried to enforce the tests, you provided, but faced
a couple of issues. Most of them, I was able to fix, though. Have a
look at the patch file ooopy-fix-setup.patch in the above link.

Hmm. I cannot reproduce any failing tests. My Python versions I'm
testing against are:
- Python 2.7.16
- Python 3.7.3
- Python 3.9.2

e.g.,

ralf@cat:67; python3 ./run_doctest.py ooopy/autosuper.py

ooopy/autosuper.py fails 0 of 5 doc-tests

ralf@cat:68; python3 ./run_doctest.py ooopy/Transformer.py

ooopy/Transformer.py fails 0 of 223 doc-tests

ralf@cat:69; python3 ./run_doctest.py ooopy/Transforms.py

ooopy/Transforms.py fails 0 of 0 doc-tests

ralf@cat:70; python3 ./run_doctest.py ooopy/Version.py
ooopy/Version.py fails 0 of 0 doc-tests

If you really want to make life of us packagers easier, please have a
look into the %check section of spec file, but this is the gist of it:

Have you a suggestion that would make life easier for you?
I could read environment variables for configuring the test directories
for example. Would that help? I usually run the tests before installing
in my sandbox but understand the wish to be able to test in the
installed environment, too.

With such interventions in place, all tests succeed, but one build
(Python 3.8 for Tumbleweed) triggers:

So this is only for 3.8?
Unfortunately Debian (which I'm mostly developing against) has skipped
3.8 altogether.

The reason seems to be a race between close and del in the zipfile
module, but I don't have a sparkling idea, how to prevent this other
than rewriting the code to use a context manager, but that would be a
major change in this central class.

Hmm that should not occur from the Traceback it's not immediately
obvious where that error lies.

Something with the autosuper stuff seems somewhat wrong:
```

from ooopy.OOoPy import OOoPy
for i in dir(OOoPy):
... print('%s: %s' % (i, getattr(ooopy, i)))
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
AttributeError: module 'ooopy' has no attribute '_OOoPy__super'
```</module></stdin>

Do you mean

...
        print ("%s: %s" % (i, getattr (OOoPy, i)))

(note the case of OOoPy)?

This produces no AttributeError for me. If I import ooopy in addition to
the 'from ooopy.OOoPy import OOoPy' above I do get the same error. But
this is expected because ooopy doesn't have _OOoPy__super. If I only do
the 'from ooopy.OOoPy import OOoPy' I'm getting:
NameError: name 'ooopy' is not defined

So this might be a bug in your test.

I have not looked into your patches yet, can you give me a hint what
they are supposed to fix?

Thanks!
Ralf
--
Ralf Schlatterbeck email: ralf@zoo.priv.at

[Ralf Schlatterbeck]

I've just pushed some changes, can you check if this fixes the traceback with python3.8 you're seeing? This is probably related to https://bugs.python.org/issue37773 and means one should never leave closing of zip files to the garbage collector.
Note that I'm now maintaining a copy on github, too.
I've also applied your patch to setup.py, thanks!
Now if we can come up with a way how you can more easily run the doctests, that would be fine.