From: SourceForge.net <no...@so...> - 2012-08-20 21:07:30
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Tracker Item Submitted) made by abadger1999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-21 07:07:05
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by grubert You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-21 07:41:52
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by abadger1999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-21 08:00:56
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by grubert You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-21 10:18:22
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by grubert You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-21 15:39:33
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by abadger1999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-21 15:53:10
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by abadger1999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:53 Message: Ah sorry -- I missed that the test_writer and dummy-writer are attached to this report. /me downloads them now. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-21 15:58:58
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by abadger1999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:58 Message: The test case fails in the same way as on python3 for me. Using python2.7 but I'm wondering if the difference is that you didn't set PYTHONPATH when running your python2 test? The system docutils is likely the last release so it would pass the test (using level=-1). You would need to make sure that you're getting the docutils module from the snapshot to get the traceback. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:53 Message: Ah sorry -- I missed that the test_writer and dummy-writer are attached to this report. /me downloads them now. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-21 16:02:53
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by abadger1999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 09:02 Message: $ export PYTHONPATH=/srv/git/python-docutils/docutils $ python2.7 test_get_writer_class.py (08:56:22):1225 Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/srv/git/python-docutils/docutils/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer Also note: For the actual unittest, probably want to add an else: to the try: except and assert if t[1] == 0 in the else. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:58 Message: The test case fails in the same way as on python3 for me. Using python2.7 but I'm wondering if the difference is that you didn't set PYTHONPATH when running your python2 test? The system docutils is likely the last release so it would pass the test (using level=-1). You would need to make sure that you're getting the docutils module from the snapshot to get the traceback. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:53 Message: Ah sorry -- I missed that the test_writer and dummy-writer are attached to this report. /me downloads them now. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-22 06:28:37
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by grubert You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: engelbert gruber (grubert) Date: 2012-08-21 23:28 Message: you are correct, 2.7 fails the same but 2.3 to 2.6 passes ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 09:02 Message: $ export PYTHONPATH=/srv/git/python-docutils/docutils $ python2.7 test_get_writer_class.py (08:56:22):1225 Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/srv/git/python-docutils/docutils/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer Also note: For the actual unittest, probably want to add an else: to the try: except and assert if t[1] == 0 in the else. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:58 Message: The test case fails in the same way as on python3 for me. Using python2.7 but I'm wondering if the difference is that you didn't set PYTHONPATH when running your python2 test? The system docutils is likely the last release so it would pass the test (using level=-1). You would need to make sure that you're getting the docutils module from the snapshot to get the traceback. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:53 Message: Ah sorry -- I missed that the test_writer and dummy-writer are attached to this report. /me downloads them now. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-22 15:29:11
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by abadger1999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-22 08:29 Message: Happens for me on python2.6: $ PYTHONPATH=$(pwd) $ export PYTHONPATH $ python2.6 test_get_writer_class.py Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/home/fedora/toshio/docutils-0.10/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer $ ls dummy-writer* test_get_writer* dummy-writer.py dummy-writer.pyc test_get_writer_class.py $ pwd /home/fedora/toshio/docutils-0.10/test python2.4 and below should definitely work because there's no level argument to __import__() there [So we end up using the docutils._compat.__import__ code that ignores level] ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 23:28 Message: you are correct, 2.7 fails the same but 2.3 to 2.6 passes ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 09:02 Message: $ export PYTHONPATH=/srv/git/python-docutils/docutils $ python2.7 test_get_writer_class.py (08:56:22):1225 Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/srv/git/python-docutils/docutils/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer Also note: For the actual unittest, probably want to add an else: to the try: except and assert if t[1] == 0 in the else. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:58 Message: The test case fails in the same way as on python3 for me. Using python2.7 but I'm wondering if the difference is that you didn't set PYTHONPATH when running your python2 test? The system docutils is likely the last release so it would pass the test (using level=-1). You would need to make sure that you're getting the docutils module from the snapshot to get the traceback. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:53 Message: Ah sorry -- I missed that the test_writer and dummy-writer are attached to this report. /me downloads them now. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-22 19:07:45
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by grubert You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: engelbert gruber (grubert) Date: 2012-08-22 12:07 Message: hei toshio you wrote --------- > Happens for me on python2.6: > > $ PYTHONPATH=$(pwd) > $ export PYTHONPATH > $ python2.6 test_get_writer_class.py > Traceback (most recent call last): > File "test_get_writer_class.py", line 7, in <module> > wr = get_writer_class(t[0]) > File "/home/fedora/toshio/docutils-0.10/docutils/writers/__init__.py", > line 136, in get_writer_class > module = __import__(writer_name, globals(), locals(), level=1) > ImportError: No module named dummy-writer > $ ls dummy-writer* test_get_writer* > dummy-writer.py dummy-writer.pyc test_get_writer_class.py > $ pwd > /home/fedora/toshio/docutils-0.10/test you are in "test" and set PYTHONPATH=$(pwd) ? next step for me is to get a unittest into test/test_writers (if no one objects) > python2.4 and below should definitely work because there's no level > argument to __import__() there [So we end up using the > docutils._compat.__import__ code that ignores level] i tried on linux 2.2 to 2.6 worked on mac 2.5 and 2.6 work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-22 08:29 Message: Happens for me on python2.6: $ PYTHONPATH=$(pwd) $ export PYTHONPATH $ python2.6 test_get_writer_class.py Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/home/fedora/toshio/docutils-0.10/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer $ ls dummy-writer* test_get_writer* dummy-writer.py dummy-writer.pyc test_get_writer_class.py $ pwd /home/fedora/toshio/docutils-0.10/test python2.4 and below should definitely work because there's no level argument to __import__() there [So we end up using the docutils._compat.__import__ code that ignores level] ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 23:28 Message: you are correct, 2.7 fails the same but 2.3 to 2.6 passes ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 09:02 Message: $ export PYTHONPATH=/srv/git/python-docutils/docutils $ python2.7 test_get_writer_class.py (08:56:22):1225 Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/srv/git/python-docutils/docutils/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer Also note: For the actual unittest, probably want to add an else: to the try: except and assert if t[1] == 0 in the else. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:58 Message: The test case fails in the same way as on python3 for me. Using python2.7 but I'm wondering if the difference is that you didn't set PYTHONPATH when running your python2 test? The system docutils is likely the last release so it would pass the test (using level=-1). You would need to make sure that you're getting the docutils module from the snapshot to get the traceback. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:53 Message: Ah sorry -- I missed that the test_writer and dummy-writer are attached to this report. /me downloads them now. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-22 19:15:57
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by abadger1999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- >Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-22 12:15 Message: Hmmm... Copy and paste error on my part. I am in the test directory but it was actually: $ PYTHONPATH=$(pwd)/.. $ export PYTHONPATH $ python2.6 test_get_writer_class.py ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-22 12:07 Message: hei toshio you wrote --------- > Happens for me on python2.6: > > $ PYTHONPATH=$(pwd) > $ export PYTHONPATH > $ python2.6 test_get_writer_class.py > Traceback (most recent call last): > File "test_get_writer_class.py", line 7, in <module> > wr = get_writer_class(t[0]) > File "/home/fedora/toshio/docutils-0.10/docutils/writers/__init__.py", > line 136, in get_writer_class > module = __import__(writer_name, globals(), locals(), level=1) > ImportError: No module named dummy-writer > $ ls dummy-writer* test_get_writer* > dummy-writer.py dummy-writer.pyc test_get_writer_class.py > $ pwd > /home/fedora/toshio/docutils-0.10/test you are in "test" and set PYTHONPATH=$(pwd) ? next step for me is to get a unittest into test/test_writers (if no one objects) > python2.4 and below should definitely work because there's no level > argument to __import__() there [So we end up using the > docutils._compat.__import__ code that ignores level] i tried on linux 2.2 to 2.6 worked on mac 2.5 and 2.6 work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-22 08:29 Message: Happens for me on python2.6: $ PYTHONPATH=$(pwd) $ export PYTHONPATH $ python2.6 test_get_writer_class.py Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/home/fedora/toshio/docutils-0.10/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer $ ls dummy-writer* test_get_writer* dummy-writer.py dummy-writer.pyc test_get_writer_class.py $ pwd /home/fedora/toshio/docutils-0.10/test python2.4 and below should definitely work because there's no level argument to __import__() there [So we end up using the docutils._compat.__import__ code that ignores level] ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 23:28 Message: you are correct, 2.7 fails the same but 2.3 to 2.6 passes ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 09:02 Message: $ export PYTHONPATH=/srv/git/python-docutils/docutils $ python2.7 test_get_writer_class.py (08:56:22):1225 Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/srv/git/python-docutils/docutils/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer Also note: For the actual unittest, probably want to add an else: to the try: except and assert if t[1] == 0 in the else. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:58 Message: The test case fails in the same way as on python3 for me. Using python2.7 but I'm wondering if the difference is that you didn't set PYTHONPATH when running your python2 test? The system docutils is likely the last release so it would pass the test (using level=-1). You would need to make sure that you're getting the docutils module from the snapshot to get the traceback. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:53 Message: Ah sorry -- I missed that the test_writer and dummy-writer are attached to this report. /me downloads them now. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |
From: SourceForge.net <no...@so...> - 2012-08-22 20:31:06
|
Bugs item #3559988, was opened at 2012-08-20 14:07 Message generated for change (Comment added) made by grubert You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed Resolution: None Priority: 5 Private: No Submitted By: Toshio Kuratomi (abadger1999) Assigned to: Nobody/Anonymous (nobody) Summary: Docutils 0.10 snapshot doesn't allow user created writers Initial Comment: Working on a snapshot since we're going to need to support python-3.3 for the next version of Fedora. A mercurial maintainer noticed that the behaviour of docutils has changed. mercurial has a locally created writer to take in rst and output manpages. With the snapshot they are unable to get docutils to load this: https://bugzilla.redhat.com/show_bug.cgi?id=849744 The change that made this is: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/docutils/writers/__init__.py?r1=7317&r2=7486 Which is to fix this bug on python3.3: https://sourceforge.net/tracker/?func=detail&aid=3541369&group_id=38414&atid=422030 It looks like the python-3.3 change is that python3.3 no longer allows __import__ to take a level argument of -1. That argument allowed __import__ to search both absolute and relative paths for the module. The current code fixed that by changing level to "1" which only searches a relative path for the module. This works for the modules inside of docutils because they are located in the parent package of the code that loads them. Out-of-tree modules break with this scheme because the module is not a child of the docutils package. To fix this the code can do something like this: try: module = __import__(writer_name, globals(), locals(), level=0) except ImportError: module = __import__(writer_name, globals(), locals(), level=1) What this does is first look for the module as an absolute import. This will be triggered if it's a user defined module. If no module is found as an absolute import it then looks in the parent of the module that called it. This is where all current modules shipped with docutils are called. This code is what is closest to level=-1 which is present in python2.x and python3 up to python3.2. It might make sense to revisit hardcoded's patch to issue 3541369 to implement this as the try: except is boilerplate in all the places we need to use __import__. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-22 13:31 Message: Fixed; thanks for the bug report. You can download a current snapshot from: http://docutils.svn.sourceforge.net/viewvc/docutils/trunk/docutils/?view=tar ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-22 12:15 Message: Hmmm... Copy and paste error on my part. I am in the test directory but it was actually: $ PYTHONPATH=$(pwd)/.. $ export PYTHONPATH $ python2.6 test_get_writer_class.py ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-22 12:07 Message: hei toshio you wrote --------- > Happens for me on python2.6: > > $ PYTHONPATH=$(pwd) > $ export PYTHONPATH > $ python2.6 test_get_writer_class.py > Traceback (most recent call last): > File "test_get_writer_class.py", line 7, in <module> > wr = get_writer_class(t[0]) > File "/home/fedora/toshio/docutils-0.10/docutils/writers/__init__.py", > line 136, in get_writer_class > module = __import__(writer_name, globals(), locals(), level=1) > ImportError: No module named dummy-writer > $ ls dummy-writer* test_get_writer* > dummy-writer.py dummy-writer.pyc test_get_writer_class.py > $ pwd > /home/fedora/toshio/docutils-0.10/test you are in "test" and set PYTHONPATH=$(pwd) ? next step for me is to get a unittest into test/test_writers (if no one objects) > python2.4 and below should definitely work because there's no level > argument to __import__() there [So we end up using the > docutils._compat.__import__ code that ignores level] i tried on linux 2.2 to 2.6 worked on mac 2.5 and 2.6 work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-22 08:29 Message: Happens for me on python2.6: $ PYTHONPATH=$(pwd) $ export PYTHONPATH $ python2.6 test_get_writer_class.py Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/home/fedora/toshio/docutils-0.10/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer $ ls dummy-writer* test_get_writer* dummy-writer.py dummy-writer.pyc test_get_writer_class.py $ pwd /home/fedora/toshio/docutils-0.10/test python2.4 and below should definitely work because there's no level argument to __import__() there [So we end up using the docutils._compat.__import__ code that ignores level] ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 23:28 Message: you are correct, 2.7 fails the same but 2.3 to 2.6 passes ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 09:02 Message: $ export PYTHONPATH=/srv/git/python-docutils/docutils $ python2.7 test_get_writer_class.py (08:56:22):1225 Traceback (most recent call last): File "test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/srv/git/python-docutils/docutils/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer Also note: For the actual unittest, probably want to add an else: to the try: except and assert if t[1] == 0 in the else. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:58 Message: The test case fails in the same way as on python3 for me. Using python2.7 but I'm wondering if the difference is that you didn't set PYTHONPATH when running your python2 test? The system docutils is likely the last release so it would pass the test (using level=-1). You would need to make sure that you're getting the docutils module from the snapshot to get the traceback. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:53 Message: Ah sorry -- I missed that the test_writer and dummy-writer are attached to this report. /me downloads them now. ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 08:39 Message: If you commit your test_get_writer_class.py I can take a look. Also note -- this is going to affect all the code touched by revision 7486, not just get_writer_class(). ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 03:18 Message: 1. cd docutils 2. cp test_get_writer_class.py test 3. passes:: python2.6 test/test_get_writer_class.py 4. for py3 set PYTHONPATH to build/lib:: export PYTHONPATH=`pwd`/build/lib/: cp test_get_writer_class.py test3 python3.2 test3/test_get_writer_class.py Traceback (most recent call last): File "test3/test_get_writer_class.py", line 7, in <module> wr = get_writer_class(t[0]) File "/Users/engelbert/projects/docutils/trunk/docutils/build/lib/docutils/writers/__init__.py", line 136, in get_writer_class module = __import__(writer_name, globals(), locals(), level=1) ImportError: No module named dummy-writer python3.3 is the same ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 01:00 Message: a unittest that fits into docutils tests would be the ultimate but this might be too much hassle :: cd docutils-trunk/docutils ../../cpython-HEAD/python.exe setup.py build ../../cpython-HEAD/python.exe test3/alltest.py works (4 errors in odt-writer and alot of warnings :: ResourceWarning: unclosed file <_io.TextIOWrapper name='functional/tests/_default.py' mode='r' encoding='UTF-8'> defaultpy = open(join_path(datadir, 'tests', '_default.py')).read() but docutils seams to work ---------------------------------------------------------------------- Comment By: Toshio Kuratomi (abadger1999) Date: 2012-08-21 00:41 Message: Do you want a test case suitable for a unittest or an end-to-end test? This is the simplest way I found to test: mkdir cleantest cd cleantest cp /usr/lib/python2.7/site-packages/docutils/writers/manpage.py testwriter.py python >>> from docutils.writers import get_writer_class >>> get_writer_class('manpage') <class docutils.writers.manpage.Writer at 0x7f94ee0aba78> >>> get_writer_class('testwriter') <class testwriter.Writer at 0x7f94eea0bc18> Under the current snapshot, get_writer_class('testwriter') will fail with a traceback. For end to end testing, this is how mercurial is using it: They have a script runrst and a custom writer hgmanpage.py which is a slightly modified version of the docutils builtin manpage.py. The runrst script is invoked twice. Once to create man pages using their custom hgmanpage.py writer and the second time to generate html using docutil's builtin html writer (docutils/writers/html4css1.py). The runrst script uses docutils.core.publish_cmdline() to make use of the writers. Under the current snapshot the hgmanpage.py module is not found by docutils and a traceback is raised. To see how all this works: wget http://www.selenic.com/mercurial/release/mercurial-2.3.tar.gz tar -xzvf mercurial-2.3.tar.gz cd mercurial-2.3/doc make all Note that I haven't tested this with python3.3 yet -- because of this bug: https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030 we don't yet have a docutils package built for python3.3. Reading the python3.3 documentation, the approach in my iniital comment shouldn't be using functionality that was removed from __import__() though. ---------------------------------------------------------------------- Comment By: engelbert gruber (grubert) Date: 2012-08-21 00:07 Message: 1. have to get a test 2. have to get py3.3 3. test from 2.4 to 3.3/trunk ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=422030&aid=3559988&group_id=38414 |