## docutils-users

 [Docutils-users] FileInput exiting on IOError From: Eli Bendersky - 2010-12-28 09:14:47 I'm using docutils programmatically for converting ReST to HTML (specifically docutils.core.publish_from_doctree). Since the usage is programmatic (controlled by my application) I must have full control of exceptions for logging. However, I noticed that the FileInput class used by docutils for various purposes exits the application (sys.exit) when it can't find a file. Reraising the IOError seems possible by passing in handle_io_errors=False, but no user-accessible publishing API seems to expose that. Am I missing something, or is this a valid feature request? Thanks in advance, and happy holidays 
 Re: [Docutils-users] FileInput exiting on IOError From: Guenter Milde - 2012-01-20 10:01:44 Dear docutils developers, I did not take action on this message when it appeared on year ago (and noone else did), but in the light of a second use case (the Leo outliner), it seems to me that a decision should be reached: David, what do you think? On 2010-12-28, Eli Bendersky wrote: > I'm using docutils programmatically for converting ReST to HTML > (specifically docutils.core.publish_from_doctree). Since the usage is > programmatic (controlled by my application) I must have full control of > exceptions for logging. However, I noticed that the FileInput class > used by docutils for various purposes exits the application (sys.exit) > when it can't find a file. > Reraising the IOError seems possible by passing in > handle_io_errors=False, but no user-accessible publishing API seems to > expose that. > Am I missing something, or is this a valid feature request? To me, this seems a valid feature request. I suppose the default behaviour of docutils.io.FileInput was defined to get the simple:: IOError: [Errno 2] Datei oder Verzeichnis nicht gefunden: u'nonexistent.txt' Unable to open source file for reading ('nonexistent.txt'). Exiting. instead of :: IOError: [Errno 2] Datei oder Verzeichnis nicht gefunden: 'nonexistent.txt' Exiting due to error. Use "--traceback" to diagnose. Please report errors to . Include "--traceback" output, Docutils version (0.9 [repository]), Python version (2.7.2+), your OS type & version, and the command line used. when calling the front-ends with a non-accessible file. The easiest fix (with the above side-effect of a distracting error message) would be to change the default behaviour: --- io.py (Revision 7307) +++ io.py (Arbeitskopie) @@ -184,7 +184,7 @@ """ def __init__(self, source=None, source_path=None, encoding=None, error_handler='strict', - autoclose=True, handle_io_errors=True, mode='rU'): + autoclose=True, handle_io_errors=False, mode='rU'): """ :Parameters: - source: either a file-like object (which is read directly), or For a nice user experience this should be combined with a better response to missing "main" input files (catching the error and reporting as "severe" system message at an more appropriate place). Thanks, Günter 
 Re: [Docutils-users] FileInput exiting on IOError From: David Goodger - 2012-01-20 14:06:11 On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: > Dear docutils developers, > > I did not take action on this message when it appeared on year ago (and > noone else did), but in the light of a second use case (the Leo outliner), > it seems to me that a decision should be reached: > > David, what do you think? > > On 2010-12-28, Eli Bendersky wrote: >> I'm using docutils programmatically for converting ReST to HTML >> (specifically docutils.core.publish_from_doctree). Since the usage is >> programmatic (controlled by my application) I must have full control of >> exceptions for logging. However, I noticed that the FileInput class >> used by docutils for various purposes exits the application (sys.exit) >> when it can't find a file. > >> Reraising the IOError seems possible by passing in >> handle_io_errors=False, but no user-accessible publishing API seems to >> expose that. > >> Am I missing something, or is this a valid feature request? > > To me, this seems a valid feature request. > > I suppose the default behaviour of docutils.io.FileInput was defined to > get the simple:: > >  IOError: [Errno 2] Datei oder Verzeichnis nicht gefunden: u'nonexistent.txt' >  Unable to open source file for reading ('nonexistent.txt'). Exiting. > > instead of :: > >  IOError: [Errno 2] Datei oder Verzeichnis nicht gefunden: 'nonexistent.txt' >  Exiting due to error.  Use "--traceback" to diagnose. >  Please report errors to . >  Include "--traceback" output, Docutils version (0.9 [repository]), >  Python version (2.7.2+), your OS type & version, and the >  command line used. > > > when calling the front-ends with a non-accessible file. Yes. > The easiest fix (with the above side-effect of a distracting > error message) would be to change the default behaviour: > > --- io.py       (Revision 7307) > +++ io.py       (Arbeitskopie) > @@ -184,7 +184,7 @@ >     """ >     def __init__(self, source=None, source_path=None, >                  encoding=None, error_handler='strict', > -                 autoclose=True, handle_io_errors=True, mode='rU'): > +                 autoclose=True, handle_io_errors=False, mode='rU'): >         """ >         :Parameters: >             - source: either a file-like object (which is read directly), or -1 on this change. If we only do this, we'll be bombarded by error reports from users who mis-typed their command lines. This is a *feature*. The issue seems to be that the handle_io_errors parameter is not accessible from client code. Turning it off doesn't make it accessible. Either we should make it accessible, or remove it completely and handle errors elsewhere. > For a nice user experience this should be combined with a better response > to missing "main" input files (catching the error and reporting as > "severe" system message at an more appropriate place). I think the current error handling *is* at the most appropriate place. If you disagree, please suggest a place to do the error handling. The problem is, removing error handling from there would require duplicating code in multiple places elsewhere. Instead, how about adding a command-line/config option to control this? Then programmatic users can handle any exceptions themselves, and regular users won't see any change. Problem: only docutils.io.FileInput & FileOutput have handle_io_errors parameters, none of the other docutils.io classes do (they don't need them, but would were we to generalize). We don't currently pass settings objects to the docutils.io classes, so we have two choices. Either add settings parameters to all appropriate docutils.io classes, or add handle_io_errors parameters to the StringInput & Output classes. I vote for the latter. Either way, we'll have to change calling code to pass in appropriate values depending on the settings value. -- David Goodger ; 
 Re: [Docutils-users] FileInput exiting on IOError From: Guenter Milde - 2012-01-22 20:45:37 On 2012-01-20, David Goodger wrote: > On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >> On 2010-12-28, Eli Bendersky wrote: >>> I'm using docutils programmatically for converting ReST to HTML >>> (specifically docutils.core.publish_from_doctree). Since the usage is >>> programmatic (controlled by my application) I must have full control of >>> exceptions for logging. However, I noticed that the FileInput class >>> used by docutils for various purposes exits the application (sys.exit) >>> when it can't find a file. ... >> The easiest fix (with the above side-effect of a distracting >> error message) would be to change the default behaviour: ... >> -                 autoclose=True, handle_io_errors=True, mode='rU'): >> +                 autoclose=True, handle_io_errors=False, mode='rU'): ... > -1 on this change. If we only do this, we'll be bombarded by error > reports from users who mis-typed their command lines. This is a > *feature*. I agree that this should be only part of the solution. > The issue seems to be that the handle_io_errors parameter is not > accessible from client code. Turning it off doesn't make it > accessible. Either we should make it accessible, or remove it > completely and handle errors elsewhere. >> For a nice user experience this should be combined with a better response >> to missing "main" input files (catching the error and reporting as >> "severe" system message at an more appropriate place). > I think the current error handling *is* at the most appropriate place. > If you disagree, please suggest a place to do the error handling. The > problem is, removing error handling from there would require > duplicating code in multiple places elsewhere. IMV, the right place is core.Publisher.report_Exceptions. Please see my patch below for a possible implementation. > Instead, how about adding a command-line/config option to control > this? Then programmatic users can handle any exceptions themselves, > and regular users won't see any change. I don't think that an additional config setting is necessary, as the "traceback" setting already switches between passing the exception to the caller or catching and reporting exceptions by core.Publisher.report_Exceptions. > Problem: only docutils.io.FileInput & FileOutput have handle_io_errors > parameters, none of the other docutils.io classes do (they don't need > them, but would were we to generalize). We don't currently pass > settings objects to the docutils.io classes, so we have two choices. > Either add settings parameters to all appropriate docutils.io classes, > or add handle_io_errors parameters to the StringInput & Output > classes. I vote for the latter. Either way, we'll have to change > calling code to pass in appropriate values depending on the settings > value. With "handle_io_errors" for all docutils.io classes, we could pass :: handle_io_errors=not(settings.traceback) to the io.FileInput/Output classes in core.Publisher.set_source() and core.Publisher.set_destination. This also allows to change the default value to False, as throughout docutils FileInput is called 5x with handle_io_errors=False and specific error handling but only one time with handle_io_errors=True. However, I prefer to (first deprecate and later) remove the handle_io_errors argument from io.File(In|Out)put and raise specific exceptions instead. This enables core.Publisher.report_Exceptions() to clear these errors. This way, error handling is done in one place and it is not necessary to pass arguments around with every instantiation of FileInput or FileOutput. Instead, using one of the io.File* classes for file input/output will "automacigally" do the right thing for both, programmatic and command line use. The following patch does not (yet) remove the error-handling code in docutils.io, nor the "handle_io_errors" argument from the calls elsewhere. The 10 lines of additional code are more than offset by 15 lines of obsoleted code. --- io.py (Revision 7307) +++ io.py (Arbeitskopie) @@ -17,6 +17,11 @@ from docutils._compat import b from docutils.error_reporting import locale_encoding, ErrorString, ErrorOutput + +class InputError(IOError): pass +class OutputError(IOError): pass + + class Input(TransformSpec): """ @@ -184,7 +189,7 @@ """ def __init__(self, source=None, source_path=None, encoding=None, error_handler='strict', - autoclose=True, handle_io_errors=True, mode='rU'): + autoclose=True, handle_io_errors=False, mode='rU'): """ :Parameters: - source: either a file-like object (which is read directly), or @@ -217,7 +222,8 @@ self.source = open(source_path, mode, **kwargs) except IOError, error: if not handle_io_errors: - raise + raise InputError(error.errno, error.strerror, + source_path) print >>self._stderr, ErrorString(error) print >>self._stderr, (u'Unable to open source' u" file for reading ('%s'). Exiting." % source_path) @@ -286,7 +292,7 @@ def __init__(self, destination=None, destination_path=None, encoding=None, error_handler='strict', autoclose=True, - handle_io_errors=True): + handle_io_errors=False): """ :Parameters: - destination: either a file-like object (which is written @@ -327,7 +333,8 @@ self.destination = open(self.destination_path, 'w', **kwargs) except IOError, error: if not self.handle_io_errors: - raise + raise OutputError(error.errno, error.strerror, + self.destination_path) print >>self._stderr, ErrorString(error) print >>self._stderr, (u'Unable to open destination file' u" for writing ('%s'). Exiting." % self.destination_path) --- core.py (Revision 7320) +++ core.py (Arbeitskopie) @@ -260,6 +261,10 @@ self.report_SystemMessage(error) elif isinstance(error, UnicodeEncodeError): self.report_UnicodeError(error) + elif isinstance(error, io.InputError): + self.report_InputError(error) + elif isinstance(error, io.OutputError): + self.report_OutputError(error) else: print >>self._stderr, u'%s' % ErrorString(error) print >>self._stderr, ("""\ @@ -275,6 +280,14 @@ % (error.level, utils.Reporter.levels[error.level])) + def report_InputError(self, error): + self._stderr.write(u'Unable to open source file for reading:\n' + u' %s\n' % ErrorString(error)) + + def report_OutputError(self, error): + self._stderr.write(u'Unable to open destination file for writing:\n' + u' %s\n' % ErrorString(error)) + def report_UnicodeError(self, error): data = error.object[error.start:error.end] self._stderr.write( Thanks, Günter 
 Re: [Docutils-users] FileInput exiting on IOError From: David Goodger - 2012-03-12 03:44:09 On Sun, Jan 22, 2012 at 15:45, Guenter Milde wrote: > On 2012-01-20, David Goodger wrote: >> On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >>> On 2010-12-28, Eli Bendersky wrote: > >>>> I'm using docutils programmatically for converting ReST to HTML >>>> (specifically docutils.core.publish_from_doctree). Since the usage is >>>> programmatic (controlled by my application) I must have full control of >>>> exceptions for logging. However, I noticed that the FileInput class >>>> used by docutils for various purposes exits the application (sys.exit) >>>> when it can't find a file. > > ... > >>> The easiest fix (with the above side-effect of a distracting >>> error message) would be to change the default behaviour: > ... >>> -                 autoclose=True, handle_io_errors=True, mode='rU'): >>> +                 autoclose=True, handle_io_errors=False, mode='rU'): > ... > >> -1 on this change. If we only do this, we'll be bombarded by error >> reports from users who mis-typed their command lines. This is a >> *feature*. > > I agree that this should be only part of the solution. > >> The issue seems to be that the handle_io_errors parameter is not >> accessible from client code. Turning it off doesn't make it >> accessible. Either we should make it accessible, or remove it >> completely and handle errors elsewhere. > >>> For a nice user experience this should be combined with a better response >>> to missing "main" input files (catching the error and reporting as >>> "severe" system message at an more appropriate place). > >> I think the current error handling *is* at the most appropriate place. >> If you disagree, please suggest a place to do the error handling. The >> problem is, removing error handling from there would require >> duplicating code in multiple places elsewhere. > > IMV, the right place is core.Publisher.report_Exceptions. > Please see my patch below for a possible implementation. > >> Instead, how about adding a command-line/config option to control >> this? Then programmatic users can handle any exceptions themselves, >> and regular users won't see any change. > > I don't think that an additional config setting is necessary, as the > "traceback" setting already switches between passing the exception to the > caller or catching and reporting exceptions by > core.Publisher.report_Exceptions. OK >> Problem: only docutils.io.FileInput & FileOutput have handle_io_errors >> parameters, none of the other docutils.io classes do (they don't need >> them, but would were we to generalize). We don't currently pass >> settings objects to the docutils.io classes, so we have two choices. >> Either add settings parameters to all appropriate docutils.io classes, >> or add  handle_io_errors parameters to the StringInput & Output >> classes. I vote for the latter. Either way, we'll have to change >> calling code to pass in appropriate values depending on the settings >> value. > > With "handle_io_errors" for all docutils.io classes, we could pass :: > >   handle_io_errors=not(settings.traceback) > > to the io.FileInput/Output classes in core.Publisher.set_source() and > core.Publisher.set_destination. Sure. > This also allows to change the default value to False, as throughout > docutils FileInput is called 5x with handle_io_errors=False and specific > error handling but only one time with handle_io_errors=True. Hold on. Best not to flip the default like that. There could be 3rd-party code out there that uses it. > However, I prefer to (first deprecate and later) remove the > handle_io_errors argument from io.File(In|Out)put and raise specific > exceptions instead. This enables core.Publisher.report_Exceptions() > to clear these errors. If we deprecate handle_io_errors, let's not change its default first. Any code out there that depends on the default *will break*, and that's unacceptable. It would be better to rip out the error-handling code altogether (I'm not suggesting we do this though). I'm worried that reversing the default will cause third-party code to break without warning. Maybe there is no such code, but we have to assume that there is, and we can't just reverse the default without any indication that there has been a change. It's hard to articulate, but this approach just feels *wrong*. Idea: change the default to None. Check for True and False values, and issue a deprecation warning for *both* cases. Docutils itself uses that code, and will have to be fixed. Eventually, remove the handle_io_errors parameter altogether. > This way, error handling is done in one place and it is not > necessary to pass arguments around with every instantiation of > FileInput or FileOutput. Instead, using one of the io.File* classes > for file input/output will "automacigally" do the right thing for > both, programmatic and command line use. In general I agree. > The following patch does not (yet) remove the error-handling code in > docutils.io, nor the "handle_io_errors" argument from the calls > elsewhere. The 10 lines of additional code are more than offset by 15 > lines of obsoleted code. Let's add a deprecation warning before removing anything. To make such a change without warning would be very rude and feels wrong. How about starting a branch for this? It's complicated enough that it would be useful for collaboration & thorough testing. -- David Goodger ; 
 Re: [Docutils-users] FileInput exiting on IOError From: Guenter Milde - 2012-02-28 13:39:13 Dear David, I am still waiting for a consensus regarding the IOError handling. On 2012-01-22, Guenter Milde wrote: > On 2012-01-20, David Goodger wrote: >> On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >>> On 2010-12-28, Eli Bendersky wrote: Maybe we can start with the simpler decisions: >>>> I'm using docutils programmatically for converting ReST to HTML >>>> (specifically docutils.core.publish_from_doctree). Since the usage is >>>> programmatic (controlled by my application) I must have full control of >>>> exceptions for logging. However, I noticed that the FileInput class >>>> used by docutils for various purposes exits the application (sys.exit) >>>> when it can't find a file. IMO this is a valid bug report. There should be an option to suppress the conversion of IOError into sys.exit() (especially for programmatic use). ... >> Instead, how about adding a command-line/config option to control >> this? Then programmatic users can handle any exceptions themselves, >> and regular users won't see any change. > I don't think that an additional config setting is necessary. The > "traceback" setting already switches between passing exceptions to the > caller or catching and reporting exceptions by > core.Publisher.report_Exceptions. Could you agree on using the "traceback" setting for IOErrors, too? Regular users would only see a change, if they use the --traceback option. Tey get a traceback instead of the short error message, e.g. Traceback (most recent call last): File "/home/milde/.bin/rst2html", line 23, in ... docutils.io.InputError: [Errno 2] Datei oder Verzeichnis nicht gefunden: u'foffy' I hope you can agree on these two points and look forward to discussing the implementation. Günter 
 Re: [Docutils-users] FileInput exiting on IOError From: David Goodger - 2012-03-12 03:41:36 On Tue, Feb 28, 2012 at 08:38, Guenter Milde wrote: > Dear David, > > I am still waiting for a consensus regarding the IOError handling. I'm sorry about that. I wrote most of a reply some time ago, but was troubled about part of it, and then life got in the way. I had a big problem with the patch as-is, and felt that a discussion first was better. I apologize for the long delay. > On 2012-01-22, Guenter Milde wrote: >> On 2012-01-20, David Goodger wrote: >>> On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >>>> On 2010-12-28, Eli Bendersky wrote: > > Maybe we can start with the simpler decisions: > >>>>> I'm using docutils programmatically for converting ReST to HTML >>>>> (specifically docutils.core.publish_from_doctree). Since the >>>>> usage is programmatic (controlled by my application) I must have >>>>> full control of exceptions for logging. However, I noticed that >>>>> the FileInput class used by docutils for various purposes exits >>>>> the application (sys.exit) when it can't find a file. > > IMO this is a valid bug report. There should be an option to > suppress the conversion of IOError into sys.exit() (especially for > programmatic use). I agree. >>> Instead, how about adding a command-line/config option to control >>> this? Then programmatic users can handle any exceptions themselves, >>> and regular users won't see any change. > >> I don't think that an additional config setting is necessary. The >> "traceback" setting already switches between passing exceptions to the >> caller or catching and reporting exceptions by >> core.Publisher.report_Exceptions. > > Could you agree on using the "traceback" setting for IOErrors, too? Yes, that makes sense. > Regular users would only see a change, if they use the --traceback > option. Tey get a traceback instead of the short error message, e.g. > >  Traceback (most recent call last): >    File "/home/milde/.bin/rst2html", line 23, in >      ... >  docutils.io.InputError: [Errno 2] Datei oder Verzeichnis nicht gefunden: u'foffy' > > I hope you can agree on these two points and look forward to > discussing the implementation. My now-completed earlier reply will follow. -- David Goodger ; 
 Re: [Docutils-users] FileInput exiting on IOError From: Guenter Milde - 2012-03-13 20:21:40 On 2012-03-12, David Goodger wrote: > On Sun, Jan 22, 2012 at 15:45, Guenter Milde wrote: >> On 2012-01-20, David Goodger wrote: >>> On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >>>> On 2010-12-28, Eli Bendersky wrote: >>>>> I'm using docutils programmatically for converting ReST to HTML >>>>> (specifically docutils.core.publish_from_doctree). Since the usage is >>>>> programmatic (controlled by my application) I must have full control of >>>>> exceptions for logging. However, I noticed that the FileInput class >>>>> used by docutils for various purposes exits the application (sys.exit) >>>>> when it can't find a file. >> ... >>> Problem: only docutils.io.FileInput & FileOutput have handle_io_errors >>> parameters, none of the other docutils.io classes do (they don't need >>> them, but would were we to generalize). We don't currently pass >>> settings objects to the docutils.io classes, so we have two choices. >>> Either add settings parameters to all appropriate docutils.io classes, >>> or add  handle_io_errors parameters to the StringInput & Output >>> classes. I vote for the latter. Either way, we'll have to change >>> calling code to pass in appropriate values depending on the settings >>> value. >> With "handle_io_errors" for all docutils.io classes, we could pass :: >>   handle_io_errors=not(settings.traceback) >> to the io.FileInput/Output classes in core.Publisher.set_source() and >> core.Publisher.set_destination. > Sure. >> This also allows to change the default value to False, as throughout >> docutils FileInput is called 5x with handle_io_errors=False and specific >> error handling but only one time with handle_io_errors=True. > Hold on. Best not to flip the default like that. There could be > 3rd-party code out there that uses it. I checked the sandbox and Sphinx but there may be more, of course. But see below for the consequences for this code. >> However, I prefer to (first deprecate and later) remove the >> handle_io_errors argument from io.File(In|Out)put and raise specific >> exceptions instead. This enables core.Publisher.report_Exceptions() >> to clear these errors. > If we deprecate handle_io_errors, let's not change its default first. > Any code out there that depends on the default *will break*, and > that's unacceptable. It would be better to rip out the error-handling > code altogether (I'm not suggesting we do this though). As the current default is handle_io_errors=True, ripping out the error-handling altogether implies changing the default behaviour. Nevertheless, I'm suggesting that we do this: Any code out there that expects the default *will continue to work*: * without IOErrors, everything stays as-is, * with IOErrors, instead of sys.exit()+short error message there will be exit + stack trace. I would not call a longer error message *breaking*. > I'm worried that reversing the default will cause third-party code to > break without warning. Maybe there is no such code, but we have to > assume that there is, and we can't just reverse the default without > any indication that there has been a change. > It's hard to articulate, but this approach just feels *wrong*. I agree with you that this is far from ideal. > Idea: change the default to None. Check for True and False values, > and issue a deprecation warning for *both* cases. Docutils itself > uses that code, and will have to be fixed. Eventually, remove the > handle_io_errors parameter altogether. The problem is, that to be useful, we need * a deprecation warning for the default case (because it changed) but * no deprecation warning for Docutils (which must use the default unless we add (and later remove) the handle_io_errors parameter to all IO-classes. One possibility would be to add the deprecation warning also to the error string of the Input/OutputErrors, i.e. in io.py:: class FileInput(): ... deprecation_warning = ... ... except IOError, error: raise InputError(error.errno, error.strerror+deprecation_warning, source_path) as the error string is not used in core.report_Exception(), the deprecation_warning will only be visible with traceback==True. >> This way, error handling is done in one place and it is not >> necessary to pass arguments around with every instantiation of >> FileInput or FileOutput. Instead, using one of the io.File* classes >> for file input/output will "automacigally" do the right thing for >> both, programmatic and command line use. > In general I agree. >> The following patch does not (yet) remove the error-handling code in >> docutils.io, nor the "handle_io_errors" argument from the calls >> elsewhere. The 10 lines of additional code are more than offset by 15 >> lines of obsoleted code. > Let's add a deprecation warning before removing anything. To make > such a change without warning would be very rude and feels wrong. OK. > How about starting a branch for this? It's complicated enough that it > would be useful for collaboration & thorough testing. I am not sure whether this is really worth the effort. While in Git, branching is fast and easy, it is a pain in SVN. Günter 
 Re: [Docutils-users] FileInput exiting on IOError From: Guenter Milde - 2012-03-20 07:53:01 On 2012-03-13, Guenter Milde wrote: > On 2012-03-12, David Goodger wrote: >> On Sun, Jan 22, 2012 at 15:45, Guenter Milde wrote: >>> On 2012-01-20, David Goodger wrote: >>>> On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >>>>> On 2010-12-28, Eli Bendersky wrote: >>>>>> I'm using docutils programmatically for converting ReST to HTML >>>>>> (specifically docutils.core.publish_from_doctree). Since the usage is >>>>>> programmatic (controlled by my application) I must have full control of >>>>>> exceptions for logging. However, I noticed that the FileInput class >>>>>> used by docutils for various purposes exits the application (sys.exit) >>>>>> when it can't find a file. This is fixed now without changing the default behaviour of docutils.io.FileInput/FileOutput. >>> ... >>>> Problem: only docutils.io.FileInput & FileOutput have handle_io_errors >>>> parameters, none of the other docutils.io classes do (they don't need >>>> them, but would were we to generalize). We don't currently pass >>>> settings objects to the docutils.io classes, so we have two choices. Third way: in core.Publisher, pass "handle_io_errors=False" in a try/except clause. This buys us time to clean up later: ... > As the current default is handle_io_errors=True, ripping out the > error-handling altogether implies changing the default behaviour. However, changing the default behaviour (without removing the option) has only minor consequences for 3rd party code (no change without IOError, traceback instead of system exit with IOError). This should be tolerable after an announcement. OTOH, removing the handle_io_errors option will seriously break any code that currently uses it to bypass the system exit. Therefore this should only be done after a longer transition period with "active deprecation warning:" >> Idea: change the default to None. Check for True and False values, >> and issue a deprecation warning for *both* cases. Docutils itself >> uses that code, and will have to be fixed. Eventually, remove the >> handle_io_errors parameter altogether. Therefore, I suggest the following roadmap: 0.9: announce change of default behaviour and deprecation of handle_io_errors to the Future changes section in the RELEASE-NOTES. 0.10: remove "system-exit on IOError" from docutils.io.FileInput/FileOutput remove handle_io_errors=False from code in docutils 0.11: RELEASE-NOTES: announce removement of handle_io_errors option io.py: write deprecation warning to stderr if it is True or False 0.12 or 0.13: remove the handle_io_errors option. Günter 
 Re: [Docutils-users] FileInput exiting on IOError From: David Goodger - 2012-03-30 14:24:06 On Tue, Mar 20, 2012 at 03:52, Guenter Milde wrote: > On 2012-03-13, Guenter Milde wrote: >> On 2012-03-12, David Goodger wrote: >>> On Sun, Jan 22, 2012 at 15:45, Guenter Milde wrote: >>>> On 2012-01-20, David Goodger wrote: >>>>> On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >>>>>> On 2010-12-28, Eli Bendersky wrote: > >>>>>>> I'm using docutils programmatically for converting ReST to HTML >>>>>>> (specifically docutils.core.publish_from_doctree). Since the usage is >>>>>>> programmatic (controlled by my application) I must have full control of >>>>>>> exceptions for logging. However, I noticed that the FileInput class >>>>>>> used by docutils for various purposes exits the application (sys.exit) >>>>>>> when it can't find a file. > > This is fixed now without changing the default behaviour of > docutils.io.FileInput/FileOutput. > >>>> ... > > >>>>> Problem: only docutils.io.FileInput & FileOutput have handle_io_errors >>>>> parameters, none of the other docutils.io classes do (they don't need >>>>> them, but would were we to generalize). We don't currently pass >>>>> settings objects to the docutils.io classes, so we have two choices. > > Third way: in core.Publisher, pass "handle_io_errors=False" in a > try/except clause. This buys us time to clean up later: > > ... > > >> As the current default is handle_io_errors=True, ripping out the >> error-handling altogether implies changing the default behaviour. > > However, changing the default behaviour (without removing the option) has > only minor consequences for 3rd party code (no change without IOError, > traceback instead of system exit with IOError). This should be tolerable > after an announcement. > > OTOH, removing the handle_io_errors option will seriously break any > code that currently uses it to bypass the system exit. Therefore this should > only be done after a longer transition period with "active deprecation > warning:" > >>> Idea: change the default to None.  Check for True and False values, >>> and issue a deprecation warning for *both* cases.  Docutils itself >>> uses that code, and will have to be fixed.  Eventually, remove the >>> handle_io_errors parameter altogether. > > Therefore, I suggest the following roadmap: I recommend moving more aggressively, especially since our releases are not frequent. > 0.9: announce change of default behaviour and deprecation of >     handle_io_errors to the Future changes section in the >     RELEASE-NOTES. > > 0.10: remove "system-exit on IOError" from docutils.io.FileInput/FileOutput >      remove handle_io_errors=False from code in docutils I suggest cleaning up all Docutils code now. > 0.11: RELEASE-NOTES: announce removement of handle_io_errors option >      io.py: write deprecation warning to stderr if it is True or False Also, why not add the deprecation warning now? > 0.12 or 0.13: remove the handle_io_errors option. Why not move this to 0.10? One release with the deprecation warnings, the next release with the full fix in place. -- David Goodger ; 
 Re: [Docutils-users] FileInput exiting on IOError From: Guenter Milde - 2012-03-30 15:23:56 On 2012-03-30, David Goodger wrote: > On Tue, Mar 20, 2012 at 03:52, Guenter Milde wrote: >> On 2012-03-13, Guenter Milde wrote: >>> On 2012-03-12, David Goodger wrote: >>>> On Sun, Jan 22, 2012 at 15:45, Guenter Milde wrote: >>>>> On 2012-01-20, David Goodger wrote: >>>>>> On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >>>>>>> On 2010-12-28, Eli Bendersky wrote: >> ... >>> As the current default is handle_io_errors=True, ripping out the >>> error-handling altogether implies changing the default behaviour. >> Therefore, I suggest the following roadmap: > I recommend moving more aggressively, especially since our releases > are not frequent. >> 0.9: announce change of default behaviour and deprecation of >>     handle_io_errors to the Future changes section in the >>     RELEASE-NOTES. >> 0.10: remove "system-exit on IOError" from docutils.io.FileInput/FileOutput >>      remove handle_io_errors=False from code in docutils > I suggest cleaning up all Docutils code now. Cleaning up Docutils code is only possible, if we change the default behaviour. I tried this but was ordered to reverse :-( >> 0.11: RELEASE-NOTES: announce removement of handle_io_errors option >>      io.py: write deprecation warning to stderr if it is True or False > Also, why not add the deprecation warning now? Because the deprection warning is annoying if you have no choice to comply! Code that wants to bypass the "system exit on IOError" default behaviour needs to set handle_io_errors=False. 3rd-party code requires this setting to support Docutils versions with the old default, i.e. up to 0.9. >> 0.12 or 0.13: remove the handle_io_errors option. > Why not move this to 0.10? One release with the deprecation warnings, > the next release with the full fix in place. Because then it is no longer for 3rd party code to support older Docutils versions (short of checking for the version in an if clause). Removing the option really breaks 3rd party code. Not only with a traceback (instead of short message) in case of errors, but with a TypeError "got an unexpected keyword argument" in any case! Keeping an ignored option for a while seems less intrusive. Günter 
 Re: [Docutils-users] FileInput exiting on IOError From: Guenter Milde - 2012-03-30 15:50:29 On 2012-03-30, Guenter Milde wrote: > On 2012-03-30, David Goodger wrote: >> On Tue, Mar 20, 2012 at 03:52, Guenter Milde wrote: >>> On 2012-03-13, Guenter Milde wrote: >>>> On 2012-03-12, David Goodger wrote: >>>>> On Sun, Jan 22, 2012 at 15:45, Guenter Milde wrote: >>>>>> On 2012-01-20, David Goodger wrote: >>>>>>> On Fri, Jan 20, 2012 at 05:01, Guenter Milde wrote: >>>>>>>> On 2010-12-28, Eli Bendersky wrote: >>> ... >>>> As the current default is handle_io_errors=True, ripping out the >>>> error-handling altogether implies changing the default behaviour. >>> Therefore, I suggest the following roadmap: >> I recommend moving more aggressively, especially since our releases >> are not frequent. >>> 0.9: announce change of default behaviour and deprecation of >>>     handle_io_errors to the Future changes section in the >>>     RELEASE-NOTES. >>> 0.10: remove "system-exit on IOError" from docutils.io.FileInput/FileOutput >>>      remove handle_io_errors=False from code in docutils >> I suggest cleaning up all Docutils code now. > Cleaning up Docutils code is only possible, if we change the default > behaviour. I tried this but was ordered to reverse :-( >>> 0.11: RELEASE-NOTES: announce removement of handle_io_errors option >>>      io.py: write deprecation warning to stderr if it is True or False >> Also, why not add the deprecation warning now? > Because the deprection warning is annoying if you have no choice to comply! > Code that wants to bypass the "system exit on IOError" default behaviour > needs to set handle_io_errors=False. 3rd-party code requires this > setting to support Docutils versions with the old default, i.e. up to 0.9. >>> 0.12 or 0.13: remove the handle_io_errors option. >> Why not move this to 0.10? One release with the deprecation warnings, >> the next release with the full fix in place. > Because then 3rd party code cannot support older Docutils versions > (short of checking for the version in an if clause). > Removing the option really breaks 3rd party code. Not only with a > traceback (instead of short message) in case of IOErrors, but with a > TypeError "got an unexpected keyword argument" in any case! > Keeping an ignored option for a while seems less intrusive. I prefer a phase when it is possible to easily support both, the old and the new implementation. New suggestion: :0.10: change of default behaviour to the equivalent of - handle_io_errors=False and deprecation of the - handle_io_errors option, - :0.11: deprecation warning to stderr if FileInput/FileOutput - is called with handle_io_errors, - :0.12: ignore handle_io_errors=True, - :0.13: remove the handle_io_errors option. + handle_io_errors=False, + ignore and deprecate the handle_io_errors option. + (allows us to clean up Docutils code and remove the error handling + code from the FileInput/FileOutput classes) + :0.10 + n: deprecation warning to stderr if FileInput/FileOutput + is called with handle_io_errors, + :0.10 + n+1: remove the handle_io_errors option. This way, 3rd-party code can * wait for release 10 before implementing any change. * use :: try: egg = FileInput('spam', handle_io_errors=False) except IOError: to support versions before 0.10 as well as up to 0.10 + n + 1 * or change somewhere to :: try: egg = FileInput('spam') except IOError: between versions 0.10 and 0.10 + n for a backwards-incompatibel but future-proof change. Günter 
 Re: [Docutils-users] FileInput exiting on IOError From: Peter Funk - 2012-03-31 11:50:22 Hello David, David Goodger schrieb am Freitag, den 30.03.2012 um 10:23: ... > I recommend moving more aggressively, especially since our releases > are not frequent. I can't resist to comment on this because I beg to differ here: IMHO docutils has become very successful and it is around for quite some time now. Keeping backwards compatibility in future releases is an very important aspect for the success of any software project. So even if you find some decision in the software ugly after some time it is often wise to arrange with it and to avoid pissing off endusers with unwanted deprecation warnings they didn't ask for. Please think of Linux distributions which have even more sparse release cycles: e.g. Debian. Debian stable end users expect to be able to upgrade every few years without running into difficult hassles. Or Ubuntu: The Ubuntu OS LTS has a release cycle of two years and the still latest recent version 10.04 still comes with docutils version 0.6 . docutils is only one part of thousands of software packages on a typical Linux installation. And many other packages already depend on docutils. (For example Plone, Sphinx, ...) Best Regards, Peter Funk -- Peter Funk, home: ✉Oldenburger Str.86, D-27777 Ganderkesee mobile:+49-179-640-8878 phone:+49-421-20419-0 ; office: ArtCom GmbH, ✉Haferwende 2, D-28357 Bremen, Germany DRUPA 3.5.-16.5.2012: Besuchen Sie uns in Halle 4 auf Stand B02 
 Re: [Docutils-users] FileInput exiting on IOError From: engelbert gruber - 2012-04-02 09:24:43 hei 1. releases are not that big an issue, i have to find some hours for running tests on at least most supported python versions and maybe some platforms. but i have the feeling that for the developers here they have released when they did commit the code. which is not that uncommon, if you see the debian packages with svn-date names for example. linux itself only puts releases to the repository, as far as i understand and gives a number to it. 2. the proposed change is not that aggresiv to the enduser, because the programm will exit with a different message, but will exit anyway. as the message should be more helpful after the change, i do not think this is really breaking much. or am i totally wrong cheers engelbert -- looking for position as rubberduck apprentice On Sat, Mar 31, 2012 at 1:50 PM, Peter Funk wrote: > Hello David, > > David Goodger schrieb am Freitag, den 30.03.2012 um 10:23: > ... >> I recommend moving more aggressively, especially since our releases >> are not frequent. > > I can't resist to comment on this because I beg to differ here: > IMHO docutils has become very successful and it is around for > quite some time now. > > Keeping backwards compatibility in future releases is an very important > aspect for the success of any software project.  So even if you find > some decision in the software ugly after some time it is often wise > to arrange with it and to avoid pissing off endusers with unwanted > deprecation warnings they didn't ask for. > > Please think of Linux distributions which have even more sparse > release cycles: e.g. Debian.  Debian stable end users expect to > be able to upgrade every few years without running into difficult > hassles.  Or Ubuntu:  The Ubuntu OS LTS has a release cycle of > two years and the still latest recent version 10.04 still comes > with docutils version 0.6 . > > docutils is only one part of thousands of software packages on a > typical Linux installation.  And many other packages already > depend on docutils.  (For example Plone, Sphinx, ...) > > Best Regards, Peter Funk > -- > Peter Funk, home: ✉Oldenburger Str.86, D-27777 Ganderkesee > mobile:+49-179-640-8878 phone:+49-421-20419-0 ; > office: ArtCom GmbH, ✉Haferwende 2, D-28357 Bremen, Germany > DRUPA 3.5.-16.5.2012: Besuchen Sie uns in Halle 4 auf Stand B02 > > ------------------------------------------------------------------------------ > This SF email is sponsosred by: > Try Windows Azure free for 90 days Click Here > http://p.sf.net/sfu/sfd2d-msazure > _______________________________________________ > Docutils-users mailing list > Docutils-users@... > https://lists.sourceforge.net/lists/listinfo/docutils-users > > Please use "Reply All" to reply to the list. -- http://darefoot.blogspot.com 
 [Docutils-users] new release? From: Guenter Milde - 2012-04-27 12:50:33 On 2012-04-02, engelbert gruber wrote: > hei > 1. releases are not that big an issue, i have to find some hours for > running tests on at least most supported python versions and maybe > some platforms. In this case, it might be a good idea to put out the next release without waiting for all the dust to settle. Maybe besides our testing, we could ask at the Sphinx list for someone to test with Sphinx. > 2. the proposed change is not that aggresiv to the enduser, because > the programm will > exit with a different message, but will exit anyway. > as the message should be more helpful after the change, i do not > think this is really > breaking much. > or am i totally wrong You are right. However, nothing is lost if we do the "behaviour change" and cleanup in the next cycle. Günter 
 Re: [Docutils-users] new release? From: Alan G Isaac - 2012-04-27 13:20:02 What is the status of the needed fix for the LaTeX writer, which was abusing the quote environment to add indentation to literal blocks? 1. This meant that quotes and literal text could not be separately styled -- certainly a bug. 2. The indentation fails to rely on the setting of the requested environment (e.g., lstlistings). I have argued vigorously that this is a bug, but Guenter worried about abandoning past formatting practice. I suggested a simple fix: use a custom environment instead of the quote environment. Then the writer can set a default indentation that can easily be overridden in a style sheet. Apologies if I overlooked a commit for this. Thanks, Alan Isaac 
 Re: [Docutils-users] new release? From: Guenter Milde - 2012-04-27 15:49:19 On 2012-04-27, Alan G Isaac wrote: > What is the status of the needed fix for the LaTeX writer, > which was abusing the quote environment to add indentation > to literal blocks? > 1. This meant that quotes and literal text could not > be separately styled -- certainly a bug. > 2. The indentation fails to rely on the setting of the > requested environment (e.g., lstlistings). I have > argued vigorously that this is a bug, but Guenter > worried about abandoning past formatting practice. > I suggested a simple fix: use a custom environment > instead of the quote environment. Then the writer > can set a default indentation that can easily be > overridden in a style sheet. There will be no fix soon (unless someone does a proper one). This should certainly not hinder a release to get out major improvements since the last one. Günter Changes Since 0.8.1 =================== * General: - New reStructuredText "code" role and directive and "code" option of the "include" directive with syntax highlighting by Pygments_. - Fix parse_option_marker for option arguments containing =. .. _Pygments: http://pygments.org/ * setup.py - Fix [ 2971827 ] and [ 3442827 ] extras/roman.py moved to docutils/utils/roman.py * docutils/frontend.py - Fix [ 3481980 ] Use os.getcwdu() in make_paths_absolute(). * docutils/io.py - Fix [ 3395948 ] (Work around encoding problems in Py3k). - mode argument for FileOutput avoids code replication in BinaryFileOutput. - New exceptions InputError and OutputError for IO errors in FileInput/FileOutput. * docutils/core.py: - No "hard" system exit on file IO errors: catch and report them in Publisher.reportException instead. Allows handling by a calling application if the configuration setting traceback is True. * docutils/utils.py -> docutils/utils/__init__.py - docutils.utils is now a package (providing a place for sub-modules) .. note:: docutils/math, docutils/error_reporting.py, and docutils/urischemes.py will move to the utils package in the next release, too. See RELEASE-NOTES__ __ RELEASE-NOTES.html - DependencyList uses io.FileOutput and 'utf8' encoding to prevent errors recording non-ASCII filenames (fixes [ 3434355 ]). - Fix relative_path() with source=None and unicode target. * docutils/parsers/rst/states.py - Fix [ 3402314 ] allow non-ASCII whitespace, punctuation characters and "international" quotes around inline markup. - Use field_marker pattern to look for start of a directive option block (fixes [ 3484857 ]). * docutils/parsers/rst/tableparser.py - Fix [ 2926161 ] for simple tables. (Combining chars in grid tables still contribute to cell width.) * docutils/writers/latex2e/__init__.py - Support the abbreviation and acronym standard roles. - Record only files required to generate the LaTeX source as dependencies. - Fix handling of missing stylesheets. - Use \setcounter{secnumdepth}{0} instead of *-versions when suppressing LaTeX section numbering. - Use \DUtitle for unsupported section levels - Apply [ 3512791 ] do not compare string literals with "is" * docutils/writers/xetex/__init__.py - Avoid code duplication with latex2e writer (solves [ 3512728 ]). * docutils/writers/html4css1/__init__.py - Change default for math-output setting to MathJax. - Fix handling of missing stylesheets. * docutils/writers/docutils_xml.py - Use the visitor pattern with default_visit()/default_depart() methods instead of minidom to facilitate special handling of selected nodes. - Support raw XML (inserted as-is inside a node). * docutils/writers/manpage.py - Do not emit comment line with trailing blank. Problematic for VCS. 
 Re: [Docutils-users] new release? From: engelbert gruber - 2012-04-28 15:04:21 does this mean release now ? On Fri, Apr 27, 2012 at 5:48 PM, Guenter Milde wrote: > On 2012-04-27, Alan G Isaac wrote: >> What is the status of the needed fix for the LaTeX writer, >> which was abusing the quote environment to add indentation >> to literal blocks? > >> 1. This meant that quotes and literal text could not >> be separately styled -- certainly a bug. > >> 2. The indentation fails to rely on the setting of the >> requested environment (e.g., lstlistings).  I have >> argued vigorously that this is a bug, but Guenter >> worried about abandoning past formatting practice. > >> I suggested a simple fix: use a custom environment >> instead of the quote environment. Then the writer >> can set a default indentation that can easily be >> overridden in a style sheet. > > > There will be no fix soon (unless someone does a proper one). > > This should certainly not hinder a release to get out major improvements > since the last one. > > Günter > > Changes Since 0.8.1 > =================== > > * General: > >  - New reStructuredText "code" role and directive and "code" option >    of the "include" directive with syntax highlighting by Pygments_. >  - Fix parse_option_marker for option arguments containing =. > > .. _Pygments: http://pygments.org/ > > * setup.py > >  - Fix [ 2971827 ] and [ 3442827 ] >    extras/roman.py moved to docutils/utils/roman.py > > * docutils/frontend.py > >  - Fix [ 3481980 ] Use os.getcwdu() in make_paths_absolute(). > > * docutils/io.py > >  - Fix [ 3395948 ] (Work around encoding problems in Py3k). >  - mode argument for FileOutput avoids code replication in >    BinaryFileOutput. >  - New exceptions InputError and OutputError for IO errors in >    FileInput/FileOutput. > > * docutils/core.py: > >  - No "hard" system exit on file IO errors: catch and report them in >    Publisher.reportException instead. Allows handling by a calling >    application if the configuration setting traceback is True. > > * docutils/utils.py -> docutils/utils/__init__.py > >  - docutils.utils is now a package (providing a place for sub-modules) > >  .. note:: docutils/math, docutils/error_reporting.py, and >     docutils/urischemes.py will move to the utils package in the next >     release, too. See RELEASE-NOTES__ > >     __ RELEASE-NOTES.html > >  - DependencyList uses io.FileOutput and 'utf8' encoding to prevent >    errors recording non-ASCII filenames (fixes [ 3434355 ]). > >  - Fix relative_path() with source=None and unicode target. > > * docutils/parsers/rst/states.py > >  - Fix [ 3402314 ] allow non-ASCII whitespace, punctuation >    characters and "international" quotes around inline markup. >  - Use field_marker pattern to look for start of a >    directive option block (fixes [ 3484857 ]). > > * docutils/parsers/rst/tableparser.py > >  - Fix [ 2926161 ] for simple tables. >    (Combining chars in grid tables still contribute to cell width.) > > * docutils/writers/latex2e/__init__.py > >  - Support the abbreviation and acronym standard roles. >  - Record only files required to generate the LaTeX source as dependencies. >  - Fix handling of missing stylesheets. >  - Use \setcounter{secnumdepth}{0} instead of *-versions >    when suppressing LaTeX section numbering. >  - Use \DUtitle for unsupported section levels >  - Apply [ 3512791 ] do not compare string literals with "is" > > * docutils/writers/xetex/__init__.py > >  - Avoid code duplication with latex2e writer (solves [ 3512728 ]). > > * docutils/writers/html4css1/__init__.py > >  - Change default for math-output setting to MathJax. >  - Fix handling of missing stylesheets. > > * docutils/writers/docutils_xml.py > >  - Use the visitor pattern with default_visit()/default_depart() methods >    instead of minidom to facilitate special handling of selected nodes. >  - Support raw XML (inserted as-is inside a node). > > * docutils/writers/manpage.py > >  - Do not emit comment line with trailing blank. Problematic for VCS. > > > > > > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Docutils-users mailing list > Docutils-users@... > https://lists.sourceforge.net/lists/listinfo/docutils-users > > Please use "Reply All" to reply to the list. -- http://darefoot.blogspot.com 
 Re: [Docutils-users] new release? From: Guenter Milde - 2012-04-28 19:14:05 On 2012-04-28, engelbert gruber wrote: > does this mean release now ? Yes, please. Günter