Re: [Audacity-devel] Exception safety and Nyquist Lisp
A free multi-track audio editor and recorder
Brought to you by:
aosiniao
From: Roger D. <rb...@cs...> - 2017-08-19 16:23:40
|
On 8/19/17 10:30 AM, Steve the Fiddle wrote: > On 19 August 2017 at 14:13, Paul Licameli <pau...@gm...> wrote: >> >> On Fri, Aug 18, 2017 at 7:46 PM, Robert Hänggi <aar...@gm...> >> wrote: >>> I would certainly appreciate a pipe to Python (NVDA runs under Python). >>> >>> Currently, I have to use a backdoor in order to e.g. get selection data. >>> >>> - Python simulates a keystroke which in turn triggers a Nyquist >>> plug-in, something like this: >>> >>> ;nyquist plug-in >>> ;version 4 >>> ;type analyze >>> ;name "Screen Reader Support" >>> (setf filename (format nil "~a~aaudacity_scr_rdr_pipe.tmp" (get-env >>> "userprofile") *file-separator*)) >>> (setf data (symbol-plist '*track* )) >>> (setq fp (open filename :direction :output)) >>> (dolist (curr data) >>> (print curr fp)) >>> (close fp) >>> "" >>> >>> The resulting file will be read by Python, removed and the desired >>> value spoken. >>> >>> I wouldn't mind having a bit more low level access, especially >>> selection functions. >>> >>> Compare e.g. Reapers console Window where you could write: >>> >>> S1-5, -4, voc* >>> >>> which would select tracks 1 to 5 but not 4 and all those that begin with >>> "voc". >>> >>> Robert >> >> Changing the selection is not an operation I worry about, as potentially >> throwing. >> >> Possibly throwing operations are those that really change the content of the >> project. >> >> PRL >> >> >>> >>> >>> >>> On 19/08/2017, Paul Licameli <pau...@gm...> wrote: >>>> On Fri, Aug 18, 2017 at 6:39 PM, James Crook <cr...@in...> wrote: >>>> >>>>> This merits a much fuller answer, but I'm going to say just a little >>>>> here. >>>>> >>>>> The extra functions for Nyquist are motivated by extra functions for >>>>> external scripting, which in turn is for Python over a pipe or TCP/IP >>>>> connection. >>>>> >>>>> The Python will want functions for working with selections, for >>>>> getting/setting window locations, for enumerating clips, for setting >>>>> preferences and so on. The big motivation there is to be able to >>>>> generate >>>>> just about every image in the manual by running a script. A great >>>>> workout >>>>> for Audacity. A big time saver for creating and updating images in the >>>>> manual. >>>>> >>>>> Can exceptions propagate across the pipe or TCP/IP connection? Well, I >>>>> personally have no intention of serialising them and 'converting' them >>>>> to >>>>> Python exceptions. So the API that the Python scripting is calling >>>>> instead >>>>> needs certain guarantees. For my needs it looks to be that any >>>>> unwinding >>>>> of objects be handled in Audacity before a result is handed back to >>>>> Python >>>>> over the pipe. Exactly the same logic seems appropriate to Nyquist. >>>>> Part >>>>> of the semantics of the scripting API is that it is a 'firewall' across >>>>> which exceptions do not pass. > So should such a "firewall" act in both directions? > > Something that I have voiced concerns about many times is that fact > that if Nyquist crashes, it takes down Audacity with it. But in return for that, Nyquist can efficiently access Audacity data -- it would be possible to send Audacity data between processes, especially given the speed of current processors, and recover from process termination or crashes, but it would be a very different design, and the design itself would have a lot of subtle points of failure. We've fixed a lot of things in Nyquist (and even the Audacity side has had its share of crash-producing bugs) and that seems to be at least a viable approach. > Even > something as "harmless" as exiting Nyquist will take down Audacity. > For example, in the Nyquist Prompt, run the command: > (exit) That's easily "fixed," but can the Python interface give a Quit command? Will future extensions make "Quit" accessible to Nyquist? I.e. is this a bug or a feature? > > Surely that should just exit Nyquist and leave Audacity running, > allowing Nyquist to be reinitialised. > > Steve > > >>>>> Is that too restrictive? I don't think so. The Python and Nyquist >>>>> scripts do need a way to 'abandon ship' when a prior error (e.g. could >>>>> not >>>>> open audacity project file) means that all subsequent actions by the >>>>> script >>>>> are pointless. I do not mandate how that is done, but instead provide >>>>> enough control that it is possible. I provide enough control that >>>>> someone >>>>> writing Python (or Nyquist) can request that Audacity report all such >>>>> errors to the user - that's the default - OR ask that they not be >>>>> reported >>>>> by dialog, and that they will handle them. Either way the script still >>>>> gets to see the error. On the Python side, the Python library can call >>>>> a >>>>> Python user supplied function on return from every call into Audacity. >>>>> The >>>>> function could be set to raise Python exceptions if that user wants, or >>>>> to >>>>> stop the script, but that is up to them. All that is guaranteed is >>>>> that >>>>> they can read results of the call into Audacity, which could be an >>>>> error >>>>> result. Any unwinding of Audacity objects will already have been done >>>>> at >>>>> that stage. Same for Nyquist. >>>>> >>>>> --James. >>>> >>>> So if I understand you right, you want Python or Lisp to do something >>>> like >>>> simulation of a user interaction sequence, such as "select this range of >>>> time; Duplicate". Each of these self-contained operations could >>>> recover >>>> from errors as implemented now. What is new might only be the reporting >>>> of >>>> errors to the scripting language. A script is composed of very high >>>> level >>>> Audacity operations, each of which remains implemented completely in >>>> C++. >>>> >>>> Scripts are for administrative tasks and not meant as extensions or >>>> plug-ins that people might share on the Forum. Importantly, we the Team >>>> would not distribute any scripts as part of the package (as we do for >>>> Nyquist effects now). It's your problem, advanced user, as a script >>>> writer, to detect and handle errors correctly. >>>> >>>> You do not want Python or Lisp to be a language in which you extend >>>> Audacity with novel editing operations, composed of low-level steps such >>>> as, "Make a temporary WaveTrack object as a copy of this track; resample >>>> it; paste it into that track; push the undo history." >>>> >>>> So then, all that I have said today may be irrelevant to this project. >>>> But >>>> perhaps not to imaginable future projects. >>>> >>>> PRL >>>> >>>> >>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 8/18/2017 11:05 PM, Paul Licameli wrote: >>>>> >>>>>> On Fri, Aug 18, 2017 at 4:54 PM, Roger Dannenberg <rb...@cs...> >>>>>> wrote: >>>>>> >>>>>> What's the ultimate goal or desired behavior? It seems to me that the >>>>>>> most we could do or should do is print some text to the user in a >>>>>>> dialog >>>>>>> box. Since the Audacity/Nyquist interface can already show text, it >>>>>>> seems like a simple mechanism could capture error information and >>>>>>> display it. I've never been a fan of exception handling (other than >>>>>>> simply returning values or error codes) because you never know if a >>>>>>> function call will return or not, so it's much harder to reason about >>>>>>> code (IMO). It sounds like Audacity is pretty committed to throwing >>>>>>> exceptions as a style. It's true that XLISP implements throw/catch >>>>>>> control structures, and it would be possible to catch C++ exceptions >>>>>>> and >>>>>>> then re-throw them as Lisp exception, and even catch them at the top >>>>>>> level and re-throw them as C++ exceptions if you fall off the XLISP >>>>>>> stack. >>>>>>> >>>>>>> -Roger >>>>>>> >>>>>>> What, indeed, is the goal? James raised mention of Nyquist in the >>>>>> context >>>>>> of command scripts, and I would like James to explain. >>>>>> >>>>>> There seems to be a desire to use Nyquist Lisp as a built-in command >>>>>> scripting language for the Audacity application, which goes beyond its >>>>>> original limited purposes for implementing effects and generators and >>>>>> analyzers. I think James wants this, I believe Steve wants it too, >>>>>> even >>>>>> more so. I say, if you want that expansion of what our Lisp can do, >>>>>> then >>>>>> you must take these other precautions. >>>>>> >>>>>> One of my big projects in Audacity 2.2.0 was to use C++ throw and >>>>>> catch >>>>>> to >>>>>> recover from disk exhaustion errors, and disk read errors too, and >>>>>> report >>>>>> them to the user. In my opinion, contra Roger, exception handling is >>>>>> a >>>>>> very good thing. It makes it very clean to encode the long-distance >>>>>> communication between the low level detection of resource exhaustion >>>>>> (which >>>>>> might strike in any of many deeply nested places) and the high level >>>>>> (where >>>>>> we recover from incomplete operations in just one place, by rollback >>>>>> of >>>>>> the >>>>>> undo state, and then make an error report). Tedious explicit >>>>>> propagation >>>>>> of error codes through many levels of stack is avoided. Rewriting the >>>>>> code >>>>>> we had was easier this way. >>>>>> >>>>>> But it also means many levels of stack in between must be written in >>>>>> an >>>>>> exception-aware way to avoid resource leaks or other undesired >>>>>> side-effects >>>>>> of operations abandoned in the middle, and I scrutinized much such >>>>>> code >>>>>> to >>>>>> be sure of its safety. Use of smart pointers is only a part of that. >>>>>> See >>>>>> also the many uses of finally(). It is not hard to adopt the "RAII" >>>>>> discipline, but you must learn it, and know the places that need it >>>>>> when >>>>>> you see them. Middle level code must be safe for exceptions to go >>>>>> through. Between throw and catch, the code must know how to "duck." >>>>>> >>>>>> If we want to bind some possibly throwing low-level operations on the >>>>>> Audacity project as Lisp functions, and then write the middle-level >>>>>> code >>>>>> for novel menu commands in Lisp instead of C++, then I want >>>>>> reassurances >>>>>> that Lisp is playing nicely with this exception handling framework >>>>>> that >>>>>> I >>>>>> implemented with much effort this past year, letting the exceptions >>>>>> through >>>>>> safely, neither "eating" them (not letting the high level know them) >>>>>> or >>>>>> "choking" on them (corrupting the Lisp runtime when they do go >>>>>> through). >>>>>> I >>>>>> outlined how to accomplish that. >>>>>> >>>>>> And as for throw, catch, and unwind-protect in XLISP -- those are in >>>>>> fact >>>>>> parts of Common Lisp. XLISP is a subset of Common Lisp (plus an >>>>>> object >>>>>> system), which includes all of the Common Lisp flow of control >>>>>> structures. >>>>>> >>>>>> PRL >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> On 8/18/17 3:31 PM, Steve the Fiddle wrote: >>>>>>> >>>>>>>> On 18 August 2017 at 14:52, Paul Licameli <pau...@gm...> >>>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>>> Some idea jottings, mostly for James, and for future reference: >>>>>>>>> If we want more Audacity functions to be callable from Nyquist >>>>>>>>> Lisp, >>>>>>>>> and >>>>>>>>> these functions might throw, and we want my new error reporting to >>>>>>>>> the >>>>>>>>> >>>>>>>> user >>>>>>>> to function just as elsewhere in Audacity, then what must be done? >>>>>>>>> See what I did at commit 8e5975b10db0, but it's not satisfactory. >>>>>>>>> >>>>>>>> It would be more helpful if you gave a link rather than expecting us >>>>>>>> to search for the repository and branch that you are referring to. >>>>>>>> >>>>>>>> Steve >>>>>>>> >>>>>>>> Catch exceptions where Nyquist calls back into C++ land, translate >>>>>>>> to >>>>>>>> error >>>>>>>> code. >>>>>>>>> At the high level where Lisp gives control back to C++, detect the >>>>>>>>> >>>>>>>> error, >>>>>>>> try to reconstruct an equivalent exception, and re-throw. >>>>>>>>> This is unsatisfactory because it doesn't generalize to all >>>>>>>>> possible >>>>>>>>> exceptions. We are less free to let the callbacks into C++ throw >>>>>>>>> other >>>>>>>>> things if need be. >>>>>>>>> Re-throw is really needed. You don't want just to absorb the >>>>>>>>> exception >>>>>>>>> >>>>>>>> at a >>>>>>>> low level. You want to propagate through levels above Nyquist so >>>>>>>> they >>>>>>>> can >>>>>>>> do their proper RAII and ultimately report the cause of failure to >>>>>>>> the >>>>>>>> user. >>>>>>>> I mention in comments that the missing thing I want to use is >>>>>>>>> std::exception_ptr. >>>>>>>>> >>>>>>>>> That's reference-counting of exception objects in the C++ runtime, >>>>>>>>> new >>>>>>>>> >>>>>>>> in >>>>>>>> C++11. You can keep an exception object alive even after leaving >>>>>>>> the >>>>>>>> catch >>>>>>>> block than handles it, and re-throw it later. Read more here: >>>>>>>>> http://en.cppreference.com/w/cpp/error/exception_ptr >>>>>>>>> But on Mac at least I know we have not yet migrated to a version of >>>>>>>>> the >>>>>>>>> library that includes it. And this is connected to the question of >>>>>>>>> abandoning Snow Leopard, which did not get reexamined this release. >>>>>>>>> I don't know what minimal versions are needed on the other >>>>>>>>> platforms >>>>>>>>> to >>>>>>>>> >>>>>>>> get >>>>>>>> us exception_ptr. >>>>>>>>> Given exception_ptr you could re-do commit 8e5975b10db0 properly >>>>>>>>> and >>>>>>>>> >>>>>>>> more >>>>>>>> generally (as to possible exception types). You could do the >>>>>>>> analogous >>>>>>>> in a >>>>>>>> few places, wrapping each foreign function, and testing for an >>>>>>>> exception to >>>>>>>> re-throw at each place where Lisp runtime returns to C++. >>>>>>>>> All this would propagate C++ exceptions, while preventing C++ stack >>>>>>>>> unwinding within the levels of stack written in C that implement >>>>>>>>> the >>>>>>>>> >>>>>>>> Lisp >>>>>>>> interpreter. >>>>>>>>> But Lisp also has its exception handling -- you knew that, right? >>>>>>>>> >>>>>>>>> Relevant forms are throw, catch, and unwind-protect, the latter >>>>>>>>> >>>>>>>> allowing you >>>>>>>> to use the equivalent of the finally template function that is much >>>>>>>> used in >>>>>>>> our C++ now. >>>>>>>>> You can examine the code implementing the Lisp interpreter -- I >>>>>>>>> read >>>>>>>>> it >>>>>>>>> >>>>>>>> all >>>>>>>> a few years ago, and forgot much -- and you will see that setjmp and >>>>>>>> longjmp >>>>>>>> implement it. >>>>>>>>> If what is thrown with Lisp throw is not caught with Lisp catch, >>>>>>>>> then >>>>>>>>> a >>>>>>>>> top-level setjmp in the Lisp runtime stops it. >>>>>>>>> >>>>>>>>> >>>>>>>>> So: beyond the use of exception_ptr, do we want gateways between >>>>>>>>> Lisp >>>>>>>>> >>>>>>>> and >>>>>>>> C++ that translate exceptions? These ideas would require a little >>>>>>>> bit >>>>>>>> of >>>>>>>> hacking the Lisp runtime, but I am sure we could figure it out. >>>>>>>>> Where the foreign function wrapper catches and saves exception_ptr, >>>>>>>>> it >>>>>>>>> >>>>>>>> would >>>>>>>> also find the current jmp_buf in the Lisp runtime and longjmp it. >>>>>>>>> Levels of Lisp code could then visit their unwind_protect handlers >>>>>>>>> in >>>>>>>>> response to a C++ exception, so that Lisp code could write RAII >>>>>>>>> too. >>>>>>>>> We must be sure the implementation of Lisp catch (which examines >>>>>>>>> the >>>>>>>>> >>>>>>>> Lisp >>>>>>>> symbol associated with a Lisp throw) will never match a C++ >>>>>>>> exception >>>>>>>> and >>>>>>>> always pass control up the chain to the next jmp_buf. >>>>>>>>> Where Lisp implements its top-level setjmp of last resort (in C), >>>>>>>>> it >>>>>>>>> >>>>>>>> could >>>>>>>> return a code to caller indicating whether there was an unhandled >>>>>>>> exception, >>>>>>>> both in case this is a Lisp exception, or it is a C++ exception. >>>>>>>>> The C++ layer that called Lisp can detect this condition, and >>>>>>>>> either >>>>>>>>> >>>>>>>> rethrow >>>>>>>> the exception_ptr, or, finding it null, could construct a new C++ >>>>>>>> exception >>>>>>>> object representing the Lisp exception, details of which we might >>>>>>>> also >>>>>>>> have >>>>>>>> saved in the Lisp code, using some foreign function. >>>>>>>>> PRL >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>> ------------------------------------------------------------ >>>>> ------------------ >>>>> Check out the vibrant tech community on one of the world's most >>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>>> _______________________________________________ >>>>> audacity-devel mailing list >>>>> aud...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/audacity-devel >>>>> >>> >>> ------------------------------------------------------------------------------ >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>> _______________________________________________ >>> audacity-devel mailing list >>> aud...@li... >>> https://lists.sourceforge.net/lists/listinfo/audacity-devel >> >> >> ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> audacity-devel mailing list >> aud...@li... >> https://lists.sourceforge.net/lists/listinfo/audacity-devel >> > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > audacity-devel mailing list > aud...@li... > https://lists.sourceforge.net/lists/listinfo/audacity-devel |