edgar wrote:
> Question:
>
> > How "safe" is it to use these in Audacity plug-ins? (are they
> > "permanent" features or might they disappear in later versions
> > of Audacity?)
>
> *SCRATCH* is a new variable, which has not extensively tested.
>
> Some first *SCRATCH* test results:
>
> Preface:
>
> In Lisp, using global variables like *SCRATCH* to store local results
> of Lisp functions is considered very bad programming style.
Yes, but the point of *SCRATCH* is to provide some way for information
to survive from one invocation of a plug-in to the next. This is not a
case of local results but one of making local information global on purpose.
> The reason
> is that local variables are lexically scoped, what means that they
> get automatically garbage-collected if not referenced any longer by
> any other Lisp code, while global variables are dynamically scoped,
> they only can get garbage-collected if explicitely set to NIL.
>
> In one sentence:
>
> Local function results referenced by global Lisp variables cannot
> get automatically garbage-collected until all global references
> are explicitely set to NIL.
>
> In the case of *SCRATCH* in Audacity there happens something very
> strange (from the Lisp point of view) but maybe very useful.
>
Depending on your point of view. The convention of putting asterisks (*)
around global variables in Lisp shows that globals are considered a
special case, but one that's common enough and important enough to have
developed a standard naming practice.
> Audacity internals:
>
> In Audacity, the Nyquist Lisp interpreter is only invoced once,
> at the time when the first Nyquist plugin or some code from
> "Effect > Nyquist Prompt" is applied. The initial state of the
> *obarray* (the Lisp array where all symbol references are stored)
> is backed-up into a C variable by the Audacity Nayquist interface.
>
Strictly speaking, XLISP is initialized only once, meaning that the heap
and system variables are initialized only once, but the interpreter is
invoked once each time you run a plug-in. The interpreter is a function
written in C (that of course may call many other C functions in the
process of interpreting XLISP code).
> After an Audacity Nyquist plugin is finished, the Nyquist Interpreter
> keeps running,
Or rather the XLISP heap is retained after the interpreter returns to
the calling C function in Audacity.
> but the Lisp *obarray* gets restored by the Audacity
> Nyquist interface from the initial value, which was stored in the
> C variable. This way all user-defined functions and variables get
> deleted from the Lisp *obarray*, and the Nyquist Lisp interpreter
> will have it's initial state again, with exception of the *SCRATCH*
> symbol, which is explicitely protected from deletion by the C-code
> in the Audacity Nyquist interface.
>
> Important: The "survival" of the *SCRATCH* symbol is a C-hack in the
> Audacity Nyquist Interface, and NOT a build-in Nyquist behaviour.
>
Or alternatively, restoring the *obarray* is a very non-standard C-hack.
A much more "normal" thing to do would be to leave the heap unaltered
(including *scratch* when a plug-in returns). In this way of looking at
things, the *scratch* behavior is completely normal Nyquist behavior and
the removal of all other globals set by the plug-in is the hack. In
retrospect, it might have been better to translate plug-ins so that all
top-level variables (that currently are globals) would be declared as
locals so that they would be cleaned up in the normal way, avoiding the
"hack" of manipulating the *obarray* when the plug-in returns. But then
this would make plug-ins have non-standard lisp behavior, so explaining
that would add some complexity to the system.
> As a side-effect of this C-hack all the backtracking pointers,
> referenced by the *SCRATCH* symbol, get deleted, only the
> reference to the *SCRATCH* symbol and it's values survive.
>
> Practical usage:
>
> The *SCRATCH* symbol offers a very ugly (but practically functional)
> work-around for writing two-pass plugins.
>
> You can write two-pass plugins as two separate plugins:
>
> 1. Write an "analysis" plugin, which stores it's result in the
> *SCRATCH* variable (you can use lists to store multiple values).
>
> Hint: an Audacity "analysis" plugin can generally also be written as a
> "process" plugin, then the plugins will not appear in different menus.
>
> 2. Write a "process" plugin, using the results from the "analysis"
> plugin, stored in the *SCRATCH* variable.
>
> If the "analysis" plugin survives it's work without crashing Audacity
> (the Audacity memory problems still exist during the plugin run-time),
> then after the plugin has finished, all backtracking pointers,
> referenced by the *SCRATCH* symbol, get deleted (by the C-hack in
> the Audacity Nyquist interface), this way all allocated memory gets
> freed, but all values of the *SCRATCH* symbol still exist and can be
> re-used in subsequent plugins without causing huge memory allocations.
>
> Practical examples:
>
> If you want to see the memory allocation, start some memory monitor
> (which tool depends on your operation system).
>
> 1. In Audacity, generate an audio track via "Generate > Tone".
>
> 2. Select the whole track, open "Effect > Nyquist Prompt" and type:
>
> (setq *SCRATCH* (peak s ny:all))
> (print *SCRATCH*)
>
> Click "Debug" to apply the Nyquist code and see the "print" result.
> In the memory monitor you can see the Audacity memory allocation
> rising. Note that if the Audacity selection is too long, the "peak"
> function will still be able to crash Audacity!
>
> After the plugin has finished, the "Debug" window will appear with
> the result of the "print" function. After closing the window the
> plugin has finished and the memory gets freed again.
>
> 3. Now open Effect > Nyquist Prompt" a second time and type only:
>
> (print *SCRATCH*)
>
> Click "Debug" to see the "print" result. The "Debug" window will
> appear with the result of the "print" function, which is still the
> same as in the first example, but this time without any significant
> memory allocation.
>
> 4. Because Lisp uses separate namespaces for variables and functions
> also a user-defined *SCRATCH* function will survive:
>
> Open Effect > Nyquist Prompt" again and type:
>
> (defun *SCRATCH* () (print 'hello))
> (*SCRATCH*)
>
> Click "Debug" to see how the *SCRATCH* function prints "HELLO".
>
> 5. Open Effect > Nyquist Prompt" again and type only:
>
> (*SCRATCH*)
>
> Click "Debug" to see that the *SCRATCH* function still prints "HELLO".
>
> Note: Defining Lisp functions in *star-notation* is even worse Lisp
> style than binding local results to global variables. I only wanted
> to demonstrate that this really works. Maybe there will appear some
> situations where this could be used as another work-around.
>
> 6. Also the property list of the *SCRATCH* symbol will survive.
>
> 7. I still haven't tested what happens with *SCRATCH* OOP objects.
>
> Summary:
>
> From the Lisp point of view the behaviour of the *SCRATCH* variable
> is nothing but a really wacky hack, but in practical work the
> *SCRATCH* variable can turn out as a really useful work-around.
>
> I have noticed no really serious problems with *SCRATCH* so far.
>
> - edgar
>
>
>
There should probably be some recommended protocols for dealing with
*scratch* since it is sort of a bottleneck for passing data among
plug-ins. I would recommend not setting *scratch* as a variable but
using it as a holder of property lists. That way, you get a whole name
space rather than a single variable name. To pass data from plugin
effectX-partA to effectX-partB,
1) assign a property name based on the effect name, e.g. 'EFFECTX
2) effectX-partA should delete any old property value:
remprop(quote(*scratch*), quote(effectx))
3) effectX-partA should compute a new property value v and save it:
putprop(quote(*scratch*), v, quote(effectx))
4) effectX-partB should access the property using
get(quote(*scratch*), quote(effectx))
5) when effectX-partB finishes, it should remove the property:
remprop(quote(*scratch*), quote(effectx)), ...
6) but there may be cases where you do some analysis and want to use
the analysis data multiple times. You might even have multiple analysis
plugins operating on different inputs to collect data to feed into a
plugin with multiple inputs. In this case (which might be quite common),
you should not call remprop(), but this has the problem of leaving data
on the *scratch* property list indefinitely, so ...
7) In cases where *scratch* data is not deleted immediately after
use, there should be another effect, e.g. effectX-partCleanup, that
simply calls remprop(quote(*scratch*), quote(effectx)), allowing the
user to explicitly free up any data stored on the 'EFFECTX property. It
would be reasonable to omit the "effectX-partCleanup" effect if the data
stored on the property list has a maximum size of, say, 10KB. The
problem we want to avoid is properties with unbounded size getting left
in the heap until Audacity is restarted.
-Roger
|
