[Audacity-nyquist] New "Silence Finder" Plug-In, Need Help on Memory Use (Alex S. Brown)
A free multi-track audio editor and recorder
Brought to you by:
aosiniao
|
From: Roger B. D. <rb...@cs...> - 2005-03-30 16:00:26
|
Sorry for the length of this reply, but there are some subtle issues = here... > I believe that the biggest memory-use growth is in the "do" loop,=20 > not the initial SETF. I calculated that for a 20-minute segment,=20 > the loop is going 1.2 million times, so just a 200 bytes of memo>ry=20 > use each round would lead to over 200 MB over the course of the = function.=20 > Maybe that is part of the problem. Memory use is subtle in Lisp and especially in Nyquist. When memory = usage increases, Nyquist runs the garbage collector, which will free anything = that is not being used. So normally, anything allocated in the do loop or elsewhere will soon be freed and not cause any problems. Problems occur when there are unintended references to data. Typically = this happens when you set a global variable to some signal value. Once you do this, Nyquist is unable to free any of the signal's samples. You might say, "so what, the signal is already in memory, so what harm = does it do to assign it to a global?" The answer is that signals are only evaluated on demand. E.g. the variable "s" that Audacity passes to = Nyquist plug-ins is really just an object that is capable of producing samples = (by reading them from disk) when asked to do so by Nyquist. Ordinarily, = Nyquist will read a block of samples from "s", process those samples, deliver = the result to Audacity, and free any intermediate samples that were used in = the computation. Nyquist's runtime system figures out automatically that = once a block of samples from "s" have been processed, they are no longer needed = and can be freed. If you assign "s" (or some function of "s") to a global, Nyquist's = runtime system assumes that you may want to use these samples in the future for something else. It cannot free the samples as they are processed, and instead appends them to a linked list in memory. When the garbage = collector runs, it sees that the huge list of samples are referenced by a (global) variable and thus cannot be freed. You are right that the do loop is involved -- the do loop is forcing = Nyquist to read "s" and reading "s" causes samples to be accumulated in memory. = But the trick is to let the garbage collection free samples after you read = them. SND-FETCH is good for this. It will read from the head of a sound and advance the "reader" to the next sample, implying sequential access and allowing the initial samples to be deallocated once they are read. SREF allows random access to the sound, which means you are going to have = some variable pointing to the sound. As you SREF further and further into the sound, you'll amass more and more samples in memory -- again, Nyquist = cannot know that you will not perform an SREF back to the beginning of the = sound, so all the sound's samples must be kept in memory. Nyquist will only = read in samples as you ask for them, so you would expect to see memory growing = in the do loop. |