Re: [Audacity-devel] White Space

[Dan H. <da...@go...>]

Hi David,

The 'white space' is related to the way Audacity stores the audio in a
track. Although a track appears to effects as an array of samples, it is
in fact a list of 'wave clips' - blocks of samples, together with
information describing the times at which the clips are positioned in
the track. This reason for this (I believe) is mainly to make it more
efficient to move chunks of audio around. It also makes it easier for
users to chop up and rearrange a track.

The white space is what you get between the clips - so called because it
is drawn differently on the screen to silent audio, by which I mean a
clip that contains samples which are zero. In many ways the white space
behaves the same way as silent audio does - and of course it sounds exactly
the same. Currently many of the effects produce one monolithic clip of
audio even if their input consisted of several separate clips. Since
some people use the facility to divide the audio into clips as part of
their workflow, it would help them if the effects preserved the clip
structure of the input. For the effects which are 'time-altering', the
positions and lengths of the clips may be different in the output to
those in the input, so it's tricky to keep track of which audio should
be in which clips.

Dan

On Thu, Aug 06, 2009 at 01:46:33AM -0700, David R. Sky wrote:
> Hi,
>
> Thanks for the sequencer plug plug *chuckle*, which needs some work  
> getting it to the coding level I did in clicktrack and vocal remover.
>
> That said, I'm still unclear what the difference between "white space" 
> and silence is - I haven't been following discussions about white space  
> _because_ it meant nothing to me. Is there an explanation that someone  
> who's v.i. could understand?
>
> Thanks - David
>
> On Wed, 5 Aug 2009, Michael Chinen wrote:
>
>> Great discussion.
>>
>>
>> On Wed, Aug 5, 2009 at 11:58 AM, Richard Ash<ri...@au...> wrote:
>>> On Wed, 2009-08-05 at 13:29 +0100, Dan Horgan wrote:
>>>> On Tue, Aug 04, 2009 at 07:35:55PM +0100, Steve wrote:
>>>>> As has been raised in several post recently, the handling of white space
>>>>> between audio clips on the same track can be confusing.
>>>
>>>>> And then the situation gets considerably more confusing when multiple
>>>>> tracks are selected and tracks are linked or not linked with a label
>>>>> track.
>>> I don't think linking or multiple track selection should ever affect
>>> white space handling vs. silence handling. The handling of both may be
>>> different to having single tracks selected or not having linking, but
>>> there shouldn't be a difference between white space and silent samples.
>>>
>>>>> How bad would it be to make everything consistent and always treat white
>>>>> space within a selection as part of the selection? That is, if the
>>>>> effect shrinks audio within the selection, then it will also shrink
>>>>> white space.
>>>
>>>> As a user I think I would expect the space between clips to behave the
>>>> same way as silent audio - as it does for the most part. (There is also
>>>> a visual similarity). However, as in the generate example (see 'release
>>>> checklist' thread), this does offer fewer options to users who do know
>>>> what they're doing.
>>>
>>> I would describe that use case as not something that Audacity is
>>> designed for. If you want to do that, the David Sky has written some
>>> very powerful sequencing plug-ins using Nyquist that can be used to do
>>> all the clip manipulations without messing with the general editing
>>> behaviour of Audacity. I'd still suggest a proper sampler / sequencer
>>> for that kind of usage.
>>>
>>>> I think having white space generally behave like silence but with a few
>>>> subtle exceptions is a bad idea - if there is to be a distinction I
>>>> think it's important to make that clear.
>>>>
>>>> Personally I'd favour always treating it as silence, unless there are
>>>> use cases where that makes things much more difficult.
>>> I agree - in general there shouldn't be a distinction between runs of
>>> silent samples and the gaps between clips on a track.
>>>
>>>> This ties in somewhat with the "Nyquist effects, SoundTouch, SBSMS and
>>>> Generators join separate clips together." problem that I've been looking
>>>> at. In fact I think the 'joining together' happens with all the effects
>>>> that use ClearAndPaste (so ChangeSpeed and NoiseRemoval as well as the
>>>> above).
>>>>
>>>> The thing is, if white space appears as silence to, say, a Nyquist
>>>> effect, then the effect has every right to produce actual silence there
>>>> in its output. At that point it isn't really possible to tell what was
>>>> originally white space and what was actually silence.
>>> This depends on what conceptually the effects are being applied to. If
>>> the effects were applied to each clip separately, then of course the
>>> output would be a set of clips of the same size as each source clip,
>>> retaining the original clip boundaries and sizes.
>> The list-tracking suggestion at the end of Richard's email may provide
>> a way to post-process the general case where there is no
>> stretching/shifting.  Its true that we could solve some cases with a
>> scaling factor, but some effects (like analysis/synthesis user
>> specified time maps) might have that factor be time varying.  Maybe it
>> would be a good idea to have a virtual method that does the suggested
>> waveclip tracking and have effects that break the rule override?
>>
>>
>> Michael
>>
>>>
>>> There are in fact two related questions here, which we are in danger of
>>> confusing:
>>> 1) Should patches of white space within a selection being processed be
>>> converted into samples?
>>> 2) Should clip boundaries within a selection being processed be lost
>>> (all clips in the source merged into a single output clip)?
>>>
>>> I think the answer to the second question is clearly no, but I'm open to
>>> argument over the first.
>>>
>>> So if we start with something like this:
>>>
>>> |--clip1--|  whitespace  |--clip2--|  white  |__clip3 is silent__|
>>>
>>> (pipe characters are clip boundaries, - are audio and _ are silent
>>> audio).
>>> Three clips, the third is silent samples, with two white space gaps
>>> between them. As I understand it now, what comes out (for an effect that
>>> doesn't make any significant changes) is:
>>>
>>> |--clip1---_______________----------_____________________________|
>>>
>>> i.e one long clip (not three or five) with two bursts of audio and a lot
>>> of silence. The original clip boundaries (except start and end) have all
>>> been lost - which could be quite annoying for lots of reasons.
>>>
>>> Let's consider the case if whitespace was considered as silence, but we
>>> maintained clip boundaries in the output:
>>>
>>> |--clip1--|___silent_____|--clip2--|_silent__|__clip3 is silent__|
>>>
>>> Now we have 5 clips, the three original ones plus two new ones which
>>> were created out of silence. This preserves the original clip structure,
>>> treats white space as though it were silence, but still leaves the
>>> effect processing the whole track in one run (if you really don't want
>>> the two silent clips, they are of course very easy to delete - double
>>> click to select and then hit delete).
>>>
>>> I think what Gale and I had originally envisaged was an output of
>>>
>>> |--clip1--|  whitespace  |--clip2--|  white  |__clip3 is silent__|
>>>
>>> just like the input (or the above after deletion), but now you point it
>>> out I can see some significant drawbacks to this idea:
>>> * It means that the effect has to process each clip on it's own, so that
>>> a clip boundary (between two waveclips) could cause audible artifacts in
>>> the processed audio
>>> * It means that if the audio after the effect carries on (like a reverb)
>>> then the tail is chopped off rather than continuing into the silence.
>>> * It assumes the output of effected silence is silence, which is a nasty
>>> assumption to make (see above for one special case of this).
>>>
>>> So I agree that whitespace that is processed may become silence when it
>>> makes sense to do so, but I insist that clip boundaries don't get munged
>>> by the process. This is probably just a case of storing a list before
>>> processing, and sticking the cut points back into the audio afterwards,
>>> in the same places. It gets a bit more complex for time-changing
>>> effects, because you have to move the clip boundaries by the effect
>>> scaling factors ...
>>>
>>> To my mind this is an acceptable solution to use, and a realistic one to
>>> attempt to implement.
>>>
>>> Richard
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
>>> trial. Simplify your report design, integration and deployment - and focus on
>>> what you do best, core application coding. Discover what's new with
>>> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>>> _______________________________________________
>>> audacity-devel mailing list
>>> aud...@li...
>>> https://lists.sourceforge.net/lists/listinfo/audacity-devel
>>>
>>
>> ------------------------------------------------------------------------------
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
>> trial. Simplify your report design, integration and deployment - and focus on
>> what you do best, core application coding. Discover what's new with
>> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>> _______________________________________________
>> audacity-devel mailing list
>> aud...@li...
>> https://lists.sourceforge.net/lists/listinfo/audacity-devel
>>

> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
> trial. Simplify your report design, integration and deployment - and focus on 
> what you do best, core application coding. Discover what's new with 
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> audacity-devel mailing list
> aud...@li...
> https://lists.sourceforge.net/lists/listinfo/audacity-devel