Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#35 audio split with clicks and rubberband time stretching strange behavior

svn_trunk
closed
nobody
None
2013-05-06
2013-01-20
Alex Chernyshoff
No

I do not know how to describe the problems exactly, but I'll try. (I apologize for my English)

  1. When you split a sound clip into parts - the next time you play, in some joints can be heard audibly click, which should not be. The situation depends more on the location in which the splitting was than the time and other factors. In the system there is no xruns. When playing material recorded without splitting sound is excellent.
    Clicks occur in a regular manner. Where they are - they are always the same every time you play.

  2. The "time stretching" by SHIFT-dragging mode something is not working on audio clips.
    The problem manifests itself in the mode "rubberband".
    The point of it is that - when I split in a few places the clip, and try to adjust the rhythm (to correct vocal tempo inaccuracies) by moving the joints (through a "shift-dragging" by the edges) - instead of the expected change in the length, the content of the sound clip unimaginable way changing. Content of parts are jumps and changes its position in time quite differently than expected - in some unpredictable manner. It is hard to explain, but it sounds awful.
    I tried to change the recording mode - 24 bit signed, 32 floating, all uncompressed, but the result is the same.

My system: Arch Linux x86_64 updated to the present day, qtractor-0.5.6 from the official repository, and qtractor-0.5.7 from sources (perfoms the same manner).

Discussion

  • 1.is kind of unavoidable and probably much more noticeable for content that is rich in low frequency energy. anyways, you're making a hard cut (split) on some point and hardly making any cross-fading at all. of course, the precise location where the split is done is extremely important. ideally it should be done on silent locations only.

    2.is hard to understand i'm afraid. what really is you seeing as badly behaving or wrong with rubberband time-stretching? pardon me, my english might not be better than yours, so that it would be of great benefit for you to explain it with the help of some reproducible evidence, a session (.qtz) file perhaps? ;)

    cheers

     
  • I apologize for the delay in response. At the moment I am very busy, but as soon as I can, I will record a track with a simple voice containing the problem, and send the project.
    Now double-checked - and the problem appears to "rubberband" mode, and the mode "WSOLA" too.
    The sequence is as follows:

    1. File -> New. View -> Buses. Create new audio bus "Vocal" with mode = "input", and 1 channel width. (mono bus)
    2. Add audio track (Track 1), at input bus Vocal, and output bus is dafault Master.
    3. Connect jack input "capture_1" (my microphone) to "Vocal/in", and connect Qtractor outputs to jack playback ports. Save session.
    4. Press "monitor" knob at "Track 1" mixer window, and arm this track for record. Now i can hear myself.
    5. Press Record, press Play, start recording. After 4-5 measures (9-10 seconds) stop. Check recorded. We see that it is normal play.
    6. Ctrl-click play-head at middle of the clip, and split. Stepping a couple of seconds ahead, again doing the splitting. We get three fragments. Save for future.
    7. With the "shift-dragging" move the left edge of the middle piece, adjusting it to the nearest grid, and then move the edge of the neighboring fragment in the same position. We ideally would be to get a change of pace without changing the pitch, with no breaks in the sound. But instead of his plans in the middle piece somehow gets a piece of the third (last) piece. At the hearing, it looks like some kind of confusion in the sound, with breaks at split points.
     
  • As I promised, I'm sending the archive "session1.qtz", where I sang a verse of a simple russian children's song, which appears in full problem "rubberband time stretching". Not a masterpiece of performance, but is suitable for example :)

    The project contains three tracks.

    The first track - the original clip, but split into 3 parts. Object of interest starts at 6 seconds. This track is unmuted by default.
    Track 3 - a copy of the first, but the joint place of fragments 1 and 2 are shifted slightly to the left in the 6 second point using the "time stretching". This track is muted by default.
    Track 2 - just midi tune. (used WhySynth)
    If alternately play tracks 1 and 3 - you can catch the essence of the problem. In track 3, the middle fragment contains absolutely not what was expected. In addition, there is very clearly audible artifacts of the algorithm "rubberband" - the sound quality is very poor compared to the original.

    Thanks for your project :)

     
    Attachments
    • ouch.

      i can now confirm that time-stretched audio clips are seriously broken in regard to their internal offset parameter setting (when not zero ie.at absolute beginning of the audio sample.file)..

      will investigate this further and try to fix it asap.

      thanks for the patience.
      cheers

       
      • Hi!
        A few words about this "time stretching"... In fact, it is very necessary and important function. Very very very much for me. In the process of recording sometimes occur rhythmic mistakes. Not everyone can just play perfectly with the metronome :)
        It would be very useful to be able to correct these mistakes are more convenient way. If there was such an opportunity - to move the position of splitting a clip without moving alternately left and right sides, and immediately moving for both splices at split point simultaneously. That would be just great!
        Still, there are times when it is necessary to shift the point of splitting in an arbitrary place, not tied to the grid. If the moving part of the split clip one by one, it will be very difficult to get to the same point. As I understand, is "shift-dragging" and "ctrl-dragging" doing the same thing? For example, it would be nice one of these combinations to determine for such behavior...
        Oh and... it would be cool to look at the sound quality of this "time stretching". It is just not satisfactory. Even with small shifts, there are any ringing and wheezing badly spoiling the sound.
        The strange thing is that the quality of the algorithm "rubberband" itself is very high. I specifically heard the sound samples on their website. The quality of the algorithm is just a wonderful...
        I do not understand why it was so bad manifests itself in "qtractor". I do not know ... perhaps this error with respect to the positioning of the clip somehow influences too.

        Thank you again for this wonderful project "qtractor". I have long been following him and I like what I see, intuitive interface, comprehensive capabilities, rationality and conciseness. Very, very good! :)

         
        • it turns out, as an early investigation result, there was one single but nasty semi-colon typo hiding in the peak files descrimination used to draw distinct waveforms of time-stretched audio clips.

          this was causing all audio clips of same sample file showing the very same and original waveform no matter each clip was time-stretched or not. iow. visual waveform display was wrong and not in sync with audio playback, which was correct. a very illusive bug indeed, now fixed on svn trunk (qtractor 0.5.7.13+). thanks

          however it doesn't fix it all. changing audio clip lengths via the shift/ctrl+click+drag procedure and altering the time-stretch factor in direct proportion is not what i would call perfect, yet. i'll try to have a closer look in the next days. be patient.

          [UPDATE:] qtractor 0.5.7.14+ (svn trunk rev.3167+) has it fixed now. hopefully. thanks again.

          otoh. imo. the rubberband algorithm quality is not that constant for all kinds of sounds. i guess, what seems right with some is bad enough for some others. also, time-stretching is more appropriate for sounds with a strong rythmic content, like drums. instead, results may and often vary a lot against melodic and/or harmonic sounds. eg. a "metalic", "robotic" sound effect may often be recognized.

          seeya

           
          Last edit: Rui Nuno Capela 2013-01-27
    • status: open --> pending
     
    • milestone: 0.5.7 --> svn_trunk
     
    • The latest version of Qtractor from SVN works well. Time stretch is working properly.
      Thanks a lot, Rui!!!

      As for the rubberband - it all remains as before.
      For sources such as the human voice, it does not work well.
      I did a tests on different audio files directly from the terminal, tried different options... Nothing good with vocals.
      However, the transformed sample from the site - "Example 1 - Jazz ensemble" sounds great. Indeed, for a different sound, rubberband works very differently.
      Sadly, basically this is very necessary for monophonic solo instruments and vocals. Especially where it is necessary to correct a little rhythmic mistakes
      made in the process of recording with a metronome.

      Of course, all that concerns rubberband is offtopic in some way ... Although there are algorithms that work well, but not open source. For example, "Reaper" uses Zplane Elastique, SoundTouch and DiracLE. Elastique quality is very very good. They have a free SDK, but the library is not free, unfortunately ...

       
      • re. rubberband...

        yes, zplane's elastique is an overall winner and has been like so for the last decade at least.

        while on the free-software camp the best we have is indeed rubberband and/or soundtouch, both are provided in qtractor (nb.the wsola time-stretching option is a ripped off version adapted from soundtouch's;).however, you're not the first, and certainly not the last to praise the quality and even performance factor of zplane's. :)

        otoh. please try the rubberband command line utility against some of your recordings. experiment with different options and see whether one fits best to your case.

        byee

         
        Last edit: Rui Nuno Capela 2013-02-04
        • Yesterday, I experimented a lot with the rubberband. The quality of the vocal sound is not very good.
          But today I tried SoundTouch v1.7.1, and was very surprised. The quality is amazing!
          Something like that:
          soundstretch session1-Track_1-6.wav o.wav -tempo=20 -speech
          Pay attention to the option "-speech" :)
          Sound is simply amazing!!
          Listening to the results over and over again, I could not see a single artifact. Sounds perfect for me!
          I am very impressed and looking forward to the future :))
          But WSOLA, which is built into Qtractor, unlike SoundTouch, just sounds distorted. Some wheezing and crackles heard good.
          I do not know how to explain it...

           
  • something fishy has been detected in the (not so) sse optimized code from soundtouch-inspired wsola time-stretching... fixed in svn trunk rev.3182+ (aka. qtractor 0.5.7.19+)

    please test and report whether wsola time-stretching sounds any better now.

    byee

     
    • WSOLA now works almost perfectly, though sometimes, I think, in the processed audio are some minor clicks in the same places. But this is not xruns, and they are not in the original recording. I still have more to do experiments, write and listen to the different tracks.
      Thank you very much, Rui! :))

       
    • Yes, Rui, I tried several times to record different tracks and that's what I discovered.
      For small changes, around +-5% distortion usually does not occur. Small distortion (wheezing) generally begins when the rate of change of more than 20%. This usually occurs on a fragment that is stretched out (the rate is slowing down).
      That is, in general, small distortions do occur.

       
      • can you try with a new build with

        ./configure --disable-sse
        
        make
        
        src/qtractor
        

        and tell whether it makes any difference re. wsola time-stretching.

         
        • Rui, it is possible that you did not see my last comment (it seems I posted it in the wrong branch of this thread). Just a month has passed, and no news...

           
          • was it the 'small distortions' comment?

            yes i've seen it but i'm afraid a can't do much more or anything about it this time. i'm afraid that's inherent to the time-stretching algorithm and specific sound samples.

            sorry

             
            • "small distortions" - small glitches, unwanted audio artifacts in some places of the the clip.
              I'll think about ways to show the problem more clearly. May be even better for this use pure sine wave. As soon as I get home, I'll try to spend a few more tests, as well as independently to process them with soundstretch command line tool, and compare between them.
              Thanks, Rui!

               
            • Now I have created and tested a simple project with a single tone of 440 Hz. Now the problems with the time stretch alone i can not see, but hear strong clicks in joint point of splitten clips. Despite the fact that the cross fading is disabled in the clip properties (F4 shortcut), feeling that a small amount was going on.
              In theory this should not be. If the clip is zero fading, the transition between tightly stacked parts of clip should come without any clicks, from which so headache :)

               
              Last edit: Alex Chernyshoff 2013-03-09
              Attachments
              • all audio clips have implicit fade-in/out to zero at start/end edges.

                please confirm that by recording a unmuted track output and look at the waveform in detail, using a proper audio sample file editor (eg. audacity).

                though, these so called micro fade-in/outs may not be enough to avoid click-popping completely. again, it really depends on the sample signal content, specially on signals with hot low-frequency power spectra. the micro fade-ins/outs may help a bit but not the definitive and absolute solution.

                cheers

                 
                • About implicit fade-in fade-out, I understand. But, I think it is not very useful to do it always at the split point of the arbitrary clip, as almost always leads to a click.
                  The attached project is the case. There pure sinusoid.
                  So, when we split the clip (logically), at split point there is almost unavoidable click. Of course, if at the point of splitting the signal amplitude is close to zero, a click will be less audible.
                  It would be nice to have some opportunity to do a default "no-fading-in-out" at the split point. If at any split point (there can be many in the original clip) will be some amount of fading at edges, time stretching the complex rhythmic pattern will be almost useless.
                  I do not know how else to explain :)
                  Well ... this is so, for example, in Reaper. When we just split the clip, at this point the sound does not change at all. There is no any fading, unless of course it is not done manually later:)

                   
  • I do it, Disabled SSE, recompile. Move the tracks here and there, listen - nothing has changed. Light distortions exactly in the same points of stretched fragment.
    (Yes, everything happens in the WSOLA mode)

    P.S. The distortions are relatively small - some kind of glitches (not a click) at fixed places of clip, regarding value and size of stretching.
    I tried recompile with more safe CFLAGS, and nothing was changed.
    P.P.S. Something strange appear. In the file "./configure" option "--disable-sse" is missing, only "--enable-sse" is present. It is possible that yesterday's experiment with recompilation is not quite correct. Or maybe I'm wrong?

     
    Last edit: Alex Chernyshoff 2013-02-07
    • status: pending --> closed