Re: [Audacity-devel] New error reporting feature for recordings

[Cliff S. <fly...@gm...>]

> On Jan 16, 2018, at 8:25 AM, Peter Sampson <pet...@gm...> wrote:
> 
> 
> 
> On Tue, Jan 16, 2018 at 1:51 PM, Paul Licameli <pau...@gm... <mailto:pau...@gm...>> wrote:
> 
> 
> On Tue, Jan 16, 2018 at 8:02 AM, Peter Sampson <pet...@gm... <mailto:pet...@gm...>> wrote:
> Hi Paul,
> 
> I've just been testing and documenting this today:
> https://alphamanual.audacityteam.org/man/New_features_in_this_release#dropouts <https://alphamanual.audacityteam.org/man/New_features_in_this_release#dropouts>
> https://alphamanual.audacityteam.org/man/Recording_Preferences#Options <https://alphamanual.audacityteam.org/man/Recording_Preferences#Options>
> 
> I tested on the latest Mac nightly using the temporary Tools menu item to simulate 
> recording errors.
> 
> I note, gratefully, that the "error label track" was a newly created label track and did not
> corrupt my existing label track.
> 
> 
> My initial comment is made in the light of your comment in this thread and in one of your commits
> >Allow reporting of drop-outs to be turned off by Recording prefs...
> >
> >Just in case it turns out to make some false positives.
> 
> If there is any chance whatsoever of false positives them I'm thinking that the default for this new
> option should be "off" rather than "on".
> 
> I would not want my perfectly good recorded audio corrupted by any such false positive.
> 
> No, I still want to have the detection enabled by default, else there is much less value in this project.  The naive users who might most be helped would not know to turn detection on, unless they are talking to someone like me and have already lost some data and not known it until listening again to the recording.
> 
> To explain a bit more:  the project was intially rolling two different detected error conditions into one.  I now think only one is too easy to get spuriously, but not the other.
> 
> I tried making the error happen naturally, not as a simulation, by running Audacity simultaneously with other programs eating a lot of CPU (in fact, another complete compilation of Audacity.)
> 
> The portAudio library itself can report an error code as an argument to the audacityAudioCallback function, where we do the work of copying the recorded samples it provides into our data structures.  This error code had just been ignored before.  I tried using this error code.  But then some experimentation on macOs convinced me these errors might be spurious.  (Perhaps they won't be spurious on other operating systems?)
> 
> It was this source of error that I stopped detecting.
> 
> But there is another source of possible error -- which I was NOT able yet to provoke into happening on my Macbook, but is more surely real data loss if it ever happens.  This other error happens if the writes of recorded sample data into block files on disk is not happening at a fast enough rate to keep up with the stream of recorded data.  It could be too much CPU is used, or it could be that the external device is too slow to keep up.
> 
> I feel more confident that if the program ever detects this, then it is a data loss error, for real.
> 
> Does anyone on the team intentionally keep some low-powered computers for testing purposes, as Gale did?  Try to make the error happen on that.  Run some CPU hog program concurrently with Audacity, while recording into a project that was previously saved onto a slow external storage device.
>  
> 
> 
> Additionally I note that the error message is rather long (TL;DR possibly) - in general shorter messages
> are to be preferred to longer ones as users are more likely to read them properly.
> 
> 
> Some error message should be agreed on so that string freeze isn't delayed too long.
> 
> Remember the project for exhaustion of disk space?  There, it was important to make some message that suggested the remedial actions to take.
> 
> I think now there are three suggestions to make: (1) close other programs, (2) don't use slow external storage devices, and now, (3) disable this warning in Preferences (if it does turn out to be spurious too often, which I doubt).
> 
> Or there could be a checkbox for "don't remind me again" instead of (3).  We have such a dialog with checkbox already.  Start with no audacity.cfg, and save a project:  there is that warning that you are saving, not exporting.  I could do an analogous thing.
> 
> For the main message text, I think also just an empty line or two breaking the long paragraph might make it a little less intimidating.
> 
> So then, can we agree on a text?
> 
> 
> 
> 
> How about slightly restructuring the message to aid readability
> 
> Recorded audio was lost at the labelled locations. Possible causes:
> 
> other applications are competing with Audacity for processor time,
> you are saving directly to a slow external storage device.
> 
> 
> (Aside re US spelling:  is it double-l "labelling" or single-l "labeling" in US spelling?  US spelling is mandated by Connie. )
> 
> 
> 
Labeling would be the normal US spelling. Labelling is accepted by the spell checker, but the dictionary says it is the British spelling.

Cliff


> Peter.
> 
> 
> 
> 
> Finally, a few more thoughts I just had.
> 
> The program itself might take a certain corrective action.  If it detects the (second, likely non-spurious kind of) error, it can't easily correct the problem mid-recording, but it can do things differently if you stop and try recording again.
> 
> Technically, it can allocate more space in the "ring buffer" that communicates recorded data between two threads.  This longer internal buffer could compensate for slowness of storage.
> 
> And finally that suggests the means by which I simulate recording errors in a development build should be different.  Instead of just throwing blocks of data away randomly, it could do the opposite to the correction:  reduce the allocation of the ring buffer to make lost data, not a near certainty as now, but rather just more likely.  I could make a dialog with a tunable parameter for the length of this buffer.  You could reduce it until errors do happen for you.  Then, you are getting more "realistic" errors.  And then you could also test whether the program's correction of that error, by automatically lengthening the buffers again, is working.
> 
> PRL
> 
> 
>  
> Peter.
> 
> On Mon, Jan 15, 2018 at 11:50 PM, Paul Licameli <pau...@gm... <mailto:pau...@gm...>> wrote:
> 
> 
> On Mon, Jan 15, 2018 at 6:23 PM, Peter Sampson <pet...@gm... <mailto:pet...@gm...>> wrote:
> 
> 
> On Mon, Jan 15, 2018 at 11:19 PM, Paul Licameli <pau...@gm... <mailto:pau...@gm...>> wrote:
> 
> 
> On Mon, Jan 15, 2018 at 5:57 PM, James Crook <cr...@in... <mailto:cr...@in...>> wrote:
> +1 to doing something about drop outs. Bravo.  I have found drop outs very annoying in the past.  Much better to know about them at the time of recording.
> 
> I think though that this is a risky change to make late in 2.2.2.
> There is not going to be a lot of time to thoroughly test it out on a range of different machines before 2.2.2 is released.
> 
> For example, it is possible that creating labels will slow the recording down further, and lead to many more drop outs, on slow machines.
> 
> If you mean the labels created by this project, no, it doesn't happen until recording is finished.
>  
> 
> I think if you are serious about this for 2.2.2, as a minimum it needs a preference that can be used to disable it.
> recording-preferences -> Report recording errors  .  The preference can be normally on by default.  If we do find it is causing problems, we can tell affected users to disable it, rather than have to issue an emergency fix release (or tell them to go back to 2.2.1).
> 
> My only concern is that it might make false-positive reports, which I have not seen yet.  You are right that making a preference to disable the warning could save us if that ever happens.
>  
>   But by default, I would want the reporting turned on.
> 
> 
> +1   I agree
> 
> Peter
> 
> Quite easily done.
> 
> I have added a checkbox to Recording preferences.  Its label is a third user-visible string for this project:
> 
> "Detect dropouts"
> 
> This is at commit d2749068f145519a6b71dd91ca72961da8f7b52d.
> 
> PRL
>  
> 
> 
> 
> But if ever it makes real reports, it will have done a very useful thing for the users.
> 
> PRL
> 
>  
> 
> --James.
> 
> 
> On 1/15/2018 8:29 PM, Paul Licameli wrote:
> I pushed it at 9777d3e88009e593035c77be2a5835ce76ce0765
> 
> Up first: what matters before string freeze:  There are two new
> user-visible strings.  Suggest detailed wording changes soon, please!
> 
> 
>     - "Errors" is a name given to a new label track.  (I decided "Recording
>     errors" was too long to appear properly in the TCP menu button.)
>     - A long message box text:
> 
> 
> "Recorded audio was lost at the labelled locations.\n\
> This may have happened because you are saving directly to a \
> slow external storage device, or because other applications are \
> competing with Audacity for processor time."
> 
> Secondly: the rationale.
> 
> In my usage of Audacity, before I get a faster computer, I did notice,
> rarely, on a Windows laptop that occasional syllables of my recorded voice
> were dropped out, as if a few milliseconds had been deleted.  I noticed
> this only becauase I proofed my work carefully.
> 
> I have heard a few complaints of similar things too from users, in a
> discussion group I manage for narrators.  I think that in one case, someone
> was trying to record directly onto a slow external device, like a thumb
> drive.
> 
> In a few other cases, the recording glitches were very dense in a short
> space, making the recordings unsalvageable.  The users ultimately decided
> that it was a symptom of the computer's internal storage devices going bad,
> because bad effects were seen with other programs.
> 
> In most cases, the default suspicion is that Audacity is competing for CPU
> with other applications or background virus checkers, which should be
> closed and disabled before you try again.  Audacity fails to keep up in
> real-time with the stream of input samples because of this.
> 
> In all these cases, the user only knows there is a problem when listening
> to the recording again.  But if Audacity has the means to detect the error,
> it should give the user some alert, sooner, so the problem might be
> corrected before recording again.
> 
> And Audacity was detecting the errors, but only writing debug messages to
> standard output that only programmers were likely to notice:
> wxPrintf(wxT("lost %d samples\n"), (int)(framesPerBuffer - len));
> 
> This new feature gives the user an alert.  I am not wholly satisfied,
> because the alert is a modal dialog that happens only after the recording
> is complete.  I think a better solution might pop up a modeless dialog
> during recording, as soon as the problem is detected, giving the user the
> option to stop the recording, but otherwise continuing it.  This would let
> a user stop a bad recording very early instead of going on ten or twenty
> minutes only to discover then that it was bad.  But this requires some more
> programming effort.  Perhaps for a later release.
> 
> On the other hand, if the errors are very few and far between, and easily
> repaired with a few pickups, then this feature should be very convenient to
> pinpoint them.
> 
> Thirdly: the details of this project.  Quoting my commit comment:
> 
> 
>      Alert user to drop-outs during recording...
> 
>      1) When the program detects this, insert zeroes into the recording to
> keep the
>      other good parts synchronized.
> 
>      2) When recording stops, a message box alerts the user, and a label
> track is
>      added showing the lost parts, labelled with consecutive numbers.
> 
>      3) A menu item visible in alpha builds only is added to Tools, to
> simulate
>      recording errors at random times and test the reporting feature.
> 
> Lastly:  I would like others to try this out and comment.  Try it with the
> simulated errors to see if you like the details of the reporting
> mechanism.  Try recordings without this, to be sure we don't get frequent
> false positives on any operating system.
> 
> PRL
> 
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot <http://sdm.link/slashdot>
> _______________________________________________
> audacity-devel mailing list
> aud...@li... <mailto:aud...@li...>
> https://lists.sourceforge.net/lists/listinfo/audacity-devel <https://lists.sourceforge.net/lists/listinfo/audacity-devel>
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot <http://sdm.link/slashdot>
> _______________________________________________
> audacity-devel mailing list
> aud...@li... <mailto:aud...@li...>
> https://lists.sourceforge.net/lists/listinfo/audacity-devel <https://lists.sourceforge.net/lists/listinfo/audacity-devel>
> 
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot <http://sdm.link/slashdot>
> _______________________________________________
> audacity-devel mailing list
> aud...@li... <mailto:aud...@li...>
> https://lists.sourceforge.net/lists/listinfo/audacity-devel <https://lists.sourceforge.net/lists/listinfo/audacity-devel>
> 
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot <http://sdm.link/slashdot>
> _______________________________________________
> audacity-devel mailing list
> aud...@li... <mailto:aud...@li...>
> https://lists.sourceforge.net/lists/listinfo/audacity-devel <https://lists.sourceforge.net/lists/listinfo/audacity-devel>
> 
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot <http://sdm.link/slashdot>
> _______________________________________________
> audacity-devel mailing list
> aud...@li... <mailto:aud...@li...>
> https://lists.sourceforge.net/lists/listinfo/audacity-devel <https://lists.sourceforge.net/lists/listinfo/audacity-devel>
> 
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
> audacity-devel mailing list
> aud...@li...
> https://lists.sourceforge.net/lists/listinfo/audacity-devel