From: Martin S. <mc...@as...> - 2006-06-23 11:55:29
|
I've been trying to figure out what the memory management conventions are for Rosegarden, but without any success. However my C++ is very rusty, given that I have been coding in pure C for many years. So I may just be missing something obvious. There are a couple of things that are puzzling me: 1. I haven't found any code that checks the return value of the new() operator, and I gather that the Qt library is compiled with exception handling turned off, by default. So, what happens when the new operator fails in Rosegarden? Does Rosegarden just crash with a seg-fault, relying on auto-saved data to allow subsequent recovery, or is there a mechanism that I haven't found, for making out-of-memory errors non-fatal? The reason that I'm asking, is because I want to know how to handle potential memory errors in the parameter area widget that I'm writing. 2. I've noticed many places where an explicit call to new doesn't appear to be matched by a subsequent call to delete. In many cases, this happens in constructors of objects that are intended to exist for the duration of the program. So such cases are relatively benign. However there are other examples, where a method function will call new() each time that it's invoked, but not subsequently call delete. An example is the RosegardenRotary::mouseDoubleClickEvent() function, in widgets.cpp. Every time that this function is invoked, it calls new(), to create a new RosegardenFloatEdit object, but then fails to delete this object before returning. Furthermore, in this case, even if the appropriate call to delete() were added, this wouldn't actually achieve much, because there's a similar problem in the RosegardenFloatEdit class itself. Whereas the RosegardenFloatEdit class has a constructor that explicitly calls new(), to create a few internal objects, and then assigns these to pointers, it subsequently relies on the default destructor that the compiler supplies for this class, which won't call delete for those pointers. Am I correct that these are memory leaks, or am I missing something basic about C++. Martin |
From: Chris C. <ca...@al...> - 2006-06-23 12:19:15
|
On Friday 23 Jun 2006 12:55, Martin Shepherd wrote: > 1. I haven't found any code that checks the return value of the new() > operator, and I gather that the Qt library is compiled with > exception handling turned off, by default. Operator new always returns a valid result, so there's no reason to check the return value. If it fails, an exception is thrown rather than returning null. You're right that we never bother to catch them -- if you run out of memory, the program simply crashes. Catching allocation failures when allocating large chunks of stuff is a good idea. It's very hard to recover meaningfully from failures in the sort of fine-grained allocation that goes on routinely in C++, though. If you can't allocate a few bytes for some transient data in a widget, you almost certainly can't warn the user or compose and write an autosave file either. If you have plenty of virtual memory and are trying to use an RT-mode audio server like JACK, your machine will probably have ground to a halt by then anyway. > 2. I've noticed many places where an explicit call to new doesn't > appear to be matched by a subsequent call to delete. In many cases, > this happens in constructors of objects that are intended to exist > for the duration of the program. So such cases are relatively > benign. However there are other examples, where a method function > will call new() each time that it's invoked, but not subsequently > call delete. An example is the > RosegardenRotary::mouseDoubleClickEvent() function, in > widgets.cpp. Every time that this function is invoked, it calls > new(), to create a new RosegardenFloatEdit object, but then fails > to delete this object before returning. I think that one is a leak. > Furthermore, in this case, > even if the appropriate call to delete() were added, this wouldn't > actually achieve much, because there's a similar problem in the > RosegardenFloatEdit class itself. Whereas the RosegardenFloatEdit > class has a constructor that explicitly calls new(), to create a > few internal objects, and then assigns these to pointers, it > subsequently relies on the default destructor that the compiler > supplies for this class, which won't call delete for those > pointers. That one isn't a leak. QObject (the Qt library object base class) destroys its children when it is destroyed itself. The widgets created in the RosegardenFloatEdit constructor all fall into that category, although that isn't necessarily obvious as the code that makes the first object a child of the dialog is hidden away in KDialogBase::makeVBoxMainWidget. That doesn't work for RosegardenRotary::mouseDoubleClickEvent, because although the object constructed is a child of the RosegardenRotary, there may be many double-clicks during the lifetime of the rotary: a new object will be created each time, and none of them will be deleted until the rotary itself is deleted (although they will all be deleted then). The dialog should have been explicitly deleted at the end of the function, or created on the stack. To complicate matters further, QDialog has an optional flag WDestructiveClose which can be set on construction. If this is set, the dialog will destroy itself when its exec() function returns, thus avoiding a leak. But that isn't set by default, and it can't be used here anyway because the code calls a method on the dialog after exec(). Chris |
From: Guillaume L. <gla...@te...> - 2006-06-23 12:42:08
|
Martin Shepherd wrote: > So, what happens when the new operator fails in Rosegarden? Does > Rosegarden just crash with a seg-fault, relying on auto-saved data > to allow subsequent recovery, Yes. As Chris said, it's worth to check if new() failed or not (and subsequently trying to recover) only when allocating really large chunks of memory. In the general case, it makes no sense, because by the time new() fails, your machine is already trashing like hell and begging for mercy. > The reason that I'm asking, is because I want to know how to handle > potential memory errors in the parameter area widget that I'm > writing. > Easy. Just ignore them :-) > pointers. Am I correct that these are memory leaks, or am I > missing something basic about C++. > To follow up on Chris's already well written answer : http://doc.trolltech.com/3.3/objecttrees.html -- Guillaume http://telegraph-road.org |
From: Martin S. <mc...@as...> - 2006-06-24 08:17:14
|
On Fri, 23 Jun 2006, Chris Cannam wrote: > Operator new always returns a valid result, so there's no reason to check the > return value. If it fails, an exception is thrown rather than returning > null. While this is probably the case for Rosegarden code, is it also true of the calls that Rosegarden makes to the Qt and KDE libraries, given that those libraries are usually compiled with C++ exceptions disabled (ie. -fno-exceptions) . For example, the following page at trolltech shows how to use Q_CHECK_PTR to check the return value of new(). http://doc.trolltech.com/3.3/debug.html > Catching allocation failures when allocating large chunks of stuff is a good > idea. It's very hard to recover meaningfully from failures in the sort of > fine-grained allocation that goes on routinely in C++, though. If you can't > allocate a few bytes for some transient data in a widget, you almost > certainly can't warn the user or compose and write an autosave file either. Once the program has allocated sufficient resources to bring up a user interface, and is running stably, if the user invokes a dialog, and the dialog fails to come up, due to lack of memory, then there is no reason why the dialog couldn't release any resources that it did manage to allocate, and then return control to its caller, such that the program is returned to the stable state that it was in before the user invoked the dialog. After that, although there might not be enough memory for the user to do anything else either, at least they have the option of trying. Furthermore, the user has the option of killing other, less important programs, to free up some memory, before trying again. In summary, I'm just trying to say that I can't see why a memory allocation failure in dialog, should take the whole program with it. Anyway, I get the impression that Rosegarden is in good company with lots of other programs, given the lack of any documentation at trolltech or kde.org, regarding error handling. So maybe the problem is so deeply ingrained at a much lower level, in the KDE and Qt libraries, that it would be pointless for Rosegarden to do anything different. I would need to look at the KDE and Qt source code to determine that, to check whether the library code check returns from new() etc.. > If you have plenty of virtual memory and are trying to use an RT-mode audio > server like JACK, your machine will probably have ground to a halt by then > anyway. Regardless, the computer shouldn't actually crash at that point, unless there is a bug in the kernel (or the computer is running Windows). So after something like this happened, I'd still be able to kill off whatever process was hogging the CPU, and recover my Rosegarden process, without losing what I was doing. > That one isn't a leak. QObject (the Qt library object base class) destroys > its children when it is destroyed itself. Okay. That makes sense. Thanks, Martin |
From: Guillaume L. <gla...@te...> - 2006-06-24 10:30:59
|
On Saturday 24 June 2006 10:17, Martin Shepherd wrote: > > Once the program has allocated sufficient resources to bring up a user > interface, and is running stably, if the user invokes a dialog, and > the dialog fails to come up, due to lack of memory, then there is no > reason why the dialog couldn't release any resources that it did > manage to allocate, and then return control to its caller, such that > the program is returned to the stable state that it was in before the > user invoked the dialog. Yes, there is. Memory allocation happens all the time. Even printing a trace may require allocating a small amount of memory. So once memory is filled you really can't do *anything*, short of exiting on the spot and possibly trying to save files (and I do mean *trying*, because this will require allocating memory too). > In summary, I'm just trying to say that I can't see why a memory > allocation failure in dialog, should take the whole program with > it. Anyway, I get the impression that Rosegarden is in good company > with lots of other programs, given the lack of any documentation at > trolltech or kde.org, regarding error handling. So maybe the problem > is so deeply ingrained at a much lower level, in the KDE and Qt > libraries, that it would be pointless for Rosegarden to do anything > different. I would need to look at the KDE and Qt source code to > determine that, to check whether the library code check returns from > new() etc.. No, the problem is simply that you just don't bother with memory allocation failures in a desktop application, except when allocating large chunks. Because, again, the machine is unusable way before this condition may occur. This is really a FAQ, all beginners with good intentions come up with it at some point. I've seen it discussed dozens of time, the answer it always the same : don't bother with it. > Regardless, the computer shouldn't actually crash at that point, > unless there is a bug in the kernel (or the computer is running > Windows). So after something like this happened, I'd still be able to > kill off whatever process was hogging the CPU, and recover my > Rosegarden process, without losing what I was doing. Have you ever sat at the keyboard of a machine in such a state ? Each action you take, be it moving the mouse or typing something, takes minutes to have an effet. Believe me, when you see it happen, you don't wait for some error message to tell you "i'm out of mem", you'll be running to the nearest xterm/konsole in hope you can do something while you can. -- Guillaume. http://www.telegraph-road.org |
From: D. M. 'S. M. <ros...@gm...> - 2006-06-24 15:00:37
|
On Saturday 24 June 2006 6:30 am, Guillaume Laurent wrote: > Have you ever sat at the keyboard of a machine in such a state ? Each > action you take, be it moving the mouse or typing something, takes minutes > to have an effet. Believe me, when you see it happen, you don't wait for > some error message to tell you "i'm out of mem", you'll be running to the > nearest xterm/konsole in hope you can do something while you can. Usually running to the nearest virtual console, because using anything in the GUI is all but utterly impossible in such a state. Then trying to figure out what is leaking, and KILL KILL KILL it, data be damned, unless it's something massively important that I haven't saved yet. I hate it when that happens. At least half the time, I just hit the reset button, and take my chances with what might not be on my disk on the rebound. It's a lot faster to fsck two 300 GB hard drives running old fashioned ext2 than to recover from a swap thrash fest. -- D. Michael 'Silvan' McIntyre ---- Silvan <dmm...@us...> Linux fanatic, and certified Geek; registered Linux user #243621 Author of Rosegarden Companion http://rosegarden.sourceforge.net/tutorial/ |
From: Martin S. <mc...@as...> - 2006-06-24 22:08:55
|
On Sat, 24 Jun 2006, Guillaume Laurent wrote: >... > This is really a FAQ, all beginners with good intentions come up with it at > some point. I've seen it discussed dozens of time, the answer it always the > same : don't bother with it. Okay, its clear that I am in a minority of one here. So now that I know the development philosophy behind Rosegarden, I will follow it. For the record, before I shut up, the reason that I brought this up, is because I personally have two decades of experience of writing software that doesn't crash, regardless of what the user does in the user interface, be it command-line or graphical. It certainly is possible, although maybe C++ makes it more difficult than C. Martin |
From: Guillaume L. <gla...@te...> - 2006-06-26 18:03:26
|
On Sunday 25 June 2006 00:08, Martin Shepherd wrote: > > Okay, its clear that I am in a minority of one here. So now that I > know the development philosophy behind Rosegarden, I will follow > it. For the record, before I shut up, the reason that I brought this > up, is because I personally have two decades of experience of writing > software that doesn't crash, regardless of what the user does in the > user interface, be it command-line or graphical. It certainly is > possible, although maybe C++ makes it more difficult than C. It certainly is possible, and I know that in some specific environments it is required, but in these cases the software is designed that way from the ground up, sometimes starting with the OS itself. However, in the case of a regular desktop app like Rosegarden, I've never seen a good solution being adopted for this. Admittedly, I can only justify of a single decade of writing software that, well, does crash from time to time, but I've seen this issue discussed by enough respected experts in their own field (Unix or C++ programming) that I'm pretty sure there's no better practical answer for this problem. -- Guillaume. http://www.telegraph-road.org |
From: Chris C. <ca...@al...> - 2006-06-25 14:24:47
|
On Saturday 24 Jun 2006 09:17, Martin Shepherd wrote: > On Fri, 23 Jun 2006, Chris Cannam wrote: > > Operator new always returns a valid result, so there's no reason to > > check the return value. If it fails, an exception is thrown rather > > than returning null. > > While this is probably the case for Rosegarden code, is it also true > of the calls that Rosegarden makes to the Qt and KDE libraries, given > that those libraries are usually compiled with C++ exceptions > disabled (ie. -fno-exceptions) . Good question. My guess would be that supplying -fno-exceptions when compiling a library that uses libstdc++ will not make any difference to whether or not an exception is thrown by a new operator in libstdc++ (i.e. the compiler won't magically switch to a version of the operator that doesn't throw). So I would expect a failed new in the library to throw std::bad_alloc which is then uncatchable because the exception is not propagated through the library. That's a guess though. It would be interesting to know. > Once the program has allocated sufficient resources to bring up a > user interface, and is running stably, if the user invokes a dialog, > and the dialog fails to come up, due to lack of memory, then there is > no reason why the dialog couldn't release any resources that it did > manage to allocate, and then return control to its caller, such that > the program is returned to the stable state that it was in before the > user invoked the dialog. The essential (usability) problem here is that the program is not really in a "stable" state even before the allocation fails. If there is no more virtual memory, then the program probably became unusable some time ago. (If the machine has a significant amount of swap.) Even then there are various different scenarios: 1. Rosegarden eats up all available virtual memory because of a bug. In this case, crashing is among the best things it can do. Somehow contriving to continue as if nothing had happened would be among the worst. I think this is the first scenario that comes to mind for most of us, because it bites us more often during development. That may explain why our response has been so negative, but it probably isn't what you were thinking of. 2. Rosegarden eats up all available virtual memory "legitimately" because the work it's trying to do needs it. It's easy to imagine a coarse recovery strategy that would at least be better than crashing and that could almost always be carried out successfully (abandon the current editing operation if there is one; close and destroy all GUI windows; save the document; pop up an error dialog; exit). One could probably implement this starting by simply catching std::bad_alloc in main, if one was confident that it would be propagated correctly. I'd certainly be happy to see something like this in Rosegarden if it actually worked. I guess in this situation, doing the minimum amount of recovery (abandoning the current dialog or operation and continuing) is unlikely to be a great strategy, because it's unlikely to return the machine to a really usable state. 3. Another application eats up all available virtual memory. Rosegarden is innocent. This is the situation in which the minimum disturbance would be ideal. You really don't want Rosegarden to do anything. Abandon no more than the current dialog or operation, and hope the other application gets killed first. I've made a bit of an assumption that one process or another is "at fault", which obviously isn't always the case. And there are various environmental complications. If another application goes mad and eats all the memory, Rosegarden is increasingly unlikely to get swapped in at all (making option 3 less likely than 2). If Rosegarden's GUI process runs out of memory, the sequencer process is unlikely to be able to continue either (and vice versa). An other application that is consuming the memory may be something Rosegarden relies on, such as the KDE file or print services. The Linux kernel also kills processes when memory is exhausted, and it doesn't always kill the right ones. I would be interested to know whether there are any C++ applications of similar size and GUI complexity to Rosegarden (and aimed at home users rather than in systems with defined service and uptime requirements) that do actually do rigorous recovery from memory allocation problems throughout -- and what percentage of those do more than just save state and bail out when something fails. Chris |
From: Martin S. <mc...@as...> - 2006-06-25 20:35:44
|
On Sun, 25 Jun 2006, Chris Cannam wrote: >... > that doesn't throw). So I would expect a failed new in the library to > throw std::bad_alloc which is then uncatchable because the exception is > not propagated through the library. >From my searches with google, this sounds about correct. As to the question of why exceptions are turned off in Qt, I have come across a couple of stories; one being that it was because exceptions weren't originally portable, and the other being that exceptions bloated the size of KDE/Qt applications, and slowed them down. If the unportability story is the correct one, then this would have made sense while it remained true. I don't think that is the case any longer though. The other story, that exceptions were disabled because they were slow, seems short-sighted. IMHO correctness and maintainability trump efficiency, and they are worth a performance hit. When I looked at C++ 14 years ago, exceptions were still problematic, and I gave up on C++ because of its apparent inability to deal with problems in constructors. For example, without exceptions, it appears to be impossible to check whether a copy-constructor (say invoked behind the scenes, by one argument of an argument-list), failed or not. Thus, the fact that exceptions are disabled in KDE and Qt, really bothers me, and I realize that without this being resolved, it would probably be pointless to try to improve the error handling in Rosegarden. > Even then there are various different scenarios: > > 1. Rosegarden eats up all available virtual memory because of a bug... Thankyou for the well thought out reply. To be honest, although I have been arguing from the viewpoint of memory exhaustion; in my experience, the most frequent cases of failed constructors have nothing to do with memory shortages at all. If a constructor checks its arguments, and finds a bad one, or it notices that a necessary external resource, (eg. /dev/snd/seq), is missing, then the constructor should throw an exception or return NULL. However if the caller of the constructor doesn't catch exceptions or check the return value of new(), then this will subsequently crash the whole program, without leaving any indication of what crashed it. If the bad arguments to the constructor were derived from user input, entering elsewhere in the program, or the missing external resource was not missing on the developer's box, then this won't be detected during developement, and the first indication of trouble, will be when a user reports that rosegarden crashed, possibly in a seemingly unrelated part of the program. Unless that user has the time and knowledge to run a debugger on rosegarden, you are then left with no idea what caused the seg-fault. Anyway, as I said at the top, without resolving KDE and Qt's lack of exceptions, there's probably no point in trying to add a formal error handling scheme to Rosegarden. Martin |
From: Guillaume L. <gla...@te...> - 2006-06-26 18:12:05
|
On Sunday 25 June 2006 22:35, Martin Shepherd wrote: > > As to the question of why exceptions are turned off in Qt, I have come > across a couple of stories; one being that it was because exceptions > weren't originally portable, and the other being that exceptions > bloated the size of KDE/Qt applications, and slowed them down. > > If the unportability story is the correct one, then this would have > made sense while it remained true. I don't think that is the case any > longer though. The other story, that exceptions were disabled because > they were slow, seems short-sighted. IMHO correctness and > maintainability trump efficiency, and they are worth a performance > hit. It's the 2nd one : exceptions were disabled for memory gains (quite substantial ones at the time, I recall). -- Guillaume. http://www.telegraph-road.org |
From: Chris C. <ca...@al...> - 2006-06-26 12:26:05
|
On Sunday 25 Jun 2006 21:35, Martin Shepherd wrote: > As to the question of why exceptions are turned off in Qt, I have come > across a couple of stories; one being that it was because exceptions > weren't originally portable, and the other being that exceptions > bloated the size of KDE/Qt applications, and slowed them down. The former sounds more likely to me. It took a long time for exception support to appear in g++ and they're still the quickest way to expose ABI differences between C++ compiler versions. There are other reasons why exceptions are used much less in C++ than in, for example, Java. One that I particularly dislike is that the throw declaration declares the set of exceptions that will be propagated out of the function, rather than statically guaranteeing that no other exceptions will be uncaught within the function. For example, this program: void f() { throw 1; } void g() throw() { f(); } int main(int argc, char **argv) { try { g(); } catch (...) { } } will (with g++ 3.3 anyway) compile without warnings but abort at run time, because g declares that it doesn't throw any exceptions but then fails to catch the one thrown by f. (It doesn't make any difference whether f declares its exception or not -- I left it undeclared to illustrate the point that the author of g might not even be aware that f can throw an exception.) The equivalent in Java would refuse to compile, quite properly in my opinion. In C++ the "solution" is never to declare exceptions at all. Chris |
From: Stephen T. <st...@to...> - 2006-06-26 15:16:51
|
On Mon, 2006-06-26 at 13:27 +0100, Chris Cannam wrote: > will (with g++ 3.3 anyway) compile without warnings but abort at run time, > because g declares that it doesn't throw any exceptions but then fails to > catch the one thrown by f. (It doesn't make any difference whether f > declares its exception or not -- I left it undeclared to illustrate the point > that the author of g might not even be aware that f can throw an exception.) > > The equivalent in Java would refuse to compile, quite properly in my opinion. > In C++ the "solution" is never to declare exceptions at all. The issue of refusing to compile and waiting to run-time is the choice of the compiler designers to be fair. I agree that the refusing to compile is a better option than the gotcha of finding out the program fails at run-time. Through the kindness of our campus library I got my hands on the 2003 C ++ specification. It says in Section 15.4 item 8 that "Whenever an exception is thrown and the search for a handler (see Section 15.3) encounters the outermost block of a function with an exception-specification, the function unexpected() is called ( See Section 15.5.2) if the exception-specification does not allow the exception." - Ref: The C++ standard - Incorporating Technical Corrigendum No. 1 So it appears that the g++ designers felt that this check should be done at run-time but it would be nice to be notified before hand that I will receive a call to unexpected() in the function g(). Stephen |