From: Richard B. <ric...@fe...> - 2004-03-02 07:59:37
|
Ok, I think I'm getting somewhere with the ghost notes thing. The events are definitely getting deleted from the Composition OK, it's just the mmapping that is causing issues as first suspected. We're doing some horribly funky stuff with mmapped segment resizing and I reckon the memset() at the end of SegmentMmapper::dump() is getting it wrong. At this stage the size of the buffer has already changed to the new smaller version (when deleting a group of notes from a segment) so any zeroing of data in that buffer is going not going to extend to the end of it anymore. I remember from conversation about crashing in the sequencer that we don't actually shrink the mmapped file anymore because that would be terminal for the sequencer. So therefore (I think) we have the same size buffer and a partial memset going on inside it removing only one or two notes. So either we make the memset at the end of dump() cleverer by sending along the maximum mmapped file size (the true size) or we reinstate the complete memset - which works in the non-playing case but according to the comment attached to it is the wrong thing to do. Anyway, thoughts and logic fixes appreciated. R |
From: Richard B. <ric...@fe...> - 2004-03-02 09:30:11
|
On Tuesday 02 March 2004 07:45, Richard Bown wrote: > So > either we make the memset at the end of dump() cleverer by sending along > the maximum mmapped file size (the true size) or we reinstate the complete > memset - which works in the non-playing case but according to the comment > attached to it is the wrong thing to do. Hmm, but additionally it would appear that SegmentMmapper::computeMmappedSize() is returning a size that has no real relation to the number of events stored in the Segment (m_segment->size()). Or it's being used wrongly. Therefore even using this value to memset the mmapped file will produce erroneous results and ghost notes. At the moment the only solution I can see is to memset the whole mmapped file in SegmentMmapper::setFileSize when the new size is less than the old. However even this may well be wrong if we can't get the size of the Segment reliably. R |
From: Chris C. <ca...@al...> - 2004-03-02 11:07:38
|
On Tuesday 02 Mar 2004 9:17 am, Richard Bown wrote: > Hmm, but additionally it would appear that > SegmentMmapper::computeMmappedSize() is returning a size that has > no real relation to the number of events stored in the Segment > (m_segment->size()). Not sure I get what you're saying here. computeMmappedSize essentially multiplies m_segment->size() by the size of a mapped event (and the repeat count). m_segment->size() is indeed the number of events in the segment, so this should be correct. Chris |
From: Chris C. <ca...@al...> - 2004-03-02 11:01:58
|
On Tuesday 02 Mar 2004 7:45 am, Richard Bown wrote: > At this stage the size of the buffer has already changed to the new > smaller version (when deleting a group of notes from a segment) so > any zeroing of data in that buffer is going not going to extend to > the end of it anymore. The important question is whether the sequencer knows the new size or not. If it does, then it shouldn't matter whether there's garbage at the end or not as it should stop before reading it. If it doesn't then that's pretty bad, and looking at it, I think that probably is the case. Currently the way the sequencer keeps track of the size is that it just looks at the file size first, and then it expects remapSegment calls when something changes, passing in the new size. remapSegment didn't originally pass in the size, I added that to deal with the crash when unrepeating. Unfortunately remapSegment is only called if the segment changes during playback, not when stopped. So that suggests that if you change the segment when playback is stopped, you will get ghost notes: but as soon as you then make another change during playback, it should resynchronise itself. Is that the case? During our discussions about unrepeating I seem to remember wondering why we didn't just write the number of events into the shared memory area itself rather than having the sequencer guess it from the file size and expect to be told when it changes. I don't remember whether we ever worked out a reason not to do that. The other big question I have about all this is what happens at the sequencer if the remap call at gui/mmappedsegment.cpp:107 results in the memory location actually having to move. (As you see, I put in a warning to remind me to keep an eye out for that case, but I've never actually seen one yet. I should just make a test program.) Chris |
From: Richard B. <ric...@fe...> - 2004-03-02 13:22:32
|
On Tuesday 02 March 2004 10:55, Chris Cannam wrote: > So that > suggests that if you change the segment when playback is stopped, you > will get ghost notes: but as soon as you then make another change > during playback, it should resynchronise itself. Is that the case? Yeah, that is the case. I'd actually thought of this and considered adding an complete memset of the mmapped block only when the sequencer is stopped - but that's a bit of a horrible hack. > During our discussions about unrepeating I seem to remember wondering > why we didn't just write the number of events into the shared memory > area itself rather than having the sequencer guess it from the file > size and expect to be told when it changes. I don't remember whether > we ever worked out a reason not to do that. Hmm, I dunno enough about the update mechanism to be able to comment on that one. R |
From: Guillaume L. <gla...@te...> - 2004-03-02 22:35:28
|
On Tuesday 02 March 2004 08:45, Richard Bown wrote: > > partial memset going on inside it removing only one or two notes. So > either we make the memset at the end of dump() cleverer by sending along > the maximum mmapped file size (the true size) or we reinstate the complete > memset - which works in the non-playing case but according to the comment > attached to it is the wrong thing to do. Indeed it isn't. The problem with zeroing out the whole segment while playing is that there's a good chance the iterator will suddenly find lots of zeroes which to it means that the end of the segment's data is reached and there's no point in advancing any further. If you recall, this was the cause of the segments suddenly going silent when being edited while played. -- Guillaume. http://www.telegraph-road.org |
From: Guillaume L. <gla...@te...> - 2004-03-02 22:39:17
|
On Tuesday 02 March 2004 14:09, Richard Bown wrote: > On Tuesday 02 March 2004 10:55, Chris Cannam wrote: > > So that > > suggests that if you change the segment when playback is stopped, you > > will get ghost notes: but as soon as you then make another change > > during playback, it should resynchronise itself. Is that the case? > > Yeah, that is the case. Quite surprising given that we should be remapping everything right before playing. > > During our discussions about unrepeating I seem to remember wondering > > why we didn't just write the number of events into the shared memory > > area itself rather than having the sequencer guess it from the file > > size and expect to be told when it changes. I don't remember whether > > we ever worked out a reason not to do that. > > Hmm, I dunno enough about the update mechanism to be able to comment on > that one. I don't remember us finding a reason why not doing it, and I think it would be a good solution for our problems. -- Guillaume. http://www.telegraph-road.org |
From: Richard B. <ric...@fe...> - 2004-03-03 16:49:36
|
On Tuesday 02 March 2004 22:26, Guillaume Laurent wrote: > > Yeah, that is the case. > > Quite surprising given that we should be remapping everything right before > playing. Well there's something else to investigate then. > > Hmm, I dunno enough about the update mechanism to be able to comment on > > that one. > > I don't remember us finding a reason why not doing it, and I think it would > be a good solution for our problems. Ok then, so who's going to do this? We're getting a bit vague here and if we can't fix it properly we should at least fix it somehow and somewhere and soon. BTW I'm fixing the segment preview problem and getting sidetracked into the performance of the segmentcanvas when scaling. If I open something big like schumann.mid and zoom right in RG eats up all available memory and eventually falls over. Something odd and new going on there I think. Or something whacky with the code I've just been playing with. Anyone else seen this? R |
From: Chris C. <ca...@al...> - 2004-03-03 17:40:36
|
On Wednesday 03 Mar 2004 4:35 pm, Richard Bown wrote: > On Tuesday 02 March 2004 22:26, Guillaume Laurent wrote: > > I don't remember us finding a reason why not doing it, and I > > think it would be a good solution for our problems. > > Ok then, so who's going to do this? We're getting a bit vague here > and if we can't fix it properly we should at least fix it somehow > and somewhere and soon. I will -- this evening if I get time. And maybe I'll discover what it was that prevented me from doing it before. Chris |
From: Chris C. <ca...@al...> - 2004-03-04 10:48:14
|
On Wednesday 03 Mar 2004 5:33 pm, Chris Cannam wrote: > On Wednesday 03 Mar 2004 4:35 pm, Richard Bown wrote: > > On Tuesday 02 March 2004 22:26, Guillaume Laurent wrote: > > > I don't remember us finding a reason why not doing it, and I > > > think it would be a good solution for our problems. > > > > Ok then, so who's going to do this? We're getting a bit vague > > here and if we can't fix it properly we should at least fix it > > somehow and somewhere and soon. > > I will -- this evening if I get time. And maybe I'll discover what > it was that prevented me from doing it before. Nothing, I think, just fiddliness. It should be done now, but please test -- this is the sort of thing it's easy to miss supposedly-obvious bugs in. Chris |
From: Guillaume L. <gla...@te...> - 2004-03-07 23:11:18
|
On Wednesday 03 March 2004 17:35, Richard Bown wrote: > On Tuesday 02 March 2004 22:26, Guillaume Laurent wrote: > > > Yeah, that is the case. > > > > Quite surprising given that we should be remapping everything right > > before playing. > > Well there's something else to investigate then. According to RosegardenSequencerApp::play(), all segments are remapped. However I realize this is of little use to the case at hand (getting ghost notes if editting when playback is stopped), since the segments mmapped files are kept in sync by the GUI even when not playing (SequenceManager::checkRefreshStatus() and SequenceManager::segmentModified()). -- Guillaume. http://www.telegraph-road.org |
From: Richard B. <ric...@fe...> - 2004-03-08 07:44:00
|
On Sunday 07 March 2004 22:55, Guillaume Laurent wrote: > According to RosegardenSequencerApp::play(), all segments are remapped. > However I realize this is of little use to the case at hand (getting ghost > notes if editting when playback is stopped), since the segments mmapped > files are kept in sync by the GUI even when not playing > (SequenceManager::checkRefreshStatus() and > SequenceManager::segmentModified()). No, the problem was we _were_ getting ghost notes during playback after editing. So something was failing at both of those places when deleting notes from a segment and the reason was the sequencer didn't know that the mmapped segment had shrunk. Anyway, suffice to say it's fixed now because Chris implemented the event count (I presume) to ensure that the sequencer knows exactly how many notes to expect. R |
From: Richard B. <ric...@fe...> - 2004-03-03 18:30:04
|
> zoom right in RG eats up all available memory Yeah, for example: Empty composition, set duration to 300 bars and zoom right in to max. It takes its time resizing and eats up 163MB in the process. Adding some debug it looks like the thing that's taking the time is the QCanvas::resize() at gui/trackeditor.cpp:304. Looking in the Qt docs it warns that QCanvas::resize() is a "slow operation". Also apparently expensive for large canvas sizes. I did a cachegrind on this op and yes there's lots of mallocing going on. I'd always assumed that some ruler calculation was causing the glitches in zoom level change until I tried it now with all the rulers switched off. So the question is should we really be scaling the QCanvas at all? Shouldn't we be scaling the QCanvasView using transformation matrices? Is this what we do on the matrix view (he asks lazily without looking) ? R |
From: Guillaume L. <gla...@te...> - 2004-03-03 21:24:35
|
On Wednesday 03 March 2004 19:16, Richard Bown wrote: > > Shouldn't we be scaling the QCanvasView using transformation matrices? Yes, that's probably our only option. > Is this what we do on the matrix view (he asks lazily without looking) ? Yes. -- Guillaume. http://www.telegraph-road.org |
From: Richard B. <ric...@fe...> - 2004-03-04 07:01:47
|
On Wednesday 03 March 2004 21:10, Guillaume Laurent wrote: > On Wednesday 03 March 2004 19:16, Richard Bown wrote: > > Shouldn't we be scaling the QCanvasView using transformation matrices? > > Yes, that's probably our only option. Eeek. This will be fun. R |
From: Chris C. <ca...@al...> - 2004-03-03 19:30:23
|
On Wednesday 03 Mar 2004 6:16 pm, Richard Bown wrote: > > zoom right in RG eats up all available memory > > Yeah, for example: Empty composition, set duration to 300 bars and > zoom right in to max. You're right, it's dreadfully slow isn't it? I had expected zooming a blank canvas to take next to no time or memory. It might be possible to improve on it by changing the canvas chunk size (see docs for QCanvas::retune()) but I'm not sure how far that would count as a fix. The resize glitch thing looks OK to me now btw -- but I am seeing one other weirdness (related or not). Try opening mandolin-sonatina.rg and setting the zoom to 1000% or 2000%. When I do that, all the previews look right, and the first and third segment blocks look right, but the second block is mysteriously truncated just before the middle of the first bar even though its preview marches on into the distance regardless. One other thing is that we used to deliberately make the component lines of the segment previews 1 pixel shorter than they theoretically should be (unless they were already only 1 pixel long) so as to distinguish between consecutive notes more clearly. That seems to have been lost. Chris |
From: Richard B. <ric...@fe...> - 2004-03-03 21:05:44
|
On Wednesday 03 March 2004 19:23, Chris Cannam wrote: > It might be possible to improve on it by changing the canvas chunk > size (see docs for QCanvas::retune()) but I'm not sure how far that > would count as a fix. Hm, I don't like this line for retune() "For example if you have a very large canvas and want to trade off speed for memory then you might set the chunk size to 32 or 64." We have to improve both speed and memory usage. > One other thing is that we used to deliberately make the component > lines of the segment previews 1 pixel shorter than they theoretically > should be (unless they were already only 1 pixel long) so as to > distinguish between consecutive notes more clearly. That seems to > have been lost. Yeah, undoubtedly. That can be factored back in though pretty easily I should imagine. I'll take a look tomorrow along with the other reported thing. R |
From: Guillaume L. <gla...@te...> - 2004-03-03 21:27:54
|
On Wednesday 03 March 2004 20:23, Chris Cannam wrote: > > The resize glitch thing looks OK to me now btw -- but I am seeing one > other weirdness (related or not). Try opening mandolin-sonatina.rg > and setting the zoom to 1000% or 2000%. When I do that, all the > previews look right, and the first and third segment blocks look > right, but the second block is mysteriously truncated just before the > middle of the first bar even though its preview marches on into the > distance regardless. By some strange coincidence I appended exactly the same report to the original BR :-). -- Guillaume. http://www.telegraph-road.org |
From: Richard B. <ric...@fe...> - 2004-03-04 07:13:20
|
On Wednesday 03 March 2004 21:14, Guillaume Laurent wrote: > On Wednesday 03 March 2004 20:23, Chris Cannam wrote: > > The resize glitch thing looks OK to me now btw -- but I am seeing one > > other weirdness (related or not). Try opening mandolin-sonatina.rg > > and setting the zoom to 1000% or 2000%. When I do that, all the > > previews look right, and the first and third segment blocks look > > right, but the second block is mysteriously truncated just before the > > middle of the first bar even though its preview marches on into the > > distance regardless. > > By some strange coincidence I appended exactly the same report to the > original BR :-). Actually all the segments are weirdly truncated at high zooms. I'm not sure this can have anything to do with the preview stuff but I'm going to have a look at the QCanvas problem that's arisen now anyway and hence all of this stuff. Dunno how far I'll get mind you. R |
From: Chris C. <ca...@al...> - 2004-03-04 09:25:34
|
On Thursday 04 Mar 2004 6:59 am, Richard Bown wrote: > Actually all the segments are weirdly truncated at high zooms. Segments have always been truncated on occasion: if I import a MIDI file with nearly 700 bars in it, then the segments all get chopped off in bar 40 (at default zoom -- this depends on the zoom level) while the segment previews continue, only for the segment rectangles to start again in bar 656! I haven't previously seen quite such an obvious example as the one Guillaume and I mentioned just now, though. I've generally assumed that even though there's no theoretical limit to the size of the QCanvas, there may well be a theoretical limit (maybe SHORT_MAX, like for pixmaps and windows in X?) to the size of a single canvas item. Doesn't quite seem like a full explanation though. (Unrelatedly, I wonder why RG creates so much blank space at the end of a composition imported from a MIDI file?) Chris |
From: Richard B. <ric...@fe...> - 2004-03-04 07:23:44
|
On Thursday 04 March 2004 06:59, Richard Bown wrote: > Actually all the segments are weirdly truncated at high zooms. I'm not > sure this can have anything to do with the preview stuff but I'm going to > have a look at the QCanvas problem that's arisen now anyway and hence all > of this stuff. Dunno how far I'll get mind you. Double eek. There doesn't appear to be any notion of QCanvasView at the SegmentCanvas level. I think I might work this on a branch. Um, how do you check into a branch? R (far too early to be doing anything like this really but there you go) |
From: Chris C. <ca...@al...> - 2004-03-04 09:29:17
|
On Thursday 04 Mar 2004 7:09 am, Richard Bown wrote: > how do you check into a branch? There are two examples in the CVS manpage in the section on "commit". Generally you do "cvs tag -b some_tag_name" followed by "cvs update -rsome_tag_name" and then your following commits will go to that branch. Chris |
From: Richard B. <ric...@fe...> - 2004-03-04 13:00:28
|
On Thursday 04 March 2004 09:22, Chris Cannam wrote: > cvs update -rsome_tag_name Ok, so what if I've got two checked out copies. One is recently updated and unmodified, the other one I've already done some branch work on. If I tag/branch the unmodified copy and then cvs update the modified branch as above can I then be sure that the already modified code in that dir will be checked into that branch rather than HEAD? Should I clean it up and reapply the changes after the branch update to be sure? R |
From: Chris C. <ca...@al...> - 2004-03-04 13:06:10
|
On Thursday 04 Mar 2004 12:46 pm, Richard Bown wrote: > On Thursday 04 March 2004 09:22, Chris Cannam wrote: > > cvs update -rsome_tag_name > > Ok, so what if I've got two checked out copies. One is recently > updated and unmodified, the other one I've already done some branch > work on. If I tag/branch the unmodified copy and then cvs update > the modified branch as above can I then be sure that the already > modified code in that dir will be checked into that branch rather > than HEAD? Yes. But you should be able to tag either copy: tagging affects the repository, not the working directory (unless you use the -c flag), so if you tag the modified copy it should simply tag the last version you updated from. The update on the modified copy should then merge any changes and record locally the fact that you're now going to commit to the branch. > Should I clean it up and reapply the changes after the > branch update to be sure? No, I'm pretty sure that's unnecessary. Remember that cvs commit tells you what branch you're committing to (in the comment at the bottom of the commit log edit page), so as long as you pay attention you can be fairly sure you're not committing to HEAD. Chris |
From: Richard B. <ric...@fe...> - 2004-03-04 13:09:28
|
On Thursday 04 March 2004 12:59, Chris Cannam wrote: > Remember that cvs commit tells you what branch you're committing to (in the > comment at the bottom of the commit log edit page) Hmm, no I'd never noticed that actually. Ta. So has anyone had a look at this CVS replacement thing yet that all the kids are talking about? R |