Re: [Mvpmc-devel] reducing memory use in MythTV client
Status: Alpha
Brought to you by:
gettler
From: <ge...@ac...> - 2008-03-10 01:08:48
|
On Sun, 09 Mar 2008 19:13:35 CDT, Roger Heflin wrote: > ge...@ac... wrote: > >> each * 500 shows if 50k). Also all of the allocations have some bookkeep >ing > >> > >> meaning that each malloc done through the internal memory management funct >ion > >> > >> has several bytes extra for keeping track of access, though it is unclear >to > >> me > >> if things would get screwed up if this was removed. > > > > The whole point of the internal memory management is to reduce memory usage >. > > The allocations have reference counts so that different parts of code can > > use the data without fear that it will be free'd out from under it. This > > would be far less of a problem without threads. The only other option woul >d > > be to copy the data, which would take more memory. > > Or implement some locking around things like the proginfo structure, if you > don't have the lock then you have to wait for it to touch it. I suspect tha >t > there may be more than one copy of the proginfo structure for a short period >of > time around a program delete/update happening, and that results in the really > > show and is why it appears to happen more around certain events (deletes) but > > also happens at other times if things line up wrong. > > Is this possible given the memory management structure in it? It looked lik >e > it was possible. The effect of the reference counted data structures is that as one thread dealloctes them, another thread may still hold references to them. As you suspect, this does mean that some data structures can be duplicated for short periods of time (ie, the currently highlighted recording will be in memory twice until the gui is refreshed). > >>> I assume that when it comes to recording groups, mvpmc is dynamically > >>> filtering and throwing away records that don't match the filter. So > >>> while it may not improve memory usage, it would improve performance if > >>> we could tell MythTV more specifically what groups we wanted. > >> Not that I can tell, it appears to load it all. > > > > Correct, all the program data is cached in memory so that it does not need > > to be constantly fetched. The data is only refetched when an event from > > the backend tells it to. > > > >>> What would improve memory is if we could tell MythTV to sort the results > >>> in our preferred way and return only the first screenfull of titles. If > >>> we can't do that via the MythTV protocol, maybe we need to resort to > >>> using the database connection (as much as I'd hate adding to that problem >). > >> I would guess just load the titles and subtitles and nothing else at all u >nti > >> l > >> it is needed to be displayed and then through that away immediately, given > th > >> e > >> size of the structure and the number of things it should reduce the memory > us > >> ed > >> quite a bit. > > > > I didn't think myth gave you a choice but to get it all. So keep in mind > > that you might be saving memory, but with 500 recordings, you would risk > > slowing down your interface quite a bit. This is true for under 100 > > recordings as well. The interface sped up quite noticeably after I added > > the event code and stopped fetching the recordings list every time the > > menu was changed. > > Given that the interface *FAILS* badly now on the mvpmc, and given the number > of > times the refresh happens and that happens pretty fast so even with 500 does >not > appear to be a serious issue. I can usually watch 1 maybe 2 shows before hav >ing > to reboot it. And given the number of refreshes I see just sitting there an >d > the speed of them, I don't think that will be an issue. The recordings data should only be refreshed on events like a delete, a recording starting, or a recording ending. So is the problem in your case watching the shows, or fetching the data from the backend? In other words, does mvpmc crash, lockup, or become unresponsive if you just let it sit there for days? Or do you actually have to watch something to see a problem? > > BTW, you and Tom both mentioned that you need to restart mvpmc every so > > often, pressumably due to memory problems. Do you see the memory useage > > of mvpmc grow over time? And does building librefmem with DEBUG defined > > point you to any memory leaks? > > Yes, I see it growing, if it hits over 10MB and generally gets horribly slow > and/or has to be green buttoned. Often I cannot even telnet into it to see >how > much ram it is using, if I am already in there, the ram usage does line up wi >th > the issues. > > I don't think at this point we even have enough ram to load the recordings wi >th > the librefmem turned on, I took a look at that, I also checked some of the co >de > paths that are common and did not see anything that looked wrong. > > But just going in and doing a few delete was significant enough to cause the > issue, also going into weather and then trying to view a mythtv video will > typically almost always cause a fatal crash when you view the video (it is ab >le > to pull up the listing, but starting a video is fatal). > > > > > Jon > > > > > > And I have every intention of setting up the minmem code in such a way that i >t > can be enabled when needed, and easily disabled to be fully functional, given > > that otherwise I know you won't accept the patches, and I do understand why. > > Roger BTW, assuming this is a memory leak of some sort, it would probably be far easier to debug while running under XWindows. Have you tried that? You can't watch a video that way, but the gui should work like on the mvp. Jon |