I want to try the updating of object database, so I add a timer to do the removeObject("1") and addFile("my_dir") every 5s. This way also can test if there is memory leak when you try to update the database. Then after 2 days, I check its memory usage using top cmd, I find it use 200MB memory and keep up and up.
So I am sure there is a memory leak when remove/add object. This seems a serious problem.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK, no details yet but I can confirm that it seems to be leaking somewhere. I basically repeated your test with the only difference, that my testcode was not inside the signal handler (was just triggered by it and executed in main.cc)
We are investigating the issue, thanks for reporting!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We found two serious and one minor leaks, they are fixed now. I tried adding and removing a directory of mp3's in a loop over night and the memory stays stable now.
So this issue should not occur in 0.8
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just have a question, I notice that the memory usage of mediatomb keep increase during I addFile() a directory with 50,000 files. Is this normal? And why it keep so much memory in hand?
Thanks ahead.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I noticed that for parsing video content libextractor allocates as much memory as it can.
Basically there are surely improvements regarding speed and memory consuption which we could/should do, but we also depend on the libraries for parsing metadata, and they do things their way.
At least we know that we are not leaking, we checked that; after the file import is complete memory should get freed and go back to "normal" levels.
The new release will introduce some improvements, especially regarding speed and CPU usage during media import. A lot of code has been rewritten and modified, that is probably also the reason why the release will take a little longer :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I want to try the updating of object database, so I add a timer to do the removeObject("1") and addFile("my_dir") every 5s. This way also can test if there is memory leak when you try to update the database. Then after 2 days, I check its memory usage using top cmd, I find it use 200MB memory and keep up and up.
So I am sure there is a memory leak when remove/add object. This seems a serious problem.
hi, thanks for the report.
let me see if I understand correctly:
you modified the code and you are calling those two functions repeatedly, right?
where are you calling them? can you please give more details on where exactly you added the calls so I can try to reproduce.
adding and removing items via the UI does not seem to have the same effect... so I would like to try it your way.
thanks,
Jin
I add a alarm signal handler in main.cc, and setup 5s to trigger alarm, the removeObject&addFile is called inside the handler.
OK, no details yet but I can confirm that it seems to be leaking somewhere. I basically repeated your test with the only difference, that my testcode was not inside the signal handler (was just triggered by it and executed in main.cc)
We are investigating the issue, thanks for reporting!
We found two serious and one minor leaks, they are fixed now. I tried adding and removing a directory of mp3's in a loop over night and the memory stays stable now.
So this issue should not occur in 0.8
Can you give me the fix now? Or tell me the changes in which files. I do not want to wait till 0.8 release. Thanks
Just have a question, I notice that the memory usage of mediatomb keep increase during I addFile() a directory with 50,000 files. Is this normal? And why it keep so much memory in hand?
Thanks ahead.
Did you compile with or without libextractor?
I noticed that for parsing video content libextractor allocates as much memory as it can.
Basically there are surely improvements regarding speed and memory consuption which we could/should do, but we also depend on the libraries for parsing metadata, and they do things their way.
At least we know that we are not leaking, we checked that; after the file import is complete memory should get freed and go back to "normal" levels.
The new release will introduce some improvements, especially regarding speed and CPU usage during media import. A lot of code has been rewritten and modified, that is probably also the reason why the release will take a little longer :)