Menu

#224 Huge memory footprint for a project with many producers

head
wont-fix
nobody
None
5
2015-02-21
2015-02-15
No

$ ./src/melt/melt timer001.mlt

Melt process allocates about 5.5 Mb every second. After 3 minutes it's +1.1Gb.

1 Attachments

Discussion

  • Dan Dennedy

    Dan Dennedy - 2015-02-17

    Some producers such as avformat, pixbuf, and qimage use mlt_cache to alleviate this problem, but the pango and qtext producers do not.

     
  • Dan Dennedy

    Dan Dennedy - 2015-02-19
    • status: open --> pending
     
  • Dan Dennedy

    Dan Dennedy - 2015-02-19

    Are you specifically interested in the timer and/or pango producers, or is the attachment just an example? There might be other ways to achieve your goal.

    If you are requesting that long (in numbers of objects) or complex projects use less memory, then I am afraid that is just the way it is, and there are no plans to address it.

     
  • Nikita Melnichenko

    I was interested to create a timer with a custom style (clock plugin settings were insufficient). This was the first solution inspired by kdenlive counter generator. Then I discovered the crash and memory problems, so I tried to find how to keyframe plugin properties (like in Sony Vegas), however it looks it's not supported in a general way yet. After some time I luckily discovered pango_keyframes.mpl example which almost does not eat memory - this was very useful and I implemented my timer based on this.

    The reason I created the bug was raising the awareness among mlt devs. I have no problem that one pango producer consumes 6Mb. The point is that at time T seconds I don't need (T - 1) old producers anymore, so I thought why keep them in memory? Ok, you may want to cache some in case of rewind or future appearance of the same producer, but there should be user-defined limits. For rendering scenario the cache is not important IMHO. And it's not specific to my example - for every project at time T mlt only need <= k sources, where k = number of tracks * number of effects per track, which is << total number of sources. Does it make sense?

     
  • Dan Dennedy

    Dan Dennedy - 2015-02-19
    • status: pending --> wont-fix
     
  • Dan Dennedy

    Dan Dennedy - 2015-02-19

    Most MLT projects and use case require random access. For cases where only sequential processing is required, mlt_playlist has a property autoclose to close the old producers:
    http://www.mltframework.org/doxygen/properties.html#_properties000006

    You can theorize all you want about general memory optimization if you are not the one who has to actually make it all work. How do you think it will affect performance if every resource needs to be re-opened and objects re-initialized for each frame? Yes, the situation can be improved, but we do not use this bug tracker for enhancement requests or assigning tasks to volunteers.

     
  • Dan Dennedy

    Dan Dennedy - 2015-02-19

    I think my last comment may sound harsh. If you have any interest to help the project with patch, code, or documentation; please let me know, and I will help you. For example, you could improve documentation for the pango producer, or update pango and other things to use the mlt_animation API. Or, we can look at how mlt_cache can reduce the memory overhead of the pango and qtext producers.

     
  • Nikita Melnichenko

    Sure, I'd be happy to help. Currently I'm overloaded working on other projects, so I will contact you once I'm ready. Keyframable properties is a must for a video editor.

     

Log in to post a comment.