#204 Composite transition causes darkening

[TheDiveo]

For the record: I'm using MLT 0.9.1 (from sunab's kdenlive-svn ppa; 0.9.1+git20140207.4945fd34-0ubuntu0~sunab~saucy1) together with Kdenlive (0.9.7+git20140202.645eed7a-0ubuntu0~sunab~saucy1). I'm using the 64bit version on a Core i7 system.

I've experiencing what I think is a longstanding bug for many MLT versions: the composite transition causes darkening the result. It seems as if the luma calculation isn't quite okay. As a good example I've attached an image showing the same clip without and with the composite transition. To the left, you see the original clip without any transition. To the right, you see the same clip on which I've placing some titling using the Composite transition. The alpha operation is set to OVER. The transparency of the composite transition is near 0, but yet not exactly 0.

I've thoroughly checked that the titling clip has a transparency of 100% in its main region, except for the bottom titling area, where it is something of 50% or so.

When comparing the two frame, you should clearly notice that the right frame is darker than the left frame. As the titling matte has 100%, the Composite transition must cause this darkening. In fact, as soon as the transition ends, there is a clear luma increase of the resulting frames.

[Dan Dennedy]

This is fixed in git commit 2e68fff due to a regression introduced Jan. 20, 2014. I will close this ticket on the next release.

[TheDiveo]

Thank you very much! However, the regression must be much, much older. I can see the darkening issue at least throughout many of my Kdenlive projects from 2013.

[Dan Dennedy]

I fixed an obvious problem that I found. If you are running a version of MLT checked out from git on 20140207, then you definitely were seeing this problem when paused in Kdenlive because it grossly affected YUV->RGB conversion.

Now, with that said, at the command line, using the frei0r.rgbparade filter attached downstream of the transition to force it into RGB conversion (frei0r filters are RGB), and to also meter it; I do see a slight shift down at frame 50 using a white input:

melt color:white -track -blank 49 color:#00000000 -transition composite -filter frei0r.rgbparade track=0

This problem is much more subtle. It is difficult for me to determine from your picture which problem you are observing. If I modify the transition to simply return the frames from the color:white track, then I still see the very slight downward luma shift. To me that suggests a small error in the YUV-RGB conversions performed by libswscale. Indeed, even the below without transition demonstrates the problem for me:

melt color:white -attach grayscale in=50 -filter frei0r.rgbparade

grayscale is a YUV filter, and the default SDL output is YUV. Thus, with frei0r in the middle, that command is causing a YUV->RGB->YUV conversion. If you add "-debug" to that command line, you will see it logging those conversions.

How to completely avoid the problem? Do not apply any RGB filters to the video and use YUV output. You see, when you are paused, srubbing, and stepping frame-by-frame in Kdenlive, it puts the SDL video output into an RGB mode. It switches SDL into YUV mode during playback. The composite transition is YUV-based (Affine is RGB). Rendering to most video file formats is done in YUV. So, you should not see the problem in rendered output if you avoid almost all filters (Effects in Kdenlive).

[TheDiveo]

Danny, thank you very much for the time you take in tracking down my issue and explaining it to me! I greatly appreciate this.

I'm not clear whether we are talking about the same situation here. I took the two stills (which I mounted side by side in the jpeg attached) from the final rendered video footage. While I notice the changes you mention in Kdenlive's user interface, this isn't my concern.

I'm primarily working with footage shot with a GoPro HD Hero 3 black edition and I almost entirely shoot in Protune. The unavoidable thing is that I need to apply R, G, and B tonal curves ... which is no problem as this is part of my grading process, so I'm changing the luma indirectly to quite some extent. That's what Protune is about.

What is bugging me and at least another Kdenlive user is that when you have a clip and on part of it apply a composite transition, then the underlying A? frames darken during the time of transition. You can clearly see this undesirable effect in the first 20+ seconds of this video: http://www.youtube.com/watch?v=cYHCrAF0vUI

Overall, I don't understand how to avoid color space conversion in Kdenlive. As you can see from the example above, the rendered output is consistent in itself, except for the transitions. How am I supposed to avoid transitions then? I don't care abou the exact reproduction of frame pixel values, but I do care about consistency, as the human eye is really made to detect such temporal inconsistency. So how do I avoid such things from happening in MLT and Kdenlive?

[TheDiveo]

Danny, one idea that I had then thinking over and over again what you wrote is to slap on a brightness effect onto the bottommost clip and to key-frame it so it covers only the same extent as the transition. I've set it to 110 (can this value be interpreted as percent?). A quick rendering test then shows that there barely is any change visible.

I consider it to be an ugly kludge, as -- albeit to my really limited understanding -- there should be no roundtrip luma conversion/rounding error to the extend we can watch it at this time. But I still have problems understanding all the ramifications that result from what you've described above.

[Dan Dennedy]

Your example video looks like the problem I fixed. Based on your description and example, I tried to reproduce the problem using last night's build, and I did not reproduce:
http://youtu.be/VWvqUBa3rm0

I applied a RGB adjustment to the video. There were other changes related to YUV/RGB conversion in January, and it could be the combination of those changes with yesterday's change fixes it and explains why you still had problems in 2013. Or, it could be related to the version of libswscale you are using. In general, I tend to use FFmpeg not older than one year and not libav.

I want you to test using our nightly build to confirm the fix:

http://builds.meltytech.com/kdenlive/kdenlive-ubuntu12.04-x86_64-20140215.tar.bz2

You do not need to install this. Simply extract the archive, open the kdenlive folder, and run start-kdenlive.

[TheDiveo]

I've downloaded the 64bit version for Ubuntu dated from 20140217 and gave it a try...

1. Rendering now looks fine. There is a barely noticeable change that probably comes from the roundtrip rounding errors ... that's neglectable and most playback devices will probably kill it even due to their own image (pre)processing. So this is gone.

   A big thank-you for fixing this bug!

2. There is something strange with MLT sometimes not really fully stressing my Core i7, but I this doesn't seem to be consistent. So I need to watch this in the future.

[Dan Dennedy]

OK, thanks for feedback and confirmation.

Re: "+" the problem is simply due to a lack of concurrency in the code. There is some capability for parallel processing in MLT, For example, the encoding is parallel with the decoding+processing. The encoding is highly optimized esp. x264, but then the bottleneck becomes decoding+processing. There is some additional parallel processing in MLT for the processing part. However, that is usually disabled because it is not tested enough - there are processing functions that are not thread-safe and require rework or locks to make them safe but which also inhibit concurrency. People generally want stability more than performance. This parallelism is exposed in Kdenlive through Settings > Configure Kdenlive > Environment > Processing threads. Due to overheads in this non-optimal implementation, the processing threads is intentionally limited to 3. Instead, we have taken a new direction with processing in MLT to use the GPU where we also improve the quality and correctness. However, that integration into Kdenlive is only available in source form in an experimental fork. You can learn more about this in this video:
http://video.fosdem.org/2014/H1301_Cornil/Saturday/Movit_Highspeed_highquality_video_filters_on_the_GPU.webm
Even with that, there is a bottleneck to transfer image data over the bus between CPU and GPU RAM. And so, our work continues.