libquicktime-devel — Discussion about development

Re: [Libquicktime-devel] Prores codec issues.

[Burkhard P. <pl...@ip...>]

Hi,

Am 07.11.2012 17:50, schrieb Al Crate:
> You can easily see this by running lqt_transcode on a prores mov.
>
>
>
> On 07/11/12 12:00, Al Crate wrote:
>> Hi, have been playing with the ProRes codec, seems to encode correctly,
>> but on decode the output is noticeably darker than the original.
>>
>> Has anyone else noticed this ?
>>
>> I've tried using ffmpeg directly and that works as expected. So I'm
>> suspecting libquicktimes conversion from YUV422P10 to RGB888.
>>

Well then you can fix the conversion or bypass the built in colormodel converter
completely. It's incredibly inefficient and only here for compatibility reasons.

Too dark video is often related to a confusion of video scaled and JPEG scaled
YCbCr. Not sure if that applies here.

Burkhard

Re: [Libquicktime-devel] Prores codec issues.

[Al C. <al...@dn...>]

You can easily see this by running lqt_transcode on a prores mov.



On 07/11/12 12:00, Al Crate wrote:
> Hi, have been playing with the ProRes codec, seems to encode correctly,
> but on decode the output is noticeably darker than the original.
>
> Has anyone else noticed this ?
>
> I've tried using ffmpeg directly and that works as expected. So I'm
> suspecting libquicktimes conversion from YUV422P10 to RGB888.
>
> ------------------------------------------------------------------------------
> LogMeIn Central: Instant, anywhere, Remote PC access and management.
> Stay in control, update software, and manage PCs from one command center
> Diagnose problems and improve visibility into emerging IT issues
> Automate, monitor and manage. Do more in less time with Central
> http://p.sf.net/sfu/logmein12331_d2d
> _______________________________________________
> Libquicktime-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libquicktime-devel
>

[Libquicktime-devel] Prores codec issues.

[Al C. <al...@dn...>]

Hi, have been playing with the ProRes codec, seems to encode correctly, 
but on decode the output is noticeably darker than the original.

Has anyone else noticed this ?

I've tried using ffmpeg directly and that works as expected. So I'm 
suspecting libquicktimes conversion from YUV422P10 to RGB888.

Re: [Libquicktime-devel] [PATCH] Pass AVCodec to avcodec_alloc_context3

[Burkhard P. <pl...@ip...>]

Hi,

Am 31.10.2012 13:03, schrieb Joseph Artsimovich:
> Hi,
>
> When you pass AVCodec to avcodec_alloc_context3(), ffmpeg will be using codec-specific defaults for AVCodecContext rather than global defaults. For instance, codec-specific qmax for DNxHD is 1024, while global default is 31. Encoding low-bitrate DNxHD with qmax=31 will likely fail, as the encoder
> won't be able to meet frame size limitations.
>
> The patch for ffmpeg that changes qmax default for DNxHD is my work as well, and I did try to convince them it should be hardcoded for fixed-bitrate formats, but they insisted I only make it codec-specific default. Back then I didn't know it will or won't be applied depending on what API you use and
> how exactly you use it. Well, getting a patch accepted into libquicktime is so quick and easy compared to ffmpeg, so here goes a patch.

Applied, seems to make sense.

Burkhard

[Libquicktime-devel] [PATCH] Pass AVCodec to avcodec_alloc_context3

[Joseph A. <jo...@mi...>]

Hi,

When you pass AVCodec to avcodec_alloc_context3(), ffmpeg will be using 
codec-specific defaults for AVCodecContext rather than global defaults. 
For instance, codec-specific qmax for DNxHD is 1024, while global 
default is 31. Encoding low-bitrate DNxHD with qmax=31 will likely fail, 
as the encoder won't be able to meet frame size limitations.

The patch for ffmpeg that changes qmax default for DNxHD is my work as 
well, and I did try to convince them it should be hardcoded for 
fixed-bitrate formats, but they insisted I only make it codec-specific 
default. Back then I didn't know it will or won't be applied depending 
on what API you use and how exactly you use it. Well, getting a patch 
accepted into libquicktime is so quick and easy compared to ffmpeg, so 
here goes a patch.

-- 
Joseph Artsimovich
Senior C++ Applications Developer
MirriAd Ltd

Re: [Libquicktime-devel] [PATCH] GOP size fix for XDCAM

[Burkhard P. <pl...@ip...>]

Hi,

Am 21.09.2012 11:34, schrieb Joseph Artsimovich:
> A one-liner fix for incorrect GOP size calculation.

Applied
Burkhard

[Libquicktime-devel] [PATCH] GOP size fix for XDCAM

[Joseph A. <jo...@mi...>]

A one-liner fix for incorrect GOP size calculation.

-- 
Joseph Artsimovich
Senior C++ Applications Developer
MirriAd Ltd

Re: [Libquicktime-devel] AAC audio

[Burkhard P. <pl...@ip...>]

Hi,

Am 17.09.2012 11:55, schrieb Phil Barrett:
>>> Has anyone done any work on getting AAC audio decode working in
>>> libquicktime using ffmpeg?
>>
>> No, libquicktime uses faad.

> libfaad2 is under GPL2 which is a problem for many projects.

Agree.

>
>> Not sure how the current ffmpeg AAC decoder compares with faad2.
>>
>> If the ffmpeg decoder doesn't support HE-AAC or multichannel AAC,
>> the faad2 decoder should be preferred and the ffmpeg decoder should
>> by
>> default only be used, if faad is not available.
>
> In 2009 the ffmpeg AAC decode was ">2x the speed of FAAD!" and in
> ffmpeg 0.6 it was made even faster.

That's no surprise actually :)
Never had performance issues with faad though.

> It's supported HE-AAC v2 since
> October 2010.

Excellent. And what about multichannel support?
Would be good if we could deprecate libfaad without any feature regressions.

Burkhard

Re: [Libquicktime-devel] AAC audio

[Phil B. <ph...@fi...>]

>> Has anyone done any work on getting AAC audio decode working in
>> libquicktime using ffmpeg?
>
> No, libquicktime uses faad.

libfaad2 is under GPL2 which is a problem for many projects.

> Not sure how the current ffmpeg AAC decoder compares with faad2.
>
> If the ffmpeg decoder doesn't support HE-AAC or multichannel AAC,
> the faad2 decoder should be preferred and the ffmpeg decoder should 
> by
> default only be used, if faad is not available.

In 2009 the ffmpeg AAC decode was ">2x the speed of FAAD!" and in 
ffmpeg 0.6 it was made even faster. It's supported HE-AAC v2 since 
October 2010.

Phil
-- 
Phil Barrett
FilmLight

Re: [Libquicktime-devel] AAC audio

[Burkhard P. <pl...@ip...>]

Hi,

Am 17.09.2012 11:39, schrieb Phil Barrett:
> Has anyone done any work on getting AAC audio decode working in
> libquicktime using ffmpeg?

No, libquicktime uses faad.

> It looks like we're about to try, but if anyone has any experience that
> will speed things up please do let me know!

If you want to do that, go ahead.

Not sure how the current ffmpeg AAC decoder compares with faad2.

If the ffmpeg decoder doesn't support HE-AAC or multichannel AAC,
the faad2 decoder should be preferred and the ffmpeg decoder should by
default only be used, if faad is not available.

Burkhard

[Libquicktime-devel] AAC audio

[Phil B. <ph...@fi...>]

Has anyone done any work on getting AAC audio decode working in  
libquicktime using ffmpeg?

It looks like we're about to try, but if anyone has any experience that  
will speed things up please do let me know!

Phil
-- 
Phil Barrett
FilmLight

Re: [Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Burkhard P. <pl...@ip...>]

Hi,

sorry was my mistake.

Am 14.09.2012 17:23, schrieb Joseph Artsimovich:
> On 14/09/2012 15:18, Burkhard Plaum wrote:
>> ...
>> Is it against current cvs?
>>
>> I get:
>>
>> patch -p0 < ../seek.patch
                  ^^^^^^^^^^^^ seeking.patch, not seek.patch (was an older one)

>> patching file plugins/ffmpeg/video.c
>> Hunk #1 FAILED at 1059.
>> Hunk #2 FAILED at 1073.
>> 2 out of 2 hunks FAILED -- saving rejects to file plugins/ffmpeg/video.c.rej

Applied :)

Burkhard


-- 
Dr. Burkhard Plaum, Institut für Plasmaforschung
Pfaffenwaldring 31, 70569 Stuttgart
Tel.: +49 711 / 685-62174, Fax.:  +49 711 / 685-63102

Re: [Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Joseph A. <jo...@mi...>]

On 14/09/2012 15:18, Burkhard Plaum wrote:
> ...
> Is it against current cvs?
>
> I get:
>
> patch -p0 < ../seek.patch
> patching file plugins/ffmpeg/video.c
> Hunk #1 FAILED at 1059.
> Hunk #2 FAILED at 1073.
> 2 out of 2 hunks FAILED -- saving rejects to file plugins/ffmpeg/video.c.rej
It is, I double checked. Can you post video.c.rej?

-- 
Joseph Artsimovich
Senior C++ Applications Developer
MirriAd Ltd

Re: [Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Burkhard P. <pl...@ip...>]

Hi,

Am 14.09.2012 15:04, schrieb Joseph Artsimovich:
> So, I tried to add support for reference B-frames but ended up completely rewriting the seeking code once again :)
> The new code is not as fast (reads frames that can be skipped, passes them to ffmpeg, skips IDCT though), but
 > a lot simpler. Reference B-frames are supported as well.

Perfect. Sometimes it's good not to apply patches right away :)

> The patch is attached.

Is it against current cvs?

I get:

patch -p0 < ../seek.patch
patching file plugins/ffmpeg/video.c
Hunk #1 FAILED at 1059.
Hunk #2 FAILED at 1073.
2 out of 2 hunks FAILED -- saving rejects to file plugins/ffmpeg/video.c.rej

Regarding your earlier question:

> If you are selectively skipping frames, it's quite hard to tell which
> frame ffmpeg has just decoded. You have to predict that yourself, as
> ffmpeg won't tell you.

That's right. For the classical case, where B-frames are never reference frames,
it's possible. But in general it gets practically unpredictable without
deep bitstream parsing.

What I did in gmerlin-avdecoder, was the following:

- Pass only the frames to ffmpeg, which are needed anyway. My code detects
   frame types and reference B-frames in the core so it's much simpler on the
   codec level (which is complicated enough anyway).

- For each encoded frame passed to ffmpeg, put the pts along with other metadata
   from the packet (e.g. timecode) into a buffer.

- For each decoded frame pulled from ffmpeg, it gets the lowest available pts
   from the buffer. Assuming that ffmpeg reorders the frames correctly (it does
   in all cases I observed), I get a relation between compressed packets and
   uncompressed frames. Then I can assign e.g. the timecodes from the compressed packets
   to decompressed frames.

- After seeking the PTS buffer needs to be cleared of course.

This method is quite dumb and there might be more elegant ones, but it works perfectly.

Burkhard

Re: [Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Joseph A. <jo...@mi...>]

So, I tried to add support for reference B-frames but ended up 
completely rewriting the seeking code once again :)
The new code is not as fast (reads frames that can be skipped, passes 
them to ffmpeg, skips IDCT though), but a lot simpler. Reference 
B-frames are supported as well.

The patch is attached.

-- 
Joseph Artsimovich
Senior C++ Applications Developer
MirriAd Ltd

Re: [Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Joseph A. <jo...@mi...>]

Just got back from holidays, therefore a late reply.


On 21/08/2012 16:35, Burkhard Plaum wrote:
> ...
>   > decoding delay. I never managed to reproduce it without multithreading, but I suspect
>   > it might be possible.
>
> The decoding delay can theoretically also change between versions.
> And it can change theoretically while decoding the same stream (e.g. when
> B-frames are used as references in H.264 streams).
Yes. Seeking code needs to work in all cases. The old code failed at 
that, the new should be fine.

> Basically, seeking can be done like this:
>
> - Detect the last I-frame before the seek target (in *display* order) and decode it
That's what the code effectively does. As this functionality is not in 
the core, it did take quite a few lines of code.

>
> - Skip B-frames, if less than 2 non-b-frames (in bitstream order) were seen since the seek.
>     This simple logic implicitely handles open GOPs and doesn't need a difference between
>     partial and full sync samples. In gmerlin-avdecoder I do this skipping by not even
>     passing these frames to libavcodec, and it works perfectly.
First you need to decide whether to pass those initial B-frames to 
ffmpeg. If you do, you can never be sure what ffmpeg is going to do with 
them. It can't decode them, so it might just pretend it hasn't seen 
them, or it might return a broken frame or return an error code. So, you 
don't pass them to ffmpeg. On top of that, you want to avoid decoding 
(and even reading) any B-frames that precede the target frame in display 
order. Once you start to selectively pass some frames to ffmpeg while 
skipping others, you have to do very careful accounting. That's where 
the complexity is coming from.

> - Decode frames until the target frame is decoded. Non-reference frames can be skipped here as well,
>     but they might not be trivial to detect in advance (for H.264 B-frames can also be references).
If you are selectively skipping frames, it's quite hard to tell which 
frame ffmpeg has just decoded. You have to predict that yourself, as 
ffmpeg won't tell you.
I didn't know B-frames can be reference frames. That would require one 
more change to the code: don't skip any B-frames in the same mini-GOP as 
the target frame, even if it precedes the target frame in display order. 
By mini-GOP I mean stuff between two reference frames.

>
> If there are generic routines, which convert between coding-order and display order for a video sample
> (using the ctts atom), and maybe between DTS (from stts) and PTS (from stts + ctts), I'm 100% sure
> that things could be done much simpler.
Generic routines would help a bit. Those that come to mind are "find 
previous I-frame in display order" and also conversion between display 
and storage orders.

>
> Otherwise I could apply this patch, but my ability and motivation to further maintain the ffmpeg
> frontend would drop significantly. Not sure if this is what people want :)
I think it's probably a good idea to apply it anyway, as the current 
code is broken. I'll make that change for supporting reference B-frames 
hopefully later this week.

-- 
Joseph Artsimovich
Senior C++ Applications Developer
MirriAd Ltd

Re: [Libquicktime-devel] Fast-start

[Burkhard P. <pl...@ip...>]

Hi,

Am 04.09.2012 17:23, schrieb Phil Barrett:
> Hi Burkhard
>
>> That's quicktime's fault, because it requires complete indices along
>> with the
>> global headers. And you don't know in advance how long the movie will
>> be.
>
> What if I did? Could we arrange for the headers to be put first and
> filled in as we go?

Not with the current API, but maybe it could be hacked in.
If you know how many packets will be in your movie, you could do something like:

- Write a moov atom (with tables slightly larger than required to compensate estimation
   errors)
- Write mdat atom (media packets)
- Re-write updated moov atom
- Write a placeholder atom ("free" or "skip" IIRC) after the moov atom and before
   the mdat.

The main problem is to know in advance how many packets will be generated and what tables
you will need (e.g. if there are keyframes or not, if there are B-frames or not).
Furthermore the audio might get split into packets by the codecs, i.e. one call
to *_encode_audio() will not be equivalent to one audio packet written to the file.

If you have an idea how to cleanly solve this, I'm open for discussions.

Burkhard

Re: [Libquicktime-devel] Fast-start

[Phil B. <ph...@fi...>]

Hi Burkhard

> That's quicktime's fault, because it requires complete indices along 
> with the
> global headers. And you don't know in advance how long the movie will 
> be.

What if I did? Could we arrange for the headers to be put first and 
filled in as we go?

Phil
-- 
Phil Barrett
FilmLight

Re: [Libquicktime-devel] Fast-start

[Burkhard P. <pl...@ip...>]

Hi,

Am 04.09.2012 16:04, schrieb Phil Barrett:
> Anyone got any thoughts on how to write fast-start (progressive download)
> movies with libquicktime?

It's done by quicktime_make_streamable().
But it copies the entire file to a second one.

> It's a bit messy having to post-process the movie to "fix" it - e.g. with
> https://github.com/danielgtaylor/qtfaststart

That's quicktime's fault, because it requires complete indices along with the
global headers. And you don't know in advance how long the movie will be.

Other formats are better here, e.g. AVI has global headers at the beginning
and indices at the end of the file.

Burkhard

[Libquicktime-devel] Fast-start

[Phil B. <ph...@fi...>]

Anyone got any thoughts on how to write fast-start (progressive download)  
movies with libquicktime?

It's a bit messy having to post-process the movie to "fix" it - e.g. with  
https://github.com/danielgtaylor/qtfaststart

Thanks

Phil
-- 
Phil Barrett
FilmLight

Re: [Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Burkhard P. <pl...@ip...>]

Hi,

Am 22.08.2012 10:31, schrieb Simon Burley:
>
> On Tue, August 21, 2012 4:35 pm, Burkhard Plaum wrote:
>
>> Otherwise I could apply this patch, but my ability and motivation to
>> further maintain the ffmpeg
>> frontend would drop significantly. Not sure if this is what people want :)
>
> I don't know about others, but what I'd really like is for ffmpeg to go
> away, and have the codecs that we use for real life every day work ported
> to libquicktime as plugins - Prores, H264, DNxHD.

Hmm with ffmpeg frontend I actually mean the plugin, which uses libavcodec to decode
(among others) Prores, H264, DNxHD.

Burkhard

Re: [Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Simon B. <si...@rp...>]

On Tue, August 21, 2012 4:35 pm, Burkhard Plaum wrote:

> Otherwise I could apply this patch, but my ability and motivation to
> further maintain the ffmpeg
> frontend would drop significantly. Not sure if this is what people want :)

I don't know about others, but what I'd really like is for ffmpeg to go
away, and have the codecs that we use for real life every day work ported
to libquicktime as plugins - Prores, H264, DNxHD.

Simon

-- 
Simon Burley
RPS Film Imaging Ltd
Mobile: 07702 732 655

Re: [Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Burkhard P. <pl...@ip...>]

Hi,

Am 09.08.2012 17:07, schrieb Joseph Artsimovich:
> So, my unified seeking code turned out not to be frame-accurate. The easiest way
> to reproduce that is by turning multithreaded decoding on, which results in higher
 > decoding delay. I never managed to reproduce it without multithreading, but I suspect
 > it might be possible.

The decoding delay can theoretically also change between versions.
And it can change theoretically while decoding the same stream (e.g. when
B-frames are used as references in H.264 streams).

> The new  version is largely rewritten from scratch, so don't try to understand it by looking at the diff.

Well I guess even when looking at the patched file I'd have problems to
understand it :)

I really believe that this could be made much simpler, maybe with some changes
to the core.

Basically, seeking can be done like this:

- Detect the last I-frame before the seek target (in *display* order) and decode it

- Skip B-frames, if less than 2 non-b-frames (in bitstream order) were seen since the seek.
   This simple logic implicitely handles open GOPs and doesn't need a difference between
   partial and full sync samples. In gmerlin-avdecoder I do this skipping by not even
   passing these frames to libavcodec, and it works perfectly.

- Decode frames until the target frame is decoded. Non-reference frames can be skipped here as well,
   but they might not be trivial to detect in advance (for H.264 B-frames can also be references).

If there are generic routines, which convert between coding-order and display order for a video sample
(using the ctts atom), and maybe between DTS (from stts) and PTS (from stts + ctts), I'm 100% sure
that things could be done much simpler.

Otherwise I could apply this patch, but my ability and motivation to further maintain the ffmpeg
frontend would drop significantly. Not sure if this is what people want :)

Burkhard

[Libquicktime-devel] [PATCH] Fix seeking broken by my earlier patch

[Joseph A. <jo...@mi...>]

So, my unified seeking code turned out not to be frame-accurate. The 
easiest way to reproduce that is by turning multithreaded decoding on, 
which results in higher decoding delay. I never managed to reproduce it 
without multithreading, but I suspect it might be possible.

The new  version is largely rewritten from scratch, so don't try to 
understand it by looking at the diff.
This time I tested both with and without multithreading.

-- 
Joseph Artsimovich
Senior C++ Applications Developer
MirriAd Ltd

Re: [Libquicktime-devel] [PATCH] ProRes support

[Burkhard P. <pl...@ip...>]

Hi,

Am 07.08.2012 17:34, schrieb Maksalov Boris:
> Updated ProRes patch attached.

Applied

Burkhard