h264 with aac audio

Help
Arclyde
2011-12-12
2013-04-25
  • John Fletcher

    John Fletcher - 2011-12-12

    Maybe take a look at this line in your init_audio_aac()

    aenc->audio_samples_per_output_frame = MPA_FRAME_SIZE;
    

    I think AAC uses a different block size - 1024?

     
  • John Fletcher

    John Fletcher - 2011-12-15

    I tried

    aenc->audio_samples_per_output_frame = 1024;
    

    and it seems to work.

    I will incorporate your patches.

     
  • Arclyde

    Arclyde - 2011-12-15

    Now this patch is little more completed, now there is libx264 options that will make baseline and main profiles http://londiste.etv.ee/users/arclyde/ingex-studio-h264-aac2.patch

    There is one problem, with PAL, top scanlines still are visible. Is there a way to use scale_image and crop top 4 lines?

     
  • John Fletcher

    John Fletcher - 2011-12-19

    A couple of questions:

    Which ffmpeg are you using?
    Could you explain what you are doing regarding image dimension (in the two profiles)?

     
  • Arclyde

    Arclyde - 2011-12-19

    I have compiled my own ffmpeg form your spec file. I updated ffmpeg to the latest 0.5 series. Updated spec file is http://londiste.etv.ee/users/arclyde/ffmpeg-DNxHD-h264-aac.spec to get it compiling I had to remove Patch13 and edit ffmpeg-0.5-dnxhd-avid-nitris.patch you can get it from http://londiste.etv.ee/users/arclyde/ffmpeg-0.5.6-dnxhd-avid-nitris.patch. With your ffmpeg there was problem with CODEC_ID_H264.

    We are tring to modify Ingex Studio to produce mp4 file for web archive. Right know we use Adobe Premiere for capturing and editing and Carbon Coders for making mp4 files. My ffmpeg_encode_av.cpp (http://londiste.etv.ee/users/arclyde/ffmpeg_encoder_av.cpp) has:

    case MaterialResolution::MPEG4_MP4_BP:
    ...
                enc->scale_image = 1;
                enc->input_width = 720;
                enc->input_height = 576;
                {
                    int width = 320;
                    int height = 180;
                    enc->inputBuffer = (uint8_t *)av_mallocz(width * height * 2);
                    avcodec_set_dimensions(codec_context, width, height);
                }
                enc->tmpFrame = (AVPicture *)av_mallocz(sizeof(AVPicture));
    

    and

    case MaterialResolution::MPEG4_MP4_MP:
    ...
                enc->scale_image = 1;
                enc->input_width = 720;
                enc->input_height = 576;
                {
                    int width = 640;
                    int height = 480;
                    enc->inputBuffer = (uint8_t *)av_mallocz(width * height * 2);
                    avcodec_set_dimensions(codec_context, width, height);
                }
                enc->tmpFrame = (AVPicture *)av_mallocz(sizeof(AVPicture));
    

    With MPEG4_MP4_MP there is first line like in this video http://londiste.etv.ee/users/arclyde/first-line.mp4

     
  • John Fletcher

    John Fletcher - 2011-12-19

    Yes, you're right that our published ffmpeg RPM seems to be missing x264 support.  Not sure why as it is in the the .spec file.

    ./configure --prefix=/usr --enable-pthreads --disable-demuxer=ogg --enable-swscale --enable-libx264 --enable-libmp3lame --enable-gpl --enable-libfaac --enable-libfaad
    

    I rebuilt the RPM from the unchanged .spec file and it seems fine, although I do get

    ‘struct AVCodecContext’ has no member named ‘weighted_p_pred’
    

    and have to comment out that and CODEC_FLAG2_MBTREE

    In the patch you were only changing image size for NTSC.  Still wonder why you change width in the Main Profile case.

    Looking at the clip "first_line.mp4", I think that data will have been present in the source image on the first active line, so more of a problem with the input data than the encoding.

     
  • Arclyde

    Arclyde - 2011-12-19

    Actually I also do not know exactly why, but in our web archive we store our video in this format 640x480. This is something to do with devices support. Maybe ipod and ipad were involved in early days.

    Yes in my patch there is no image size change included. New patch has size change http://londiste.etv.ee/users/arclyde/ingex-studio-h264-aac_20111219.patch

    And the data is present in source image this is widescreen signaling data http://en.wikipedia.org/wiki/Widescreen_signaling in line 23.

    And this CODEC_FLAG2_MBTREE is taken from ffmpeg-0.5.6 libx264-baseline.ffpreset file there is line flags2=-wpred-dct8x8+mbtree and this same line is in main preset file.

     
  • John Fletcher

    John Fletcher - 2011-12-19

    I updated ffmpeg-DNxHD-h264-aac-0.5-11.x86_64.rpm to enable x264 support.

    I updated CVS to add support for the H264/AAC/MP4 resolutions.  I've left the image scaling disabled as this seems to be a specialist requirement.  However, it is easily turned on via:

    const bool H264_IMAGE_SCALE = true;
    

    in ffmpeg_encoder_av.cpp

    Thanks to arclyde for contributing the code.

     
  • Arclyde

    Arclyde - 2011-12-19

    I modified write_video_frame function

        if (enc->scale_image)
        {
            // Set up parameters for scale
            enc->scale_context = sws_getCachedContext(enc->scale_context,
                        enc->input_width, enc->input_height,                   // input WxH
                        c->pix_fmt,
                        c->width, c->height, // output WxH
                        c->pix_fmt,
                        SWS_FAST_BILINEAR,
                        NULL, NULL, NULL);
            // Input image goes into tmpFrame
            avpicture_fill((AVPicture*)enc->tmpFrame, (uint8_t*)p_video,
                c->pix_fmt,
                enc->input_width, enc->input_height);
            // Skip top 4 lines.
            // See SMPTE 314M-2005
            enc->tmpFrame->data[0] += 4 * enc->tmpFrame->linesize[0];
            enc->tmpFrame->data[1] += 2 * enc->tmpFrame->linesize[1];
            enc->tmpFrame->data[2] += 2 * enc->tmpFrame->linesize[2];
            // Output of scale operation goes into inputFrame
            avpicture_fill((AVPicture*)enc->inputFrame, enc->inputBuffer,
                c->pix_fmt,
                c->width, c->height);
            // Perform the scale
            sws_scale(enc->scale_context,
                enc->tmpFrame->data, enc->tmpFrame->linesize,
                0, enc->input_height,
                enc->inputFrame->data, enc->inputFrame->linesize);
        }
    

    This removes top lines but it makes lines green and red. Like in this clip http://londiste.etv.ee/users/arclyde/last_line.mp4

     
  • John Fletcher

    John Fletcher - 2011-12-21

    Are you trying to skip 4 lines to hide the widescreen signalling data?

     
  • John Fletcher

    John Fletcher - 2011-12-21

    I rearranged the code so that where there is cropping (e.g. for 480 line NTSC), this happens before any scaling.  That should make it easier for you to add an option for cropping lines off PAL image.

     
  • Arclyde

    Arclyde - 2011-12-24

    I downloaded latest version and looks like scaling code has memory leak in it. Setting H264_SCALE_IMAGE to true and Recorder memory usage will crow till out of memory will come.

     
  • John Fletcher

    John Fletcher - 2011-12-30

    OK, will look into that.

     
  • John Fletcher

    John Fletcher - 2011-12-30

    Memory leak should be fixed now.

     
  • Arclyde

    Arclyde - 2012-02-27

    I discovered that there is something wrong with audio, at least with aac. VLC plays mp4 without problems, but flowplayer (http://flowplayer.org/) does not have audio. Also newer ffmpeg gives error when trying to do something like this ffmpeg -i 20120227_10483418_portaalid_REC_ERR_SDI1_input0_0.mp4 -vcodec copy -acodec copy -f mp4 test.mp4
    Then I get error  malformated aac bitstream, use -absf aac_adtstoasc
    I added printig on pkt.size to write_audio_frame function in file ffmpeg_encoder_av.cpp and this shows that first 3 frames have pkt-size == 0.
    And

     
  • Michael Anisimoff

    Hi *,

    I've now also tried out the h264 profiles.

    x264 always had vbv-underflows when I started a recording, resulting in some blockiness/artefacts for the first few frames.
    This can be resolved by letting x264 decide the size for it's buffer:

    -- /home/ingex/ap-workspace/ingex/studio/common/ffmpeg_setup_codec.cpp 2012-04-18 17:01:54.000000000 +0200
    +++ ffmpeg_setup_codec.cpp      2012-04-25 18:19:07.931311633 +0200
    @@ -289,7 +289,7 @@ void ffmpeg_setup_video_codec(MaterialRe
                 codec_context->bit_rate = 512000;
                 codec_context->rc_max_rate = 512000;
                 codec_context->rc_min_rate = 0;
    -            codec_context->rc_buffer_size = 512000;
    +           codec_context->rc_buffer_size = 0;
                 codec_context->coder_type = 0;
                 codec_context->flags |= CODEC_FLAG_LOOP_FILTER;
                 codec_context->me_cmp = FF_CMP_CHROMA;
    @@ -345,7 +345,7 @@ void ffmpeg_setup_video_codec(MaterialRe
                 codec_context->bit_rate = 1024000;
                 codec_context->rc_max_rate = 1024000;
                 codec_context->rc_min_rate = 0;
    -            codec_context->rc_buffer_size = 1024000;
    +           codec_context->rc_buffer_size = 0;
                 codec_context->coder_type = 1;
                 codec_context->flags |= CODEC_FLAG_LOOP_FILTER;
                 codec_context->me_cmp = FF_CMP_CHROMA;

    Further I have the same problem with the aac audio as arclyde described in Post 17.
    When I use jwplayer (a flash player similar to the above mentioned flowplayer) there is no audio playback, too.
    Playback using Quicktime doesn't work also: "error -2041: an invalid sample description was found in the movie"
    As described by arclyde newer (>0.5) FFmpeg versions result in the following error: " malformated aac bitstream, use -absf aac_adtstoasc".

    If you then do as advised in this error message and use the aac_adtstoasc bitstream (aac_adtoasc_bsf.c)  filter to order to "create an MPEG-4 AudioSpecificConfig from an MPEG-2/4 ADTS header and removes the ADTS header" the file will playback fine, since the ADTS is not needed in mp4 container and should be stripped.

    ffmpeg -i file_adts.mp4 -vcodec copy -acodec copy -absf aac_adtstoasc -f mp4 file_asc.mp4

    aac_adtstoasc function was introduced to FFmpeg in 2009:
    http://web.archiveorange.com/archive/v/yR2T4OAJgYE65QmH0wp9

    Could this be used to maybe get a valid audio for h264/acc mp4 files?

     
  • Arclyde

    Arclyde - 2012-05-09

    this flash playback problem was solved form me when I changed function init_audio_aac
    aenc->audio_outbuf_size = 960;

    Then VLC 1.x had clicking artifacts but VLC 2.x and flash was OK

     
  • John Fletcher

    John Fletcher - 2012-05-09

    I've added the patches suggested by nieselito (post 18) - thanks for that.

     
  • Michael Anisimoff

    this flash playback problem was solved form me when I changed function init_audio_aac aenc->audio_outbuf_size = 960;

    I'll give that a try. Thank you!

     
  • Michael Anisimoff

    Changing audio_buf_size didn't help, but I could fix the audio problem by simply commenting out "//codec_context->flags |= CODEC_FLAG_GLOBAL_HEADER;" in ffmpeg_setup_codec.cpp.
    Now the resulting mp4 have audio which is played back in jwplayer and also in quicktime without an error.

    -- /home/ingex/ap-workspace/ingex/studio/common/ffmpeg_setup_codec.cpp 2012-05-15 16:07:15.434373426 +0200
    +++ ffmpeg_setup_codec.cpp      2012-05-15 17:32:23.466005023 +0200
    @@ -795,7 +795,7 @@ void ffmpeg_setup_audio_codec(MaterialRe
             codec_context->codec_id = CODEC_ID_AAC;
             codec_context->profile = FF_PROFILE_AAC_LOW;
             codec_context->codec_tag = MKTAG('m', 'p', '4', 'a');
    -        //codec_context->flags |= CODEC_FLAG_GLOBAL_HEADER;
    +        codec_context->flags |= CODEC_FLAG_GLOBAL_HEADER;
             //codec_context->cutoff = 0.9;
             //codec_context->flags |= CODEC_FLAG_QSCALE;
             codec_context->sample_fmt = SAMPLE_FMT_S16;

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks