Help save net neutrality! Learn more.
Close

#56 MJPEG encoding causes segmentation fault

v1.1.4pre
closed-fixed
encoding (15)
5
2007-03-05
2006-10-25
No

Using xvidcap 1.1.4rc2 under Mandriva 2007 for x86-64
(AMD64 processor), trying to encode with MJPEG codec
causes a segmentation fault. Don't really know what
the deal is or how to debug, so here's some sort of
debug output.

xtoffmpeg.c XImageToFFMPEG(): Entering
xtoffmpeg.c XImageToFFMPEG(): doing x2ffmpeg init for
targetCodec 6
xtoffmpeg.c XImageToFFMPEG(): got color info
xtoffmpeg.c XImageToFFMPEG(): alpha_mask: 0xFF000000
xtoffmpeg.c XImageToFFMPEG(): alpha_shift: 24
xtoffmpeg.c XImageToFFMPEG(): red_shift: 16
xtoffmpeg.c XImageToFFMPEG(): green_shift: 8
xtoffmpeg.c XImageToFFMPEG(): blue_shift: 0
xtoffmpeg.c XImageToFFMPEG(): image->byte_order: 0,
msb=1, lsb=0
xtoffmpeg.c XImageToFFMPEG(): found AVOutputFormat avi
it expects a number in the filename (0=no/1=yes) 0
xtoffmpeg.c XImageToFFMPEG(): found based on: target 6
- targetCodec 6 - ffmpeg codec id 8
xtoffmpeg.c add_video_stream(): Entering
xtoffmpeg.c add_video_stream(): pix_fmt_mask 32768,
has alpha 1, input_pixfmt 6, output pixfmt 15
xtoffmpeg.c add_video_stream(): bitrate = 576000
xtoffmpeg.c add_video_stream(): Leaving with 1 streams
in oc
xtoffmpeg.c prepareOutputFile(): Entering with
filename test.avi format context 0xc98d50 and number 0
xtoffmpeg.c prepareOutputFile(): Leaving with filename
file:///tmp/test.avi
xtoffmpeg.c XImageToFFMPEG(): leaving xffmpeg init
xtoffmpeg.c XImageToFFMPEG(): codec 0x92d220
xtoffmpeg.c XImageToFFMPEG(): p_inpic 0xcc9dc0
xtoffmpeg.c XImageToFFMPEG(): p_outpic 0xcc9ee0
xtoffmpeg.c XImageToFFMPEG(): outpic_buf 0xcca000
xtoffmpeg.c XImageToFFMPEG(): outbuf 0xce6210
xtoffmpeg.c XImageToFFMPEG(): output_file 0xc98d50
xtoffmpeg.c XImageToFFMPEG(): file_oformat 0x927fc0
xtoffmpeg.c XImageToFFMPEG(): out_st 0xc90130
xtoffmpeg.c XImageToFFMPEG(): image size 115200 -
input pixfmt 6 - out_size 0
xtoffmpeg.c XImageToFFMPEG(): audio_pts 1024 -
video_pts -1
xtoffmpeg.c XImageToFFMPEG(): c_info (nil) -
scratchbuf8bit 0xffffffff
xtoffmpeg.c dump32bit(): Entering with image 0xc7a430
xtoffmpeg.c dump32bit(): Leaving
xtoffmpeg.c XImageToFFMPEG(): calling encode_video
with codec 0xc99d80, outbuf 0xce6210, outbuf size
13525520, output frame 0xcc9ee0

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1082132800 (LWP 11807)]
0x00000000004c544c in dct_quantize_trellis_c
(s=0xc9a490, block=0xcc91b0, n=0, qscale=8,
overflow=0x407f570c) at mpegvideo.c:5880
5880 int score= distoration +
length[UNI_AC_ENC_INDEX(run, level)]*lambda;
(gdb) bt
#0 0x00000000004c544c in dct_quantize_trellis_c
(s=0xc9a490, block=0xcc91b0, n=0, qscale=8,
overflow=0x407f570c) at mpegvideo.c:5880
#1 0x00000000004bdd08 in encode_thread (c=0xc99d80,
arg=0xc9a490) at mpegvideo.c:4463
#2 0x000000000049b5a3 in avcodec_default_execute
(c=0xc99d80, func=0x4b94ba <encode_thread>,
arg=0xc9a570, ret=0x0, count=1) at utils.c:424
#3 0x00000000004c4b31 in encode_picture (s=0xc9a490,
picture_number=0) at mpegvideo.c:5697
#4 0x00000000004a7a0b in MPV_encode_picture
(avctx=0xc99d80, buf=0xce6210 "�", buf_size=115400,
data=0xcc9ee0) at mpegvideo.c:2505
#5 0x000000000049bd62 in avcodec_encode_video
(avctx=0xc99d80, buf=0xce6210 "�", buf_size=115400,
pict=0xcc9ee0) at utils.c:930
#6 0x00000000004379f6 in XImageToFFMPEG (fp=0xc8fef0,
image=0xc7a430, job=0xc8c880) at xtoffmpeg.c:1486
#7 0x000000000041f2c7 in TCbCaptureSHM (xtp=0xc8c880,
id=0x0) at capture.c:919
#8 0x0000000000423dd3 in do_record_thread
(job=0xc8c880) at gnome_ui.c:960
#9 0x00002b55423af360 in start_thread () from /lib64/
libpthread.so.0
#10 0x00002b554283252d in clone () from /lib64/
libc.so.6
#11 0x0000000000000000 in ?? ()

Discussion

  • Karl H. Beckers

    Karl H. Beckers - 2006-10-25
    • assigned_to: nobody --> charly4711
    • status: open --> open-accepted
     
  • Karl H. Beckers

    Karl H. Beckers - 2006-10-25

    Logged In: YES
    user_id=782084

    (accepted)
    can reproduce problem ... researching

     
  • Karl H. Beckers

    Karl H. Beckers - 2006-10-25

    Logged In: YES
    user_id=782084

    (update)
    it's definetely the trellis quant that's causing the
    segfault. mjpeg quality is very bad anyhow, though. Will try
    with recent ffmpeg (though I loathe to upgrade at this stage)

     
  • Karl H. Beckers

    Karl H. Beckers - 2006-10-25

    Logged In: YES
    user_id=782084

    (update)
    the segfault with trellis quant and mjpeg is still there
    with ffmpeg svn 6791.
    mjpeg quality is considerably better, so an upgrade "might"
    be worth the risk.
    rescaling and a/v sync need to be tested, though.
    Will check with ffmpeg team why trellis quant segfaults with
    mjpeg.

     
  • Karl H. Beckers

    Karl H. Beckers - 2006-10-25

    Logged In: YES
    user_id=782084

    (update)
    ahh ... btw. to fix the segfault when compiling from source,
    find the following line in src/xtoffmpeg.c:
    st->codec->flags |= ( CODEC_FLAG_TRELLIS_QUANT |
    CODEC_FLAG2_FAST );

    and change it to:
    st->codec->flags |= CODEC_FLAG2_FAST;

     
  • Karl H. Beckers

    Karl H. Beckers - 2006-10-30

    Logged In: YES
    user_id=782084

    (update)
    the original issue requires two fixes in libavcodec:
    1) adding trellis quantiser to mjpeg
    2) checking for the presence of trellis in a codec before
    calling the trellis quantiser

    I have quickfixed this is the version of libavcodec bundled
    with xvidcap and will revisit the issue for a better fix
    later. (Seems I cannot get away with just using libavcodec
    without understanding what's going on behind the scenes
    after all anyway.)
    If you REALLY want to build against a shared version of
    libavcodec (which I would advise against), you need to patch
    xtoffmpeg.c as described below.

    Btw., I've also bumped the libavcodec version to svn6791 to
    improve mjpeg quality.
    Will keep this open till fix is in libavcodec.

     
  • Karl H. Beckers

    Karl H. Beckers - 2006-10-30
    • status: open-accepted --> open-postponed
     
  • Karl H. Beckers

    Karl H. Beckers - 2007-03-05
    • status: open-postponed --> closed-fixed
     
  • Karl H. Beckers

    Karl H. Beckers - 2007-03-05

    Logged In: YES
    user_id=782084
    Originator: NO

    (closing, fixed)
    adding a trellis quantiser to mjpeg in ffmpeg is beyond my current skill (and time at hand to learn those things).
    A condition has been added to xvidcap to disable trellis quant with mjpeg, to make this work with future and unpatched versions of libav*

     

Log in to post a comment.