[Opencore-amr-devel] Out of bounds crash

[Robert S. <st...@zo...>]

Hi there,

i am running libvo-aacenc as component of a server side video messaging
service.
I am using it through the ffmpeg api interface. The service encodes a lot
of videos a day (6 digit number) and it crashes from time to time.

I am running on ubuntu 12.04 with a self compiled libvo-aacenc downloaded
source version  (0.1.3)

(gdb) info stack
#0  0x00007fa8fbdf6b7b in raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
#1  0x00000000004ae4aa in SignalHandler (p_iSig=11) at src/main.cpp:109
#2  <signal handler called>
#3  0x00007fa8f550ea62 in voAACEnc_pow2_xy (x=<optimized out>, y=28) at
aacenc/basic_op/oper_32b.c:358
#4  0x00007fa8f5511844 in allowMoreHoles (desiredPe=1079, nChannels=2,
ahParam=0x7fa8ec5e9f82, ahFlag=0x7fa8f250cc54, peData=0x7fa8f250c790,
psyOutElement=0x7fa8ec5eb128, psyOutChannel=0x7fa8ec5eb388)
    at aacenc/src/adj_thr.c:676
#5  adaptThresholdsToPe (msaParam=0x7fa8ec5e9f88, ahParam=0x7fa8ec5e9f82,
desiredPe=1028, nChannels=2, peData=0x7fa8f250c790,
logSfbEnergy=0x7fa8f250cd44, psyOutElement=0x7fa8ec5eb128,
    psyOutChannel=<optimized out>) at aacenc/src/adj_thr.c:812
#6  AdjustThresholds (adjThrState=<optimized out>,
AdjThrStateElement=0x7fa8ec5e9f7c, psyOutChannel=<optimized out>,
psyOutElement=0x7fa8ec5eb128, chBitDistribution=0x7fa8f250d210,
    logSfbEnergy=0x7fa8f250cd44, sfbNRelevantLines=0x7fa8ec5ea094,
qcOE=0x7fa8ec5eb108, elBits=0x7fa8ec5e9f4c, nChannels=2, maxBitFac=830) at
aacenc/src/adj_thr.c:1187
#7  0x00007fa8f551a66e in QCMain (hQC=0x7fa8ec5e9f3c,
elBits=0x7fa8ec5e9f4c, adjThrStateElement=0x7fa8ec5e9f7c,
psyOutChannel=0x7fa8ec5eb388, psyOutElement=0x7fa8ec5eb128,
qcOutChannel=0x7fa8ec5ea278,
    qcOutElement=0x7fa8ec5eb108, nChannels=2, ancillaryDataBytes=0) at
aacenc/src/qc_main.c:298
#8  0x00007fa8f550f5af in AacEncEncode (aacEnc=0x7fa8ec5e9f20,
timeSignal=<optimized out>, ancBytes=<optimized out>,
numAncBytes=<optimized out>, outBytes=<optimized out>,
numOutBytes=0x7fa8f250d398)
    at aacenc/src/aacenc_core.c:180
#9  0x00007fa8f550ef2f in voAACEncGetOutputData (hCodec=0x7fa8ec5e9f20,
pOutput=0x7fa8f250d390, pOutInfo=0x7fa8f250d340) at aacenc/src/aacenc.c:247
#10 0x00007fa8f9493b91 in aac_encode_frame (avctx=0x7fa8a884bc00,
avpkt=0x7fa8f250d890, frame=0x7fa8ef5fae00, got_packet_ptr=0x7fa8f250da00)
at libavcodec/libvo-aacenc.c:165
#11 0x00007fa8f95ed0b8 in avcodec_encode_audio2 (avctx=0x7fa8a884bc00,
avpkt=0x7fa8f250d890, frame=0x7fa8ef5fae00, got_packet_ptr=0x7fa8f250da00)
at libavcodec/utils.c:1532
...

It looks like, that there is a negative index used on the pow2 lookup table.
I currently thinking about the best fix and try to understand how this
function is used in the context. I could simply implement a bounds check,
for at least negative values, or i need to find out the cause of the
negative value.
Maybe you could help me with that second thing?
I will try for my self in the meantime...

Kind regards,
Robert Stoll