#172 SegFault in videostab2

[Anonymous]

After several minutes of rendering, I get a segfault with this backtrace (mlt 0.8.0) :

#0 interpolateBiLin (channel=0 '\000', N=3 '\003', width=<optimized out>,
img=0x7fffa0c1d010 "\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\337\346\332\337\346\331\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\331\337\346\332\337\346\334\340\350\335\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334", <incomplete sequence \340>..., y=1079, x=335,
rv=0x7fffeb1e92e3 "\237\221]\243\222b\244\223e\243\221e\235\213`\225\203Y\221~U\221|S\227\202Y\235\211`\236\211`\236\206^\234\201Z\232}V\235~U\235~S\235}P\236~R\236~R\237\177S\237\177S\241\201U\240\202U\246\212\\\247\216`\245\216_\242\215^\236\212Z\232\207W\236\212Z\235\211Y\233\210W\230\204U\236\210\\\250\221g\255\227l\253\224j\255\227l\256\226l\260\227m\255\223j\250\215c\245\212^\237\205V\243\210W\235\201P\236\201O\231{H\235~J\237\201K\233}G\241\203M\240\203N\234\202M\233\202M\232\201L\230\177I\232\203K\231\202J\232\201K\232\201L\231\177L\232\177N\234\177P\237\203S\246\214Y\242\210"..., height=<optimized out>, def=<optimized out>) at transform_image.c:155
#1 interpolateBiLin (
rv=0x7fffeb1e92e3 "\237\221]\243\222b\244\223e\243\221e\235\213`\225\203Y\221~U\221|S\227\202Y\235\211`\236\211`\236\206^\234\201Z\232}V\235~U\235~S\235}P\236~R\236~R\237\177S\237\177S\241\201U\240\202U\246\212\\\247\216`\245\216_\242\215^\236\212Z\232\207W\236\212Z\235\211Y\233\210W\230\204U\236\210\\\250\221g\255\227l\253\224j\255\227l\256\226l\260\227m\255\223j\250\215c\245\212^\237\205V\243\210W\235\201P\236\201O\231{H\235~J\237\201K\233}G\241\203M\240\203N\234\202M\233\202M\232\201L\230\177I\232\203K\231\202J\232\201K\232\201L\231\177L\232\177N\234\177P\237\203S\246\214Y\242\210"..., x=335, y=1079,
img=0x7fffa0c1d010 "\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\336\346\333\337\346\332\337\346\331\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\332\337\346\331\337\346\332\337\346\334\340\350\335\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334\340\350\334", <incomplete sequence \340>..., width=<optimized out>, height=<optimized out>, def=<optimized out>, N=3 '\003', channel=0 '\000') at transform_image.c:140
#2 0x00007ffff75c87fd in transformRGB (td=0x16fdfe0) at transform_image.c:269
#3 0x00007ffff75ca008 in transform_filter_video (self=0x16fdfe0, frame=<optimized out>, pixelformat=mlt_image_rgb24) at transform_image.c:707
#4 0x00007ffff75c482c in filter_get_image (frame=0x7fffbdad4090, image=0x7ffff2ae5e90, format=0x6d63a0, width=0x7ffff2ae5e98, height=0x7fff00000438, writable=<optimized out>) at filter_videostab2.c:209
#5 0x00007ffff7da0e3a in mlt_frame_get_image (self=<optimized out>, buffer=0x7ffff2ae5e90, format=0x6d63a0, width=0x7ffff2ae5e98, height=0x7ffff2ae5e9c, writable=0) at mlt_frame.c:452
#6 0x00007ffff7db39ed in producer_get_image (self=0x7fffe79fcea0, buffer=0x7ffff2ae5e90, format=0x6d63a0, width=0x7ffff2ae5e98, height=0x7ffff2ae5e9c, writable=<optimized out>) at mlt_tractor.c:273
#7 0x00007ffff7da0e3a in mlt_frame_get_image (self=<optimized out>, buffer=0x7ffff2ae5e90, format=0x6d63a0, width=0x7ffff2ae5e98, height=0x7ffff2ae5e9c, writable=0) at mlt_frame.c:452
#8 0x00007ffff7db39ed in producer_get_image (self=0x7fffe4828f00, buffer=0x7ffff2ae5e90, format=0x6d63a0, width=0x7ffff2ae5e98, height=0x7ffff2ae5e9c, writable=<optimized out>) at mlt_tractor.c:273
#9 0x00007ffff7da0e3a in mlt_frame_get_image (self=<optimized out>, buffer=0x7ffff2ae5e90, format=0x6d63a0, width=0x7ffff2ae5e98, height=0x7ffff2ae5e9c, writable=0) at mlt_frame.c:452
#10 0x00007ffff7db0b44 in consumer_read_ahead_thread (arg=0x6d6320) at mlt_consumer.c:755
#11 0x0000003ac5e07d96 in ?? () from /lib64/libpthread.so.0
#12 0x0000003ac52eaa1d in clone () from /lib64/libc.so.6

[Anonymous]

The segault is caused by this line :

153 short v1 = PIXN(img, x_c, y_c, width, height, N, channel);

(gdb) print width
$1 = 1920
(gdb) print height
$2 = 1080
(gdb) print x_c
$3 = 336
(gdb) print y_c
$4 = 1080
(gdb) print N
$5 = 3 '\003'
(gdb) print channel
$6 = 0 '\000'
(gdb) print y
$7 = 1079
(gdb) print y_f
$8 = 1079
(gdb) print y_c
$9 = 1080

The value of y_c seems incorrect (as equal to height), doesn't it ?

[Anonymous]

I confirm that the following patch removes the SegFault but I'm unable to say if it gives the expected result :

diff --git a/src/modules/videostab/transform_image.c b/src/modules/videostab/transform_image.c
index a2e6de5..c1ea1d9 100644
--- a/src/modules/videostab/transform_image.c
+++ b/src/modules/videostab/transform_image.c
@@ -141,7 +141,7 @@ void interpolateBiLin(unsigned char *rv, float x, float y,
unsigned char* img, int width, int height,
unsigned char def,unsigned char N, unsigned char channel)
{
- if (x < 0 || x > width-1 || y < 0 || y > height - 1) {
+ if (x < 0 || x > width - 2 || y < 0 || y > height - 2) {
interpolateBiLinBorder(rv, x, y, img, width, height, def,N,channel);
} else {
int x_f = myfloor(x);

[marco]

i see the problem, x_f = round down f, x_c round up.
so y=1079 will read the pixel at 1080 (y_c).

this should alway get the wrong result (since 1080 is one line over height)

all functions that use PIXN must be ensured not to use y>=height. (same for x+.. )

i'll fix that in git after some tests.
question is, why did that not always happen, because it will always take 1 line to much of every image.

[Dan Dennedy]

fixed in git commit cd38a6 by g-marco. I will close this on the next release.

[Dan Dennedy]

Closing this because there is new release including the fix.