[xine-devel] osd SEGV patch

[Christer P. <pa...@no...>]

Hi!
I fixed a problem in the osd rle encoder that sometime causes xine to 
coredump. The code relies on that overlay->num_rle starts out with a 
zero value, but the memset() supposed to make sure it is accidentally 
used the size of a pointer to an overlay_t instead of the actual size of 
overlay_t.

Perhaps num_rle should instead be initialized explicitly instead of 
relying on memset()? That would make the code easier to understand if 
nothing else. I mean, there's even a comment in there saying that the 
memset is not necessary.

I also fixed another problem I discovered where the encoder incorrectly 
compares the number of entries in the rle array to the actual size of 
the array in the code that checks if more space needs to be added to the 
array. This will cause unnecessary realloc()'s on the array.

While on the subject on rle encoding, I understand that the overlays 
uses the rle encoding format used by DVD subtitle overlays. What amazes 
me from studying the osd rle encoder is that there seems to be no way of 
encoding a sequence of non-repeating values. For each sequence of 
non-repeating values there's a 3-byte penalty for each byte in that 
sequence. Very stupid IMHO, since it's extremely easy to add.
Any comments?

-- 
Christer Palm