#93 invalid shift and out of bound memory access

[Dingbao Xie]

Hi,
after testing libjpeg-turbo with afl for a night, I found several undefined behaviors in jpegtrans.
To reproduce them, you need to build the source code with flag '-fsanitize=undefined', then execute cmd 'jpegtrans $file' (attached is the input files). You will see the following error information:
jdhuff.c:577:11: runtime error: left shift of negative value -1
jdhuff.c:603:15: runtime error: left shift of negative value -1
jdhuff.c:111:31: runtime error: index 8 out of bounds for type 'd_derived_tbl [4]'
jdhuff.c:113:31: runtime error: index 8 out of bounds for type 'd_derived_tbl [4]'
jdphuff.c:172:35: runtime error: index 6 out of bounds for type 'd_derived_tbl *[4]'
jdarith.c:309:53: runtime error: left shift of negative value -1
jdphuff.c:332:13: runtime error: left shift of negative value -1
jdphuff.c:339:32: runtime error: left shift of negative value -6

[Dingbao Xie]

version 1.4.1

[DRC]

Reproduced, but these issues also exist in all of the libjpeg releases. If they were specific to libjpeg-turbo, then I would be be glad to fix them, but given that they have existed in libjpeg since at least before 1998, my inclination is to say that, if they haven't caused any real (user-visible) bugs prior to now, they probably never will.

[DRC]

Fixed in https://github.com/libjpeg-turbo/libjpeg-turbo/commit/0877b38f35664b7333bdd5214df56cf5b1bdba8d