#608 heap-buffer-overflow in ImportRLEPixels of coders/miff.c

[galycannon]

There is a heap-buffer-overflow in function ImportRLEPixels of coders/miff.c whick can be reproduced as below.
gm convert ./heap-buffer-overflow_ImportRLEPixels /dev/null

=================================================================
==27431==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61500000fed1 at pc 0x00000071ab86 bp 0x7ffcc60ecb20 sp 0x7ffcc60ecb10
WRITE of size 1 at 0x61500000fed1 thread T0
    #0 0x71ab85 in ImportRLEPixels coders/miff.c:428
    #1 0x729d09 in ReadMIFFImage coders/miff.c:1850
    #2 0x477730 in ReadImage magick/constitute.c:1607
    #3 0x421598 in ConvertImageCommand magick/command.c:4365
    #4 0x436b0d in MagickCommand magick/command.c:8889
    #5 0x45f2ca in GMCommandSingle magick/command.c:17434
    #6 0x45f516 in GMCommand magick/command.c:17487
    #7 0x40cc65 in main utilities/gm.c:61
    #8 0x7f5e4f4fe82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #9 0x40cb78 in _start (/home/graphicsmagick-code/utilities/gm+0x40cb78)

0x61500000fed1 is located 0 bytes to the right of 465-byte region [0x61500000fd00,0x61500000fed1)
allocated by thread T0 here:
    #0 0x7f5e5228b602 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98602)
    #1 0x4df677 in MagickRealloc magick/memory.c:494
    #2 0x501826 in OpenCache magick/pixel_cache.c:2497
    #3 0x507f01 in ModifyCache magick/pixel_cache.c:4584
    #4 0x4fb19e in SetCacheNexus magick/pixel_cache.c:922
    #5 0x508d90 in SetCacheViewPixels magick/pixel_cache.c:4881
    #6 0x508e6a in SetImagePixels magick/pixel_cache.c:4947
    #7 0x7298da in ReadMIFFImage coders/miff.c:1831
    #8 0x477730 in ReadImage magick/constitute.c:1607
    #9 0x421598 in ConvertImageCommand magick/command.c:4365
    #10 0x436b0d in MagickCommand magick/command.c:8889
    #11 0x45f2ca in GMCommandSingle magick/command.c:17434
    #12 0x45f516 in GMCommand magick/command.c:17487
    #13 0x40cc65 in main utilities/gm.c:61
    #14 0x7f5e4f4fe82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: heap-buffer-overflow coders/miff.c:428 ImportRLEPixels
Shadow bytes around the buggy address:
  0x0c2a7fff9f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2a7fff9f90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2a7fff9fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2a7fff9fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c2a7fff9fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c2a7fff9fd0: 00 00 00 00 00 00 00 00 00 00[01]fa fa fa fa fa
  0x0c2a7fff9fe0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2a7fff9ff0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2a7fffa000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2a7fffa010: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2a7fffa020: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==27431==ABORTING

System Configuration:

Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:    16.04
Codename:   xenial

GraphicsMagick version:

GraphicsMagick 1.4 snapshot-20190423 Q8 http://www.GraphicsMagick.org/
Copyright (C) 2002-2019 GraphicsMagick Group.
Additional copyrights and licenses apply to this software.
See http://www.GraphicsMagick.org/www/Copyright.html for details.

Feature Support:
  Native Thread Safe       yes
  Large Files (> 32 bit)   yes
  Large Memory (> 32 bit)  yes
  BZIP                     yes
  DPS                      no
  FlashPix                 no
  FreeType                 yes
  Ghostscript (Library)    no
  JBIG                     yes
  JPEG-2000                yes
  JPEG                     yes
  Little CMS               yes
  Loadable Modules         no
  Solaris mtmalloc         no
  OpenMP                   yes (201307 "4.0")
  PNG                      yes
  TIFF                     yes
  TRIO                     no
  Solaris umem             no
  WebP                     yes
  WMF                      yes
  X11                      yes
  XML                      yes
  ZLIB                     yes

Host type: x86_64-pc-linux-gnu

Configured using the command:
  ./configure  'CC=gcc' 'CXX=g++' 'CFLAGS=-g -fsanitize=address -fno-omit-frame-pointer -fsanitize=leak' '--enable-shared=no'

Final Build Parameters:
  CC       = gcc
  CFLAGS   = -fopenmp -g -fsanitize=address -fno-omit-frame-pointer -fsanitize=leak -Wall -pthread
  CPPFLAGS = -I/usr/include/freetype2 -I/usr/include/libxml2
  CXX      = g++
  CXXFLAGS = -pthread
  LDFLAGS  = 
  LIBS     = -ljbig -lwebp -lwebpmux -llcms2 -ltiff -lfreetype -ljasper -ljpeg -lpng12 -lwmflite -lXext -lSM -lICE -lX11 -llzma -lbz2 -lxml2 -lz -lm -lpthread

[Bob Friesenhahn]

This problem is fixed by changeset 15991:bc99af93614d, which addresses the typo which resulted in this heap overflow as well as undefined behavior caused by large shifts. Thank you very much for this report.