GifQuantizeBuffer: Width*Height integer overflow → heap OOB writequantize.c:62-200, function GifQuantizeBuffer(unsigned int Width, unsigned int Height, int *ColorMapSize, const GifByteType *RedInput, const GifByteType *GreenInput, const GifByteType *BlueInput, GifByteType *OutputBuffer, GifColorType *OutputColorMap).
Tested against giflib 6.1.3 (latest release, shipped 2026-04-12).
GifQuantizeBuffer is part of the public C API of giflib:
getarg.h:40 and was deliberately restored to the library at 5.0 because client applications depend on it (quantize.c:8-10 comment: "This doesn't really belong in the core library, was undocumented, and was removed in 4.2. Then it turned out some client apps were actually using it, so it was restored in 5.0.").gif2rgb.c (which uses it in the rgb2gif direction with attacker-controllable input dimensions when used as a service).egif_fuzz_common.cc calls it but with Width = 100 hardcoded — the bug condition was never explored by the existing harness.Width * Height (CWE-190 → CWE-787)/* quantize.c:90 */
for (i = 0; i < (int)(Width * Height); i++) {
Width and Height are unsigned int. The product is computed in unsigned int arithmetic and then cast to int for the loop comparison.
For Width * Height > INT_MAX (smallest square trigger: 46341 * 46341 = 2,147,488,281 > 2,147,483,647 = INT_MAX), the cast is implementation-defined per C99 §6.3.1.3 and on every mainstream platform produces a negative value (-2,147,479,015 for the smallest-square trigger). The loop body never executes; every ColorArrayEntries[i].Count stays at zero.
Downstream chain:
/* quantize.c:111-115 */
for (i = 0; i < COLOR_ARRAY_SIZE; i++) {
if (ColorArrayEntries[i].Count > 0) {
break;
}
}
/* quantize.c:117 */
QuantizedColor = &ColorArrayEntries[i];
/* quantize.c:119-127 */
NewColorSubdiv[0].QuantizedColors = QuantizedColor;
NumOfEntries = 1;
while (++i < COLOR_ARRAY_SIZE) { /* skipped, i is already COLOR_ARRAY_SIZE */
if (ColorArrayEntries[i].Count > 0) {
QuantizedColor->Pnext = &ColorArrayEntries[i];
QuantizedColor = &ColorArrayEntries[i];
NumOfEntries++;
}
}
QuantizedColor->Pnext = NULL; /* <-- heap OOB write of 8 bytes */
The find-non-empty loop exits with i == COLOR_ARRAY_SIZE == 32768, so &ColorArrayEntries[32768] is one-past-end. The C standard permits forming it but not dereferencing it; line 126 writes 8 bytes (a QuantizedColorType *) past the end of the 32768-entry, 24-byte-per-entry allocation made at quantize.c:75-76.
Secondary OOB writes/reads exist at quantize.c:151-156 in SubdivColorMap if execution continues past the first write (the same one-past-end pointer is walked through Pnext).
Built giflib 6.1.3 unmodified with -fsanitize=address, then linked the poc.c reproducer (in this directory) against libgif.a:
$ ASAN_OPTIONS=detect_leaks=0 ./poc
[poc] Width=46341 Height=46341 uint product=2147488281 int cast=-2147479015
=================================================================
==136242==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f64059ef810 at pc 0x562d06a9c033 bp 0x7fff65db6850 sp 0x7fff65db6840
WRITE of size 8 at 0x7f64059ef810 thread T0
#0 0x562d06a9c032 in GifQuantizeBuffer /tmp/giflib-audit/giflib/quantize.c:126
#1 0x562d06a9b5f5 in main /tmp/giflib-audit/poc/poc.c:45
...
0x7f64059ef810 is located 16 bytes to the right of 786432-byte region [0x7f640592f800,0x7f64059ef800)
allocated by thread T0 here:
#0 0x7f6408a5c887 in __interceptor_malloc
#1 0x562d06a9b8b1 in GifQuantizeBuffer /tmp/giflib-audit/giflib/quantize.c:75
Region size 786,432 = 32768 * sizeof(QuantizedColorType) = 32768 * 24. The 16-byte offset past the end corresponds to the Pnext field of the (one-past-end) QuantizedColorType struct on x86-64 (16 bytes of {RGB[3] + NewColorIndex + long Count} precede Pnext).
Any Width * Height whose 32-bit unsigned product exceeds INT_MAX:
| Width | Height | uint product | (int) cast | trigger? |
|---|---|---|---|---|
| 46340 | 46340 | 0x7FFF7C30 | 2147119152 | no (just under INT_MAX) |
| 46341 | 46341 | 0x80001259 | -2147479015 | yes (smallest square) |
| 50000 | 50000 | 0x95028200 | -1795524096 | yes |
| 65535 | 65535 | 0xFFFE0001 | -131071 | yes (max 16-bit GIF dims) |
| 65536 | 65536 | 0x00000000 | 0 | yes (wraps to zero) |
| 0 | any | 0 | 0 | yes (zero-dimension trips same path) |
The function does not validate Width or Height at entry.
GifQuantizeBuffer is reachable from any program that converts RGB pixel data to a paletted/indexed format via the giflib API and accepts attacker-controllable dimensions. The bug condition does not require valid pixel data — Red/Green/BlueInput are never read once the sampling loop is skipped.
Examples of exposed surfaces (illustrative, not enumerated audits):
gif2rgb -r (rgb2gif) path with -s W H on attacker-supplied input.See patch.diff. The change adds 10 lines (1 include, 9 statement lines) to quantize.c:
#include <limits.h> for INT_MAX.Width == 0, Height == 0, and any (Width, Height) whose product would exceed INT_MAX in unsigned int arithmetic.GIF_ERROR (giflib's existing error convention; matches the OOM path at line 78).The check uses Width > (unsigned int)INT_MAX / Height — the standard overflow-safe form. No new system dependencies, no behavior change for any valid Width * Height <= INT_MAX, no abort().
added the patch.diff attachment couldn't more than one attachment at a time
I have successfully created a patch and submitted a Merge Request for this issue here:
https://sourceforge.net/p/giflib/code/merge-requests/21/
The patch adds an input validation check at the top of GifQuantizeBuffer() that rejects zero dimensions and dimension pairs whose product would overflow INT_MAX, preventing the heap OOB write.