#59 unsquashfs segfaults with musl libc due to stack overflow

[Natanael Copa]

unsquashfs build with musl libc will segfault. The problem is:

 void *inflator(void *arg)
 {
       char tmp[block_size];

where block size is set from the squshfs image.

On musl libc the default stack size is 80Kb which is a lot smaller than glibc who allocates 8MB by default. This is why this is not visible on glibc.

I am pretty sure you can trigger it with a crafted squashfs image on glibc too. Just say that the block size is bigger than 8Mb.

The following fixes it (the free(tmp) is not really needed):

--- ./squashfs-tools/unsquashfs.c.orig
+++ ./squashfs-tools/unsquashfs.c
@@ -2099,7 +2099,9 @@
  */
 void *inflator(void *arg)
 {
-       char tmp[block_size];
+       char *tmp = malloc(block_size);
+       if(tmp == NULL)
+               EXIT_UNSQUASH("Out of memory allocating block buffer\n");

        while(1) {
                struct cache_entry *entry = queue_get(to_inflate);
@@ -2122,6 +2124,7 @@
                 */
                cache_block_ready(entry, res == -1);
        }
+       free(tmp);
 }

Credits to dalias (musl libc maintainer) who found it and suggested the fix.