[Squashfs-devel] Deadlock using mksquashfs 3.3

[Scott A C. <sc...@cs...>]

There appears to be a deadlock in this version of mksquashfs,
depending on the block size. The point at which it deadlocks is
deterministic and does not change from run to run or when changing the
-processors. It only changes on the block size.

I'm building a 220mb filesystem with 44k files (image size ~80mb).


For instance:

# /mksquashfs * ../z_16k  -b 16k -processors 1
Parallel mksquashfs: Using 1 processor
Creating little endian 3.1 filesystem on ../z_16k, block size 16384.
[====================================                        ] 28737/47276  60%

# mksquashfs * ../z_64k -b 64k -processors 1 
Parallel mksquashfs: Using 1 processor
Creating little endian 3.1 filesystem on ../z_64k, block size 65536.
[===============================================             ] 34865/44159  78%

It works for 256k:

# mksquashfs * ../z_256k -b 256k -processors 1 
Parallel mksquashfs: Using 1 processor
Creating little endian 3.1 filesystem on ../z_256k, block size 262144.
[============================================================] 44022/44022 100%


Hope this helps. I can hand you the ~70mb tar file containing the
directory tree if you wish.


Scott.




***** Backtrace of all threads when deadlocked *****

/////////////////////////


(gdb) info threads
  5 Thread 0x40800950 (LWP 10179)  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  4 Thread 0x41001950 (LWP 10180)  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  3 Thread 0x41802950 (LWP 10181)  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  2 Thread 0x42003950 (LWP 10182)  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
  1 Thread 0x2b48eb49c6e0 (LWP 10178)  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0




#0  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#1  0x0000000000405a5f in get_fragment (buffer=0x696b20 "PSID",
    fragment=0x5c8b9b0, cached_fragment=0x61694c) at mksquashfs.c:1244
#2  0x0000000000405ef3 in duplicate (file_size=1336, bytes=0,
    block_list=0x7fffc02330a8, start=0x7fffc02330a0,
#fragment=0x7fffc02330b0, 
    file_buffer=0x2aaaac030aa0, buffer_list=0x0, blocks=0, checksum=0,
    fragment_checksum=2353, checksum_flag=1) at mksquashfs.c:1616
#3  0x0000000000409308 in write_file_frag_dup (inode=0x7fffc023a790,
    dir_ent=0x1b95980, size=<value optimized out>,
    duplicate_file=0x7fffc02331ac, file_buffer=0x2aaaac030aa0,
    checksum=<value optimized out>) at mksquashfs.c:1964


//////

(gdb) thread 2
[Switching to thread 2 (Thread 0x42003950 (LWP 10182))]#0
0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib/libpthread.so.0
(gdb) bt
#0  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#1  0x0000000000402f8b in queue_get (queue=0xc22410) at mksquashfs.c:482
#2  0x0000000000406312 in frag_deflator (arg=<value optimized out>)
    at mksquashfs.c:1855

//////
[Switching to thread 3 (Thread 0x41802950 (LWP 10181))]#0
0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib/libpthread.so.0
(gdb) bt
#0  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#1  0x0000000000402f8b in queue_get (queue=0xc1a110) at mksquashfs.c:482
#2  0x0000000000406491 in deflator (arg=<value optimized out>)
    at mksquashfs.c:1826

/////

[Switching to thread 4 (Thread 0x41001950 (LWP 10180))]#0   
0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib/libpthread.so.0
(gdb) bt
#0  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 ()
   from /lib/libpthread.so.0
#1  0x000000000040309f in queue_put (queue=0xc1e280, data=0x0)
    at mksquashfs.c:467 
#2  0x00000000004031b9 in writer (arg=<value optimized out>)
    at mksquashfs.c:1774

/////

[Switching to thread 5 (Thread 0x40800950 (LWP 10179))]#0
0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib/libpthread.so.0
(gdb) bt
#0  0x00002b48eacabb99 in pthread_cond_wait@@GLIBC_2.3.2 ()   
   from /lib/libpthread.so.0
#1  0x000000000040320b in alloc_get (allocator=0xc224e0) at mksquashfs.c:388
#2  0x0000000000403342 in reader_read_file (dir_ent=0x1bf6100)
    at mksquashfs.c:1663
#3  0x0000000000403501 in reader_scan (dir=0x1bf0220) at mksquashfs.c:1730
#4  0x000000000040351d in reader_scan (dir=0x188a3a0) at mksquashfs.c:1733  
#5  0x000000000040351d in reader_scan (dir=0xc22ab0) at mksquashfs.c:1733