Menu

#24 filebench hanging in sempost/semblock flowops

pending
None
7
2014-07-31
2010-07-21
David Lutz
No

A filebench workload based on oltp.f was seen to deadlock on some runs, with most shadow threads in flowoplib_sempost(), most dbwr threads and the lgwr thread in flowoplib_semblock(), and a few of the shadow and dbwr threads in fileset_pick(). All of the dbwr and lgwr aio threads were in _aio_idle(). The hang was observed repeatedly, on roughly 1 out of every 50 runs. When it did occur, it always happened within a few seconds of the "Running..." message and the start of I/O operations.

Examining all of the processes with dbx during a hang, I see that they all use the same sys_semid, but the sbuf[0].sem_num and sbuf[1].sem_num used by the shadow threads that are in flowoplib_sempost() do not match any of the sem_nums used the dbwr or lgwr threads that are in flowoplib_semblock(). This suggest that we may either get into a state where the shadow threads and dbwr threads disagree on the correct semaphores to use, or that the shadows are posting to semaphores associated dbwr threads stuck in fileset_pick(), and vice versa.

filebench-1.4.8 was compiled on Solaris 10 with SunStudio 12, and executed on a 128 strand T5240.

The attached myoltp.f is the workload definition used, including 64 shadow, 20 dbwr, and one lgwr process. go_filebench was run with the following input:

load my-oltp
set $dir=/mydir1/filebench
set $logdir=/mydir2/filebench
set $filesize=17179869184
set $logfilesize=17179869184
set $iosize=4096
set $dbwriosize=8192
set $logiosize=12288
run 6000

Discussion

  • David Lutz

    David Lutz - 2010-07-21

    workload definition

     
  • David Lutz

    David Lutz - 2010-07-21

    The process list of a hung run:

    28790 /opt/filebench/bin/go_filebench
    28801 shadow -a shadow -i 1 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28802 shadow -a shadow -i 2 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28803 shadow -a shadow -i 3 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28804 shadow -a shadow -i 4 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28805 shadow -a shadow -i 5 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28806 shadow -a shadow -i 6 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28807 shadow -a shadow -i 7 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28808 shadow -a shadow -i 8 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28809 shadow -a shadow -i 9 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28810 shadow -a shadow -i 10 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28811 shadow -a shadow -i 11 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28812 shadow -a shadow -i 12 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28813 shadow -a shadow -i 13 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28814 shadow -a shadow -i 14 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28815 shadow -a shadow -i 15 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28816 shadow -a shadow -i 16 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28817 shadow -a shadow -i 17 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28818 shadow -a shadow -i 18 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28819 shadow -a shadow -i 19 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28820 shadow -a shadow -i 20 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28821 shadow -a shadow -i 21 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28822 shadow -a shadow -i 22 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28823 shadow -a shadow -i 23 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28824 shadow -a shadow -i 24 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28825 shadow -a shadow -i 25 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28826 shadow -a shadow -i 26 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28827 shadow -a shadow -i 27 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28828 shadow -a shadow -i 28 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28829 shadow -a shadow -i 29 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28830 shadow -a shadow -i 30 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28831 shadow -a shadow -i 31 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28832 shadow -a shadow -i 32 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28833 shadow -a shadow -i 33 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28834 shadow -a shadow -i 34 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28835 shadow -a shadow -i 35 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28836 shadow -a shadow -i 36 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28837 shadow -a shadow -i 37 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28838 shadow -a shadow -i 38 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28839 shadow -a shadow -i 39 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28840 shadow -a shadow -i 40 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28841 shadow -a shadow -i 41 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28842 shadow -a shadow -i 42 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28843 shadow -a shadow -i 43 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28844 shadow -a shadow -i 44 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28845 shadow -a shadow -i 45 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28846 shadow -a shadow -i 46 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28847 shadow -a shadow -i 47 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28848 shadow -a shadow -i 48 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28849 shadow -a shadow -i 49 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28850 shadow -a shadow -i 50 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28851 shadow -a shadow -i 51 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28852 shadow -a shadow -i 52 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28853 shadow -a shadow -i 53 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28854 shadow -a shadow -i 54 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28855 shadow -a shadow -i 55 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28856 shadow -a shadow -i 56 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28857 shadow -a shadow -i 57 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28858 shadow -a shadow -i 58 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28859 shadow -a shadow -i 59 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28860 shadow -a shadow -i 60 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28861 shadow -a shadow -i 61 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28862 shadow -a shadow -i 62 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28863 shadow -a shadow -i 63 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28864 shadow -a shadow -i 64 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28865 dbwr -a dbwr -i 1 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28866 dbwr -a dbwr -i 2 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28867 dbwr -a dbwr -i 3 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28868 dbwr -a dbwr -i 4 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28869 dbwr -a dbwr -i 5 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28870 dbwr -a dbwr -i 6 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28871 dbwr -a dbwr -i 7 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28872 dbwr -a dbwr -i 8 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28873 dbwr -a dbwr -i 9 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28874 dbwr -a dbwr -i 10 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28875 dbwr -a dbwr -i 11 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28876 dbwr -a dbwr -i 12 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28877 dbwr -a dbwr -i 13 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28878 dbwr -a dbwr -i 14 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28879 dbwr -a dbwr -i 15 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28880 dbwr -a dbwr -i 16 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28881 dbwr -a dbwr -i 17 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28882 dbwr -a dbwr -i 18 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28883 dbwr -a dbwr -i 19 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28884 dbwr -a dbwr -i 20 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4
    28885 lgwr -a lgwr -i 1 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    The stacks from the above processes:
    28843: shadow -a shadow -i 43 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdeac)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf68, 1, 0, ffffffff7fffdeac) + 38

    0000000100031fac threadflow_init (ffffffff392b41e0, 10003ee40, 10000, 2b, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed50, 2b, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeea8, ffffffff7fffeef8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39425e40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28809: shadow -a shadow -i 9 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdecc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf88, 1, 0, ffffffff7fffdecc) + 38

    0000000100031fac threadflow_init (ffffffff392b1980, 10003ee40, 10000, 9, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed70, 9, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeec8, ffffffff7fffef18, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393e9240, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28830: shadow -a shadow -i 30 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf7c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe038, 1, 0, ffffffff7fffdf7c) + 38

    0000000100031fac threadflow_init (ffffffff392b3270, 10003ee40, 10000, 1e, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee20, 1e, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef78, ffffffff7fffefc8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39402740, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28881: dbwr -a dbwr -i 17 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde4c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf08, 1, 0, ffffffff7fffde4c) + 38

    0000000100031fac threadflow_init (ffffffff392b6f00, 10003ee40, 10000, 11, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecf0, 11, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee48, ffffffff7fffee98, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39409340, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a280, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a360, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 100159fe0, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 100159f00, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28854: shadow -a shadow -i 54 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffddfc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdeb8, 1, 0, ffffffff7fffddfc) + 38

    0000000100031fac threadflow_init (ffffffff392b4ef0, 10003ee40, 10000, 36, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffeca0, 36, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffedf8, ffffffff7fffee48, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393ff140, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28844: shadow -a shadow -i 44 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde9c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf58, 1, 0, ffffffff7fffde9c) + 38

    0000000100031fac threadflow_init (ffffffff392b4310, 10003ee40, 10000, 2c, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed40, 2c, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee98, ffffffff7fffeee8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3941c9c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28825: shadow -a shadow -i 25 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffddcc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffde88, 1, 0, ffffffff7fffddcc) + 38

    0000000100031fac threadflow_init (ffffffff392b2c80, 10003ee40, 10000, 19, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffec70, 19, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffedc8, ffffffff7fffee18, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39404fc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28849: shadow -a shadow -i 49 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde4c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf08, 1, 0, ffffffff7fffde4c) + 38

    0000000100031fac threadflow_init (ffffffff392b4900, 10003ee40, 10000, 31, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecf0, 31, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee48, ffffffff7fffee98, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393fd640, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28848: shadow -a shadow -i 48 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde5c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf18, 1, 0, ffffffff7fffde5c) + 38

    0000000100031fac threadflow_init (ffffffff392b47d0, 10003ee40, 10000, 30, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed00, 30, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee58, ffffffff7fffeea8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3941bc40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28814: shadow -a shadow -i 14 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde7c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf38, 1, 0, ffffffff7fffde7c) + 38

    0000000100031fac threadflow_init (ffffffff392b1f70, 10003ee40, 10000, e, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed20, e, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee78, ffffffff7fffeec8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7dffbdb4, 2, ffffffff7dffbd98)

    0000000100012394 flowop_start (ffffffff393ef0c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28868: dbwr -a dbwr -i 4 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf1c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfd8, 1, 0, ffffffff7fffdf1c) + 38

    0000000100031fac threadflow_init (ffffffff392b5f90, 10003ee40, 10000, 4, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedc0, 4, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef18, ffffffff7fffef68, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39420d40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a360, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a280, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a1a0, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a0c0, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28821: shadow -a shadow -i 21 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde0c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdec8, 1, 0, ffffffff7fffde0c) + 38

    0000000100031fac threadflow_init (ffffffff392b27c0, 10003ee40, 10000, 15, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecb0, 15, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee08, ffffffff7fffee58, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f4f40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28845: shadow -a shadow -i 45 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde8c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf48, 1, 0, ffffffff7fffde8c) + 38

    0000000100031fac threadflow_init (ffffffff392b4440, 10003ee40, 10000, 2d, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed30, 2d, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee88, ffffffff7fffeed8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393fe3c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28832: shadow -a shadow -i 32 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf5c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe018, 1, 0, ffffffff7fffdf5c) + 38

    0000000100031fac threadflow_init (ffffffff392b34d0, 10003ee40, 10000, 20, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee00, 20, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef58, ffffffff7fffefa8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f92c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28819: shadow -a shadow -i 19 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde2c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdee8, 1, 0, ffffffff7fffde2c) + 38

    0000000100031fac threadflow_init (ffffffff392b2560, 10003ee40, 10000, 13, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecd0, 13, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee28, ffffffff7fffee78, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f3440, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28827: shadow -a shadow -i 27 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdfac)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe068, 1, 0, ffffffff7fffdfac) + 38

    0000000100031fac threadflow_init (ffffffff392b2ee0, 10003ee40, 10000, 1b, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee50, 1b, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffefa8, ffffffff7fffeff8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39415040, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28811: shadow -a shadow -i 11 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdeac)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf68, 1, 0, ffffffff7fffdeac) + 38

    0000000100031fac threadflow_init (ffffffff392b1be0, 10003ee40, 10000, b, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed50, b, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeea8, ffffffff7fffeef8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393e9fc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28861: shadow -a shadow -i 61 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf8c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe048, 1, 0, ffffffff7fffdf8c) + 38

    0000000100031fac threadflow_init (ffffffff392b5740, 10003ee40, 10000, 3d, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee30, 3d, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef88, ffffffff7fffefd8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3941f240, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28858: shadow -a shadow -i 58 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdfbc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe078, 1, 0, ffffffff7fffdfbc) + 38

    0000000100031fac threadflow_init (ffffffff392b53b0, 10003ee40, 10000, 3a, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee60, 3a, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffefb8, ffffffff7ffff008, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3941ffc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28812: shadow -a shadow -i 12 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde9c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf58, 1, 0, ffffffff7fffde9c) + 38

    0000000100031fac threadflow_init (ffffffff392b1d10, 10003ee40, 10000, c, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed40, c, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee98, ffffffff7fffeee8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393ee340, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28853: shadow -a shadow -i 53 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde0c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdec8, 1, 0, ffffffff7fffde0c) + 38

    0000000100031fac threadflow_init (ffffffff392b4dc0, 10003ee40, 10000, 35, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecb0, 35, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee08, ffffffff7fffee58, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393fc8c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28828: shadow -a shadow -i 28 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf9c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe058, 1, 0, ffffffff7fffdf9c) + 38

    0000000100031fac threadflow_init (ffffffff392b3010, 10003ee40, 10000, 1c, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee40, 1c, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef98, ffffffff7fffefe8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff394085c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28841: shadow -a shadow -i 41 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdecc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf88, 1, 0, ffffffff7fffdecc) + 38

    0000000100031fac threadflow_init (ffffffff392b3f80, 10003ee40, 10000, 29, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed70, 29, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeec8, ffffffff7fffef18, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3941d740, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28880: dbwr -a dbwr -i 16 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde5c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf18, 1, 0, ffffffff7fffde5c) + 38

    0000000100031fac threadflow_init (ffffffff392b6dd0, 10003ee40, 10000, 10, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed00, 10, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee58, ffffffff7fffeea8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db01c lwp_cond_wait (ffffffff22010ec8, ffffffff22010f08, 0, 1)

    ffffffff7e7d08dc cond_wait_kernel (ffffffff22010ec8, ffffffff22010f08, 0, 0, 1, 1) + 4c

    ffffffff7e7d0bdc cond_wait_common (ffffffff22010ec8, ffffffff22010f08, 0, ffffffff22010f0c, 16b6d0, 0) + 2bc

    ffffffff7e7d0cf0 cond_wait (ffffffff22010ec8, ffffffff22010f08, 0, 10000dce8, 0, ffffffff7f700a00) + 10

    ffffffff7e7d0d2c pthread_cond_wait (ffffffff22010ec8, ffffffff22010f08, ff000000, 0, 0, 0) + 8

    000000010000dce8 fileset_pick (ffffffff22010df0, 10, 1f, 0, 0, fffc00) + 70

    0000000100013e58 flowoplib_pickfile (ffffffff7e0fbad0, ffffffff6f401090, 10, 1f, 0, 0) + 138

    0000000100016cbc flowoplib_openfile_common (ffffffff393fadc0, ffffffff6f401090, 7c, 1001474bc, ffffffff7f700a00, 0) + 184

    00000001000143ec flowoplib_filesetup (ffffffff393fadc0, ffffffff6f401090, ffffffff7e0fbdb8, ffffffff7e0fbda8, 0, 1) + 15c

    00000001000147b0 flowoplib_iosetup (ffffffff393fadc0, ffffffff6f401090, ffffffff7e0fbdb8, ffffffff7e0fbdc0, ffffffff7e0fbda8, 2000) + 38

    000000010000a39c fb_lfsflow_aiowrite (ffffffff393fadc0, ffffffff6f401090, ffffffff393fadc0, ffffffff6f401090, 1, 10002a00000) + 4c

    0000000100012394 flowop_start (ffffffff393fadc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (13, 2, 100159b80, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (14, 2, 100159c60, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (15, 2, 100159d40, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, ffffffff7f705a00, 0) + 28

    ffffffff7e503284 _aio_do_request (100159a30, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28813: shadow -a shadow -i 13 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde8c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf48, 1, 0, ffffffff7fffde8c) + 38

    0000000100031fac threadflow_init (ffffffff392b1e40, 10003ee40, 10000, d, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed30, d, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee88, ffffffff7fffeed8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393e7740, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28874: dbwr -a dbwr -i 10 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdebc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf78, 1, 0, ffffffff7fffdebc) + 38

    0000000100031fac threadflow_init (ffffffff392b66b0, 10003ee40, 10000, a, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed60, a, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeeb8, ffffffff7fffef08, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff393fbb40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 100159fe0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a360, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 100159f00, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a280, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28842: shadow -a shadow -i 42 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdebc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf78, 1, 0, ffffffff7fffdebc) + 38

    0000000100031fac threadflow_init (ffffffff392b40b0, 10003ee40, 10000, 2a, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed60, 2a, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeeb8, ffffffff7fffef08, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39426bc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28863: shadow -a shadow -i 63 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf6c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe028, 1, 0, ffffffff7fffdf6c) + 38

    0000000100031fac threadflow_init (ffffffff392b59a0, 10003ee40, 10000, 3f, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee10, 3f, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef68, ffffffff7fffefb8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff394250c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28883: dbwr -a dbwr -i 19 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde2c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdee8, 1, 0, ffffffff7fffde2c) + 38

    0000000100031fac threadflow_init (ffffffff392b7160, 10003ee40, 10000, 13, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecd0, 13, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee28, ffffffff7fffee78, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39404240, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a1a0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a360, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a280, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 100159fe0, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28865: dbwr -a dbwr -i 1 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf4c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe008, 1, 0, ffffffff7fffdf4c) + 38

    0000000100031fac threadflow_init (ffffffff392b5c00, 10003ee40, 10000, 1, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedf0, 1, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef48, ffffffff7fffef98, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39421ac0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 100159f00, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a360, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a280, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 100159fe0, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28847: shadow -a shadow -i 47 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde6c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf28, 1, 0, ffffffff7fffde6c) + 38

    0000000100031fac threadflow_init (ffffffff392b46a0, 10003ee40, 10000, 2f, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed10, 2f, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee68, ffffffff7fffeeb8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3940bbc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28878: dbwr -a dbwr -i 14 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde7c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf38, 1, 0, ffffffff7fffde7c) + 38

    0000000100031fac threadflow_init (ffffffff392b6b70, 10003ee40, 10000, e, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed20, e, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee78, ffffffff7fffeec8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db01c lwp_cond_wait (ffffffff22010ec8, ffffffff22010f08, 0, 1)

    ffffffff7e7d08dc cond_wait_kernel (ffffffff22010ec8, ffffffff22010f08, 0, 0, 1, 1) + 4c

    ffffffff7e7d0bdc cond_wait_common (ffffffff22010ec8, ffffffff22010f08, 0, ffffffff22010f0c, 16b6d0, 0) + 2bc

    ffffffff7e7d0cf0 cond_wait (ffffffff22010ec8, ffffffff22010f08, 1, 10000dce8, ffffffff7e948240, ffffffff7f700a00) + 10

    ffffffff7e7d0d2c pthread_cond_wait (ffffffff22010ec8, ffffffff22010f08, ff000000, 0, 0, 0) + 8

    000000010000dce8 fileset_pick (ffffffff22010df0, 10, 1c, 0, 0, fffc00) + 70

    0000000100013e58 flowoplib_pickfile (ffffffff7e0fbad0, ffffffff6f3fe7e0, 10, 1c, 0, 0) + 138

    0000000100016cbc flowoplib_openfile_common (ffffffff393f8540, ffffffff6f3fe7e0, 7b, 1001474bc, ffffffff7f700a00, 0) + 184

    00000001000143ec flowoplib_filesetup (ffffffff393f8540, ffffffff6f3fe7e0, ffffffff7e0fbdb8, ffffffff7e0fbda8, 0, 1) + 15c

    00000001000147b0 flowoplib_iosetup (ffffffff393f8540, ffffffff6f3fe7e0, ffffffff7e0fbdb8, ffffffff7e0fbdc0, ffffffff7e0fbda8, 2000) + 38

    000000010000a39c fb_lfsflow_aiowrite (ffffffff393f8540, ffffffff6f3fe7e0, ffffffff393f8540, ffffffff6f3fe7e0, 1, 10003b00000) + 4c

    0000000100012394 flowop_start (ffffffff393f8540, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (13, 2, 100159b80, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (14, 2, 100159c60, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (15, 2, 100159d40, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (16, 2, 100159e20, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28803: shadow -a shadow -i 3 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf2c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfe8, 1, 0, ffffffff7fffdf2c) + 38

    0000000100031fac threadflow_init (ffffffff392b1260, 10003ee40, 10000, 3, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedd0, 3, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef28, ffffffff7fffef78, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393e5c40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28856: shadow -a shadow -i 56 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdddc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffde98, 1, 0, ffffffff7fffdddc) + 38

    0000000100031fac threadflow_init (ffffffff392b5150, 10003ee40, 10000, 38, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffec80, 38, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffedd8, ffffffff7fffee28, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff394235c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28867: dbwr -a dbwr -i 3 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf2c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfe8, 1, 0, ffffffff7fffdf2c) + 38

    0000000100031fac threadflow_init (ffffffff392b5e60, 10003ee40, 10000, 3, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedd0, 3, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef28, ffffffff7fffef78, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff393f77c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a360, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a280, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 100159fe0, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a0c0, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28873: dbwr -a dbwr -i 9 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdecc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf88, 1, 0, ffffffff7fffdecc) + 38

    0000000100031fac threadflow_init (ffffffff392b6580, 10003ee40, 10000, 9, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed70, 9, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeec8, ffffffff7fffef18, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff394178c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a1a0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 100159fe0, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a360, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a280, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28817: shadow -a shadow -i 17 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde4c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf08, 1, 0, ffffffff7fffde4c) + 38

    0000000100031fac threadflow_init (ffffffff392b2300, 10003ee40, 10000, 11, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecf0, 11, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee48, ffffffff7fffee98, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f1940, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28835: shadow -a shadow -i 35 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf2c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfe8, 1, 0, ffffffff7fffdf2c) + 38

    0000000100031fac threadflow_init (ffffffff392b3860, 10003ee40, 10000, 23, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedd0, 23, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef28, ffffffff7fffef78, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39427940, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28869: dbwr -a dbwr -i 5 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf0c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfc8, 1, 0, ffffffff7fffdf0c) + 38

    0000000100031fac threadflow_init (ffffffff392b60c0, 10003ee40, 10000, 5, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedb0, 5, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef08, ffffffff7fffef58, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff3940ae40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 100159fe0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a280, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a360, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a1a0, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28816: shadow -a shadow -i 16 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde5c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf18, 1, 0, ffffffff7fffde5c) + 38

    0000000100031fac threadflow_init (ffffffff392b21d0, 10003ee40, 10000, 10, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed00, 10, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee58, ffffffff7fffeea8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f0bc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28831: shadow -a shadow -i 31 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf6c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe028, 1, 0, ffffffff7fffdf6c) + 38

    0000000100031fac threadflow_init (ffffffff392b33a0, 10003ee40, 10000, 1f, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee10, 1f, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef68, ffffffff7fffefb8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3941e4c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28850: shadow -a shadow -i 50 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde3c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdef8, 1, 0, ffffffff7fffde3c) + 38

    0000000100031fac threadflow_init (ffffffff392b4a30, 10003ee40, 10000, 32, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffece0, 32, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee38, ffffffff7fffee88, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393ffec0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28840: shadow -a shadow -i 40 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdedc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf98, 1, 0, ffffffff7fffdedc) + 38

    0000000100031fac threadflow_init (ffffffff392b3e50, 10003ee40, 10000, 28, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed80, 28, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeed8, ffffffff7fffef28, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39400c40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28864: shadow -a shadow -i 64 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf5c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe018, 1, 0, ffffffff7fffdf5c) + 38

    0000000100031fac threadflow_init (ffffffff392b5ad0, 10003ee40, 10000, 40, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee00, 40, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef58, ffffffff7fffefa8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39405d40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28826: shadow -a shadow -i 26 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdfbc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe078, 1, 0, ffffffff7fffdfbc) + 38

    0000000100031fac threadflow_init (ffffffff392b2db0, 10003ee40, 10000, 1a, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee60, 1a, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffefb8, ffffffff7ffff008, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393fa040, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28823: shadow -a shadow -i 23 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffddfc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdeb8, 1, 0, ffffffff7fffddfc) + 38

    0000000100031fac threadflow_init (ffffffff392b2a20, 10003ee40, 10000, 17, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffeca0, 17, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffedf8, ffffffff7fffee48, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f5cc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28818: shadow -a shadow -i 18 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde3c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdef8, 1, 0, ffffffff7fffde3c) + 38

    0000000100031fac threadflow_init (ffffffff392b2430, 10003ee40, 10000, 12, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffece0, 12, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee38, ffffffff7fffee88, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f26c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28870: dbwr -a dbwr -i 6 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdefc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfb8, 1, 0, ffffffff7fffdefc) + 38

    0000000100031fac threadflow_init (ffffffff392b61f0, 10003ee40, 10000, 6, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffeda0, 6, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeef8, ffffffff7fffef48, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff394193c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a1a0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a280, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 100159f00, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a360, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28875: dbwr -a dbwr -i 11 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdeac)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf68, 1, 0, ffffffff7fffdeac) + 38

    0000000100031fac threadflow_init (ffffffff392b67e0, 10003ee40, 10000, b, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed50, b, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeea8, ffffffff7fffeef8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39424340, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 100159fe0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a360, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 100159f00, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a280, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28790: /opt/filebench/bin/go_filebench

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7da12c nanosleep (ffffffff7fffd520, ffffffff7fffd510)

    0000000100025dec parser_pause (1770, 10003d460, 0, fffffffffffffff8, 0, 100167458) + 54

    0000000100025f3c parser_run (1001673f0, 0, 1001470d8, 18164c, ffffffff7e7bad70, ffffffff7e93c000) + 7c

    000000010001e8b0 yyparse (ffffffff7fffe4d8, ffffffff7fffe4d8, 100146ab0, 9, ffffffff7fffe4e1, a) + 8c0

    000000010002346c main (1, ffffffff7fffee48, ffffffff7fffee58, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 714

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7da12c nanosleep (ffffffff7defbe60, ffffffff7defbe50)

    0000000100007a74 eventgen_thread (0, 1fc000, 0, 0, 100007a30, 0) + 44

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7db230 waitid (7, 0, ffffffff7dcfbe40, 1)

    000000010002d1d4 procflow_createnwait (0, 1fc000, 0, 0, 10002d110, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28820: shadow -a shadow -i 20 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde1c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffded8, 1, 0, ffffffff7fffde1c) + 38

    0000000100031fac threadflow_init (ffffffff392b2690, 10003ee40, 10000, 14, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecc0, 14, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee18, ffffffff7fffee68, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f41c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28815: shadow -a shadow -i 15 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde6c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf28, 1, 0, ffffffff7fffde6c) + 38

    0000000100031fac threadflow_init (ffffffff392b20a0, 10003ee40, 10000, f, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed10, f, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee68, ffffffff7fffeeb8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393efe40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28808: shadow -a shadow -i 8 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdedc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf98, 1, 0, ffffffff7fffdedc) + 38

    0000000100031fac threadflow_init (ffffffff392b1850, 10003ee40, 10000, 8, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed80, 8, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeed8, ffffffff7fffef28, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393e4140, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28806: shadow -a shadow -i 6 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdefc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfb8, 1, 0, ffffffff7fffdefc) + 38

    0000000100031fac threadflow_init (ffffffff392b15f0, 10003ee40, 10000, 6, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffeda0, 6, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeef8, ffffffff7fffef48, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393ec840, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28829: shadow -a shadow -i 29 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf8c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe048, 1, 0, ffffffff7fffdf8c) + 38

    0000000100031fac threadflow_init (ffffffff392b3140, 10003ee40, 10000, 1d, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee30, 1d, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef88, ffffffff7fffefd8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3940e440, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28877: dbwr -a dbwr -i 13 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde8c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf48, 1, 0, ffffffff7fffde8c) + 38

    0000000100031fac threadflow_init (ffffffff392b6a40, 10003ee40, 10000, d, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed30, d, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee88, ffffffff7fffeed8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff394286c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 100159fe0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a0c0, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a280, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a360, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28810: shadow -a shadow -i 10 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdebc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf78, 1, 0, ffffffff7fffdebc) + 38

    0000000100031fac threadflow_init (ffffffff392b1ab0, 10003ee40, 10000, a, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed60, a, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeeb8, ffffffff7fffef08, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393e84c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28837: shadow -a shadow -i 37 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf0c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfc8, 1, 0, ffffffff7fffdf0c) + 38

    0000000100031fac threadflow_init (ffffffff392b3ac0, 10003ee40, 10000, 25, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedb0, 25, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef08, ffffffff7fffef58, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393f6a40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28804: shadow -a shadow -i 4 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf1c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfd8, 1, 0, ffffffff7fffdf1c) + 38

    0000000100031fac threadflow_init (ffffffff392b1390, 10003ee40, 10000, 4, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedc0, 4, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef18, ffffffff7fffef68, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393ead40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28871: dbwr -a dbwr -i 7 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdeec)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfa8, 1, 0, ffffffff7fffdeec) + 38

    0000000100031fac threadflow_init (ffffffff392b6320, 10003ee40, 10000, 7, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed90, 7, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeee8, ffffffff7fffef38, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39418640, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a1a0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a360, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 100159f00, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a280, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28855: shadow -a shadow -i 55 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffddec)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdea8, 1, 0, ffffffff7fffddec) + 38

    0000000100031fac threadflow_init (ffffffff392b5020, 10003ee40, 10000, 37, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffec90, 37, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffede8, ffffffff7fffee38, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39413540, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28834: shadow -a shadow -i 34 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf3c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdff8, 1, 0, ffffffff7fffdf3c) + 38

    0000000100031fac threadflow_init (ffffffff392b3730, 10003ee40, 10000, 22, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffede0, 22, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef38, ffffffff7fffef88, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39407840, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28866: dbwr -a dbwr -i 2 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf3c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdff8, 1, 0, ffffffff7fffdf3c) + 38

    0000000100031fac threadflow_init (ffffffff392b5d30, 10003ee40, 10000, 2, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffede0, 2, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef38, ffffffff7fffef88, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff3942af40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 100159fe0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a280, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a360, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a1a0, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28859: shadow -a shadow -i 59 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdfac)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe068, 1, 0, ffffffff7fffdfac) + 38

    0000000100031fac threadflow_init (ffffffff392b54e0, 10003ee40, 10000, 3b, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee50, 3b, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffefa8, ffffffff7fffeff8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3941a140, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28885: lgwr -a lgwr -i 1 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde0c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdec8, 1, 0, ffffffff7fffde0c) + 38

    0000000100031fac threadflow_init (ffffffff392b73c0, 10003ee40, 10000, 1, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecb0, 1, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee08, ffffffff7fffee58, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff394034c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (13, 2, 100159b80, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (13, 2, 100159b80, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (13, 2, 100159b80, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (13, 2, 100159b80, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28822: shadow -a shadow -i 22 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffddec)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdea8, 1, 0, ffffffff7fffddec) + 38

    0000000100031fac threadflow_init (ffffffff392b28f0, 10003ee40, 10000, 16, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffec90, 16, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffede8, ffffffff7fffee38, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3942a1c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28836: shadow -a shadow -i 36 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf1c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfd8, 1, 0, ffffffff7fffdf1c) + 38

    0000000100031fac threadflow_init (ffffffff392b3990, 10003ee40, 10000, 24, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedc0, 24, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef18, ffffffff7fffef68, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3940d6c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28801: shadow -a shadow -i 1 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf4c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe008, 1, 0, ffffffff7fffdf4c) + 38

    0000000100031fac threadflow_init (ffffffff392b1000, 10003ee40, 10000, 1, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedf0, 1, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef48, ffffffff7fffef98, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393ebac0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28876: dbwr -a dbwr -i 12 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde9c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf58, 1, 0, ffffffff7fffde9c) + 38

    0000000100031fac threadflow_init (ffffffff392b6910, 10003ee40, 10000, c, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed40, c, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee98, ffffffff7fffeee8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39416b40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a0c0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 100159fe0, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a360, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a280, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28879: dbwr -a dbwr -i 15 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde6c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf28, 1, 0, ffffffff7fffde6c) + 38

    0000000100031fac threadflow_init (ffffffff392b6ca0, 10003ee40, 10000, f, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed10, f, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee68, ffffffff7fffeeb8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff3940a0c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a360, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a280, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 100159fe0, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a1a0, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28846: shadow -a shadow -i 46 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde7c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf38, 1, 0, ffffffff7fffde7c) + 38

    0000000100031fac threadflow_init (ffffffff392b4570, 10003ee40, 10000, 2e, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed20, 2e, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee78, ffffffff7fffeec8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39429440, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28872: dbwr -a dbwr -i 8 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdedc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdf98, 1, 0, ffffffff7fffdedc) + 38

    0000000100031fac threadflow_init (ffffffff392b6450, 10003ee40, 10000, 8, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed80, 8, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeed8, ffffffff7fffef28, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39410cc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a360, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 10015a280, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 100159fe0, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a0c0, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28882: dbwr -a dbwr -i 18 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde3c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdef8, 1, 0, ffffffff7fffde3c) + 38

    0000000100031fac threadflow_init (ffffffff392b7030, 10003ee40, 10000, 12, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffece0, 12, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee38, ffffffff7fffee88, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdc6, 1, ffffffff7e0fbda8)

    0000000100012394 flowop_start (ffffffff39415dc0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1b, 2, 10015a0c0, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1c, 2, 100159fe0, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (19, 2, 10015a360, 100159790, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (1a, 2, 10015a280, 100159a30, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28862: shadow -a shadow -i 62 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf7c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe038, 1, 0, ffffffff7fffdf7c) + 38

    0000000100031fac threadflow_init (ffffffff392b5870, 10003ee40, 10000, 3e, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee20, 3e, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef78, ffffffff7fffefc8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39411a40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28833: shadow -a shadow -i 33 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf4c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe008, 1, 0, ffffffff7fffdf4c) + 38

    0000000100031fac threadflow_init (ffffffff392b3600, 10003ee40, 10000, 21, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedf0, 21, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef48, ffffffff7fffef98, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff394142c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28839: shadow -a shadow -i 39 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdeec)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfa8, 1, 0, ffffffff7fffdeec) + 38

    0000000100031fac threadflow_init (ffffffff392b3d20, 10003ee40, 10000, 27, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed90, 27, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeee8, ffffffff7fffef38, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff394019c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28884: dbwr -a dbwr -i 20 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde1c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffded8, 1, 0, ffffffff7fffde1c) + 38

    0000000100031fac threadflow_init (ffffffff392b7290, 10003ee40, 10000, 14, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecc0, 14, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee18, ffffffff7fffee68, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db01c lwp_cond_wait (ffffffff22010ec8, ffffffff22010f08, 0, 1)

    ffffffff7e7d08dc cond_wait_kernel (ffffffff22010ec8, ffffffff22010f08, 0, 0, 1, 1) + 4c

    ffffffff7e7d0bdc cond_wait_common (ffffffff22010ec8, ffffffff22010f08, 0, ffffffff22010f0c, 16b6d0, 0) + 2bc

    ffffffff7e7d0cf0 cond_wait (ffffffff22010ec8, ffffffff22010f08, 1, 1, ffffffff7e948240, ffffffff7f700a00) + 10

    ffffffff7e7d0d2c pthread_cond_wait (ffffffff22010ec8, ffffffff22010f08, ff000000, 0, 0, 0) + 8

    000000010000dce8 fileset_pick (ffffffff22010df0, 10, 38, 0, 0, fffc00) + 70

    0000000100013e58 flowoplib_pickfile (ffffffff7e0fbad0, ffffffff6f416498, 10, 38, 0, 0) + 138

    0000000100016cbc flowoplib_openfile_common (ffffffff3940ff40, ffffffff6f416498, 7d, 1001474bc, ffffffff7f700a00, 0) + 184

    00000001000143ec flowoplib_filesetup (ffffffff3940ff40, ffffffff6f416498, ffffffff7e0fbdb8, ffffffff7e0fbda8, 0, 1) + 15c

    00000001000147b0 flowoplib_iosetup (ffffffff3940ff40, ffffffff6f416498, ffffffff7e0fbdb8, ffffffff7e0fbdc0, ffffffff7e0fbda8, 2000) + 38

    000000010000a39c fb_lfsflow_aiowrite (ffffffff3940ff40, ffffffff6f416498, ffffffff3940ff40, ffffffff6f416498, 1, 10002e00000) + 4c

    0000000100012394 flowop_start (ffffffff3940ff40, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 3 / thread# 3 --------------------

    ffffffff7e7da8cc kaio (6, 0, ffffffff7e948240, 10, ffffffff7e949ed8, ffffffff7f701200)

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 4 / thread# 4 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159020, 100159030, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159020, 100159030, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100158fb0, 1, 100159030, 0, ffffffff7e949ed8, 0) + 28

    ffffffff7e503610 _aio_send_sigev (100158fb0, 0, 104a4c, ffffffff7e7d6d2c, ffffffff7e5035ac, ffffffff7e608000) + 64

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 5 / thread# 5 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159170, 100159180, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159170, 100159180, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159100, 1, 100159180, 0, ffffffff7f702200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159100, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 6 / thread# 6 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001592c0, 1001592d0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001592c0, 1001592d0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159250, 1, 1001592d0, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (13, 2, 100159b80, 100159250, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 7 / thread# 7 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159410, 100159420, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159410, 100159420, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001593a0, 1, 100159420, 0, ffffffff7f703200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001593a0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 8 / thread# 8 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159560, 100159570, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159560, 100159570, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001594f0, 1, 100159570, 0, 0, 0) + 28

    ffffffff7e503284 _aio_do_request (14, 2, 100159c60, 1001594f0, ffffffff7e608000, 1) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 9 / thread# 9 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (1001596b0, 1001596c0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (1001596b0, 1001596c0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159640, 1, 1001596c0, 0, ffffffff7f704200, 0) + 28

    ffffffff7e503284 _aio_do_request (100159640, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 10 / thread# 10 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159800, 100159810, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159800, 100159810, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159790, 1, 100159810, 0, ffffffff7f704a00, 0) + 28

    ffffffff7e503284 _aio_do_request (100159790, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 11 / thread# 11 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159950, 100159960, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159950, 100159960, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (1001598e0, 1, 100159960, 0, ffffffff7f705200, 0) + 28

    ffffffff7e503284 _aio_do_request (1001598e0, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    ----------------- lwp# 12 / thread# 12 --------------------

    ffffffff7e7d6f80 lwp_park (0, 0, 0)

    ffffffff7e7d076c cond_wait_queue (100159aa0, 100159ab0, 0, 0, ffffffff7e949fc8, 0) + 4c

    ffffffff7e7d0cf0 cond_wait (100159aa0, 100159ab0, 0, 0, 0, ffffffff7e948240) + 10

    ffffffff7e503b48 _aio_idle (100159a30, 1, 100159ab0, 0, ffffffff7f705a00, 0) + 28

    ffffffff7e503284 _aio_do_request (100159a30, 10000, 0, 0, ffffffff7e5031c0, ffffffff7e608000) + c4

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28805: shadow -a shadow -i 5 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf0c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfc8, 1, 0, ffffffff7fffdf0c) + 38

    0000000100031fac threadflow_init (ffffffff392b14c0, 10003ee40, 10000, 5, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffedb0, 5, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef08, ffffffff7fffef58, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393ed5c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28851: shadow -a shadow -i 51 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde2c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdee8, 1, 0, ffffffff7fffde2c) + 38

    0000000100031fac threadflow_init (ffffffff392b4b60, 10003ee40, 10000, 33, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecd0, 33, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee28, ffffffff7fffee78, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39406ac0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28838: shadow -a shadow -i 38 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdefc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfb8, 1, 0, ffffffff7fffdefc) + 38

    0000000100031fac threadflow_init (ffffffff392b3bf0, 10003ee40, 10000, 26, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffeda0, 26, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeef8, ffffffff7fffef48, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3940c940, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28807: shadow -a shadow -i 7 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdeec)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdfa8, 1, 0, ffffffff7fffdeec) + 38

    0000000100031fac threadflow_init (ffffffff392b1720, 10003ee40, 10000, 7, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffed90, 7, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffeee8, ffffffff7fffef38, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393e4ec0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28852: shadow -a shadow -i 52 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffde1c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffded8, 1, 0, ffffffff7fffde1c) + 38

    0000000100031fac threadflow_init (ffffffff392b4c90, 10003ee40, 10000, 34, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffecc0, 34, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffee18, ffffffff7fffee68, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3941aec0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28860: shadow -a shadow -i 60 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf9c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffe058, 1, 0, ffffffff7fffdf9c) + 38

    0000000100031fac threadflow_init (ffffffff392b5610, 10003ee40, 10000, 3c, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffee40, 3c, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef98, ffffffff7fffefe8, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff39422840, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28857: shadow -a shadow -i 57 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffddcc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffde88, 1, 0, ffffffff7fffddcc) + 38

    0000000100031fac threadflow_init (ffffffff392b5280, 10003ee40, 10000, 39, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffec70, 39, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffedc8, ffffffff7fffee18, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff394127c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28802: shadow -a shadow -i 2 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdf3c)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffdff8, 1, 0, ffffffff7fffdf3c) + 38

    0000000100031fac threadflow_init (ffffffff392b1130, 10003ee40, 10000, 2, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffede0, 2, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffef38, ffffffff7fffef88, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff393e69c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

    28824: shadow -a shadow -i 24 -s ffffffff20000000 -m /var/tmp/fbenchaiaWo4

    ----------------- lwp# 1 / thread# 1 --------------------

    ffffffff7e7daf68 lwp_wait (2, ffffffff7fffdddc)

    ffffffff7e7d34dc _thrp_join (2, 0, ffffffff7fffde98, 1, 0, ffffffff7fffdddc) + 38

    0000000100031fac threadflow_init (ffffffff392b2b50, 10003ee40, 10000, 18, a, 0) + 1ec

    000000010002cff0 procflow_exec (ffffffff7fffec80, 18, 10003970f, 100147550, ffffffff7e944674, ffffffff7e7bc37c) + 1c0

    00000001000232b0 main (9, ffffffff7fffedd8, ffffffff7fffee28, ffffffff7e74bb30, ffffffff7ed00100, ffffffff7f700200) + 558

    00000001000065fc _start (0, 0, 0, 0, 0, 0) + 17c

    ----------------- lwp# 2 / thread# 2 --------------------

    ffffffff7e7db6a0 semsys (4, 77, ffffffff7e0fbdb4, 2, ffffffff7e0fbd98)

    0000000100012394 flowop_start (ffffffff3940f1c0, 1fc000, 0, 0, 100011ec8, 1) + 4cc

    ffffffff7e7d6ee0 _lwp_start (0, 0, 0, 0, 0, 0)

     
  • David Lutz

    David Lutz - 2010-07-21

    The csv list of semaphores used by the processes that are in flowoplib_sempost() or flowoplib_semblock():

    pid,sys_semid,sbuf[0].sem_num,sbuf[0].sem_op,sbuf[1].sem_num,sbuf[1].sem_op
    28843,119,4U,1,5U,-1
    28809,119,4U,1,5U,-1
    28830,119,4U,1,5U,-1
    28881,119,13U,700,12U,-700
    28854,119,4U,1,5U,-1
    28844,119,4U,1,5U,-1
    28825,119,4U,1,5U,-1
    28849,119,4U,1,5U,-1
    28848,119,4U,1,5U,-1
    28814,119,4U,1,5U,-1
    28868,119,33U,700,32U,-700
    28821,119,4U,1,5U,-1
    28845,119,4U,1,5U,-1
    28832,119,4U,1,5U,-1
    28819,119,4U,1,5U,-1
    28827,119,4U,1,5U,-1
    28811,119,4U,1,5U,-1
    28861,119,4U,1,5U,-1
    28858,119,4U,1,5U,-1
    28812,119,4U,1,5U,-1
    28853,119,4U,1,5U,-1
    28828,119,4U,1,5U,-1
    28841,119,4U,1,5U,-1
    28813,119,4U,1,5U,-1
    28874,119,7U,700,6U,-700
    28842,119,4U,1,5U,-1
    28863,119,4U,1,5U,-1
    28883,119,11U,700,10U,-700
    28865,119,35U,700,34U,-700
    28847,119,4U,1,5U,-1
    28803,119,4U,1,5U,-1
    28856,119,4U,1,5U,-1
    28867,119,1U,700,0,-700
    28873,119,27U,700,26U,-700
    28817,119,4U,1,5U,-1
    28835,119,4U,1,5U,-1
    28869,119,17U,700,16U,-700
    28816,119,4U,1,5U,-1
    28831,119,4U,1,5U,-1
    28850,119,4U,1,5U,-1
    28840,119,4U,1,5U,-1
    28864,119,4U,1,5U,-1
    28826,119,4U,1,5U,-1
    28823,119,4U,1,5U,-1
    28818,119,4U,1,5U,-1
    28870,119,31U,700,30U,-700
    28875,119,37U,700,36U,-700
    28820,119,4U,1,5U,-1
    28815,119,4U,1,5U,-1
    28808,119,4U,1,5U,-1
    28806,119,4U,1,5U,-1
    28829,119,4U,1,5U,-1
    28877,119,39U,700,38U,-700
    28810,119,4U,1,5U,-1
    28837,119,4U,1,5U,-1
    28804,119,4U,1,5U,-1
    28871,119,29U,700,28U,-700
    28855,119,4U,1,5U,-1
    28834,119,4U,1,5U,-1
    28866,119,41U,700,40U,-700
    28859,119,4U,1,5U,-1
    28822,119,4U,1,5U,-1
    28836,119,4U,1,5U,-1
    28801,119,4U,1,5U,-1
    28876,119,25U,700,24U,-700
    28879,119,15U,700,14U,-700
    28846,119,4U,1,5U,-1
    28872,119,21U,700,20U,-700
    28882,119,23U,700,22U,-700
    28862,119,4U,1,5U,-1
    28833,119,4U,1,5U,-1
    28839,119,4U,1,5U,-1
    28805,119,2U,1,3U,-1
    28851,119,4U,1,5U,-1
    28838,119,4U,1,5U,-1
    28807,119,4U,1,5U,-1
    28852,119,4U,1,5U,-1
    28860,119,4U,1,5U,-1
    28857,119,4U,1,5U,-1
    28802,119,4U,1,5U,-1
    28824,119,4U,1,5U,-1

     
  • Vasily Tarasov

    Vasily Tarasov - 2011-05-27
    • priority: 5 --> 7
    • status: open --> open-accepted
     
  • Vasily Tarasov

    Vasily Tarasov - 2014-07-31
    • Status: open-accepted --> pending