Menu

#378 decompressing are very slow with large number of files due to getdents64 calls

open
nobody
None
5
2021-09-04
2020-12-22
gyko
No

When the archive contains a large number of small files, the decompressor makes too many possible unnecessary calls to the getdents64 function. Here is the strace log:

strace -c 7z e test.7z

...
Everything is Ok

Folders: 13
Files: 23859
Size:       273479029
Compressed: 20727696
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 97,50   84,770124         186    453847           getdents64
  0,57    0,494513          10     47743         4 openat
  0,54    0,473838          19     24513           write
  0,31    0,268364           5     47739           close
  0,22    0,192505           8     23859     23859 readlink

In this example there were 453847 calls to getdents64 (97% of cpu time) for 23859 files

Discussion

  • Igor Pavlov

    Igor Pavlov - 2020-12-22

    It's problem of p7zip's implementation of one function of 7-Zip.

     
  • necros

    necros - 2021-09-04

    +1

     

Log in to post a comment.