#11 par2cmdline multi processor (win32)

Unstable (example)
open
Peter C
5
2006-09-23
2006-09-23
No

This patch uses the latest CVS files of par2cmdline,
with patches from macgerard for multiple cpu's on
GNU/Linux, and enables multiple cpu usage on Win32.
This significantly increases performance for especially
the creation of par2-archives.
Added is a patch file for the source of par2cmdline
0.4, plus a compiled binary for win32 - needs
pthreadVC2.dll (for example in c:\windows\system32).
Requirements for compilation: pthreads-win32
(http://sourceware.org/pthreads-win32/)

This patch is untested, usage is entirely at your own
risk. This patch is solely released here to allow the
original developers to easily incorporate this patch
into their own software.

Discussion

  • Logged In: YES
    user_id=439782

    Just a small remark wrg the performance for repairing and
    verifying archives:
    the strategy originally implemented by macgerard for
    repairing/verifying par2-archives was that each CPU has 1
    thread that verifies 1 file. So, when taking 2 CPU's as
    example, it will verify 2 files simultaneously. While
    theoretically this could work, practically it won't, simply
    because the overhead of the harddrive reading 2 (large)
    files simultaneously is simply too big. The CPU never gets
    enough work to do, because the harddrive can't keep up.
    A possible solution (but not implemented!) could be to
    change the repair/verify algorithm in the following way:
    * only repair/verify 1 file at a time
    * read in 2 blocks of 50% physical blocksize of the
    harddrive for 1 file - for example for a physical 16 KB
    block, read in 2* 8 KB of 1 file
    * process these 2 blocks simultaneously in separate threads

    This would make the processing bit complicated, and I have
    not checked out the actual algorithm to see if this is
    easily implemented. If it were, however, this solution
    should be much more efficient.