Re: [Mssqlcompressed-users] Multithreaded compression

[Clay L. <cl...@le...>]

Louie,

Whoops!  I've missed documenting the multithread option on the website.

Each destination file has its own thread (including the pipeline to the file
such as compression), so you can specify multiple files with multiple "path"
parameters in the "local" storage plugin:

msbp.exe backup "db(database=model)" zip64
"local(path=c:\model1.bak.zip;path=c:\model2.bak.zip)"

The reason it was done this way is that some compression file formats, like
gzip, do not allow for multithreaded compression very easily, and I didn't
want to make a custom file formats that similar products have done so that
the backups msbp.exe produces is easy to restore without any proprietary
software.

I originally (before this project) played around with backing up to a
Windows compressed folder, but it kept locking up for larger backups (over
32 GB I think), which lead to this project.  I don't know if Windows files
compression is multicore.  I would suspect that it is single-core for one
file, but if you back up to multiple files you could achieve multicore
compression.

Cheers,
Clay


2009/12/15 Louie Gregg <lou...@ie...>

>  Clay
>
>
>
> Excellent stuff, the SQL world has been crying out for something like this.
> Not only compression, but backups across the network – I have a few servers
> to backup, and want them all on the one USB disk. Excellent.
>
>
>
> You said: *Multithreaded Compression**. SQL Server Compressed Backup can
> compress your backups using multiple threads to take advantage of multiple
> cores.*
>
>
>
> How do I get this to work? Can I select how many cores are active? Runtime
> selection? Much used or beta or ...?
>
>
>
> My backup ran on one core, I have eight cores, and the disks weren’t busy
>  I did dig around your code, trying to find an answer. I hoped it would be a
> param on zip64(), but I see it has only two/one in its schema
> (Zip64Transform.cs). Can’t find any other params that might control it.
> Considered that it might be additional transforms in the pipeline, but that
> sounds difficult, even nonsensical. I’m guessing its a compilation option
> L.
>
>
>
> I have thought of another option: write an uncompressed backup onto a
> windows-compressed file system. Not possible with vanilla SQL2005, easy I
> expect with your handy program. Though I’m not sure that windows would
> compress multi-core. Any comments?
>
>
>
> Regards
>
> Louie Gregg
>
>
>
> IT Dept, IESA Ltd
>
> lou...@ie...
>
> 01925 454409
>
> 0759 577 3206
>
>
>
>
> IESA Limited, a company registered in England and Wales with the company
> number 4188491 and VAT Number GB 703 4081 74 whose registered office is
> Dallam Lane, Warrington, Cheshire, WA2 7PZ.
>
> For more details of our services visit *** http://www.iesa.co.uk ***
>
> This communication contains information which is confidential and may also
> be privileged. It is for the exclusive use of the addressee. If you are not
> the addressee please note that any distribution, reproduction, copying,
> publication or use of this communication or the information in it is
> prohibited. If you have received this communication in error, please
> telephone us immediately (UK +44 1925 454400) and also delete the
> communication from your computer. The views expressed in this communication
> may not necessarily be the views held by IESA Limited.
>
>
> ------------------------------------------------------------------------------
> Return on Information:
> Google Enterprise Search pays you back
> Get the facts.
> http://p.sf.net/sfu/google-dev2dev
>
> _______________________________________________
> Mssqlcompressed-users mailing list
> Mss...@li...
> https://lists.sourceforge.net/lists/listinfo/mssqlcompressed-users
>
>