Recent changes to Using_Snappy_in_Picard

Using_Snappy_in_Picard modified by Alec Wysoker

Alec Wysoker — Mon, 30 Jun 2014 15:10:20 -0000

--- v7
+++ v8
@@ -1,4 +1,4 @@
-[Snappy](http://code.google.com/p/snappy/) is an open-source compression library that emphasizes fast compression over maximum compression. [Snappy-java](http://code.google.com/p/snappy-java/) is a Java JNI wrapper for Snappy. If the Snappy library is installed on a system and made available to the JVM then Picard programs that sort large amounts of data will use it to compress temporary data before writing it to disk. This is a performance optimization only and is entirely optional. Snappy can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may vary. 
+[Snappy](http://code.google.com/p/snappy/) is an open-source compression library that emphasizes fast compression over maximum compression. [Snappy-java](http://code.google.com/p/snappy-java/) is a Java JNI wrapper for Snappy. If the Snappy-java library is installed on a system and made available to the JVM then Picard programs that sort large amounts of data will use it to compress temporary data before writing it to disk. This is a performance optimization only and is entirely optional. Snappy can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may vary.

@@ -7,23 +7,22 @@

 ## Requirements for using Snappy

-In order for Picard to use Snappy, two requirements must be met: 1) the Snappy-java classes must be available on Java's classpath; and 2) the Snappy DLL (dynamic-link library) must be available to Snappy-java. 
+In order for Picard to use Snappy, two requirements must be met: 1) the Snappy-java classes must be available on Java's classpath; and 2) the Snappy-java DLL (dynamic-link library) must be available to Snappy-java. 

 ### Making Snappy-java classes available

-If you use one of the executable jars in the Picard bundle (e.g. SortSam.jar), the Snappy-java classes are included in the executable jar so you need do nothing additional. However, if you are using sam-<version#>.jar, then you must also add the Snappy-java jar to the classpath. The jar can be found in the picard-tools-<version#>.zip file. 
+On Linux only, if you use one of the executable jars in the Picard bundle (e.g. SortSam.jar), the Snappy-java classes are included in the executable jar so you need do nothing additional. However, if you are using sam-<version#>.jar, or it you are on , then you must also add the Snappy-java jar to the classpath. The jar can be found in the picard-tools-<version#>.zip file. 

-### Obtaining the Snappy DLL
+### Obtaining the Snappy-java DLL

-Currently, the Snappy project only distributes source code. You can download the Snappy sources and build for your platform. Alternately, the Snappy-java jarfile contains Snappy DLLs for the most popular platforms, so you may extract it from there. 
+TYhe Snappy-java jarfile contains Snappy DLLs for the most popular platforms, so you may extract it from there. Note that the Snappy-java DLL is not the same as the Snappy DLL, because it contains JNI (Jave native interface) code in addition to the Snappy code itself. 

 ### Making the Snappy DLL available

-If the Snappy jar is on your classpath, Snappy-java can find the Snappy (DLL) automatically. However, if you use one of the executable jars (e.g. SortSam.jar), then you must make the Snappy DLL available for Snappy-java to load. There are several ways this can be done: 
+If the Snappy jar is on your classpath, Snappy-java can find the Snappy (DLL) automatically. The Linux version of the Snappy-java DLL is included in the executable jars. On other platforms, you must make the Snappy DLL available for Snappy-java to load. There are several ways this can be done: 

-  * On Linux systems, add the path to the Snappy DLL to the LD_LIBRARY_PATH environment variable. This does not work on Mac, and I'm not sure about Windows. 
-  * Set the org.xerial.snappy.lib.path system property to point to the directory containing the Snappy DLL. E.g. add an argument like the following to your Java command line: `-Dorg.xerial.snappy.lib.path=<directory containing Snappy DLL>`. 
-  * Put the Snappy-java jar on your classpath, e.g. add an argumet like the following to your Java commane line: `-classpath snappy-java<version#>.jar`. 
+  * Set the org.xerial.snappy.lib.path system property to point to the directory containing the Snappy-java DLL. E.g. add an argument like the following to your Java command line: `-Dorg.xerial.snappy.lib.path=<directory containing Snappy DLL>`. 
+  * Put the Snappy-java jar on your classpath, e.g. add an argument like the following to your Java command line: `-classpath snappy-java<version#>.jar`. 

 ### Controlling Snappy in Picard

Using_Snappy_in_Picard modified by Alec Wysoker

Alec Wysoker — Mon, 30 Jun 2014 15:10:20 -0000

--- v6
+++ v7
@@ -1,4 +1,4 @@
-[[Snappy](http://code.google.com/p/snappy/)] is an open-source compression library that emphasizes fast compression over maximum compression. [[Snappy-java](http://code.google.com/p/snappy-java/)] is a Java JNI wrapper for Snappy. If the Snappy library is installed on a system and made available to the JVM then Picard programs that sort large amounts of data will use it to compress temporary data before writing it to disk. This is a performance optimization only and is entirely optional. Snappy can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may vary. 
+[Snappy](http://code.google.com/p/snappy/) is an open-source compression library that emphasizes fast compression over maximum compression. [Snappy-java](http://code.google.com/p/snappy-java/) is a Java JNI wrapper for Snappy. If the Snappy library is installed on a system and made available to the JVM then Picard programs that sort large amounts of data will use it to compress temporary data before writing it to disk. This is a performance optimization only and is entirely optional. Snappy can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may vary.

Using_Snappy_in_Picard modified by Alec Wysoker

Alec Wysoker — Mon, 30 Jun 2014 15:10:20 -0000

--- v5
+++ v6
@@ -1,4 +1,4 @@
-[[Snappy](http://code.google.com/p/snappy/)] is an open-source compression library that emphasizes fast compression over maximum compression. [[Snappy-java](http://code.google.com/p/snappy-java/)] is a Java JNI wrapper for Snappy. The Picard library and Picard programs that use SortingCollection to sort large data sets (mostly SAMRecords) optionally use Snappy to compress temporary files written in the process of doing a disk-based merge sort. This can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may vary. 
+[[Snappy](http://code.google.com/p/snappy/)] is an open-source compression library that emphasizes fast compression over maximum compression. [[Snappy-java](http://code.google.com/p/snappy-java/)] is a Java JNI wrapper for Snappy. If the Snappy library is installed on a system and made available to the JVM then Picard programs that sort large amounts of data will use it to compress temporary data before writing it to disk. This is a performance optimization only and is entirely optional. Snappy can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may vary.

Using_Snappy_in_Picard modified by Alec Wysoker

Alec Wysoker — Mon, 30 Jun 2014 15:10:20 -0000

--- v4
+++ v5
@@ -23,7 +23,7 @@

   * On Linux systems, add the path to the Snappy DLL to the LD_LIBRARY_PATH environment variable. This does not work on Mac, and I'm not sure about Windows. 
   * Set the org.xerial.snappy.lib.path system property to point to the directory containing the Snappy DLL. E.g. add an argument like the following to your Java command line: `-Dorg.xerial.snappy.lib.path=<directory containing Snappy DLL>`. 
-  * You can put the Snappy-java jar on your classpath, e.g. add an argumet like the following to your Java commane line: `-classpath snappy-java<version#>.jar`. 
+  * Put the Snappy-java jar on your classpath, e.g. add an argumet like the following to your Java commane line: `-classpath snappy-java<version#>.jar`. 

 ### Controlling Snappy in Picard

Using_Snappy_in_Picard modified by Alec Wysoker

Alec Wysoker — Mon, 30 Jun 2014 15:10:20 -0000

--- v3
+++ v4
@@ -21,9 +21,9 @@

 If the Snappy jar is on your classpath, Snappy-java can find the Snappy (DLL) automatically. However, if you use one of the executable jars (e.g. SortSam.jar), then you must make the Snappy DLL available for Snappy-java to load. There are several ways this can be done: 

-  * On Linux systems, if the LD_LIBRARY_PATH environment variable contains the path to the Snappy DLL, Snappy-java can find it. This does not work on Mac, and I'm not sure about Windows. 
-  * You can set the org.xerial.snappy.lib.path system property to point to the directory containing the Snappy DLL. E.g. add and argument like the following to your Java command line: `-Dorg.xerial.snappy.lib.path=<directory containing Snappy DLL>`. 
-  * You can put the Snappy-java jar on your classpath, e.g. `-classpath snappy-java<version#>.jar`. 
+  * On Linux systems, add the path to the Snappy DLL to the LD_LIBRARY_PATH environment variable. This does not work on Mac, and I'm not sure about Windows. 
+  * Set the org.xerial.snappy.lib.path system property to point to the directory containing the Snappy DLL. E.g. add an argument like the following to your Java command line: `-Dorg.xerial.snappy.lib.path=<directory containing Snappy DLL>`. 
+  * You can put the Snappy-java jar on your classpath, e.g. add an argumet like the following to your Java commane line: `-classpath snappy-java<version#>.jar`. 

 ### Controlling Snappy in Picard

Using_Snappy_in_Picard modified by Alec Wysoker

Alec Wysoker — Mon, 30 Jun 2014 15:10:20 -0000

--- v2
+++ v3
@@ -1,4 +1,4 @@
-[[Snappy](http://code.google.com/p/snappy/)] is an open-source compression library that emphasizes fast compression over maximum compression. [[Snappy-java](http://code.google.com/p/snappy-java/)] is a Java JNI wrapper for Snappy. The Picard library and Picard programs that use SortingCollection to sort large data sets (mostly SAMRecords) optionally use Snappy to compress temporary files written in the process of doing a disk-based merge sort. This can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may depend on many factors. 
+[[Snappy](http://code.google.com/p/snappy/)] is an open-source compression library that emphasizes fast compression over maximum compression. [[Snappy-java](http://code.google.com/p/snappy-java/)] is a Java JNI wrapper for Snappy. The Picard library and Picard programs that use SortingCollection to sort large data sets (mostly SAMRecords) optionally use Snappy to compress temporary files written in the process of doing a disk-based merge sort. This can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may vary.

Using_Snappy_in_Picard modified by Alec Wysoker

Alec Wysoker — Mon, 30 Jun 2014 15:10:19 -0000

--- v1
+++ v2
@@ -2,6 +2,8 @@

+
+[TOC]

 ## Requirements for using Snappy

@@ -11,6 +13,10 @@

 If you use one of the executable jars in the Picard bundle (e.g. SortSam.jar), the Snappy-java classes are included in the executable jar so you need do nothing additional. However, if you are using sam-<version#>.jar, then you must also add the Snappy-java jar to the classpath. The jar can be found in the picard-tools-<version#>.zip file. 

+### Obtaining the Snappy DLL
+
+Currently, the Snappy project only distributes source code. You can download the Snappy sources and build for your platform. Alternately, the Snappy-java jarfile contains Snappy DLLs for the most popular platforms, so you may extract it from there. 
+
 ### Making the Snappy DLL available

 If the Snappy jar is on your classpath, Snappy-java can find the Snappy (DLL) automatically. However, if you use one of the executable jars (e.g. SortSam.jar), then you must make the Snappy DLL available for Snappy-java to load. There are several ways this can be done: 
@@ -18,3 +24,9 @@
   * On Linux systems, if the LD_LIBRARY_PATH environment variable contains the path to the Snappy DLL, Snappy-java can find it. This does not work on Mac, and I'm not sure about Windows. 
   * You can set the org.xerial.snappy.lib.path system property to point to the directory containing the Snappy DLL. E.g. add and argument like the following to your Java command line: `-Dorg.xerial.snappy.lib.path=<directory containing Snappy DLL>`. 
   * You can put the Snappy-java jar on your classpath, e.g. `-classpath snappy-java<version#>.jar`. 
+
+### Controlling Snappy in Picard
+
+In order to see if Snappy has been found by Picard, pass `-Dsnappy.loader.verbosity=true` on the Java command line. When SortingCollection first tries to sort something, a message will be printed to stderr indicating whether or not Snappy-java and Snappy DLL have been found. 
+
+In order to force Snappy not to be used by Picard, pass `-Dsnappy.disable=true` on the Java command line.

Using_Snappy_in_Picard modified by Alec Wysoker

Alec Wysoker — Mon, 30 Jun 2014 15:10:19 -0000

[Snappy] is an open-source compression library that emphasizes fast compression over maximum compression. [Snappy-java] is a Java JNI wrapper for Snappy. The Picard library and Picard programs that use SortingCollection to sort large data sets (mostly SAMRecords) optionally use Snappy to compress temporary files written in the process of doing a disk-based merge sort. This can reduce file I/O, and in some cases the savings in file I/O more than compensates for the time spent compressing and decompressing, and results in faster execution times. We have seen sort times reduced by nearly 25%, although your mileage may depend on many factors.

Requirements for using Snappy

In order for Picard to use Snappy, two requirements must be met: 1) the Snappy-java classes must be available on Java's classpath; and 2) the Snappy DLL (dynamic-link library) must be available to Snappy-java.

Making Snappy-java classes available

If you use one of the executable jars in the Picard bundle (e.g. SortSam.jar), the Snappy-java classes are included in the executable jar so you need do nothing additional. However, if you are using sam-<version#>.jar, then you must also add the Snappy-java jar to the classpath. The jar can be found in the picard-tools-<version#>.zip file.

Making the Snappy DLL available

If the Snappy jar is on your classpath, Snappy-java can find the Snappy (DLL) automatically. However, if you use one of the executable jars (e.g. SortSam.jar), then you must make the Snappy DLL available for Snappy-java to load. There are several ways this can be done:

On Linux systems, if the LD_LIBRARY_PATH environment variable contains the path to the Snappy DLL, Snappy-java can find it. This does not work on Mac, and I'm not sure about Windows.
You can set the org.xerial.snappy.lib.path system property to point to the directory containing the Snappy DLL. E.g. add and argument like the following to your Java command line: -Dorg.xerial.snappy.lib.path=<directory containing Snappy DLL>.
You can put the Snappy-java jar on your classpath, e.g. -classpath snappy-java<version#>.jar.