Recent changes to LambdaD

LambdaD modified by Guobo Chen

Guobo Chen — Mon, 27 Apr 2015 04:48:40 -0000

--- v16
+++ v17
@@ -1,5 +1,5 @@
 **LambdaMeta**
-This function is designed to evaluate the quality of summary statistics.  It can calculate LambdaMeta for a pair of summary statistics on genetic effects; also can calculate Fst between cohorts on allele frequency.
+This function is designed to evaluate the quality of summary statistics. It can calculate LambdaMeta for a pair of summary statistics on genetic effects; also can calculate Fst between cohorts on allele frequency.
 LambdaMeta is a statistic invented to measure heterogeneity/homogeneity between cohorts.  Under the null hypothesis, lambdameta=1 if the pair of cohorts are completely independent and no biological difference, such as different genetic architecture, or technical difference, such as different analysis protocols. When there are overlapping samples, LambdaMeta < 1; when there are bilogical/techncal difference, LambdaMeta >1. Often, LambdaMeta is slightly greater than zero.

 **Master command**
@@ -46,15 +46,12 @@
 "--key markerID effect SE Ref_Allele Other_allele Chromosome POS Pval"
 Pval will be used to calculate genomic inflation factor for each cohort.

---fst
-When --fst is switched on, it will calculate fst statistic between cohorts.
-Of note, when --fst is on, the column for allele frequency and s.e. for allele frequency should be specified in --key.  For example, the --key should be
-"--key markerID RAF RAF_SE Ref_Allele Other_Allele CHR POS Pval"
-
 --top <arg>
 This option tells the program only the top X files listed in --meta-batch will be compared to all files.  For example, if there are 10 summary statistic files included in --meta-batch, when "--top 1" is used, it only calculate lambdaMeta (of fst) between the first file and other files.
 In practice, if only want to calculate fst between the cohort to 1KG European samples, the user can put the summary statistic file for 1KG as the first file in --meta-batch and use "--top 1" option.

+--verbose
+When this option is switched on, the detailed results for SNPs that are used for calculating lambdametw will be saved in "*.lam.gz" for each pair of cohorts. 

 Examples

@@ -62,15 +59,16 @@
 java -jar gear.jar lam --meta-batch metalist.txt --qt-size qt-sample-size.txt --key SNP BETA SE A1 A2 CHR BP P --out test
 java -jar gear.jar lam --meta-batch metalist.txt --qt-size cc-sample-size.txt --key SNP BETA SE A1 A2 CHR BP P --me 50000 --out test

-java -jar gear.jar lam --meta-batch metalist.txt --qt-size qt-sample-size.txt --key SNP RAF RAFSE A1 A2 CHR BP P --out test
+java -jar gear.jar lam --meta-batch metalist.txt --qt-size qt-sample-size.txt --key SNP BETA SE A1 A2 CHR BP P --top 1 --out test

-java -jar gear.jar lam --meta-batch metalist.txt --cc-size cc-sample-size.txt --out test
-java -jar gear.jar lam --meta-batch 1kg_OtherCohort.txt --qt-size qt-sample-size.txt  --key SNP RAF RAFSE A1 A2 CHR BP P --out test
+java -jar gear.jar lam --meta-batch metalist.txt --qt-size qt-sample-size.txt --key SNP BETA SE A1 A2 CHR BP P --verbose --out test
 ~~~~~~~
-When calculating LambdaMeta over genetic effects.
-Two result files will be found. One is ".lmat", which has the covariance and estimated overlapping samples; the other is ".lamB", which saves the lambdaD for SNPs. When "--fst" option is on, "lamF" and ".fst" file will be found.

-In "lamB"/"lambF" file, using "ChiExp" and "ChiObs" as x- and y-axis, respectively, a plot can be made for each pair of cohorts. The slop of the line should be very close to 1 if the quality is fairly good.
+In "*.lmat", please find the lambdameta for each pair of cohorts.
+In "*.gc", please find the lambda_gc for each cohort using the reported pvalues (first column) and the reported beta/se (second column).  These two columns should be nearly identical.  But if they are different, it may indicate come cohorts have uploaded summary results that have already been corrected for lambda_gc.
+ 
+
+In "lamB"/"lambF" file, using "ChiExp" and "ChiObs" as x- and y-axis, respectively, a plot can be made for each pair of cohorts. The slop of the line should be very close to 1 if the quality of summary statistics is fairly good.

 **Overlapping samples deflates LambdaMeta**
 ![Overlapping sample](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/BD_CD.png)

LambdaD modified by Guobo Chen

Guobo Chen — Sun, 22 Feb 2015 11:32:29 -0000

--- v15
+++ v16
@@ -1,5 +1,5 @@
 **LambdaMeta**
-This function is designed to evaluate the quality of summary statistics.  It can calculate LambdaMeta for a pair of summary statistics; also can calculate Fst between cohorts.
+This function is designed to evaluate the quality of summary statistics.  It can calculate LambdaMeta for a pair of summary statistics on genetic effects; also can calculate Fst between cohorts on allele frequency.
 LambdaMeta is a statistic invented to measure heterogeneity/homogeneity between cohorts.  Under the null hypothesis, lambdameta=1 if the pair of cohorts are completely independent and no biological difference, such as different genetic architecture, or technical difference, such as different analysis protocols. When there are overlapping samples, LambdaMeta < 1; when there are bilogical/techncal difference, LambdaMeta >1. Often, LambdaMeta is slightly greater than zero.

 **Master command**
@@ -67,15 +67,13 @@
 java -jar gear.jar lam --meta-batch metalist.txt --cc-size cc-sample-size.txt --out test
 java -jar gear.jar lam --meta-batch 1kg_OtherCohort.txt --qt-size qt-sample-size.txt  --key SNP RAF RAFSE A1 A2 CHR BP P --out test
 ~~~~~~~
+When calculating LambdaMeta over genetic effects.
+Two result files will be found. One is ".lmat", which has the covariance and estimated overlapping samples; the other is ".lamB", which saves the lambdaD for SNPs. When "--fst" option is on, "lamF" and ".fst" file will be found.

-Two result files will be found. One is ".lmat", which has the covariance and estimated overlapping samples; the other is ".lamB", which saves the lambdaD for SNPs.
-
-When "--fst" option is on, "lamF" and ".fst" file will be found.
-
-In "lamB"/"lambF", using "ChiExp" as the x-axis, and "ChiObs" as y-axis, a plot can be made for each pair of cohorts.  The slop of the line should be very close to 1 if the quality is fairly good.
+In "lamB"/"lambF" file, using "ChiExp" and "ChiObs" as x- and y-axis, respectively, a plot can be made for each pair of cohorts. The slop of the line should be very close to 1 if the quality is fairly good.

 **Overlapping samples deflates LambdaMeta**
 ![Overlapping sample](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/BD_CD.png)

 **Heterogeneity inflates LambdaMeta**
-![Fairly good](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/2_16.png)
+![Fairly good](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/1_2.png)

LambdaD modified by Guobo Chen

Guobo Chen — Sun, 22 Feb 2015 11:27:09 -0000

--- v14
+++ v15
@@ -72,10 +72,10 @@

 When "--fst" option is on, "lamF" and ".fst" file will be found.

-In "lamB"/"lambF", using "ChiExp" as the x-axis, and "ChiObs" as y-axis, a plot can be made for each pair of cohort.  The slop of the line should be very close to 1 if the quality is fairly good.
+In "lamB"/"lambF", using "ChiExp" as the x-axis, and "ChiObs" as y-axis, a plot can be made for each pair of cohorts.  The slop of the line should be very close to 1 if the quality is fairly good.

-Overlapping samples deflates LambdaMeta
-![Overlapping sample](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/BD_CDlam.png)
+**Overlapping samples deflates LambdaMeta**
+![Overlapping sample](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/BD_CD.png)

-Heterogeneity inflates LambdaMeta
+**Heterogeneity inflates LambdaMeta**
 ![Fairly good](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/2_16.png)

LambdaD modified by Guobo Chen

Guobo Chen — Sun, 22 Feb 2015 11:24:06 -0000

--- v13
+++ v14
@@ -74,4 +74,8 @@

 In "lamB"/"lambF", using "ChiExp" as the x-axis, and "ChiObs" as y-axis, a plot can be made for each pair of cohort.  The slop of the line should be very close to 1 if the quality is fairly good.

+Overlapping samples deflates LambdaMeta
+![Overlapping sample](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/BD_CDlam.png)

+Heterogeneity inflates LambdaMeta
+![Fairly good](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/2_16.png)

LambdaD modified by Guobo Chen

Guobo Chen — Sun, 22 Feb 2015 11:18:37 -0000

--- v12
+++ v13
@@ -1,6 +1,6 @@
 **LambdaMeta**
 This function is designed to evaluate the quality of summary statistics.  It can calculate LambdaMeta for a pair of summary statistics; also can calculate Fst between cohorts.
-LambdaMeta is a statistic invented to measure heterogeneity/homogeneity between cohorts.  Under the null hypothesis, lambdameta=1 if the pair of cohorts are completely independent and no biological, such as different genetic architecture, or technical difference, such as different analysis protocols.  When there are overlapping samples, LambdaMeta < 1; when there are bilogical/techncal difference, LambdaMeta >1.  Often, LambdaMeta is slightly greater than zero.
+LambdaMeta is a statistic invented to measure heterogeneity/homogeneity between cohorts.  Under the null hypothesis, lambdameta=1 if the pair of cohorts are completely independent and no biological difference, such as different genetic architecture, or technical difference, such as different analysis protocols. When there are overlapping samples, LambdaMeta < 1; when there are bilogical/techncal difference, LambdaMeta >1. Often, LambdaMeta is slightly greater than zero.

 **Master command**
 lam
@@ -56,7 +56,7 @@
 In practice, if only want to calculate fst between the cohort to 1KG European samples, the user can put the summary statistic file for 1KG as the first file in --meta-batch and use "--top 1" option.

-Example
+Examples

 ~~~~~~~
 java -jar gear.jar lam --meta-batch metalist.txt --qt-size qt-sample-size.txt --key SNP BETA SE A1 A2 CHR BP P --out test
@@ -68,6 +68,10 @@
 java -jar gear.jar lam --meta-batch 1kg_OtherCohort.txt --qt-size qt-sample-size.txt  --key SNP RAF RAFSE A1 A2 CHR BP P --out test
 ~~~~~~~

-Two result files will be found. One is ".lmat", which has the covariance and estimated overlapping samples; the other is ".lam", which saves the lambdaD for SNPs.
+Two result files will be found. One is ".lmat", which has the covariance and estimated overlapping samples; the other is ".lamB", which saves the lambdaD for SNPs.

-When "--fst" option is on, the ".fst" file will be found.
+When "--fst" option is on, "lamF" and ".fst" file will be found.
+
+In "lamB"/"lambF", using "ChiExp" as the x-axis, and "ChiObs" as y-axis, a plot can be made for each pair of cohort.  The slop of the line should be very close to 1 if the quality is fairly good.
+
+

LambdaD modified by Guobo Chen

Guobo Chen — Sun, 22 Feb 2015 11:14:50 -0000

--- v11
+++ v12
@@ -1,89 +1,73 @@
-**LambdaD**
+**LambdaMeta**
+This function is designed to evaluate the quality of summary statistics.  It can calculate LambdaMeta for a pair of summary statistics; also can calculate Fst between cohorts.
+LambdaMeta is a statistic invented to measure heterogeneity/homogeneity between cohorts.  Under the null hypothesis, lambdameta=1 if the pair of cohorts are completely independent and no biological, such as different genetic architecture, or technical difference, such as different analysis protocols.  When there are overlapping samples, LambdaMeta < 1; when there are bilogical/techncal difference, LambdaMeta >1.  Often, LambdaMeta is slightly greater than zero.

 **Master command**
 lam

---meta/--meta-gz
-Specify the meta-data in text/gz format one by one.
+The meta-data should have these columns: "SNP", "BETA" (or "OR" for case-control studies), "SE", "A1", "A2", "CHR", "BP", and "P". The keywords are case-insensitive, and there is no requirement for the order of these columns. "A1" is the reference allele, and "A2" is the other allele. Other columns such as reference allele frequency, standard error of allele frequency can also be included. 

---cc
-Specify the numbers of cases and controls for each meta file which are for case-control studies.
+SNP  | CHR | BP | A1 | A2 | OR/Beta | SE | P | RAF | RAF_SE
+-----|-----|----|----|----|---------|----|---|---|
+snp1 | 1 | 100 | G  | T |1.05 | 0.03 | 0.03 | 0.10 | 0.35 | 0.032
+snp2 | 2 | 200 |  T  | A |0.95 | 0.033 | 0.12 | 0.3 | 0.03

---qt
-Specify the sample size for each meta file which are for quantitative traits.
+The program will automatically eliminate ambiguous loci, such as A/T and G/C loci. In the example, the second row, which has ambiguous alleles will be eliminated.
+Of note, all the summary statistic files should have the same column names, but their order can be different.

-The meta-data should have these columns: "SNP", "OR" (or "beta" for quantitative traits), "SE", "A1", "A2", "CHR", "BP", and "P". The last three key words are optional. The keywords are case-insensitive, and there is no requirement for the order of these columns. "A1" is the reference allele, and "A2" is the other allele.  If "A2" column is absent, the program will not eliminate ambiguous loci, such as A/T and G/C loci.
+**Options**

-SNP  | CHR | BP | A1 | A2 | OR/Beta | SE | P 
------|-----|----|----|----|---------|----|
-snp1 | 1 | 100 | G  | T |1.05 | 0.03 | 0.03 | 0.10
-snp2 | 2 | 200 |  T  | A |0.95 | 0.033 | 0.12
+--meta-batch <arg>
+Specify the batch file that each line contains one meta file.
+It looks like:
+gwas1.txt
+gwas2.txt
+...

-The second row, which has ambiguous alleles will be eliminated. When "A1" and "A2" are both available, the program will automatically eliminate ambiguous loci, such as A/T and G/C loci.
+--qt-size <arg>
+Specify the file in which each line contains the sample size for the file at the corresponding row in meta-batch.
+It looks like:
+100
+200
+...
+
+--cc-size <arg>
+For case-control studies, each line has two elements, the number of cases and the number of controls for each corresponding file.
+It looks like
+200 300
+1000 800
+...
+
+--me <arg>
+Specifies the number of markers that should be sampled for calculating LambdaMeta or --fst.  By default it samples 30000 markers.
+
+--key <args>
+Although summary statistic files have all the columns required, their names may different. For example, "markerID" for "SNP", "effect" for "beta", "SE" for "SE", "Ref_Allele" for "A1", "Other_Allele" for "A2",  "Chromosome" for "CHR", "POS" for "BP", "Pval" for "P", "RAF" for "RAF", and "freq_se" for "RAF_SE". Then this option should be used as
+"--key markerID effect SE Ref_Allele Other_allele Chromosome POS Pval"
+Pval will be used to calculate genomic inflation factor for each cohort.
+
+--fst
+When --fst is switched on, it will calculate fst statistic between cohorts.
+Of note, when --fst is on, the column for allele frequency and s.e. for allele frequency should be specified in --key.  For example, the --key should be
+"--key markerID RAF RAF_SE Ref_Allele Other_Allele CHR POS Pval"
+
+--top <arg>
+This option tells the program only the top X files listed in --meta-batch will be compared to all files.  For example, if there are 10 summary statistic files included in --meta-batch, when "--top 1" is used, it only calculate lambdaMeta (of fst) between the first file and other files.
+In practice, if only want to calculate fst between the cohort to 1KG European samples, the user can put the summary statistic file for 1KG as the first file in --meta-batch and use "--top 1" option.
+

 Example

 ~~~~~~~
-gear lam --meta m1.txt m2.txt --cc 100 130 150 200
-gear lam --meta-gz m1.txt m2.txt --cc 100 130 150 200
+java -jar gear.jar lam --meta-batch metalist.txt --qt-size qt-sample-size.txt --key SNP BETA SE A1 A2 CHR BP P --out test
+java -jar gear.jar lam --meta-batch metalist.txt --qt-size cc-sample-size.txt --key SNP BETA SE A1 A2 CHR BP P --me 50000 --out test

+java -jar gear.jar lam --meta-batch metalist.txt --qt-size qt-sample-size.txt --key SNP RAF RAFSE A1 A2 CHR BP P --out test
+
+java -jar gear.jar lam --meta-batch metalist.txt --cc-size cc-sample-size.txt --out test
+java -jar gear.jar lam --meta-batch 1kg_OtherCohort.txt --qt-size qt-sample-size.txt  --key SNP RAF RAFSE A1 A2 CHR BP P --out test
 ~~~~~~~
-Notes: the numbers of cases and controls for m1 are 100 and 130, respectively; the numbers of cases and controls for m2 are 150 and 200, respectively.
-
-~~~~~~~
-gear lam --meta m1.txt  m2.txt --qt 100 200
-gear lam --meta-gz m1.gz  m2.gz --qt 100 200
-~~~~~~~
-Notes: the numbers of the sample size for m1 and m2 are 100 and 200, respectively.
-
-**Batch format**
-When there are many GWAS summary statistic files, it is much easier to write a batch file, which contains meta files.
-
---meta-batch <arg>
-Specify the batch file that each line contains one meta file.
-It should look like:
-gwas1.txt
-gwas2.txt
-...
-
-Or the files can be in gz format
---meta-gz-batch <arg>
-Specify the batch file that each line contains one meta file in gz format.
-It should look like:
-gwas1.gz
-gwas2.gz
-...
-
-And when the files are organized batch format, the sample size should be given in either --qt-batch for quantitative traits or --cc-batch for case-control studies.
-
---qt-size
-Specify the file that each line contains the sample size for the file at the corresponding row in the --meta-batch/--meta-gz-batch.
-For example:
-100
-200
-...
-
---cc-size
-Similar to --qt-batch, but each line has two elements, which are the number of cases and the number of controls for the corresponding files.
-For example:
-200 300
-1000 800
-...
-
---verbose/--verbose-gz
-Once verbose/verbose-gz option is switched on, the result for all selected SNPs will be saved in text/gz format in ".lam".  If this option is not switched on, only 100 fingerprint SNPs, whose lambdaD is at 1,2,3,..,100 percentile of the sequence, will be saved in text format in ".lam".

 Two result files will be found. One is ".lmat", which has the covariance and estimated overlapping samples; the other is ".lam", which saves the lambdaD for SNPs.

-Example
-
-~~~~~~~
-gear lam --meta-batch metalist.txt --cc-size cc-sample-size.txt --out test
-gear lam --mate-gz-batch metagzlist.txt --cc-size cc-sample-size.txt --out test
-
-gear lam --meta-batch metalist.txt --qt-size cc-sample-size.txt --out test
-gear lam --mate-gz-batch metagzlist.txt --qt-size cc-sample-size.txt --out test
-
-gear lam --meta-batch metalist.txt --qt-size cc-sample-size.txt --verbose --out test
-gear lam --mate-gz-batch metagzlist.txt --qt-size cc-sample-size.txt --verbose-gz --out test
-~~~~~~~
+When "--fst" option is on, the ".fst" file will be found.

LambdaD modified by Guobo Chen

Guobo Chen — Mon, 12 Jan 2015 02:11:51 -0000

--- v10
+++ v11
@@ -56,14 +56,14 @@

 And when the files are organized batch format, the sample size should be given in either --qt-batch for quantitative traits or --cc-batch for case-control studies.

---qt-batch
+--qt-size
 Specify the file that each line contains the sample size for the file at the corresponding row in the --meta-batch/--meta-gz-batch.
 For example:
 100
 200
 ...

---cc-batch
+--cc-size
 Similar to --qt-batch, but each line has two elements, which are the number of cases and the number of controls for the corresponding files.
 For example:
 200 300
@@ -78,12 +78,12 @@
 Example

 ~~~~~~~
-gear lam --meta-batch metalist.txt --cc-batch cc-sample-size.txt --out test
-gear lam --mate-gz-batch metagzlist.txt --cc-batch cc-sample-size.txt --out test
+gear lam --meta-batch metalist.txt --cc-size cc-sample-size.txt --out test
+gear lam --mate-gz-batch metagzlist.txt --cc-size cc-sample-size.txt --out test

-gear lam --meta-batch metalist.txt --qt-batch cc-sample-size.txt --out test
-gear lam --mate-gz-batch metagzlist.txt --qt-batch cc-sample-size.txt --out test
+gear lam --meta-batch metalist.txt --qt-size cc-sample-size.txt --out test
+gear lam --mate-gz-batch metagzlist.txt --qt-size cc-sample-size.txt --out test

-gear lam --meta-batch metalist.txt --qt-batch cc-sample-size.txt --verbose --out test
-gear lam --mate-gz-batch metagzlist.txt --qt-batch cc-sample-size.txt --verbose-gz --out test
+gear lam --meta-batch metalist.txt --qt-size cc-sample-size.txt --verbose --out test
+gear lam --mate-gz-batch metagzlist.txt --qt-size cc-sample-size.txt --verbose-gz --out test
 ~~~~~~~

LambdaD modified by Guobo Chen

Guobo Chen — Wed, 05 Nov 2014 22:10:48 -0000

--- v9
+++ v10
@@ -12,15 +12,14 @@
 --qt
 Specify the sample size for each meta file which are for quantitative traits.

-The meta-data should have these columns: "SNP", "CHR", "BP", "A1", "A2", "OR" (or "beta" for quantitative traits), "SE" and "P". The keywords are case-insensitive, and there is no requirement for the order of these columns. "A1" is the reference allele, and "A2" is the other allele.  If "A2" column is absent, the program will not eliminate ambiguous loci, such as A/T and G/C loci.
+The meta-data should have these columns: "SNP", "OR" (or "beta" for quantitative traits), "SE", "A1", "A2", "CHR", "BP", and "P". The last three key words are optional. The keywords are case-insensitive, and there is no requirement for the order of these columns. "A1" is the reference allele, and "A2" is the other allele.  If "A2" column is absent, the program will not eliminate ambiguous loci, such as A/T and G/C loci.

 SNP  | CHR | BP | A1 | A2 | OR/Beta | SE | P 
 -----|-----|----|----|----|---------|----|
 snp1 | 1 | 100 | G  | T |1.05 | 0.03 | 0.03 | 0.10
 snp2 | 2 | 200 |  T  | A |0.95 | 0.033 | 0.12

-The second row, which has ambiguous alleles will be eliminated.
-"CHR", "BP", and "A2" are optional.  When "A1" and "A2" are both available, the program will automatically eliminate ambiguous loci, such as A/T and G/C loci.
+The second row, which has ambiguous alleles will be eliminated. When "A1" and "A2" are both available, the program will automatically eliminate ambiguous loci, such as A/T and G/C loci.

 Example

LambdaD modified by Guobo Chen

Guobo Chen — Wed, 05 Nov 2014 01:32:53 -0000

LambdaD modified by Guobo Chen

Guobo Chen — Fri, 04 Jul 2014 03:47:44 -0000

--- v7
+++ v8
@@ -88,6 +88,3 @@
 gear lam --meta-batch metalist.txt --qt-batch cc-sample-size.txt --verbose --out test
 gear lam --mate-gz-batch metagzlist.txt --qt-batch cc-sample-size.txt --verbose-gz --out test
 ~~~~~~~
-
-
-![BD](https://sourceforge.net/p/gbchen/wiki/LambdaD/attachment/BC_CDlam.png)