Menu
▾
▴
Re: [GeneX-dev] CyberT bug
|
From: Michael P. <mic...@ho...> - 2001-02-02 21:00:36
|
Harry, I notice that the "lnp" column doesn't seem to be the natural logarithm of the "p"... When I plot lnp vs. p in xgobi, I get a scatter plot rather than a clear functional relationship and even when p<1, displayed values for lnp are positive. Am I missing something? Regards, Michael Pear ----- Original Message ----- From: "Harry Mangalam" <man...@ho...> To: <td...@ho...> Cc: "genexdev at SF" <gen...@li...> Sent: Friday, February 02, 2001 11:28 AM Subject: [GeneX-dev] CyberT bug > Hi Tony, > > I think there is a bug in the CyberT interface (or R code). Michael Pear, who has been stress testing the whole of GeneX is generating C+E data that he has loaded into his GeneX database and trying > to analyze it via the CyberT C+E tool. > > He found that > ===> > 2) There appears to be a "minrep" level set to 2 when calling "doitall" that > is to prevent t-testing on too small a number of replicates. However, it seems that there is still a "p" value and fold computed for spots that violate this minimum requirement...am I missing > something? We are finding the some of our best "p" values are for spots that do not satisfy the "minrep" requirement. > <=== > > Have I gotten this right below? And if so, there is a mismatch between what I understand to be happening and what the doitall script is doing. Is my suggestion below the correct approach or is there > a deeper bit that I'm missing? > > I've partially tracked it down to bad/misleading documentation at minimum. > > The values that he's trying to analyze have negative #s in them. He currently has the option of indicating that they are either below backgound (in which case they are set to 0) > > That's what this means in the C+E form: > == > Values less than [ ] will be set to 0 and ignored in calculations. (Leave blank to include all values). > == > It turns out that it's actually NOT ignored in calculations. It's USED in the calculations AS ZERO, as ZERO is a valid number for this calculation. I think the interface should allow you to set the > number to the cutoff (or even another number) or to set it to 'NA' (which indicates a true missing value - see below) > > However, you CAN tell the program to IGNORE the values by setting negative values to 'NA' (as is described in the help, but that's off the main page). That results in what you would expect for the > above case. This is also reasonably easy to fix, so that the above FORM widget should be rephrased as: > > == > Leave the following value blank to include all values. > Values less than [___] will be set to > <> that value and included in calculations as that vlaue > <> NA and ignored in further calculations. > > where <> is a radio-type button. > == > This should also be available from both the DB data entry and the upload data entry. > > > > Thanks > hjm > > > Harry Mangalam wrote: > > > > Hi Michael, > > > > Is this from data passed in directly from a database query? or did you upload it from a file. The difference is that if it comes in from the DB, some variables are already set (the minrep param is > > one of them I think). > > > > However the fact that minrep is getting set to 2 and that's getting passed to doitall is weird. I'm checking now.. > > > > And attached to this is the replacement 'dendro.jar' file that may fix your applet problem. (actually this has been deleted b/c of the size restriction on the list - I sent it and then I as admin got > > the request to approve it... proving the point that idiots rule!) > > > > hjm > > > > Michael Pear wrote: > > > > > > > > 2) There appears to be a "minrep" level set to 2 when calling "doitall" that is to prevent > > > t-testing on too > > > > > small a number of replicates. However, it seems that there is still a "p" value and fold > > > computed for spots > > > > > that violate this minimum requirement...am I missing something? We are finding the some of our > > > best > > > > > "p" values are for spots that do not satisfy the "minrep" requirement. > > > > > > > > Hmm - that shouldn't be the case - it should refuse to calculate these. No wonder that they're > > > the best p's are with these - they'd tend to have less variability. This is with the C+E cybert? > > > > > > Yes, this is with C+E cybert. I have attached a snippet from the output of one of the runs. > > > See, for example, the line "125", which gives a p value of ~.08, but has only one replicate from the > > > control set. > > > > > > Michael Pear > > > > > > > -- > Cheers, > Harry > > Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || man...@ho... -------------------------------------------------------------------------------- > R version R 1.2.1 (2001-01-15). > Cyber-T version 6.2C+E > hdarray library version 3.5 > Date of this analysis (YYYY-MM-DD) was 2001-02-01 > > Row# "Lab_0" "C_1" "C_2" "C_3" "E_4" "E_5" "E_6" "C#obs" "E#obs" "C.Mn" "E.Mn" "C.SD" "E.SD" "ClnMn" "ElnMn" "ClnSD" "ElnSD" "t" "t.df" "t.var" "lnt" "lnt.df" "lnt.var" "p" "lnp" "fold" > "1" "E06a" 5576 760 7256 2960 6640 208 3 3 4530.66667 3269.3333 3371.801497 3227.138258 8.049710 7.377117 1.23367848 1.811931358 -0.468087469 4 1.091664e+00 -0.531451529 4 2.157146e+00 0.6640889197 0.6232484745 -1.385808 > "2" "A06l" 1356 1112 4660 2336 1332 1460 3 3 2376.00000 1709.3333 1981.760833 546.469883 7.557660 7.412275 0.77635473 0.301356647 -0.561699930 4 1.315134e+01 -0.302373474 4 6.636802e+00 0.6043010773 0.7774383745 -1.390016 > "3" "A09l" 2324 3448 7748 6672 2652 3428 3 3 4506.66667 4250.6667 2862.782795 2132.530266 8.283928 8.276159 0.61388331 0.476192638 -0.124211301 4 1.802131e+00 -0.017319955 4 1.661905e+00 0.9071397521 0.9870108454 -1.060226 > "4" "A03b" 5208 2520 7256 4428 8592 4780 3 3 4994.66667 5933.3333 2375.196273 2309.189757 8.426516 8.642162 0.54089721 0.362656878 0.490784928 4 1.057986e+00 0.573552423 4 2.224527e+00 0.6492853349 0.5969769219 1.187934 > ....deleted by mrp.... from 5 to 117........ > "118" "F03d" 2160 1000 3028 1272 1396 392 3 3 2062.66667 1020.0000 1017.497584 547.386518 7.533759 6.786991 0.56783494 0.707971826 -1.563062449 4 3.455243e+00 -1.425188057 4 1.554489e+00 0.1930757537 0.2272320109 -2.022222 > "119" "F10h" 1632 528 2760 1332 1044 0 3 2 1640.00000 1188.0000 1116.021505 203.646753 7.196548 7.072626 0.84506959 0.172266827 -0.538910534 3 3.003241e+01 -0.194727236 3 2.406477e+01 0.6273715995 0.8580476335 -1.380471 > "120" "F04h" 5392 3824 2812 1384 11768 7740 3 3 4009.33333 6964.0000 1299.946666 5235.312407 8.261125 8.520010 0.32567807 1.134326860 0.948713071 4 1.621938e+01 0.379952063 4 1.213108e+01 0.3964880135 0.7232937678 1.736947 > "121" "E09c" 148 0 216 500 0 0 2 1 182.00000 500.0000 48.083261 NA 5.186245 6.214608 0.26733313 NA 9.352941200 1 NA 5.440122224 1 NA 0.0678086626 0.1157311046 2.747253 > "122" "A01a" 268 608 1184 1260 1652 2436 3 3 686.66667 1782.6667 463.039235 598.789891 6.359272 7.448907 0.74414032 0.331363300 2.507904893 4 1.672297e+00 2.316893154 4 5.043138e+00 0.0662052301 0.0814137514 2.596117 > "123" "E08b" 140 0 0 0 0 0 1 0 140.00000 NA NA NA 4.941642 NA NA NA NA NA NA NA NA NA NA NA -4.555358 > "124" "C13d" 720 0 328 336 2132 2524 2 3 524.00000 1664.0000 277.185858 1166.663619 6.186132 7.105176 0.55595394 1.118684399 1.292860150 3 1.771533e+01 1.039873073 3 4.048905e+00 0.2866235380 0.3748412897 3.175573 > "125" "C05c" 0 0 140 648 1764 1928 1 3 140.00000 1446.6667 NA 696.509392 4.941642 7.171156 NA 0.605483351 3.249364817 2 NA 6.377766652 2 NA 0.0830792373 0.0237135922 10.333333 > "126" "C06c" 64 0 192 712 1796 2016 2 3 128.00000 1508.0000 90.509668 698.077360 4.708189 7.223422 0.77683620 0.570477855 2.641157058 3 5.948633e+01 4.261059559 3 1.854306e+00 0.0775789836 0.0237069214 11.781250 > "127" "C12d" 0 0 148 0 1600 2588 1 2 148.00000 2094.0000 NA 698.621500 4.997212 7.618200 NA 0.340034745 3.939271254 1 NA 10.900758910 1 NA 0.1582654014 0.0582384116 14.148649 > "128" "C03d" 1320 88 252 1020 2176 1664 3 3 553.33333 1620.0000 668.997260 579.254694 5.730718 7.343260 1.36520027 0.384184592 2.087769692 4 1.333858e+00 1.969362771 4 1.262737e+01 0.1050953318 0.1202572649 2.927711 > "129" "C01c" 1304 40 196 636 2404 2024 3 3 513.33333 1688.0000 689.165679 930.659981 5.380062 7.284306 1.74439186 0.723163700 1.756905339 4 1.823622e+00 1.746628529 4 5.818551e+00 0.1537712858 0.1556279159 3.288312 > "130" "E09d" 448 0 480 400 0 0 2 1 464.00000 400.0000 22.627417 NA 6.139290 5.991465 0.04878533 NA -4.000000000 1 NA -4.285225084 1 NA 0.1559582608 0.1459496634 -1.160000 > "131" "A04n" 208 0 0 956 0 1092 1 2 208.00000 1024.0000 NA 96.166522 5.337538 6.929262 NA 0.094051031 12.000000030 1 NA 23.934215759 1 NA 0.0529293520 0.0265832699 4.923077 > "132" "C03c" 328 176 1144 1452 1724 1996 3 3 549.33333 1724.0000 520.574042 272.000000 6.001928 7.444000 0.95322859 0.159267948 3.463996747 4 3.662918e+00 2.584470449 4 3.582097e+01 0.0257238787 0.0610387862 3.138350 > "133" "C09c" 196 48 0 0 1500 1536 2 2 122.00000 1518.0000 104.651804 25.455844 4.574658 7.325079 0.99483818 0.016770117 18.330377969 2 1.690123e+01 3.909309297 2 3.519109e+03 0.0029629453 0.0596397842 12.442623 > "134" "C08c" 80 0 336 308 1992 2512 2 3 208.00000 1604.0000 181.019336 1152.090274 5.099569 7.051943 1.01475800 1.150608970 1.615738817 3 4.050635e+01 1.931682793 3 1.285673e+00 0.2045667699 0.1489203548 7.711538 > "135" "E03a" 1176 128 2088 1264 0 0 3 1 1130.66667 1264.0000 980.786079 NA 6.521955 7.142037 1.47440896 NA 0.235464294 2 NA 0.728436654 2 NA 0.8357625114 0.5420918997 1.117925 > "136" "E04a" 516 104 1528 1468 0 0 3 1 716.00000 1468.0000 732.764628 NA 6.074071 7.291656 1.35189679 NA 1.777517852 2 NA 1.559970479 2 NA 0.2174587699 0.2591289297 2.050279 > "137" "E05a" 168 0 432 132 0 356 2 2 300.00000 244.0000 186.676190 158.391919 5.596195 5.378866 0.66783521 0.701541008 -0.323488724 2 1.389031e+00 -0.317317028 2 1.103488e+00 0.7770180413 0.7810664174 -1.229508 > "138" "E02a" 464 0 380 268 0 0 2 1 422.00000 268.0000 59.396970 NA 6.040028 5.590987 0.14121863 NA -3.666666643 1 NA -4.496856203 1 NA 0.1695013200 0.1393032210 -1.574627 > "139" "E07a" 96 0 580 836 0 0 2 1 338.00000 836.0000 342.239682 NA 5.463688 6.728629 1.27185876 NA 2.057851240 1 NA 1.406521521 1 NA 0.2879682878 0.3934647906 2.473373 > "140" "E08a" 460 176 824 24 160 0 3 2 486.66667 92.0000 324.822003 96.166522 6.005294 4.126614 0.77951032 1.341466406 -1.595538586 3 1.140888e+01 -2.052931524 3 2.961528e+00 0.2088620248 0.1323907921 -5.289855 > "141" "E10d" 0 0 120 68 0 0 1 1 120.00000 68.0000 NA NA 4.787492 4.219508 NA NA NA NA NA NA NA NA NA NA -1.764706 > "142" "E09a" 8 0 172 200 0 0 2 1 90.00000 200.0000 115.965512 NA 3.613468 5.298317 2.16944104 NA 1.341463416 1 NA 1.098318075 1 NA 0.4078095000 0.4701925623 2.222222 > "143" "E10a" 208 0 44 216 0 0 2 1 126.00000 216.0000 115.965512 NA 4.560864 5.375278 1.09838322 NA 1.097560977 1 NA 1.048591515 1 NA 0.4704111023 0.4849025444 1.714286 > "144" "E01c" 124 0 392 412 0 0 2 1 258.00000 412.0000 189.504617 NA 5.395772 6.021023 0.81386596 NA 1.149253734 1 NA 1.086466922 1 NA 0.4558611556 0.4736324257 1.596899 > "145" "A02a" 272 0 588 612 1644 2676 2 3 430.00000 1644.0000 223.445743 1032.000000 5.991265 7.237899 0.54512621 0.751714932 1.560069984 3 2.133120e+01 1.979842155 3 1.901570e+00 0.2166496378 0.1420792749 3.823256 > "146" "A10n" 596 128 2984 896 516 172 3 3 1236.00000 528.0000 1531.791108 362.149141 6.414430 6.063847 1.57463421 0.840182631 -0.779083306 4 1.789057e+01 -0.340228643 4 3.512465e+00 0.4794492261 0.7508008055 -2.340909 > "147" "A01m" 440 112 428 908 0 288 3 2 326.66667 598.0000 186.003584 438.406204 5.621466 6.237102 0.78211434 0.811959331 1.006945660 3 5.555341e+00 0.851307744 3 1.077775e+00 0.3881401740 0.4571561542 1.830612 > "148" "A14a" 0 0 80 0 1296 2136 1 2 80.00000 1716.0000 NA 593.969696 4.382027 7.416864 NA 0.353307546 3.895238097 1 NA 12.147795012 1 NA 0.1599805033 0.0522883020 21.450000 > "149" "C11a" 108 0 0 840 1500 2436 1 3 108.00000 1592.0000 NA 801.967580 4.682131 7.281578 NA 0.533060179 3.205071455 2 NA 8.446277649 2 NA 0.0851044902 0.0137294462 14.740741 > "150" "C12b" 1356 0 3588 3804 2152 2528 2 3 2472.00000 2828.0000 1578.262336 865.896068 7.698822 7.917715 0.68805432 0.293658657 0.338134551 3 3.322208e+00 0.516777969 3 5.489842e+00 0.7575419629 0.6409934591 1.144013 > "151" "C06b" 800 112 1216 1172 2004 1952 3 3 709.33333 1709.3333 557.556574 466.070095 6.168811 7.415326 1.27333550 0.302406395 2.383451771 4 1.431118e+00 1.649683517 4 1.772980e+01 0.0757045623 0.1743526276 2.409774 > "152" "C09b" 240 0 1032 948 1804 2376 2 3 636.00000 1709.3333 560.028571 718.691403 6.209946 7.375097 1.03139657 0.471531771 1.754913839 3 1.646890e+00 1.799973401 3 4.784422e+00 0.1775434070 0.1696844484 2.687631 > "153" "C14b" 612 64 840 12 2316 2020 3 3 505.33333 1449.3333 398.844999 1253.534736 5.769672 5.947785 1.40394146 2.999720287 1.242956724 4 9.877896e+00 0.093146241 4 4.565239e+00 0.2817652114 0.9302663066 2.868074 > "154" "C11b" 552 0 932 1556 2304 2404 2 3 742.00000 2088.0000 268.700577 463.430685 6.575440 7.625722 0.37037176 0.239834044 3.605443372 3 2.974626e+00 3.967992796 3 2.384812e+00 0.0366211721 0.0286030347 2.814016 > "155" "C06e" 524 352 548 1564 1220 1712 3 3 474.66667 1498.6667 106.908060 252.422926 6.143799 7.302342 0.24366383 0.175436897 6.470021676 4 5.574895e+00 6.683277859 4 1.929036e+00 0.0029399907 0.0026061603 3.157303 > "156" "B09n" 56 0 280 332 332 340 2 3 168.00000 334.6667 158.391919 4.618802 4.830071 5.813072 1.13804446 0.013747084 1.994794132 3 1.176000e+03 1.638636483 3 6.853261e+03 0.1400309567 0.1998172900 1.992063 > "157" "C02b" 28 0 184 388 1376 1944 2 3 106.00000 1236.0000 110.308658 787.390627 4.273570 6.920148 1.33129203 0.848422229 1.916037228 3 5.095201e+01 2.801880431 3 2.462196e+00 0.1512253995 0.0677471289 11.660377 > "158" "E13e" 1296 0 504 292 240 0 2 2 900.00000 266.0000 560.028571 36.769553 6.694807 5.578696 0.66783521 0.138674161 -1.597570412 2 2.319763e+02 -2.314123224 2 2.319250e+01 0.2512310064 0.1467226203 -3.383459 > "159" "B06n" 52 0 528 140 652 492 2 3 290.00000 428.0000 336.582828 261.931289 5.110170 5.873389 1.63896927 0.819104993 0.523146114 3 1.651236e+00 0.721524435 3 4.003709e+00 0.6370542121 0.5227365272 1.475862 > "160" "C12a" 36 0 2144 1440 1952 2320 2 3 1090.00000 1904.0000 1490.581095 441.959274 5.626974 7.532777 2.88988148 0.241464562 0.955539189 3 1.137488e+01 1.242619558 3 1.432364e+02 0.4097982329 0.3022846530 1.746789 > "161" "A10m" 1888 1952 4828 3136 2084 1764 3 3 2889.33333 2328.0000 1679.239510 717.807774 7.867357 7.722696 0.53271982 0.296039601 -0.532387023 4 5.472792e+00 -0.411124812 4 3.238158e+00 0.6226570518 0.7020532942 -1.241123 > "162" "A11m" 892 1552 2988 2356 528 1412 3 3 1810.66667 1432.0000 1071.674080 914.164099 7.381042 7.095526 0.60515264 0.760108777 -0.465615070 4 1.374286e+00 -0.508991835 4 1.577690e+00 0.6657128830 0.6375498929 -1.264432 > "163" "A11l" 124 200 836 900 336 636 3 3 386.66667 624.0000 390.985081 282.191424 5.615743 6.358235 0.99298346 0.499747355 0.852524044 4 1.919697e+00 1.156870407 4 3.948053e+00 0.4419610495 0.3117081595 1.613793 > "164" "A12l" 92 80 1096 664 772 748 3 3 422.66667 728.0000 583.154639 56.709788 5.301079 6.588223 1.47246755 0.079475763 0.902624708 4 1.057429e+02 1.511855717 4 3.432591e+02 0.4177655924 0.2051069822 1.722397 > "165" "A12m" 688 552 2488 1148 680 1260 3 3 1242.66667 1029.3333 1080.631914 307.670820 6.888857 6.902245 0.81322076 0.332495850 -0.328864005 4 1.233624e+01 0.026393745 4 5.981973e+00 0.7587560326 0.9802075636 -1.207254 > "166" "A06k" 36 448 932 780 0 532 3 2 472.00000 656.0000 448.481884 175.362482 5.508548 6.467969 1.70688523 0.270574714 0.530533839 3 6.540583e+00 0.749427909 3 3.979552e+01 0.6325043242 0.5080128786 1.389831 > "167" "A12j" 652 208 708 960 948 1160 3 3 522.66667 1022.6667 273.944033 119.085404 6.126676 6.925821 0.68465386 0.113065164 2.899234362 4 5.291839e+00 1.994676390 4 3.666777e+01 0.0441518340 0.1168246949 1.956633 > "168" "A05n" 3032 1776 3468 2548 2700 1888 3 3 2758.66667 2378.6667 878.492648 431.672716 7.883477 7.762448 0.35401820 0.192009221 -0.672420407 4 4.141590e+00 -0.520510424 4 3.399437e+00 0.5381608754 0.6301908844 -1.159753 > "169" "A07n" 260 0 72 592 0 200 2 2 166.00000 396.0000 132.936075 > |
