genex-dev — GeneX developers

[GeneX-dev] Re: xcluster bug

[Harry M. <man...@ho...>]

Hi there,

Thanks for passing this info on to us - if xcluster does die with that message, did you look at the run-time messages from xcluster to see if it indicated anything amiss?

This is problem related to Gavin's code that he indicated that he was working on fixing (hence the trapping of that error).  It may or may not have interfered with the generation of the results, but
if you want to send me the data and parameters that generated the bug, I'll test it myself and get back to you about it.

I don't think anything changed with cluster from the version you installed (1.0.0pre7)to the latest one (1.0.2) but there are some other things that have improved, notably  the CyberT interface,
library and scripts.  It should be a pretty easy upgrade (the install script should detect a pre-existing installation and smoothly upgrade over it).

Cheers
Harry


"Miller.Jonathan" wrote:
> 
> I've been working on installing a GeneX server on a Solaris
> workstation here at the NIEHS to see if it's something we could use.
> When checking xcluster, though, I find that it dies with a system
> error:
>         !! System Error: @ line: 275 (Warning!! xcluster exited
> abnormally!)
> and a suggestion that the cause is a known bug with xcluster.  I can't
> quite tell, though, if the bug is something Gavin Sherlock is working
> on or if it's NCGR people.  Do you know the status of this bug?  It
> would be useful for us to have GeneX fully working to decide whether
> to continue using it.
> 
> If you need to see how our server is configured, the page is at:
> http://dukie.niehs.nih.gov/cgi-bin/genex/samples/status.pl
> 
> Thank you for your time,
> 
> Jonathan Miller
> Bioinformatics System Administrator
> NIEHS MicroArray Center (ITSS)

-- 
Cheers,
Harry

Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || man...@ho...

[GeneX-dev] GeneX version 1.0.2 available

[Harry M. <man...@ho...>]

..at:

http://genex.ncgr.org/genex/download/genex-server/GeneX-Server-1.0.2.tar.gz

along with Version 1.0.1 of the Curation Tool that supports intermediate saves.

http://genex.ncgr.org/genex/download/curation_tool/ [platform]

So please give 'em a try to see what's changed - mostly changes to the CyberT forms, analysis, and output.  

genex.ncgr.org (2x1GHz PIIs, 2GB RAM, no waiting) has been upgraded  (2x1GHz PIIs, 2GB RAM, no waiting) to version 1.0.2 as well.  It was a completely automatic update, so anyone wanting to try it
should have no problem in upgrading either.

hjm

Release Notes for Release 1.0.2
-------------------------------
This is the 3rd public release, more bugfixes, some corrections in the CyberT R
library and associated forms and scripts.  Most new development is going on in 
GeneX version 2 branch, but things that get developed for it that can be
migrated  backwards are being migrated.  Analyses are largely applicable to both
versions so they'll be added as they're integrated.

See ChangeLog for the full list of things that have changed, but briefly:
  * CyberT Paired analysis changed quite a bit to allow level of expression to be
    used as input for calculation of Bayesian estimate of variance (was using 
    magnitude of ratio, which was incorrect).
  * CyberT now allows setting of decimal points in output using a general-
    purpose filter.  This user setting will be generally applied in later 
    versions.
  * Other format and output changes in CyberT to address user feedback (output 
    sorted on 'p' values, rather than listing in input order, etc).
  * some minor changes in the installation script to address unclear instructions.
  * lots of changes in the database, but none that will affect this version of the
    server.   

Release Notes for Release 1.0.1
-------------------------------
This is second public release, mainly a more stable bugfix edition and more 
features for developers.
We have had a number of developer releases - thanks to our testers.

Changes for 1.0.1
-----------------
* many patches and fixes, mostly having to do with broken links and installation
        glitches (see Changelog for the full list).
* CyberT changes:
        - hdarray R lib updated to reflect increased confidence in Bayesian stats
   - form & analytical scripts merged to make further changes easier (now 1 
        form and 1 analytical script; Paired & C+E analyses now coherent in tems
      of features.
   - output format changed to present all results sorted on raw p value
        and moved data 1 tab/column right so it doesn't conflict with header info


-- 
Cheers,
Harry

Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || man...@ho...

Re: [GeneX-dev] Heads-up installation in places other than genex

[Harry M. <man...@ho...>]

Sorry Jason,

You're right, but when you tell the script that you want to install in a differnet location, it doesn't propagate the new LOCAL_ROOT to the rest of the vars as it did inthe 1st place - you have to
specifically tell it that you want to either place it  in a different location or tell it to manually refresh the lib location.  

Not incorrect, just different behavior than I expected... too used to just hitting 'RETURN'  I'm not sure it's worth changing it at this point..

hjm



"Jason E. Stewart" wrote:
> 
> "Harry Mangalam" <man...@ho...> writes:
> 
> > Not needed to be corrected for this go-round, but soon - and in fact
> > I might try to figger it out as well or lock it in the install-all
> > script..
> >
> > If you try to install the genex apps in any place other than
> > /usr/local/genex, there are some places where this is hard-coded:
> 
> I believe this is incorrect. /usr/local is not hardcode in any of the
> perl scripts. If you check query_intro.pl.in you will see that it is
> %%GENEX_EXTRALIBS%%, so this means that your EXTRALIBS is messed
> up.
> 
> Include your Config.pm and I can figure out what happened.
> 
> jas.
> 
> _______________________________________________
> Genex-dev mailing list
> Gen...@li...
> http://lists.sourceforge.net/lists/listinfo/genex-dev

-- 
Cheers,
Harry

Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || man...@ho...

[GeneX-dev] API changes

[<ja...@op...>]

Hey All,

I just checked in some important changes to the API. 

First there are now only 5 tests which don't run completely, and all
of these except XMLUtils.t are due to the lack of ArrayMeasurement and
ExperimentSet data in the DB:

Failed Test  Status Wstat Total Fail  Failed  List of failed
-------------------------------------------------------------------------------
t/Citation.t    255 65280     5    3  60.00%  3-5
t/DBUtils.t     255 65280    21    7  33.33%  9-12, 15, 19-20
t/Provider.t    255 65280     2    1  50.00%  2
t/Sample.t      255 65280     5    3  60.00%  3-5
t/XMLUtils.t    255 65280     4    3  75.00%  2-4

The XMLUtils.t breakage is a troubling difference between XML::DOM and
XML::Xerces that I haven't fixed yet.

Second, linking tables are now handled properly by the API. I had
special cased them in many different situations, and it made for a
horrible spaghetti mess of code. Once I realized that linking tables
are just like any other table with primary keys, a solution became
obvious. I had to fully support tables without primary keys. 

This is now handled through the judicious use of UNIQUE constraints in
the DB. Now, in the lowest level of the API when a primary key is
needed to lookup information in the DB, unique columns can be used
instead. 

This has two immediate useful effects:
* insert_db() is now available to *all* classes
* insert_db() can now check to see if an entry is already present
  in the DB before the insert.

This means that all code for inserting data into the DB can call
insert_db() on any value without fear of inserting multiple copies. I
make great use of this in the scripts that bootstrap the initial DB
data. I can re-run the bootstrap code over and over and only one copy
of each entry is added to the DB.

The only table without UNIQUE constraints, GenexAdmin, does not have
this behavior. 

jas.

Re: [GeneX-dev] Heads-up installation in places other than genex

[<ja...@op...>]

"Harry Mangalam" <man...@ho...> writes:

> Not needed to be corrected for this go-round, but soon - and in fact
> I might try to figger it out as well or lock it in the install-all
> script..
> 
> If you try to install the genex apps in any place other than
> /usr/local/genex, there are some places where this is hard-coded:

I believe this is incorrect. /usr/local is not hardcode in any of the
perl scripts. If you check query_intro.pl.in you will see that it is
%%GENEX_EXTRALIBS%%, so this means that your EXTRALIBS is messed
up. 

Include your Config.pm and I can figure out what happened.

jas.

[GeneX-dev] Heads-up installation in places other than genex

[Harry M. <man...@ho...>]

Not needed to be corrected for this go-round, but soon - and in fact I might try to figger it out as well or lock it in the install-all script..

If you try to install the genex apps in any place other than /usr/local/genex, there are some places where this is hard-coded:

http://cx408397-b.irvn1.occa.home.com/cgi-bin/genex/gxquery/query_intro.pl

gives:
Software error:

Can't locate Bio/Genex.pm in @INC (@INC contains: /usr/local/genex/perl5 /usr/local/genex/perl5 /usr/lib/perl5/5.00503/i386-linux /usr/lib/perl5/5.00503 /usr/lib/perl5/site_perl/5.005/i386-linux
/usr/lib/perl5/site_perl/5.005 .) at /home/httpd/cgi-bin/genex/gxquery/query_intro.pl line 23.
BEGIN failed--compilation aborted at /home/httpd/cgi-bin/genex/gxquery/query_intro.pl line 23.


-- 
Cheers,
Harry

Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || man...@ho...

[GeneX-dev] Re: Cyber-T data problems

[Harry M. <man...@ho...>]

Hi Jeanette,

Thanks for the feedback - I LIKE hassles - it's the only way I know that =
people are using it...

Please let me know of likes/dislikes and what you'd like to see it suppor=
t.  NO guarantees, but if you don't speak up, it certainly won't happen.

Again, thanks for taking the time to complain.

Harry




Jeanette Nance Mcclintick wrote:
> =

> Harry,
> =

> The # was not my problem. The previous version set all negative numbers=
 to
> zero by default. This newer version does not. When I used to option to =
set
> all values less than 0 to 0, it worked. Sorry for the hassles.
> =

> Jeanette
> =

> On Fri, 27 Apr 2001, Harry Mangalam wrote:
> =

> > Hi Jeannette,
> >
> > > I'll give it a try, probably Monday. Previously, CyberT ignored any=
thing
> > > that had a blank in the first column.  I thought that was why there=
 was a
> > > check box with  ' Delete lines with NULL Labels'. (But then again, =
I have
> > > lot's of label columns!! not just the 1st one.) So I will make sure=
 to use
> > > the '#'.
> >
> >
> > Hmm - you're right - that would result in them ignoring them - that w=
asn't the point of adding that feature, but it would work.  However, usin=
g '#'s in the 1st column is generally better....
> >
> > > It may be helpful to add that error message to your FAQ, for idiots=
 like
> > > me that ignore the documentation!
> >
> > :)
> >
> >
> > > I designated all of those columns as labels because I want to see t=
hem in
> > > the output. I have a perl script that puts the columns in the order=
 that
> > > I want to see them. Thanks for the info though. If this is a burden=
 to
> > > your system, I could leave them out and put them back in after the =
CyberT
> > > runs. I do trim the data so that I don't send genes that are mostly=

> > > absent, because it's a waste of cpu and memory.
> >
> > I'm pretty sure it won;t overload CyberT - the labels are just passed=
 thru adn I thought they would just get in the way as they're all identic=
al (P).  However, if they do you some good (or removing
> > them requires more work on your end), by all means leave them in.
> >
> > > When I have more time, I
> > > would like to install CyberT locally.
> >
> > Sure thing - I've left the local install instructions languish a bit =
while we've been installing it as part of GeneX; I'll have to re-work it =
to bring things up to snuff.
> >
> > Harry
> >
> > >
> > > Have a good weekend,
> > > Jeanette
> > >
> > > On Fri, 27 Apr 2001, Harry Mangalam wrote:
> > >
> > > > Jeanette McClintick wrote:
> > > >
> > > > > I used the following URL:
> > > > >
> > > > > http://genomics.biochem.uci.edu/genex/cybert/
> > > > >
> > > > > to get to:
> > > > >
> > > > > http://genomics.biochem.uci.edu/cgi-bin/genex/cybert/CyberT-7.0=
=2Eform.p
> > > > > l?DATATYPE=3DCE
> > > >
> > > > OK - that's the right one.
> > > >
> > > > > I have included the tab-delimited file that I used (Affymetrix =
data).
> > > > >
> > > > > I used columns 1 3 5 7 as control, 9 11 13 15 as experiment
> > > > > la bels: 0 2 4 6 8 10 12 14 16 17
> > > >
> > > > Why did you use these columns for labels?  '0' I can see using, b=
ut the rest are just 'P' and that many label columns would just mess thin=
gs up visually.  You don;t have to tell CyberT to consider
> > > > them labels so it will avoid them - only tell it the columns that=
 you want included in the output.
> > > >
> > > > the reason the file didn;t work is that you didn;t prefix the top=
 2 lines with:
> > > > #
> > > > which tells CyberT to consider them comments, hence it thought th=
ey were data and tried to treat them as such, so everything bombed.
> > > >
> > > > I'm sending you the data file back with the comment marks in plac=
e (with DOS end-of-lines converted to Unix end-of-lines - hope this doesn=
't mess things up for you - either way Cybert can digest them
> > > > fine.  this file worked fine for me on my system..
> > > >
> > > > I'm having some problems with my net today so I can't actually co=
nnect to genomics to verify that this will work there, but I'm pretty sur=
e my version and the version there are in sync.
> > > >
> > > > Try it and let me know if it works..
> > > >
> > > > harry
> > > >
> > > >
> > > >
> > > > >
> > > > > I didn't use the Bayesian function.
> > > > > ---------------------------------------------------------------=
-------
> > > > > ------------------------------------------------------
> > > > > I had previously used: http://128.200.5.223/cybert/  but I coul=
dn't
> > > > > get that to work today. At this site, the analysis ran, but whe=
n I
> > > > > tried to retrieve the output, I got a message saying that it wa=
sn't
> > > > > found on the server.  ERROR:
> > > > >
> > > > > Not Found
> > > > >
> > > > > The requested URL /tmp/hda28880988310487/SIGGENES was not found=
 on
> > > > > this server.
> > > > >
> > > > > Apache/1.3.14 Server at genomics.biochem.uci.edu Port 80
> > > >
> > > > that site is an old one - I took it down to prevent people from u=
sing an archaic version.  But thaks for telling me - I'll add a
> > > >
> > > > > ---------------------------------------------------------------=
-------
> > > > > ---------------------------------------------------
> > > > > Since I had bookmarked the form directly, I thought maybe I was=
 using
> > > > > an out of date version,  so I looked for an updataed site and u=
sed the
> > > > > URL listed at the top of this message.
> > > > >
> > > > > Thanks for your help,
> > > > >
> > > > > -Jeanette McClintick
> > > > > Indiana University
> > > > > Medical and Molecular Genetics
> > > > > Graduate Student
> > > > >
> > > > > -------------------
> > > > > > Hi there,
> > > > > >
> > > > > > Thanks for using CyberT - Yes, I'd like to see the bad input =
file,
> > > > > if you don't mind.  Also, what URL did you use to access CyberT=
 (th
> > > > > ere are a fair number of CyberT installations out there now and=

> > > > > > some are not as up-to-date as others..
> > > > > >
> > > > > > Best wishes
> > > > > > Harry
> > > > > >
> > > > > >
> > > > > > Jeanette Nance Mcclintick wrote:
> > > > > > >
> > > > > > > Harry,
> > > > > > >
> > > > > > > I am trying to run the C+E analysis via the web page. I rec=
 eived
> > > > > the
> > > > > > > following error message:
> > > > > > >
> > > > > > > Error in cov(x, y, use =3D use) : miss ing observations in =
cov/cor
> > > > > > > In addition: There were 50 or more warnings (use warnings()=
 to see
> > > > > the
> > > > > > > first 50)
> > > > > > > Execution halted
> > > > > > >
> > > > > > > I tried using 3 differe nt input files, 2 of which I ran
> > > > > successfully
> > > > > > > previously.
> > > > > > >
> > > > > > > I can send input if you like.
> > > > > > >
> > > > > > > Thanks for your help,
> > > > > > >
> > > > > > > Jeanette McClintick
> > > > > >
> > > > > > --
> > > > > > Cheers,
> > > > > > Harry
> > > > > >
> > > > > > Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... ||
> > > > > man...@ho...
> > > > > >
> > > > > >
> > > > > =EF=A3=BF
> > > > >
> > > > >   -------------------------------------------------------------=
-------------------------------------------------------------------------=
------------------------------------------------------------------
> > > > >                               Name: Duguid01-abs4-26-01.cyb
> > > > >    Duguid01-abs4-26-01.cyb    Type: unspecified type (applicati=
on/octet-stream)
> > > > >                           Encoding: base64
> > > >
> > > > --
> > > > Cheers,
> > > > Harry
> > > >
> > > > Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || manga=
la...@ho...
> >
> > --
> > Cheers,
> > Harry
> >
> > Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || mangalam@=
home.com
> >
> >

-- =

Cheers,
Harry

Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || mangalam@home=
=2Ecom

[GeneX-dev] gendb.pl

[<ja...@op...>]

Hey All,

For those of you that wish to create a genex2 DB for testing data
loading I've added a new script in the Genex/scripts directory. 

All you need to do is run it from within the Genex/ dir as:

scripts/gendb.pl --name=your_name@host --dbname=your_db_name

and it will do *all* the rest:
1) create the DB 
2) add the basic users
3) add the controlled vocab
4) add the species
5) add the spotters
6) add the software

scanners to be added soon.

HTH,
jas.

Re: [GeneX-dev] Re: Cyber-T

[<ja...@op...>]

"Harry Mangalam" <man...@ho...> writes:

> The only thing that you can help to assure its longevity (besides
> showering us with unrestricted cash) is to keep using it and give
> voice to whatever things irk you in the interface and execution in
> the future.  If you happen to publish something and the CyberT
> analysis helped you, you might consider citing the paper that was
> just accepted:

I would also add tell all your friends, relatives, and business
acquaintences (even any enemies) about Cyber-T and GeneX. 

;-)

Also, if you believe that open algorithms and open information are
important, we'll be circulating a net petition shortly to ask the
major funding agencies to require open source licenses for any
software funded by public money. Stay tuned.

jas.

[GeneX-dev] Re: Cyber-T

[Harry M. <man...@ho...>]

Hi Barry,

Sorry for the delay in replying - I got behind in my email and I missed yours in the 1st round of catch-up.

Thanks very much for your kind note - we *LIKE* kind notes, altho we have to admit that well-considered UNkind notes can be more helpful long term :).  I'm cc:ing this to our development group who
appreciate as much encouragement as they can get.

The only thing that you can help to assure its longevity (besides showering us with unrestricted cash) is to keep using it and give voice to whatever things irk you in the interface and execution in
the future.  If you happen to publish something and the CyberT analysis helped you, you might consider citing the paper that was just accepted:

Anthony D. Long, Harry J. Mangalam, Bob Y. P. Chan, Lorenzo Tolleri, G. Wesley Hatfield, and Pierre Baldi.  Gene Expression Profiling in Escherichia coli K12: Improved statistical inference from DNA
microarray data using analysis of variance and a Bayesian statistical framework
Papers in Press: J. Biol. Chem. published March 20, 2001 as
10.1074/jbc.M010192200

(at www.jbc.org under papers in press).

We are working on some pretty (but useful, we swear!) graphical displays for both CyberT as well as other GeneX analyses.  

Have you tried the xgobi output of CyberT?  It will sooon be possible to view the output in your browser as oppposed to having to buy a commercial Xserver.

Thanks again, and Best wishes!

Harry Mangalam


"Ford, Barry" wrote:
> 
> Cyber-T is excellent. The interface is simple and clean,
> and the results very quick.  Really the first and most
> useful for analysis (as opposed to pretty display)
>  of microarray  data. If there is anything I can do to
> help ensure its survival?
> 
> Dr. Barry Ford
> Consumer and Clinical
> Radiation Protection Bureau
> Health Canada
> 775 Brookfield Road
> 613-952-4627 or 613-998-8493
> bar...@hc...

-- 
Cheers,
Harry

Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || man...@ho...

Re: [GeneX-dev] Controlling CytberT output format...

[Michael P. <mic...@ho...>]

Harry,

I think that controlling formatting through R would be the
appropriate way to do it in version 1.
We should discuss the output filters as a GeneX Version 2
implementation as where exactly formatting gets introduced is
a matter of discussion. The natural place in Version 1 is in
the program putting out the file that gets displayed/downloaded.

If others have opinions and ideas on the output filter approach,
I hope they will chime in. As I have expressed, I'd like to see
the raw analysis results stored in the database, then formatting
occur appropriately on query and display by the user.

Regards,

Michael Pear
----- Original Message -----
From: "Harry J. Mangalam" <hj...@nc...>
To: <gen...@li...>
Sent: Thursday, April 26, 2001 7:50 AM
Subject: Re: [GeneX-dev] Controlling CytberT output format...


> Hi Michael,
>
> OK - After a fun-filled morning o' televconferencing, I'll hopefully be able
> to dedicate this aft to adding these and some other things to CyberT - I
> have quite a little list now.
>
> I'll probably add the formatting as an internal R command as R supports that
> exactly, but I'll think about how to impliment the general output filter
> that you & Jason talked about; that's obviously the better way to do it.
>
> hjm
>
>
> On Thu, 26 Apr 2001, Michael Pear wrote:
>
> > Hi Harry,
> >
> >  Whatever you make optional is great. Yes, especially the input data columns and
> > digits after the decimal (not actually significant digits since the ones to the left of the
> > decimal count there too). While you are in there, I think it would be good to
> > change the output headings, using some more verbose headings. For example
> > 'Cmn' -> Control Mean
> > and 'p' -> p-value
> > 'lnp' -> p-value(log data)
> >
> > I've had my own bug-ridden days, but I've been working with computers:-) At least
> > the results for me aren't physical discomfort, just mental anguish!
> > Hope you had fun anyways.
> >
> > Regards,
> >
> > Michael Pear
> > ----- Original Message -----
> > From: "Harry J. Mangalam" <hj...@nc...>
> > To: <gen...@li...>
> > Sent: Wednesday, April 25, 2001 1:08 PM
> > Subject: Re: [GeneX-dev] Controlling CytberT output format...
> >
> >
> > > Yup that would be me and yes, we can do all this pretty easily.   Let me
> > > remind myself what the output looks like and I'll send you some sample utput
> > > to comment on.
> > >
> > > So you want the input data either optional or moved to the end of the
> > > report and and the teh number of significant digits reduced to (perhaps
> > > optionally) 3, right?
> > >
> > > Hey. when people start asking for things like this, it means they're
> > > actually using it enough to get annoyed with it.   This is good news I
> > > think..
> > >
> > > Just back from 3 bug-ridden days in the wilds.  Will start in on it after a
> > > shower..
> > >
> > > hjm
> > >
> > >
> > > On Mon, 23 Apr 2001, Michael Pear wrote:
> > >
> > > > How can I reduce the number of digits displayed after the decimal point
> > > > in the CyberT results? Right now, results are printed with between 5 and 8
> > > > digits after the decimal, far beyond what I would expect to be signfiicant.
> > > > I'd like to reduce it to 2 or 3.
> > > >
> > > > Also, I'd like to eliminate printing of the input data (C_ and E_ columns) in the CyberT
output,
> > or at least move them to the end of the table. The most important columns for interpreting
> > > > the results are currently at the very end of each row....I'd like to move them
> > > > nearer to the gene name.
> > > >
> > > > I presume all of this happens in the R code, which I am far from fluent in. Can someone
point
> > the way (probably you, Harry!)?
> > > >
> > > > Regards,
> > > >
> > > > Michael Pear
> > > >
> > > >
> > >
> > >
> > > _______________________________________________
> > > Genex-dev mailing list
> > > Gen...@li...
> > > http://lists.sourceforge.net/lists/listinfo/genex-dev
> >
> >
> > _______________________________________________
> > Genex-dev mailing list
> > Gen...@li...
> > http://lists.sourceforge.net/lists/listinfo/genex-dev
> >
>
>
> _______________________________________________
> Genex-dev mailing list
> Gen...@li...
> http://lists.sourceforge.net/lists/listinfo/genex-dev
>

Re: [GeneX-dev] Controlling CytberT output format...

[Harry J. M. <hj...@nc...>]

Hi Michael,

OK - After a fun-filled morning o' televconferencing, I'll hopefully be able
to dedicate this aft to adding these and some other things to CyberT - I
have quite a little list now.

I'll probably add the formatting as an internal R command as R supports that
exactly, but I'll think about how to impliment the general output filter
that you & Jason talked about; that's obviously the better way to do it.

hjm


On Thu, 26 Apr 2001, Michael Pear wrote:

> Hi Harry,
> 
>  Whatever you make optional is great. Yes, especially the input data columns and
> digits after the decimal (not actually significant digits since the ones to the left of the
> decimal count there too). While you are in there, I think it would be good to
> change the output headings, using some more verbose headings. For example
> 'Cmn' -> Control Mean
> and 'p' -> p-value
> 'lnp' -> p-value(log data)
> 
> I've had my own bug-ridden days, but I've been working with computers:-) At least
> the results for me aren't physical discomfort, just mental anguish!
> Hope you had fun anyways.
> 
> Regards,
> 
> Michael Pear
> ----- Original Message -----
> From: "Harry J. Mangalam" <hj...@nc...>
> To: <gen...@li...>
> Sent: Wednesday, April 25, 2001 1:08 PM
> Subject: Re: [GeneX-dev] Controlling CytberT output format...
> 
> 
> > Yup that would be me and yes, we can do all this pretty easily.   Let me
> > remind myself what the output looks like and I'll send you some sample utput
> > to comment on.
> >
> > So you want the input data either optional or moved to the end of the
> > report and and the teh number of significant digits reduced to (perhaps
> > optionally) 3, right?
> >
> > Hey. when people start asking for things like this, it means they're
> > actually using it enough to get annoyed with it.   This is good news I
> > think..
> >
> > Just back from 3 bug-ridden days in the wilds.  Will start in on it after a
> > shower..
> >
> > hjm
> >
> >
> > On Mon, 23 Apr 2001, Michael Pear wrote:
> >
> > > How can I reduce the number of digits displayed after the decimal point
> > > in the CyberT results? Right now, results are printed with between 5 and 8
> > > digits after the decimal, far beyond what I would expect to be signfiicant.
> > > I'd like to reduce it to 2 or 3.
> > >
> > > Also, I'd like to eliminate printing of the input data (C_ and E_ columns) in the CyberT output,
> or at least move them to the end of the table. The most important columns for interpreting
> > > the results are currently at the very end of each row....I'd like to move them
> > > nearer to the gene name.
> > >
> > > I presume all of this happens in the R code, which I am far from fluent in. Can someone point
> the way (probably you, Harry!)?
> > >
> > > Regards,
> > >
> > > Michael Pear
> > >
> > >
> >
> >
> > _______________________________________________
> > Genex-dev mailing list
> > Gen...@li...
> > http://lists.sourceforge.net/lists/listinfo/genex-dev
> 
> 
> _______________________________________________
> Genex-dev mailing list
> Gen...@li...
> http://lists.sourceforge.net/lists/listinfo/genex-dev
> 

Re: [GeneX-dev] Controlling CytberT output format...

[Michael P. <mic...@ho...>]

Hi Harry,

 Whatever you make optional is great. Yes, especially the input data columns and
digits after the decimal (not actually significant digits since the ones to the left of the
decimal count there too). While you are in there, I think it would be good to
change the output headings, using some more verbose headings. For example
'Cmn' -> Control Mean
and 'p' -> p-value
'lnp' -> p-value(log data)

I've had my own bug-ridden days, but I've been working with computers:-) At least
the results for me aren't physical discomfort, just mental anguish!
Hope you had fun anyways.

Regards,

Michael Pear
----- Original Message -----
From: "Harry J. Mangalam" <hj...@nc...>
To: <gen...@li...>
Sent: Wednesday, April 25, 2001 1:08 PM
Subject: Re: [GeneX-dev] Controlling CytberT output format...


> Yup that would be me and yes, we can do all this pretty easily.   Let me
> remind myself what the output looks like and I'll send you some sample utput
> to comment on.
>
> So you want the input data either optional or moved to the end of the
> report and and the teh number of significant digits reduced to (perhaps
> optionally) 3, right?
>
> Hey. when people start asking for things like this, it means they're
> actually using it enough to get annoyed with it.   This is good news I
> think..
>
> Just back from 3 bug-ridden days in the wilds.  Will start in on it after a
> shower..
>
> hjm
>
>
> On Mon, 23 Apr 2001, Michael Pear wrote:
>
> > How can I reduce the number of digits displayed after the decimal point
> > in the CyberT results? Right now, results are printed with between 5 and 8
> > digits after the decimal, far beyond what I would expect to be signfiicant.
> > I'd like to reduce it to 2 or 3.
> >
> > Also, I'd like to eliminate printing of the input data (C_ and E_ columns) in the CyberT output,
or at least move them to the end of the table. The most important columns for interpreting
> > the results are currently at the very end of each row....I'd like to move them
> > nearer to the gene name.
> >
> > I presume all of this happens in the R code, which I am far from fluent in. Can someone point
the way (probably you, Harry!)?
> >
> > Regards,
> >
> > Michael Pear
> >
> >
>
>
> _______________________________________________
> Genex-dev mailing list
> Gen...@li...
> http://lists.sourceforge.net/lists/listinfo/genex-dev

Re: [GeneX-dev] Controlling CytberT output format...

[Harry J. M. <hj...@nc...>]

Yup that would be me and yes, we can do all this pretty easily.   Let me
remind myself what the output looks like and I'll send you some sample utput
to comment on.

So you want the input data either optional or moved to the end of the
report and and the teh number of significant digits reduced to (perhaps
optionally) 3, right?

Hey. when people start asking for things like this, it means they're
actually using it enough to get annoyed with it.   This is good news I
think..

Just back from 3 bug-ridden days in the wilds.  Will start in on it after a
shower..

hjm


On Mon, 23 Apr 2001, Michael Pear wrote:

> How can I reduce the number of digits displayed after the decimal point 
> in the CyberT results? Right now, results are printed with between 5 and 8
> digits after the decimal, far beyond what I would expect to be signfiicant.
> I'd like to reduce it to 2 or 3.
> 
> Also, I'd like to eliminate printing of the input data (C_ and E_ columns) in the CyberT output, or at least move them to the end of the table. The most important columns for interpreting
> the results are currently at the very end of each row....I'd like to move them
> nearer to the gene name. 
> 
> I presume all of this happens in the R code, which I am far from fluent in. Can someone point the way (probably you, Harry!)?
> 
> Regards,
> 
> Michael Pear
> 
> 

Re: [GeneX-dev] Controlling CytberT output format...

[Michael P. <mic...@ho...>]

Jason,

 In the long term, the analysis results should be returned to the database,
and any formatting for the user handled through queries on these results. The CyberT
results are particularly suitable for returning to the database as each column
could be put back in as an array measurement. Many other analyses or data
processing results that can be represented in a table will easily map back to
the original database....clusters would require a more sophisticated handling.

In this arrangement, the flow looks like: 

GeneX DB --> Input Filter --> Analysis Tool --> Output Filter --> GeneX DB -> Query Results ->User

Of course, the initial "Query Results"  could be generated as part of the analysis flow
to return the complete set of analysis results (essentially reproducing what is there now)
But, it also opens the door for comparisons among arrays for questions like
"Where is the expression level of this gene changed significantly(as measured by p value)?"

Regards,

Michael Pear


----- Original Message ----- 
From: "Jason E. Stewart" <ja...@op...>
To: <gen...@li...>
Sent: Sunday, April 22, 2001 9:46 PM
Subject: Re: [GeneX-dev] Controlling CytberT output format...


> "Michael Pear" <mic...@ho...> writes:
> 
> > How can I reduce the number of digits displayed after the decimal
> > point in the CyberT results? Right now, results are printed with
> > between 5 and 8 digits after the decimal, far beyond what I would
> > expect to be signfiicant.  I'd like to reduce it to 2 or 3.
> > 
> > Also, I'd like to eliminate printing of the input data (C_ and E_
> > columns) in the CyberT output, or at least move them to the end of
> > the table. The most important columns for interpreting the results
> > are currently at the very end of each row....I'd like to move them
> > nearer to the gene name.
> 
> Harry's out camping until Weds. I believe you're correct, Michael, and
> this happens in the R code, and is out of my league.
> 
> The following is intended to provoke a discussion not to solve your
> immediate problem.
> 
> I think we can generalize your request:
> * there will be many tools we want to interface to genex
> * many will have strict I/O requirements that will be difficult for us
>   to modify
> * we want to have flexibility to enter the data we have, and get the
>   output in the form that best suits us.
> 
> I think we can get this by adding an input filter and an output filter
> into the pipeline for each application we connect:
> 
> GeneX --> Input Filter --> Analysis Tool --> Output Filter --> User
> 
> The input filter could:
> * Handle missing data (convert all missing data to standard symbol or
>   just strip rows with missing data)
> * Add necessary header info for tool
> * modify column format
> 
> The output filter could:
> * handle num of significant digits
> * column ordering in output
> * convert missing data symbols
> 
> I imagine, the input filter would be written once per program, whereas
> the output filter could be specified by each user when running the
> program. This filter is one of the things that could be stored in the
> user's workspace in the DB.
> 
> jas.
> 
> _______________________________________________
> Genex-dev mailing list
> Gen...@li...
> http://lists.sourceforge.net/lists/listinfo/genex-dev

Re: [GeneX-dev] Controlling CytberT output format...

[<ja...@op...>]

"Michael Pear" <mic...@ho...> writes:

> How can I reduce the number of digits displayed after the decimal
> point in the CyberT results? Right now, results are printed with
> between 5 and 8 digits after the decimal, far beyond what I would
> expect to be signfiicant.  I'd like to reduce it to 2 or 3.
> 
> Also, I'd like to eliminate printing of the input data (C_ and E_
> columns) in the CyberT output, or at least move them to the end of
> the table. The most important columns for interpreting the results
> are currently at the very end of each row....I'd like to move them
> nearer to the gene name.

Harry's out camping until Weds. I believe you're correct, Michael, and
this happens in the R code, and is out of my league.

The following is intended to provoke a discussion not to solve your
immediate problem.

I think we can generalize your request:
* there will be many tools we want to interface to genex
* many will have strict I/O requirements that will be difficult for us
  to modify
* we want to have flexibility to enter the data we have, and get the
  output in the form that best suits us.

I think we can get this by adding an input filter and an output filter
into the pipeline for each application we connect:

GeneX --> Input Filter --> Analysis Tool --> Output Filter --> User

The input filter could:
* Handle missing data (convert all missing data to standard symbol or
  just strip rows with missing data)
* Add necessary header info for tool
* modify column format

The output filter could:
* handle num of significant digits
* column ordering in output
* convert missing data symbols

I imagine, the input filter would be written once per program, whereas
the output filter could be specified by each user when running the
program. This filter is one of the things that could be stored in the
user's workspace in the DB.

jas.

[GeneX-dev] Controlling CytberT output format...

[Michael P. <mic...@ho...>]

How can I reduce the number of digits displayed after the decimal point=20
in the CyberT results? Right now, results are printed with between 5 and =
8
digits after the decimal, far beyond what I would expect to be =
signfiicant.
I'd like to reduce it to 2 or 3.

Also, I'd like to eliminate printing of the input data (C_ and E_ =
columns) in the CyberT output, or at least move them to the end of the =
table. The most important columns for interpreting
the results are currently at the very end of each row....I'd like to =
move them
nearer to the gene name.=20

I presume all of this happens in the R code, which I am far from fluent =
in. Can someone point the way (probably you, Harry!)?

Regards,

Michael Pear

[GeneX-dev] Re: contributing code to GeneX

[<ja...@op...>]

Michael Pear writes:

> In order to incorporate contributions like this for testing,
> feedback, etc. without introducing unnecessary changes in the stable
> trees, I was thinking that creating a tagged branch would allow
> access to a GeneX version with this code incorporated. Later, if it
> were deemed appropriate to include as part of the general
> distribution, the branch could be merged with its parent branch.

Yes, this approach would work fine. In a perfect world, it would be
best to branch of the devel tree, and not the stable tree. But since
so many major changes have happened in the devel tree, it seems wisest
to do this in the stable tree for now.

> Would this be a reasonable model for contributors to add
> experimental code? Are special privileges required to create a cvs
> tagged branch? I am currently set up to check code in on
> genex.ncgr.org.

If you have commit privelege, you can add tags. Branches are just a
special kind of tag. 

Check out genex-server with the 'Rel-1_0_1' tag, and then turn create
a new branch (using cvs tag -b <tagname>), call it something like
'1_0_1-embperl'. Then you do a cvs update -r <tagname> and the source
that you originally checked out with the static tag is converted to
the new branch you created. All commits you make from that code will
go into your branch.

If you would like more help I'd be happy to assist.

jas.

[GeneX-dev] Contributing to the GeneX source code...

[Michael P. <mic...@ho...>]

I've been working on some preliminary array management and data processing
pages that I'd like to contribute to the GeneX tree. The pages require Embperl, which
diverges a bit from the current perl/CGI model in the GeneX server. I decided to use Embperl
because of the simple session support which I use to restrict experiment viewing to
a logged in user and the philosophy of code embedded in an html page (similar to ASP, JSP).

In order to incorporate contributions like this for testing, feedback, etc. without introducing
unnecessary changes in the stable trees, I was thinking that creating a tagged branch
would allow access to a GeneX version with this code incorporated. Later, if it were deemed
appropriate to include as part of the general distribution, the branch could be merged
with its parent branch.

Would this be a reasonable model for contributors to add experimental code? Are
special privileges required to create a cvs tagged branch? I am currently set up to check
code in on genex.ncgr.org.

Michael Pear

[GeneX-dev] Re: Something to watch..

[<ja...@op...>]

Hey Michael,

Welcome Back!

"Michael Pear" <mic...@ho...> writes:

> When installing V1.01, I encountered something that you may want to
> watch out for when using selectrow_array. That is, if 0 rows are
> returned in the construct:
> 
> $my_row=$dbh->selectrow_array($sql)[0] | die $DBI::errstr;
> 
> then the program dies even though there isn't really an error (and
> no error is reported, just the line number). A separate check and
> handling of the 0 row case may be needed.

Yes, this small shortcut can cause problems. This is one reason why I
modified get_objects*() to always return something in the API I
control. Unfortunately I can't control DBI. So we'll have to be aware
of how we use it.

Thanks for the tip.
jas.

[GeneX-dev] GeneX Installation Issue: Error in DBD::Pg V0.96

[Michael P. <mic...@ho...>]

*Attention anyone doing a new installation of GeneX along with the =
required Perl modules*

In the process of installing GeneX on a new server, I discovered
that the perl module DBD:Pg V0.96 (the latest available through CPAN)
will not properly connect to the Postgres database with the
database users specified in the install process.=20

It turns out that this problem was found and a patch suggested
in the message below:

http://www.mail-archive.com/dbi...@pe.../msg01788.html


Instead of patching, you can find and install V0.95 of DBD::Pg, although =
CPAN
naturally leads you to V0.96.

Please note, if you have V0.96,  fixing this problem is CRITICAL to =
proper operation
of GeneX. If you don't fix it during installation, then you will not be =
able to access the database
from from CGI scripts.

An indication that this is happening on your system is a message like:

DBI->connect(dbname=3Dtempate1) failed: FATAL 1: SetUserID: user 'root' =
is not in 'pg_shadow' at ./install-all.pl line 1567

(This prevented installation).

You may get by installation, if you happen to use a user that is also a =
postgres database user, but it
will get you later when running the web-server.

Regards,

Michael Pear

[GeneX-dev] Re: GeneX back up

[<ja...@op...>]

"Systems Administrator" <sy...@ge...> writes:

> With 2 PII 1000 Mhz Procs.

Seems to run CVS pretty fast ;-)

Thanks!
jas.

[GeneX-dev] GeneX Processor Upgrades

[John U. <jp...@nc...>]

Hi,

This is a reminder that GeneX will go down for upgrades at 6 PM
Mountain Time.

I will send a followup email as soon as the machine is back up.

-John

[GeneX-dev] genex2 now merged into CVS HEAD

[<ja...@op...>]

Hey,

So I've merged the experimental genex2 branch into the main trunk of
the genex-server module. The significance of this is that if you do a
straight 'cvs update genex-server', you will be getting the 1.1
development branch.

I also created a branch tag for the 1.0.1 release, 'Rel-1_0_1-branch',
that we can use to update the 1.0 server code. So if you want the
working code using this branch.

The merge went amazingly smooth. The only problems were files that I
had modified in both branches, in total only 7 files. I was expecting
more, and happy that it only took an hour to merge, and check it all
back in, given the amount of differences between the two branches.

jas.

[GeneX-dev] genex.ncgr.org will be down tonight for new CPUs

[Harry M. <man...@ho...>]

Starting 6:00pm MST, 5:00 pm Pacific TODAY, genex will be taken down for inserting new, faster CPU's. (2x1GHz). Thanks, John!

Please plan to hold CVS commits, analyses, etc until after a message goes out announcing its revival.

I'll also put an announcement on the web site to this effect as well.

- 
Cheers,
Harry

Harry J Mangalam -- (949) 856 2847 (v&f) -- hj...@nc... || man...@ho...