gusdev-rad-issues — Send issues relating to the RAD componenet here

[Gusdev-rad-issues] Re: [Gusdev-gusdev] about uploading RAD data problem

[Elisabetta M. <man...@pc...>]

Hi Sucheta,
RAD3.OntologyEntry.table_id points to Core.TableInfo.table_id and 
identifies the table. RAD3.OntologyEntry.row_id points to the primary key 
of the relevant entry in the table corresponding to table_id.
For example, the entry in our current RAD3.OntologyEntry:

ONTOLOGY_ENTRY_ID	PARENT_ID	TABLE_ID	ROW_ID	
EXTERNAL_DATABASE_RELEASE_ID	SOURCE_ID	URI	NAME	CATEGORY	
VALUE	...
460	504	3032	29603	8015	#stimulus_or_stress_design	
(null)	PerturbationalDesign	ExperimentDesignType	
stimulus_or_stress_design...

has table_id=3032 which corresponds to the table SRes.MGEDOntologyTerm 
(that is this is the table in our Core.TableInfo with table_id=3032), the 
row_id=29603 tells us that if we to the query:

select * from SRes.MGEDOntologyTerm where mged_ontology_term_id=29603

we'll get the stimulus_or_stress_design entry.
In other words, the entry in RAD3.OntologyEntry with ontology_entry_id=460 
corresponds to the ontology term stored in SRes.MGEDONtologyTerm with 
primary_key=29603 in that table.

I'll let Trish address your remaining questions (1 and 2), as 
she can probably provide you with more details on that than me.
Elisabetta

On Thu, 13 May 2004, Sucheta Tripathy wrote:

> Hello,
> 
> It may appear little stupid, but I was browsing through all the core tables 
> as well as the sres.ontology group of tables, but could not find a row_id 
> corresponding to a table. Am I missing something? Anyway, this is not an 
> important problem.
> 
> Here are some more important questions.
> 1. Have you downloaded the sres.ontology data from NCBI?
> 2. For the RAD3.ontologyentry how do you generate data?
> 
> Many thanks
> 
> Sucheta
> 
> 
> At 09:56 AM 5/13/2004 -0400, Elisabetta Manduchi wrote:
> 
> >Sucheta,
> >I'm cc-ing the rad-issues list as this is a point of general interest.
> >The point is that the ontology terms are actually stored in the SRes name
> >space. There you'll find a variety of tables, both for the MGED ontologies
> >and for Taxon, etc. RAD3.OntologyEntry is merely used to pull in those
> >ontology terms that are actually used frequently in RAD, so that most RAD
> >tables point to RAD3.OntologyEntry rather than directly to the SRes
> >ontology tables (the exception is RAD3.BioSource that for taxon points
> >directly to SRes). Thus, a typical term in RAD3.OntologyEntry is actually
> >coming from one of the terms stored in an appropriate SRes table,
> >precisely in the table pointed by table_id and from the term in that
> >table pointed by row_id.
> >(As an aside, there is an additional component: for the
> >StudyAnnotator we have pull-down menus for those fields coming from
> >ontologies and since for some categories there are many terms even in
> >RAD3.OntologyEntry we use RAD3.ProjectLink to link to each Project those
> >OntologyEntry terms that are relevant to that project, so that only those
> >would be displayed in the StudyAnnotator when one logs in for a certain
> >project.)
> >Elisabetta
> >
> >On Thu, 13 May 2004, Sucheta Tripathy wrote:
> >
> > > Hi Elisabetta,
> > >
> > > Many thanks for pointing me to the dump file. Got it. However, while 
> > having
> > > a cursory look I could not understand what the row_id in the table means.
> > >
> > > Here is a snippet of the first few lines of the dump file.
> > >
> > > ONTOLOGY_ENTRY_ID       PARENT_ID       TABLE_ID        ROW_ID
> > > EXTERNAL_DATABASE_RELEASE_ID    SOURCE_ID       URI     NAME    CATEGORY
> > >      VALUE   DEFINITION
> > > 217     (null)  3032    1185    7054    #cell_lysate    (null)  (null)
> > > MaterialType    cell_lysate     cell lysate, a collection of cells whose
> > > membranes have been disrupted
> > >
> > > Thanks again
> > >
> > > Sucheta
> > >
> > > At 05:52 PM 5/12/2004 -0400, you wrote:
> > >
> > > >Hi Sucheta,
> > > >in the StudyAnnotator tarball available at the RAD download site (follow
> > > >link from RAD website under GetRAD) we have a dump of RAD3.OntologyEntry
> > > >but it's kind of old. I'm cc-ing Trish Whetzel who is in our group and
> > > >part of the MGED Ontology group, she can probably point you to a more
> > > >recent dump.
> > > >Elisabetta
> > > >
> > > >
> > > >On Wed, 12 May 2004, Sucheta Tripathy wrote:
> > > >
> > > > > Hi Elisabetta,
> > > > >
> > > > > I am back to bother you again!!
> > > > > Now that we are going to upload real data to the module, I was
> > > > wondering if
> > > > > I can get hold of a template file for OntologyEntry. This is just to
> > > > get an
> > > > > idea how the terms are defined.
> > > > > Do you have a template data file that goes to RAD3.OntologyEntry?
> > > > >
> > > > > Thanks
> > > > >
> > > > > Sucheta
> 

-- 
Elisabetta Manduchi

Computational Biology and Informatics Laboratory
Center for Bioinformatics
University of Pennsylvania
1428 Blockley Hall
423 Guardian Drive
Philadelphia, PA 19104-6021

phone: 215-573-4408
fax: 215 573-3111
email: man...@pc...
web: http://www.cbil.upenn.edu/~manduchi

---

[Gusdev-rad-issues] Re: [Gusdev-gusdev] about uploading RAD data problem

[Sucheta T. <su...@vb...>]

Hello,

It may appear little stupid, but I was browsing through all the core tables 
as well as the sres.ontology group of tables, but could not find a row_id 
corresponding to a table. Am I missing something? Anyway, this is not an 
important problem.

Here are some more important questions.
1. Have you downloaded the sres.ontology data from NCBI?
2. For the RAD3.ontologyentry how do you generate data?

Many thanks

Sucheta


At 09:56 AM 5/13/2004 -0400, Elisabetta Manduchi wrote:

>Sucheta,
>I'm cc-ing the rad-issues list as this is a point of general interest.
>The point is that the ontology terms are actually stored in the SRes name
>space. There you'll find a variety of tables, both for the MGED ontologies
>and for Taxon, etc. RAD3.OntologyEntry is merely used to pull in those
>ontology terms that are actually used frequently in RAD, so that most RAD
>tables point to RAD3.OntologyEntry rather than directly to the SRes
>ontology tables (the exception is RAD3.BioSource that for taxon points
>directly to SRes). Thus, a typical term in RAD3.OntologyEntry is actually
>coming from one of the terms stored in an appropriate SRes table,
>precisely in the table pointed by table_id and from the term in that
>table pointed by row_id.
>(As an aside, there is an additional component: for the
>StudyAnnotator we have pull-down menus for those fields coming from
>ontologies and since for some categories there are many terms even in
>RAD3.OntologyEntry we use RAD3.ProjectLink to link to each Project those
>OntologyEntry terms that are relevant to that project, so that only those
>would be displayed in the StudyAnnotator when one logs in for a certain
>project.)
>Elisabetta
>
>On Thu, 13 May 2004, Sucheta Tripathy wrote:
>
> > Hi Elisabetta,
> >
> > Many thanks for pointing me to the dump file. Got it. However, while 
> having
> > a cursory look I could not understand what the row_id in the table means.
> >
> > Here is a snippet of the first few lines of the dump file.
> >
> > ONTOLOGY_ENTRY_ID       PARENT_ID       TABLE_ID        ROW_ID
> > EXTERNAL_DATABASE_RELEASE_ID    SOURCE_ID       URI     NAME    CATEGORY
> >      VALUE   DEFINITION
> > 217     (null)  3032    1185    7054    #cell_lysate    (null)  (null)
> > MaterialType    cell_lysate     cell lysate, a collection of cells whose
> > membranes have been disrupted
> >
> > Thanks again
> >
> > Sucheta
> >
> > At 05:52 PM 5/12/2004 -0400, you wrote:
> >
> > >Hi Sucheta,
> > >in the StudyAnnotator tarball available at the RAD download site (follow
> > >link from RAD website under GetRAD) we have a dump of RAD3.OntologyEntry
> > >but it's kind of old. I'm cc-ing Trish Whetzel who is in our group and
> > >part of the MGED Ontology group, she can probably point you to a more
> > >recent dump.
> > >Elisabetta
> > >
> > >
> > >On Wed, 12 May 2004, Sucheta Tripathy wrote:
> > >
> > > > Hi Elisabetta,
> > > >
> > > > I am back to bother you again!!
> > > > Now that we are going to upload real data to the module, I was
> > > wondering if
> > > > I can get hold of a template file for OntologyEntry. This is just to
> > > get an
> > > > idea how the terms are defined.
> > > > Do you have a template data file that goes to RAD3.OntologyEntry?
> > > >
> > > > Thanks
> > > >
> > > > Sucheta

[Gusdev-rad-issues] Re: [Gusdev-gusdev] about uploading RAD data problem

[Elisabetta M. <man...@pc...>]

Sucheta,
I'm cc-ing the rad-issues list as this is a point of general interest.
The point is that the ontology terms are actually stored in the SRes name 
space. There you'll find a variety of tables, both for the MGED ontologies 
and for Taxon, etc. RAD3.OntologyEntry is merely used to pull in those 
ontology terms that are actually used frequently in RAD, so that most RAD 
tables point to RAD3.OntologyEntry rather than directly to the SRes 
ontology tables (the exception is RAD3.BioSource that for taxon points 
directly to SRes). Thus, a typical term in RAD3.OntologyEntry is actually 
coming from one of the terms stored in an appropriate SRes table, 
precisely in the table pointed by table_id and from the term in that 
table pointed by row_id.
(As an aside, there is an additional component: for the 
StudyAnnotator we have pull-down menus for those fields coming from 
ontologies and since for some categories there are many terms even in 
RAD3.OntologyEntry we use RAD3.ProjectLink to link to each Project those 
OntologyEntry terms that are relevant to that project, so that only those 
would be displayed in the StudyAnnotator when one logs in for a certain 
project.)
Elisabetta

On Thu, 13 May 2004, Sucheta Tripathy wrote:

> Hi Elisabetta,
> 
> Many thanks for pointing me to the dump file. Got it. However, while having 
> a cursory look I could not understand what the row_id in the table means.
> 
> Here is a snippet of the first few lines of the dump file.
> 
> ONTOLOGY_ENTRY_ID       PARENT_ID       TABLE_ID        ROW_ID 
> EXTERNAL_DATABASE_RELEASE_ID    SOURCE_ID       URI     NAME    CATEGORY 
>      VALUE   DEFINITION
> 217     (null)  3032    1185    7054    #cell_lysate    (null)  (null) 
> MaterialType    cell_lysate     cell lysate, a collection of cells whose 
> membranes have been disrupted
> 
> Thanks again
> 
> Sucheta
> 
> At 05:52 PM 5/12/2004 -0400, you wrote:
> 
> >Hi Sucheta,
> >in the StudyAnnotator tarball available at the RAD download site (follow
> >link from RAD website under GetRAD) we have a dump of RAD3.OntologyEntry
> >but it's kind of old. I'm cc-ing Trish Whetzel who is in our group and
> >part of the MGED Ontology group, she can probably point you to a more
> >recent dump.
> >Elisabetta
> >
> >
> >On Wed, 12 May 2004, Sucheta Tripathy wrote:
> >
> > > Hi Elisabetta,
> > >
> > > I am back to bother you again!!
> > > Now that we are going to upload real data to the module, I was 
> > wondering if
> > > I can get hold of a template file for OntologyEntry. This is just to 
> > get an
> > > idea how the terms are defined.
> > > Do you have a template data file that goes to RAD3.OntologyEntry?
> > >
> > > Thanks
> > >
> > > Sucheta

Re: [Gusdev-rad-issues] Re: [Gusdev-gusdev] about uploading RAD data problem

[Elisabetta M. <man...@pc...>]

Hi Sucheta,
if you go to http://www.cbil.upenn.edu/RAD/php/RAD-installation.php? and 
look under point 2 you will see the flow of the RAD pipeline that we have 
for our instance of RAD.
Summarizing, before you can load the results for a given quantification: 
the study must have been created, the assay, acquisition, and 
quantification entries have to be present in the db (and the array itself 
too since assay points to it).
The array can be loaded via the ArrayLoader plugin. All of the rest
can be loaded using the suite of webforms StudyAnnotator 
(whose code we make available). It's enough to complete the annotation in 
Module I (From assay to quantification) in order to get to the point where 
the results can be loaded, the remaining annotation can be done 
subsequently.
Alternatively, the above entries can be inserted into the database via a 
plugin. For the case of MAS5 data we have recently developed a new plugin 
that automatically enteres all the information for Module I by reading the 
.EXP and .RPT files (I gather one can ask GCOS to output these files too). 
This plugin is not in the Sanger cvs as it contains brief parts that are 
specific to our instance of RAD, but we can make it available upon 
request to those who would like to use it, after they modify those parts 
to suit their instances.
Elisabetta

On Tue, 4 May 2004, Sucheta Tripathy wrote:

> 
> Hi Elissabetta,
> 
> For me to use ArayResultLoader.pm , I have to first upload data into
> several other parent tables i.e; RAD3:ElementImp,
> RAD3::Array,
> RAD3::OntologyEntry and all the child tables of RAD3::Array.
> 
> Which plugin do I use to do all that?
> 
> Thanks
> 
> Sucheta
> --
> Sucheta Tripathy
> Virginia Bioinformatics Institute Phase-I
> Washington street.
> Virginia Tech.
> Blacksburg,VA 24061-0447
> phone:(540)231-8138
> Fax:  (540) 231-2606

[Gusdev-rad-issues] Re: [Gusdev-gusdev] about uploading RAD data problem

[Sucheta T. <su...@vb...>]

Hi Elissabetta,

For me to use ArayResultLoader.pm , I have to first upload data into
several other parent tables i.e; RAD3:ElementImp,
RAD3::Array,
RAD3::OntologyEntry and all the child tables of RAD3::Array.

Which plugin do I use to do all that?

Thanks

Sucheta
--
Sucheta Tripathy
Virginia Bioinformatics Institute Phase-I
Washington street.
Virginia Tech.
Blacksburg,VA 24061-0447
phone:(540)231-8138
Fax:  (540) 231-2606

[Gusdev-rad-issues] Re: [Gusdev-gusdev] about uploading RAD data problem

[Elisabetta M. <man...@pc...>]

Hi Sucheta,
my understanding is that GCOS is a wrapper around MAS 5, i.e. it has a db 
layer which in particular stores the output as per the MAS 5 
probe analysis algorithm.
The output you describe below is indeed basically that which you'll get in 
a metrics file and the measurements you show indeed fit the 
AffymetrixMAD5 view (see the schema browser 
http://www.gusdb.org/cgi-bin/schemaBrowser for a description of this view).
As for the plugin, the one in question is the ArrayResultLoader (not the 
ArrayLoader, which instead is used to load information about the array 
itself). The ArrayResultLoader is very generic and it will work with any 
view of either ElementResultImp or CompositeElementeEsultImp (so it can be 
used even if new views are created to accomodate a new quantification 
software). 
You can look at the details of the documentation at
http://www.gusdb.org/documentation/plugins/GUS-RAD-Plugin-ArrayResultLoader.html,
but basically the main part is that concering the format of data_file 
input into the plugin. The header should contain columns with names 
spelled exactly as those of the relevant attributes of the view one needs 
to populate plus additional identifier column(s) (which in the case of 
Affy would just be a column with the Affy_id and header "name").
For the AffymetrixMAS5 view the data_file would have a header like (case 
sensitive):

name	signal	detection	detection_p_value	stat_pairs	
stat_pairs_used

(not all these fields are mandatory). If there are additional columns with 
other names they can stay but will be ignored.
Elisabetta
P.S. For RAD specific emails you might want to subscribe to the 
source-fourge mailing list at 
https://lists.sourceforge.net/lists/listinfo/gusdev-rad-issues.
I've cc-ed it in this reply as I thought it could be useful to other RAD 
users.

On Tue, 4 May 2004, Sucheta Tripathy wrote:

> Hi Elisabetta,
> 
> Thanks for all the info!!
> 
> We are currently using a software called gene chip operating software(GCOS) 
> from affy instead of MAS5. Now I am not sure if it generates the matrix 
> files from the .CHP file. It takes either the .cel or the .dat file and 
> gives an output in a four column format as below:
> Affy_id         Y1_Signal       Y1_Detection    Descriptions
> AFFX-MurIL2_at  5.9             A               "M16762 Mouse interleukin 2 
> (IL-2) gene, exon 4"
> 
> Please tell me if this type of files are some of the files that you upload 
> to RAD3.
> 
> What about the ArrayLoader plugin. What input file does it take?
> 
> Thanks
> 
> Sucheta
> 
> At 12:17 PM 5/4/2004 -0400, you wrote:
> 
> >Hi Sucheta,
> >
> >1. .CHP files. These are not in text format, but through MAS 5.0 one can
> >generate their corresponding text files, i.e. the Metrics files. These can
> >be loaded into RAD. More precisely minor parsing, of the Metrics files is
> >needed in order to use them as input of the ArrayResultLoader
> >(basically just the header needs to be parsed a bit, see doc for
> >this plugin).
> >We have recently written another plugin, the BathArrayResult loader,
> >which loads in batch results for a group of assays. For this plugin
> >there is no need of parsing of Metrics files, all needed is an xml
> >configuration file. We have not deposited this plugin into the Sanger cvs
> >repository yet, but we can make available upon request in the meantime.
> >
> >2. .CEL files. Up to recently these were in text format and in principle
> >could be loaded into RAD using the ArrayResultLoader (and in fact, we have
> >loaded a few in our instance of RAD). However we have now opted not to
> >load the probe cell data into RAD. We keep the .CEL files on our
> >filesystem and store in RAD a pointer to their location (see point 4
> >below).
> >
> >3. For image files like .DAT files (and similarly for .tif files), we
> >store in RAD a pointer to their location, but and store the files  in our
> >filesystem.
> >
> >4. Summarizing how we currently deal with a given assay, with files
> >myassay.data, myassay.CEL, myassay.CHP (and myassay_Metrics.txt):
> >
> >a. The assay has one child in RAD3.Acquisition and the
> >RAD3.Acquisition.uri of this points to the location of the .DAT file in
> >our filesystem.
> >
> >b. The acquisition in (a) has TWO children in RAD.Quantification (and
> >these are related in RAD3.RelatedQuantification): one corresponds to the
> >.CEL quantification (i.e. the probe cell analysis) and its uri points to
> >the location of the .CEL file in our system, the other corresponds to the
> >.CHP (probe set) analysis and its uri points to the location of the
> >corresponding Metrics file (or .CHP file) in our file system.
> >
> >c. We load the results corresponding to the .CHP file (from the Metrics
> >file) into the AffymetrixMAS5 view of CompositeElementResultImp. Every
> >entry in the latter will have quantification_id pointing to the
> >quantification_id of the quantification in (b) corresponding to the probe
> >set analysis (.CHP).
> >
> >Hope this clarifies things a bit,
> >Elisabetta
> >
> >On Tue, 4 May 2004, Sucheta Tripathy wrote:
> >
> > > Hello Elisabetta and Sam,
> > >
> > > I am new to RAD modules in GUS. Now we have some microarray data, which is
> > > available in 3 different file formats(.cel,.dat,.CHP). Which of these 3
> > > files can be uploaded to RAD tables. None of the tables are in text format.
> > >
> > > Thanks in advance.
> > >
> > > Sucheta
> > >
> > > At 07:55 PM 11/16/2003 -0800, sam wang wrote:
> > > >Dear Elisabetta,
> > > >
> > > >thank you very much for your great help! now I am trying according to 
> > your
> > > >suggestion, besides, it seems a little complicated :-)
> > > >
> > > >Thanks again,
> > > >Sammy
> > > >
> > > >Elisabetta Manduchi <man...@pc...> wrote:
> > > >Dear Sam,
> > > >regarding the part of your question referring to the manufacturer_id
> > > >error, if you look at schemaBrowser for RAD3.Array at
> > > >http://www.gusdb.org/cgi-bin/schemaBrowser?db=CBILBLD&table=RAD3::Array 
> > &path=RAD3::Array
> > > >
> > > >you should see that manufacturer_id and platform_type_id are nonnullable.
> > > >Thus a valid manufacturer_id and a valid platform_type_id must be given
> > > >when running the ArrayLoader plugin (see also plugin documentation at
> > > >http://www.gusdb.org/documentation/plugins/GUS-RAD-Plugin-ArrayLoader.h 
> > tml).
> > > >This means that before you can load an array, you will need to have
> > > >entered the manufacturer in the SRes.Contact (this can be done also
> > > >via the StudyAnnotator Contact form) and you should have in
> > > >RAD3.OntologyEntry
> > > >an entry that describes the platform type. MGED has an ontology available
> > > >that we use to populate RAD3.OntologyEntry for terms referring to
> > > >platform types, namely the ontology for class=TechnologyType. The idea is
> > > >that in RAD3.OntologyEntry we store the instances of this class,
> > > >all of which have category set to 'TechnologyType". Similarly for
> > > >SubstrateType, an array entry in RAD3.Array has a substrate_type_id that
> > > >points to a term in RAd3.OntologyEntry describing the substrate type. For
> > > >this we use the instances of the MGED SubstrateType class, that is we
> > > >enter in RAD3.OntologyEntry an entry for each such instance, all having
> > > >category set to "SubstrateType". To get the MGED instatnces (individuals)
> > > >for TechnologyType and SubstrateType, you can go to
> > > >http://mged.sourceforge.net/ontologies/MGEDontology.php#TechnologyType and
> > > >http://mged.sourceforge.net/ontologies/MGEDontology.php#SubstrateType
> > > >I believe that, even though substrate_type_id is nullable in RAD3.Array,
> > > >the ArrayLoader might require that this value is provided in the arguments
> > > >anyway. Junmin, cc'ed in this email, can confirm this, as he is the author
> > > >of the ArrayLoader plugin.
> > > >Elisabetta
> > > >
> > > >On Fri, 14 Nov 2003, sam wang wrote:
> > > >
> > > > > Dear friends,
> > > > >
> > > > > after I download the new version of DbiDbHandle.pm,
> > > > > though it can go more steps, but there is no any data
> > > > > in my database, besides, there is an error "ERROR a
> > > > > VALID -- manufacturer_id must be on the commandline
> > > > > manufacturer_id = 0", I don't know the meaning and if
> > > > > it's the problem that I can't upload data? if so, how
> > > > > can I correct it?
> > > > >
> > > > > Thank you very much for your help!
> > > > > Sammy
> > > > >
> > > > >
> > > > > please check the following error message:
> > > > >
> > > > > [oracle@GUS oracle]$ ga GUS::RAD::Plugin::ArrayLoader
> > > > > --cfg_file
> > > > > /opt/gus/projects/RAD/DataLoad/config/ArrayLoader.cfg
> > > > > --data_file /home/oracle/RADDOTSdataset/assay5quan3
> > > > > --manufacturer_id 0 --platform_type_id 0
> > > > > Reading properties from
> > > > > /opt/gus/gus_home/config/GUS-PluginMgr.prop
> > > > > Reading properties from /home/oracle/.gus.properties
> > > > > DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> > > > > ::st are not setup, RootClass ignored at
> > > > > /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> > > > > line 152
> > > > > DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> > > > > ::st are not setup, RootClass ignored at
> > > > > /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> > > > > line 152
> > > > > DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> > > > > ::st are not setup, RootClass ignored at
> > > > > /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> > > > > line 152
> > > > > DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> > > > > ::st are not setup, RootClass ignored at
> > > > > /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> > > > > line 152
> > > > > Fri Nov 14 19:43:23 2003 ALGINVID 3
> > > > > Fri Nov 14 19:43:23 2003 ARGS algoinvo
> > > > > 1
> > > > > Fri Nov 14 19:43:23 2003 ARGS cfg_file
> > > > > /opt/gus/projects/RAD/DataLoad/config/ArrayLoader.cfg
> > > > > Fri Nov 14 19:43:23 2003 ARGS comment
> > > > > Fri Nov 14 19:43:23 2003 ARGS commit
> > > > > Fri Nov 14 19:43:23 2003 ARGS data_file
> > > > > /home/oracle/RADDOTSdataset/assay5quan3
> > > > > Fri Nov 14 19:43:23 2003 ARGS debug
> > > > > Fri Nov 14 19:43:23 2003 ARGS group
> > > > > Fri Nov 14 19:43:23 2003 ARGS gusconfigfile
> > > > > /home/oracle/.gus.properties
> > > > > Fri Nov 14 19:43:23 2003 ARGS
> > > > > manufacturer_id 0
> > > > > Fri Nov 14 19:43:23 2003 ARGS noWarning
> > > > > Fri Nov 14 19:43:23 2003 ARGS
> > > > > platform_type_id 0
> > > > > Fri Nov 14 19:43:23 2003 ARGS project
> > > > > Fri Nov 14 19:43:23 2003 ARGS protocol_id
> > > > > Fri Nov 14 19:43:23 2003 ARGS restart
> > > > > Fri Nov 14 19:43:23 2003 ARGS sqlVerbose
> > > > > Fri Nov 14 19:43:23 2003 ARGS
> > > > > substrate_type_id
> > > > > Fri Nov 14 19:43:23 2003 ARGS testnumber
> > > > > Fri Nov 14 19:43:23 2003 ARGS usage
> > > > > Fri Nov 14 19:43:23 2003 ARGS user
> > > > > Fri Nov 14 19:43:23 2003 ARGS verbose
> > > > > Fri Nov 14 19:43:23 2003 ARGS veryVerbose
> > > > > Fri Nov 14 19:43:23 2003 COMMIT commit off
> > > > > Fri Nov 14 19:43:23 2003 ERROR a VALID --
> > > > > manufacturer_id must be on the commandline
> > > > > manufacturer_id = 0
> > > > > Fri Nov 14 19:43:23 2003 RESULT
> > > > > [oracle@GUS oracle]$
> >
> >
> >
> >-------------------------------------------------------
> >This SF.Net email is sponsored by: Oracle 10g
> >Get certified on the hottest thing ever to hit the market... Oracle 10g.
> >Take an Oracle 10g class now, and we'll give you the exam FREE.
> >http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
> >_______________________________________________
> >Gusdev-gusdev mailing list
> >Gus...@li...
> >https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
> 

-- 
Elisabetta Manduchi

Computational Biology and Informatics Laboratory
Center for Bioinformatics
University of Pennsylvania
1428 Blockley Hall
423 Guardian Drive
Philadelphia, PA 19104-6021

phone: 215-573-4408
fax: 215 573-3111
email: man...@pc...
web: http://www.cbil.upenn.edu/~manduchi

---

[Gusdev-rad-issues] Re: [Gusdev-gusdev] about uploading RAD data problem (fwd)

[Elisabetta M. <man...@pc...>]

---------- Forwarded message ----------
Date: Tue, 4 May 2004 12:17:45 -0400 (EDT)
From: Elisabetta Manduchi <man...@pc...>
To: Sucheta Tripathy <su...@vb...>
Cc: sam wang <gus...@ya...>, gus...@li...
Subject: Re: [Gusdev-gusdev] about uploading RAD data problem


Hi Sucheta,

1. .CHP files. These are not in text format, but through MAS 5.0 one can 
generate their corresponding text files, i.e. the Metrics files. These can 
be loaded into RAD. More precisely minor parsing, of the Metrics files is 
needed in order to use them as input of the ArrayResultLoader 
(basically just the header needs to be parsed a bit, see doc for 
this plugin). 
We have recently written another plugin, the BathArrayResult loader, 
which loads in batch results for a group of assays. For this plugin
there is no need of parsing of Metrics files, all needed is an xml 
configuration file. We have not deposited this plugin into the Sanger cvs 
repository yet, but we can make available upon request in the meantime.

2. .CEL files. Up to recently these were in text format and in principle 
could be loaded into RAD using the ArrayResultLoader (and in fact, we have 
loaded a few in our instance of RAD). However we have now opted not to 
load the probe cell data into RAD. We keep the .CEL files on our 
filesystem and store in RAD a pointer to their location (see point 4 
below).

3. For image files like .DAT files (and similarly for .tif files), we 
store in RAD a pointer to their location, but and store the files  in our 
filesystem.

4. Summarizing how we currently deal with a given assay, with files 
myassay.data, myassay.CEL, myassay.CHP (and myassay_Metrics.txt):

a. The assay has one child in RAD3.Acquisition and the 
RAD3.Acquisition.uri of this points to the location of the .DAT file in 
our filesystem.

b. The acquisition in (a) has TWO children in RAD.Quantification (and 
these are related in RAD3.RelatedQuantification): one corresponds to the 
.CEL quantification (i.e. the probe cell analysis) and its uri points to 
the location of the .CEL file in our system, the other corresponds to the 
.CHP (probe set) analysis and its uri points to the location of the 
corresponding Metrics file (or .CHP file) in our file system.

c. We load the results corresponding to the .CHP file (from the Metrics 
file) into the AffymetrixMAS5 view of CompositeElementResultImp. Every 
entry in the latter will have quantification_id pointing to the 
quantification_id of the quantification in (b) corresponding to the probe 
set analysis (.CHP).

Hope this clarifies things a bit,
Elisabetta

On Tue, 4 May 2004, Sucheta Tripathy wrote:

> Hello Elisabetta and Sam,
> 
> I am new to RAD modules in GUS. Now we have some microarray data, which is 
> available in 3 different file formats(.cel,.dat,.CHP). Which of these 3 
> files can be uploaded to RAD tables. None of the tables are in text format.
> 
> Thanks in advance.
> 
> Sucheta
> 
> At 07:55 PM 11/16/2003 -0800, sam wang wrote:
> >Dear Elisabetta,
> >
> >thank you very much for your great help! now I am trying according to your 
> >suggestion, besides, it seems a little complicated :-)
> >
> >Thanks again,
> >Sammy
> >
> >Elisabetta Manduchi <man...@pc...> wrote:
> >Dear Sam,
> >regarding the part of your question referring to the manufacturer_id
> >error, if you look at schemaBrowser for RAD3.Array at
> >http://www.gusdb.org/cgi-bin/schemaBrowser?db=CBILBLD&table=RAD3::Array&path=RAD3::Array 
> >
> >you should see that manufacturer_id and platform_type_id are nonnullable.
> >Thus a valid manufacturer_id and a valid platform_type_id must be given
> >when running the ArrayLoader plugin (see also plugin documentation at
> >http://www.gusdb.org/documentation/plugins/GUS-RAD-Plugin-ArrayLoader.html).
> >This means that before you can load an array, you will need to have
> >entered the manufacturer in the SRes.Contact (this can be done also
> >via the StudyAnnotator Contact form) and you should have in 
> >RAD3.OntologyEntry
> >an entry that describes the platform type. MGED has an ontology available
> >that we use to populate RAD3.OntologyEntry for terms referring to
> >platform types, namely the ontology for class=TechnologyType. The idea is
> >that in RAD3.OntologyEntry we store the instances of this class,
> >all of which have category set to 'TechnologyType". Similarly for
> >SubstrateType, an array entry in RAD3.Array has a substrate_type_id that
> >points to a term in RAd3.OntologyEntry describing the substrate type. For
> >this we use the instances of the MGED SubstrateType class, that is we
> >enter in RAD3.OntologyEntry an entry for each such instance, all having
> >category set to "SubstrateType". To get the MGED instatnces (individuals)
> >for TechnologyType and SubstrateType, you can go to
> >http://mged.sourceforge.net/ontologies/MGEDontology.php#TechnologyType and
> >http://mged.sourceforge.net/ontologies/MGEDontology.php#SubstrateType
> >I believe that, even though substrate_type_id is nullable in RAD3.Array,
> >the ArrayLoader might require that this value is provided in the arguments
> >anyway. Junmin, cc'ed in this email, can confirm this, as he is the author
> >of the ArrayLoader plugin.
> >Elisabetta
> >
> >On Fri, 14 Nov 2003, sam wang wrote:
> >
> > > Dear friends,
> > >
> > > after I download the new version of DbiDbHandle.pm,
> > > though it can go more steps, but there is no any data
> > > in my database, besides, there is an error "ERROR a
> > > VALID -- manufacturer_id must be on the commandline
> > > manufacturer_id = 0", I don't know the meaning and if
> > > it's the problem that I can't upload data? if so, how
> > > can I correct it?
> > >
> > > Thank you very much for your help!
> > > Sammy
> > >
> > >
> > > please check the following error message:
> > >
> > > [oracle@GUS oracle]$ ga GUS::RAD::Plugin::ArrayLoader
> > > --cfg_file
> > > /opt/gus/projects/RAD/DataLoad/config/ArrayLoader.cfg
> > > --data_file /home/oracle/RADDOTSdataset/assay5quan3
> > > --manufacturer_id 0 --platform_type_id 0
> > > Reading properties from
> > > /opt/gus/gus_home/config/GUS-PluginMgr.prop
> > > Reading properties from /home/oracle/.gus.properties
> > > DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> > > ::st are not setup, RootClass ignored at
> > > /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> > > line 152
> > > DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> > > ::st are not setup, RootClass ignored at
> > > /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> > > line 152
> > > DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> > > ::st are not setup, RootClass ignored at
> > > /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> > > line 152
> > > DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> > > ::st are not setup, RootClass ignored at
> > > /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> > > line 152
> > > Fri Nov 14 19:43:23 2003 ALGINVID 3
> > > Fri Nov 14 19:43:23 2003 ARGS algoinvo
> > > 1
> > > Fri Nov 14 19:43:23 2003 ARGS cfg_file
> > > /opt/gus/projects/RAD/DataLoad/config/ArrayLoader.cfg
> > > Fri Nov 14 19:43:23 2003 ARGS comment
> > > Fri Nov 14 19:43:23 2003 ARGS commit
> > > Fri Nov 14 19:43:23 2003 ARGS data_file
> > > /home/oracle/RADDOTSdataset/assay5quan3
> > > Fri Nov 14 19:43:23 2003 ARGS debug
> > > Fri Nov 14 19:43:23 2003 ARGS group
> > > Fri Nov 14 19:43:23 2003 ARGS gusconfigfile
> > > /home/oracle/.gus.properties
> > > Fri Nov 14 19:43:23 2003 ARGS
> > > manufacturer_id 0
> > > Fri Nov 14 19:43:23 2003 ARGS noWarning
> > > Fri Nov 14 19:43:23 2003 ARGS
> > > platform_type_id 0
> > > Fri Nov 14 19:43:23 2003 ARGS project
> > > Fri Nov 14 19:43:23 2003 ARGS protocol_id
> > > Fri Nov 14 19:43:23 2003 ARGS restart
> > > Fri Nov 14 19:43:23 2003 ARGS sqlVerbose
> > > Fri Nov 14 19:43:23 2003 ARGS
> > > substrate_type_id
> > > Fri Nov 14 19:43:23 2003 ARGS testnumber
> > > Fri Nov 14 19:43:23 2003 ARGS usage
> > > Fri Nov 14 19:43:23 2003 ARGS user
> > > Fri Nov 14 19:43:23 2003 ARGS verbose
> > > Fri Nov 14 19:43:23 2003 ARGS veryVerbose
> > > Fri Nov 14 19:43:23 2003 COMMIT commit off
> > > Fri Nov 14 19:43:23 2003 ERROR a VALID --
> > > manufacturer_id must be on the commandline
> > > manufacturer_id = 0
> > > Fri Nov 14 19:43:23 2003 RESULT
> > > [oracle@GUS oracle]$

[Gusdev-rad-issues] New release of the RAD Study Annotator tarball

[Elisabetta M. <man...@pc...>]

The most recent release of the RAD Study-Annotator 
(StudyAnnotator_v1-1-0.tar.gz) is now available at 
http://www.cbil.upenn.edu/downloads/RAD/.
A tarball containing the suite of tutorials to accompany this release of 
the Study Annotator is also available at the same site 
(StudyAnnotatorTutorial_v1-1-0.tar.gz).
Elisabetta

-- 
Elisabetta Manduchi

Computational Biology and Informatics Laboratory
Center for Bioinformatics
University of Pennsylvania
1428 Blockley Hall
423 Guardian Drive
Philadelphia, PA 19104-6021

phone: 215-573-4408
fax: 215 573-3111
email: man...@pc...
web: http://www.cbil.upenn.edu/~manduchi

---

[Gusdev-rad-issues] Re: [Gusdev-gusdev] about uploading RAD data problem (fwd)

[Elisabetta M. <man...@pc...>]

---------- Forwarded message ----------
Date: Fri, 14 Nov 2003 21:24:06 -0500 (EST)
From: Elisabetta Manduchi <man...@pc...>
To: sam wang <gus...@ya...>
Cc: Trish Whetzel <wh...@pc...>, Junmin Liu <ju...@pc...>,
     gus...@li...
Subject: Re: [Gusdev-gusdev] about uploading RAD data problem


Dear Sam,
regarding the part of your question referring to the manufacturer_id 
error, if you look at schemaBrowser for RAD3.Array at 
http://www.gusdb.org/cgi-bin/schemaBrowser?db=CBILBLD&table=RAD3::Array&path=RAD3::Array
you should see that manufacturer_id and platform_type_id are nonnullable. 
Thus a valid manufacturer_id and a valid platform_type_id must be given 
when running the ArrayLoader plugin (see also plugin documentation at 
http://www.gusdb.org/documentation/plugins/GUS-RAD-Plugin-ArrayLoader.html). 
This means that before you can load an array, you will need to have 
entered the manufacturer in the SRes.Contact (this can be done also 
via the StudyAnnotator Contact form) and you should have in RAD3.OntologyEntry 
an entry that describes the platform type. MGED has an ontology available 
 that we use to populate RAD3.OntologyEntry for terms referring to 
platform types, namely the ontology for class=TechnologyType. The idea is 
that in RAD3.OntologyEntry we store the instances of this class, 
all of which have category set to 'TechnologyType". Similarly for 
SubstrateType, an array entry in RAD3.Array has a substrate_type_id that 
points to a term in RAd3.OntologyEntry describing the substrate type. For 
this we use the instances of the MGED SubstrateType class, that is we 
enter in RAD3.OntologyEntry an entry for each such instance, all having 
category set to "SubstrateType". To get the MGED instatnces (individuals) 
for TechnologyType and SubstrateType, you can go to 
http://mged.sourceforge.net/ontologies/MGEDontology.php#TechnologyType and 
http://mged.sourceforge.net/ontologies/MGEDontology.php#SubstrateType
I believe that, even though substrate_type_id is nullable in RAD3.Array, 
the ArrayLoader might require that this value is provided in the arguments 
anyway. Junmin, cc'ed in this email, can confirm this, as he is the author 
of the ArrayLoader plugin.
Elisabetta

On Fri, 14 Nov 2003, sam wang wrote:

> Dear friends,
> 
> after I download the new version of DbiDbHandle.pm,
> though it can go more steps, but there is no any data
> in my database, besides, there is an error "ERROR   a
> VALID -- manufacturer_id must be on the commandline
> manufacturer_id = 0", I don't know the meaning and if
> it's the problem that I can't upload data? if so, how
> can I correct it?
> 
> Thank you very much for your help!
> Sammy
> 
> 
> please check the following error message:
> 
> [oracle@GUS oracle]$ ga GUS::RAD::Plugin::ArrayLoader
> --cfg_file
> /opt/gus/projects/RAD/DataLoad/config/ArrayLoader.cfg
> --data_file /home/oracle/RADDOTSdataset/assay5quan3
> --manufacturer_id 0 --platform_type_id 0
> Reading properties from
> /opt/gus/gus_home/config/GUS-PluginMgr.prop
> Reading properties from /home/oracle/.gus.properties
> DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> ::st are not setup, RootClass ignored at
> /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> line 152
> DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> ::st are not setup, RootClass ignored at
> /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> line 152
> DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> ::st are not setup, RootClass ignored at
> /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> line 152
> DBI subclasses 'GUS::ObjRelP::DbiDbHandle::db' and
> ::st are not setup, RootClass ignored at
> /opt/gus/gus_home/lib/perl/GUS/ObjRelP/DbiDatabase.pm
> line 152
> Fri Nov 14 19:43:23 2003        ALGINVID        3
> Fri Nov 14 19:43:23 2003        ARGS    algoinvo      
>  1
> Fri Nov 14 19:43:23 2003        ARGS    cfg_file      
>  /opt/gus/projects/RAD/DataLoad/config/ArrayLoader.cfg
> Fri Nov 14 19:43:23 2003        ARGS    comment
> Fri Nov 14 19:43:23 2003        ARGS    commit
> Fri Nov 14 19:43:23 2003        ARGS    data_file     
>  /home/oracle/RADDOTSdataset/assay5quan3
> Fri Nov 14 19:43:23 2003        ARGS    debug
> Fri Nov 14 19:43:23 2003        ARGS    group
> Fri Nov 14 19:43:23 2003        ARGS    gusconfigfile 
>  /home/oracle/.gus.properties
> Fri Nov 14 19:43:23 2003        ARGS   
> manufacturer_id 0
> Fri Nov 14 19:43:23 2003        ARGS    noWarning
> Fri Nov 14 19:43:23 2003        ARGS   
> platform_type_id        0
> Fri Nov 14 19:43:23 2003        ARGS    project
> Fri Nov 14 19:43:23 2003        ARGS    protocol_id
> Fri Nov 14 19:43:23 2003        ARGS    restart
> Fri Nov 14 19:43:23 2003        ARGS    sqlVerbose
> Fri Nov 14 19:43:23 2003        ARGS   
> substrate_type_id
> Fri Nov 14 19:43:23 2003        ARGS    testnumber
> Fri Nov 14 19:43:23 2003        ARGS    usage
> Fri Nov 14 19:43:23 2003        ARGS    user
> Fri Nov 14 19:43:23 2003        ARGS    verbose
> Fri Nov 14 19:43:23 2003        ARGS    veryVerbose
> Fri Nov 14 19:43:23 2003        COMMIT  commit off
> Fri Nov 14 19:43:23 2003        ERROR   a VALID --
> manufacturer_id must be on the commandline
> manufacturer_id = 0
> Fri Nov 14 19:43:23 2003        RESULT
> [oracle@GUS oracle]$
> 
> 
> 
> __________________________________
> Do you Yahoo!?
> Protect your identity with Yahoo! Mail AddressGuard
> http://antispam.yahoo.com/whatsnewfree
> 
> 
> -------------------------------------------------------
> This SF. Net email is sponsored by: GoToMyPC
> GoToMyPC is the fast, easy and secure way to access your computer from
> any Web browser or wireless device. Click here to Try it Free!
> https://www.gotomypc.com/tr/OSDN/AW/Q4_2003/t/g22lp?Target=mm/g22lp.tmpl
> _______________________________________________
> Gusdev-gusdev mailing list
> Gus...@li...
> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
> 

-- 
Elisabetta Manduchi

Computational Biology and Informatics Laboratory
Center for Bioinformatics
University of Pennsylvania
1428 Blockley Hall
423 Guardian Drive
Philadelphia, PA 19104-6021

phone: 215-573-4408
fax: 215 573-3111
email: man...@pc...
web: http://www.cbil.upenn.edu/~manduchi

---