gusdev-gusdev — Topics concerning GUS development

Re: [Gusdev-gusdev] LoadAnnotatedSeqs plugin underway

[Steve F. <sfi...@pc...>]

paul-  see in line

steve

Paul Mooney wrote:

>
> On 10 Dec 2004, at 12:52, Steve Fischer wrote:
>
>> paul-
>>
>> ok, i see.
>>
>> are there any other examples besides curation in which you have 
>> placed structured data in qualifiers?     are there examples of 
>> standard embl qualifiers in which you expect to find structured data 
>> and parse it?
>>
>
> After talking with Arnaud it seems we can take each 
> qualifier/structured field and create a new feature, with each one of 
> its qualifiers holding one piece of data. This would fit into your 
> mapping scheme.
>
ok.  great.  i was wondering about that.

so does that mean that we can expect that no qualifiers will contain 
structured data that needs to be parsed?

>> in the case of curation, where do you put that info in GUS?
>
>
> It will probably end up as a note, for now at least.
>
>>
>> about systematic_ids, i understand what you've said.   one thing 
>> though.  how do they relate to gene names?
>
>
ok, but, what i'm driving at is that the unflattener uses gene name 
(/gene=) to decide what features go together in one gene model.   
really, it wouldn't matter what the value of the /gene= is, as long as 
it is identical for all features that belong to the gene.   is that 
consistent with your use of /gene?

> They are the gene names :)
> Standard EMBL uses a /gene qualifier for the gene symbol and 
> /standard_name for the human readable name.
> During sequencing and annotation using a single /gene conveys no 
> meaning as to how stable/temporary the ID is.
>
>> steve
>>
>> Paul Mooney wrote:
>>
>>>
>>> On 9 Dec 2004, at 23:21, Steve Fischer wrote:
>>>
>>>> paul-
>>>>
>>>> let me start digesting this by email.
>>>>
>>>> about your extensions to EMBL.  the bioPerl model we are parsing 
>>>> into is based on generic features, tags and annotation.  as long as 
>>>> the extensions can be parsed into those objects we're half way 
>>>> there.   are the extensions syntactically consistent w/ standard 
>>>> embl files, but varying only in the particulars of what the data is 
>>>> called?
>>>
>>>
>>>
>>> We have additional qualifiers with values. The values hold 
>>> structured information (say key=value pairs).
>>> Bioperl will quite happily parse them into tags and values.
>>> What controls the mapping of a tag to a GUS objects(s)?
>>> What parses the structured information out to populate the object(s) 
>>> and fill in the objects fields (which is another mapping)?
>>>
>>> Something like this non-EMBL standard entry, curation, has several 
>>> values in a fixed field format;
>>>
>>>     /curation="name; origin; date; permission; type; dbref; notes ..."
>>> i.e.
>>>     /curation="Matt Berriman; genedb; 20020128; public; comment"
>>>
>>> How do we specify where to put this in GUS? It's very PSU specific. 
>>> Perhaps some sort of hook with specifying some perl code elsewhere 
>>> to handle it?
>>> We currently store GO annotation in EMBL like this;
>>>
>>>     /GO="aspect=process; GOid=GO:0006810; term=transport; 
>>> evidence=ISS; db_xref=GOC:unpublished; with=SPTR:Q9UQ36; date=20001122"
>>>
>>> as EMBL only has the format /db_xref="GO:00123" but I hope there is 
>>> a GO flat file loader so we don't have to worry about this in the 
>>> future.
>>>
>>>> about building the hierarchy.  if you looked at the bioperl api for 
>>>> the unflattener, you'd see that its unflattening uses gene name as 
>>>> a clue to deciding what features go together in a particular gene 
>>>> model.
>>>>
>>>> can gene name be relied upon to identify all the features that are 
>>>> associated with this gene?
>>>
>>>
>>>
>>> You can switch to use any qualifier you like to identify groups, but 
>>> you can only specify *one*.
>>> We can have 2 :)
>>> In the same sequence a gene may be identified by systematic_id.
>>> Another gene in the same sequence maybe identified by 
>>> temporary_systematic_id.
>>> Eventually all genes will get a systematic_id but not straight away.
>>>
>>> In theory it should be easy to modify the flattener to use a 'best 
>>> name first' policy.
>>>
>>> For TIGR XML you'd have PUB_LOCUS and LUCUS as the best names, in 
>>> that order. Their too mix identifiers but since the XML already has 
>>> a hierarchy you might get away with it????
>>>
>>>
>>>> finally, about the GO stuff, yes, we can probably reuse your code.
>>>>
>>>> steve
>>>>
>>>>
>>>> Paul Mooney wrote:
>>>>
>>>>>
>>>>> On 9 Dec 2004, at 19:31, Steve Fischer wrote:
>>>>>
>>>>>> paul-
>>>>>>
>>>>>> hey.  do you want to set up a time to chat so i can catch you up 
>>>>>> on what we have in mind?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> At the moment I'm curious how much can be achieved via a generic 
>>>>> plugin. I think the plugin will need plugin's to do specialised 
>>>>> parts :) However I'd be glad to give my assistance to the effort. 
>>>>> Below are my random thoughts I've just had on the matter;
>>>>>
>>>>>
>>>>> Here at the PSU we store an awful lot of info that can not be 
>>>>> stored in a standard EMBL file, hence we have extended it to fit 
>>>>> out own needs. As an example we use several name qualifiers for 
>>>>> genes;
>>>>>
>>>>>     . systematic_id           - the name cast in stone
>>>>>     . temporary_systematic_id - the name as it is currently known
>>>>>     . previous_systematic_id  - as it was known
>>>>>     . gene                    - EMBL standard qualifier
>>>>>
>>>>> Hence just trying to unflatten the EMBL file is tricky because 
>>>>> systematic and temporary_sysetmatic_ids are mixed in the same 
>>>>> sequence, hence building the hierarchy would need specialised 
>>>>> code. TIGR XML has the same issue though so maybe its not too 
>>>>> specialised after all :/ (PUB_LOCUS and LOCUS has a direct mapping 
>>>>> to systematic_id and temporary_systematic_id).
>>>>>
>>>>> Something like this entry;
>>>>>     /curation="name; origin; date; permission; type; dbref; notes 
>>>>> ..."
>>>>> i.e.
>>>>>     /curation="Matt Berriman; genedb; 20020128; public; comment"
>>>>> is unique to the PSU and I'm not sure where it fits in GUS.
>>>>>
>>>>> However;
>>>>>
>>>>> I have code that creates GO entries - supply a high level function 
>>>>> with all the standard GO fields and it creates the 5 rows (?) in 
>>>>> the different tables as required. This is definitely something 
>>>>> that can be shared across centres, perhaps in a code library. All 
>>>>> your code has to do is parse out the GO fields from the data. No 
>>>>> reason why it couldn't accept a GO Bioperl object (I presume one 
>>>>> exists).
>>>>>
>>>>> Perhaps the parsing needs to a super class for each data source 
>>>>> and then sub-classed by each centre?
>>>>>
>>>>> Ok, enough ramblings. Does any of this make sense?
>>>>> Paul.
>>>>>
>>>>>> steve
>>>>>>
>>>>>> Chris Stoeckert wrote:
>>>>>>
>>>>>>> Hi Steve,
>>>>>>> Thanks for putting this out on gusdev. Marie-Adele indicated 
>>>>>>> that Paul Mooney was very interested in this and I will likely 
>>>>>>> meet with him about this when I visit in January. Please include 
>>>>>>> him in email correspondence when not addressed to the general 
>>>>>>> gusdev list.
>>>>>>> Thanks,
>>>>>>> Chris
>>>>>>>
>>>>>>> On Dec 9, 2004, at 2:11 PM, Steve Fischer wrote:
>>>>>>>
>>>>>>>> folks-
>>>>>>>>
>>>>>>>> the UGA folks and CBIL folks have started collaborating on a 
>>>>>>>> new plugin called LoadAnnotatedSeqs.   It will use BioPerl to 
>>>>>>>> parse the input data.
>>>>>>>>
>>>>>>>> We expect it to take annotated sequences (NA at first) in 
>>>>>>>> genbank, tigr xml and embl formats (plus any others supported 
>>>>>>>> by the bioPerl parser).
>>>>>>>>
>>>>>>>> It will take an XML file that describes the mapping from the 
>>>>>>>> input features to GUS features, and SO features.
>>>>>>>> It will also hard code special cases to handle qualifer data 
>>>>>>>> that is distributed to tables outside of the NAFeature tables.
>>>>>>>>
>>>>>>>> For our projects we will be developing a mapping that unifies 
>>>>>>>> the semantics of the data we are getting from our different 
>>>>>>>> sources and formats.
>>>>>>>> (we plan to work with the PSU folks to incorporate the 
>>>>>>>> knowledge they have acquired in their work to make an EMBL parser)
>>>>>>>>
>>>>>>>> ideas and suggestions are encouraged.
>>>>>>>>
>>>>>>>> steve
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -------------------------------------------------------
>>>>>>>> SF email is sponsored by - The IT Product Guide
>>>>>>>> Read honest & candid reviews on hundreds of IT Products from 
>>>>>>>> real users.
>>>>>>>> Discover which products truly live up to the hype. Start 
>>>>>>>> reading now. http://productguide.itmanagersjournal.com/
>>>>>>>> _______________________________________________
>>>>>>>> Gusdev-gusdev mailing list
>>>>>>>> Gus...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>

Re: [Gusdev-gusdev] installing gus

[Jinal J. <jjh...@vb...>]

Do you have  $PROJECT_HOME/install/bin in your path?


On Friday 10 December 2004 12:16 am, marc jackson wrote:
> In the documentation, I keep seeing reference to a "build" command.
>
> build GUS install -append -skipJaaveCompiling     [ for example]
>
> This is not a unix/linux command I'm aware of.  I installed the binary
> version of ant from ant.apache.org and I can't find this build command.
>
> mmm ..... Where does this come from?
>
> Regards and Thanks,
>
> Marc

Re: [Gusdev-gusdev] LoadAnnotatedSeqs plugin underway

[Paul M. <pj...@sa...>]

On 10 Dec 2004, at 12:52, Steve Fischer wrote:

> paul-
>
> ok, i see.
>
> are there any other examples besides curation in which you have placed 
> structured data in qualifiers?     are there examples of standard embl 
> qualifiers in which you expect to find structured data and parse it?
>

After talking with Arnaud it seems we can take each 
qualifier/structured field and create a new feature, with each one of 
its qualifiers holding one piece of data. This would fit into your 
mapping scheme.

> in the case of curation, where do you put that info in GUS?

It will probably end up as a note, for now at least.

>
> about systematic_ids, i understand what you've said.   one thing 
> though.  how do they relate to gene names?

They are the gene names :)
Standard EMBL uses a /gene qualifier for the gene symbol and 
/standard_name for the human readable name.
During sequencing and annotation using a single /gene conveys no 
meaning as to how stable/temporary the ID is.

> steve
>
> Paul Mooney wrote:
>
>>
>> On 9 Dec 2004, at 23:21, Steve Fischer wrote:
>>
>>> paul-
>>>
>>> let me start digesting this by email.
>>>
>>> about your extensions to EMBL.  the bioPerl model we are parsing 
>>> into is based on generic features, tags and annotation.  as long as 
>>> the extensions can be parsed into those objects we're half way 
>>> there.   are the extensions syntactically consistent w/ standard 
>>> embl files, but varying only in the particulars of what the data is 
>>> called?
>>
>>
>> We have additional qualifiers with values. The values hold structured 
>> information (say key=value pairs).
>> Bioperl will quite happily parse them into tags and values.
>> What controls the mapping of a tag to a GUS objects(s)?
>> What parses the structured information out to populate the object(s) 
>> and fill in the objects fields (which is another mapping)?
>>
>> Something like this non-EMBL standard entry, curation, has several 
>> values in a fixed field format;
>>
>>     /curation="name; origin; date; permission; type; dbref; notes ..."
>> i.e.
>>     /curation="Matt Berriman; genedb; 20020128; public; comment"
>>
>> How do we specify where to put this in GUS? It's very PSU specific. 
>> Perhaps some sort of hook with specifying some perl code elsewhere to 
>> handle it?
>> We currently store GO annotation in EMBL like this;
>>
>>     /GO="aspect=process; GOid=GO:0006810; term=transport; 
>> evidence=ISS; db_xref=GOC:unpublished; with=SPTR:Q9UQ36; 
>> date=20001122"
>>
>> as EMBL only has the format /db_xref="GO:00123" but I hope there is a 
>> GO flat file loader so we don't have to worry about this in the 
>> future.
>>
>>> about building the hierarchy.  if you looked at the bioperl api for 
>>> the unflattener, you'd see that its unflattening uses gene name as a 
>>> clue to deciding what features go together in a particular gene 
>>> model.
>>>
>>> can gene name be relied upon to identify all the features that are 
>>> associated with this gene?
>>
>>
>> You can switch to use any qualifier you like to identify groups, but 
>> you can only specify *one*.
>> We can have 2 :)
>> In the same sequence a gene may be identified by systematic_id.
>> Another gene in the same sequence maybe identified by 
>> temporary_systematic_id.
>> Eventually all genes will get a systematic_id but not straight away.
>>
>> In theory it should be easy to modify the flattener to use a 'best 
>> name first' policy.
>>
>> For TIGR XML you'd have PUB_LOCUS and LUCUS as the best names, in 
>> that order. Their too mix identifiers but since the XML already has a 
>> hierarchy you might get away with it????
>>
>>
>>> finally, about the GO stuff, yes, we can probably reuse your code.
>>>
>>> steve
>>>
>>>
>>> Paul Mooney wrote:
>>>
>>>>
>>>> On 9 Dec 2004, at 19:31, Steve Fischer wrote:
>>>>
>>>>> paul-
>>>>>
>>>>> hey.  do you want to set up a time to chat so i can catch you up 
>>>>> on what we have in mind?
>>>>
>>>>
>>>>
>>>>
>>>> At the moment I'm curious how much can be achieved via a generic 
>>>> plugin. I think the plugin will need plugin's to do specialised 
>>>> parts :) However I'd be glad to give my assistance to the effort. 
>>>> Below are my random thoughts I've just had on the matter;
>>>>
>>>>
>>>> Here at the PSU we store an awful lot of info that can not be 
>>>> stored in a standard EMBL file, hence we have extended it to fit 
>>>> out own needs. As an example we use several name qualifiers for 
>>>> genes;
>>>>
>>>>     . systematic_id           - the name cast in stone
>>>>     . temporary_systematic_id - the name as it is currently known
>>>>     . previous_systematic_id  - as it was known
>>>>     . gene                    - EMBL standard qualifier
>>>>
>>>> Hence just trying to unflatten the EMBL file is tricky because 
>>>> systematic and temporary_sysetmatic_ids are mixed in the same 
>>>> sequence, hence building the hierarchy would need specialised code. 
>>>> TIGR XML has the same issue though so maybe its not too specialised 
>>>> after all :/ (PUB_LOCUS and LOCUS has a direct mapping to 
>>>> systematic_id and temporary_systematic_id).
>>>>
>>>> Something like this entry;
>>>>     /curation="name; origin; date; permission; type; dbref; notes 
>>>> ..."
>>>> i.e.
>>>>     /curation="Matt Berriman; genedb; 20020128; public; comment"
>>>> is unique to the PSU and I'm not sure where it fits in GUS.
>>>>
>>>> However;
>>>>
>>>> I have code that creates GO entries - supply a high level function 
>>>> with all the standard GO fields and it creates the 5 rows (?) in 
>>>> the different tables as required. This is definitely something that 
>>>> can be shared across centres, perhaps in a code library. All your 
>>>> code has to do is parse out the GO fields from the data. No reason 
>>>> why it couldn't accept a GO Bioperl object (I presume one exists).
>>>>
>>>> Perhaps the parsing needs to a super class for each data source and 
>>>> then sub-classed by each centre?
>>>>
>>>> Ok, enough ramblings. Does any of this make sense?
>>>> Paul.
>>>>
>>>>> steve
>>>>>
>>>>> Chris Stoeckert wrote:
>>>>>
>>>>>> Hi Steve,
>>>>>> Thanks for putting this out on gusdev. Marie-Adele indicated that 
>>>>>> Paul Mooney was very interested in this and I will likely meet 
>>>>>> with him about this when I visit in January. Please include him 
>>>>>> in email correspondence when not addressed to the general gusdev 
>>>>>> list.
>>>>>> Thanks,
>>>>>> Chris
>>>>>>
>>>>>> On Dec 9, 2004, at 2:11 PM, Steve Fischer wrote:
>>>>>>
>>>>>>> folks-
>>>>>>>
>>>>>>> the UGA folks and CBIL folks have started collaborating on a new 
>>>>>>> plugin called LoadAnnotatedSeqs.   It will use BioPerl to parse 
>>>>>>> the input data.
>>>>>>>
>>>>>>> We expect it to take annotated sequences (NA at first) in 
>>>>>>> genbank, tigr xml and embl formats (plus any others supported by 
>>>>>>> the bioPerl parser).
>>>>>>>
>>>>>>> It will take an XML file that describes the mapping from the 
>>>>>>> input features to GUS features, and SO features.
>>>>>>> It will also hard code special cases to handle qualifer data 
>>>>>>> that is distributed to tables outside of the NAFeature tables.
>>>>>>>
>>>>>>> For our projects we will be developing a mapping that unifies 
>>>>>>> the semantics of the data we are getting from our different 
>>>>>>> sources and formats.
>>>>>>> (we plan to work with the PSU folks to incorporate the knowledge 
>>>>>>> they have acquired in their work to make an EMBL parser)
>>>>>>>
>>>>>>> ideas and suggestions are encouraged.
>>>>>>>
>>>>>>> steve
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -------------------------------------------------------
>>>>>>> SF email is sponsored by - The IT Product Guide
>>>>>>> Read honest & candid reviews on hundreds of IT Products from 
>>>>>>> real users.
>>>>>>> Discover which products truly live up to the hype. Start reading 
>>>>>>> now. http://productguide.itmanagersjournal.com/
>>>>>>> _______________________________________________
>>>>>>> Gusdev-gusdev mailing list
>>>>>>> Gus...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>

Re: [Gusdev-gusdev] LoadAnnotatedSeqs plugin underway

[Steve F. <sfi...@pc...>]

paul-

ok, i see.

are there any other examples besides curation in which you have placed 
structured data in qualifiers?     are there examples of standard embl 
qualifiers in which you expect to find structured data and parse it?

in the case of curation, where do you put that info in GUS?

about systematic_ids, i understand what you've said.   one thing 
though.  how do they relate to gene names? 

steve

Paul Mooney wrote:

>
> On 9 Dec 2004, at 23:21, Steve Fischer wrote:
>
>> paul-
>>
>> let me start digesting this by email.
>>
>> about your extensions to EMBL.  the bioPerl model we are parsing into 
>> is based on generic features, tags and annotation.  as long as the 
>> extensions can be parsed into those objects we're half way there.   
>> are the extensions syntactically consistent w/ standard embl files, 
>> but varying only in the particulars of what the data is called?
>
>
> We have additional qualifiers with values. The values hold structured 
> information (say key=value pairs).
> Bioperl will quite happily parse them into tags and values.
> What controls the mapping of a tag to a GUS objects(s)?
> What parses the structured information out to populate the object(s) 
> and fill in the objects fields (which is another mapping)?
>
> Something like this non-EMBL standard entry, curation, has several 
> values in a fixed field format;
>
>     /curation="name; origin; date; permission; type; dbref; notes ..."
> i.e.
>     /curation="Matt Berriman; genedb; 20020128; public; comment"
>
> How do we specify where to put this in GUS? It's very PSU specific. 
> Perhaps some sort of hook with specifying some perl code elsewhere to 
> handle it?
> We currently store GO annotation in EMBL like this;
>
>     /GO="aspect=process; GOid=GO:0006810; term=transport; 
> evidence=ISS; db_xref=GOC:unpublished; with=SPTR:Q9UQ36; date=20001122"
>
> as EMBL only has the format /db_xref="GO:00123" but I hope there is a 
> GO flat file loader so we don't have to worry about this in the future.
>
>> about building the hierarchy.  if you looked at the bioperl api for 
>> the unflattener, you'd see that its unflattening uses gene name as a 
>> clue to deciding what features go together in a particular gene model.
>>
>> can gene name be relied upon to identify all the features that are 
>> associated with this gene?
>
>
> You can switch to use any qualifier you like to identify groups, but 
> you can only specify *one*.
> We can have 2 :)
> In the same sequence a gene may be identified by systematic_id.
> Another gene in the same sequence maybe identified by 
> temporary_systematic_id.
> Eventually all genes will get a systematic_id but not straight away.
>
> In theory it should be easy to modify the flattener to use a 'best 
> name first' policy.
>
> For TIGR XML you'd have PUB_LOCUS and LUCUS as the best names, in that 
> order. Their too mix identifiers but since the XML already has a 
> hierarchy you might get away with it????
>
>
>> finally, about the GO stuff, yes, we can probably reuse your code.
>>
>> steve
>>
>>
>> Paul Mooney wrote:
>>
>>>
>>> On 9 Dec 2004, at 19:31, Steve Fischer wrote:
>>>
>>>> paul-
>>>>
>>>> hey.  do you want to set up a time to chat so i can catch you up on 
>>>> what we have in mind?
>>>
>>>
>>>
>>>
>>> At the moment I'm curious how much can be achieved via a generic 
>>> plugin. I think the plugin will need plugin's to do specialised 
>>> parts :) However I'd be glad to give my assistance to the effort. 
>>> Below are my random thoughts I've just had on the matter;
>>>
>>>
>>> Here at the PSU we store an awful lot of info that can not be stored 
>>> in a standard EMBL file, hence we have extended it to fit out own 
>>> needs. As an example we use several name qualifiers for genes;
>>>
>>>     . systematic_id           - the name cast in stone
>>>     . temporary_systematic_id - the name as it is currently known
>>>     . previous_systematic_id  - as it was known
>>>     . gene                    - EMBL standard qualifier
>>>
>>> Hence just trying to unflatten the EMBL file is tricky because 
>>> systematic and temporary_sysetmatic_ids are mixed in the same 
>>> sequence, hence building the hierarchy would need specialised code. 
>>> TIGR XML has the same issue though so maybe its not too specialised 
>>> after all :/ (PUB_LOCUS and LOCUS has a direct mapping to 
>>> systematic_id and temporary_systematic_id).
>>>
>>> Something like this entry;
>>>     /curation="name; origin; date; permission; type; dbref; notes ..."
>>> i.e.
>>>     /curation="Matt Berriman; genedb; 20020128; public; comment"
>>> is unique to the PSU and I'm not sure where it fits in GUS.
>>>
>>> However;
>>>
>>> I have code that creates GO entries - supply a high level function 
>>> with all the standard GO fields and it creates the 5 rows (?) in the 
>>> different tables as required. This is definitely something that can 
>>> be shared across centres, perhaps in a code library. All your code 
>>> has to do is parse out the GO fields from the data. No reason why it 
>>> couldn't accept a GO Bioperl object (I presume one exists).
>>>
>>> Perhaps the parsing needs to a super class for each data source and 
>>> then sub-classed by each centre?
>>>
>>> Ok, enough ramblings. Does any of this make sense?
>>> Paul.
>>>
>>>> steve
>>>>
>>>> Chris Stoeckert wrote:
>>>>
>>>>> Hi Steve,
>>>>> Thanks for putting this out on gusdev. Marie-Adele indicated that 
>>>>> Paul Mooney was very interested in this and I will likely meet 
>>>>> with him about this when I visit in January. Please include him in 
>>>>> email correspondence when not addressed to the general gusdev list.
>>>>> Thanks,
>>>>> Chris
>>>>>
>>>>> On Dec 9, 2004, at 2:11 PM, Steve Fischer wrote:
>>>>>
>>>>>> folks-
>>>>>>
>>>>>> the UGA folks and CBIL folks have started collaborating on a new 
>>>>>> plugin called LoadAnnotatedSeqs.   It will use BioPerl to parse 
>>>>>> the input data.
>>>>>>
>>>>>> We expect it to take annotated sequences (NA at first) in 
>>>>>> genbank, tigr xml and embl formats (plus any others supported by 
>>>>>> the bioPerl parser).
>>>>>>
>>>>>> It will take an XML file that describes the mapping from the 
>>>>>> input features to GUS features, and SO features.
>>>>>> It will also hard code special cases to handle qualifer data that 
>>>>>> is distributed to tables outside of the NAFeature tables.
>>>>>>
>>>>>> For our projects we will be developing a mapping that unifies the 
>>>>>> semantics of the data we are getting from our different sources 
>>>>>> and formats.
>>>>>> (we plan to work with the PSU folks to incorporate the knowledge 
>>>>>> they have acquired in their work to make an EMBL parser)
>>>>>>
>>>>>> ideas and suggestions are encouraged.
>>>>>>
>>>>>> steve
>>>>>>
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------
>>>>>> SF email is sponsored by - The IT Product Guide
>>>>>> Read honest & candid reviews on hundreds of IT Products from real 
>>>>>> users.
>>>>>> Discover which products truly live up to the hype. Start reading 
>>>>>> now. http://productguide.itmanagersjournal.com/
>>>>>> _______________________________________________
>>>>>> Gusdev-gusdev mailing list
>>>>>> Gus...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>>>>
>>>>>
>>>>>
>>>>
>>

Re: [Gusdev-gusdev] LoadAnnotatedSeqs plugin underway

[Paul M. <pj...@sa...>]

On 9 Dec 2004, at 23:21, Steve Fischer wrote:

> paul-
>
> let me start digesting this by email.
>
> about your extensions to EMBL.  the bioPerl model we are parsing into 
> is based on generic features, tags and annotation.  as long as the 
> extensions can be parsed into those objects we're half way there.   
> are the extensions syntactically consistent w/ standard embl files, 
> but varying only in the particulars of what the data is called?

We have additional qualifiers with values. The values hold structured 
information (say key=value pairs).
Bioperl will quite happily parse them into tags and values.
What controls the mapping of a tag to a GUS objects(s)?
What parses the structured information out to populate the object(s) 
and fill in the objects fields (which is another mapping)?

Something like this non-EMBL standard entry, curation, has several 
values in a fixed field format;

     /curation="name; origin; date; permission; type; dbref; notes ..."
i.e.
     /curation="Matt Berriman; genedb; 20020128; public; comment"

How do we specify where to put this in GUS? It's very PSU specific. 
Perhaps some sort of hook with specifying some perl code elsewhere to 
handle it?
We currently store GO annotation in EMBL like this;

     /GO="aspect=process; GOid=GO:0006810; term=transport; evidence=ISS; 
db_xref=GOC:unpublished; with=SPTR:Q9UQ36; date=20001122"

as EMBL only has the format /db_xref="GO:00123" but I hope there is a 
GO flat file loader so we don't have to worry about this in the future.

> about building the hierarchy.  if you looked at the bioperl api for 
> the unflattener, you'd see that its unflattening uses gene name as a 
> clue to deciding what features go together in a particular gene model.
>
> can gene name be relied upon to identify all the features that are 
> associated with this gene?

You can switch to use any qualifier you like to identify groups, but 
you can only specify *one*.
We can have 2 :)
In the same sequence a gene may be identified by systematic_id.
Another gene in the same sequence maybe identified by 
temporary_systematic_id.
Eventually all genes will get a systematic_id but not straight away.

In theory it should be easy to modify the flattener to use a 'best name 
first' policy.

For TIGR XML you'd have PUB_LOCUS and LUCUS as the best names, in that 
order. Their too mix identifiers but since the XML already has a 
hierarchy you might get away with it????


> finally, about the GO stuff, yes, we can probably reuse your code.
>
> steve
>
>
> Paul Mooney wrote:
>
>>
>> On 9 Dec 2004, at 19:31, Steve Fischer wrote:
>>
>>> paul-
>>>
>>> hey.  do you want to set up a time to chat so i can catch you up on 
>>> what we have in mind?
>>
>>
>>
>> At the moment I'm curious how much can be achieved via a generic 
>> plugin. I think the plugin will need plugin's to do specialised parts 
>> :) However I'd be glad to give my assistance to the effort. Below are 
>> my random thoughts I've just had on the matter;
>>
>>
>> Here at the PSU we store an awful lot of info that can not be stored 
>> in a standard EMBL file, hence we have extended it to fit out own 
>> needs. As an example we use several name qualifiers for genes;
>>
>>     . systematic_id           - the name cast in stone
>>     . temporary_systematic_id - the name as it is currently known
>>     . previous_systematic_id  - as it was known
>>     . gene                    - EMBL standard qualifier
>>
>> Hence just trying to unflatten the EMBL file is tricky because 
>> systematic and temporary_sysetmatic_ids are mixed in the same 
>> sequence, hence building the hierarchy would need specialised code. 
>> TIGR XML has the same issue though so maybe its not too specialised 
>> after all :/ (PUB_LOCUS and LOCUS has a direct mapping to 
>> systematic_id and temporary_systematic_id).
>>
>> Something like this entry;
>>     /curation="name; origin; date; permission; type; dbref; notes ..."
>> i.e.
>>     /curation="Matt Berriman; genedb; 20020128; public; comment"
>> is unique to the PSU and I'm not sure where it fits in GUS.
>>
>> However;
>>
>> I have code that creates GO entries - supply a high level function 
>> with all the standard GO fields and it creates the 5 rows (?) in the 
>> different tables as required. This is definitely something that can 
>> be shared across centres, perhaps in a code library. All your code 
>> has to do is parse out the GO fields from the data. No reason why it 
>> couldn't accept a GO Bioperl object (I presume one exists).
>>
>> Perhaps the parsing needs to a super class for each data source and 
>> then sub-classed by each centre?
>>
>> Ok, enough ramblings. Does any of this make sense?
>> Paul.
>>
>>> steve
>>>
>>> Chris Stoeckert wrote:
>>>
>>>> Hi Steve,
>>>> Thanks for putting this out on gusdev. Marie-Adele indicated that 
>>>> Paul Mooney was very interested in this and I will likely meet with 
>>>> him about this when I visit in January. Please include him in email 
>>>> correspondence when not addressed to the general gusdev list.
>>>> Thanks,
>>>> Chris
>>>>
>>>> On Dec 9, 2004, at 2:11 PM, Steve Fischer wrote:
>>>>
>>>>> folks-
>>>>>
>>>>> the UGA folks and CBIL folks have started collaborating on a new 
>>>>> plugin called LoadAnnotatedSeqs.   It will use BioPerl to parse 
>>>>> the input data.
>>>>>
>>>>> We expect it to take annotated sequences (NA at first) in genbank, 
>>>>> tigr xml and embl formats (plus any others supported by the 
>>>>> bioPerl parser).
>>>>>
>>>>> It will take an XML file that describes the mapping from the input 
>>>>> features to GUS features, and SO features.
>>>>> It will also hard code special cases to handle qualifer data that 
>>>>> is distributed to tables outside of the NAFeature tables.
>>>>>
>>>>> For our projects we will be developing a mapping that unifies the 
>>>>> semantics of the data we are getting from our different sources 
>>>>> and formats.
>>>>> (we plan to work with the PSU folks to incorporate the knowledge 
>>>>> they have acquired in their work to make an EMBL parser)
>>>>>
>>>>> ideas and suggestions are encouraged.
>>>>>
>>>>> steve
>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> SF email is sponsored by - The IT Product Guide
>>>>> Read honest & candid reviews on hundreds of IT Products from real 
>>>>> users.
>>>>> Discover which products truly live up to the hype. Start reading 
>>>>> now. http://productguide.itmanagersjournal.com/
>>>>> _______________________________________________
>>>>> Gusdev-gusdev mailing list
>>>>> Gus...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>>>
>>>>
>>>
>

Re: [Gusdev-gusdev] installing gus

[marc j. <de...@ho...>]

In the documentation, I keep seeing reference to a "build" command.

build GUS install -append -skipJaaveCompiling     [ for example]

This is not a unix/linux command I'm aware of.  I installed the binary 
version of ant from ant.apache.org and I can't find this build command.

mmm ..... Where does this come from?

Regards and Thanks,

Marc

Re: [Gusdev-gusdev] LoadAnnotatedSeqs plugin underway

[Steve F. <sfi...@pc...>]

paul-

let me start digesting this by email.

about your extensions to EMBL.  the bioPerl model we are parsing into is 
based on generic features, tags and annotation.  as long as the 
extensions can be parsed into those objects we're half way there.   are 
the extensions syntactically consistent w/ standard embl files, but 
varying only in the particulars of what the data is called?

about building the hierarchy.  if you looked at the bioperl api for the 
unflattener, you'd see that its unflattening uses gene name as a clue to 
deciding what features go together in a particular gene model.

can gene name be relied upon to identify all the features that are 
associated with this gene?

finally, about the GO stuff, yes, we can probably reuse your code.

steve


Paul Mooney wrote:

>
> On 9 Dec 2004, at 19:31, Steve Fischer wrote:
>
>> paul-
>>
>> hey.  do you want to set up a time to chat so i can catch you up on 
>> what we have in mind?
>
>
>
> At the moment I'm curious how much can be achieved via a generic 
> plugin. I think the plugin will need plugin's to do specialised parts 
> :) However I'd be glad to give my assistance to the effort. Below are 
> my random thoughts I've just had on the matter;
>
>
> Here at the PSU we store an awful lot of info that can not be stored 
> in a standard EMBL file, hence we have extended it to fit out own 
> needs. As an example we use several name qualifiers for genes;
>
>     . systematic_id           - the name cast in stone
>     . temporary_systematic_id - the name as it is currently known
>     . previous_systematic_id  - as it was known
>     . gene                    - EMBL standard qualifier
>
> Hence just trying to unflatten the EMBL file is tricky because 
> systematic and temporary_sysetmatic_ids are mixed in the same 
> sequence, hence building the hierarchy would need specialised code. 
> TIGR XML has the same issue though so maybe its not too specialised 
> after all :/ (PUB_LOCUS and LOCUS has a direct mapping to 
> systematic_id and temporary_systematic_id).
>
> Something like this entry;
>     /curation="name; origin; date; permission; type; dbref; notes ..."
> i.e.
>     /curation="Matt Berriman; genedb; 20020128; public; comment"
> is unique to the PSU and I'm not sure where it fits in GUS.
>
> However;
>
> I have code that creates GO entries - supply a high level function 
> with all the standard GO fields and it creates the 5 rows (?) in the 
> different tables as required. This is definitely something that can be 
> shared across centres, perhaps in a code library. All your code has to 
> do is parse out the GO fields from the data. No reason why it couldn't 
> accept a GO Bioperl object (I presume one exists).
>
> Perhaps the parsing needs to a super class for each data source and 
> then sub-classed by each centre?
>
> Ok, enough ramblings. Does any of this make sense?
> Paul.
>
>> steve
>>
>> Chris Stoeckert wrote:
>>
>>> Hi Steve,
>>> Thanks for putting this out on gusdev. Marie-Adele indicated that 
>>> Paul Mooney was very interested in this and I will likely meet with 
>>> him about this when I visit in January. Please include him in email 
>>> correspondence when not addressed to the general gusdev list.
>>> Thanks,
>>> Chris
>>>
>>> On Dec 9, 2004, at 2:11 PM, Steve Fischer wrote:
>>>
>>>> folks-
>>>>
>>>> the UGA folks and CBIL folks have started collaborating on a new 
>>>> plugin called LoadAnnotatedSeqs.   It will use BioPerl to parse the 
>>>> input data.
>>>>
>>>> We expect it to take annotated sequences (NA at first) in genbank, 
>>>> tigr xml and embl formats (plus any others supported by the bioPerl 
>>>> parser).
>>>>
>>>> It will take an XML file that describes the mapping from the input 
>>>> features to GUS features, and SO features.
>>>> It will also hard code special cases to handle qualifer data that 
>>>> is distributed to tables outside of the NAFeature tables.
>>>>
>>>> For our projects we will be developing a mapping that unifies the 
>>>> semantics of the data we are getting from our different sources and 
>>>> formats.
>>>> (we plan to work with the PSU folks to incorporate the knowledge 
>>>> they have acquired in their work to make an EMBL parser)
>>>>
>>>> ideas and suggestions are encouraged.
>>>>
>>>> steve
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> SF email is sponsored by - The IT Product Guide
>>>> Read honest & candid reviews on hundreds of IT Products from real 
>>>> users.
>>>> Discover which products truly live up to the hype. Start reading 
>>>> now. http://productguide.itmanagersjournal.com/
>>>> _______________________________________________
>>>> Gusdev-gusdev mailing list
>>>> Gus...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>>
>>>
>>

Re: [Gusdev-gusdev] building gene models in LoadAnnotatedSeqs

[Steve F. <sfi...@pc...>]

by the way, the API page for the unflattener is here:

http://doc.bioperl.org/releases/bioperl-1.4/Bio/SeqFeature/Tools/Unflattener.html

steve

Aaron J. Mackey wrote:

>
> In general, BioPerl documentation is better viewed via:
>
>   http://doc.bioperl.org/
>
> On Dec 9, 2004, at 2:24 PM, Steve Fischer wrote:
>
>>
>> here is the documentation for the unflattener:   
>> http://cvs.bioperl.org/cgi-bin/viewcvs/viewcvs.cgi/bioperl-live/Bio/ 
>> SeqFeature/Tools/Unflattener.pm?rev=1.25&cvsroot=bioperl&content- 
>> type=text/vnd.viewcvs-markup
>>
>>
> -- 
> Aaron J. Mackey, Ph.D.
> Dept. of Biology, Goddard 212
> University of Pennsylvania       email:  am...@pc...
> 415 S. University Avenue         office: 215-898-1205
> Philadelphia, PA  19104-6017     fax:    215-746-6697
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now. 
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> Gusdev-gusdev mailing list
> Gus...@li...
> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev

Re: [Gusdev-gusdev] How are the dotsver etc. schemas are used

[Paul M. <pj...@sa...>]

Hi,

while on the subject, how many people have used the version tables to 
access old data?
We can store the old data but we don't (yet?) pulll it out.
If I wanted to reconstruct what a gene looked like (sequence, GO 
assignments, Pfam, etc) 2 weeks ago I think it would take quite a bit 
of coding. Not sure if the object layer can help...?

Paul.

On 9 Dec 2004, at 19:52, Deborah Pinney wrote:

> Hi Derek,
> No you don't want to version the version tables so you don't want to 
> set their is_versioned attribute to 1, just the non-version tables. As 
> I said, when using the objects to insert, update or delete, there is 
> automatic versioning in your case (is_versioned =1 for all the 
> tables). With a script that uses DBI, the script must explicity insert 
> into the version tables before deleting. If you are deleting using 
> something like sqlplus, there will not be any versioning unless you do 
> it, again explicity, prior to deleting from teh non-version table. I 
> don't know how you deleted but if you did it with dbvis or sqlplus, 
> there would be no entries into the version tables. So, you can't 
> conclude that versioning is not working.
>
>                                                                        
>                                                             -Debbie
>
>
> Dequan (Derek) Kong wrote:
>
>> Hi Deborah:
>>
>> Thanks!
>>
>> I checked my database.  I found that the  is_version of 
>> core.tableinfo is all set to 1 in the standard schema tables, but 
>> they are set to 0 in all the tables of the version  schema.  Do I 
>> have to change all of them to 1.  Would you please explain a little 
>> bit more about how the version system in GUS works?  I have deleted 
>> many rows from my GUS database, but I did not find any row inserted 
>> into the table of version schema.  Is this means that my version 
>> system does not work or I need to configure other stuff to make it 
>> work!
>>
>> By the way, how and when did those 1 or 0 in the core.tableinfo are 
>> set?
>>
>> Thanks again for your help!
>>
>> Derek
>>
>>
>>
>>
>> At 01:16 PM 12/9/2004 -0500, Deborah Pinney wrote:
>>
>>> Hi Derek,
>>> I'm sorry to disappoint you but I have not written such a script and 
>>> don't know of one. The table objects can automatically version and 
>>> will if core.TableInfo.is_versioned is set to 1 (not nullable).  
>>> When versioning is on, rows are put into the corresponding 
>>> spacever.tablever when rows are being deleted or updated. The 
>>> primary key of the version tables is a combination of PK and 
>>> modification date (from the original row). Apart from the objects, 
>>> there are scripts for specific purposes that put rows in the 
>>> corresponding version table prior to  deleting them. These scripts  
>>> generally  do this using DBI do statements.
>>>
>>> Does anyone else know about a plugin or script dealing with 
>>> versioning?
>>>
>>>                                                                     
>>> -Debbie
>>>
>>>
>>>
>>> Dequan (Derek) Kong wrote:
>>>
>>>> To Debbie Pinney
>>>>
>>>> Hi Debbie:
>>>>
>>>> Sucheta told me that you have written a plugin or program in CBIL 
>>>> to control the data Version using  the ver schemas in GUS.  Can I 
>>>> get the plugin or program if you have one?
>>>>
>>>> Thanks!
>>>>
>>>> Derek
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> At 10:19 AM 12/9/2004 -0500, Sucheta Tripathy wrote:
>>>>
>>>>> Hi Derek,
>>>>>
>>>>> The ver schema is used for storing any deleted objects from the 
>>>>> main database.
>>>>>
>>>>>  The deleteentry.pl script does not take care of it. Either the 
>>>>> user
>>>>> has to do it himself or get it from CBIL (they have a script, 
>>>>> which does that.)
>>>>>
>>>>> cheers
>>>>>
>>>>> Sucheta
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> At 08:14 AM 12/9/2004 -0500, Dequan (Derek) Kong wrote:
>>>>>
>>>>>>> Hi:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Can anyone tell me how the vers schemas are used in the version 
>>>>>> control of the GUS?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> Derek
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------
>>>>>> SF email is sponsored by - The IT Product Guide
>>>>>> Read honest & candid reviews on hundreds of IT Products from real 
>>>>>> users.
>>>>>> Discover which products truly live up to the hype. Start reading 
>>>>>> now. http://productguide.itmanagersjournal.com/
>>>>>> _______________________________________________
>>>>>> Gusdev-gusdev mailing list
>>>>>> Gus...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------
>>>> SF email is sponsored by - The IT Product Guide
>>>> Read honest & candid reviews on hundreds of IT Products from real 
>>>> users.
>>>> Discover which products truly live up to the hype. Start reading 
>>>> now. http://productguide.itmanagersjournal.com/
>>>> _______________________________________________
>>>> Gusdev-gusdev mailing list
>>>> Gus...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>>
>>
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real 
> users.
> Discover which products truly live up to the hype. Start reading now. 
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> Gusdev-gusdev mailing list
> Gus...@li...
> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>

Re: [Gusdev-gusdev] building gene models in LoadAnnotatedSeqs

[Steve F. <sfi...@pc...>]

aha.   in addition to the synopsis section, there is the General 
Documentation section at the end of the API doc for a given module.  (i 
didn't notice that)

steve

Aaron J. Mackey wrote:

>
> In general, BioPerl documentation is better viewed via:
>
>   http://doc.bioperl.org/
>
> On Dec 9, 2004, at 2:24 PM, Steve Fischer wrote:
>
>>
>> here is the documentation for the unflattener:   
>> http://cvs.bioperl.org/cgi-bin/viewcvs/viewcvs.cgi/bioperl-live/Bio/ 
>> SeqFeature/Tools/Unflattener.pm?rev=1.25&cvsroot=bioperl&content- 
>> type=text/vnd.viewcvs-markup
>>
>>
> -- 
> Aaron J. Mackey, Ph.D.
> Dept. of Biology, Goddard 212
> University of Pennsylvania       email:  am...@pc...
> 415 S. University Avenue         office: 215-898-1205
> Philadelphia, PA  19104-6017     fax:    215-746-6697

Re: [Gusdev-gusdev] building gene models in LoadAnnotatedSeqs

[Aaron J. M. <am...@pc...>]

In general, BioPerl documentation is better viewed via:

   http://doc.bioperl.org/

On Dec 9, 2004, at 2:24 PM, Steve Fischer wrote:

>
> here is the documentation for the unflattener:   
> http://cvs.bioperl.org/cgi-bin/viewcvs/viewcvs.cgi/bioperl-live/Bio/ 
> SeqFeature/Tools/Unflattener.pm?rev=1.25&cvsroot=bioperl&content- 
> type=text/vnd.viewcvs-markup
>
>
--
Aaron J. Mackey, Ph.D.
Dept. of Biology, Goddard 212
University of Pennsylvania       email:  am...@pc...
415 S. University Avenue         office: 215-898-1205
Philadelphia, PA  19104-6017     fax:    215-746-6697

Re: [Gusdev-gusdev] How are the dotsver etc. schemas are used

[Deborah P. <pi...@pc...>]

Hi Derek,
No you don't want to version the version tables so you don't want to set 
their is_versioned attribute to 1, just the non-version tables. As I 
said, when using the objects to insert, update or delete, there is 
automatic versioning in your case (is_versioned =1 for all the tables). 
With a script that uses DBI, the script must explicity insert into the 
version tables before deleting. If you are deleting using something like 
sqlplus, there will not be any versioning unless you do it, again 
explicity, prior to deleting from teh non-version table. I don't know 
how you deleted but if you did it with dbvis or sqlplus, there would be 
no entries into the version tables. So, you can't conclude that 
versioning is not working.

                                                                                                                                    
-Debbie


Dequan (Derek) Kong wrote:

> Hi Deborah:
>
> Thanks!
>
> I checked my database.  I found that the  is_version of core.tableinfo 
> is all set to 1 in the standard schema tables, but they are set to 0 
> in all the tables of the version  schema.  Do I have to change all of 
> them to 1.  Would you please explain a little bit more about how the 
> version system in GUS works?  I have deleted many rows from my GUS 
> database, but I did not find any row inserted into the table of 
> version schema.  Is this means that my version system does not work or 
> I need to configure other stuff to make it work!
>
> By the way, how and when did those 1 or 0 in the core.tableinfo are set?
>
> Thanks again for your help!
>
> Derek
>
>
>
>
> At 01:16 PM 12/9/2004 -0500, Deborah Pinney wrote:
>
>> Hi Derek,
>> I'm sorry to disappoint you but I have not written such a script and 
>> don't know of one. The table objects can automatically version and 
>> will if core.TableInfo.is_versioned is set to 1 (not nullable).  When 
>> versioning is on, rows are put into the corresponding 
>> spacever.tablever when rows are being deleted or updated. The primary 
>> key of the version tables is a combination of PK and modification 
>> date (from the original row). Apart from the objects, there are 
>> scripts for specific purposes that put rows in the corresponding 
>> version table prior to  deleting them. These scripts  generally  do 
>> this using DBI do statements.
>>
>> Does anyone else know about a plugin or script dealing with versioning?
>>
>>                                                                     
>> -Debbie
>>
>>
>>
>> Dequan (Derek) Kong wrote:
>>
>>> To Debbie Pinney
>>>
>>> Hi Debbie:
>>>
>>> Sucheta told me that you have written a plugin or program in CBIL to 
>>> control the data Version using  the ver schemas in GUS.  Can I get 
>>> the plugin or program if you have one?
>>>
>>> Thanks!
>>>
>>> Derek
>>>
>>>
>>>
>>>
>>>
>>> At 10:19 AM 12/9/2004 -0500, Sucheta Tripathy wrote:
>>>
>>>> Hi Derek,
>>>>
>>>> The ver schema is used for storing any deleted objects from the 
>>>> main database.
>>>>
>>>>  The deleteentry.pl script does not take care of it. Either the user
>>>> has to do it himself or get it from CBIL (they have a script, which 
>>>> does that.)
>>>>
>>>> cheers
>>>>
>>>> Sucheta
>>>>
>>>>
>>>>
>>>>
>>>> At 08:14 AM 12/9/2004 -0500, Dequan (Derek) Kong wrote:
>>>>
>>>>>> Hi:
>>>>>
>>>>>
>>>>>
>>>>> Can anyone tell me how the vers schemas are used in the version 
>>>>> control of the GUS?
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Derek
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -------------------------------------------------------
>>>>> SF email is sponsored by - The IT Product Guide
>>>>> Read honest & candid reviews on hundreds of IT Products from real 
>>>>> users.
>>>>> Discover which products truly live up to the hype. Start reading 
>>>>> now. http://productguide.itmanagersjournal.com/
>>>>> _______________________________________________
>>>>> Gusdev-gusdev mailing list
>>>>> Gus...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>>>
>>>
>>>
>>>
>>>
>>> -------------------------------------------------------
>>> SF email is sponsored by - The IT Product Guide
>>> Read honest & candid reviews on hundreds of IT Products from real 
>>> users.
>>> Discover which products truly live up to the hype. Start reading 
>>> now. http://productguide.itmanagersjournal.com/
>>> _______________________________________________
>>> Gusdev-gusdev mailing list
>>> Gus...@li...
>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>
>

[Gusdev-gusdev] building gene models in LoadAnnotatedSeqs

[Steve F. <sfi...@pc...>]

folks-

We expect that LoadAnnotatedSeqs will be using bioPerl's 
Bio::SeqFeature::Tools::Unflattener to create the feature hierarchy from 
genbank/embl records.   (the tigr xml already has that info).

the unflattener (written by Chris Mungall) has a "smart" algorithm to 
infer containment relationships from those implicit in the sloppy 
genbank syntax.

here is the documentation for the unflattener:  
http://cvs.bioperl.org/cgi-bin/viewcvs/viewcvs.cgi/bioperl-live/Bio/SeqFeature/Tools/Unflattener.pm?rev=1.25&cvsroot=bioperl&content-type=text/vnd.viewcvs-markup

if you are interested, you might want to check out the algorithm.

by default, the unflattener only handles:
  Gene, RNA, CDS, exon, intron and psuedogenes

probably we'll start out either using that default, or, modifying it to 
include, say, polyA_signals, etc.

we would leave to future iterations of the plugin the ability to 
configure the plugin to specify what containment to use.

steve

[Gusdev-gusdev] LoadAnnotatedSeqs plugin underway

[Steve F. <sfi...@pc...>]

folks-

the UGA folks and CBIL folks have started collaborating on a new plugin 
called LoadAnnotatedSeqs.   It will use BioPerl to parse the input data.

We expect it to take annotated sequences (NA at first) in genbank, tigr 
xml and embl formats (plus any others supported by the bioPerl parser).

It will take an XML file that describes the mapping from the input 
features to GUS features, and SO features. 

It will also hard code special cases to handle qualifer data that is 
distributed to tables outside of the NAFeature tables.

For our projects we will be developing a mapping that unifies the 
semantics of the data we are getting from our different sources and 
formats.  

(we plan to work with the PSU folks to incorporate the knowledge they 
have acquired in their work to make an EMBL parser)

ideas and suggestions are encouraged.

steve

Re: [Gusdev-gusdev] How are the dotsver etc. schemas are used

[Dequan (D. K. <dk...@vb...>]

Hi Deborah:

Thanks!

I checked my database.  I found that the  is_version of core.tableinfo is 
all set to 1 in the standard schema tables, but they are set to 0 in all 
the tables of the version  schema.  Do I have to change all of them to 
1.  Would you please explain a little bit more about how the version system 
in GUS works?  I have deleted many rows from my GUS database, but I did not 
find any row inserted into the table of version schema.  Is this means that 
my version system does not work or I need to configure other stuff to make 
it work!

By the way, how and when did those 1 or 0 in the core.tableinfo are set?

Thanks again for your help!

Derek




At 01:16 PM 12/9/2004 -0500, Deborah Pinney wrote:
>Hi Derek,
>I'm sorry to disappoint you but I have not written such a script and don't 
>know of one. The table objects can automatically version and will if 
>core.TableInfo.is_versioned is set to 1 (not nullable).  When versioning 
>is on, rows are put into the corresponding spacever.tablever when rows are 
>being deleted or updated. The primary key of the version tables is a 
>combination of PK and modification date (from the original row). Apart 
>from the objects, there are scripts for specific purposes that put rows in 
>the corresponding version table prior to  deleting them. These 
>scripts  generally  do this using DBI do statements.
>
>Does anyone else know about a plugin or script dealing with versioning?
>
>                                                                     -Debbie
>
>
>
>Dequan (Derek) Kong wrote:
>
>>To Debbie Pinney
>>
>>Hi Debbie:
>>
>>Sucheta told me that you have written a plugin or program in CBIL to 
>>control the data Version using  the ver schemas in GUS.  Can I get the 
>>plugin or program if you have one?
>>
>>Thanks!
>>
>>Derek
>>
>>
>>
>>
>>
>>At 10:19 AM 12/9/2004 -0500, Sucheta Tripathy wrote:
>>
>>>Hi Derek,
>>>
>>>The ver schema is used for storing any deleted objects from the main 
>>>database.
>>>
>>>  The deleteentry.pl script does not take care of it. Either the user
>>>has to do it himself or get it from CBIL (they have a script, which does 
>>>that.)
>>>
>>>cheers
>>>
>>>Sucheta
>>>
>>>
>>>
>>>
>>>At 08:14 AM 12/9/2004 -0500, Dequan (Derek) Kong wrote:
>>>
>>>>>Hi:
>>>>
>>>>
>>>>Can anyone tell me how the vers schemas are used in the version control 
>>>>of the GUS?
>>>>
>>>>Thanks!
>>>>
>>>>Derek
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>-------------------------------------------------------
>>>>SF email is sponsored by - The IT Product Guide
>>>>Read honest & candid reviews on hundreds of IT Products from real users.
>>>>Discover which products truly live up to the hype. Start reading now. 
>>>>http://productguide.itmanagersjournal.com/
>>>>_______________________________________________
>>>>Gusdev-gusdev mailing list
>>>>Gus...@li...
>>>>https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>
>>
>>
>>
>>-------------------------------------------------------
>>SF email is sponsored by - The IT Product Guide
>>Read honest & candid reviews on hundreds of IT Products from real users.
>>Discover which products truly live up to the hype. Start reading now. 
>>http://productguide.itmanagersjournal.com/
>>_______________________________________________
>>Gusdev-gusdev mailing list
>>Gus...@li...
>>https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev

Re: [Gusdev-gusdev] How are the dotsver etc. schemas are used in the version control?

[Deborah P. <pi...@pc...>]

Hi Derek,
I'm sorry to disappoint you but I have not written such a script and 
don't know of one. The table objects can automatically version and will 
if core.TableInfo.is_versioned is set to 1 (not nullable).  When 
versioning is on, rows are put into the corresponding spacever.tablever 
when rows are being deleted or updated. The primary key of the version 
tables is a combination of PK and modification date (from the original 
row). Apart from the objects, there are scripts for specific purposes 
that put rows in the corresponding version table prior to  deleting 
them. These scripts  generally  do this using DBI do statements.

Does anyone else know about a plugin or script dealing with versioning?
                   
                                                                     -Debbie



Dequan (Derek) Kong wrote:

> To Debbie Pinney
>
> Hi Debbie:
>
> Sucheta told me that you have written a plugin or program in CBIL to 
> control the data Version using  the ver schemas in GUS.  Can I get the 
> plugin or program if you have one?
>
> Thanks!
>
> Derek
>
>
>
>
>
> At 10:19 AM 12/9/2004 -0500, Sucheta Tripathy wrote:
>
>> Hi Derek,
>>
>> The ver schema is used for storing any deleted objects from the main 
>> database.
>>
>>  The deleteentry.pl script does not take care of it. Either the user  
>> has to do it himself or get it from CBIL (they have a script, which 
>> does that.)
>>
>> cheers
>>
>> Sucheta
>>
>>
>>
>>
>> At 08:14 AM 12/9/2004 -0500, Dequan (Derek) Kong wrote:
>>
>>>> Hi:
>>>
>>>
>>> Can anyone tell me how the vers schemas are used in the version 
>>> control of the GUS?
>>>
>>> Thanks!
>>>
>>> Derek
>>>
>>>
>>>
>>>
>>>
>>> -------------------------------------------------------
>>> SF email is sponsored by - The IT Product Guide
>>> Read honest & candid reviews on hundreds of IT Products from real 
>>> users.
>>> Discover which products truly live up to the hype. Start reading 
>>> now. http://productguide.itmanagersjournal.com/
>>> _______________________________________________
>>> Gusdev-gusdev mailing list
>>> Gus...@li...
>>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>>
>
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now. 
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> Gusdev-gusdev mailing list
> Gus...@li...
> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev

Re: [Gusdev-gusdev] How are the dotsver etc. schemas are used in the version control?

[Dequan (D. K. <dk...@vb...>]

To Debbie Pinney

Hi Debbie:

Sucheta told me that you have written a plugin or program in CBIL to 
control the data Version using  the ver schemas in GUS.  Can I get the 
plugin or program if you have one?

Thanks!

Derek





At 10:19 AM 12/9/2004 -0500, Sucheta Tripathy wrote:
>Hi Derek,
>
>The ver schema is used for storing any deleted objects from the main database.
>
>  The deleteentry.pl script does not take care of it. Either the user  has 
> to do it himself or get it from CBIL (they have a script, which does that.)
>
>cheers
>
>Sucheta
>
>
>
>
>At 08:14 AM 12/9/2004 -0500, Dequan (Derek) Kong wrote:
>
>>>Hi:
>>
>>Can anyone tell me how the vers schemas are used in the version control 
>>of the GUS?
>>
>>Thanks!
>>
>>Derek
>>
>>
>>
>>
>>
>>-------------------------------------------------------
>>SF email is sponsored by - The IT Product Guide
>>Read honest & candid reviews on hundreds of IT Products from real users.
>>Discover which products truly live up to the hype. Start reading now. 
>>http://productguide.itmanagersjournal.com/
>>_______________________________________________
>>Gusdev-gusdev mailing list
>>Gus...@li...
>>https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev

Re: [Gusdev-gusdev] How are the dotsver etc. schemas are used in the version control?

[Sucheta T. <su...@vb...>]

Hi Derek,

The ver schema is used for storing any deleted objects from the main database.

  The deleteentry.pl script does not take care of it. Either the user  has 
to do it himself or get it from CBIL (they have a script, which does that.)

cheers

Sucheta




At 08:14 AM 12/9/2004 -0500, Dequan (Derek) Kong wrote:

>>Hi:
>
>Can anyone tell me how the vers schemas are used in the version control of 
>the GUS?
>
>Thanks!
>
>Derek
>
>
>
>
>
>-------------------------------------------------------
>SF email is sponsored by - The IT Product Guide
>Read honest & candid reviews on hundreds of IT Products from real users.
>Discover which products truly live up to the hype. Start reading now. 
>http://productguide.itmanagersjournal.com/
>_______________________________________________
>Gusdev-gusdev mailing list
>Gus...@li...
>https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev

[Gusdev-gusdev] How are the dotsver etc. schemas are used in the version control?

[Dequan (D. K. <dk...@vb...>]

>Hi:

Can anyone tell me how the vers schemas are used in the version control of 
the GUS?

Thanks!

Derek

[Gusdev-gusdev] proposals for WDK 1.2

[Steve F. <sfi...@pc...>]

folks-

i have started a wiki page describing proposals for upcoming features 
for the WDK.

they are in a rough priority order, according only to me.   now is your 
chance to offer suggestions/requests.

we will start any day now on the next release cycle, so PLEASE LET US 
KNOW what you think we should be working on, and if you have ideas about 
what i have outlined.

http://www.gusdb.org/wiki/index.php/GusWdkDevelopment

steve

Re: Re: [Gusdev-gusdev] UpdateGusFromXML and updating records

[Ed R. <ed_...@be...>]

One thing I have encountered using updategusfromxml is that it allows you to bypass the sequences that should maintain data integrity.  I'm just briging this up here as a matter of concern for future development.

Any ideas?

-Ed


> 
> From: Angel Pizarro <an...@pc...>
> Date: 2004/12/07 Tue PM 04:51:50 EST
> To: gusdev <Gus...@li...>
> Subject: Re: [Gusdev-gusdev] UpdateGusFromXML and updating records
> 
> I was actually pretty surprised that you did not run into that error 
> previously. All of the GUS XML that I have dealt with had lower case 
> elements for the table columns
> 
> So now we know that:
> 1) Elements that are table column name must be lowercase and in the 
> order they appear in the view/table
> 2) A column must be all on one line with no leading/trailing whitespace
> 3) Updates are triggered by settig the PK column
> 
> angel
> 
> Kevin Murphy wrote:
> 
> > Angel Pizarro wrote:
> >
> >> The GUS XML parser is not a real XML parser. The order of the columns 
> >> matter. Put <PROTOCOL_ID> before <PROTOCOL_TYPE_ID> 
> >
> >
> > Here is another important note on updating records using 
> > UpdateGusFromXML:
> >
> > If you specify the primary key, that column name must be in lower 
> > case.  The other column names are not case-sensitive.
> >
> > Kevin Murphy
> >
> 
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now. 
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> Gusdev-gusdev mailing list
> Gus...@li...
> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
> 

Re: [Gusdev-gusdev] installing gus

[Ed R. <ed_...@be...>]

We have a much more recently written installation guide along with some helper scripts and advice on bootstrapping, etc.    You will have to get in touch with me about where to get this documentation, etc, as it is all on our site and password protected.

At somepoint, when I am not busy re-programming some of the pluggins, I would like to update this and put it on the wiki.

-Ed


> 
> From: "marc jackson" <de...@ho...>
> Date: 2004/12/08 Wed AM 10:52:40 EST
> To: gus...@li...
> Subject: [Gusdev-gusdev] installing gus
> 
> 

Ed Robinson
255 Deerfield Rd
Bogart, GA 30622
(706)425-9181


Sweet Caroline
good times never seemed so good.
I've been inclined
to believe they never would.
     --Neil Diamond


We're just a bunch of idiots.
      --Johnny Damon

Re: [Gusdev-gusdev] installing gus

[Sucheta T. <su...@vb...>]

Hi,

It is our fault. We originally wrote it for our lab and did not quite clean 
it before releasing.

I can send you the bootstrap data as well as the .sh scripts.

Sucheta


At 03:52 PM 12/8/2004 +0000, marc jackson wrote:
>Hello,
>
>Documentation for installing GUS doesn't seem accurate.  I can't find any 
>tarballs at the Sanger institute web page, and I'm not finding any *.sh 
>scripts in the information that is downloaded with the cvs commands.
>
>Any help would be appreciated.
>
>Regards,
>
>Marc
>------------------------------------------------------- SF email is 
>sponsored by - The IT Product Guide Read honest & candid reviews on 
>hundreds of IT Products from real users. Discover which products truly 
>live up to the hype. Start reading now. 
>http://productguide.itmanagersjournal.com/ 
>_______________________________________________ Gusdev-gusdev mailing list 
>Gus...@li... 
>https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev

Re: [Gusdev-gusdev] installing gus

[Michael S. <msa...@pc...>]

Hi Marc,

I'd take a look at some of the additional documentation available at:

http://gusdb.org/documentation.html

The "Fidel and Sucheta's installation guide put together by Fidel Salas, 
Sucheta Tripathy and Tejal Karkhanis at the Virginia Bioinformtics 
Institute." is particularly good and should be of help.

--Mike


marc jackson wrote:
> Hello,
>  
> Documentation for installing GUS doesn't /seem/ accurate.  I can't find 
> any tarballs at the Sanger institute web page, and I'm not finding any 
> *.sh scripts in the information that is downloaded with the cvs commands. 
>  
> Any help would be appreciated.
>  
> Regards,
>  
> Marc
> ------------------------------------------------------- SF email is 
> sponsored by - The IT Product Guide Read honest & candid reviews on 
> hundreds of IT Products from real users. Discover which products truly 
> live up to the hype. Start reading now. 
> http://productguide.itmanagersjournal.com/ 
> _______________________________________________ Gusdev-gusdev mailing 
> list Gus...@li... 
> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev

[Gusdev-gusdev] RAD3 Process Tables deprecation

[Elisabetta M. <man...@pc...>]

Here at CBIL we have decided to deprecate the usage of the tables 
RAD3.Process*** and to utilize instead a new view of 
RAD3.AnalysisResultImp to store the results of a series of 
DataTransformation protocols. The Analysis tables were created 
subsequently to the Process tables and have a more mature layout, which 
should facilitate queries. They were originally created to store 
down-stream analysis results (e.g. for differential expression or 
clustering) but they are also suitable for pre-processing result storage.
The following describes what we are implementing:

1. We have created a RAD3.ProtocolStep table to identify protocols 
consisting of an ordered series of (sub) protocols. Every protocol of the 
latter form is linked through RAD3.ProtocolStep to its components. Its 
type will be "transformation_protocol_series" 
(DataTransformationProtocolType).

2. We'll create a linking table RAD3.StudyAnalysis to facilitate queries.

3. We'll create a new view of RAD3.AnalysisResultImp, called 
RAD3.DataTransformationResult, which will store all results from a series 
of preprocessing steps obtained from protocols with type 
DataTransformationProtocolType. This will be a simple view with just 
5 relevant attributes: table_id, row_id (pointing to the spot or to the 
probeset on the array; a soft link), and 
float_value/int_value/string_value 
(which will be what's now currently stored in RAD3.ProcessResult.value).

4. When a series of processing steps is used, we will only store the 
results from the *final* step and the corresponding protocol will be of 
type "transformation_protocol_series".

5. For processes which are simple normalizations of a given quantification 
(or of a pair of quantifications for 2-channel data), the input to the 
analysis will be a LogicalGroup consisting just of that quantification 
(resp. pair of quantifications). For processes which involve averaging 
across quantifications the input LogicalGroup will contain 2 or more 
quantifications

6. In terms of plugins, the current AnalysisResultLoader in the 
GUS/RAD cvs repository at Sanger will do. 
(Enhancements or auxiliary plugins might be written for the specific case 
in which the analysis is an up-stream (=low level=processing) one. So this 
plugin will do the job that the ProcessResultLoader was doing for the 
Process tables.)

Elisabetta