Re: [Gmod-schema] Re: xml question

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

On Tue, 15 Feb 2005, Scott Cain wrote:

> On Tue, 2005-02-15 at 13:09 -0800, Chris Mungall wrote:
> >
> > On Tue, 15 Feb 2005, Scott Cain wrote:
> >
> > > Hi Linda,
> > >
> > > I am cc'ing this to the schema mailing list because it is an importan=
t
> > > question.
> > >
> > > I am inclined to say that the whole schema/chado/dat/ directory, whic=
h
> > > contains chado.* should be removed as it is quite out of date; any
> > > objections?  (It is currently ignored when distributions are created
> > > because it isn't in the MANIFEST file).
> >
> > No objections from me
> >
> > > Also, I would advocate removing the Flybase specific files from XML-
> > > XORT/conf/ , including chado.ddl, chado.dtd, ddl.properties and the
> > > trigger files.  They could be moved to another directory in the
> > > repository like contrib/ .
> >
> > In what sense are these FlyBase specific files (other than perhaps the
> > trigger code)?
>
> They are FlyBase specific because they are for the FlyBase flavor of
> chado, which could certainly confuse non-flybase folks.

but isn't chado.ddl the same for everyone?

ah, perhaps not, as some people may include/exclude certain modules, i
see.

> > Presumably most of these are all files that are auto-generated by the X=
ORT
> > Makefile.PL, so there is no need to keep them in cvs
>
> Probably so--so they should be removed.
> >
> > The trigger code may benefit other non-FB users. It seems that the
> > appropriate place to keep this is in
> > chado/modules/<module>/flybase/flybase-<module>-triggers.sql
>
> There are more generic triggers that I've been working on that reside in
> schema/chado/modules/sequence/apollo-bridge/ . I don't much care where
> the flybase triggers go, but your suggestion seems fine.
>
> > > Now to address you questions (mostly with more question): first, are =
you
> > > using the current XML-XORT (ie, the separate repository, not the one
> > > under the schema repository)? Next, when you installed gmod, did you =
set
> > > up the environment variables as suggested in the INSTALL doc?  If so,
> > > you should get a prompt to use Bio::GMOD::Config when you run XORT's
> > > Makefile.PL.  Answering yes to that should make the XORT installer us=
e
> > > complete.sql which was installed when you installed gmod (it probably
> > > went in /usr/local/gmod/src/modules).  Finally note that I just fixed=
 a
> > > bug in the XORT installer, so if you are using an anonymous checkout,=
 it
> > > may take a day or so for you to get the bug fix (I can send you the
> > > fixed file separately if you want).
> >
> > I have to admit I'm still a bit confused by the installation process -
> > perhaps XORT should have it's own INSTALL document?
>
> Whoops! Yes, there should be one.  Anyone want to write it?
>
> >  Maybe my confusion
> > comes from the fact that I'm currently looking at just XORT & Chado and
> > not trying to install GMOD as a whole. Is this not to be recommended?
>
> Right; so if you start with installing gmod/chado from it's Makefile.PL,
> one of the things that gets installed is Bio::GMOD::Config, which reads
> configuration information from files in /usr/local/gmod/conf.  The XORT
> Makefile.PL checks for the existence of that module and prompts the user
> to use that information.

OK, I see. I still feel that this a bit back-to-front, as I think of xort
as being a dependency of chado-xml rather than vice versa

> > Is there any reason not to cvs rm the deprecated schema/XMLTools/XORT/
> > directory, since that is now superseded by gmod/XML-XORT?
>
> No, I was reluctant to do it because I didn't want to be responsible for
> losing the history of the files.  If Pinglei doesn't mind, I can ask the
> SF admins to remove that directory.

I think it's fine just to cvs rm the files. the directories will hang
around annoyingly since there's no way of truly deleting them with cvs,
but they will be invisible so long as people cvs update with the prune
option

> > Note that the xort test suite seems to fail at the moment, due to the
> > absence of a XML::XORT module
>
> Right again; nobody has taken the time to write tests.

fair enough - but the 'test' that is there fails...

> > Scott - is there any plan to switch to using the SQL::T templates for
> > generating the xort conf files? On the one hand, the existing
> > xort_ddl_properties_creator.pl is a little hacky and is perhaps not rob=
ust
> > to changes in the style of DDL. On the other hand, this script seems ha=
ppy
> > with things like function definitions in the complete.sql file (which I
> > believe SQL::T will barf at). Furthermore, I'm a little worried by the
> > 'CPAN bloat' in SQL::T - it used to be a fairly lightweight module, but
> > the list of dependencies seems to have grown rather a lot. I was
> > previously enthusiastic about a switch to using SQL::T but I'm not so s=
ure
> > now.
>
> I sympathize with your concerns about CPAN bloat.  Bundle::GMOD has
> quite a few modules in it.  My philosophy lately has been, if it
> installs fine with the cpan shell, who cares how many modules there are?
> That said, SQL::T is currently an optional module for gmod; users only
> need it if they want to modify the schema.  If they don't modify the
> schema, the installer uses default files that are created when I create
> a distribution (well, anytime the schema tables get changed really).  I
> wouldn't mind seeing pregenerated files for XORT living in the chado cvs
> that get used as the default files, while if the user changes the
> schema, they have to let XORT regenerate the files.

That sounds fine (so long as they're kept in sync of course!)

> As for migrating to SQL::T for XORT, it doesn't matter much to me.  I
> wrote a proof of concept template for generating ddl_properties using
> SQL::T last fall, but it didn't quite work.  I'm sure if we wanted to go
> down that path we could fix it.  The question I have is this: how
> fragile is xort_ddl_properties_creator.pl?  Should we invest the time
> migrating now to save headaches later?

I don't know...

> > I think you and Pinglei have done an excellent job documenting all this
> > code & making it easy to use. I think the XORT stuff is 95% there.
>
> Thanks.
> >
> > I have one request as far as XORT is concerned. Would it not be possibl=
e
> > to allow runtime config rather than install-time config. All this setti=
ng
> > of chado variables at Makefile.PL stage doesn't seem quite right. I'm n=
ot
> > sure why things have to be set in Constants.pm.
>
> I am inclined to agree, but I don't have the time at the moment to make
> the changes.

well it sounds like Don may already have the code to do this...

> > Surely it would be fairly easy to make Constants.pm lookup a config fil=
e
> > at runtime? This could be a standard ini file like this:
> >
> > [database]
> > name=3Dchado
> > conf_dir=3D~/chado/conf
> > dbport=3D5432
> >
> > [database]
> > name=3Dtest_chado
> > conf_dir=3D~/test_chado/conf
> >
> > or some xml equivalent?
> >
> > > Scott
> > >
> > >
> > >
> > > On Tue, 2005-02-15 at 10:53 +0100, Linda Sperling wrote:
> > > > Hi Scott,
> > > >
> > > > The problem persists as concerns seeing double features, any insigh=
t is
> > > > welcome.
> > > >
> > > > I have a (probably stupid) question (and Olivier is not around toda=
y).
> > > > Maybe you can answer it.
> > > >
> > > > I am spending a lot of effort trying to transitiion our stock
> > > > collection database into the chado genetic module. I now understand=
 it,
> > > > I understand how it will work for our stocks and for our rnai data,=
 and
> > > > I even have added a submodule (on paper) for the stocks (as suggest=
ed
> > > > by whoever put comments in the genetic.sql). Great.
> > > >
> > > > So I decided to poke into the XORT and XML stuff, to figure out how=
 to
> > > > dump our present database into chado xml, so that Olivier can then =
work
> > > > on how to load it and test it.
> > > >
> > > > I cannot find any xml or dtd or ddl file that incorporates the curr=
ent
> > > > (I hope!) genetic module. I can only find things that mention genot=
ype
> > > > and phenotype and interaction tables which no longer exist in the c=
hado
> > > > genetic.sql!!!!
> > > >
> > > > Is this because I am supposed to know how to generate the dtd mysel=
f
> > > > from the chado.sql ?
> > > >
> > > > Or is this because pinglei and co. have not yet payed attention to =
the
> > > > new genetic module in the XML-XORT package?
> > > >
> > > >
> > > > I am crossing my fingers that the chado.sql (which  corresponds to =
Dave
> > > > Emmert's presentation at the April GMOD meeting in Cambridge) **is*=
*
> > > > the recent version -- because it will work for us! But I am surpris=
ed
> > > > that the chado.dtd and chado.ddl in CVS under  gmod/XML-XORT/conf i=
s
> > > > not up to date.
> > > >
> > > > any insight welcome!
> > > >
> > > > thanks,
> > > > Linda
> > > >
> > > >
> > > >
> > > > On Feb 14, 2005, at 3:45 PM, Scott Cain wrote:
> > > >
> > > > > Hi Linda,
> > > > >
> > > > > I just wanted to let you know I haven't forgotten about this.  I =
ran
> > > > > into a problem last week while doing a test install of chado--mak=
e sure
> > > > > that Olivier doesn't upgrade to the current version of Graph (0.5=
5); it
> > > > > either makes it impossible to load GO or makes it VERY slow (so s=
low
> > > > > that the two are indistinguishable on my computer).  Anyway, I sh=
ould
> > > > > be
> > > > > moving on to gbrowse problems today.
> > > > >
> > > > > Scott
> > > > >
> > > > >
> > > > > On Mon, 2005-02-07 at 17:56 +0100, Linda Sperling wrote:
> > > > >> Yes, he is using the yeast data, as well as Paramecium data, eit=
her
> > > > >> dataset gives the problem.
> > > > >>
> > > > >> But if you don't have that problem with the yeast data...  how c=
an you
> > > > >> figure out what is going wrong ??
> > > > >>
> > > > >> L
> > > > >>
> > > > >>
> > > > >> On Feb 7, 2005, at 5:40 PM, Scott Cain wrote:
> > > > >>
> > > > >>> I've got ideas out the wazoo; the trick, as always, is to narro=
w it
> > > > >>> down
> > > > >>> to the right one :-)
> > > > >>>
> > > > >>> I suspect that there is a poorly constructed query in the adapt=
or.
> > > > >>> I'm
> > > > >>> installing postgres as I write this. I'll install the yeast dat=
a and
> > > > >>> do
> > > > >>> some testing.  Is that what you are using now?
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Scott
> > > > >>>
> > > > >>>
> > > > >>> On Mon, 2005-02-07 at 17:34 +0100, Linda Sperling wrote:
> > > > >>>> Scott,
> > > > >>>>
> > > > >>>> Unfortunately, that's not it.
> > > > >>>> If you have any other ideas...
> > > > >>>>
> > > > >>>> Thanks,
> > > > >>>> Linda
> > > > >>>> On Feb 7, 2005, at 4:44 PM, Scott Cain wrote:
> > > > >>>>
> > > > >>>>> Hi Linda,
> > > > >>>>>
> > > > >>>>> I've seen this before, though I don't remember feature_dbxref=
 being
> > > > >>>>> involved.  Perhaps this is not the same problem but is relate=
d.  I
> > > > >>>>> don't
> > > > >>>>> have postgres installed on my new hard drive yet (I do have t=
he
> > > > >>>>> source
> > > > >>>>> code though, so it's just a matter of time :-).  Ask Olivier =
to
> > > > >>>>> find
> > > > >>>>> the
> > > > >>>>> following snippet of code in Browser.pm in the feature2label
> > > > >>>>> method:
> > > > >>>>>
> > > > >>>>>   # WARNING: if too many features start showing up in tracks,
> > > > >>>>> uncomment
> > > > >>>>>   # the following line and comment the one after that.
> > > > >>>>>   # @label    =3D $self->type2label($basetype,$length) unless=
 @label;
> > > > >>>>>   push @label,$self->type2label($basetype,$length);
> > > > >>>>>
> > > > >>>>> and switch the commenting on the last two lines and see if th=
at
> > > > >>>>> helps.
> > > > >>>>> Browser.pm should be in a path like
> > > > >>>>> perl5/site_perl/5.8.5/i386-linux-
> > > > >>>>> thread-multi/Bio/Graphics.
> > > > >>>>>
> > > > >>>>> Thanks,
> > > > >>>>> Scott
> > > > >>>>>
> > > > >>>>> On Mon, 2005-02-07 at 14:53 +0100, Linda Sperling wrote:
> > > > >>>>>> Hello Scott,
> > > > >>>>>>
> > > > >>>>>> =09We are having a strange problem with gbrowse running on c=
hado.
> > > > >>>>>>
> > > > >>>>>> =09Olivier is loading GFF3 format into chado. The database s=
eems to
> > > > >>>>>> be
> > > > >>>>>> correctly loaded. However, any feature with a dbxref in the =
9th
> > > > >>>>>> GFF3
> > > > >>>>>> field, comes up double in gbrowse! The feature_dbxref and db=
xref
> > > > >>>>>> tables
> > > > >>>>>> are loaded correctly.  Any feature without dbxref shows up o=
nce,
> > > > >>>>>> as
> > > > >>>>>> expected (and desired).
> > > > >>>>>>
> > > > >>>>>> =09Any ideas?
> > > > >>>>>>
> > > > >>>>>> Thanks,
> > > > >>>>>> Linda
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> Linda Sperling
> > > > >>>>>> Centre de G=C3=A9n=C3=A9tique Mol=C3=A9culaire
> > > > >>>>>> CNRS
> > > > >>>>>> Avenue de la Terrasse
> > > > >>>>>> 91198 Gif-sur-Yvette CEDEX
> > > > >>>>>> FRANCE
> > > > >>>>>>
> > > > >>>>>> spe...@cg...
> > > > >>>>>> +33 (0)1 69 82 32 09 =09(telephone)
> > > > >>>>>> +33 (0)1 69 82 31 50=09(fax)
> > > > >>>>>> http://paramecium.cgm.cnrs-gif.fr
> > > > >>>>>>
> > > > >>>>>> SAUVONS LA RECHERCHE
> > > > >>>>>> http://recherche-en-danger.apinc.org
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>> --
> > > > >>>>> -------------------------------------------------------------=
------
> > > > >>>>> --
> > > > >>>>> --
> > > > >>>>> -
> > > > >>>>> Scott Cain, Ph. D.
> > > > >>>>> ca...@cs...
> > > > >>>>> GMOD Coordinator (http://www.gmod.org/)
> > > > >>>>> 216-392-3087
> > > > >>>>> Cold Spring Harbor Laboratory
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>> --
> > > > >>> ---------------------------------------------------------------=
------
> > > > >>> --
> > > > >>> -
> > > > >>> Scott Cain, Ph. D.
> > > > >>> ca...@cs...
> > > > >>> GMOD Coordinator (http://www.gmod.org/)
> > > > >>> 216-392-3087
> > > > >>> Cold Spring Harbor Laboratory
> > > > >>
> > > > >>
> > > > >>
> > > > > --
> > > > > -----------------------------------------------------------------=
------
> > > > > -
> > > > > Scott Cain, Ph. D.
> > > > > ca...@cs...
> > > > > GMOD Coordinator (http://www.gmod.org/)
> > > > > 216-392-3087
> > > > > Cold Spring Harbor Laboratory
> > > > >
> > > > Linda Sperling
> > > > Centre de G=C3=A9n=C3=A9tique Mol=C3=A9culaire
> > > > CNRS
> > > > Avenue de la Terrasse
> > > > 91198 Gif-sur-Yvette CEDEX
> > > > FRANCE
> > > >
> > > > spe...@cg...
> > > > +33 (0)1 69 82 32 09 =09(telephone)
> > > > +33 (0)1 69 82 31 50=09(fax)
> > > > http://paramecium.cgm.cnrs-gif.fr
> > > >
> > > > SAUVONS LA RECHERCHE
> > > > http://recherche-en-danger.apinc.org
> > > >
> > > >
> > > >
> > >
> >
> >
>