Menu
▾
▴
genex-dev
|
From: Hilmar L. <la...@gn...> - 2001-11-27 01:35:53
|
Hi all,
I tried to find the DataLoader in the distribution, but was unable
to locate it. Quoting from the homepage:
Mar 3, 2001 - DataLoader project started to support large-scale, server-side
data entry for GeneX. In Perl. Fewer annotation requirements.
Did I overlook something, or has this been abandoned, or has this
finally turned into the XML loader?
-hilmar
--
-------------------------------------------------------------
Hilmar Lapp email: hil...@ya...
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------
|
|
From: Todd F. P. <tf...@nc...> - 2001-11-27 04:27:54
|
We are going to work on this next week and create a release of a very simple loader. Todd Peterson NCGR On Mon, 26 Nov 2001, Hilmar Lapp wrote: > Hi all, > > I tried to find the DataLoader in the distribution, but was unable > to locate it. Quoting from the homepage: > > Mar 3, 2001 - DataLoader project started to support large-scale, server-side > data entry for GeneX. In Perl. Fewer annotation requirements. > > Did I overlook something, or has this been abandoned, or has this > finally turned into the XML loader? > > > -hilmar > -- > ------------------------------------------------------------- > Hilmar Lapp email: hil...@ya... > GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 > ------------------------------------------------------------- > > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > |
|
From: <ja...@op...> - 2001-11-27 05:07:49
|
Hey Hilmar! Interesting to see that Novartis has taken an interest. "Hilmar Lapp" <la...@gn...> writes: > Quoting "Jason E. Stewart" <ja...@op...>: > > > There needs to be some significant > > database changes to make it work properly, and this will be released > > in the Genex-2 branch under development. > > May I ask how this will in general work, and what the required DB > changes were about? Mind you, this is for the upcoming Genex-2 version, not the code that is already available. Genex-2 was underway before the MAGE model was finalized. The primary change between Genex-1 and Genex-2 was a far more useful security model. In Genex-1 you can only protect ExperimentSets, ArrayMeasurements, and AM_Spots. All the rest is world viewable. Genex-2 enables you to protect *all* data: protocols, samples, contacts, etc. It also introduces audit information so you can track what was changed and by whom. And it introduces a generic authentication mechanism used by all CGI scripts -- so you have to login to the system before viewing data, making queries, manipulating data. Because MAGE is now (mostly) finalized, a good deal of the plans for Genex-2 will be the renaming of objects/tables to fit with MAGE nomenclature plus the addition of a number of additional tables/objects specified by MAGE. Genex-2 will *not* be fully MAGE compliant, but it will have major pieces. > Some background as to why I'm interested in the details: With my > previous employer we actually together with a consultant developed a > high-throughput general database loader, which would take any > record-oriented input file and load it to any relational > database. The limitation is obviously SQL on the DB end; i.e., > anything you cannot load through SQL cannot be loaded with that > tool. That limits you to a) insert into 1 table at a time, or b) > insert into 1 view at a time, provided you can attach insert > triggers to the view (which you can in Oracle), or c) call a stored > procedure. We used b) and c), with all the relational logic > (LU,PK,FK etc) staying within the DB. I'm actually trying to get > them to release the code (Java), not sure how successful this is > going to be. Sounds pretty cool. If you wanted to, that could easily be hosted at the MAGEstk site (mged.sf.net), the GeneX site, or at OpenInformatics (www.openinformatics.org). The Genex-2 data loader will *not* be a general purpose solution, it will strictly handle microarray data. You will need to specify two templates in order to use the loader: 1) the ArrayLayout (or ArrayDesign if you speak MAGE) 2) the QuantitiationDimension (from MAGE) that is defined by the combination of array technology and feature extraction software you used. This is a mapping that describes how many columns are in the output file, what their data type is and what the semantic meaning of the column is Once they are specified it just a matter of slurping in rows of data from the array files and entering them into the appropriate table in the DB. A major change in Genex-2 will be how the AM_Spots table is handle. In Genex-1 there is a single table into which all data is smashed. This works, and it is very general, but it creates too many problems. The solutiont that we've decided to pursue in Genex-2 is to use a different AM_Spots table for each new QuantitationDimension. That way if your data generates an array of 80 floats for each spot (or Feature in MAGE speak), all of those 80 numbers will go into a single row in the AM_Spots table for that technology. Genex-1 would force you to create 80 ArrayMeasurements each with a single value/spot in the AM_Spots table (yuck!). > > In the mean time, I took code > > that was graciously donated by Michael Pear, and got a data loader > > working for Genex-1. > > > > In the meantime, if you want to help pre-test the code, let me know. > > Sure. Especially if it helps me migrate a couple of thousand chip data > to our local GeneX in order to test its performance. You can check the code out from CVS. You'll want to use the 'Rel-1_0_1-branch' branch. Info on how to get the code from CVS is at: https://sourceforge.net/cvs/?group_id=16453 Once you've logged in you'll want to do the following: cvs -d:pserver:ano...@cv...:/cvsroot/genex \ co -r Rel-1_0_1-branch genex-server except of course you want it all on one line without the backslash... That will give you a working copy of GeneX-Server-1.0.5. The dataloader is in the affyloader/ directory. !!! WARNING !!! There isn't a huge amount of documentation available on the code. I've added a USAGE to each and a --help flag that *should* print out useful info, but YMMV. Please write to the list if you need help. You'll want to run a complete install even if you already have a working GeneX installation: there were two changes to the DB one to fix a bug in AL_Spots (the primary key was not being auto-generated), and the other is the addition of a view on the AM_Spots table. So you want to make sure that the DB installer runs and downloads the new DB init file (1.0.5) from the internet. BTW, GeneX has a nice feature for updating an existing installation. Check out the section on 'Updating an installation' in the INSTALL file. jas. |
|
From: Hilmar L. <la...@gn...> - 2001-11-27 19:46:10
|
Quoting "Jason E. Stewart" <ja...@op...>:
>
> Genex-2 enables you to protect *all* data: protocols, samples,
> contacts, etc. It also introduces audit information so you can track
> what was changed and by whom. And it introduces a generic
> authentication mechanism used by all CGI scripts -- so you have to
> login to the system before viewing data, making queries, manipulating
> data.
>
Is there an ERD for the new schema already? (BTW which tool did you use
to create the current one which is available on the website?)
> Because MAGE is now (mostly) finalized, a good deal of the plans for
> Genex-2 will be the renaming of objects/tables to fit with MAGE
> nomenclature plus the addition of a number of additional
> tables/objects specified by MAGE. Genex-2 will *not* be fully MAGE
> compliant, but it will have major pieces.
>
Do you already know which parts of MAGE will not be implemented
and/or covered by Genex-2?
> QuantitationDimension. That way if your data generates an array of 80
> floats for each spot (or Feature in MAGE speak), all of those 80
> numbers will go into a single row in the AM_Spots table for that
> technology. Genex-1 would force you to create 80 ArrayMeasurements
> each with a single value/spot in the AM_Spots table (yuck!).
So you're going to denormalize. Did you run into performance problems,
and if so, on which end, or in which situations? (Trying to learn from
your experience.)
-hilmar
--
-------------------------------------------------------------
Hilmar Lapp email: la...@gn...
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------
|
|
From: Todd F. P. <tf...@nc...> - 2001-11-27 19:57:12
|
I have placed the ERWin file on the genex website at: http://genebox.ncgr.org/download/DB/ Todd Peterson NCGR |
|
From: Hilmar L. <la...@gn...> - 2001-11-28 03:16:04
|
Quoting "Todd F. Peterson" <tf...@nc...>: > I have placed the ERWin file on the genex website at: > http://genebox.ncgr.org/download/DB/ > It seems that the present ERD has some considerable differences from the one available on the website, and I'm having difficulties getting hold of our person who's got an ERwin license. If someone could post an image of that, it'd help. -hilmar -- ------------------------------------------------------------- Hilmar Lapp email: la...@gn... GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 ------------------------------------------------------------- |
|
From: <ja...@op...> - 2001-11-28 03:41:14
|
"Hilmar Lapp" <la...@gn...> writes: > Quoting "Todd F. Peterson" <tf...@nc...>: > > > I have placed the ERWin file on the genex website at: > > http://genebox.ncgr.org/download/DB/ > > > > It seems that the present ERD has some considerable differences > from the one available on the website, and I'm having difficulties > getting hold of our person who's got an ERwin license. If someone > could post an image of that, it'd help. Just to warn you, if by 'the present ERD' you mean the proposed changes to Genex-2 that I've been meandering about, no diagram exists. The diagram that Todd posted is for Genex-1, and it is out of date with the latest changes that will be released in 1.0.5 (because I don't have an ERwin license either ;-) jas. |
|
From: Todd F. P. <tf...@nc...> - 2001-11-28 04:18:46
|
That's why I still advocate DoME. Will take a look at the other tool mentioned. Todd On 27 Nov 2001, Jason E. Stewart wrote: > "Hilmar Lapp" <la...@gn...> writes: > > > Quoting "Todd F. Peterson" <tf...@nc...>: > > > > > I have placed the ERWin file on the genex website at: > > > http://genebox.ncgr.org/download/DB/ > > > > > > > It seems that the present ERD has some considerable differences > > from the one available on the website, and I'm having difficulties > > getting hold of our person who's got an ERwin license. If someone > > could post an image of that, it'd help. > > Just to warn you, if by 'the present ERD' you mean the proposed > changes to Genex-2 that I've been meandering about, no diagram > exists. The diagram that Todd posted is for Genex-1, and it is out of > date with the latest changes that will be released in 1.0.5 (because I > don't have an ERwin license either ;-) > > jas. > > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > |
|
From: <ja...@op...> - 2001-11-28 05:02:52
|
"Todd F. Peterson" <tf...@nc...> writes: > That's why I still advocate DoME. Will take a look at the other tool > mentioned. I have time at this point, so I will take a look at DoME, thanks Todd. jas. |
|
From: Todd F. P. <tf...@nc...> - 2001-11-28 06:59:44
|
It's too bad I've been swamped with PathDB work. Had a pretty good start on customization of supplied UML format to produce DTD or code. Starting to get back to genex after some reorg'ing up there. Will be spending more time with DoME to make it work well. Have modeled a good portion of the MAGE UML model with it. todd On 27 Nov 2001, Jason E. Stewart wrote: > "Todd F. Peterson" <tf...@nc...> writes: > > > That's why I still advocate DoME. Will take a look at the other tool > > mentioned. > > I have time at this point, so I will take a look at DoME, thanks Todd. > > jas. > > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > |
|
From: Hilmar L. <la...@gn...> - 2001-11-28 17:07:36
|
Quoting "Jason E. Stewart" <ja...@op...>:
>
> Just to warn you, if by 'the present ERD' you mean the proposed
> changes to Genex-2 that I've been meandering about, no diagram
> exists.
I meant the diagram for the 1.0.x branch I checked out (I installed
1.0.4).
> The diagram that Todd posted is for Genex-1, and it is out of
> date with the latest changes that will be released in 1.0.5
That's what I realized. It's also dated with respect to 1.0.4, which
I realized after looking at the tables directly. It seems that a
number of things I was about to point out were fixed, whereas other
problems were introduced. Before going into any more potentially dated
details I really want to have the schema in front of me.
BTW as for ArgoUML, I don't mind installing another tool, as long as it's
free or we (and everyone else interested in a live version) have a license.
It also turns out that we have a license for Oracle designer, so that would
be fine with me too.
-hilmar
--
-------------------------------------------------------------
Hilmar Lapp email: la...@gn...
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------
|
|
From: <ja...@op...> - 2001-11-28 17:30:14
|
"Hilmar Lapp" <la...@gn...> writes: > Quoting "Jason E. Stewart" <ja...@op...>: > > > The diagram that Todd posted is for Genex-1, and it is out of > > date with the latest changes that will be released in 1.0.5 > > That's what I realized. It's also dated with respect to 1.0.4, which > I realized after looking at the tables directly. It seems that a > number of things I was about to point out were fixed, whereas other > problems were introduced. Before going into any more potentially dated > details I really want to have the schema in front of me. Right. The scripts in DB/tdscripts/ are mostly up-to-date. Once upon a time they were always up to date, because they were used to generate new versions of the DB. The person who took over the role of DB maintainer after me noticed that Postgres's pg_dump utility would export the schema into the DB dumps if you asked it, and stopped maintaining the tdscripts. This was an unfortunate accident that has caused a number of problems. In Genex-2, the DB is created from a DBMS-independent XML representation of the tables, so the XML files are always the most current representation of the schema. It also makes it pretty easy to plug in a new DBMS into GeneX by simply overriding the default XML -> SQL translator where it differs from Postgres. But that doesn't help you at the moment. Until I can get funds to pay for the time it will take to produce an new ER diagram for Genex-1 and Genex-2, I will have to work on other things. I'll be talking to Bill Pearson at UVA later on, and perhaps he'll agree to my doing this. jas. |
|
From: Hilmar L. <la...@gn...> - 2001-11-28 20:13:52
|
Quoting "Jason E. Stewart" <ja...@op...>:
>
> In Genex-2, the DB is created from a DBMS-independent XML
> representation of the tables, so the XML files are always the most
> current representation of the schema. It also makes it pretty easy to
> plug in a new DBMS into GeneX by simply overriding the default XML ->
> SQL translator where it differs from Postgres.
>
Sounds cool. Do you use a specific XML editing tool/tree viewer?
-hilmar
--
-------------------------------------------------------------
Hilmar Lapp email: la...@gn...
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------
|
|
From: Hilmar L. <la...@gn...> - 2001-11-29 18:35:40
|
Quoting "Jason E. Stewart" <ja...@op...>:
>
> All the DB interfaces are created to be SQL generic. Instead of having
> a bunch of hard-coded SQL ('SELECT foo FROM bar WHERE baz="NAN"') I
> created an SQL abstraction layer using three functions
> create_select_sql(), create_insert_sql(), and
> create_update_sql(). With these you can plug in backends for any
> relational DB to produce the SQL you need.
>
Is this framework already available already? If so, from the GeneX repository
or MAGEstk?
I'm trying to reverse engineer the ERD, unsuccessful so far. ERwin would
not reverse engineer from SQL, and doesn't have an idea about Postgres.
My next idea was port to Oracle, and reverse engineer with Designer.
Can the tool framework you mentioned reverse engineer from SQL DDL?
BTW the DDL I was able to dump did not contain FK constraints (bad for reverse
engineering). Was I missing something, or does the schema not contain FK
constraints? (If the latter, was this a design decision?)
-hilmar
--
-------------------------------------------------------------
Hilmar Lapp email: la...@gn...
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------
|
|
From: <ja...@op...> - 2001-11-29 19:39:49
|
"Hilmar Lapp" <la...@gn...> writes:
> Quoting "Jason E. Stewart" <ja...@op...>:
>
> >
> > All the DB interfaces are created to be SQL generic. Instead of having
> > a bunch of hard-coded SQL ('SELECT foo FROM bar WHERE baz="NAN"') I
> > created an SQL abstraction layer using three functions
> > create_select_sql(), create_insert_sql(), and
> > create_update_sql(). With these you can plug in backends for any
> > relational DB to produce the SQL you need.
> >
>
> Is this framework already available already? If so, from the GeneX repository
> or MAGEstk?
So all of the create_* functions are all in Genex-1.x, the piece that
isn't there is the XML based schema description.
> I'm trying to reverse engineer the ERD, unsuccessful so far. ERwin would
> not reverse engineer from SQL, and doesn't have an idea about Postgres.
> My next idea was port to Oracle, and reverse engineer with Designer.
One of the reasons I detested ERWin.
> Can the tool framework you mentioned reverse engineer from SQL DDL?
It's an API for helping applications build DBMS-independent queries, I
have no idea if it would be of any use for reverse engineering. I'm
actually hoping to replace the system I wrote with some of the
existing CPAN modules that do very similar things,
e.g. SQL::QueryBuilder::Simple.
> BTW the DDL I was able to dump did not contain FK constraints (bad
> for reverse engineering). Was I missing something, or does the
> schema not contain FK constraints? (If the latter, was this a design
> decision?)
Sorry, DDL = ???
That is one of the irritating consequences of the incident I mentioned
the other day. When management of the DB and schema were taken over by
another individual, he began using pg_dump instead of maintaining the
table definition scripts. Well, GeneX was originally built on
Postgres-6.5, which didn't support foreign key constraints. By the
time we switched to Postgres-7.x which does, it was too late. The
current DB dump had thrown out all the FK constraints :-(
The good news is that in Genex-2.x, they are all back. As well as a
number of other useful constraints for column uniqueness that makes it
a lot easier to check if a given piece of data, e.g. an array design,
is already in the DB or not. This proved very useful for controlling
the insertion and updating of data.
jas.
|
|
From: Hilmar L. <la...@gn...> - 2001-11-30 00:57:42
|
Quoting "Jason E. Stewart" <ja...@op...>:
>
> Sorry, DDL = ???
Data Definition Language. The piece of SQL that defines tables, indexes,
constraints, etc. As opposed to DML (Data Manipulation Language).
So the more we exchange emails, the more it seems to me that it
may be much more useful to implement a specific intermediate system
here first, and put GeneX-2 on trial in order to improve until
release. Any timeline for that already?
-hilmar
--
-------------------------------------------------------------
Hilmar Lapp email: la...@gn...
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------
|
|
From: Todd F. P. <tf...@nc...> - 2001-11-30 01:39:52
|
Y'all, I have a Java application which currently reverse-engineers a
Sybase database into an Oracle database. I don't think it would be much
trouble to generalize this a bit to do Postgres to Oracle or whatever to
whatever. The utility is pretty basic right now...only does tables, views,
some constraints and some permissions. But, it has proved useful as a 'get
started' utility for reverse endgineering of database structure.
Have been meaning to work on it more. Will provide a prototype soon.
Todd
P.S. DDL = data definition language (create table, etc.)
On 29 Nov 2001, Jason E. Stewart wrote:
> "Hilmar Lapp" <la...@gn...> writes:
>
> > Quoting "Jason E. Stewart" <ja...@op...>:
> >
> > >
> > > All the DB interfaces are created to be SQL generic. Instead of having
> > > a bunch of hard-coded SQL ('SELECT foo FROM bar WHERE baz="NAN"') I
> > > created an SQL abstraction layer using three functions
> > > create_select_sql(), create_insert_sql(), and
> > > create_update_sql(). With these you can plug in backends for any
> > > relational DB to produce the SQL you need.
> > >
> >
> > Is this framework already available already? If so, from the GeneX repository
> > or MAGEstk?
>
> So all of the create_* functions are all in Genex-1.x, the piece that
> isn't there is the XML based schema description.
>
> > I'm trying to reverse engineer the ERD, unsuccessful so far. ERwin would
> > not reverse engineer from SQL, and doesn't have an idea about Postgres.
> > My next idea was port to Oracle, and reverse engineer with Designer.
>
> One of the reasons I detested ERWin.
>
> > Can the tool framework you mentioned reverse engineer from SQL DDL?
>
> It's an API for helping applications build DBMS-independent queries, I
> have no idea if it would be of any use for reverse engineering. I'm
> actually hoping to replace the system I wrote with some of the
> existing CPAN modules that do very similar things,
> e.g. SQL::QueryBuilder::Simple.
>
> > BTW the DDL I was able to dump did not contain FK constraints (bad
> > for reverse engineering). Was I missing something, or does the
> > schema not contain FK constraints? (If the latter, was this a design
> > decision?)
>
> Sorry, DDL = ???
>
> That is one of the irritating consequences of the incident I mentioned
> the other day. When management of the DB and schema were taken over by
> another individual, he began using pg_dump instead of maintaining the
> table definition scripts. Well, GeneX was originally built on
> Postgres-6.5, which didn't support foreign key constraints. By the
> time we switched to Postgres-7.x which does, it was too late. The
> current DB dump had thrown out all the FK constraints :-(
>
> The good news is that in Genex-2.x, they are all back. As well as a
> number of other useful constraints for column uniqueness that makes it
> a lot easier to check if a given piece of data, e.g. an array design,
> is already in the DB or not. This proved very useful for controlling
> the insertion and updating of data.
>
> jas.
>
> _______________________________________________
> Genex-dev mailing list
> Gen...@li...
> https://lists.sourceforge.net/lists/listinfo/genex-dev
>
|
|
From: Todd F. P. <tf...@nc...> - 2001-11-30 03:45:26
|
of course, i could generalize it further to create a DoME UML model of
some sort.
todd
On Thu, 29 Nov 2001, Todd F. Peterson wrote:
> Y'all, I have a Java application which currently reverse-engineers a
> Sybase database into an Oracle database. I don't think it would be much
> trouble to generalize this a bit to do Postgres to Oracle or whatever to
> whatever. The utility is pretty basic right now...only does tables, views,
> some constraints and some permissions. But, it has proved useful as a 'get
> started' utility for reverse endgineering of database structure.
>
> Have been meaning to work on it more. Will provide a prototype soon.
>
> Todd
>
> P.S. DDL = data definition language (create table, etc.)
>
> On 29 Nov 2001, Jason E. Stewart wrote:
>
> > "Hilmar Lapp" <la...@gn...> writes:
> >
> > > Quoting "Jason E. Stewart" <ja...@op...>:
> > >
> > > >
> > > > All the DB interfaces are created to be SQL generic. Instead of having
> > > > a bunch of hard-coded SQL ('SELECT foo FROM bar WHERE baz="NAN"') I
> > > > created an SQL abstraction layer using three functions
> > > > create_select_sql(), create_insert_sql(), and
> > > > create_update_sql(). With these you can plug in backends for any
> > > > relational DB to produce the SQL you need.
> > > >
> > >
> > > Is this framework already available already? If so, from the GeneX repository
> > > or MAGEstk?
> >
> > So all of the create_* functions are all in Genex-1.x, the piece that
> > isn't there is the XML based schema description.
> >
> > > I'm trying to reverse engineer the ERD, unsuccessful so far. ERwin would
> > > not reverse engineer from SQL, and doesn't have an idea about Postgres.
> > > My next idea was port to Oracle, and reverse engineer with Designer.
> >
> > One of the reasons I detested ERWin.
> >
> > > Can the tool framework you mentioned reverse engineer from SQL DDL?
> >
> > It's an API for helping applications build DBMS-independent queries, I
> > have no idea if it would be of any use for reverse engineering. I'm
> > actually hoping to replace the system I wrote with some of the
> > existing CPAN modules that do very similar things,
> > e.g. SQL::QueryBuilder::Simple.
> >
> > > BTW the DDL I was able to dump did not contain FK constraints (bad
> > > for reverse engineering). Was I missing something, or does the
> > > schema not contain FK constraints? (If the latter, was this a design
> > > decision?)
> >
> > Sorry, DDL = ???
> >
> > That is one of the irritating consequences of the incident I mentioned
> > the other day. When management of the DB and schema were taken over by
> > another individual, he began using pg_dump instead of maintaining the
> > table definition scripts. Well, GeneX was originally built on
> > Postgres-6.5, which didn't support foreign key constraints. By the
> > time we switched to Postgres-7.x which does, it was too late. The
> > current DB dump had thrown out all the FK constraints :-(
> >
> > The good news is that in Genex-2.x, they are all back. As well as a
> > number of other useful constraints for column uniqueness that makes it
> > a lot easier to check if a given piece of data, e.g. an array design,
> > is already in the DB or not. This proved very useful for controlling
> > the insertion and updating of data.
> >
> > jas.
> >
> > _______________________________________________
> > Genex-dev mailing list
> > Gen...@li...
> > https://lists.sourceforge.net/lists/listinfo/genex-dev
> >
>
>
> _______________________________________________
> Genex-dev mailing list
> Gen...@li...
> https://lists.sourceforge.net/lists/listinfo/genex-dev
>
|
|
From: Hilmar L. <la...@gn...> - 2001-11-30 19:32:44
|
Sounds very helpful. I don't dare to ask whether you got time :)
-hilmar
--
-------------------------------------------------------------
Hilmar Lapp email: la...@gn...
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------
Quoting "Todd F. Peterson" <tf...@nc...>:
> of course, i could generalize it further to create a DoME UML model of
> some sort.
>
> todd
>
> On Thu, 29 Nov 2001, Todd F. Peterson wrote:
>
> > Y'all, I have a Java application which currently reverse-engineers a
> > Sybase database into an Oracle database. I don't think it would be
> much
> > trouble to generalize this a bit to do Postgres to Oracle or whatever
> to
> > whatever. The utility is pretty basic right now...only does tables,
> views,
> > some constraints and some permissions. But, it has proved useful as a
> 'get
> > started' utility for reverse endgineering of database structure.
> >
> > Have been meaning to work on it more. Will provide a prototype soon.
> >
> > Todd
> >
|
|
From: Todd F. P. <tf...@nc...> - 2001-11-30 20:04:19
|
I need to make time for it since that will help not only GeneX, but other projects as well. Hopefully, next week will allow me some time to do some work on this. Todd On Fri, 30 Nov 2001, Hilmar Lapp wrote: > > Sounds very helpful. I don't dare to ask whether you got time :) > > -hilmar > -- > ------------------------------------------------------------- > Hilmar Lapp email: la...@gn... > GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 > ------------------------------------------------------------- > > Quoting "Todd F. Peterson" <tf...@nc...>: > > > of course, i could generalize it further to create a DoME UML model of > > some sort. > > > > todd > > > > On Thu, 29 Nov 2001, Todd F. Peterson wrote: > > > > > Y'all, I have a Java application which currently reverse-engineers a > > > Sybase database into an Oracle database. I don't think it would be > > much > > > trouble to generalize this a bit to do Postgres to Oracle or whatever > > to > > > whatever. The utility is pretty basic right now...only does tables, > > views, > > > some constraints and some permissions. But, it has proved useful as a > > 'get > > > started' utility for reverse endgineering of database structure. > > > > > > Have been meaning to work on it more. Will provide a prototype soon. > > > > > > Todd > > > > |
|
From: Todd F. P. <tf...@nc...> - 2001-12-12 16:05:35
Attachments:
genex-lite.gif
|
I have completed a loader that uses a very simple schema. The original datafile is referenced int the File_Properties table. When data is needed it is loaded into a temporary table on-the-fly. When database is near capacity, or performance gets bad. These tables may be unloaded. It's kind of a caching mechanism. Attached is picture of schema. Will try to get this on the sourceforge site or the genex.ncgr.org site soon. Todd On Fri, 30 Nov 2001, Hilmar Lapp wrote: > > Sounds very helpful. I don't dare to ask whether you got time :) > > -hilmar > -- > ------------------------------------------------------------- > Hilmar Lapp email: la...@gn... > GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 > ------------------------------------------------------------- > > Quoting "Todd F. Peterson" <tf...@nc...>: > > > of course, i could generalize it further to create a DoME UML model of > > some sort. > > > > todd > > > > On Thu, 29 Nov 2001, Todd F. Peterson wrote: > > > > > Y'all, I have a Java application which currently reverse-engineers a > > > Sybase database into an Oracle database. I don't think it would be > > much > > > trouble to generalize this a bit to do Postgres to Oracle or whatever > > to > > > whatever. The utility is pretty basic right now...only does tables, > > views, > > > some constraints and some permissions. But, it has proved useful as a > > 'get > > > started' utility for reverse endgineering of database structure. > > > > > > Have been meaning to work on it more. Will provide a prototype soon. > > > > > > Todd > > > > |
|
From: <ja...@op...> - 2001-12-14 03:23:33
|
"Todd F. Peterson" <tf...@nc...> writes: > I have completed a loader that uses a very simple schema. The > original datafile is referenced int the File_Properties table. When > data is needed it is loaded into a temporary table on-the-fly. When > database is near capacity, or performance gets bad. These tables may > be unloaded. It's kind of a caching mechanism. Attached is picture > of schema. Will try to get this on the sourceforge site or the > genex.ncgr.org site soon. Hey Todd, Sorry, I'm a bit confused. Why are you loading to a separate schema and not to GeneX proper? How does that help get data into GeneX? jas. |
|
From: Todd F. P. <tf...@nc...> - 2001-12-14 07:32:33
|
It's not far off. with some views, it could mimic the genex proper tables. todd On 13 Dec 2001, Jason E. Stewart wrote: > "Todd F. Peterson" <tf...@nc...> writes: > > > I have completed a loader that uses a very simple schema. The > > original datafile is referenced int the File_Properties table. When > > data is needed it is loaded into a temporary table on-the-fly. When > > database is near capacity, or performance gets bad. These tables may > > be unloaded. It's kind of a caching mechanism. Attached is picture > > of schema. Will try to get this on the sourceforge site or the > > genex.ncgr.org site soon. > > Hey Todd, > > Sorry, I'm a bit confused. Why are you loading to a separate schema > and not to GeneX proper? How does that help get data into GeneX? > > jas. > > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > |
|
From: Todd F. P. <tf...@nc...> - 2001-12-14 07:52:54
|
oh yeah, it's fast and easy. that was not the case with the curation tool/xml mechanism. we figure, first get the data in the db and then annotate as necessary. this will be good for small labs. hopefully, the design will allow scaleability to larger environments. also, after loading one additional dataset, i saw a big decrease in performance from the db. i'm attempting to solve that problem. the mechanism i am proposing allows the flexibility of having all of the data in the db, or just what is needed. bottom line, we're experimenting by taking small steps and designing/implementing each one well while building on previous steps. todd On 13 Dec 2001, Jason E. Stewart wrote: > "Todd F. Peterson" <tf...@nc...> writes: > > > I have completed a loader that uses a very simple schema. The > > original datafile is referenced int the File_Properties table. When > > data is needed it is loaded into a temporary table on-the-fly. When > > database is near capacity, or performance gets bad. These tables may > > be unloaded. It's kind of a caching mechanism. Attached is picture > > of schema. Will try to get this on the sourceforge site or the > > genex.ncgr.org site soon. > > Hey Todd, > > Sorry, I'm a bit confused. Why are you loading to a separate schema > and not to GeneX proper? How does that help get data into GeneX? > > jas. > > _______________________________________________ > Genex-dev mailing list > Gen...@li... > https://lists.sourceforge.net/lists/listinfo/genex-dev > |
|
From: <ja...@op...> - 2001-12-15 23:08:11
|
"Todd F. Peterson" <tf...@nc...> writes: > oh yeah, it's fast and easy. that was not the case with the curation > tool/xml mechanism. we figure, first get the data in the db and then > annotate as necessary. this will be good for small labs. hopefully, > the design will allow scaleability to larger environments. I'm all for lightwieght, fast, and easy, as well as annotation after the data is in the DB. None of that impacts what schema you use. If you invent a whole new schema you'll have to invent completely new tools that can work with that schema. If you store the data in a subset of the GeneX schema, you can use all the tools that work for GeneX. What is the advantage of the new schema? jas. PS. I'm on the list, you don't have to reply to me personally. |
Thanks for helping keep SourceForge clean.
X