Re: [Gusdev-gusdev] Pipeline Manager change

[Dave B. <db...@pc...>]

Hey John--see below.

On Tue, 26 Oct 2004, John Iodice wrote:

> Adding a pipeline step for new external database release IDs is a great
> idea.  I have noticed in the past that when people insert a record
> "manually" (that is, with UpdateGusFromXML or SubmitRow), they tend to
> supply only the minimum info, to the extent of populating required
> fields with the string of "unknown".  I've been guilty of this myself.
> Hopefully, records created programmatically will document the DB release
> better.

Me too, but what I had in mind wasn't explicit functionality to load new 
DB releases, just a generic step in the Manager that exits and prints out 
information to the user.  I do remember writing a simple plugin that loads 
a new release into ExternalDatabaseRelease; it is in Common but I don't 
think it adds much over SubmitRow except to name ExternalDatabaseRelease 
attributes explicitly as parameters.

>
> Question: is there a way to avoid editing the properties file to include
> the new release ID?
>
> In many cases, we want the latest release of the given external
> database.  Could we create a utility that takes an external database ID
> and finds the ID of its latest release?

The only thing I could think of is to query for the latest release for a 
particular database, but this is dangerous; there could well be new 
releases loaded that for one reason or another we don't want to use (I 
think that is the case with GO right now; there is a new release loaded 
since the last time we made rules against GO terms.)  Maybe the latest 
release could be the default though, unless the user overrides it with a 
property.

I'd be interested to hear Steve's plans for loading 3rd party data and 
whether they include handling entries in the XDBR table.

Dave



>
> If we could do something like that, it would not only simplify a manual
> step, it would completely automate it.
>
> On Mon, 2004-10-25 at 17:39, Dave Barkan wrote:
>> Hey all,
>>
>> I noticed something that is a bit of a pain in the neck when running our
>> pipeline; whenever we load external data, we have to make sure that there
>> is a new entry in the ExternalDatabaseRelease table for that data.  The
>> way I've always handled this is to make those by hand before the pipeline
>> runs and set the database release IDs in the pipeline properties file.
>>
>> I think a better way would be to have a step in the Pipeline that does it
>> for you for each database release you have to load.  Values for the
>> database release entry could either go in the properties file or be parsed
>> from the file you are loading depending on availability.
>>
>> This is easy enough to implement, but often we need to use the database
>> release ids later in the pipeline.  There isn't any way to automatically
>> set these as internal properties; we really need to add them by hand to
>> the pipeline properties file.
>>
>> I am going to add a method to Manager.pm that the pipeline can call named
>> "waitForUser" (or something similar).  It will just exit the pipeline with
>> a message, and in this case it can say "Please set the following property
>> in the pipeline properties file:  flyDB=XXXX".  It is sort of a generic
>> implementation of the 'exitToLiniac' method that we have already.  Then
>> the user can set this property and start the pipeline again.
>>
>> Dave
>>
>>
>> -------------------------------------------------------
>> This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
>> Use IT products in your business? Tell us what you think of them. Give us
>> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
>> http://productguide.itmanagersjournal.com/guidepromo.tmpl
>> _______________________________________________
>> Gusdev-gusdev mailing list
>> Gus...@li...
>> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Gusdev-gusdev mailing list
> Gus...@li...
> https://lists.sourceforge.net/lists/listinfo/gusdev-gusdev
>