[Gusdev-gusdev] Pipeline Manager change

[Dave B. <db...@pc...>]

Hey all,

I noticed something that is a bit of a pain in the neck when running our 
pipeline; whenever we load external data, we have to make sure that there 
is a new entry in the ExternalDatabaseRelease table for that data.  The 
way I've always handled this is to make those by hand before the pipeline 
runs and set the database release IDs in the pipeline properties file.

I think a better way would be to have a step in the Pipeline that does it 
for you for each database release you have to load.  Values for the 
database release entry could either go in the properties file or be parsed 
from the file you are loading depending on availability.

This is easy enough to implement, but often we need to use the database 
release ids later in the pipeline.  There isn't any way to automatically 
set these as internal properties; we really need to add them by hand to 
the pipeline properties file.

I am going to add a method to Manager.pm that the pipeline can call named 
"waitForUser" (or something similar).  It will just exit the pipeline with 
a message, and in this case it can say "Please set the following property 
in the pipeline properties file:  flyDB=XXXX".  It is sort of a generic 
implementation of the 'exitToLiniac' method that we have already.  Then 
the user can set this property and start the pipeline again.

Dave