pipeline130-developers Mailing List for pipeline cs130 (Page 4)

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Sorry, forgot the attachment.

N

CODE FREEZE

I have updated the cluster so that the group will have pre-compiled jar
files and example files.  I tested a few to make sure they work (I tested
the submit, list, and status commands, nice work all!).

Attached is the testing manual (I haven't spell-checked yet).

I am going to now format the Design and Requirements docs from the wiki
(don't change the wiki either).

I will send out another email by 5pm with all three docs (testing manual,
design doc, requirements doc).

Nils

Hi Jason,

Let us know how you'd like us to test your app. Feel free to send the information directly to me (cs...@uc...) or to our entire group at pip...@li.... 

Thanks!

-Chris
I will add those to the manual.  I can set up the branch for them at 3pm.

Nils

On 5/27/09 3:09 AM, "Chris Swan" <cs...@uc...> wrote:

> configuration or arguments are required. The Manager
> works as follows:

I agree that we should create a tag, but we should try to have a pre-built 
copy of that tag on the cluster for them, preferably in jar form so that 
they can run the applications easier without the classpath stuff. Ideally 
they could just log in and run "java -jar wfengine.jar", "java -jar 
wfmanager.jar ...". You just need to copy the built files that NetBeans 
produces. I can take care of that if necessary.

Also, your ssh command says to enter "ssh 
cs1...@ro..." when it should be @neuron.

We should probably mention something like:
To run the Engine, no configuration or arguments are required. The Manager 
works as follows:

Submitting a workflow from an xml file
    wfmanager submit -e engineHostname -w workflowXmlFile

View a list of all workflows (optionally filtered by name)
    wfmanager list -e engineHostname [-n workflowName]

View that status of a particular workflow, specified by the unique ID found 
in the listing
    wfmanager status -e engineHostname -w workflowID

Where engineHostname should be localhost.

I've improved the clarity of the workflow/job statuses, and added some 
progress messages at various points. I am going to add some better 
error-checking on arguments and also try to make the engine acknowledge 
receipt of workflows by returning their ID.

-Chris

--------------------------------------------------
From: "Nils Homer" <nil...@uc...>
Sent: Tuesday, May 26, 2009 8:15 PM
To: <pip...@li...>
Subject: Re: [Pipeline130-developers] Outside testing

> Attached is a first pass at the instructions.  We also need to compile the
> new design and requirements document.  I can do that too if no one 
> objects.
>
> Feel free to comment on missing sections.  I think we should create a tag
> for the code to be tested by the other group.  I also created a new user
> account for their group.  Note: I asked them to email us when they want to
> test the code since we only want one instance running at a time etc.
>
> Nils
>
>
> On 5/25/09 9:59 PM, "cs...@uc..." <cs...@uc...> wrote:
>
>> So the professor updated the website:
>> http://www.cs.ucla.edu/~rupak/Courses/spring09/projects/proj5.html
>>
>> The highlight:
>>
>> You must send an email by 6:00pm on May 27 to the team who has
>> reviewed your project (cc the TAs) with complete instructions on how
>> to download/install/run the code. (This could for example, be a
>> special account on your server.) Teams that do not provide access to
>> their code in a timely manner will be penalized. If your requirements
>> and design documents have changed since the design review, send them
>> too to your testing team.
>>
>>
>> Each team must submit a copy of this completed document by 11:59pm on
>> 6/2/2009.
>>
>>
>> We need to set up some simple scripts for them to run to start
>> workflows of various complexities, along with instructions. And we
>> need to do it soon!
>>
>> -Chris
>>
>> ------------------------------------------------------------------------------
>> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
>> is a gathering of tech-side developers & brand creativity professionals. 
>> Meet
>> the minds behind Google Creative Lab, Visual Complexity, Processing, &
>> iPhoneDevCamp asthey present alongside digital heavyweights like 
>> Barbarian
>> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
>> _______________________________________________
>> Pipeline130-developers mailing list
>> Pip...@li...
>> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
>
>

> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals. 
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp as they present alongside digital heavyweights like 
> Barbarian
> Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com

> _______________________________________________
> Pipeline130-developers mailing list
> Pip...@li...
> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
> 

I have uploaded the new changes (templated job args and templated command
args) to the repository as well as updated the design document.  I will code
up some demos for the testing group.

Lets shoot for 3pm as a hard deadline to get all the code changes (don't
break it -> note to self) as well as all revisions to the design and
requirement documents.

At 3pm I will create the instructions document, the design document, and the
requirements document.  The latter two will be taken from the wiki and will
be uploaded to the repository for storage.  I will also create a "tagged"
version in the "tags" directory for the testers to use.

Let me know if anyone has any questions.

Nils 

I am going to modify the XML schema (and source code) to remove the
arguments to the command (since it is a shell script) and add arguments for
job submission (useful for specifying required resources, architecture,
etc.).  I will then try to update the examples.  If you have any code or
example programs that rely on this that are committed to the repository let
me know where and I will update it.

I will setup a test user account and write some sample scripts.  I can also
write the instructions later tonight.  Does this sound ok to everyone if I
take this one on?

Chris: what ports/hostname were you using to run the jobs?

Nils

On 5/25/09 9:59 PM, "cs...@uc..." <cs...@uc...> wrote:

> So the professor updated the website:
> http://www.cs.ucla.edu/~rupak/Courses/spring09/projects/proj5.html
> 
> The highlight:
> 
> You must send an email by 6:00pm on May 27 to the team who has
> reviewed your project (cc the TAs) with complete instructions on how
> to download/install/run the code. (This could for example, be a
> special account on your server.) Teams that do not provide access to
> their code in a timely manner will be penalized. If your requirements
> and design documents have changed since the design review, send them
> too to your testing team.
> 
> 
> Each team must submit a copy of this completed document by 11:59pm on
> 6/2/2009.
> 
> 
> We need to set up some simple scripts for them to run to start
> workflows of various complexities, along with instructions. And we
> need to do it soon!
> 
> -Chris
> 
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals. Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> _______________________________________________
> Pipeline130-developers mailing list
> Pip...@li...
> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers

So the professor updated the website:
http://www.cs.ucla.edu/~rupak/Courses/spring09/projects/proj5.html

The highlight:

You must send an email by 6:00pm on May 27 to the team who has
reviewed your project (cc the TAs) with complete instructions on how
to download/install/run the code. (This could for example, be a
special account on your server.) Teams that do not provide access to
their code in a timely manner will be penalized. If your requirements
and design documents have changed since the design review, send them
too to your testing team.

Each team must submit a copy of this completed document by 11:59pm on
6/2/2009.

We need to set up some simple scripts for them to run to start
workflows of various complexities, along with instructions. And we
need to do it soon!

-Chris

We should probably consider a design for how we would support this, and 
include it in our docs (and maybe even our presentation). In terms of 
implementation, not sure if we should bother with it now -- our primary use 
case is in a lab where everyone is trusted with root access anyway, right?

--------------------------------------------------
From: "Nils Homer" <nil...@uc...>
Sent: Sunday, May 24, 2009 5:03 PM
To: <pip...@li...>
Subject: [Pipeline130-developers] User permissions

> Another problem that I thought of is that the user of a wfmanager instance
> should not be able to access other users' workflows.  This could be a
> security concern.
>
> Nils
>
>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals. 
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> _______________________________________________
> Pipeline130-developers mailing list
> Pip...@li...
> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
> 

I like #1. Isn't it already a requirement of SGE that all commands are shell 
scripts ( http://people.sc.fsu.edu/~burkardt/examples/sge/sge.html )? 
Therefore, for the lab at least, it shouldn't require any changes. I don't 
really understand your argument against this: "there cannot be any arguments 
to a shell script in this context". DRMAA JobTemplates take a list of 
parameters and the scripts can parse these parameters however they'd like. 
If we simply state that all commands must be scripts, failure to do so will 
be immediately discovered by the user when their job doesn't work.

-Chris

--------------------------------------------------
From: "Nils Homer" <nil...@uc...>
Sent: Sunday, May 24, 2009 5:02 PM
To: <pip...@li...>
Subject: [Pipeline130-developers] Question

> As Chris mentioned before, you submit a shell script not a binary 
> executable
> to the cluster.  So we have two options.
>
> 1. Require that all "commands" are in fact shell scripts.
>
> 2. Wrap every "command" in a shell script.
>
> #1 would push the complexity to the "editor", since then there cannot be 
> any
> arguments to a shell script in this context.
>
> #2 could be a problem since where do we store all of these shell scripts?
> We could have a common engine run directory with each wrapper shell script
> named by their unique id etc.
>
> I prefer #2 since then users do not have to put their command line
> executables into a shell script before creating a workflow.
>
> Nils
>
>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals. 
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> _______________________________________________
> Pipeline130-developers mailing list
> Pip...@li...
> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
> 

Another problem that I thought of is that the user of a wfmanager instance
should not be able to access other users' workflows.  This could be a
security concern.

Nils

As Chris mentioned before, you submit a shell script not a binary executable
to the cluster.  So we have two options.

1. Require that all "commands" are in fact shell scripts.

2. Wrap every "command" in a shell script.

#1 would push the complexity to the "editor", since then there cannot be any
arguments to a shell script in this context.

#2 could be a problem since where do we store all of these shell scripts?
We could have a common engine run directory with each wrapper shell script
named by their unique id etc.

I prefer #2 since then users do not have to put their command line
executables into a shell script before creating a workflow.

Nils

I was able to get ANT to build our code on the cluster.  I then used the
command ant suggested to run the engine/manager (it needed the classpath).

For example:

java -cp 
"/home/cs130/nhomer/pipeline130/trunk/wflib/dist/wflib.jar:/home/cs130/nhome
r/pipeline130/trunk/wfmanager/java-getopt-1.0.13.jar:/home/cs130/nhomer/pipe
line130/trunk/wfmanager/dist/wfmanager.jar" wfmanager.Main

Nils

You can use any other command installed on the system (ex. Echo, awk, sed,
sort, uniq, cat, tail, head, grep, ...)

On 5/24/09 8:53 AM, "Justin (Weiguang Si)" <fo...@gm...> wrote:

>    By the way, besides "sleep", what other legal commands can we use?
> 
> 2009/5/23 Nils Homer <nil...@uc...>
>> Everything needs a test case if possible.  Only do what you think is
>> reasonable. 
>> 
>> 
>> 
>> On 5/23/09 6:29 PM, "Justin (Weiguang Si)" <fo...@gm...> wrote:
>> 
>>>    I saw them. So templated workflows also need test cases?
>>> 
>>> 2009/5/23 Nils Homer <nil...@uc...>
>>>> 
>>>> Re templated workflows.
>>>> 
>>>> Check out the examples directory.  Templated workflows should exist there.
>>>> 
>>>> Nils
>>>> 
>>>> 
>>>> 
>>>> ---------------------------------------------------------------------------
>>>> ---
>>>> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
>>>> is a gathering of tech-side developers & brand creativity professionals.
>>>> Meet
>>>> the minds behind Google Creative Lab, Visual Complexity, Processing, &
>>>> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
>>>> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
>>>> <http://www.creativitycat.com/>  <http://www.creativitycat.com/>
>>>> _______________________________________________
>>>> Pipeline130-developers mailing list
>>>> Pip...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
>>> 
>>> 
>> 
>> 
>> 
----------------------------------------------------------------------------->>
-
>> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
>> is a gathering of tech-side developers & brand creativity professionals. Meet
>> the minds behind Google Creative Lab, Visual Complexity, Processing, &
>> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
>> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
>> <http://www.creativitycat.com/>
>> _______________________________________________
>> Pipeline130-developers mailing list
>> Pip...@li...
>> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
>> 
> 
> 

   By the way, besides "sleep", what other legal commands can we use?

2009/5/23 Nils Homer <nil...@uc...>

> Everything needs a test case if possible.  Only do what you think is
> reasonable.
>
>
> On 5/23/09 6:29 PM, "Justin (Weiguang Si)" <fo...@gm...> wrote:
>
>     I saw them. So templated workflows also need test cases?
>
> 2009/5/23 Nils Homer <nil...@uc...>
>
>
> Re templated workflows.
>
> Check out the examples directory.  Templated workflows should exist there.
>
> Nils
>
>
>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals.
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> <http://www.creativitycat.com/> <http://www.creativitycat.com/>
> _______________________________________________
> Pipeline130-developers mailing list
> Pip...@li...
> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals.
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> _______________________________________________
> Pipeline130-developers mailing list
> Pip...@li...
> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
>
>

-- 
--------------------------------------------------
Justin (Weiguang Si)

Ph.D Candidate
Department of Computer Science
University of California, Los Angeles

fo...@gm...
fo...@uc...

Everything needs a test case if possible.  Only do what you think is
reasonable.

On 5/23/09 6:29 PM, "Justin (Weiguang Si)" <fo...@gm...> wrote:

>    I saw them. So templated workflows also need test cases?
> 
> 2009/5/23 Nils Homer <nil...@uc...>
>> 
>> Re templated workflows.
>> 
>> Check out the examples directory.  Templated workflows should exist there.
>> 
>> Nils
>> 
>> 
>> 
>> 
----------------------------------------------------------------------------->>
-
>> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
>> is a gathering of tech-side developers & brand creativity professionals. Meet
>> the minds behind Google Creative Lab, Visual Complexity, Processing, &
>> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
>> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
>> <http://www.creativitycat.com/>
>> _______________________________________________
>> Pipeline130-developers mailing list
>> Pip...@li...
>> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
> 
> 

   I saw them. So templated workflows also need test cases?

2009/5/23 Nils Homer <nil...@uc...>

>
> Re templated workflows.
>
> Check out the examples directory.  Templated workflows should exist there.
>
> Nils
>
>
>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals.
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> _______________________________________________
> Pipeline130-developers mailing list
> Pip...@li...
> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
>

-- 
--------------------------------------------------
Justin (Weiguang Si)

Ph.D Candidate
Department of Computer Science
University of California, Los Angeles

fo...@gm...
fo...@uc...

Re templated workflows.

Check out the examples directory.  Templated workflows should exist there.

Nils

>    (1) I wrote two test cases according to the format of files in the
> examples file folder. I don't know whether they are right.
>        One (test.samename.xml) is the test case which has commands with the
> same name, the other (test.cyclic.xml) is the test case which has a cycle.
>        To design a invalid workflow of cyclic graph which is complex enough
> for our test, how many nodes should it have approximately?

Those both look good -- commit them to SVN.

I think we might want to have one or two more cycle-detecting test  
cases just because it is a tricky situation with a lot of corner  
cases. For example, we should have one with a simple two-cycle (i.e. 1  
-> 2 -> 1). That's the only thing I can think of at the moment, but  
there could be more.

>    (2) About the file
> "failure.dependent.start.xml","failure.dependent.mid.xml" and
> "failure.dependent.end.xml" in the examples file folder, I am not clear why
> they fail, especially abou the file "failure.dependent.end.xml".

I'm not at home right now so I can't look up those files, but I  
believe the failure is that one of the scripts they run always returns  
an error code, so the jobs should be marked as "failed".

>    (3) About other test cases, yesterday Nils told me I should look into the
> details of the code to design test cases. Did he mean where the exception
> might happen, like unique command name, valid node type, etc? Or I should
> design cases where the program doesn't cover.

Both. You should try to design cases that test specific things we  
catch, but also things that we do not check for that should be errors.

>    (4) Should I design test cases of templated workflow?

At the moment I'm not totally sure how the templated workflows have  
been implemented. Nils might be able to better answer this.

Another thing we need to do is write the code that actually executes  
these tests.

-Chris

Quoting "Justin (Weiguang Si)" <fo...@gm...>:

>      I have some questions about the test cases.
>
>    (1) I wrote two test cases according to the format of files in the
> examples file folder. I don't know whether they are right.
>        One (test.samename.xml) is the test case which has commands with the
> same name, the other (test.cyclic.xml) is the test case which has a cycle.
>        To design a invalid workflow of cyclic graph which is complex enough
> for our test, how many nodes should it have approximately?
>
>    (2) About the file
> "failure.dependent.start.xml","failure.dependent.mid.xml" and
> "failure.dependent.end.xml" in the examples file folder, I am not clear why
> they fail, especially abou the file "failure.dependent.end.xml".
>
>    (3) About other test cases, yesterday Nils told me I should look into the
> details of the code to design test cases. Did he mean where the exception
> might happen, like unique command name, valid node type, etc? Or I should
> design cases where the program doesn't cover.
>
>    (4) Should I design test cases of templated workflow?
>
>     Thanks a lot for your help.
>

   It seems that my questions are not displayed properly. The following are
my questions:

   (1) I wrote two test cases according to the format of files in the
examples file folder. I don't know whether they are right.
       One (test.samename.xml) is the test case which has commands with the
same name, the other (test.cyclic.xml) is the test case which has a cycle.
       To design a invalid workflow of cyclic graph which is complex enough
for our test, how many nodes should it have approximately?

   (2) About the file
"failure.dependent.start.xml","failure.dependent.mid.xml" and
"failure.dependent.end.xml" in the examples file folder, I am not clear why
they fail, especially abou the file "failure.dependent.end.xml".

   (3) About other test cases, yesterday Nils told me I should look into the
details of the code to design test cases. Did he mean where the exception
might happen, like unique command name, valid node type, etc? Or I should
design cases where the program doesn't cover.

   (4) Should I design test cases of templated workflow?

    Thanks a lot for your help.

2009/5/23 Justin (Weiguang Si) <fo...@gm...>

>    I have some questions about the test cases.
>
>
>    (1) I wrote two test cases according to the format of files in the
> examples file folder. I don't know whether they are right.
>        One (test.samename.xml) is the test case which has commands with the
> same name, the other (test.cyclic.xml) is the test case which has a cycle.
>        To design a invalid workflow of cyclic graph which is complex enough
> for our test, how many nodes should it have approximately?
>
>    (2) About the file
> "failure.dependent.start.xml","failure.dependent.mid.xml" and
> "failure.dependent.end.xml" in the examples file folder, I am not clear why
> they fail, especially abou the file "failure.dependent.end.xml".
>
>    (3) About other test cases, yesterday Nils told me I should look into
> the details of the code to design test cases. Did he mean where the
> exception might happen, like unique command name, valid node type, etc? Or I
> should design cases where the program doesn't cover.
>
>    (4) Should I design test cases of templated workflow?
>
>     Thanks a lot for your help.
>

-- 
--------------------------------------------------
Justin (Weiguang Si)

Ph.D Candidate
Department of Computer Science
University of California, Los Angeles

fo...@gm...
fo...@uc...

I added some functions and classes that get a list of the workflow  
names and statuses.  I threw one in there that gets the list of all  
workflows with a given name and their id's and statuses.  So with that  
function, a user can query by name and using that get the unique  
Identifier of the one he wants.  Included is the corresponding time  
since the workflow was started.  In these functions however, I do have  
to iterate over every single job to determine if the workflow is  
complete or not, so still not as efficient as it could be, like if  
there was a way to store the workflow status in the workflow.

-Mike

On May 22, 2009, at 5:54 PM, Chris Swan wrote:

> Oops -- accidentally sent to Mike directly...
>
> --------------------------------------------------
> From: "Chris Swan" <cs...@uc...>
> Sent: Friday, May 22, 2009 5:51 PM
> To: "Mike Hess" <mh...@uc...>
> Subject: Re: [Pipeline130-developers] Meeting
>
>>> Should ./wfmanager list get a list of workflowStatusReports?   
>>> Then  just
>>> print out simple statuses?  Or should I make a new class with  less
>>> information.  It may save on the efficiency but will require more   
>>> work.
>>
>> The problem I see with sending over all the WorkflowStatusReports  
>> is that
>> there could be a LOT of them -- and we should have an option to  
>> list every
>> workflow that the Engine knows about, past and present. In that  
>> scenario,
>> it seems like it'd be nicer to only worry about the smaller list of  
>> names
>> + IDs + status, but I do see the benefit of just sending over the  
>> regular
>> reports...
>>
>>
>>> And for looking up workflows by a nice name, should I add a  
>>> dateTime  to
>>> a workflow?  Or another option may be getting a list of workflows   
>>> with
>>> the same name, and either 1) printing them all out, or  2)   
>>> printing it
>>> out if it only has one object in the list. If it has more  then one
>>> object however, it could list all the workflows with that  name as  
>>> per
>>> the "./wfmanager list"
>>
>> It might be a good idea to add the time to Workflows, at the very  
>> least
>> for reporting purposes. Your list idea sounds good, but won't be  
>> useful
>> unless we provide submit time anyway (how will they tell them apart
>> otherwise?). At this point, I don't think it's super important to  
>> hammer
>> out the details as long as we have something working.
>>
>> -Chris
>>
>> --------------------------------------------------
>> From: "Mike Hess" <mh...@uc...>
>> Sent: Friday, May 22, 2009 5:25 PM
>> To: <cs...@uc...>
>> Subject: Re: [Pipeline130-developers] Meeting
>>
>>> Should ./wfmanager list get a list of workflowStatusReports?   
>>> Then  just
>>> print out simple statuses?  Or should I make a new class with  less
>>> information.  It may save on the efficiency but will require more   
>>> work.
>>>
>>> And for looking up workflows by a nice name, should I add a  
>>> dateTime  to
>>> a workflow?  Or another option may be getting a list of workflows   
>>> with
>>> the same name, and either 1) printing them all out, or  2)   
>>> printing it
>>> out if it only has one object in the list. If it has more  then one
>>> object however, it could list all the workflows with that  name as  
>>> per
>>> the "./wfmanager list"
>>>
>>>
>>>
>>> -Mike
>>>
>>> On May 22, 2009, at 3:23 PM, cs...@uc... wrote:
>>>
>>>> Last night I hooked up the manager's ability to query workflow   
>>>> status.
>>>> Theoretically, you can look up a workflow's status with ./  
>>>> wfmanager
>>>> status -e enginehost -w workflowID . However, there's no  way to
>>>> discover workflow IDs. There are two things we should do to  fix  
>>>> this:
>>>>
>>>> 1) Add a "./wfmanager list" that lists all workflows by name and  
>>>> ID,
>>>> with their status (completed, failed, pending).
>>>>
>>>> 2) Allow looking up workflows by a nice name. If multiple  
>>>> workflows  are
>>>> found with the same name (a common scenario), show the most
>>>> recently-submitted one. To implement this we need to actually give
>>>> workflows names specified in the XML file, which I don't think we  
>>>> do
>>>> right now.
>>>>
>>>> -Chris
>>>>
>>>> Quoting Mike Hess <mh...@uc...>:
>>>>
>>>>> I was sick so I wasn't able to make it to the meeting.  I think we
>>>>> have to have a working version done soon for alpha testing, so I  
>>>>> was
>>>>> wondering what are the key functions that still need to be coded  
>>>>> so I
>>>>> can help out with them.  I was looking through the code and I  
>>>>> wasn't
>>>>> sure what else needed to be done.
>>>>>
>>>>>
>>>>> -Mike
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Register Now for Creativity and Technology (CaT), June 3rd, NYC.  
>>>>> CaT
>>>>> is a gathering of tech-side developers & brand creativity
>>>>> professionals. Meet
>>>>> the minds behind Google Creative Lab, Visual Complexity,   
>>>>> Processing, &
>>>>> iPhoneDevCamp asthey present alongside digital heavyweights like
>>>>> Barbarian
>>>>> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
>>>>> _______________________________________________
>>>>> Pipeline130-developers mailing list
>>>>> Pip...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/pipeline130- 
>>>>> developers
>>>>>
>>>>
>>>>
>>>
>>>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity  
> professionals. Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like  
> Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> _______________________________________________
> Pipeline130-developers mailing list
> Pip...@li...
> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers

Oops -- accidentally sent to Mike directly...

--------------------------------------------------
From: "Chris Swan" <cs...@uc...>
Sent: Friday, May 22, 2009 5:51 PM
To: "Mike Hess" <mh...@uc...>
Subject: Re: [Pipeline130-developers] Meeting

>> Should ./wfmanager list get a list of workflowStatusReports?  Then  just 
>> print out simple statuses?  Or should I make a new class with  less 
>> information.  It may save on the efficiency but will require more  work.
>
> The problem I see with sending over all the WorkflowStatusReports is that 
> there could be a LOT of them -- and we should have an option to list every 
> workflow that the Engine knows about, past and present. In that scenario, 
> it seems like it'd be nicer to only worry about the smaller list of names 
> + IDs + status, but I do see the benefit of just sending over the regular 
> reports...
>
>
>> And for looking up workflows by a nice name, should I add a dateTime  to 
>> a workflow?  Or another option may be getting a list of workflows  with 
>> the same name, and either 1) printing them all out, or  2)  printing it 
>> out if it only has one object in the list. If it has more  then one 
>> object however, it could list all the workflows with that  name as per 
>> the "./wfmanager list"
>
> It might be a good idea to add the time to Workflows, at the very least 
> for reporting purposes. Your list idea sounds good, but won't be useful 
> unless we provide submit time anyway (how will they tell them apart 
> otherwise?). At this point, I don't think it's super important to hammer 
> out the details as long as we have something working.
>
> -Chris
>
> --------------------------------------------------
> From: "Mike Hess" <mh...@uc...>
> Sent: Friday, May 22, 2009 5:25 PM
> To: <cs...@uc...>
> Subject: Re: [Pipeline130-developers] Meeting
>
>> Should ./wfmanager list get a list of workflowStatusReports?  Then  just 
>> print out simple statuses?  Or should I make a new class with  less 
>> information.  It may save on the efficiency but will require more  work.
>>
>> And for looking up workflows by a nice name, should I add a dateTime  to 
>> a workflow?  Or another option may be getting a list of workflows  with 
>> the same name, and either 1) printing them all out, or  2)  printing it 
>> out if it only has one object in the list. If it has more  then one 
>> object however, it could list all the workflows with that  name as per 
>> the "./wfmanager list"
>>
>>
>>
>> -Mike
>>
>> On May 22, 2009, at 3:23 PM, cs...@uc... wrote:
>>
>>> Last night I hooked up the manager's ability to query workflow  status. 
>>> Theoretically, you can look up a workflow's status with ./ wfmanager 
>>> status -e enginehost -w workflowID . However, there's no  way to 
>>> discover workflow IDs. There are two things we should do to  fix this:
>>>
>>> 1) Add a "./wfmanager list" that lists all workflows by name and ID, 
>>> with their status (completed, failed, pending).
>>>
>>> 2) Allow looking up workflows by a nice name. If multiple workflows  are 
>>> found with the same name (a common scenario), show the most 
>>> recently-submitted one. To implement this we need to actually give 
>>> workflows names specified in the XML file, which I don't think we do 
>>> right now.
>>>
>>> -Chris
>>>
>>> Quoting Mike Hess <mh...@uc...>:
>>>
>>>> I was sick so I wasn't able to make it to the meeting.  I think we
>>>> have to have a working version done soon for alpha testing, so I was
>>>> wondering what are the key functions that still need to be coded so I
>>>> can help out with them.  I was looking through the code and I wasn't
>>>> sure what else needed to be done.
>>>>
>>>>
>>>> -Mike
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
>>>> is a gathering of tech-side developers & brand creativity 
>>>> professionals. Meet
>>>> the minds behind Google Creative Lab, Visual Complexity,  Processing, &
>>>> iPhoneDevCamp asthey present alongside digital heavyweights like 
>>>> Barbarian
>>>> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
>>>> _______________________________________________
>>>> Pipeline130-developers mailing list
>>>> Pip...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers
>>>>
>>>
>>>
>>
>> 

I was sick so I wasn't able to make it to the meeting.  I think we  
have to have a working version done soon for alpha testing, so I was  
wondering what are the key functions that still need to be coded so I  
can help out with them.  I was looking through the code and I wasn't  
sure what else needed to be done.

-Mike

  I should take the responsibility to design several test cases of invalid
workflows. Let's talk about it in today's disscussion.

pipeline130-developers Mailing List for pipeline cs130 (Page 4)

pipeline130-developers — Developer Mailing List