You can subscribe to this list here.
2009 |
Jan
|
Feb
|
Mar
|
Apr
(37) |
May
(113) |
Jun
(62) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
---|
From: Nils H. <nil...@uc...> - 2009-05-14 00:57:00
|
Just to let everyone know, I am still testing out the DRMAA interface via the drmaa example in the sandbox folder (note: there are hard-coded links). Anyhow, if someone figures out how to get a job to successfully run (mine are always aborted after submission), let me know. Nis |
From: Chris S. <cs...@uc...> - 2009-05-14 00:42:59
|
Although that does produce an error, the actual project should still be built. Check the output directory. I thought that I had removed the junit stuff from the engine, but maybe it didn't go through. I will experiment with it more later. -Chris -------------------------------------------------- From: "Nils Homer" <nil...@uc...> Sent: Wednesday, May 13, 2009 5:32 PM To: <pip...@li...> Subject: [Pipeline130-developers] Build with ANT > Looks like some stuff is not included in the svn repository. Can anyone > confirm that they were able to build the wfengine via "ant" on the > cluster? > > Nilss > > BUILD FAILED > /home/cs130/nhomer/pipeline130/trunk/wfengine/nbproject/build-impl.xml:571: > The following error occurred while executing this line: > /home/cs130/nhomer/pipeline130/trunk/wfengine/nbproject/build-impl.xml:220: > Could not create task or type of type: junit. > > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK > i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > |
From: Nils H. <nil...@uc...> - 2009-05-14 00:33:00
|
Looks like some stuff is not included in the svn repository. Can anyone confirm that they were able to build the wfengine via "ant" on the cluster? Nilss BUILD FAILED /home/cs130/nhomer/pipeline130/trunk/wfengine/nbproject/build-impl.xml:571: The following error occurred while executing this line: /home/cs130/nhomer/pipeline130/trunk/wfengine/nbproject/build-impl.xml:220: Could not create task or type of type: junit. |
From: Nils H. <nil...@uc...> - 2009-05-13 04:39:00
|
1. The manager does parse command line options and there are stubs in place already! 2. I have a test script that works on the cluster using DRMAA. I wont be near a computer from Friday until after the exam so I apologize in advance. 3. Example XML files can be found on the cluster in $HOME/examples Nils On 5/12/09 7:47 PM, "cs...@uc..." <cs...@uc...> wrote: > Hey guys, > > I got an email from someone asking to be pointed in the right > direction. Like I said, I'm going to try to get some concrete tasks > written up in the bug tracker -- mainly code stuff but possibly > documentation tasks as well. However, we should all also be > contributing to the planning aspect. If you happen to see something > that needs to be done, try to take note of it. > > FIRST: please let us know immediately if you don't have SVN working. > You really need to be able to check out the code. > > Once you have it working, here are some things that need to be done: > > Document stuff: > Update the Design document to describe how we handle our Workflows. We > have Workflow objects which contain Jobs, Jobs keep track of their own > status, etc... the WFManager parses the XML documents into this > structure, and we also serialize and transmit this structure as the > main form of communication between Manager and Engine. When we query > for Workflow status, we get a different structure that contains > information specifically pertaining to job statuses. Maybe draw a > diagram of these structures? This mailing list and the code itself are > good places to look for info on these things. > > Testing stuff: > Design test cases, and try implementing them if possible! A good place > to start would be testing Mike's workflow status printing routines, > since they are pretty self-contained. Make a test scaffolding that > generates a Workflow, populate it with Jobs that have interesting > states, and print out the status. Make sure everything comes out > looking right. > > Coding stuff: > Just try to dive in somewhere. Make small changes at first, then try > adding functionality. The Engine->Cluster interface definitely needs > work, but I think Nils is currently investigating -- that said, don't > be shy to try getting it going yourself. As far as I know, the Manager > currently parses command-line arguments but doesn't really examine > them. If you could have it call different stub functions based on the > command entered, that would be very helpful. > > The Engine needs its entire "polling" loop done. You can just make an > empty stub PollEngine() method and write the rest of the loop assuming > that it updated the running workflows. Make sure you clearly mark > which parts are unfinished! > > -Chris > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: <cs...@uc...> - 2009-05-13 02:50:35
|
Hey guys, I got an email from someone asking to be pointed in the right direction. Like I said, I'm going to try to get some concrete tasks written up in the bug tracker -- mainly code stuff but possibly documentation tasks as well. However, we should all also be contributing to the planning aspect. If you happen to see something that needs to be done, try to take note of it. FIRST: please let us know immediately if you don't have SVN working. You really need to be able to check out the code. Once you have it working, here are some things that need to be done: Document stuff: Update the Design document to describe how we handle our Workflows. We have Workflow objects which contain Jobs, Jobs keep track of their own status, etc... the WFManager parses the XML documents into this structure, and we also serialize and transmit this structure as the main form of communication between Manager and Engine. When we query for Workflow status, we get a different structure that contains information specifically pertaining to job statuses. Maybe draw a diagram of these structures? This mailing list and the code itself are good places to look for info on these things. Testing stuff: Design test cases, and try implementing them if possible! A good place to start would be testing Mike's workflow status printing routines, since they are pretty self-contained. Make a test scaffolding that generates a Workflow, populate it with Jobs that have interesting states, and print out the status. Make sure everything comes out looking right. Coding stuff: Just try to dive in somewhere. Make small changes at first, then try adding functionality. The Engine->Cluster interface definitely needs work, but I think Nils is currently investigating -- that said, don't be shy to try getting it going yourself. As far as I know, the Manager currently parses command-line arguments but doesn't really examine them. If you could have it call different stub functions based on the command entered, that would be very helpful. The Engine needs its entire "polling" loop done. You can just make an empty stub PollEngine() method and write the rest of the loop assuming that it updated the running workflows. Make sure you clearly mark which parts are unfinished! -Chris |
From: Nils H. <nil...@uc...> - 2009-05-12 07:10:43
|
Excellent, now we can write the interface to the DRMAA library and still use Netbeans! Thanks Chris. On 5/11/09 11:57 PM, "Chris Swan" <cs...@uc...> wrote: > I just checked in a copy of drmaa.jar to the wfengine root directory and added > it to the NetBeans project. I think we can now link to it and get code > completion for it, but of course running it locally will probably cause a > runtime error. > > To compile on the cluster, you can just do an svn checkout from your folder > (mkdir <yourname> after logging in if you haven't already, then cd into it, > then svn co https://pipeline130.svn.sourceforge.net/svnroot/pipeline130 > pipeline130 ). cd into the directory of the project you're interested in (the > one containing build.xml) and build it with Ant by just typing "ant". The > compiled code should be in the "./build/classes" directory. >From this folder, > type "java wfengine.Main" (change as appropriate). > > Let me know if this doesn't work. > > -Chris > > From: Chris Swan <mailto:cs...@uc...> > Sent: Monday, May 11, 2009 9:41 PM > To: pip...@li... > Subject: Re: [Pipeline130-developers] Group participation > > I haven't yet tried the test cluster aside from logging in, but hopefully will > soon. NetBeans generates build scripts for Ant, an XML-based make-style > utility that is installed on the cluster, so I think it will be OK. We just > might have to have a fake library for it to link to on our local machines so > we don't get compilation errors. > > -Chris > > From: Nils Homer <mailto:nil...@uc...> > Sent: Monday, May 11, 2009 8:29 PM > To: pip...@li... > Subject: Re: [Pipeline130-developers] Group participation > > Has anyone tried logging onto the test cluster? There is not GUI so maybe we > should move away from using netbeans for development. I say this since I was > working on the DRMAA/Engine interface and you need to have the drmaa library > for linking, which isn¹t available on my local computer (I am not going to > install SGE on my computer). Anyways, we could move back to a simpler source > tree hierarchy and just have Makefiles. Then we could use vim or whatever to > develop on the cluster, which has DRMAA installed. > > Alternatively, if anyone finds a way to link DRMAA for local development let > me know. I am starting to like this IDE. > > Nils > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > > > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Chris S. <cs...@uc...> - 2009-05-12 06:58:04
|
Re: [Pipeline130-developers] Group participationI just checked in a copy of drmaa.jar to the wfengine root directory and added it to the NetBeans project. I think we can now link to it and get code completion for it, but of course running it locally will probably cause a runtime error. To compile on the cluster, you can just do an svn checkout from your folder (mkdir <yourname> after logging in if you haven't already, then cd into it, then svn co https://pipeline130.svn.sourceforge.net/svnroot/pipeline130 pipeline130 ). cd into the directory of the project you're interested in (the one containing build.xml) and build it with Ant by just typing "ant". The compiled code should be in the "./build/classes" directory. From this folder, type "java wfengine.Main" (change as appropriate). Let me know if this doesn't work. -Chris From: Chris Swan Sent: Monday, May 11, 2009 9:41 PM To: pip...@li... Subject: Re: [Pipeline130-developers] Group participation I haven't yet tried the test cluster aside from logging in, but hopefully will soon. NetBeans generates build scripts for Ant, an XML-based make-style utility that is installed on the cluster, so I think it will be OK. We just might have to have a fake library for it to link to on our local machines so we don't get compilation errors. -Chris From: Nils Homer Sent: Monday, May 11, 2009 8:29 PM To: pip...@li... Subject: Re: [Pipeline130-developers] Group participation Has anyone tried logging onto the test cluster? There is not GUI so maybe we should move away from using netbeans for development. I say this since I was working on the DRMAA/Engine interface and you need to have the drmaa library for linking, which isn't available on my local computer (I am not going to install SGE on my computer). Anyways, we could move back to a simpler source tree hierarchy and just have Makefiles. Then we could use vim or whatever to develop on the cluster, which has DRMAA installed. Alternatively, if anyone finds a way to link DRMAA for local development let me know. I am starting to like this IDE. Nils -------------------------------------------------------------------------------- ------------------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com -------------------------------------------------------------------------------- _______________________________________________ Pipeline130-developers mailing list Pip...@li... https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Chris S. <cs...@uc...> - 2009-05-12 04:41:48
|
Re: [Pipeline130-developers] Group participationI haven't yet tried the test cluster aside from logging in, but hopefully will soon. NetBeans generates build scripts for Ant, an XML-based make-style utility that is installed on the cluster, so I think it will be OK. We just might have to have a fake library for it to link to on our local machines so we don't get compilation errors. -Chris From: Nils Homer Sent: Monday, May 11, 2009 8:29 PM To: pip...@li... Subject: Re: [Pipeline130-developers] Group participation Has anyone tried logging onto the test cluster? There is not GUI so maybe we should move away from using netbeans for development. I say this since I was working on the DRMAA/Engine interface and you need to have the drmaa library for linking, which isn't available on my local computer (I am not going to install SGE on my computer). Anyways, we could move back to a simpler source tree hierarchy and just have Makefiles. Then we could use vim or whatever to develop on the cluster, which has DRMAA installed. Alternatively, if anyone finds a way to link DRMAA for local development let me know. I am starting to like this IDE. Nils |
From: Mike H. <mh...@uc...> - 2009-05-12 03:50:12
|
Yep I've been able to log into the cluster but I haven't actually tested anything there because I don't really remember how to the installation and testing. Personally I would rather move to vim at the last moment possible but that's just me and I realize it likely will be inevitable. Mike On May 11, 2009, at 8:29 PM, Nils Homer <nil...@uc...> wrote: > I agree. I will not be around Friday or over the weekend as I will > be out of town on important business. Just a heads up. > > Has anyone tried logging onto the test cluster? There is not GUI so > maybe we should move away from using netbeans for development. I say > this since I was working on the DRMAA/Engine interface and you need > to have the drmaa library for linking, which isn’t available on my l > ocal computer (I am not going to install SGE on my computer). Anywa > ys, we could move back to a simpler source tree hierarchy and just h > ave Makefiles. Then we could use vim or whatever to develop on the > cluster, which has DRMAA installed. > > Alternatively, if anyone finds a way to link DRMAA for local > development let me know. I am starting to like this IDE. > > Nils > > > On 5/11/09 7:42 PM, "Chris Swan" <cs...@uc...> wrote: > > Hey guys, > > As Nils mentioned in discussion on Friday, our group has a major > participation problem. We are supposed to be in active development > right now, but only four of us have made any commits to source > control. That alone is not necessarily a problem, except that those > who haven't written code also haven't really done anything else > either. We've already turned in three assignments, and the vast > majority of all three was written by two or three of us with almost > no input by anyone else. You guys all knew the projects were due, > how to do them, and what was needed. > > The bottom line is that everybody has to put work into this project. > When I write an email to the group asking for your thoughts, that is > an easy invitation to participate. Even if you don't think you know > enough about the problem, your clarifying questions can bring new > insight to the group. And saying "I agree with everything you say" > is not enough, because I know fully well that my designs aren't > perfect. We need to have more discussion like the debate Nils and I > had over Workflow Templates... except there should be more than two > viewpoints. > > In the next couple days I'm going to try getting the bug tracking > system working so we can better keep track of what needs to be done. > But you should be taking initiative even before then. Do something > for the project. Update a document on the wiki, start working on > automated tests, work on a feature, start building a feature, flesh > out an existing feature... do anything. > > Don't be scared to ask questions, even simple ones. I am more than > happy to answer a question if it means you are thinking about the > project and preparing to contribute. I didn't intend this email to > come across as mean, but it all had to be said. > > -Chris > > --- > --- > --- > --------------------------------------------------------------------- > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > > --- > --- > --- > --------------------------------------------------------------------- > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Nils H. <nil...@uc...> - 2009-05-12 03:30:26
|
I agree. I will not be around Friday or over the weekend as I will be out of town on important business. Just a heads up. Has anyone tried logging onto the test cluster? There is not GUI so maybe we should move away from using netbeans for development. I say this since I was working on the DRMAA/Engine interface and you need to have the drmaa library for linking, which isn¹t available on my local computer (I am not going to install SGE on my computer). Anyways, we could move back to a simpler source tree hierarchy and just have Makefiles. Then we could use vim or whatever to develop on the cluster, which has DRMAA installed. Alternatively, if anyone finds a way to link DRMAA for local development let me know. I am starting to like this IDE. Nils On 5/11/09 7:42 PM, "Chris Swan" <cs...@uc...> wrote: > Hey guys, > > As Nils mentioned in discussion on Friday, our group has a major participation > problem. We are supposed to be in active development right now, but only four > of us have made any commits to source control. That alone is not necessarily a > problem, except that those who haven't written code also haven't really done > anything else either. We've already turned in three assignments, and the vast > majority of all three was written by two or three of us with almost no input > by anyone else. You guys all knew the projects were due, how to do them, and > what was needed. > > The bottom line is that everybody has to put work into this project. When I > write an email to the group asking for your thoughts, that is an easy > invitation to participate. Even if you don't think you know enough about the > problem, your clarifying questions can bring new insight to the group. And > saying "I agree with everything you say" is not enough, because I know fully > well that my designs aren't perfect. We need to have more discussion like the > debate Nils and I had over Workflow Templates... except there should be more > than two viewpoints. > > In the next couple days I'm going to try getting the bug tracking system > working so we can better keep track of what needs to be done. But you should > be taking initiative even before then. Do something for the project. Update a > document on the wiki, start working on automated tests, work on a feature, > start building a feature, flesh out an existing feature... do anything. > > Don't be scared to ask questions, even simple ones. I am more than happy to > answer a question if it means you are thinking about the project and preparing > to contribute. I didn't intend this email to come across as mean, but it all > had to be said. > > -Chris > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Chris S. <cs...@uc...> - 2009-05-12 02:42:57
|
Hey guys, As Nils mentioned in discussion on Friday, our group has a major participation problem. We are supposed to be in active development right now, but only four of us have made any commits to source control. That alone is not necessarily a problem, except that those who haven't written code also haven't really done anything else either. We've already turned in three assignments, and the vast majority of all three was written by two or three of us with almost no input by anyone else. You guys all knew the projects were due, how to do them, and what was needed. The bottom line is that everybody has to put work into this project. When I write an email to the group asking for your thoughts, that is an easy invitation to participate. Even if you don't think you know enough about the problem, your clarifying questions can bring new insight to the group. And saying "I agree with everything you say" is not enough, because I know fully well that my designs aren't perfect. We need to have more discussion like the debate Nils and I had over Workflow Templates... except there should be more than two viewpoints. In the next couple days I'm going to try getting the bug tracking system working so we can better keep track of what needs to be done. But you should be taking initiative even before then. Do something for the project. Update a document on the wiki, start working on automated tests, work on a feature, start building a feature, flesh out an existing feature... do anything. Don't be scared to ask questions, even simple ones. I am more than happy to answer a question if it means you are thinking about the project and preparing to contribute. I didn't intend this email to come across as mean, but it all had to be said. -Chris |
From: Nils H. <nil...@uc...> - 2009-05-11 23:10:57
|
I have been running DRMAA testing on the cluster and the interface seems fairly clean. One part we forgot to include is the actual arguments when submitting a job (not the arguments to the command). An example of a job argument is to use the BASH shell as the default shell for the job, or to use a computers with an amd64 architecture, or that this job uses 8 threads and therefore should request 8 slots. I will update the XSD to include these job arguments. Also, I have been working hard getting the test cluster running. I have it pretty well setup, with all home directories now using autofs and LDAP for authentication. Let me know if you experience any problems on the test cluster. Nils |
From: Nils H. <nil...@uc...> - 2009-05-11 19:28:59
|
I have started writing test cases and I am storing them on the cluster: /export/home/cs130/examples I will add them to the repository once I am finished writing all of them according to our design and req docs. Feel free to add a new test case, just remember to add a description to the README so we all can understand what you are trying to do. Nils |
From: Chris S. <cs...@uc...> - 2009-05-11 09:06:32
|
In my mind it should be a completely independent data structure, generated by the Jobs themselves, containing no references to actual Job objects. So you ask the workflow for a WorkflowStatusReport, which it returns to you. Inside that are a bunch of JobStatusReports generated by each job individually (aggregated by the workflow). I see it looking something like this: // treating this class basically like a struct for now class JobStatusReport { public string name; // user-friendly name of job public string id; // internal-use name of job public string command; public JobStatus status; public JobDelayReason delayReason; // some representation of why the job isn't finished, if applicable. For now could just be a string suitable for printing, but eventually could include the job ID(s) of prereq jobs public DateTime timeElapsed; // useful eventually } Again, we don't want anybody touching the privates of Jobs or Workflows if at all possible. But this is just my opinion of how it should work... feel free to argue against it. -Chris -------------------------------------------------- From: "Mike Hess" <mh...@uc...> Sent: Monday, May 11, 2009 1:30 AM To: <pip...@li...> Subject: Re: [Pipeline130-developers] Any other status functions? > Yea you're probably right. I'll fix it tomorrow. Shouldn't be that > hard to implements, I just did it a quick dirty way. Could a > JobStatusReport just have a Job that it can access the data of, then a > couple of print functions? Because I could do that in like 15 minutes. > > -Mike > > On May 11, 2009, at 12:44 AM, Chris Swan wrote: > >> Nice work so far! I haven't given the code a real review yet, but >> one thing >> I think we should decide is whether the primary logic for these >> functions >> belongs in the Manager or in the Workflow/Job itself. I see a few >> ways to do >> this: >> >> 1) The current way (or at least my understanding of it): From within >> the >> WorkflowManager, iterate over the jobs >> >> 2) A way that promotes the "smart nodes, dumb consumers" idea: The >> Manager >> calls workflow.PrintStatus(), the Workflow calls Job.printStatus(), >> etc., >> and the workflow/jobs print themselves. >> >> 3) Probably the best way overall: The Manager calls >> workflow.GetStatus(), >> which returns a data structure containing all the relevant >> information -- >> essentially, a list of "JobStatusReport" objects that contain all the >> information about the job, but without the parent/children >> information. The >> Manager can then iterate over this data structure in order to print >> the >> info. The benefit of this way is that we can use this very same code >> to make >> our GUI app, only changing what we do as we iterate. All of the >> intelligence >> is still maintained within the Workflow/Job. >> >> Maybe for now this isn't a high priority, but I do think we should >> try to >> avoid ever directly looking at the job's parents and children. Those >> should >> be private and not have a public accessor. In the meantime, maybe we >> can >> make the Manager's reporting module into a friend class. >> >> What do you guys think? >> >> -Chris >> >> -------------------------------------------------- >> From: "Mike Hess" <mh...@uc...> >> Sent: Sunday, May 10, 2009 2:27 PM >> To: "CS 130" <pip...@li...> >> Subject: [Pipeline130-developers] Any other status functions? >> >>> Hey I was wondering if you guys had any other status functions that >>> you guys can think of. >>> One's I've implemented are: >>> Take in a workflow and print out status of all its Jobs >>> Take in a workflow and print out jobs that are holding up completion >>> Take in a job and print out why the job hasn't started >>> >>> >>> Also I had to mess around with some other classes and I wanted to >>> make >>> sure its OK how I did it. I took the JobStatus enum out of the Job >>> class and made it its own file because it wasn't letting me access >>> it. I also added functions that return a Job's parents and return a >>> workflow's jobs so I could access them. >>> >>> >>> -Mike >>> >>> ------------------------------------------------------------------------------ >>> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >>> Your >>> production scanning environment may not be a perfect world - but >>> thanks to >>> Kodak, there's a perfect scanner to get the job done! With the NEW >>> KODAK >>> i700 >>> Series Scanner you'll get full speed at 300 dpi even with all image >>> processing features enabled. http://p.sf.net/sfu/kodak-com >>> _______________________________________________ >>> Pipeline130-developers mailing list >>> Pip...@li... >>> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers >>> >> >> ------------------------------------------------------------------------------ >> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >> Your >> production scanning environment may not be a perfect world - but >> thanks to >> Kodak, there's a perfect scanner to get the job done! With the NEW >> KODAK i700 >> Series Scanner you'll get full speed at 300 dpi even with all image >> processing features enabled. http://p.sf.net/sfu/kodak-com >> _______________________________________________ >> Pipeline130-developers mailing list >> Pip...@li... >> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK > i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > |
From: Mike H. <mh...@uc...> - 2009-05-11 08:54:14
|
One problem I do see with it however is I do need the list of parents to figure out the failure point in the "determine why a job hasn't started" function, soooo just having a job won't work, but having it extend a job might. Mike On May 11, 2009, at 1:30 AM, Mike Hess <mh...@uc...> wrote: > Yea you're probably right. I'll fix it tomorrow. Shouldn't be that > hard to implements, I just did it a quick dirty way. Could a > JobStatusReport just have a Job that it can access the data of, then a > couple of print functions? Because I could do that in like 15 > minutes. > > -Mike > > On May 11, 2009, at 12:44 AM, Chris Swan wrote: > >> Nice work so far! I haven't given the code a real review yet, but >> one thing >> I think we should decide is whether the primary logic for these >> functions >> belongs in the Manager or in the Workflow/Job itself. I see a few >> ways to do >> this: >> >> 1) The current way (or at least my understanding of it): From within >> the >> WorkflowManager, iterate over the jobs >> >> 2) A way that promotes the "smart nodes, dumb consumers" idea: The >> Manager >> calls workflow.PrintStatus(), the Workflow calls Job.printStatus(), >> etc., >> and the workflow/jobs print themselves. >> >> 3) Probably the best way overall: The Manager calls >> workflow.GetStatus(), >> which returns a data structure containing all the relevant >> information -- >> essentially, a list of "JobStatusReport" objects that contain all the >> information about the job, but without the parent/children >> information. The >> Manager can then iterate over this data structure in order to print >> the >> info. The benefit of this way is that we can use this very same code >> to make >> our GUI app, only changing what we do as we iterate. All of the >> intelligence >> is still maintained within the Workflow/Job. >> >> Maybe for now this isn't a high priority, but I do think we should >> try to >> avoid ever directly looking at the job's parents and children. Those >> should >> be private and not have a public accessor. In the meantime, maybe we >> can >> make the Manager's reporting module into a friend class. >> >> What do you guys think? >> >> -Chris >> >> -------------------------------------------------- >> From: "Mike Hess" <mh...@uc...> >> Sent: Sunday, May 10, 2009 2:27 PM >> To: "CS 130" <pip...@li...> >> Subject: [Pipeline130-developers] Any other status functions? >> >>> Hey I was wondering if you guys had any other status functions that >>> you guys can think of. >>> One's I've implemented are: >>> Take in a workflow and print out status of all its Jobs >>> Take in a workflow and print out jobs that are holding up completion >>> Take in a job and print out why the job hasn't started >>> >>> >>> Also I had to mess around with some other classes and I wanted to >>> make >>> sure its OK how I did it. I took the JobStatus enum out of the Job >>> class and made it its own file because it wasn't letting me access >>> it. I also added functions that return a Job's parents and return a >>> workflow's jobs so I could access them. >>> >>> >>> -Mike >>> >>> --- >>> --- >>> --- >>> --- >>> ------------------------------------------------------------------ >>> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >>> Your >>> production scanning environment may not be a perfect world - but >>> thanks to >>> Kodak, there's a perfect scanner to get the job done! With the NEW >>> KODAK >>> i700 >>> Series Scanner you'll get full speed at 300 dpi even with all image >>> processing features enabled. http://p.sf.net/sfu/kodak-com >>> _______________________________________________ >>> Pipeline130-developers mailing list >>> Pip...@li... >>> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers >>> >> >> --- >> --- >> --- >> --------------------------------------------------------------------- >> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >> Your >> production scanning environment may not be a perfect world - but >> thanks to >> Kodak, there's a perfect scanner to get the job done! With the NEW >> KODAK i700 >> Series Scanner you'll get full speed at 300 dpi even with all image >> processing features enabled. http://p.sf.net/sfu/kodak-com >> _______________________________________________ >> Pipeline130-developers mailing list >> Pip...@li... >> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > > > --- > --- > --- > --------------------------------------------------------------------- > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Mike H. <mh...@uc...> - 2009-05-11 08:31:20
|
Yea you're probably right. I'll fix it tomorrow. Shouldn't be that hard to implements, I just did it a quick dirty way. Could a JobStatusReport just have a Job that it can access the data of, then a couple of print functions? Because I could do that in like 15 minutes. -Mike On May 11, 2009, at 12:44 AM, Chris Swan wrote: > Nice work so far! I haven't given the code a real review yet, but > one thing > I think we should decide is whether the primary logic for these > functions > belongs in the Manager or in the Workflow/Job itself. I see a few > ways to do > this: > > 1) The current way (or at least my understanding of it): From within > the > WorkflowManager, iterate over the jobs > > 2) A way that promotes the "smart nodes, dumb consumers" idea: The > Manager > calls workflow.PrintStatus(), the Workflow calls Job.printStatus(), > etc., > and the workflow/jobs print themselves. > > 3) Probably the best way overall: The Manager calls > workflow.GetStatus(), > which returns a data structure containing all the relevant > information -- > essentially, a list of "JobStatusReport" objects that contain all the > information about the job, but without the parent/children > information. The > Manager can then iterate over this data structure in order to print > the > info. The benefit of this way is that we can use this very same code > to make > our GUI app, only changing what we do as we iterate. All of the > intelligence > is still maintained within the Workflow/Job. > > Maybe for now this isn't a high priority, but I do think we should > try to > avoid ever directly looking at the job's parents and children. Those > should > be private and not have a public accessor. In the meantime, maybe we > can > make the Manager's reporting module into a friend class. > > What do you guys think? > > -Chris > > -------------------------------------------------- > From: "Mike Hess" <mh...@uc...> > Sent: Sunday, May 10, 2009 2:27 PM > To: "CS 130" <pip...@li...> > Subject: [Pipeline130-developers] Any other status functions? > >> Hey I was wondering if you guys had any other status functions that >> you guys can think of. >> One's I've implemented are: >> Take in a workflow and print out status of all its Jobs >> Take in a workflow and print out jobs that are holding up completion >> Take in a job and print out why the job hasn't started >> >> >> Also I had to mess around with some other classes and I wanted to >> make >> sure its OK how I did it. I took the JobStatus enum out of the Job >> class and made it its own file because it wasn't letting me access >> it. I also added functions that return a Job's parents and return a >> workflow's jobs so I could access them. >> >> >> -Mike >> >> ------------------------------------------------------------------------------ >> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >> Your >> production scanning environment may not be a perfect world - but >> thanks to >> Kodak, there's a perfect scanner to get the job done! With the NEW >> KODAK >> i700 >> Series Scanner you'll get full speed at 300 dpi even with all image >> processing features enabled. http://p.sf.net/sfu/kodak-com >> _______________________________________________ >> Pipeline130-developers mailing list >> Pip...@li... >> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers >> > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Chris S. <cs...@uc...> - 2009-05-11 07:44:26
|
Nice work so far! I haven't given the code a real review yet, but one thing I think we should decide is whether the primary logic for these functions belongs in the Manager or in the Workflow/Job itself. I see a few ways to do this: 1) The current way (or at least my understanding of it): From within the WorkflowManager, iterate over the jobs 2) A way that promotes the "smart nodes, dumb consumers" idea: The Manager calls workflow.PrintStatus(), the Workflow calls Job.printStatus(), etc., and the workflow/jobs print themselves. 3) Probably the best way overall: The Manager calls workflow.GetStatus(), which returns a data structure containing all the relevant information -- essentially, a list of "JobStatusReport" objects that contain all the information about the job, but without the parent/children information. The Manager can then iterate over this data structure in order to print the info. The benefit of this way is that we can use this very same code to make our GUI app, only changing what we do as we iterate. All of the intelligence is still maintained within the Workflow/Job. Maybe for now this isn't a high priority, but I do think we should try to avoid ever directly looking at the job's parents and children. Those should be private and not have a public accessor. In the meantime, maybe we can make the Manager's reporting module into a friend class. What do you guys think? -Chris -------------------------------------------------- From: "Mike Hess" <mh...@uc...> Sent: Sunday, May 10, 2009 2:27 PM To: "CS 130" <pip...@li...> Subject: [Pipeline130-developers] Any other status functions? > Hey I was wondering if you guys had any other status functions that > you guys can think of. > One's I've implemented are: > Take in a workflow and print out status of all its Jobs > Take in a workflow and print out jobs that are holding up completion > Take in a job and print out why the job hasn't started > > > Also I had to mess around with some other classes and I wanted to make > sure its OK how I did it. I took the JobStatus enum out of the Job > class and made it its own file because it wasn't letting me access > it. I also added functions that return a Job's parents and return a > workflow's jobs so I could access them. > > > -Mike > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK > i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > |
From: Mike H. <mh...@uc...> - 2009-05-10 22:43:31
|
nvmd... figured it out. I'll commit it again. -Mike On May 10, 2009, at 3:35 PM, Mike Hess wrote: > Alright, I'll mess around with it some more. I just don't think i > could figure out the syntax and google wasn't helping much. If you > have any example syntax that would be helpful but otherwise, I'll just > try to get it to work. > > -Mike > > On May 10, 2009, at 3:27 PM, Nils Homer wrote: > >> Looks great! I would keep the JobStatus enum in the Job class. >> Lets debug >> why you cannot access the enum... >> >> Nils >> >> >> On 5/10/09 2:27 PM, "Mike Hess" <mh...@uc...> wrote: >> >>> Hey I was wondering if you guys had any other status functions that >>> you guys can think of. >>> One's I've implemented are: >>> Take in a workflow and print out status of all its Jobs >>> Take in a workflow and print out jobs that are holding up completion >>> Take in a job and print out why the job hasn't started >>> >>> >>> Also I had to mess around with some other classes and I wanted to >>> make >>> sure its OK how I did it. I took the JobStatus enum out of the Job >>> class and made it its own file because it wasn't letting me access >>> it. I also added functions that return a Job's parents and return a >>> workflow's jobs so I could access them. >>> >>> >>> -Mike >>> >>> ------------------------------------------------------------------------------ >>> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >>> Your >>> production scanning environment may not be a perfect world - but >>> thanks to >>> Kodak, there's a perfect scanner to get the job done! With the NEW >>> KODAK i700 >>> Series Scanner you'll get full speed at 300 dpi even with all image >>> processing features enabled. http://p.sf.net/sfu/kodak-com >>> _______________________________________________ >>> Pipeline130-developers mailing list >>> Pip...@li... >>> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers >> >> >> >> ------------------------------------------------------------------------------ >> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >> Your >> production scanning environment may not be a perfect world - but >> thanks to >> Kodak, there's a perfect scanner to get the job done! With the NEW >> KODAK i700 >> Series Scanner you'll get full speed at 300 dpi even with all image >> processing features enabled. http://p.sf.net/sfu/kodak-com >> _______________________________________________ >> Pipeline130-developers mailing list >> Pip...@li... >> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Mike H. <mh...@uc...> - 2009-05-10 22:39:09
|
I think I did manage to get the package checked in. Try it out. -Mike On May 9, 2009, at 8:01 PM, Chris Swan wrote: > Nils is working on getting the gnu.getopt package checked in to SVN. > We're > not totally sure how Java/Netbeans package management works so we're > still > figuring it out. > > In the meantime, you can just download the package locally. Better > yet, > check it in yourself so we all can benefit! > > -Chris > > -------------------------------------------------- > From: "Mike Hess" <mh...@uc...> > Sent: Saturday, May 09, 2009 6:12 PM > To: "CS 130" <pip...@li...> > Subject: [Pipeline130-developers] so i'm a netbeans noooob > >> How do I get the project to build? I think I'm missing the >> gnu.getopt >> library. how do i get this? heres my error: >> /Users/mhess/NetBeansProjects/trunk/wfmanager/src/wfmanager/ >> MainSubmit.java:8: package gnu.getopt does not exist >> import gnu.getopt.*; >> >> -Mike >> >> ------------------------------------------------------------------------------ >> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >> Your >> production scanning environment may not be a perfect world - but >> thanks to >> Kodak, there's a perfect scanner to get the job done! With the NEW >> KODAK >> i700 >> Series Scanner you'll get full speed at 300 dpi even with all image >> processing features enabled. http://p.sf.net/sfu/kodak-com >> _______________________________________________ >> Pipeline130-developers mailing list >> Pip...@li... >> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers >> > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Mike H. <mh...@uc...> - 2009-05-10 22:36:33
|
Alright, I'll mess around with it some more. I just don't think i could figure out the syntax and google wasn't helping much. If you have any example syntax that would be helpful but otherwise, I'll just try to get it to work. -Mike On May 10, 2009, at 3:27 PM, Nils Homer wrote: > Looks great! I would keep the JobStatus enum in the Job class. > Lets debug > why you cannot access the enum... > > Nils > > > On 5/10/09 2:27 PM, "Mike Hess" <mh...@uc...> wrote: > >> Hey I was wondering if you guys had any other status functions that >> you guys can think of. >> One's I've implemented are: >> Take in a workflow and print out status of all its Jobs >> Take in a workflow and print out jobs that are holding up completion >> Take in a job and print out why the job hasn't started >> >> >> Also I had to mess around with some other classes and I wanted to >> make >> sure its OK how I did it. I took the JobStatus enum out of the Job >> class and made it its own file because it wasn't letting me access >> it. I also added functions that return a Job's parents and return a >> workflow's jobs so I could access them. >> >> >> -Mike >> >> ------------------------------------------------------------------------------ >> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! >> Your >> production scanning environment may not be a perfect world - but >> thanks to >> Kodak, there's a perfect scanner to get the job done! With the NEW >> KODAK i700 >> Series Scanner you'll get full speed at 300 dpi even with all image >> processing features enabled. http://p.sf.net/sfu/kodak-com >> _______________________________________________ >> Pipeline130-developers mailing list >> Pip...@li... >> https://lists.sourceforge.net/lists/listinfo/pipeline130-developers > > > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! > Your > production scanning environment may not be a perfect world - but > thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW > KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Nils H. <nil...@uc...> - 2009-05-10 22:28:10
|
Looks great! I would keep the JobStatus enum in the Job class. Lets debug why you cannot access the enum... Nils On 5/10/09 2:27 PM, "Mike Hess" <mh...@uc...> wrote: > Hey I was wondering if you guys had any other status functions that > you guys can think of. > One's I've implemented are: > Take in a workflow and print out status of all its Jobs > Take in a workflow and print out jobs that are holding up completion > Take in a job and print out why the job hasn't started > > > Also I had to mess around with some other classes and I wanted to make > sure its OK how I did it. I took the JobStatus enum out of the Job > class and made it its own file because it wasn't letting me access > it. I also added functions that return a Job's parents and return a > workflow's jobs so I could access them. > > > -Mike > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Mike H. <mh...@uc...> - 2009-05-10 21:27:39
|
Hey I was wondering if you guys had any other status functions that you guys can think of. One's I've implemented are: Take in a workflow and print out status of all its Jobs Take in a workflow and print out jobs that are holding up completion Take in a job and print out why the job hasn't started Also I had to mess around with some other classes and I wanted to make sure its OK how I did it. I took the JobStatus enum out of the Job class and made it its own file because it wasn't letting me access it. I also added functions that return a Job's parents and return a workflow's jobs so I could access them. -Mike |
From: Nils H. <nil...@uc...> - 2009-05-10 19:08:55
|
That's it. On 5/10/09 12:04 PM, "Mike Hess" <mh...@uc...> wrote: > http://www.urbanophile.com/arenn/hacking/download.html > > Or where do you get it from if thats not it? > > > -Mike > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Pipeline130-developers mailing list > Pip...@li... > https://lists.sourceforge.net/lists/listinfo/pipeline130-developers |
From: Mike H. <mh...@uc...> - 2009-05-10 19:05:08
|
http://www.urbanophile.com/arenn/hacking/download.html Or where do you get it from if thats not it? -Mike |
From: Chris S. <cs...@uc...> - 2009-05-10 07:46:09
|
I figure, given Mike's earlier message, we should talk a little bit about comments. Firstly, I think he's totally right that we should avoid block comments inside method bodies unless there's a really good reason for it. I will convert any in-method block comments to regular block comments for the reason Mike stated (to allow block-commenting around the comments for debugging purposes). More importantly though, I think we need to discuss *what* we comment. For example, in Job.java we have a bunch of methods like this: /** * * @param id ID of the job **/ public void setID(String id) { this.id = id; } Is this comment necessary? I think absolutely not. It only means readers have to scroll more to find the important stuff. If the name of the method, variable, parameter or return value completely describes what it does (and in most cases it definitely should), then don't bother commenting. Here is a method that I wrote last night in WorkflowEngine.java: /** * Starts all the ready jobs of all workflows. Probably should only be called * when resuming from a paused state. */ public void startReadyJobs() { for (Workflow w : workflows) { w.startReadyJobs(); } } In this case, I think it's a good idea to mention that it starts ready jobs of all workflows, so the first sentence is justified. However, I might not have written the comment just to say that. The really important part is that I specified the circumstances this function should be called. The comment serves as a warning to anyone who might feel like calling it later: it might make you think, "Hey, should I really be calling this here?" Then of course there are the completely obtuse functions. Not all methods can be as simple as startReadyJobs, and not everything can be broken down into tiny, simple parts (but many things can, so please try to do so!). For example, Justin recently committed the method forwardWorkflows in WorkflowEngine.java. It seems to do something kind of complicated... but I have absolutely no idea what that is. "forwardWorkflows" isn't a very meaningful name and I can't figure out what it does just by reading it. This is the case where we definitely do need a "descriptive" comment -- or else a clearer name. Justin, if you could fix that up, it'd be really helpful -- none of us can use your function until we figure out what it does. I'm not going to touch any comments right now (though I did last night), but I probably will once we have a chance to discuss the issue. What do you guys think? -Chris |