Re: [Opaltoolkit-users] collecting statistics of Opal jobs

[Luca C. <luc...@gm...>]

Malcom,
you can look into Opal database and in the job_info table you have
both the base_url field where you can get the output directory (in the
form of http://blabla/app123451523) and then there is the field handle
that if I'm not wrong it should contain the SGE JobID if you are using
DRMAA.

Sincerely,
Luca


On Tue, Dec 1, 2009 at 8:01 PM, Malcolm Tobias <mt...@wu...> wrote:
>
> I'd like to be able collect some statistics for the jobs running on our Opal
> server (e.g. run time, memory usage, some info about the input parameters)
> and am wondering about the best way to go about this.  One important detail
> is that I'm using the DRMAA Job Manager.  It seems like there are multiple
> approaches, but I'm thinking the cleanest approach might be to parse the
> temporary directories (where I'd have direct access to the input files), but
> I'm not sure how to correlate the Opal job number (app1259160756494) with the
> job number that the queuing system associates with the job.  If I had this
> latter info, I could query the queuing system to find the run time and memory
> usage.  I noticed that this information does show up in the Tomcat logs, but
> it never shows up at the same time as the Opal job number:
>
> 2009-11-25 08:52:36,501 DEBUG
> edu.sdsc.nbcr.opal.manager.DRMAAJobManager.launchJob(DRMAAJobManager.java:210) -
> Working
> directory: /export/home/opal/jakarta-tomcat-5.0.30/webapps/ROOT/app1259160756494/
> ...
> 2009-11-25 08:52:36,545 INFO
> edu.sdsc.nbcr.opal.manager.DRMAAJobManager.launchJob(DRMAAJobManager.java:233) -
> DRMAA job has been submitted with id 4549
>
> I could assume that the line after the launchJob message contains the queuing
> system job id, but I can imagine this getting screwed up if multiple jobs are
> running at the same time which is entirely possible.
>
> Can anyone recommend a better solution?
>
> Would it be possible to include the queuing system job id into the temporary
> directory somehow (say in a file called jobid)?
>
> Thanks in advance,
> Malcolm
>
>
>
> --
> Malcolm Tobias
> 314.362.1594
>
>
> ------------------------------------------------------------------------------
> Join us December 9, 2009 for the Red Hat Virtual Experience,
> a free event focused on virtualization and cloud computing.
> Attend in-depth sessions from your desk. Your couch. Anywhere.
> http://p.sf.net/sfu/redhat-sfdev2dev
> _______________________________________________
> Opaltoolkit-users mailing list
> Opa...@li...
> https://lists.sourceforge.net/lists/listinfo/opaltoolkit-users
>