Re: [Opaltoolkit-users] collecting statistics of Opal jobs

[Malcolm T. <mt...@wu...>]

Luca,

I wasn't aware of this database, but it sounds like just what I need.
Is this stored on the server somewhere?  Is it a flat file, or can you provide 
me with some pointers on how I'd go about extracting fields?

Cheers,
Malcolm

On Tuesday 01 December 2009 01:25:58 pm Luca Clementi wrote:
> Malcom,
> you can look into Opal database and in the job_info table you have
> both the base_url field where you can get the output directory (in the
> form of http://blabla/app123451523) and then there is the field handle
> that if I'm not wrong it should contain the SGE JobID if you are using
> DRMAA.
>
> Sincerely,
> Luca
>
> On Tue, Dec 1, 2009 at 8:01 PM, Malcolm Tobias <mt...@wu...> wrote:
> > I'd like to be able collect some statistics for the jobs running on our
> > Opal server (e.g. run time, memory usage, some info about the input
> > parameters) and am wondering about the best way to go about this.  One
> > important detail is that I'm using the DRMAA Job Manager.  It seems like
> > there are multiple approaches, but I'm thinking the cleanest approach
> > might be to parse the temporary directories (where I'd have direct access
> > to the input files), but I'm not sure how to correlate the Opal job
> > number (app1259160756494) with the job number that the queuing system
> > associates with the job.  If I had this latter info, I could query the
> > queuing system to find the run time and memory usage.  I noticed that
> > this information does show up in the Tomcat logs, but it never shows up
> > at the same time as the Opal job number:
> >
> > 2009-11-25 08:52:36,501 DEBUG
> > edu.sdsc.nbcr.opal.manager.DRMAAJobManager.launchJob(DRMAAJobManager.java
> >:210) - Working
> > directory:
> > /export/home/opal/jakarta-tomcat-5.0.30/webapps/ROOT/app1259160756494/
> > ...
> > 2009-11-25 08:52:36,545 INFO
> > edu.sdsc.nbcr.opal.manager.DRMAAJobManager.launchJob(DRMAAJobManager.java
> >:233) - DRMAA job has been submitted with id 4549
> >
> > I could assume that the line after the launchJob message contains the
> > queuing system job id, but I can imagine this getting screwed up if
> > multiple jobs are running at the same time which is entirely possible.
> >
> > Can anyone recommend a better solution?
> >
> > Would it be possible to include the queuing system job id into the
> > temporary directory somehow (say in a file called jobid)?
> >
> > Thanks in advance,
> > Malcolm
> >
> >
> >
> > --
> > Malcolm Tobias
> > 314.362.1594
> >
> >
> > -------------------------------------------------------------------------
> >----- Join us December 9, 2009 for the Red Hat Virtual Experience,
> > a free event focused on virtualization and cloud computing.
> > Attend in-depth sessions from your desk. Your couch. Anywhere.
> > http://p.sf.net/sfu/redhat-sfdev2dev
> > _______________________________________________
> > Opaltoolkit-users mailing list
> > Opa...@li...
> > https://lists.sourceforge.net/lists/listinfo/opaltoolkit-users



-- 
Malcolm Tobias
314.362.1594