gug-cvs Mailing List for Grid Underground (Page 5)
Status: Planning
Brought to you by:
szferi
Menu
▾
▴
gug-cvs — CVS commits
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(67) |
Oct
(232) |
Nov
(76) |
Dec
(36) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
(100) |
Feb
(89) |
Mar
(44) |
Apr
(71) |
May
(34) |
Jun
(81) |
Jul
(43) |
Aug
(42) |
Sep
(1) |
Oct
(87) |
Nov
(11) |
Dec
(9) |
| 2007 |
Jan
(203) |
Feb
(149) |
Mar
(94) |
Apr
(70) |
May
(35) |
Jun
(27) |
Jul
(4) |
Aug
|
Sep
(18) |
Oct
|
Nov
|
Dec
|
|
From: Roczei G. <ro...@us...> - 2007-04-30 20:20:44
|
Update of /cvsroot/gug/gug/gug/cli In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv24641/gug/cli Modified Files: clgr.py Log Message: jsdl namespace bugfix Index: clgr.py =================================================================== RCS file: /cvsroot/gug/gug/gug/cli/clgr.py,v retrieving revision 1.48 retrieving revision 1.49 diff -C2 -d -r1.48 -r1.49 *** clgr.py 22 Apr 2007 20:53:37 -0000 1.48 --- clgr.py 30 Apr 2007 20:20:42 -0000 1.49 *************** *** 253,257 **** jsdl = proxy.GetActivityJSDLDocuments([grid_id])[0] xml = SimpleXML(memory=jsdl) ! nss = {'jsdl' : "http://schemas.ogf.org/jsdl/2005/11/jsdl"} storagedir = xml.get_content('//jsdl:Source', nss = nss)[0][7:] print 'storagedir =', storagedir --- 253,257 ---- jsdl = proxy.GetActivityJSDLDocuments([grid_id])[0] xml = SimpleXML(memory=jsdl) ! nss = {'jsdl' : "http://schemas.ggf.org/jsdl/2005/11/jsdl"} storagedir = xml.get_content('//jsdl:Source', nss = nss)[0][7:] print 'storagedir =', storagedir *************** *** 485,489 **** proxy.GetActivityJSDLDocuments(grid_ids)) for grid_id in jsdls.keys(): ! nss = {'jsdl' : "http://schemas.ogf.org/jsdl/2005/11/jsdl"} jobname = jsdls[grid_id].get_content(\ '//jsdl:JobName', nss = nss)[0] --- 485,489 ---- proxy.GetActivityJSDLDocuments(grid_ids)) for grid_id in jsdls.keys(): ! nss = {'jsdl' : "http://schemas.ggf.org/jsdl/2005/11/jsdl"} jobname = jsdls[grid_id].get_content(\ '//jsdl:JobName', nss = nss)[0] *************** *** 555,559 **** proxy.GetActivityJSDLDocuments(grid_ids)) for grid_id in jsdls.keys(): ! nss = {'jsdl' : "http://schemas.ogf.org/jsdl/2005/11/jsdl"} jobname = jsdls[grid_id].get_content(\ '//jsdl:JobName', nss = nss)[0] --- 555,559 ---- proxy.GetActivityJSDLDocuments(grid_ids)) for grid_id in jsdls.keys(): ! nss = {'jsdl' : "http://schemas.ggf.org/jsdl/2005/11/jsdl"} jobname = jsdls[grid_id].get_content(\ '//jsdl:JobName', nss = nss)[0] |
|
From: Roczei G. <ro...@us...> - 2007-04-30 20:19:06
|
Update of /cvsroot/gug/gug/doc/user In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv23822/doc/user Modified Files: user_manual_hu.tex Log Message: jsdl namespace bugfix Index: user_manual_hu.tex =================================================================== RCS file: /cvsroot/gug/gug/doc/user/user_manual_hu.tex,v retrieving revision 1.5 retrieving revision 1.6 diff -C2 -d -r1.5 -r1.6 *** user_manual_hu.tex 14 Mar 2007 13:59:14 -0000 1.5 --- user_manual_hu.tex 30 Apr 2007 20:19:06 -0000 1.6 *************** *** 123,127 **** \begin{cop} <?xml version="1.0" encoding="UTF-8"?> ! <jsdl:JobDefinition xmlns="http://schemas.ogf.org/jsdl/2005/11/jsdl" xmlns:jsdl="http://schemas.ogf.org/jsdl/2005/11/jsdl" xmlns:jsdl-posix="http://schemas.ggf.org/jsdl/2005/06/jsdl-posix" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jsdl-gug="http://gug.grid.niif.hu/jsdl/2005/11/jsdl-gug"> <jsdl:JobDescription> --- 123,127 ---- \begin{cop} <?xml version="1.0" encoding="UTF-8"?> ! <jsdl:JobDefinition xmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:jsdl-posix="http://schemas.ggf.org/jsdl/2005/06/jsdl-posix" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jsdl-gug="http://gug.grid.niif.hu/jsdl/2005/11/jsdl-gug"> <jsdl:JobDescription> |
|
From: Roczei G. <ro...@us...> - 2007-04-30 20:18:28
|
Update of /cvsroot/gug/gug/doc/user In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv23650/doc/user Modified Files: user_manual_en.tex Log Message: jsdl namespace bugfix Index: user_manual_en.tex =================================================================== RCS file: /cvsroot/gug/gug/doc/user/user_manual_en.tex,v retrieving revision 1.2 retrieving revision 1.3 diff -C2 -d -r1.2 -r1.3 *** user_manual_en.tex 14 Mar 2007 13:59:14 -0000 1.2 --- user_manual_en.tex 30 Apr 2007 20:18:23 -0000 1.3 *************** *** 123,127 **** \begin{cop} <?xml version="1.0" encoding="UTF-8"?> ! <jsdl:JobDefinition xmlns="http://schemas.ogf.org/jsdl/2005/11/jsdl" xmlns:jsdl="http://schemas.ogf.org/jsdl/2005/11/jsdl" xmlns:jsdl-posix="http://schemas.ggf.org/jsdl/2005/06/jsdl-posix" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jsdl-gug="http://gug.grid.niif.hu/jsdl/2005/11/jsdl-gug"> <jsdl:JobDescription> --- 123,127 ---- \begin{cop} <?xml version="1.0" encoding="UTF-8"?> ! <jsdl:JobDefinition xmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:jsdl-posix="http://schemas.ggf.org/jsdl/2005/06/jsdl-posix" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jsdl-gug="http://gug.grid.niif.hu/jsdl/2005/11/jsdl-gug"> <jsdl:JobDescription> |
|
From: Roczei G. <ro...@us...> - 2007-04-23 00:04:27
|
Update of /cvsroot/gug/gug/gug/service/exec In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv18679 Modified Files: exec.py Log Message: New exec GIS attribute: disk total/free space Index: exec.py =================================================================== RCS file: /cvsroot/gug/gug/gug/service/exec/exec.py,v retrieving revision 1.52 retrieving revision 1.53 diff -C2 -d -r1.52 -r1.53 *** exec.py 20 Apr 2007 20:34:51 -0000 1.52 --- exec.py 23 Apr 2007 00:04:26 -0000 1.53 *************** *** 218,221 **** --- 218,222 ---- self.gis = gis_proxy(initdata['ctl']) self.CKPTPath = initdata.get('CKPTPath','None') + self.disk_space_path = initdata.get('DiskSpacePath', '/') path = initdata['DataDir'] path = path.replace('$HOSTNAME$', sysinfo.hostname()) *************** *** 375,378 **** --- 376,386 ---- else: data['new_activity'] = 1 + + data['free_space'] = str(sysinfo.total_and_free_space(self.disk_space_path)[1]) + data['total_space'] = str(sysinfo.total_and_free_space(self.disk_space_path)[0]) + + for item in ['mem_total', 'mem_free', 'swap_free','swap_total']: + data[item] = str(float(data[item]) * 1024.0) + return """<?xml version='1.0' encoding="utf-8" ?> <ServiceDescription> *************** *** 403,406 **** --- 411,417 ---- <FreeCpuNum>%(free_cpu)s</FreeCpuNum> <ProcessPerCPU>%(processpercpu)s</ProcessPerCPU> + <LoadAVG>%(load_avg)s</LoadAVG> + <TotalDiskSpace>%(total_space)s</TotalDiskSpace> + <FreeDiskSpace>%(free_space)s</FreeDiskSpace> </ClusterElement> </ServiceDescription>""" % data |
|
From: Roczei G. <ro...@us...> - 2007-04-22 23:41:55
|
Update of /cvsroot/gug/gug/etc In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv7664 Modified Files: site.conf.default Log Message: new config parameter: <DiskSpacePath>/</DiskSpacePath> Index: site.conf.default =================================================================== RCS file: /cvsroot/gug/gug/etc/site.conf.default,v retrieving revision 1.47 retrieving revision 1.48 diff -C2 -d -r1.47 -r1.48 *** site.conf.default 3 Apr 2007 16:48:37 -0000 1.47 --- site.conf.default 22 Apr 2007 23:41:54 -0000 1.48 *************** *** 100,103 **** --- 100,104 ---- <ProcessPerCPU>1</ProcessPerCPU> <DataDir>exec_data</DataDir> + <DiskSpacePath>/</DiskSpacePath> </InitData> </Service> |
|
From: Roczei G. <ro...@us...> - 2007-04-22 23:41:36
|
Update of /cvsroot/gug/gug/etc In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv7143 Modified Files: site.conf.ce.default Log Message: new config parameter: <DiskSpacePath>/</DiskSpacePath> Index: site.conf.ce.default =================================================================== RCS file: /cvsroot/gug/gug/etc/site.conf.ce.default,v retrieving revision 1.7 retrieving revision 1.8 diff -C2 -d -r1.7 -r1.8 *** site.conf.ce.default 14 Mar 2007 09:32:06 -0000 1.7 --- site.conf.ce.default 22 Apr 2007 23:41:35 -0000 1.8 *************** *** 41,44 **** --- 41,45 ---- <CKPTPath>/usr/local/src/gug/patch/ckpt/ckpt</CKPTPath> <DataDir>/home/gug/exec_data/$HOSTNAME$</DataDir> + <DiskSpacePath>/</DiskSpacePath> </InitData> </Service> |
|
From: Roczei G. <ro...@us...> - 2007-04-22 21:27:01
|
Update of /cvsroot/gug/gug/gug/service/cm In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv11081 Modified Files: cm.conf.default Log Message: clean Index: cm.conf.default =================================================================== RCS file: /cvsroot/gug/gug/gug/service/cm/cm.conf.default,v retrieving revision 1.8 retrieving revision 1.9 diff -C2 -d -r1.8 -r1.9 *** cm.conf.default 19 Mar 2007 11:30:54 -0000 1.8 --- cm.conf.default 22 Apr 2007 21:27:00 -0000 1.9 *************** *** 7,12 **** <TimeOut>600</TimeOut> <DBConnect>cm_data</DBConnect> - <BatchDefault>gug/service/cm/batch_job.xml.default</BatchDefault> - <BinaryDefault>gug/service/cm/binary_job.xml.default</BinaryDefault> <Architecture>i686</Architecture> <OperatingSystem>LINUX</OperatingSystem> --- 7,10 ---- |
|
From: Roczei G. <ro...@us...> - 2007-04-22 21:25:42
|
Update of /cvsroot/gug/gug/gug/service/cm In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv10690 Modified Files: cm.conf.n0.default Log Message: clean Index: cm.conf.n0.default =================================================================== RCS file: /cvsroot/gug/gug/gug/service/cm/cm.conf.n0.default,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** cm.conf.n0.default 14 Mar 2007 04:59:49 -0000 1.3 --- cm.conf.n0.default 22 Apr 2007 21:25:42 -0000 1.4 *************** *** 7,12 **** <TimeOut>600</TimeOut> <DBConnect>/home/gug/cm_data</DBConnect> - <BatchDefault>/usr/local/src/gug/gug/service/cm/batch_job.xml.default</BatchDefault> - <BinaryDefault>/usr/local/src/gug/gug/service/cm/binary_job.xml.default</BinaryDefault> <Architecture>i686</Architecture> <OperatingSystem>LINUX</OperatingSystem> --- 7,10 ---- |
|
From: Roczei G. <ro...@us...> - 2007-04-22 21:05:21
|
Update of /cvsroot/gug/gug/gug/service/cm In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv2718/gug/service/cm Removed Files: binary_job.xml.default Log Message: clean --- binary_job.xml.default DELETED --- |
|
From: Roczei G. <ro...@us...> - 2007-04-22 21:04:09
|
Update of /cvsroot/gug/gug/gug/service/cm In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv2309/gug/service/cm Removed Files: batch_job.xml.default Log Message: clean --- batch_job.xml.default DELETED --- |
|
From: Roczei G. <ro...@us...> - 2007-04-22 20:54:50
|
Update of /cvsroot/gug/gug/gug/common In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv31230 Modified Files: jsdl_parser.py Log Message: bugfix Index: jsdl_parser.py =================================================================== RCS file: /cvsroot/gug/gug/gug/common/jsdl_parser.py,v retrieving revision 1.10 retrieving revision 1.11 diff -C2 -d -r1.10 -r1.11 *** jsdl_parser.py 20 Apr 2007 20:32:27 -0000 1.10 --- jsdl_parser.py 22 Apr 2007 20:54:49 -0000 1.11 *************** *** 79,85 **** """ for item in desc.keys(): if item not in SupportedFeatures.keys(): ! raise UnsupportedFeatureFault, "Unknown feature: %s" % item try: --- 79,89 ---- """ + for item in desc.keys(): if item not in SupportedFeatures.keys(): ! if item[0] == '#': ! del desc[item] ! else: ! raise UnsupportedFeatureFault, "Unknown feature: %s" % item try: *************** *** 102,108 **** if directory_structure: desc['executable'] = 'bin/' + desc['executable'] ! desc['input'] = 'input/' + desc['input'] ! desc['output'] = 'output/' + desc['output'] ! desc['error'] = 'output/' + desc['error'] jsdldoc += """ --- 106,112 ---- if directory_structure: desc['executable'] = 'bin/' + desc['executable'] ! desc['input'] = 'input/' + desc.get('input', 'stdin.txt') ! desc['output'] = 'output/' + desc.get('output', 'stdout.txt') ! desc['error'] = 'output/' + desc.get('error', 'stderr.txt') jsdldoc += """ |
|
From: Roczei G. <ro...@us...> - 2007-04-22 20:54:10
|
Update of /cvsroot/gug/gug/gug/cli In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv30883 Modified Files: cm.py Log Message: bugfix Index: cm.py =================================================================== RCS file: /cvsroot/gug/gug/gug/cli/cm.py,v retrieving revision 1.20 retrieving revision 1.21 diff -C2 -d -r1.20 -r1.21 *** cm.py 20 Apr 2007 20:30:37 -0000 1.20 --- cm.py 22 Apr 2007 20:54:10 -0000 1.21 *************** *** 30,36 **** def _cm(self): ! gis = get_working_client(self.gis_peers) ! cm_data = gis.get({'type' : self.send_job_to}, True)[0] ! return get_working_client(cm_data[0]['urls']) def _submit(self, args): --- 30,54 ---- def _cm(self): ! ! try: ! gis = get_working_client(self.gis_peers) ! except Exception, error: ! print "Error message: %s" % error ! sys.exit(-1) ! ! i = 0 ! while True: ! try: ! cm_data = gis.get({'type' : self.send_job_to}, True)[0] ! break ! except: ! if i < 3: ! sleep(1) ! i = i + 1 ! continue ! else: ! print "I did not found Cluster Manager service, try again later!" ! ! return get_working_client(cm_data[0]['urls']) def _submit(self, args): *************** *** 372,375 **** --- 390,394 ---- output = output.txt error = error.txt + #comment string1 universe = Binary checkpoint = False *************** *** 381,384 **** --- 400,404 ---- totalvirtualmemory = 4000 jobproject = gug + #comment string2 Directory stucture: |
|
From: Roczei G. <ro...@us...> - 2007-04-22 20:53:40
|
Update of /cvsroot/gug/gug/gug/cli In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv30448 Modified Files: clgr.py Log Message: New feature: comment in the submit file Index: clgr.py =================================================================== RCS file: /cvsroot/gug/gug/gug/cli/clgr.py,v retrieving revision 1.47 retrieving revision 1.48 diff -C2 -d -r1.47 -r1.48 *** clgr.py 3 Apr 2007 16:49:36 -0000 1.47 --- clgr.py 22 Apr 2007 20:53:37 -0000 1.48 *************** *** 316,319 **** --- 316,320 ---- input = input.txt output = output.txt + #comment string1 error = error.txt universe = Binary *************** *** 323,326 **** --- 324,328 ---- operatingsystem = LINUX totalcpucount = 1 + #comment string2 totalphysicalmemory = 2000 totalvirtualmemory = 4000 |
|
From: Roczei G. <ro...@us...> - 2007-04-20 20:34:52
|
Update of /cvsroot/gug/gug/gug/service/exec In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv5325 Modified Files: exec.py Log Message: The CM's directory structure depending removed Index: exec.py =================================================================== RCS file: /cvsroot/gug/gug/gug/service/exec/exec.py,v retrieving revision 1.51 retrieving revision 1.52 diff -C2 -d -r1.51 -r1.52 *** exec.py 2 Apr 2007 03:32:29 -0000 1.51 --- exec.py 20 Apr 2007 20:34:51 -0000 1.52 *************** *** 70,78 **** batchjob = False checkpoint = desc.get('checkpoint','0') if checkpoint == '1' and not batchjob: args.insert(0, self.my_exec.CKPTPath) args.insert(1, '-n') ! args.insert(2, 'bin/' + str(desc['job_migration']) + '--' + pid + '.ckpt') args.insert(3, '-a') args.insert(4, 'SIGUSR1') --- 70,81 ---- batchjob = False + if desc['directory_structure']: + desc['job_migration'] = 'bin/' + str(desc['job_migration']) + checkpoint = desc.get('checkpoint','0') if checkpoint == '1' and not batchjob: args.insert(0, self.my_exec.CKPTPath) args.insert(1, '-n') ! args.insert(2, str(desc['job_migration']) + '--' + pid + '.ckpt') args.insert(3, '-a') args.insert(4, 'SIGUSR1') *************** *** 118,122 **** self.errorfd = None ! my_exec.execids.add({'grid_id':pid, 'environment':environment, 'args':args, 'initialdir':desc['initialdir'],'checkpoint':checkpoint, 'batchjob':batchjob, 'status':BES_OVERALL_STATE_NEW, 'mem_usage':-1, 'cpu_usage':-1, 'exit_code':None, 'desc':desc , 'shutdown':False, 'job_migration' : desc.get('job_migration',0)}) def execute(self,job_id): --- 121,125 ---- self.errorfd = None ! my_exec.execids.add({'grid_id':pid, 'environment':environment, 'args':args, 'initialdir':desc['initialdir'],'checkpoint':checkpoint, 'batchjob':batchjob, 'status':BES_OVERALL_STATE_NEW, 'mem_usage':-1, 'cpu_usage':-1, 'exit_code':None, 'desc':desc , 'shutdown':False, 'job_migration' : desc.get('job_migration',0), 'directory_structure' : desc['directory_structure']}) def execute(self,job_id): *************** *** 566,573 **** if job['status'] != BES_OVERALL_STATE_SUSPENDED: raise JobNotSuspended ! ckpt_check1 = path.join(job['initialdir'], 'bin/' + str(job['job_migration']) + '--' + job_id +'.ckpt.check1') ! ckpt_check2 = path.join(job['initialdir'], 'bin/' + str(job['job_migration']) + '--' + job_id +'.ckpt.check2') ! ckpt_copy = path.join(job['initialdir'], 'bin/' + str(job['job_migration']) + '--' + job_id +'.ckpt.copy') ! ckpt = path.join(job['initialdir'], 'bin/' + str(job['job_migration']) + '--' + job_id +'.ckpt') if not path.exists(ckpt) and not path.exists(ckpt_copy): --- 569,580 ---- if job['status'] != BES_OVERALL_STATE_SUSPENDED: raise JobNotSuspended ! ! if job['directory_structure']: ! job['job_migration'] = 'bin/' + str(job['job_migration']) ! ! ckpt_check1 = path.join(job['initialdir'], str(job['job_migration']) + '--' + job_id +'.ckpt.check1') ! ckpt_check2 = path.join(job['initialdir'], str(job['job_migration']) + '--' + job_id +'.ckpt.check2') ! ckpt_copy = path.join(job['initialdir'], str(job['job_migration']) + '--' + job_id +'.ckpt.copy') ! ckpt = path.join(job['initialdir'], str(job['job_migration']) + '--' + job_id +'.ckpt') if not path.exists(ckpt) and not path.exists(ckpt_copy): |
|
From: Roczei G. <ro...@us...> - 2007-04-20 20:33:46
|
Update of /cvsroot/gug/gug/gug/service/cm In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv4918 Modified Files: cm.py Log Message: The CM's directory structure depending removed Index: cm.py =================================================================== RCS file: /cvsroot/gug/gug/gug/service/cm/cm.py,v retrieving revision 1.36 retrieving revision 1.37 diff -C2 -d -r1.36 -r1.37 *** cm.py 10 Apr 2007 05:59:10 -0000 1.36 --- cm.py 20 Apr 2007 20:33:45 -0000 1.37 *************** *** 307,314 **** return good_proxies ! def _submit(self, jsdl): """ ! This will queuded the job to the pool and the _sched task will schedule it later. ! The desc is a dictionary. The Exec service input is desc! start(desc) --- 307,317 ---- return good_proxies ! def submit(self, auth, jsdl, directory_structure = False): # CM client default ! return self._submit(jsdl, directory_structure) ! ! def _submit(self, jsdl, directory_structure = True): # CLGR client default """ ! This _submit method will put the job in the pool and the _sched task will schedule it later. ! The desc is a dictionary data. The Exec service input parameter is desc! start(desc) *************** *** 316,320 **** try: pid = mkuid() ! desc = self._create_submit_desc(jsdl) self.cmids.add({'grid_id' : pid, 'exec_job_id' : False, 'exec_id' : False, 'desc': desc, \ 'last_check_time' : time.time(), 'status' : BES_OVERALL_STATE_NEW, 'jsdl' : jsdl,\ --- 319,323 ---- try: pid = mkuid() ! desc = self._create_submit_desc(jsdl, directory_structure) self.cmids.add({'grid_id' : pid, 'exec_job_id' : False, 'exec_id' : False, 'desc': desc, \ 'last_check_time' : time.time(), 'status' : BES_OVERALL_STATE_NEW, 'jsdl' : jsdl,\ *************** *** 537,548 **** self.freecpunum = freecpunum ! def _create_submit_desc(self, jsdl_str): out = {} try: ! out = create_desc_from_jsdl(jsdl_str) print out #out['walltimelimit'] = 1209600 # 2 weeks out['ckpt_period'] = self.CKPTPeriod out['job_migration'] = 0 print out --- 540,552 ---- self.freecpunum = freecpunum ! def _create_submit_desc(self, jsdl_str, directory_structure): out = {} try: ! out = create_desc_from_jsdl(jsdl_str, directory_structure) print out #out['walltimelimit'] = 1209600 # 2 weeks out['ckpt_period'] = self.CKPTPeriod + out['directory_structure'] = directory_structure out['job_migration'] = 0 print out |
|
From: Roczei G. <ro...@us...> - 2007-04-20 20:32:31
|
Update of /cvsroot/gug/gug/gug/common In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv4096 Modified Files: jsdl_parser.py Log Message: The CM's directory structure depending removed Index: jsdl_parser.py =================================================================== RCS file: /cvsroot/gug/gug/gug/common/jsdl_parser.py,v retrieving revision 1.9 retrieving revision 1.10 diff -C2 -d -r1.9 -r1.10 *** jsdl_parser.py 10 Apr 2007 04:28:24 -0000 1.9 --- jsdl_parser.py 20 Apr 2007 20:32:27 -0000 1.10 *************** *** 69,73 **** raise UnsupportedFeatureFault, "Connection error: %s" % host ! def create_jsdl_from_desc(desc, staging_data = None): """ --- 69,73 ---- raise UnsupportedFeatureFault, "Connection error: %s" % host ! def create_jsdl_from_desc(desc, staging_data = None, directory_structure = True): """ *************** *** 99,110 **** </jsdl:JobIdentification> """ % (desc['name'], desc.get('description','No description.'), desc.get('jobproject','gug')) ! jsdldoc += """ <jsdl:Application> <jsdl:ApplicationName>%s</jsdl:ApplicationName> <jsdl-hpcpa:BasicHPCProfileApplication> ! <jsdl-hpcpa:Executable>bin/%s</jsdl-hpcpa:Executable>""" \ % (desc['name'], desc['executable']) ! if desc.has_key('arguments'): list = desc['arguments'].split() --- 99,116 ---- </jsdl:JobIdentification> """ % (desc['name'], desc.get('description','No description.'), desc.get('jobproject','gug')) ! ! if directory_structure: ! desc['executable'] = 'bin/' + desc['executable'] ! desc['input'] = 'input/' + desc['input'] ! desc['output'] = 'output/' + desc['output'] ! desc['error'] = 'output/' + desc['error'] ! jsdldoc += """ <jsdl:Application> <jsdl:ApplicationName>%s</jsdl:ApplicationName> <jsdl-hpcpa:BasicHPCProfileApplication> ! <jsdl-hpcpa:Executable>%s</jsdl-hpcpa:Executable>""" \ % (desc['name'], desc['executable']) ! if desc.has_key('arguments'): list = desc['arguments'].split() *************** *** 113,122 **** if desc.has_key('input'): ! jsdldoc += """<jsdl-hpcpa:Input>input/%s</jsdl-hpcpa:Input>""" % desc['input'] ! ! jsdldoc += """<jsdl-hpcpa:Output>output/%s</jsdl-hpcpa:Output>""" \ % desc.get('output',desc['name'] + '.stdout') jsdldoc += """ ! <jsdl-hpcpa:Error>output/%s</jsdl-hpcpa:Error>""" \ % desc.get('error',desc['name'] + '.stderr') jsdldoc += """ --- 119,128 ---- if desc.has_key('input'): ! jsdldoc += """<jsdl-hpcpa:Input>%s</jsdl-hpcpa:Input>""" % desc['input'] ! ! jsdldoc += """<jsdl-hpcpa:Output>%s</jsdl-hpcpa:Output>""" \ % desc.get('output',desc['name'] + '.stdout') jsdldoc += """ ! <jsdl-hpcpa:Error>%s</jsdl-hpcpa:Error>""" \ % desc.get('error',desc['name'] + '.stderr') jsdldoc += """ *************** *** 239,243 **** ! def create_desc_from_jsdl(jsdl_str): """ Standards: --- 245,249 ---- ! def create_desc_from_jsdl(jsdl_str, directory_structure = True): """ Standards: |
|
From: Roczei G. <ro...@us...> - 2007-04-20 20:30:42
|
Update of /cvsroot/gug/gug/gug/cli In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv2502 Modified Files: cm.py Log Message: The CM's directory structure depending removed Index: cm.py =================================================================== RCS file: /cvsroot/gug/gug/gug/cli/cm.py,v retrieving revision 1.19 retrieving revision 1.20 diff -C2 -d -r1.19 -r1.20 *** cm.py 4 Apr 2007 12:41:10 -0000 1.19 --- cm.py 20 Apr 2007 20:30:37 -0000 1.20 *************** *** 46,50 **** if args[0] == '--jsdl': jsdldoc = file(args[1]).read() ! print 'Job ID: ', cm.CreateActivity(jsdldoc) return else: --- 46,50 ---- if args[0] == '--jsdl': jsdldoc = file(args[1]).read() ! print 'Job ID: ', cm.submit(jsdldoc, False) return else: *************** *** 53,57 **** verbose = True jsdldoc = self._classad_to_jsdl(args[0], verbose) ! print 'Job ID: ', cm.CreateActivity(jsdldoc) return except Exception, error: --- 53,57 ---- verbose = True jsdldoc = self._classad_to_jsdl(args[0], verbose) ! print 'Job ID: ', cm.submit(jsdldoc, False) return except Exception, error: *************** *** 299,303 **** if verbose: print 'desc: ', desc ! jsdl = create_jsdl_from_desc(desc) if verbose: print 'jsdl: ', jsdl --- 299,303 ---- if verbose: print 'desc: ', desc ! jsdl = create_jsdl_from_desc(desc, None, False) if verbose: print 'jsdl: ', jsdl *************** *** 355,358 **** --- 355,359 ---- print "Argument error: %s" % args self.usage() + def _help(self): print "The submit file's features:\n" *************** *** 372,376 **** error = error.txt universe = Binary ! checkpoint = True username = john architecture = other, i686 --- 373,377 ---- error = error.txt universe = Binary ! checkpoint = False username = john architecture = other, i686 *************** *** 384,396 **** /home/gug/test/ ! |__bin/foo | ! |__input/input.txt | ! |__output/output.txt | ! |__error/error.txt | |__classad_file Command for classad parsing --- 385,399 ---- /home/gug/test/ ! |__foo | ! |__input.txt | ! |__output.txt | ! |__error.txt | |__classad_file + | + |__job.jsdl Command for classad parsing |
|
From: Roczei G. <ro...@us...> - 2007-04-10 05:59:11
|
Update of /cvsroot/gug/gug/gug/service/cm In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv1731 Modified Files: cm.py Log Message: bugfix Index: cm.py =================================================================== RCS file: /cvsroot/gug/gug/gug/service/cm/cm.py,v retrieving revision 1.35 retrieving revision 1.36 diff -C2 -d -r1.35 -r1.36 *** cm.py 10 Apr 2007 05:38:41 -0000 1.35 --- cm.py 10 Apr 2007 05:59:10 -0000 1.36 *************** *** 105,109 **** try: job['error_message'] = \ ! file(os.path.join(job['desc']['initialdir'],job['error'])).read() except IOError, error: #There is no error file pass --- 105,109 ---- try: job['error_message'] = \ ! file(os.path.join(job['desc']['initialdir'],job['desc']['error'])).read() except IOError, error: #There is no error file pass |
|
From: Roczei G. <ro...@us...> - 2007-04-10 05:38:42
|
Update of /cvsroot/gug/gug/gug/service/cm In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv26591 Modified Files: cm.py Log Message: bugfix Index: cm.py =================================================================== RCS file: /cvsroot/gug/gug/gug/service/cm/cm.py,v retrieving revision 1.34 retrieving revision 1.35 diff -C2 -d -r1.34 -r1.35 *** cm.py 10 Apr 2007 05:24:34 -0000 1.34 --- cm.py 10 Apr 2007 05:38:41 -0000 1.35 *************** *** 235,239 **** result = item['exec_job_id'] if item['exec_id'] != exec_id: ! log.msg('This job: %s is MIGRATED from [Execid: %s] to [Execid: %s]' % (item['grid_id'], item['exec_id'], exec_id)) log.msg('Job REVOKED: %s'% item['grid_id']) except JobNotSuspended: --- 235,239 ---- result = item['exec_job_id'] if item['exec_id'] != exec_id: ! log.msg('The %s job MIGRATED from [Execid: %s] to [Execid: %s]' % (item['grid_id'], item['exec_id'], exec_id)) log.msg('Job REVOKED: %s'% item['grid_id']) except JobNotSuspended: |
|
From: Roczei G. <ro...@us...> - 2007-04-10 05:24:36
|
Update of /cvsroot/gug/gug/gug/service/cm In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv21344 Modified Files: cm.py Log Message: bugfix Index: cm.py =================================================================== RCS file: /cvsroot/gug/gug/gug/service/cm/cm.py,v retrieving revision 1.33 retrieving revision 1.34 diff -C2 -d -r1.33 -r1.34 *** cm.py 4 Apr 2007 12:33:11 -0000 1.33 --- cm.py 10 Apr 2007 05:24:34 -0000 1.34 *************** *** 79,83 **** client = self.execes[job['exec_id']] except KeyError: ! log.msg('Exec\'s job access error, Exec ID: %s' % job['exec_id']) self._check(job) self.cmids.commit(job['grid_id']) --- 79,83 ---- client = self.execes[job['exec_id']] except KeyError: ! log.msg('Job access error: %s, Exec ID: %s' % (job['grid_id'], job['exec_id'])) self._check(job) self.cmids.commit(job['grid_id']) *************** *** 95,103 **** new_status = result['status'] if job['status'] != new_status: ! log.msg('Job status changed from %s to %s: %s' % \ (overall_states[job['status']], overall_states[new_status], job['grid_id'])) job['status'] = new_status if result['status'] == BES_OVERALL_STATE_UNKNOWN: ! log.error('Job\'s state unknown: %s' % job['grid_id']) log.error('Rescheduling...') job['status'] = BES_OVERALL_STATE_NEW --- 95,103 ---- new_status = result['status'] if job['status'] != new_status: ! log.msg('Job\'s status changed from %s to %s: %s' % \ (overall_states[job['status']], overall_states[new_status], job['grid_id'])) job['status'] = new_status if result['status'] == BES_OVERALL_STATE_UNKNOWN: ! log.error('Job\'s status unknown: %s' % job['grid_id']) log.error('Rescheduling...') job['status'] = BES_OVERALL_STATE_NEW *************** *** 236,244 **** if item['exec_id'] != exec_id: log.msg('This job: %s is MIGRATED from [Execid: %s] to [Execid: %s]' % (item['grid_id'], item['exec_id'], exec_id)) ! log.msg('Job REVOKED: %s'% job_id) except JobNotSuspended: item['status'] = BES_OVERALL_STATE_RUNNING item['cpu_usage'] = 'Unknown' ! log.msg('Job REVOKED: %s'% job_id) item['mem_usage'] = 'Unknown' result = item['exec_job_id'] --- 236,244 ---- if item['exec_id'] != exec_id: log.msg('This job: %s is MIGRATED from [Execid: %s] to [Execid: %s]' % (item['grid_id'], item['exec_id'], exec_id)) ! log.msg('Job REVOKED: %s'% item['grid_id']) except JobNotSuspended: item['status'] = BES_OVERALL_STATE_RUNNING item['cpu_usage'] = 'Unknown' ! log.msg('Job REVOKED: %s'% item['grid_id']) item['mem_usage'] = 'Unknown' result = item['exec_job_id'] |
|
From: Roczei G. <ro...@us...> - 2007-04-10 04:28:31
|
Update of /cvsroot/gug/gug/gug/common In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv32441 Modified Files: jsdl_parser.py Log Message: bugfix Index: jsdl_parser.py =================================================================== RCS file: /cvsroot/gug/gug/gug/common/jsdl_parser.py,v retrieving revision 1.8 retrieving revision 1.9 diff -C2 -d -r1.8 -r1.9 *** jsdl_parser.py 28 Mar 2007 11:38:51 -0000 1.8 --- jsdl_parser.py 10 Apr 2007 04:28:24 -0000 1.9 *************** *** 57,67 **** input = input.upper().strip() if input == "TRUE" or input == "1": ! return 1 elif input == "FALSE" or input == "0": ! return 0 ! elif input == '1': ! return 1 ! elif input == '0': ! return 0 else: raise UnsupportedFeatureFault, 'Bad checkpoint value: %s' % input --- 57,63 ---- input = input.upper().strip() if input == "TRUE" or input == "1": ! return '1' elif input == "FALSE" or input == "0": ! return '0' else: raise UnsupportedFeatureFault, 'Bad checkpoint value: %s' % input |
|
From: Nagy Z. <zs...@us...> - 2007-04-04 14:15:02
|
Update of /cvsroot/gug/gug/gug/host In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv19099/gug/host Modified Files: timed.py Log Message: if 'no more timed' do not start any tasks Index: timed.py =================================================================== RCS file: /cvsroot/gug/gug/gug/host/timed.py,v retrieving revision 1.12 retrieving revision 1.13 diff -C2 -d -r1.12 -r1.13 *** timed.py 19 Feb 2007 16:29:34 -0000 1.12 --- timed.py 4 Apr 2007 14:14:15 -0000 1.13 *************** *** 117,120 **** --- 117,123 ---- If once is True (the default is False) then the task only runs once. """ + if not more_timed: + print 'Cannot start', self, '- no more timed tasks allowed.' + return False if not self.running_lock.acquire(0): print 'Cannot start', self, '- already running.' |
|
From: Nagy Z. <zs...@us...> - 2007-04-04 14:13:50
|
Update of /cvsroot/gug/gug/gug/host In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv18926/gug/host Modified Files: http.py Log Message: bugfix Index: http.py =================================================================== RCS file: /cvsroot/gug/gug/gug/host/http.py,v retrieving revision 1.18 retrieving revision 1.19 diff -C2 -d -r1.18 -r1.19 *** http.py 3 Apr 2007 14:38:42 -0000 1.18 --- http.py 4 Apr 2007 14:13:43 -0000 1.19 *************** *** 256,259 **** --- 256,263 ---- return 'SecureHTTPListenerIPv6 (%s)' % id(self) + def server_bind(self): + self.socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1) + HTTPListener.server_bind(self) + class HTTPServer: """ This class is handling (Secure)HTTPListener instances. """ *************** *** 327,331 **** try: # creating the secure listener object ! listener = SecureHTTPListener6(('',port), RequestHandler, self.ssl_contexts['::']) log.msg('%s is bound to [::]:%d' % (listener, port)) self.listeners[('::',port)] = listener --- 331,335 ---- try: # creating the secure listener object ! listener = SecureHTTPListenerIPv6(('',port), RequestHandler, self.ssl_contexts['::']) log.msg('%s is bound to [::]:%d' % (listener, port)) self.listeners[('::',port)] = listener *************** *** 343,347 **** # create a secure listener object for each ip if ':' in ip: ! listener = SecureHTTPListener6((ip,port), \ RequestHandler, self.ssl_contexts[ip]) else: --- 347,351 ---- # create a secure listener object for each ip if ':' in ip: ! listener = SecureHTTPListenerIPv6((ip,port), \ RequestHandler, self.ssl_contexts[ip]) else: *************** *** 392,396 **** ip4, ip6, port, path = location() for interface in [(ip4, port), (ip6, port)]: ! if interface: # if this is the first location with this interface if not interfaces.has_key(interface): --- 396,400 ---- ip4, ip6, port, path = location() for interface in [(ip4, port), (ip6, port)]: ! if interface[0]: # if this is the first location with this interface if not interfaces.has_key(interface): *************** *** 410,414 **** ip4, ip6, port, path = location() for interface in [(ip4, port), (ip6, port)]: ! if interface: # try to remove it if it is there at all if interfaces.has_key(interface): --- 414,418 ---- ip4, ip6, port, path = location() for interface in [(ip4, port), (ip6, port)]: ! if interface[0]: # try to remove it if it is there at all if interfaces.has_key(interface): |
|
From: Nagy Z. <zs...@us...> - 2007-04-04 14:13:21
|
Update of /cvsroot/gug/gug/gug/host In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv18547/gug/host Modified Files: ctl.py Log Message: do not stop tasks before cleanup Index: ctl.py =================================================================== RCS file: /cvsroot/gug/gug/gug/host/ctl.py,v retrieving revision 1.26 retrieving revision 1.27 diff -C2 -d -r1.26 -r1.27 *** ctl.py 3 Apr 2007 12:40:50 -0000 1.26 --- ctl.py 4 Apr 2007 14:13:18 -0000 1.27 *************** *** 284,289 **** try: log.msg('Controller is shutting down.') - log.msg('Set "no more timed".') - timed.no_more_timed() log.msg('Stopping server.') self.server.stop() --- 284,287 ---- *************** *** 292,295 **** --- 290,295 ---- if self.services[sid]['instance']: self.stop_service(sid) + log.msg('Set "no more timed".') + timed.no_more_timed() runo = timed.number_of_running_tasks() while runo > 0: |
|
From: Roczei G. <ro...@us...> - 2007-04-04 12:41:15
|
Update of /cvsroot/gug/gug/gug/cli In directory sc8-pr-cvs2.sourceforge.net:/tmp/cvs-serv16072 Modified Files: cm.py Log Message: xml killed info Index: cm.py =================================================================== RCS file: /cvsroot/gug/gug/gug/cli/cm.py,v retrieving revision 1.18 retrieving revision 1.19 diff -C2 -d -r1.18 -r1.19 *** cm.py 4 Apr 2007 12:33:11 -0000 1.18 --- cm.py 4 Apr 2007 12:41:10 -0000 1.19 *************** *** 233,236 **** --- 233,238 ---- xml += " \t<SuspendedJobs>%s" % item[1]['suspended'] xml += "</SuspendedJobs>\n" + xml += " \t<KilledJobs>%s" % item[1]['killed'] + xml += "</KilledJobs>\n" xml += " \t<CompletedJobs>%s" % item[1]['completed'] xml += "</CompletedJobs>\n" *************** *** 260,263 **** --- 262,267 ---- xml += " \t<TotalSuspendedJobs>%s" % item[1]['suspended'] xml += "</TotalSuspendedJobs>\n" + xml += " \t<TotalKilledJobs>%s" % item[1]['suspended'] + xml += "</TotalKilledJobs>\n" xml += " \t<TotalCompletedJobs>%s" % item[1]['completed'] xml += "</TotalCompletedJobs>\n" |
