dispy uncaught exception
Distributed and Parallel Computing with/for Python.
Brought to you by:
pgiri
With dispy 4.5.5 and asyncoro 3.5 I see this error in dispyscheduler:
2015-09-10 22:05:53,742 - asyncoro - uncaught exception in cleanup_computation/139708078636568:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/asyncoro/init.py", line 3226, in _schedule
retval = coro._generator.send(coro._value)
File "/usr/local/bin/dispyscheduler.py", line 1022, in cleanup_computation
os.remove(pkl_path)
OSError: [Errno 2] No such file or directory: '/tmp/dispy/scheduler/1441858917550_97673ba1c86da56034a4'
The attached patch is a workaround.
Thank you very much for the report and the patch.
Could you reproduce this issue, possibly with a simple test case? While the patch fixes the issue, it would be better to fix the cause; i.e., the file must have been removed already (if the file was removed externally, then this issue need not be fixed, and if the file was removed by dispyscheduler, then this should be fixed).
Note that the exceptions raised in a coroutine with asyncoro are contained within that coroutine and the program continues to work otherwise, so this exception is not really a problem - dispyscheduler would conitnue to work.
If it is possible to reproduce this issue, can you try the patch attached instead? I am assuming that 'cleanup_computation' is called more than once that is causing the issue. Attached patch avoids executing that path more than once.
I have committed a combination of above patch and a version of your patch; if you can test GitHub repository and confirm, it would be great.
View and moderate all "bugs Discussion" comments posted by this user
Mark all as spam, and block user from posting to "Bugs"
Thanks for the quick response!
I'm trying dispyscheduler with your patch and will let you know how it goes
View and moderate all "bugs Discussion" comments posted by this user
Mark all as spam, and block user from posting to "Bugs"
I confirm that your patch works, I see dispyscheduler log output like this:
2015-09-15 19:06:01,904 - dispyscheduler - Received reply for job 140578524239944 from 192.168.16.92
2015-09-15 19:06:01,922 - dispyscheduler - Removing "/tmp/dispy/scheduler/192.168.16.93/compute_ralQ4r"
2015-09-15 19:06:01,923 - dispyscheduler - Pending jobs: 0
2015-09-15 19:06:02,088 - dispyscheduler - Invalid computation "1442194951264" to cleanup ignored