Hi. First of all let me say that this is a great project. It's really easy to use the power of distributed computing with pymw even for a noob like me. I have a question though' about what kinds of files can be included in a workunit bundle. I know there can be the data files and additional python modules that will be zipped and included in the bundle (by passing them to PyMW\_Master.submit\_task). Is it possible to also include .so files? I have a python project that also includes extensions made with swig, e.g. c extension libraries. Can I somehow pass them to submit_task too, like it's done with modules?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've tried to investigate this a bit further. But I don't understand the way the zipped module files are transferred to the client. Here is how I understand it. (Since I want to use boinc, it's all from a boinc perspective):
PyMW_Master.submit\_task zippes the data files and the modules and creates a PyMW\_Task object. The boinc interface then takes that task object and puts the data files into the right place. Then boinc takes over and transferres everything to the client. But I can't see where the boinc interface puts the modules. The zipped module files don't seem to be touched by the boinc interface at all. Or am I overlooking something?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for your comments. Sorry for the delay in replying, but I thought Sourceforge would email me when someone posts here.
You should be able to include any type of file in the workunit bundle. The bundle is unzipped by _pymw_worker_manager(), which also cleans up the files after the task finishes. I believe the bundle files are placed in a temp directory, which is actually a bug we need to fix (they should be in a specified worker directory). If you're still having trouble with this, feel free to message me directly.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi. First of all let me say that this is a great project. It's really easy to use the power of distributed computing with pymw even for a noob like me. I have a question though' about what kinds of files can be included in a workunit bundle. I know there can be the data files and additional python modules that will be zipped and included in the bundle (by passing them to PyMW\_Master.submit\_task). Is it possible to also include .so files? I have a python project that also includes extensions made with swig, e.g. c extension libraries. Can I somehow pass them to submit_task too, like it's done with modules?
I've tried to investigate this a bit further. But I don't understand the way the zipped module files are transferred to the client. Here is how I understand it. (Since I want to use boinc, it's all from a boinc perspective):
PyMW_Master.submit\_task zippes the data files and the modules and creates a PyMW\_Task object. The boinc interface then takes that task object and puts the data files into the right place. Then boinc takes over and transferres everything to the client. But I can't see where the boinc interface puts the modules. The zipped module files don't seem to be touched by the boinc interface at all. Or am I overlooking something?
Thanks for your comments. Sorry for the delay in replying, but I thought Sourceforge would email me when someone posts here.
You should be able to include any type of file in the workunit bundle. The bundle is unzipped by _pymw_worker_manager(), which also cleans up the files after the task finishes. I believe the bundle files are placed in a temp directory, which is actually a bug we need to fix (they should be in a specified worker directory). If you're still having trouble with this, feel free to message me directly.