#19 JobCluster.submit_node() broken in 4.10.6

[Mikhail T.]

The method is invoking self.submit_job_id_node() without passing the specified node -- the called function then interprets the first of the *args as the desired node...

Only the JobCluster has this bug -- the SharedJobCluster is Ok.

The fix is simple -- please, push out the 4.10.7 -- because the later releases have certain incompatibilities requiring wider code-changes for existing users.

@@ -2730,13 +2730,13 @@
     def submit_node(self, node, *args, **kwargs):
         """Submit a job for execution at 'node' with the given
         arguments. 'node' can be an instance of DispyNode (e.g., as
         received in cluster/job status callback) or IP address or host
         name.

         Arguments should be serializable and should correspond to
         arguments for computation used when cluster is created.
         """

-        return self.submit_job_id_node(None, *args, **kwargs)
+        return self.submit_job_id_node(None, node, *args, **kwargs)

     def submit_job_id_node(self, job_id, node, *args, **kwargs):

[Giridhar Pemmasani]

Thanks for submitting bug and fix.

As you know, 4.10.x is quite old release. While I can commit this fix in github, releasing will confuse many users that use new(er) releases. Since this issue is fixed in 4.11.0, I would prefer not to push this.

If you don't mind, can you describe your use case?

Thanks again!

[Mikhail T.]

If you don't mind, can you describe your use case?

I forgot the details, but I recall, that, when I attempted to upgrade our Dispy PIP here 2+ years ago -- to 4.11.0 -- the existing code stopped working. I think, it had to do with the tasks remaining mutable in 4.10.x -- but in 4.11 they became private copies after cluster.submit() returns.

Our code is written to retry any failed task once more. The failure-count is part of the task -- which became impossible in 4.11 -- that's my best recollection :-(.

Of course, we could change our code -- but the management does not want to "fix it, if it aint broke"...

Meanwhile, the 4.10.6 works for our main project, which uses submit(). But it broke the secondary (rarely used) program -- which uses submit_node(). I had to investigate -- the investigation resulting in this ticket. We rolled back to 4.10.5 here.

[Giridhar Pemmasani]

Well, the last couple of years I have been working on improving and adding features to dispy to make it enterprise product (with support). One major feature is to integrate with multiple cloud platforms so applications can dynamically allocate and use cloud computing (in addition to local/remote compute nodes) to meet deadlines / peak demands. If you are using dispy in enterprise, would you or your team be interested in exploring? If so, email me at pgiri@yahoo.com or giri@uniaccel.com and I will send you some details.