Re: [Algorithms] General purpose task parallel threading approach

[Gregory J. <gj...@da...>]

Honestly, Alexander, at this point you largely are re-implementing TBB. Not
that this by itself should give you pause, but (and maybe this is lost on me
over the past 180 or so posts), what are you offering that TBB doesn't?
Given that a library such as asynclib will be considered by those starting a
new project, and not likely by those already using a particular solution
(whether commercial or in-house), what is(are) asynclib's differentiating
factor(s)?

 

On a side note, if you want to know what would make asynclib more appealing
in general, I'd recommend spending more time in the TBB forums, where plenty
of feedback on this topic is available.

 

Regards

Greg

 

From: asy...@gm... [mailto:asy...@gm...] 
Sent: Wednesday, April 15, 2009 11:09 AM
To: Game Development Algorithms
Subject: Re: [Algorithms] General purpose task parallel threading approach

 

>So, in my world, I know what dependencies a task has before that task is
>The trade-off is that new
>dependencies can be introduced, but not synchronized
That's an ok tradeoff for your project, but for someone else these
restrictions may seem too exotic because a problem might be more dynamic in
nature.
What I try to do is to provide enough possibilities for you guys to achieve
what you want with less  amount of movements. What you say, Jon, could be
achieved if you'd simply call async::Wait not inside the task but outside
it. You could wait not directly on a task, but on a proxy synchronization
object.  You could make a lot fewer synchronization points (relative to a
task) very far outside the tasks you work with, and resolve the dependency
for a bigger number of tasks, etc, etc. As I've with point 2 you can do the
same thing in many ways, where each way gives you slightly different
possibilites.  Of course you'll probably want to wrap some repetive actions
you do into a set of another primitives or functions to ease programming for
your version of the multithreading model which you view as a best approach
to solve your specific problem.

I have actually a question regarding point 1 (where you say a task needs to
be executed inside thread's stack and nowhere else). I think I know what you
want to say here:
a. - you need to have a growable stack
b. - you need cpu to access the same piece of memory from task to task.

As I can see now the only thing I can't provide is 'a.'.   I'll work on 'b.'
together with new optimized scheduler  which will reuse the same memory for
tasks as much as possible to prevent cache thrashing on a particular cpu.
Point a. is an OS feature which I can't provide in usermode, though I do
provide a way to handle 3rd party library calls for which you don't know
stack requirements, and in the future I'll try provide some automation for
the code running in small stack environment.


The whole purpose of asynclib idea was to simplify programming. Anything
asynclib does you can do it manually, for example, a sync point in the
middle of a task can be achieved without asynclib by splitting the task into
2 functions - one does work before sync point, another does it after, and
register 2nd part in some interlocked queue which will be traversed by some
other code. Task suspension can be achieved by saving data you need to
restore in some structure and restoring it later and so on. Can you squeze
out a bit more performance ? Of course you can, since you have all required
information you could an instruction here and there.
You can do all that, but for that you need to write code and spend
programmer's time on that, horrify the code by introducing a quadrillion of
callbacks, completion functions, etc. etc, where as I try to provide a good
enough solution which handles all that automatically, and seems like that
the only thing I can't handle is stack growing, any other aspect of
behaviour is achievable one way or another with a very acceptable
performance imho (yet I've noted some performance advices you mentioned and
will work on them).

Alexander.
 

2009/4/15 Jon Watte <jw...@gm...>

asy...@gm... wrote:
> When a task gets blocked by a dependency, worker thread can start
> executing other tasks recursively.
>

So, in my world, I know what dependencies a task has before that task is
started. Thus, I can prove that the task will not block, and thus no
recursion (or additional stacks) necessary. The trade-off is that new
dependencies can be introduced, but not synchronized, until after the
current task invocation ends. The benefit is that scheduling can be a
lot smarter, and the requirements for running a task are known and can
be bounded (very important in embedded and otherwise limited environments).

And, because tasks do not block while executing, this also means that
there are no extra context switches -- each task gets switched in;
that's the extent of switching. Thus, slightly less pressure on the
context switcher (but optimizing switching is still worthwhile, up to a
point).

Sincerely,

jw



----------------------------------------------------------------------------
--
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
GDAlgorithms-list mailing list
GDA...@li...
https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
Archives:
http://sourceforge.net/mailarchive/forum.php?forum_name=gdalgorithms-list




-- 
Regards,
Alexander Karnakov