#65 Auto Multi-Core Support

John Cerney

This patch adds support for splitting up numerical processing between multiple
parallel processor threads (or pthreads) using new functions "set_autopthread_targ"
and "set_autopthread_targ". This can improve processing performance (by greater than
2-4X in most cases) by taking advantage of multi-core and/or multi-processor machines.

See the new file Basic/Pod/ParallelCPU.pod for details.

This patch is against PDL 2.4.7 and is in git "format-patch" format


<< < 1 2 (Page 2 of 2)
  • John Cerney

    John Cerney - 2011-05-09

    The redefinition of warn to PDL->pdl_warn keeps perl's warn from being called in PP code when multi-threading. Calling perl's warn while multi-threading can cause a segfault. The new PDL->pdl_warn function saves warning messages while multi-threading and prints them out when the multi-threading is completed.

    Calling perl's warn inside of PP code appears to be fairly common. It shows up in the PDL package code, and in external libraries PP code (like some I have written). This is why the C preprocessor macro was added to define warn to PDL->pdl_warn. The change keeps existing PDL PP-code working with the new multi-threading support without any code changes.

    I think the simple fix for the GSL glue-code (gsl_integ.pd) is to undef the PDL->pdl_warn macro definition in this file, since it is not needed here anyway. The alternative is to have everywhere a warn was called changed to PDL->warn, which will cause segfaults when multi-threading if any warn to PDL->warn changes are missed. Basically I would prefer to work compilation issues like this when they come up rather than working strange segfault problems later.

    If there are no objections, I will make this fix to gsl_integ.pd in the patch and apply to the main trunk in git.

  • David Mertens

    David Mertens - 2011-05-09

    I see. Perl's warn has been a part of the API for some time, so we can't just remove it as I have proposed. That would be breaking backward compatibility. In that case, your solution is much better than mine.

    The only issue I have with the redefinition of warn is that it leads to nonsensical compiler errors when somebody uses the variable named 'warn', as was the case with the GSL INTEG bindings. The errors are caught at compile time, but I was only able to make sense of them because I knew they were due to the new patch. A note should be added to the PP docs that this is a reserved word so that if people run into problems, they will know why.

    That having been said, I think you can apply your patch since it will have no adverse effects. I'll test it when I get a chance.

  • John Cerney

    John Cerney - 2011-05-12
    • status: open --> open-accepted
  • John Cerney

    John Cerney - 2011-05-12

    Applied patch (with changes so gsl_integ.pdl compiles) to master in git.

  • John Cerney

    John Cerney - 2011-05-12
    • status: open-accepted --> closed-accepted
<< < 1 2 (Page 2 of 2)

Log in to post a comment.