pocl-devel

[pocl-devel] Unexpected results when running OpenCL kernels with pocl

[Matthias B. <ma...@ma...>]

Hi,

I am experiencing an incorrect behaviour
when running OpenCL kernels via pocl:

I wrote an application in C/C++
calling kernels written in OpenCL C.
It is a numerical simulation of the propagation
of an electrical stimulus on tissue,
from point to point on a 3D mesh
with thousands of points.

On one computer I run it
with Intel's Open Source Beignet driver
on an Ivy Bridge GT2 featuring 16 compute units.
Calculation is time consuming, but the results are good:
A wavefront of excitation forms and travels across
the mesh as one would expect.

When I run the identical source code with pocl
on a machine featuring a Xeon L5640 showing 23 compute units,
the results are completely different i.e. incorrect:
The stimulus is not propagating in a wavefront,
but instead wanders in random lines across the mesh.

The platform is
OpenCL 1.2 pocl 1.6, None+Asserts, LLVM 9.0.1,
RELOC, SLEEF, DISTRO, POCL_DEBUG.
The device is
pthread-Intel(R) Xeon(R) CPU  L5640  @ 2.27GHz
OpenCL 1.2 pocl HSTR: pthread-x86_64-pc-linux-gnu-westmere.

Unfortunately, I can't share the original source code,
but I suspect some sort of race condition or the
queue not being executed in the correct order.
Inserting global memory barriers in all kernels
and adding queue flush & finish calls
in the main program after every kernel call
didn't change anything though.

Do you have any ideas or suggestions?

Cheers,
Matthias

Re: [pocl-devel] Unexpected results when running OpenCL kernels with pocl

[Pekka J. <pek...@tu...>]

Hi Matthias,

Lately the reported issues have been more often race conditions or other
bugs in the application side code rather than bugs in PoCL. These can
appear as the program working as expected (by luck) in one OpenCL
platform, but not the other.

Can you try with the latest version (3.0-RC1) and the latest LLVM
to see if there's any difference? If it fails, a simplified reproducer
is needed to track the issue - otherwise it's only guesswork.

BR,
Pekka

On 5/20/22 00:56, Matthias Bock wrote:
> Hi,
> 
> I am experiencing an incorrect behaviour
> when running OpenCL kernels via pocl:
> 
> I wrote an application in C/C++
> calling kernels written in OpenCL C.
> It is a numerical simulation of the propagation
> of an electrical stimulus on tissue,
> from point to point on a 3D mesh
> with thousands of points.
> 
> On one computer I run it
> with Intel's Open Source Beignet driver
> on an Ivy Bridge GT2 featuring 16 compute units.
> Calculation is time consuming, but the results are good:
> A wavefront of excitation forms and travels across
> the mesh as one would expect.
> 
> When I run the identical source code with pocl
> on a machine featuring a Xeon L5640 showing 23 compute units,
> the results are completely different i.e. incorrect:
> The stimulus is not propagating in a wavefront,
> but instead wanders in random lines across the mesh.
> 
> The platform is
> OpenCL 1.2 pocl 1.6, None+Asserts, LLVM 9.0.1,
> RELOC, SLEEF, DISTRO, POCL_DEBUG.
> The device is
> pthread-Intel(R) Xeon(R) CPU  L5640  @ 2.27GHz
> OpenCL 1.2 pocl HSTR: pthread-x86_64-pc-linux-gnu-westmere.
> 
> Unfortunately, I can't share the original source code,
> but I suspect some sort of race condition or the
> queue not being executed in the correct order.
> Inserting global memory barriers in all kernels
> and adding queue flush & finish calls
> in the main program after every kernel call
> didn't change anything though.
> 
> Do you have any ideas or suggestions?
> 
> Cheers,
> Matthias
> 
> 
> _______________________________________________
> pocl-devel mailing list
> poc...@li...
> https://lists.sourceforge.net/lists/listinfo/pocl-devel

[pocl-devel] MacOS and POCL challenges

[Noah R. <noa...@gm...>]

Hi,
   I upgraded to an M1 Pro Mac and found that MacOS's built-in OpenCL
implementation no longer supports the CPU device.  The GPU is
supported -- for now at least, but doesn't do fp64.
  I'm very motivated to have a local OpenCL platform on my Mac that
supports CPU device.  This is by far the most productive way for me to
first debug and develop kernels versus GPU-based debugging or on
remote systems.   So I return to POCL but for the first time trying to
use it on a Mac.  I'm having struggles.  (latest releases MacOS
12.3.1, XCode 13.4)

My problems trying to build and run the latest POCL are more complex,
so I'll first mention a problem trying to use POCL acquired with the
Conda package manager this week.   I can build and run a simple
hello-world program with one kernel and it computes the right answers
using POCL.
OpenCL Devices Detected:
Platform: Portable Computing Language
Type: CPU, Name: pthread


Unfortunately, when I run my large computational science application,
I get a runtime error shortly after the first call to
clEnqueueNDRangeKernel().  The function returns CL_SUCCESS, another
thread raises SIGABRT, I assume after the following output:

ld: dynamic main executables must link with libSystem.dylib for
architecture arm64
error: linker command failed with exit code 1 (use -v to see invocation)
Final linking of kernel physical_initial_condition_evaluation failed.

Since my hello-world app works, yet the app I care about doesn't,  my
leading suspicion so far is some confusion about which clang / llvm is
getting utilized.  I've tried to be careful about what PATH is set and
I'm not setting DYLD_LIBRARY_PATH in any way.
I'd love to pass "-v" to the POCL link as suggested, but I'm not sure
how to do that.
The main application *is* linked with -lSystem so that error message
is probably referring to the POCL kernel build.
It looks like POCL_MSG_PRINT_LLVM could be helpful, but I think that
is only set when POCL is built, and I'm not able to build.

Ok,  so any suggestions about the above problem would be appreciated!


Sometime I'd really like to be able to build POCL myself.

I can tell that some special steps are needed to get the latest POCL
to build for MacOS.  Who is maintaining that Conda package?   Are the
build steps available?

   Lesser question stemming from my failed attempts to build POCL from source:
Question:  What is the difference between the "basic" device and
"pthread" device?  I'm not finding documentation for either host
devices.
   One reason I ask is that I found problems building "pthread" due to
Apple's pthread.h not including pthread_barrier support.      Looks
like two lines in CMakeLists.txt under if(ENABLE_HOST_CPU_DEVICES)
could be modified to turn off the pthreads device.   I also had
success naively providing  implementations of the missing functions in
file pthread_scheduler.c



Warm regards,

Noah

Re: [pocl-devel] MacOS and POCL challenges

[Isuru F. <is...@gm...>]

I think the issue is that you don't have the conda environment activated.
Activating the conda environment sets the SDKROOT env variable
so that clang can find the macOS SDK.

I maintain the pocl conda package with some other maintainers. You
can find our build scripts at
https://github.com/conda-forge/pocl-feedstock/blob/main/recipe/build.sh

Isuru

On Fri, May 20, 2022 at 9:38 AM Noah Reddell <noa...@gm...>
wrote:

> Hi,
>    I upgraded to an M1 Pro Mac and found that MacOS's built-in OpenCL
> implementation no longer supports the CPU device.  The GPU is
> supported -- for now at least, but doesn't do fp64.
>   I'm very motivated to have a local OpenCL platform on my Mac that
> supports CPU device.  This is by far the most productive way for me to
> first debug and develop kernels versus GPU-based debugging or on
> remote systems.   So I return to POCL but for the first time trying to
> use it on a Mac.  I'm having struggles.  (latest releases MacOS
> 12.3.1, XCode 13.4)
>
> My problems trying to build and run the latest POCL are more complex,
> so I'll first mention a problem trying to use POCL acquired with the
> Conda package manager this week.   I can build and run a simple
> hello-world program with one kernel and it computes the right answers
> using POCL.
> OpenCL Devices Detected:
> Platform: Portable Computing Language
> Type: CPU, Name: pthread
>
>
> Unfortunately, when I run my large computational science application,
> I get a runtime error shortly after the first call to
> clEnqueueNDRangeKernel().  The function returns CL_SUCCESS, another
> thread raises SIGABRT, I assume after the following output:
>
> ld: dynamic main executables must link with libSystem.dylib for
> architecture arm64
> error: linker command failed with exit code 1 (use -v to see invocation)
> Final linking of kernel physical_initial_condition_evaluation failed.
>
> Since my hello-world app works, yet the app I care about doesn't,  my
> leading suspicion so far is some confusion about which clang / llvm is
> getting utilized.  I've tried to be careful about what PATH is set and
> I'm not setting DYLD_LIBRARY_PATH in any way.
> I'd love to pass "-v" to the POCL link as suggested, but I'm not sure
> how to do that.
> The main application *is* linked with -lSystem so that error message
> is probably referring to the POCL kernel build.
> It looks like POCL_MSG_PRINT_LLVM could be helpful, but I think that
> is only set when POCL is built, and I'm not able to build.
>
> Ok,  so any suggestions about the above problem would be appreciated!
>
>
> Sometime I'd really like to be able to build POCL myself.
>
> I can tell that some special steps are needed to get the latest POCL
> to build for MacOS.  Who is maintaining that Conda package?   Are the
> build steps available?
>
>    Lesser question stemming from my failed attempts to build POCL from
> source:
> Question:  What is the difference between the "basic" device and
> "pthread" device?  I'm not finding documentation for either host
> devices.
>    One reason I ask is that I found problems building "pthread" due to
> Apple's pthread.h not including pthread_barrier support.      Looks
> like two lines in CMakeLists.txt under if(ENABLE_HOST_CPU_DEVICES)
> could be modified to turn off the pthreads device.   I also had
> success naively providing  implementations of the missing functions in
> file pthread_scheduler.c
>
>
>
> Warm regards,
>
> Noah
>
>
> _______________________________________________
> pocl-devel mailing list
> poc...@li...
> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>

Re: [pocl-devel] MacOS and POCL challenges

[Noah R. <noa...@gm...>]

Hi Isuru,
    Thank you for maintaining the conda package and the suggestion.

It is true I don't have the conda environment activated when building
/ running my large app.   I installed conda for the sole purpose of
POCL and generally use homebrew for similar needs.   I'm not seeing
any difference in build/run of hello-opencl whether I have conda
activated or not.  SDKROOT is not set either way.

Starting with a fresh shell:
----------------------------------------------
Last login: Wed May 18 22:48:50 on ttys005
noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
-Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
/Users/noah/Downloads/hello_opencl.c
noah@Spitzer ~ % ./a.out
Num Platforms: 1
platform[0] name: Portable Computing Language
platform[0] Num Devices: 1
device[0] name: pthread
function definition may only have a distinct !dbg attachment
i64 (i32)* @_Z13get_global_idj
Computed '1024/1024' correct values!
noah@Spitzer ~ % echo $SDKROOT

noah@Spitzer ~ % source /Users/noah/miniforge3/etc/profile.d/conda.sh
noah@Spitzer ~ % conda activate
(base) noah@Spitzer ~ % echo $SDKROOT

(base) noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
-Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
/Users/noah/Downloads/hello_opencl.c
(base) noah@Spitzer ~ % ./a.out
Num Platforms: 1
platform[0] name: Portable Computing Language
platform[0] Num Devices: 1
device[0] name: pthread
function definition may only have a distinct !dbg attachment
i64 (i32)* @_Z13get_global_idj
Computed '1024/1024' correct values!

(base) noah@Spitzer ~ % which clang
/usr/bin/clang
----------------------------------------------


I'll check out your build.sh recipe.    At first glance, it is not
clear to me how you're dealing with the MacOS pthread not having
pthread_barrier_t and related functions.  When I tried to build POCL
last week, those pthread parts are needed.

Cheers,
-Noah


On Fri, May 20, 2022 at 7:51 AM Isuru Fernando <is...@gm...> wrote:
>
> I think the issue is that you don't have the conda environment activated.
> Activating the conda environment sets the SDKROOT env variable
> so that clang can find the macOS SDK.
>
> I maintain the pocl conda package with some other maintainers. You
> can find our build scripts at https://github.com/conda-forge/pocl-feedstock/blob/main/recipe/build.sh
>
> Isuru
>
> On Fri, May 20, 2022 at 9:38 AM Noah Reddell <noa...@gm...> wrote:
>>
>> Hi,
>>    I upgraded to an M1 Pro Mac and found that MacOS's built-in OpenCL
>> implementation no longer supports the CPU device.  The GPU is
>> supported -- for now at least, but doesn't do fp64.
>>   I'm very motivated to have a local OpenCL platform on my Mac that
>> supports CPU device.  This is by far the most productive way for me to
>> first debug and develop kernels versus GPU-based debugging or on
>> remote systems.   So I return to POCL but for the first time trying to
>> use it on a Mac.  I'm having struggles.  (latest releases MacOS
>> 12.3.1, XCode 13.4)
>>
>> My problems trying to build and run the latest POCL are more complex,
>> so I'll first mention a problem trying to use POCL acquired with the
>> Conda package manager this week.   I can build and run a simple
>> hello-world program with one kernel and it computes the right answers
>> using POCL.
>> OpenCL Devices Detected:
>> Platform: Portable Computing Language
>> Type: CPU, Name: pthread
>>
>>
>> Unfortunately, when I run my large computational science application,
>> I get a runtime error shortly after the first call to
>> clEnqueueNDRangeKernel().  The function returns CL_SUCCESS, another
>> thread raises SIGABRT, I assume after the following output:
>>
>> ld: dynamic main executables must link with libSystem.dylib for
>> architecture arm64
>> error: linker command failed with exit code 1 (use -v to see invocation)
>> Final linking of kernel physical_initial_condition_evaluation failed.
>>
>> Since my hello-world app works, yet the app I care about doesn't,  my
>> leading suspicion so far is some confusion about which clang / llvm is
>> getting utilized.  I've tried to be careful about what PATH is set and
>> I'm not setting DYLD_LIBRARY_PATH in any way.
>> I'd love to pass "-v" to the POCL link as suggested, but I'm not sure
>> how to do that.
>> The main application *is* linked with -lSystem so that error message
>> is probably referring to the POCL kernel build.
>> It looks like POCL_MSG_PRINT_LLVM could be helpful, but I think that
>> is only set when POCL is built, and I'm not able to build.
>>
>> Ok,  so any suggestions about the above problem would be appreciated!
>>
>>
>> Sometime I'd really like to be able to build POCL myself.
>>
>> I can tell that some special steps are needed to get the latest POCL
>> to build for MacOS.  Who is maintaining that Conda package?   Are the
>> build steps available?
>>
>>    Lesser question stemming from my failed attempts to build POCL from source:
>> Question:  What is the difference between the "basic" device and
>> "pthread" device?  I'm not finding documentation for either host
>> devices.
>>    One reason I ask is that I found problems building "pthread" due to
>> Apple's pthread.h not including pthread_barrier support.      Looks
>> like two lines in CMakeLists.txt under if(ENABLE_HOST_CPU_DEVICES)
>> could be modified to turn off the pthreads device.   I also had
>> success naively providing  implementations of the missing functions in
>> file pthread_scheduler.c
>>
>>
>>
>> Warm regards,
>>
>> Noah
>>
>>
>> _______________________________________________
>> pocl-devel mailing list
>> poc...@li...
>> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>
> _______________________________________________
> pocl-devel mailing list
> poc...@li...
> https://lists.sourceforge.net/lists/listinfo/pocl-devel

Re: [pocl-devel] MacOS and POCL challenges

[Isuru F. <is...@gm...>]

Ah, that makes sense now. Can you try doing the following?

export SDKROOT=$(xcrun --show-sdk-path)

(Note that the clang you are using and the one from conda that pocl is
using are two different things)

Isuru

On Fri, May 20, 2022 at 10:41 AM Noah Reddell <noa...@gm...>
wrote:

> Hi Isuru,
>     Thank you for maintaining the conda package and the suggestion.
>
> It is true I don't have the conda environment activated when building
> / running my large app.   I installed conda for the sole purpose of
> POCL and generally use homebrew for similar needs.   I'm not seeing
> any difference in build/run of hello-opencl whether I have conda
> activated or not.  SDKROOT is not set either way.
>
> Starting with a fresh shell:
> ----------------------------------------------
> Last login: Wed May 18 22:48:50 on ttys005
> noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
> /Users/noah/Downloads/hello_opencl.c
> noah@Spitzer ~ % ./a.out
> Num Platforms: 1
> platform[0] name: Portable Computing Language
> platform[0] Num Devices: 1
> device[0] name: pthread
> function definition may only have a distinct !dbg attachment
> i64 (i32)* @_Z13get_global_idj
> Computed '1024/1024' correct values!
> noah@Spitzer ~ % echo $SDKROOT
>
> noah@Spitzer ~ % source /Users/noah/miniforge3/etc/profile.d/conda.sh
> noah@Spitzer ~ % conda activate
> (base) noah@Spitzer ~ % echo $SDKROOT
>
> (base) noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
> /Users/noah/Downloads/hello_opencl.c
> (base) noah@Spitzer ~ % ./a.out
> Num Platforms: 1
> platform[0] name: Portable Computing Language
> platform[0] Num Devices: 1
> device[0] name: pthread
> function definition may only have a distinct !dbg attachment
> i64 (i32)* @_Z13get_global_idj
> Computed '1024/1024' correct values!
>
> (base) noah@Spitzer ~ % which clang
> /usr/bin/clang
> ----------------------------------------------
>
>
> I'll check out your build.sh recipe.    At first glance, it is not
> clear to me how you're dealing with the MacOS pthread not having
> pthread_barrier_t and related functions.  When I tried to build POCL
> last week, those pthread parts are needed.
>
> Cheers,
> -Noah
>
>
> On Fri, May 20, 2022 at 7:51 AM Isuru Fernando <is...@gm...> wrote:
> >
> > I think the issue is that you don't have the conda environment activated.
> > Activating the conda environment sets the SDKROOT env variable
> > so that clang can find the macOS SDK.
> >
> > I maintain the pocl conda package with some other maintainers. You
> > can find our build scripts at
> https://github.com/conda-forge/pocl-feedstock/blob/main/recipe/build.sh
> >
> > Isuru
> >
> > On Fri, May 20, 2022 at 9:38 AM Noah Reddell <
> noa...@gm...> wrote:
> >>
> >> Hi,
> >>    I upgraded to an M1 Pro Mac and found that MacOS's built-in OpenCL
> >> implementation no longer supports the CPU device.  The GPU is
> >> supported -- for now at least, but doesn't do fp64.
> >>   I'm very motivated to have a local OpenCL platform on my Mac that
> >> supports CPU device.  This is by far the most productive way for me to
> >> first debug and develop kernels versus GPU-based debugging or on
> >> remote systems.   So I return to POCL but for the first time trying to
> >> use it on a Mac.  I'm having struggles.  (latest releases MacOS
> >> 12.3.1, XCode 13.4)
> >>
> >> My problems trying to build and run the latest POCL are more complex,
> >> so I'll first mention a problem trying to use POCL acquired with the
> >> Conda package manager this week.   I can build and run a simple
> >> hello-world program with one kernel and it computes the right answers
> >> using POCL.
> >> OpenCL Devices Detected:
> >> Platform: Portable Computing Language
> >> Type: CPU, Name: pthread
> >>
> >>
> >> Unfortunately, when I run my large computational science application,
> >> I get a runtime error shortly after the first call to
> >> clEnqueueNDRangeKernel().  The function returns CL_SUCCESS, another
> >> thread raises SIGABRT, I assume after the following output:
> >>
> >> ld: dynamic main executables must link with libSystem.dylib for
> >> architecture arm64
> >> error: linker command failed with exit code 1 (use -v to see invocation)
> >> Final linking of kernel physical_initial_condition_evaluation failed.
> >>
> >> Since my hello-world app works, yet the app I care about doesn't,  my
> >> leading suspicion so far is some confusion about which clang / llvm is
> >> getting utilized.  I've tried to be careful about what PATH is set and
> >> I'm not setting DYLD_LIBRARY_PATH in any way.
> >> I'd love to pass "-v" to the POCL link as suggested, but I'm not sure
> >> how to do that.
> >> The main application *is* linked with -lSystem so that error message
> >> is probably referring to the POCL kernel build.
> >> It looks like POCL_MSG_PRINT_LLVM could be helpful, but I think that
> >> is only set when POCL is built, and I'm not able to build.
> >>
> >> Ok,  so any suggestions about the above problem would be appreciated!
> >>
> >>
> >> Sometime I'd really like to be able to build POCL myself.
> >>
> >> I can tell that some special steps are needed to get the latest POCL
> >> to build for MacOS.  Who is maintaining that Conda package?   Are the
> >> build steps available?
> >>
> >>    Lesser question stemming from my failed attempts to build POCL from
> source:
> >> Question:  What is the difference between the "basic" device and
> >> "pthread" device?  I'm not finding documentation for either host
> >> devices.
> >>    One reason I ask is that I found problems building "pthread" due to
> >> Apple's pthread.h not including pthread_barrier support.      Looks
> >> like two lines in CMakeLists.txt under if(ENABLE_HOST_CPU_DEVICES)
> >> could be modified to turn off the pthreads device.   I also had
> >> success naively providing  implementations of the missing functions in
> >> file pthread_scheduler.c
> >>
> >>
> >>
> >> Warm regards,
> >>
> >> Noah
> >>
> >>
> >> _______________________________________________
> >> pocl-devel mailing list
> >> poc...@li...
> >> https://lists.sourceforge.net/lists/listinfo/pocl-devel
> >
> > _______________________________________________
> > pocl-devel mailing list
> > poc...@li...
> > https://lists.sourceforge.net/lists/listinfo/pocl-devel
>
>
> _______________________________________________
> pocl-devel mailing list
> poc...@li...
> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>

Re: [pocl-devel] MacOS and POCL challenges

[Noah R. <noa...@gm...>]

I find no difference exporting SDKROOT at build time,  run time, or both.

ld: dynamic main executables must link with libSystem.dylib for
architecture arm64
error: linker command failed with exit code 1 (use -v to see invocation)
Final linking of kernel phase_initial_condition_evaluation failed.

(base) noah@Spitzer ~ % which xcrun
/usr/bin/xcrun
(base) noah@Spitzer ~ % xcrun --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

When are you supposing SDKROOT in the environment affects behavior?

I tried to use dtruss to catch whatever system calls are happening
under clEnqueueNDRangeKernel() but that system utility isn't catching
anything.  ...MacOS security I think.

-Noah

On Fri, May 20, 2022 at 9:14 AM Isuru Fernando <is...@gm...> wrote:
>
> Ah, that makes sense now. Can you try doing the following?
>
> export SDKROOT=$(xcrun --show-sdk-path)
>
> (Note that the clang you are using and the one from conda that pocl is using are two different things)
>
> Isuru
>
> On Fri, May 20, 2022 at 10:41 AM Noah Reddell <noa...@gm...> wrote:
>>
>> Hi Isuru,
>>     Thank you for maintaining the conda package and the suggestion.
>>
>> It is true I don't have the conda environment activated when building
>> / running my large app.   I installed conda for the sole purpose of
>> POCL and generally use homebrew for similar needs.   I'm not seeing
>> any difference in build/run of hello-opencl whether I have conda
>> activated or not.  SDKROOT is not set either way.
>>
>> Starting with a fresh shell:
>> ----------------------------------------------
>> Last login: Wed May 18 22:48:50 on ttys005
>> noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
>> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
>> /Users/noah/Downloads/hello_opencl.c
>> noah@Spitzer ~ % ./a.out
>> Num Platforms: 1
>> platform[0] name: Portable Computing Language
>> platform[0] Num Devices: 1
>> device[0] name: pthread
>> function definition may only have a distinct !dbg attachment
>> i64 (i32)* @_Z13get_global_idj
>> Computed '1024/1024' correct values!
>> noah@Spitzer ~ % echo $SDKROOT
>>
>> noah@Spitzer ~ % source /Users/noah/miniforge3/etc/profile.d/conda.sh
>> noah@Spitzer ~ % conda activate
>> (base) noah@Spitzer ~ % echo $SDKROOT
>>
>> (base) noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
>> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
>> /Users/noah/Downloads/hello_opencl.c
>> (base) noah@Spitzer ~ % ./a.out
>> Num Platforms: 1
>> platform[0] name: Portable Computing Language
>> platform[0] Num Devices: 1
>> device[0] name: pthread
>> function definition may only have a distinct !dbg attachment
>> i64 (i32)* @_Z13get_global_idj
>> Computed '1024/1024' correct values!
>>
>> (base) noah@Spitzer ~ % which clang
>> /usr/bin/clang
>> ----------------------------------------------
>>
>>
>> I'll check out your build.sh recipe.    At first glance, it is not
>> clear to me how you're dealing with the MacOS pthread not having
>> pthread_barrier_t and related functions.  When I tried to build POCL
>> last week, those pthread parts are needed.
>>
>> Cheers,
>> -Noah
>>
>>
>> On Fri, May 20, 2022 at 7:51 AM Isuru Fernando <is...@gm...> wrote:
>> >
>> > I think the issue is that you don't have the conda environment activated.
>> > Activating the conda environment sets the SDKROOT env variable
>> > so that clang can find the macOS SDK.
>> >
>> > I maintain the pocl conda package with some other maintainers. You
>> > can find our build scripts at https://github.com/conda-forge/pocl-feedstock/blob/main/recipe/build.sh
>> >
>> > Isuru
>> >
>> > On Fri, May 20, 2022 at 9:38 AM Noah Reddell <noa...@gm...> wrote:
>> >>
>> >> Hi,
>> >>    I upgraded to an M1 Pro Mac and found that MacOS's built-in OpenCL
>> >> implementation no longer supports the CPU device.  The GPU is
>> >> supported -- for now at least, but doesn't do fp64.
>> >>   I'm very motivated to have a local OpenCL platform on my Mac that
>> >> supports CPU device.  This is by far the most productive way for me to
>> >> first debug and develop kernels versus GPU-based debugging or on
>> >> remote systems.   So I return to POCL but for the first time trying to
>> >> use it on a Mac.  I'm having struggles.  (latest releases MacOS
>> >> 12.3.1, XCode 13.4)
>> >>
>> >> My problems trying to build and run the latest POCL are more complex,
>> >> so I'll first mention a problem trying to use POCL acquired with the
>> >> Conda package manager this week.   I can build and run a simple
>> >> hello-world program with one kernel and it computes the right answers
>> >> using POCL.
>> >> OpenCL Devices Detected:
>> >> Platform: Portable Computing Language
>> >> Type: CPU, Name: pthread
>> >>
>> >>
>> >> Unfortunately, when I run my large computational science application,
>> >> I get a runtime error shortly after the first call to
>> >> clEnqueueNDRangeKernel().  The function returns CL_SUCCESS, another
>> >> thread raises SIGABRT, I assume after the following output:
>> >>
>> >> ld: dynamic main executables must link with libSystem.dylib for
>> >> architecture arm64
>> >> error: linker command failed with exit code 1 (use -v to see invocation)
>> >> Final linking of kernel physical_initial_condition_evaluation failed.
>> >>
>> >> Since my hello-world app works, yet the app I care about doesn't,  my
>> >> leading suspicion so far is some confusion about which clang / llvm is
>> >> getting utilized.  I've tried to be careful about what PATH is set and
>> >> I'm not setting DYLD_LIBRARY_PATH in any way.
>> >> I'd love to pass "-v" to the POCL link as suggested, but I'm not sure
>> >> how to do that.
>> >> The main application *is* linked with -lSystem so that error message
>> >> is probably referring to the POCL kernel build.
>> >> It looks like POCL_MSG_PRINT_LLVM could be helpful, but I think that
>> >> is only set when POCL is built, and I'm not able to build.
>> >>
>> >> Ok,  so any suggestions about the above problem would be appreciated!
>> >>
>> >>
>> >> Sometime I'd really like to be able to build POCL myself.
>> >>
>> >> I can tell that some special steps are needed to get the latest POCL
>> >> to build for MacOS.  Who is maintaining that Conda package?   Are the
>> >> build steps available?
>> >>
>> >>    Lesser question stemming from my failed attempts to build POCL from source:
>> >> Question:  What is the difference between the "basic" device and
>> >> "pthread" device?  I'm not finding documentation for either host
>> >> devices.
>> >>    One reason I ask is that I found problems building "pthread" due to
>> >> Apple's pthread.h not including pthread_barrier support.      Looks
>> >> like two lines in CMakeLists.txt under if(ENABLE_HOST_CPU_DEVICES)
>> >> could be modified to turn off the pthreads device.   I also had
>> >> success naively providing  implementations of the missing functions in
>> >> file pthread_scheduler.c
>> >>
>> >>
>> >>
>> >> Warm regards,
>> >>
>> >> Noah
>> >>
>> >>
>> >> _______________________________________________
>> >> pocl-devel mailing list
>> >> poc...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>> >
>> > _______________________________________________
>> > pocl-devel mailing list
>> > poc...@li...
>> > https://lists.sourceforge.net/lists/listinfo/pocl-devel
>>
>>
>> _______________________________________________
>> pocl-devel mailing list
>> poc...@li...
>> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>
> _______________________________________________
> pocl-devel mailing list
> poc...@li...
> https://lists.sourceforge.net/lists/listinfo/pocl-devel

Re: [pocl-devel] MacOS and POCL challenges

[Isuru F. <is...@gm...>]

SDKROOT is not the problem then.

I think you need the `ld` executable found in `/Users/noah/miniforge3/bin`
in the PATH.
(That ld is a patched ld to workaround a setting we have added in conda to
workaround
issues with SDKROOT)

This can probably be fixed in the conda package, but not sure.

Isuru

On Fri, May 20, 2022 at 11:49 AM Noah Reddell <noa...@gm...>
wrote:

> I find no difference exporting SDKROOT at build time,  run time, or both.
>
> ld: dynamic main executables must link with libSystem.dylib for
> architecture arm64
> error: linker command failed with exit code 1 (use -v to see invocation)
> Final linking of kernel phase_initial_condition_evaluation failed.
>
> (base) noah@Spitzer ~ % which xcrun
> /usr/bin/xcrun
> (base) noah@Spitzer ~ % xcrun --show-sdk-path
>
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
>
> When are you supposing SDKROOT in the environment affects behavior?
>
> I tried to use dtruss to catch whatever system calls are happening
> under clEnqueueNDRangeKernel() but that system utility isn't catching
> anything.  ...MacOS security I think.
>
> -Noah
>
> On Fri, May 20, 2022 at 9:14 AM Isuru Fernando <is...@gm...> wrote:
> >
> > Ah, that makes sense now. Can you try doing the following?
> >
> > export SDKROOT=$(xcrun --show-sdk-path)
> >
> > (Note that the clang you are using and the one from conda that pocl is
> using are two different things)
> >
> > Isuru
> >
> > On Fri, May 20, 2022 at 10:41 AM Noah Reddell <
> noa...@gm...> wrote:
> >>
> >> Hi Isuru,
> >>     Thank you for maintaining the conda package and the suggestion.
> >>
> >> It is true I don't have the conda environment activated when building
> >> / running my large app.   I installed conda for the sole purpose of
> >> POCL and generally use homebrew for similar needs.   I'm not seeing
> >> any difference in build/run of hello-opencl whether I have conda
> >> activated or not.  SDKROOT is not set either way.
> >>
> >> Starting with a fresh shell:
> >> ----------------------------------------------
> >> Last login: Wed May 18 22:48:50 on ttys005
> >> noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
> >> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
> >> /Users/noah/Downloads/hello_opencl.c
> >> noah@Spitzer ~ % ./a.out
> >> Num Platforms: 1
> >> platform[0] name: Portable Computing Language
> >> platform[0] Num Devices: 1
> >> device[0] name: pthread
> >> function definition may only have a distinct !dbg attachment
> >> i64 (i32)* @_Z13get_global_idj
> >> Computed '1024/1024' correct values!
> >> noah@Spitzer ~ % echo $SDKROOT
> >>
> >> noah@Spitzer ~ % source /Users/noah/miniforge3/etc/profile.d/conda.sh
> >> noah@Spitzer ~ % conda activate
> >> (base) noah@Spitzer ~ % echo $SDKROOT
> >>
> >> (base) noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
> >> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
> >> /Users/noah/Downloads/hello_opencl.c
> >> (base) noah@Spitzer ~ % ./a.out
> >> Num Platforms: 1
> >> platform[0] name: Portable Computing Language
> >> platform[0] Num Devices: 1
> >> device[0] name: pthread
> >> function definition may only have a distinct !dbg attachment
> >> i64 (i32)* @_Z13get_global_idj
> >> Computed '1024/1024' correct values!
> >>
> >> (base) noah@Spitzer ~ % which clang
> >> /usr/bin/clang
> >> ----------------------------------------------
> >>
> >>
> >> I'll check out your build.sh recipe.    At first glance, it is not
> >> clear to me how you're dealing with the MacOS pthread not having
> >> pthread_barrier_t and related functions.  When I tried to build POCL
> >> last week, those pthread parts are needed.
> >>
> >> Cheers,
> >> -Noah
> >>
> >>
> >> On Fri, May 20, 2022 at 7:51 AM Isuru Fernando <is...@gm...>
> wrote:
> >> >
> >> > I think the issue is that you don't have the conda environment
> activated.
> >> > Activating the conda environment sets the SDKROOT env variable
> >> > so that clang can find the macOS SDK.
> >> >
> >> > I maintain the pocl conda package with some other maintainers. You
> >> > can find our build scripts at
> https://github.com/conda-forge/pocl-feedstock/blob/main/recipe/build.sh
> >> >
> >> > Isuru
> >> >
> >> > On Fri, May 20, 2022 at 9:38 AM Noah Reddell <
> noa...@gm...> wrote:
> >> >>
> >> >> Hi,
> >> >>    I upgraded to an M1 Pro Mac and found that MacOS's built-in OpenCL
> >> >> implementation no longer supports the CPU device.  The GPU is
> >> >> supported -- for now at least, but doesn't do fp64.
> >> >>   I'm very motivated to have a local OpenCL platform on my Mac that
> >> >> supports CPU device.  This is by far the most productive way for me
> to
> >> >> first debug and develop kernels versus GPU-based debugging or on
> >> >> remote systems.   So I return to POCL but for the first time trying
> to
> >> >> use it on a Mac.  I'm having struggles.  (latest releases MacOS
> >> >> 12.3.1, XCode 13.4)
> >> >>
> >> >> My problems trying to build and run the latest POCL are more complex,
> >> >> so I'll first mention a problem trying to use POCL acquired with the
> >> >> Conda package manager this week.   I can build and run a simple
> >> >> hello-world program with one kernel and it computes the right answers
> >> >> using POCL.
> >> >> OpenCL Devices Detected:
> >> >> Platform: Portable Computing Language
> >> >> Type: CPU, Name: pthread
> >> >>
> >> >>
> >> >> Unfortunately, when I run my large computational science application,
> >> >> I get a runtime error shortly after the first call to
> >> >> clEnqueueNDRangeKernel().  The function returns CL_SUCCESS, another
> >> >> thread raises SIGABRT, I assume after the following output:
> >> >>
> >> >> ld: dynamic main executables must link with libSystem.dylib for
> >> >> architecture arm64
> >> >> error: linker command failed with exit code 1 (use -v to see
> invocation)
> >> >> Final linking of kernel physical_initial_condition_evaluation failed.
> >> >>
> >> >> Since my hello-world app works, yet the app I care about doesn't,  my
> >> >> leading suspicion so far is some confusion about which clang / llvm
> is
> >> >> getting utilized.  I've tried to be careful about what PATH is set
> and
> >> >> I'm not setting DYLD_LIBRARY_PATH in any way.
> >> >> I'd love to pass "-v" to the POCL link as suggested, but I'm not sure
> >> >> how to do that.
> >> >> The main application *is* linked with -lSystem so that error message
> >> >> is probably referring to the POCL kernel build.
> >> >> It looks like POCL_MSG_PRINT_LLVM could be helpful, but I think that
> >> >> is only set when POCL is built, and I'm not able to build.
> >> >>
> >> >> Ok,  so any suggestions about the above problem would be appreciated!
> >> >>
> >> >>
> >> >> Sometime I'd really like to be able to build POCL myself.
> >> >>
> >> >> I can tell that some special steps are needed to get the latest POCL
> >> >> to build for MacOS.  Who is maintaining that Conda package?   Are the
> >> >> build steps available?
> >> >>
> >> >>    Lesser question stemming from my failed attempts to build POCL
> from source:
> >> >> Question:  What is the difference between the "basic" device and
> >> >> "pthread" device?  I'm not finding documentation for either host
> >> >> devices.
> >> >>    One reason I ask is that I found problems building "pthread" due
> to
> >> >> Apple's pthread.h not including pthread_barrier support.      Looks
> >> >> like two lines in CMakeLists.txt under if(ENABLE_HOST_CPU_DEVICES)
> >> >> could be modified to turn off the pthreads device.   I also had
> >> >> success naively providing  implementations of the missing functions
> in
> >> >> file pthread_scheduler.c
> >> >>
> >> >>
> >> >>
> >> >> Warm regards,
> >> >>
> >> >> Noah
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> pocl-devel mailing list
> >> >> poc...@li...
> >> >> https://lists.sourceforge.net/lists/listinfo/pocl-devel
> >> >
> >> > _______________________________________________
> >> > pocl-devel mailing list
> >> > poc...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/pocl-devel
> >>
> >>
> >> _______________________________________________
> >> pocl-devel mailing list
> >> poc...@li...
> >> https://lists.sourceforge.net/lists/listinfo/pocl-devel
> >
> > _______________________________________________
> > pocl-devel mailing list
> > poc...@li...
> > https://lists.sourceforge.net/lists/listinfo/pocl-devel
>
>
> _______________________________________________
> pocl-devel mailing list
> poc...@li...
> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>

Re: [pocl-devel] MacOS and POCL challenges

[Isuru F. <is...@gm...>]

Should be fixed in https://github.com/conda-forge/pocl-feedstock/pull/77

Isuru

On Fri, May 20, 2022 at 12:20 PM Isuru Fernando <is...@gm...> wrote:

> SDKROOT is not the problem then.
>
> I think you need the `ld` executable found in `/Users/noah/miniforge3/bin`
> in the PATH.
> (That ld is a patched ld to workaround a setting we have added in conda to
> workaround
> issues with SDKROOT)
>
> This can probably be fixed in the conda package, but not sure.
>
> Isuru
>
> On Fri, May 20, 2022 at 11:49 AM Noah Reddell <noa...@gm...>
> wrote:
>
>> I find no difference exporting SDKROOT at build time,  run time, or both.
>>
>> ld: dynamic main executables must link with libSystem.dylib for
>> architecture arm64
>> error: linker command failed with exit code 1 (use -v to see invocation)
>> Final linking of kernel phase_initial_condition_evaluation failed.
>>
>> (base) noah@Spitzer ~ % which xcrun
>> /usr/bin/xcrun
>> (base) noah@Spitzer ~ % xcrun --show-sdk-path
>>
>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
>>
>> When are you supposing SDKROOT in the environment affects behavior?
>>
>> I tried to use dtruss to catch whatever system calls are happening
>> under clEnqueueNDRangeKernel() but that system utility isn't catching
>> anything.  ...MacOS security I think.
>>
>> -Noah
>>
>> On Fri, May 20, 2022 at 9:14 AM Isuru Fernando <is...@gm...> wrote:
>> >
>> > Ah, that makes sense now. Can you try doing the following?
>> >
>> > export SDKROOT=$(xcrun --show-sdk-path)
>> >
>> > (Note that the clang you are using and the one from conda that pocl is
>> using are two different things)
>> >
>> > Isuru
>> >
>> > On Fri, May 20, 2022 at 10:41 AM Noah Reddell <
>> noa...@gm...> wrote:
>> >>
>> >> Hi Isuru,
>> >>     Thank you for maintaining the conda package and the suggestion.
>> >>
>> >> It is true I don't have the conda environment activated when building
>> >> / running my large app.   I installed conda for the sole purpose of
>> >> POCL and generally use homebrew for similar needs.   I'm not seeing
>> >> any difference in build/run of hello-opencl whether I have conda
>> >> activated or not.  SDKROOT is not set either way.
>> >>
>> >> Starting with a fresh shell:
>> >> ----------------------------------------------
>> >> Last login: Wed May 18 22:48:50 on ttys005
>> >> noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
>> >> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
>> >> /Users/noah/Downloads/hello_opencl.c
>> >> noah@Spitzer ~ % ./a.out
>> >> Num Platforms: 1
>> >> platform[0] name: Portable Computing Language
>> >> platform[0] Num Devices: 1
>> >> device[0] name: pthread
>> >> function definition may only have a distinct !dbg attachment
>> >> i64 (i32)* @_Z13get_global_idj
>> >> Computed '1024/1024' correct values!
>> >> noah@Spitzer ~ % echo $SDKROOT
>> >>
>> >> noah@Spitzer ~ % source /Users/noah/miniforge3/etc/profile.d/conda.sh
>> >> noah@Spitzer ~ % conda activate
>> >> (base) noah@Spitzer ~ % echo $SDKROOT
>> >>
>> >> (base) noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
>> >> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
>> >> /Users/noah/Downloads/hello_opencl.c
>> >> (base) noah@Spitzer ~ % ./a.out
>> >> Num Platforms: 1
>> >> platform[0] name: Portable Computing Language
>> >> platform[0] Num Devices: 1
>> >> device[0] name: pthread
>> >> function definition may only have a distinct !dbg attachment
>> >> i64 (i32)* @_Z13get_global_idj
>> >> Computed '1024/1024' correct values!
>> >>
>> >> (base) noah@Spitzer ~ % which clang
>> >> /usr/bin/clang
>> >> ----------------------------------------------
>> >>
>> >>
>> >> I'll check out your build.sh recipe.    At first glance, it is not
>> >> clear to me how you're dealing with the MacOS pthread not having
>> >> pthread_barrier_t and related functions.  When I tried to build POCL
>> >> last week, those pthread parts are needed.
>> >>
>> >> Cheers,
>> >> -Noah
>> >>
>> >>
>> >> On Fri, May 20, 2022 at 7:51 AM Isuru Fernando <is...@gm...>
>> wrote:
>> >> >
>> >> > I think the issue is that you don't have the conda environment
>> activated.
>> >> > Activating the conda environment sets the SDKROOT env variable
>> >> > so that clang can find the macOS SDK.
>> >> >
>> >> > I maintain the pocl conda package with some other maintainers. You
>> >> > can find our build scripts at
>> https://github.com/conda-forge/pocl-feedstock/blob/main/recipe/build.sh
>> >> >
>> >> > Isuru
>> >> >
>> >> > On Fri, May 20, 2022 at 9:38 AM Noah Reddell <
>> noa...@gm...> wrote:
>> >> >>
>> >> >> Hi,
>> >> >>    I upgraded to an M1 Pro Mac and found that MacOS's built-in
>> OpenCL
>> >> >> implementation no longer supports the CPU device.  The GPU is
>> >> >> supported -- for now at least, but doesn't do fp64.
>> >> >>   I'm very motivated to have a local OpenCL platform on my Mac that
>> >> >> supports CPU device.  This is by far the most productive way for me
>> to
>> >> >> first debug and develop kernels versus GPU-based debugging or on
>> >> >> remote systems.   So I return to POCL but for the first time trying
>> to
>> >> >> use it on a Mac.  I'm having struggles.  (latest releases MacOS
>> >> >> 12.3.1, XCode 13.4)
>> >> >>
>> >> >> My problems trying to build and run the latest POCL are more
>> complex,
>> >> >> so I'll first mention a problem trying to use POCL acquired with the
>> >> >> Conda package manager this week.   I can build and run a simple
>> >> >> hello-world program with one kernel and it computes the right
>> answers
>> >> >> using POCL.
>> >> >> OpenCL Devices Detected:
>> >> >> Platform: Portable Computing Language
>> >> >> Type: CPU, Name: pthread
>> >> >>
>> >> >>
>> >> >> Unfortunately, when I run my large computational science
>> application,
>> >> >> I get a runtime error shortly after the first call to
>> >> >> clEnqueueNDRangeKernel().  The function returns CL_SUCCESS, another
>> >> >> thread raises SIGABRT, I assume after the following output:
>> >> >>
>> >> >> ld: dynamic main executables must link with libSystem.dylib for
>> >> >> architecture arm64
>> >> >> error: linker command failed with exit code 1 (use -v to see
>> invocation)
>> >> >> Final linking of kernel physical_initial_condition_evaluation
>> failed.
>> >> >>
>> >> >> Since my hello-world app works, yet the app I care about doesn't,
>> my
>> >> >> leading suspicion so far is some confusion about which clang / llvm
>> is
>> >> >> getting utilized.  I've tried to be careful about what PATH is set
>> and
>> >> >> I'm not setting DYLD_LIBRARY_PATH in any way.
>> >> >> I'd love to pass "-v" to the POCL link as suggested, but I'm not
>> sure
>> >> >> how to do that.
>> >> >> The main application *is* linked with -lSystem so that error message
>> >> >> is probably referring to the POCL kernel build.
>> >> >> It looks like POCL_MSG_PRINT_LLVM could be helpful, but I think that
>> >> >> is only set when POCL is built, and I'm not able to build.
>> >> >>
>> >> >> Ok,  so any suggestions about the above problem would be
>> appreciated!
>> >> >>
>> >> >>
>> >> >> Sometime I'd really like to be able to build POCL myself.
>> >> >>
>> >> >> I can tell that some special steps are needed to get the latest POCL
>> >> >> to build for MacOS.  Who is maintaining that Conda package?   Are
>> the
>> >> >> build steps available?
>> >> >>
>> >> >>    Lesser question stemming from my failed attempts to build POCL
>> from source:
>> >> >> Question:  What is the difference between the "basic" device and
>> >> >> "pthread" device?  I'm not finding documentation for either host
>> >> >> devices.
>> >> >>    One reason I ask is that I found problems building "pthread" due
>> to
>> >> >> Apple's pthread.h not including pthread_barrier support.      Looks
>> >> >> like two lines in CMakeLists.txt under if(ENABLE_HOST_CPU_DEVICES)
>> >> >> could be modified to turn off the pthreads device.   I also had
>> >> >> success naively providing  implementations of the missing functions
>> in
>> >> >> file pthread_scheduler.c
>> >> >>
>> >> >>
>> >> >>
>> >> >> Warm regards,
>> >> >>
>> >> >> Noah
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> pocl-devel mailing list
>> >> >> poc...@li...
>> >> >> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>> >> >
>> >> > _______________________________________________
>> >> > pocl-devel mailing list
>> >> > poc...@li...
>> >> > https://lists.sourceforge.net/lists/listinfo/pocl-devel
>> >>
>> >>
>> >> _______________________________________________
>> >> pocl-devel mailing list
>> >> poc...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>> >
>> > _______________________________________________
>> > pocl-devel mailing list
>> > poc...@li...
>> > https://lists.sourceforge.net/lists/listinfo/pocl-devel
>>
>>
>> _______________________________________________
>> pocl-devel mailing list
>> poc...@li...
>> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>>
>

Re: [pocl-devel] MacOS and POCL challenges

[Noah R. <noa...@gm...>]

Hi Isuru,

   Indeed, modifying PATH by prepending /Users/noah/miniforge3/bin at
runtime resolved this issue.  So the late-binding link at
clEnqueueNDRangeKernel() was calling the wrong ld executable.
    Thank you for identifying this!! :-)  The workaround is
straightforward.  I appreciate you've already worked out a long term
fix!  In general I do not want to pull-in conda into my run
environment.


This will probably turn into a separate thread,   but I note that
about 50% of my runs fail with "clBuildProgram failed. Returned code:
-11".  (CL_BUILD_PROGRAM_FAILURE) This is my big app and this doesn't
happen on other platforms.  The build log contains no clear errors.
The referenced source cache files are not present by the time
clBuildProgram returns.

 Build-log ( 698 bytes):
warning: /Users/noah/.cache/pocl/kcache/tempfile_i2Ph3x.cl:2401:2: TODO:
warning: /Users/noah/.cache/pocl/kcache/tempfile_i2Ph3x.cl:2413:2:
review the below TODO
warning: /Users/noah/.cache/pocl/kcache/tempfile_i2Ph3x.cl:2401:2: TODO:
warning: /Users/noah/.cache/pocl/kcache/tempfile_i2Ph3x.cl:2413:2:
review the below TODO

Device pthread failed to build the program, log: warning:
/Users/noah/.cache/pocl/kcache/tempfile_i2Ph3x.cl:2401:2: TODO:
warning: /Users/noah/.cache/pocl/kcache/tempfile_i2Ph3x.cl:2413:2:
review the below TODO
warning: /Users/noah/.cache/pocl/kcache/tempfile_i2Ph3x.cl:2401:2: TODO:
warning: /Users/noah/.cache/pocl/kcache/tempfile_i2Ph3x.cl:2413:2:
review the below TODO



The above output was when running under lldb debugger.  Extra weird
the output is different when not under the debugger.

clBuildProgram failed. Returned code: -11
 Build-log ( 56 bytes):
 Device pthread failed to build the program, log: (null)


And remember,  half of the time, everything works correctly.  I see
this output from the kernel build instead:
1 warning generated.
2 warnings generated.


Feels like a race condition.   I don't think I'm calling
clBuildProgram in parallel, but I might be.  That should be allowable
though, I think.  In total, 12 clBuildPrograms are called over 2-3
seconds.

Warm regards,
-Noah

On Fri, May 20, 2022 at 10:39 AM Isuru Fernando <is...@gm...> wrote:
>
> Should be fixed in https://github.com/conda-forge/pocl-feedstock/pull/77
>
> Isuru
>
> On Fri, May 20, 2022 at 12:20 PM Isuru Fernando <is...@gm...> wrote:
>>
>> SDKROOT is not the problem then.
>>
>> I think you need the `ld` executable found in `/Users/noah/miniforge3/bin` in the PATH.
>> (That ld is a patched ld to workaround a setting we have added in conda to workaround
>> issues with SDKROOT)
>>
>> This can probably be fixed in the conda package, but not sure.
>>
>> Isuru
>>
>> On Fri, May 20, 2022 at 11:49 AM Noah Reddell <noa...@gm...> wrote:
>>>
>>> I find no difference exporting SDKROOT at build time,  run time, or both.
>>>
>>> ld: dynamic main executables must link with libSystem.dylib for
>>> architecture arm64
>>> error: linker command failed with exit code 1 (use -v to see invocation)
>>> Final linking of kernel phase_initial_condition_evaluation failed.
>>>
>>> (base) noah@Spitzer ~ % which xcrun
>>> /usr/bin/xcrun
>>> (base) noah@Spitzer ~ % xcrun --show-sdk-path
>>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
>>>
>>> When are you supposing SDKROOT in the environment affects behavior?
>>>
>>> I tried to use dtruss to catch whatever system calls are happening
>>> under clEnqueueNDRangeKernel() but that system utility isn't catching
>>> anything.  ...MacOS security I think.
>>>
>>> -Noah
>>>
>>> On Fri, May 20, 2022 at 9:14 AM Isuru Fernando <is...@gm...> wrote:
>>> >
>>> > Ah, that makes sense now. Can you try doing the following?
>>> >
>>> > export SDKROOT=$(xcrun --show-sdk-path)
>>> >
>>> > (Note that the clang you are using and the one from conda that pocl is using are two different things)
>>> >
>>> > Isuru
>>> >
>>> > On Fri, May 20, 2022 at 10:41 AM Noah Reddell <noa...@gm...> wrote:
>>> >>
>>> >> Hi Isuru,
>>> >>     Thank you for maintaining the conda package and the suggestion.
>>> >>
>>> >> It is true I don't have the conda environment activated when building
>>> >> / running my large app.   I installed conda for the sole purpose of
>>> >> POCL and generally use homebrew for similar needs.   I'm not seeing
>>> >> any difference in build/run of hello-opencl whether I have conda
>>> >> activated or not.  SDKROOT is not set either way.
>>> >>
>>> >> Starting with a fresh shell:
>>> >> ----------------------------------------------
>>> >> Last login: Wed May 18 22:48:50 on ttys005
>>> >> noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
>>> >> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
>>> >> /Users/noah/Downloads/hello_opencl.c
>>> >> noah@Spitzer ~ % ./a.out
>>> >> Num Platforms: 1
>>> >> platform[0] name: Portable Computing Language
>>> >> platform[0] Num Devices: 1
>>> >> device[0] name: pthread
>>> >> function definition may only have a distinct !dbg attachment
>>> >> i64 (i32)* @_Z13get_global_idj
>>> >> Computed '1024/1024' correct values!
>>> >> noah@Spitzer ~ % echo $SDKROOT
>>> >>
>>> >> noah@Spitzer ~ % source /Users/noah/miniforge3/etc/profile.d/conda.sh
>>> >> noah@Spitzer ~ % conda activate
>>> >> (base) noah@Spitzer ~ % echo $SDKROOT
>>> >>
>>> >> (base) noah@Spitzer ~ % clang -g -L/Users/noah/miniforge3/lib/
>>> >> -Wl,-rpath,/Users/noah/miniforge3/lib/  -lOpenCL
>>> >> /Users/noah/Downloads/hello_opencl.c
>>> >> (base) noah@Spitzer ~ % ./a.out
>>> >> Num Platforms: 1
>>> >> platform[0] name: Portable Computing Language
>>> >> platform[0] Num Devices: 1
>>> >> device[0] name: pthread
>>> >> function definition may only have a distinct !dbg attachment
>>> >> i64 (i32)* @_Z13get_global_idj
>>> >> Computed '1024/1024' correct values!
>>> >>
>>> >> (base) noah@Spitzer ~ % which clang
>>> >> /usr/bin/clang
>>> >> ----------------------------------------------
>>> >>
>>> >>
>>> >> I'll check out your build.sh recipe.    At first glance, it is not
>>> >> clear to me how you're dealing with the MacOS pthread not having
>>> >> pthread_barrier_t and related functions.  When I tried to build POCL
>>> >> last week, those pthread parts are needed.
>>> >>
>>> >> Cheers,
>>> >> -Noah
>>> >>
>>> >>
>>> >> On Fri, May 20, 2022 at 7:51 AM Isuru Fernando <is...@gm...> wrote:
>>> >> >
>>> >> > I think the issue is that you don't have the conda environment activated.
>>> >> > Activating the conda environment sets the SDKROOT env variable
>>> >> > so that clang can find the macOS SDK.
>>> >> >
>>> >> > I maintain the pocl conda package with some other maintainers. You
>>> >> > can find our build scripts at https://github.com/conda-forge/pocl-feedstock/blob/main/recipe/build.sh
>>> >> >
>>> >> > Isuru
>>> >> >
>>> >> > On Fri, May 20, 2022 at 9:38 AM Noah Reddell <noa...@gm...> wrote:
>>> >> >>
>>> >> >> Hi,
>>> >> >>    I upgraded to an M1 Pro Mac and found that MacOS's built-in OpenCL
>>> >> >> implementation no longer supports the CPU device.  The GPU is
>>> >> >> supported -- for now at least, but doesn't do fp64.
>>> >> >>   I'm very motivated to have a local OpenCL platform on my Mac that
>>> >> >> supports CPU device.  This is by far the most productive way for me to
>>> >> >> first debug and develop kernels versus GPU-based debugging or on
>>> >> >> remote systems.   So I return to POCL but for the first time trying to
>>> >> >> use it on a Mac.  I'm having struggles.  (latest releases MacOS
>>> >> >> 12.3.1, XCode 13.4)
>>> >> >>
>>> >> >> My problems trying to build and run the latest POCL are more complex,
>>> >> >> so I'll first mention a problem trying to use POCL acquired with the
>>> >> >> Conda package manager this week.   I can build and run a simple
>>> >> >> hello-world program with one kernel and it computes the right answers
>>> >> >> using POCL.
>>> >> >> OpenCL Devices Detected:
>>> >> >> Platform: Portable Computing Language
>>> >> >> Type: CPU, Name: pthread
>>> >> >>
>>> >> >>
>>> >> >> Unfortunately, when I run my large computational science application,
>>> >> >> I get a runtime error shortly after the first call to
>>> >> >> clEnqueueNDRangeKernel().  The function returns CL_SUCCESS, another
>>> >> >> thread raises SIGABRT, I assume after the following output:
>>> >> >>
>>> >> >> ld: dynamic main executables must link with libSystem.dylib for
>>> >> >> architecture arm64
>>> >> >> error: linker command failed with exit code 1 (use -v to see invocation)
>>> >> >> Final linking of kernel physical_initial_condition_evaluation failed.
>>> >> >>
>>> >> >> Since my hello-world app works, yet the app I care about doesn't,  my
>>> >> >> leading suspicion so far is some confusion about which clang / llvm is
>>> >> >> getting utilized.  I've tried to be careful about what PATH is set and
>>> >> >> I'm not setting DYLD_LIBRARY_PATH in any way.
>>> >> >> I'd love to pass "-v" to the POCL link as suggested, but I'm not sure
>>> >> >> how to do that.
>>> >> >> The main application *is* linked with -lSystem so that error message
>>> >> >> is probably referring to the POCL kernel build.
>>> >> >> It looks like POCL_MSG_PRINT_LLVM could be helpful, but I think that
>>> >> >> is only set when POCL is built, and I'm not able to build.
>>> >> >>
>>> >> >> Ok,  so any suggestions about the above problem would be appreciated!
>>> >> >>
>>> >> >>
>>> >> >> Sometime I'd really like to be able to build POCL myself.
>>> >> >>
>>> >> >> I can tell that some special steps are needed to get the latest POCL
>>> >> >> to build for MacOS.  Who is maintaining that Conda package?   Are the
>>> >> >> build steps available?
>>> >> >>
>>> >> >>    Lesser question stemming from my failed attempts to build POCL from source:
>>> >> >> Question:  What is the difference between the "basic" device and
>>> >> >> "pthread" device?  I'm not finding documentation for either host
>>> >> >> devices.
>>> >> >>    One reason I ask is that I found problems building "pthread" due to
>>> >> >> Apple's pthread.h not including pthread_barrier support.      Looks
>>> >> >> like two lines in CMakeLists.txt under if(ENABLE_HOST_CPU_DEVICES)
>>> >> >> could be modified to turn off the pthreads device.   I also had
>>> >> >> success naively providing  implementations of the missing functions in
>>> >> >> file pthread_scheduler.c
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> Warm regards,
>>> >> >>
>>> >> >> Noah
>>> >> >>
>>> >> >>
>>> >> >> _______________________________________________
>>> >> >> pocl-devel mailing list
>>> >> >> poc...@li...
>>> >> >> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>>> >> >
>>> >> > _______________________________________________
>>> >> > pocl-devel mailing list
>>> >> > poc...@li...
>>> >> > https://lists.sourceforge.net/lists/listinfo/pocl-devel
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> pocl-devel mailing list
>>> >> poc...@li...
>>> >> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>>> >
>>> > _______________________________________________
>>> > pocl-devel mailing list
>>> > poc...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/pocl-devel
>>>
>>>
>>> _______________________________________________
>>> pocl-devel mailing list
>>> poc...@li...
>>> https://lists.sourceforge.net/lists/listinfo/pocl-devel
>
> _______________________________________________
> pocl-devel mailing list
> poc...@li...
> https://lists.sourceforge.net/lists/listinfo/pocl-devel

[pocl-devel] POCL global memory size limits

[Noah R. <noa...@gm...>]

Hi,

   I'm wondering why the POCL memory limits are so small relative to
my machine RAM and other OpenCL implementations.  Can I increase these
device limits?
    I'm running POCL 3.0 from conda-forge on an M1 Pro Mac with 64GB
RAM.   Install and self-reported properties:
pocl                      3.0                  h422b2f1_0    conda-forge

CL_PLATFORM_PROFILE: FULL_PROFILE
CL_PLATFORM_VERSION: OpenCL 3.0 PoCL 3.0-rc2  Apple, Release, RELOC,
LLVM 14.0.4, SLEEF, FP16, POCL_DEBUG
CL_DEVICE_NAME: pthread
CL_DEVICE_TYPE: CPU


POCL Reporting from clGetDeviceInfo:
(app built with -DCL_TARGET_OPENCL_VERSION=120)

CL_DEVICE_GLOBAL_MEM_SIZE: 2,530,467,840 bytes.     (~2 GB)
CL_DEVICE_MAX_MEM_ALLOC_SIZE: 1,073,741,824 bytes.   (~1 GB)

Apple's OpenCL implementation is reporting much more availability
total and per-buffer.
CL_DEVICE_GLOBAL_MEM_SIZE: 51,539,607,552 bytes.  (~51 GB)
CL_DEVICE_MAX_MEM_ALLOC_SIZE: 9,663,676,416 bytes.  (~9 GB)

The observed POCL memory limits are hampering the type of simulations
I am able to run.  I've seen other platforms limit to 4GB allocations
before, but not in a while.

Warm regards,

Noah

Re: [pocl-devel] POCL global memory size limits

[Pekka J. <pek...@tu...>]

Hi Noah,

This is likely due to not having proper detection implemented
for Apple. Normally we use hwloc for detecting the mem. Is it
not available for you or does it report the RAM wrongly?

The code is around here if you are interested in investigating and 
perhaps contributing a fix:

https://github.com/pocl/pocl/blob/master/lib/CL/devices/topology/pocl_topology.c#L121

HTH,
Pekka

On 12.7.2022 20.54, Noah Reddell wrote:
> Hi,
> 
>     I'm wondering why the POCL memory limits are so small relative to
> my machine RAM and other OpenCL implementations.  Can I increase these
> device limits?
>      I'm running POCL 3.0 from conda-forge on an M1 Pro Mac with 64GB
> RAM.   Install and self-reported properties:
> pocl                      3.0                  h422b2f1_0    conda-forge
> 
> CL_PLATFORM_PROFILE: FULL_PROFILE
> CL_PLATFORM_VERSION: OpenCL 3.0 PoCL 3.0-rc2  Apple, Release, RELOC,
> LLVM 14.0.4, SLEEF, FP16, POCL_DEBUG
> CL_DEVICE_NAME: pthread
> CL_DEVICE_TYPE: CPU
> 
> 
> POCL Reporting from clGetDeviceInfo:
> (app built with -DCL_TARGET_OPENCL_VERSION=120)
> 
> CL_DEVICE_GLOBAL_MEM_SIZE: 2,530,467,840 bytes.     (~2 GB)
> CL_DEVICE_MAX_MEM_ALLOC_SIZE: 1,073,741,824 bytes.   (~1 GB)
> 
> Apple's OpenCL implementation is reporting much more availability
> total and per-buffer.
> CL_DEVICE_GLOBAL_MEM_SIZE: 51,539,607,552 bytes.  (~51 GB)
> CL_DEVICE_MAX_MEM_ALLOC_SIZE: 9,663,676,416 bytes.  (~9 GB)
> 
> The observed POCL memory limits are hampering the type of simulations
> I am able to run.  I've seen other platforms limit to 4GB allocations
> before, but not in a while.
> 
> Warm regards,
> 
> Noah
> 
> 
> _______________________________________________
> pocl-devel mailing list
> poc...@li...
> https://lists.sourceforge.net/lists/listinfo/pocl-devel