Hello there,
at first i want to thank you for your great work on ViennaCL! I am trying to implement your library to speed up some calculations within the simulation framework i am developing. In need to be able to solve sparse linear systems with around 1.000.000 unknowns or more.
In my system i am working with stl based sparse matrices. An have ViennaCL with OpenCL enabled an link the NVIDIA OpenCL.lib.
When i am simulating small Systems everything works perfectly. Bun at around 100.000 unknowns the System crashes throwing " viennacl::ocl::out_of_resources an Speicherposition 0x0040bca0.." when its running on GPU the display driver also crashes.
This error also happens when using CPU (OpenCL) or pure CPU (OpenMP). I am using Visual Studio 2010 on Windows 7 64Bit.
Is there something wrong in my implementation or are there some size limitations coming with the library?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
the out-of-resources-error should not occur at such small sizes. Which GPU do you have in your system? My only explanation is that you have an integrated laptop GPU with small GPU RAM. How many nonzeros do you have per row in your sparse matrix?
However, the error should not show up with OpenMP. It suggests that OpenCL is still in use. Can you recompile the code without VIENNACL_WITH_OPENCL being defined in order to make sure that OpenMP is used?
Best regards,
Karli
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2013-04-12
Hi Karl,
thanks for your fast answer. I was also very curios about this and i am pretty sure i did something wrong on my side. Just don't have an idea what it may be. I did recompile with the OPENMP define only and still, the crash occurred at around the same size ( I am sorry it wasn't at 100.000 unknowns, lowest value i got it to crash with was 203401) >The GPU in my current system is a Quadro FX 580 with 512MB ran also tested on a nvs 4200m with 1GB of memory. I know its not the best setup at the moment… I am pretty sure it uses the right GPU since on both system the graphic card driver crashes in the process.
Nonzeros per row in my system is maximum of 26 minimum 1. I used BICStab and CG. On Bicstab the crash occurred earlier, on CG i can calculate systems a bit larger. Its alway the inner_prod function.
Now, using OpenMP only the crash ocures at a other position: sparse_matrix_operations.hpp at line 126 result_buf = dot_prod; Memory Access violation..
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2013-04-12
Hello Karl,
i am sorry! i knew it was my mistake, now i can also explain why!
The matrix was in a bad condition (some all 0 rows added mistakenly on the bottom). However, sorry for wasting time.
On smaller systems this wasn't an issue, even with this mistake in my code, i got good results. But when the systems grew larger this causes the unexplained crash.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ok, the Quadro with 512 MB might run short on GPU memory because the system puts a lot of stuff in there already. A 4200m might suffer from the same problems, since it shares memory with main RAM (if I'm not mistaken). Did you check the RAM consumption during the run?
BiCGStab crashing earlier than CG is consistent, because it has slightly higher memory requirements. The inner_prod() kernel uses shared memory and uses a small additional buffer for the reduction, so this is also consistent with running out of memory.
Does the OpenMP case crash at the same system sizes as OpenCL? The error in sparse_matrix_operations.hpp occurs during the CSR sparse matrix-vector product, which is a fairly trivial kernel and usually breaks only if the data in the row/column buffers is corrupted. Did you use viennacl::copy() for the setup, or did you use the .set() member function in the compressed_matrix?
Best regards,
Karli
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ok, never mind, this explains the issue then. Anyway, ViennaCL should ideally report the matrix to be singular rather than leading to such issues. I'll improve the code to get better error reports :-)
Thanks and best regards,
Karli
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello there,
at first i want to thank you for your great work on ViennaCL! I am trying to implement your library to speed up some calculations within the simulation framework i am developing. In need to be able to solve sparse linear systems with around 1.000.000 unknowns or more.
In my system i am working with stl based sparse matrices. An have ViennaCL with OpenCL enabled an link the NVIDIA OpenCL.lib.
When i am simulating small Systems everything works perfectly. Bun at around 100.000 unknowns the System crashes throwing " viennacl::ocl::out_of_resources an Speicherposition 0x0040bca0.." when its running on GPU the display driver also crashes.
This error also happens when using CPU (OpenCL) or pure CPU (OpenMP). I am using Visual Studio 2010 on Windows 7 64Bit.
Is there something wrong in my implementation or are there some size limitations coming with the library?
Hi,
the out-of-resources-error should not occur at such small sizes. Which GPU do you have in your system? My only explanation is that you have an integrated laptop GPU with small GPU RAM. How many nonzeros do you have per row in your sparse matrix?
However, the error should not show up with OpenMP. It suggests that OpenCL is still in use. Can you recompile the code without VIENNACL_WITH_OPENCL being defined in order to make sure that OpenMP is used?
Best regards,
Karli
Hi Karl,
thanks for your fast answer. I was also very curios about this and i am pretty sure i did something wrong on my side. Just don't have an idea what it may be. I did recompile with the OPENMP define only and still, the crash occurred at around the same size ( I am sorry it wasn't at 100.000 unknowns, lowest value i got it to crash with was 203401) >The GPU in my current system is a Quadro FX 580 with 512MB ran also tested on a nvs 4200m with 1GB of memory. I know its not the best setup at the moment… I am pretty sure it uses the right GPU since on both system the graphic card driver crashes in the process.
Nonzeros per row in my system is maximum of 26 minimum 1. I used BICStab and CG. On Bicstab the crash occurred earlier, on CG i can calculate systems a bit larger. Its alway the inner_prod function.
Now, using OpenMP only the crash ocures at a other position: sparse_matrix_operations.hpp at line 126 result_buf = dot_prod; Memory Access violation..
Hello Karl,
i am sorry! i knew it was my mistake, now i can also explain why!
The matrix was in a bad condition (some all 0 rows added mistakenly on the bottom). However, sorry for wasting time.
On smaller systems this wasn't an issue, even with this mistake in my code, i got good results. But when the systems grew larger this causes the unexplained crash.
Hey,
ok, the Quadro with 512 MB might run short on GPU memory because the system puts a lot of stuff in there already. A 4200m might suffer from the same problems, since it shares memory with main RAM (if I'm not mistaken). Did you check the RAM consumption during the run?
BiCGStab crashing earlier than CG is consistent, because it has slightly higher memory requirements. The inner_prod() kernel uses shared memory and uses a small additional buffer for the reduction, so this is also consistent with running out of memory.
Does the OpenMP case crash at the same system sizes as OpenCL? The error in sparse_matrix_operations.hpp occurs during the CSR sparse matrix-vector product, which is a fairly trivial kernel and usually breaks only if the data in the row/column buffers is corrupted. Did you use viennacl::copy() for the setup, or did you use the .set() member function in the compressed_matrix?
Best regards,
Karli
ok, never mind, this explains the issue then. Anyway, ViennaCL should ideally report the matrix to be singular rather than leading to such issues. I'll improve the code to get better error reports :-)
Thanks and best regards,
Karli