#92 Thread -safe fourier and cosine transforms

Next_Release
closed
None
1
2013-05-24
2012-12-20
andy_panov
No

Please find attached patch with thread-safe implementation of fourier-cosine transforms.

I've tested it with FFTW and ACML. It still requires testing with MKL (I do not have this library available, but made some investigations regarding the transforms implementation and tried to compile with the MKL headers).

PS I guess, following should be mentioned in ITPP documentation:
Both AMD and Intel recommend to use single-threaded math libraries with omp-enabled code. So, if user wants to control multi-threading he/she use single-threading versions of these libraries. Multi-threaded versions of the libraries should be used to improve the performance of single-threaded ITPP builds.

1 Attachments

Discussion

1 2 > >> (Page 1 of 2)
  • andy_panov

    andy_panov - 2012-12-20

    Oops, forgot to include most recent changes...

     
  • Bogdan Cristea

    Bogdan Cristea - 2012-12-23
    • status: open --> accepted
    • assigned_to: Bogdan Cristea
     
  • andy_panov

    andy_panov - 2012-12-27

    Sorry for the late response. I was away for awhile and could not look into the sources.

    Please find attached file with review comments. I have not attached the modified sources since I need some further feedback from you.

     
  • Bogdan Cristea

    Bogdan Cristea - 2012-12-29

    Hi

    No problem for the late response and thank you for your contribution to IT++.

    Below are my answers to review2.txt:
    4. using have_fourier_transform function and related seems to be redundant since all FFT functions have a different definition when a FFT library is not available (see for example void idct(const vec &in, vec &out))
    5. I would prefer a more clean code here, i.e. all tests should be executed serially when no openmp support is available instead of separately define tests for serial and parallel execution
    8. ok for the ACML version, I'll test on my side with the 64 bits version
    9. see 4.
    15. I prefer here to use templated types for InType and OutType in order to easily maintain and expand the code. Another possible solution would be to use typelists:

    template <class T, class U>
    struct TypeList {
      typedef vec<T> InType;
      typedef vec<U> OutType;
    };
    

    then provide as template parameter for the Transform class the correct instantiation of the typelist. Maybe the first template parameter (TransformType) could be replaced by the typelist. Please choose the best approach that suits you.

    You could add me in the authors list is you feel that your code has been significantly changed by my observations, but I let this entirely at your discretion.

    regards
    Bogdan

     
    Last edit: Bogdan Cristea 2012-12-29
  • andy_panov

    andy_panov - 2013-01-07

    Hi Bogdan,

    Here is the patch reworked based on review comments:
    -All tests are executed in multithreaded context if OMP is available
    -Transfrom/Locked_Transfrom classes reworked with type lists (traits)

    I decided to leave have_fourier_transform() function in transforms header. Here are the reasons behind this decision:
    all transform functions are replaced with stubs if FFT library is not available. These stubs use it_error macro. This macro can generate exception or just abort execution of test program depending on the library build options. Abort is default for Windows builds.
    As far as I understand, it is highly desirable to continue the tests execution even if library is built without FFT support, so we should know in advance (before calling transfrom functions) if transforms are available in the current build. have_fourier_transfrom()-type functions is the best option from my point of view since -
    1. it does not affect existing users code.
    2. config.h defines are not exposed in itpp headers. Library users do not have to learn them and defines can be changed without any problems in future versions.

    I do not insist on this solution but I can not come up with better one so far.

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-01-12

    comited patch in rev 1922. Andy, please update your sources and retest on your side

     
  • andy_panov

    andy_panov - 2013-01-14

    Done. Gtests passed successfully with ACML and prebuilt LAPACK-FFTW3 on my machine. Thank you for the corrections!

    Old MSVC ACML test projects are broken for a long time (MSVC Express 2010), so I fixed them locally and run old tests with ACML and prebuilt LAPACK-FFTW3. I found no regression (all passed).

    Here is the random_core_test.cpp with minor fix (there was a warning on MSVC regarding unknown pragma)

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-04-07

    Hi Andy
    I have just committed changes for detecting MKL11. However, it seems that FFT does not work anymore (Transforms test fail on Linux, while on Windows Channel test fail with an error in transforms.cpp). Please have a look. In order to use MKL 11 you need first to point PATH environment variable to MKL ddls folder (I have tested on Win x86_64).
    regards
    Bogdan

     
  • andy_panov

    andy_panov - 2013-04-07

    Hi Bogdan,
    I'll install a trial version of the MKL and look into it.
    Andy

     
  • andy_panov

    andy_panov - 2013-04-07

    Hi Bogdan,

    Here is my status so far:
    configure fails for Intel MKL 11 on 32-bit Windows:

    The C compiler identification is MSVC 16.0.40219.1
    The CXX compiler identification is MSVC 16.0.40219.1
    Check for working C compiler using: Visual Studio 10
    Check for working C compiler using: Visual Studio 10 -- works
    Detecting C compiler ABI info
    Detecting C compiler ABI info - done
    Check for working CXX compiler using: Visual Studio 10
    Check for working CXX compiler using: Visual Studio 10 -- works
    Detecting CXX compiler ABI info
    Detecting CXX compiler ABI info - done
    A library with BLAS API found.
    A library with BLAS API found.
    A library with LAPACK API found.
    A library with BLAS API found.
    CMake Warning at CMakeLists.txt:191 (message):
    FFT library not found.

    CMake Warning at CMakeLists.txt:195 (message):
    You can still compile IT++ but the functionality will be reduced.

    Configuring done

    with BLA_VENDOR = Intel11

    It seems, cmake can not find DftiCommitDescriptor inside the mkl_rt.lib (naming conventions are different ????). I'll investigate it further.

    Andy.

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-04-08

    You could comment that test in FindFFT.cmake and see if IT++ compiles, but it is strange since IT++ uses DftiCommitDescriptor

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-04-08

    Could you try instead with mkl_intel_c.lib ?

     
  • andy_panov

    andy_panov - 2013-04-08

    Hi Bogdan,

    I've found a bug in transforms.cpp. Here is the fixed version.

    PS I still was unable to make cmake to detect MKL FFT libs on windows 32-bit with MKL 11 Trial. I had to manually force library detection.

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-04-08

    Hi Andy
    Thanks for your quick reply. What dll have you used for linking ? I would use mkl_rt.dll as this seems to be the easiest way.
    Could you tell me your development platform ? OS, cmake version ? I would like to try on a VM.
    regards
    Bogdan

     
  • andy_panov

    andy_panov - 2013-04-08

    Hi Bogdan

    I used mkl_rt.dll and the latest version of cmake (2.8.10.2). I am running 32 bit Windows 7 Home Basic with SP1 installed.

    Andy

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-04-09

    Hi Andy
    I have checked your patch and everything seems fine. In your system configuration MKL library is detected, but you need to specify the path to mkl_rt.lib (in System Properties). Sorry it is my mistake. I have updated accordingly the installation instr.
    regards
    Bogdan

     
  • andy_panov

    andy_panov - 2013-04-09

    Hi Bogdan

    Detection worked well for me with following modifications (please see attached file). Otherwise cl.exe inside the check_function_exists() complains about unknown library (FFT_LIBRARIES-NOTFOUND) and fails to check if DftiCommitDescriptor exists.

    Hope this helps.
    Andy

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-04-09

    Hi Andy
    I am a little puzzled about your patch for FindFFT.cmake because when ${LIBRARIES} is empty it shouldn't make any difference in adding or not to the list. Also, on my side, the behavior you report cannot be reproduced.
    regards
    Bogdan

     
    • andy_panov

      andy_panov - 2013-04-09

      So am I. When I tried to debug the thing with message(), I noticed that ${${LIBRARIES}} expands to "FFT_LIBRARIES-NOTFOUND" after set(${LIBRARIES}).
      I also expected that ${LIBRARIES} would become empty. I cleared the cache in the build folder first and run cmake after that.

       
      Last edit: andy_panov 2013-04-09
  • andy_panov

    andy_panov - 2013-04-09

    Following example illustrates my point. Just extract attached file to some folder and run cmake. Following is the result of execution on my machine:

    D:\Development\itpp\cmake_test>call C:\Portable\cmake-2.8.10.2-win32-x86\bin\cmake .
    TEST_FILE=TEST_FILE-NOTFOUND
    TEST_FILE=TEST_FILE-NOTFOUND
    bb=10
    bb=
    -- Configuring done
    error opening key: Software\Microsoft\VisualStudio\10.0\vsmacros\OtherProjects7

    error opening key: Software\Microsoft\VisualStudio\10.0\vsmacros\RecordingProject7

    error opening key: Software\Microsoft\VisualStudio\10.0\vsmacros\OtherProjects7

    -- Generating done
    error opening key: Software\Microsoft\VisualStudio\10.0\vsmacros\OtherProjects7

    error opening key: Software\Microsoft\VisualStudio\10.0\vsmacros\RecordingProject7

    -- Build files have been written to: D:/Development/itpp/cmake_test

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-04-10

    Hi Andy
    I have corrected FindFFT.cmake, where it was indeed a bug. Please update your repository and let be know if you still have problems.
    regards
    Bogdan

     
  • andy_panov

    andy_panov - 2013-04-10

    Hi Bogdan,

    I've just tried to run cmake on the empty build folder and FFT library was detected successfully. Thank you very much!

    Could you please add verbose notification that library is found as it is done in Find_LAPCK/Find_BLAS?

    ~~~~~~~~~~~~~~~
    if(NOT LAPACK_FIND_QUIETLY)
    if(LAPACK_FOUND)
    message(STATUS "A library with LAPACK API found.")
    else(LAPACK_FOUND)
    if(LAPACK_FIND_REQUIRED)
    message(FATAL_ERROR
    "A required library with LAPACK API not found. Please specify library location."
    )
    else(LAPACK_FIND_REQUIRED)
    message(STATUS
    "A library with LAPACK API not found. Please specify library location."
    )
    endif(LAPACK_FIND_REQUIRED)
    endif(LAPACK_FOUND)
    endif(NOT LAPACK_FIND_QUIETLY)
    ~~~~~~~~~~~~~~~~~~

    It would be easier to verify the install.

    Regards,
    Andy

     
  • Bogdan Cristea

    Bogdan Cristea - 2013-04-10

    done, I'll port the rest of the unit tests to Google framework and after this I think we are ready for a new release.

     
1 2 > >> (Page 1 of 2)

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks