From: Laurent El S. <lau...@id...> - 2012-04-25 18:44:21
|
Hi Patrik, Very good guess! After applying the proposed change, the program correctly works. The compiler used is the stock version of gcc shipped with Ubuntu 12.04 beta: user@ubuntu32-12-04:~/bug_blitz$ gcc --version gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 Thanks and cheers, Laurent On 4/25/2012, "Patrik Jonsson" <co...@fa...> wrote: >Hmm. There's type aliasing being done there, maybe that's what's >making the compiler unhappy? > >If you change that line to "dataBlockAddress_ =new P_type[length];" >and the deallocation on line 64 to delete datablockAddress_ instead, >does that change something? > >cheers, > >/Patrik > >On Wed, Apr 25, 2012 at 7:13 AM, Laurent El Shafey ><lau...@id...> wrote: >> Hello, >> >> I've tried to understand a bit more the problem. It seems that this is >> related to the way the memory is allocated by blitz: >> - When I allocate the memory myself, and attach the blitz arrays to the >> memory blocks, then everything works fine. >> - If blitz allocates the memory of the arrays, then I'm getting the >> segmentation fault in the LAPACK function. >> >> Memory allocation is performed by this function in blitz/memblock.cc: >> template<typename P_type> >> inline void MemoryBlock<P_type>::allocate(sizeType length) >> >> I've recorded the variables values before the allocation using gdb, for >> one of the problematic 2D 3x3 double array. This was done on Ubuntu >> 12.04 beta 32 bits (I guess this affect any linux 32 bits distribution): >> length=9 >> vecWidth=1 >> byteWidth=8 >> cacheLineSize=64 >> minLengthToAlign=1024 >> numBytes=72 >> The only difference I see is here: >> 112 // no shifting to cache line >> 113 dBA_tv_ = >> 114 new typename simdTypes<P_type>::vecType[length/vecWidth]; >> 115 data_= dataBlockAddress_; >> >> It seems that this way of allocating the memory is related to my problem: >> new double[9]; vs new typename simdTypes<P_type>::vecType[72]; (with >> P_type=double) >> >> vecType being defined by: >> typedef TinyVector<T, vecWidth> vecType; >> >> Any clue? >> >> Thanks and cheers, >> Laurent >> >> On 4/20/2012, "Laurent El Shafey" <lau...@id...> wrote: >> >>>Hi Patrik, >>> >>>Thanks a lot for your answer. I've just tried your propositions to >>>determine the cause of the problem. >>>Strangely, if I allocate the C-style arrays with new, and make the blitz >>>arrays to point to these memory blocks, then everything is working fine. >>>Please find the detailed behaviours below. >>> >>>Cheers, >>>Laurent >>> >>>On 4/19/2012, "Patrik Jonsson" <co...@fa...> wrote: >>> >>>>Hi Laurent, >>>> >>>>That's weird, it should work fine. Can you confirm that >>>>isStorageContiguous() returns true on those arrays? And that >>>>dataFirst() ==data()? >>> >>>I confirm that both isStorageContiguous() and (dataFirst()==data()) are >>>true for all the blitz arrays. >>> >>>> >>>>Alignment shouldn't be an issue if you haven't specified a simd width >>>>since then blitz does exactly what you do when allocating. >>>> >>>>Couple of more things to try: >>>>* If you allocate your C-style arrays and then use the "preexisting >>>>memory" Array constructor to make Arrays that point to that memory, >>>>does it still crash? >>>Everything works fine in this case. >>> >>>>* If you make the blitz Arrays much longer (but still tell LAPACK that >>>>it's only 3x3 elements) does it still crash? >>>Everything works fine in this case. >>> >>>>* Does it crash if you use any one of the blitz arrays and the rest C arrays? >>>The problem is only with the three 2D blitz arrays. Using the memory >>>block of the 1D array works just fine. If for only one of the three 2D >>>arrays, I try to use the memory block, then it crashes. >>> >>>>* If you manually print out A.data()[0...8] do you get the values you >>>>initialized with? >>>All the values are printed correctly, and the program crashes after that >>>when calling the LAPACK function. >>> >>>>* Did you configure with --enable-64bit on 64-bit? Though that >>>>shouldn't matter unless you have huge arrays. >>>I've realized that I was always setting --enable-64bit for both 32 bits >>>and 64 bits platforms. I've disabled it for 32 bits, but still get the >>>same behaviour for all the operations above. >>> >>>> >>>>There's no obvious reason this shouldn't work, so I think you'll have >>>>to poke around a bit to see if you can figure out exactly under what >>>>circumstances it happens. >>>> >>>>Regards, >>>> >>>>/Patrik >>>> >>>> >>>>On Thu, Apr 19, 2012 at 3:59 PM, Laurent El Shafey >>>><lau...@id...> wrote: >>>>> Hello, >>>>> >>>>> I'm experiencing the following problem when interfacing Blitz++ arrays >>>>> with a LAPACK function (dgesdd). >>>>> To sum up, I'm getting a segmentation fault when using the C-pointers to >>>>> the memory blocks of the 2D blitz::Array (which are supposed by >>>>> construction to be contiguous in memory), whereas everything works fine >>>>> when I'm directly using C-style arrays of the same size allocated with >>>>> the new operator. >>>>> >>>>> The problem has been found while developing this library: >>>>> http://idiap.github.com/bob/. >>>>> The data containers chosen for this library are Blitz++ arrays. LAPACK >>>>> functions require C-style arrays to be passed. Therefore, I'm passing >>>>> the pointers to the memory blocks of the blitz::Array to the LAPACK >>>>> function (using the Blitz++ data() method). Obviously, the blitz::Array >>>>> should be contiguous in memory. Therefore, I've not set a SIMD width, >>>>> and I'm not processing the arrays before passing them to the LAPACK >>>>> function (In the attached example, arrays are created/allocated as in >>>>> blitz::Array<double,2> A(3,3), for one of them initialized, and then the >>>>> data() pointers are passed to LAPACK). As the code to reproduce the bug >>>>> is a bit long, I've attached it and you can also find it at the end of >>>>> this mail. Only 2D blitz::Array are leading to the problem. >>>>> >>>>> If you compile this example (g++ test.cc -pthread -llapack -g -o test), >>>>> and run it, you will get the following segmentation fault: >>>>> Program received signal SIGSEGV, Segmentation fault. >>>>> 0xb726c19b in ATL_dgemvT_a1_x1_b0_y1 () from /usr/lib/libblas.so.3gf >>>>> >>>>> If you replace the pointers to the memory blocks of the 2D blitz::Array >>>>> by memory blocks of the same size allocated with the new operator (Just >>>>> uncomment the lines starting by "//UNCOMMENT" in the file attached), >>>>> then everything is working fine. >>>>> >>>>> Strangely, the problem only occurs when using Ubuntu 32 bits (this has >>>>> been checked on both Ubuntu 10.04 and 12.04 beta), and the latest >>>>> checkout of Blitz++ from the mercurial repository. >>>>> If Ubuntu 64 bits OR the latest (old) release Blitz++ 0.9 is used >>>>> instead, everything is working fine, and there is no segmentation fault. >>>>> >>>>> This looks like a data alignment problem, but I don't manage to >>>>> understand the problem completely, and hence, to find a nice fix. >>>>> >>>>> Thanks in advance for your help. >>>>> >>>>> With kind regards, >>>>> Laurent >>>>> >>>>> p.s. 1: Of course, the dependence LAPACK needs to be installed (Ubuntu >>>>> package liblapack3gf). >>>>> p.s. 2: For blitz++, I've installed it through the following ppa which >>>>> seems to be up to date: >>>>> https://launchpad.net/~biometrics/+archive/blitz/+packages >>>>> >>>>> >>>>> <---------------- SOURCE CODE TO REPRODUCE THE BUG ----------------> >>>>> >>>>> #include <blitz/array.h> >>>>> #include <algorithm> >>>>> #include <iostream> >>>>> >>>>> // Declaration of the external LAPACK function (Divide and conquer SVD) >>>>> extern "C" void dgesdd_( const char *jobz, const int *M, const int *N, >>>>> double *A, const int *lda, double *S, double *U, const int* ldu, double >>>>> *VT, >>>>> const int *ldvt, double *work, const int *lwork, int *iwork, int *info); >>>>> >>>>> int main() >>>>> { >>>>> const int N=3; >>>>> blitz::Array<double,2> A(N,N); >>>>> A = 0.8147, 0.9134, 0.2785, 0.9058, 0.6324, 0.5469, 0.1270, 0.0975, >>>>> 0.9575; >>>>> blitz::Array<double,1> S(N); >>>>> blitz::Array<double,2> U(N,N); >>>>> blitz::Array<double,2> Vt(N,N); >>>>> >>>>> // Initialises LAPACK variables >>>>> const char jobz = 'A'; // Get All left singular vectors >>>>> int info = 0; >>>>> const int M = N; >>>>> const int lda = N; >>>>> const int ldu = N; >>>>> const int ldvt = N; >>>>> // Integer (workspace) array, dimension (8*min(M,N)) >>>>> >>>>> const int l_iwork = 8*N; >>>>> int *iwork = new int[l_iwork]; >>>>> // Initialises C arrays from blitz array (contiguous memory) >>>>> double* A_lapack = A.data(); >>>>> //UNCOMMENT A_lapack = new double[N*N]; >>>>> //UNCOMMENT for(size_t i=0; i<N*N; ++i) A_lapack[i] = A(i); >>>>> >>>>> double *S_lapack = S.data(); >>>>> double *U_lapack = Vt.data(); >>>>> //UNCOMMENT U_lapack = new double[N*N]; >>>>> double *VT_lapack = U.data(); >>>>> //UNCOMMENT VT_lapack = new double[N*N]; >>>>> >>>>> // Call the LAPACK function: >>>>> // A/ Query the optimal size of the working array >>>>> const int lwork_query = -1; >>>>> double work_query; >>>>> dgesdd_( &jobz, &M, &N, A_lapack, &lda, S_lapack, U_lapack, &ldu, >>>>> VT_lapack, &ldvt, &work_query, &lwork_query, iwork, &info ); >>>>> // B/ Compute >>>>> const int lwork = static_cast<int>(work_query); >>>>> double *work = new double[lwork]; >>>>> dgesdd_( &jobz, &N, &N, A_lapack, &lda, S_lapack, U_lapack, &ldu, >>>>> VT_lapack, &ldvt, work, &lwork, iwork, &info ); >>>>> >>>>> // TODO: Check info variable >>>>> >>>>> std::cout << S << std::endl; >>>>> // Free memory >>>>> delete [] iwork; >>>>> delete [] work; >>>>> //UNCOMMENT delete [] A_lapack; >>>>> //UNCOMMENT delete [] U_lapack; >>>>> //UNCOMMENT delete [] VT_lapack; >>>>> } >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> For Developers, A Lot Can Happen In A Second. >>>>> Boundary is the first to Know...and Tell You. >>>>> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! >>>>> http://p.sf.net/sfu/Boundary-d2dvs2 >>>>> _______________________________________________ >>>>> Blitz-devel mailing list >>>>> Bli...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/blitz-devel >>>>>) >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Blitz-devel mailing list >> Bli...@li... >> https://lists.sourceforge.net/lists/listinfo/blitz-devel |