On 06/14/14 02:07, Jorge Jimenez wrote:

What I have:

a multi array X, which is a 3XD complex array * #encode * # frames

Array< Array<complex<float>,3>,2 > X = recon.read_data(data);

Why is this a multi-array?

You've created a two-dimensional array, where each entry is, separately,  a three-dimensional array of entries with type complex<float>.

This seems unnecessarily complicated, since it puts your actual data behind a few levels of indirection and it hides structure that you later assume.

Onces done that, if I have 3 encodes in the first frame I should end up whit data set2 = 400x400x(800*3) = 400x400x2400. This set will be my first array in my new multi array matrix Y or Y(0).
This is a case in point.  Is each 3D array going to be of the same size across encodes and/or frames?  Your multi-array structure permits each chunk to be of different size, but then you assume that (at least within a frame) each is indeed of the same size when you wish to concatenate them.

Will you ever need a particularly clever way of accessing, say, the first encode of each frame, across all frames?  That's permitted by your multi-array structure, but based on some rough guesses about your subsequent workflow that might not be *necessary*.  Will you have the same number of encodes of each frame?  That's required by your data organization, but it may not be necessary.

Please be aware of the amount of memory that you're casually throwing around.  Your first frame mentioned above will use, by itself, about 1.4GiB of memory.  Keeping more than a handful of frames in memory at a time will be beyond the capabilities of many workstations, so you may wish to use data structures more amenable to offline processing.

If you *are* aware of the memory complexities involved and you are guaranteed to have the same data size per (encode,frame), then the most useful array organization would be a five-dimensional array of type array< complex<float>, 5>, with indices (x,y,z,enc,frame).

Multi-arrays should be used sparingly; Blitz++ is good at math-on-arrays but less good at heavy data-structure work.  You should use them as you would an array type in MATLAB or NumPy.

 
Array< Array<float,3>,1 > Y; //Array for all encodes in same frame
Please also be aware that this doesn't actually give the Y array any size, so even if your code compiled the subsequent unpack-in-Y would do a very wrong thing and probably (if you're lucky) segfault.