## pytables-users

 [Pytables-users] subarray extraction and update From: Bernard KAPLAN - 2005-07-25 09:36:37 ```Dear all, I am using pytables mostly to store data in the form of simple two-dimensional arrays. For my calculations I often need to extract and update submatrices of these arrays. Unfortunately the row and column indexes I am using can not be described as slices because those can be quite random. Can anyone teach me a "good" (meaning memory efficient, fast and elegant) way to code submatrix extraction and update ? The ideal for me would be a code close to this one : x=array([1,2,5,8]) # row index y=array([0,3,7]) # column index a=data[x,y] # returns an array of size [4,3], 'data' is a pytables 2-d array data[x,y]=0 # set the submatrix defined by x rows and y columns to 0 data[x,y]=rand([4,3]) # update of a submatrix data[x,y]=data2[x,y] # copy of a submatrix from one array into another Sincerely, Bernard KAPLAN ```
 Re: [Pytables-users] subarray extraction and update From: Antonio Valentino - 2005-07-25 10:57:39 ```Alle 11:36, luned=EC 25 luglio 2005, Bernard KAPLAN ha scritto: > Dear all, hi Bernard [...] > x=3Darray([1,2,5,8]) # row index > y=3Darray([0,3,7]) # column index > a=3Ddata[x,y] # returns an array of size [4,3], 'data' is a pytables = 2-d > array > data[x,y]=3D0 # set the submatrix defined by x rows and y columns to = 0 > data[x,y]=3Drand([4,3]) # update of a submatrix > data[x,y]=3Ddata2[x,y] # copy of a submatrix from one array into anot= her I think that what you are asking for can't be done in an "elegant" way. HDF5 supports hyperslabs selection that is powerfull and elegant but it=20 requires reqular spacing in the selection, see=20 http://hdf.ncsa.uiuc.edu/HDF5/doc/RM_H5S.html#Dataspace-SelectHyperslab In order to use hyperslabs you have to be able to express the sub-matrix=20 selection in terms of "start, stride, count, and block". It seems to me that it is not your case. > Sincerely, > > Bernard KAPLAN > > > > ------------------------------------------------------- > SF.Net email is sponsored by: Discover Easy Linux Migration Strategies > from IBM. Find simple to follow Roadmaps, straightforward articles, > informative Webcasts and more! Get everything you need to get up to > speed, fast. http://ads.osdn.com/?ad_id=3D7477&alloc_id=3D16492&op=3Dcl= ick > _______________________________________________ > Pytables-users mailing list > Pytables-users@... > https://lists.sourceforge.net/lists/listinfo/pytables-users ciao --=20 Antonio Valentino ```
 Re: [Pytables-users] subarray extraction and update From: Bernard KAPLAN - 2005-07-25 14:44:50 ```Dear Antonio, In hdf5 you can define progressively a selected region (adding elements=20 one by one) by use of either "H5Sselect=5Felements" with H5S=5FSELECT=5FAPP= END=20 or "H5Sselect=5Fhyperslab" with H5S=5FSELECT=5FOR. Then you can read or wri= te=20 the selected elements at once with "H5Dread" or "H5Dwrite". Is there a function in pytables that use this functionality ? How would=20 you do to extract a submatrix in pytables ? Bernard|| Antonio Valentino wrote: >Alle 11:36, luned=EC 25 luglio 2005, Bernard KAPLAN ha scritto: > =20 > >>Dear all, >> =20 >> > >hi Bernard > >[...] > > =20 > >>x=3Darray([1,2,5,8]) # row index >>y=3Darray([0,3,7]) # column index >>a=3Ddata[x,y] # returns an array of size [4,3], 'data' is a pytables 2-d >>array >>data[x,y]=3D0 # set the submatrix defined by x rows and y columns to 0 >>data[x,y]=3Drand([4,3]) # update of a submatrix >>data[x,y]=3Ddata2[x,y] # copy of a submatrix from one array into another >> =20 >> > >I think that what you are asking for can't be done in an "elegant" way. >HDF5 supports hyperslabs selection that is powerfull and elegant but it=20 >requires reqular spacing in the selection, see=20 > >http://hdf.ncsa.uiuc.edu/HDF5/doc/RM=5FH5S.html#Dataspace-SelectHyperslab > >In order to use hyperslabs you have to be able to express the sub-matrix=20 >selection in terms of "start, stride, count, and block". >It seems to me that it is not your case. > > =20 > >>Sincerely, >> >>Bernard KAPLAN >> >> >> >>------------------------------------------------------- >>SF.Net email is sponsored by: Discover Easy Linux Migration Strategies >>from IBM. Find simple to follow Roadmaps, straightforward articles, >>informative Webcasts and more! Get everything you need to get up to >>speed, fast. http://ads.osdn.com/?ad=5Fid=3D7477&alloc=5Fid=3D16492&op=3D= click >>=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F >>Pytables-users mailing list >>Pytables-users@... >>https://lists.sourceforge.net/lists/listinfo/pytables-users >> =20 >> > >ciao > > =20 > ```
 Re: [Pytables-users] subarray extraction and update From: Antonio Valentino - 2005-07-25 15:33:00 ```Alle 16:44, luned=EC 25 luglio 2005, Bernard KAPLAN ha scritto: > Dear Antonio, > > In hdf5 you can define progressively a selected region (adding elements > one by one) by use of either "H5Sselect_elements" with H5S_SELECT_APPEN= D > or "H5Sselect_hyperslab" with H5S_SELECT_OR. Then you can read or write > the selected elements at once with "H5Dread" or "H5Dwrite". Yes, you are right. The selection operation is not as immediate as in you= r=20 example, but afrer selection you can read all data at once. > Is there a function in pytables that use this functionality ? How would > you do to extract a submatrix in pytables ? I think that at the moment this functionality is not available. Surely=20 Francesk or Ivan can give you more explainations. Usually I get an entire block of data from arrays=20 data =3D h5f.myarray[0:100, 0:100] I never encountered a situation like yours, sorry :(( > Bernard|| > > Antonio Valentino wrote: > >Alle 11:36, luned=EC 25 luglio 2005, Bernard KAPLAN ha scritto: > >>Dear all, > > > >hi Bernard > > > >[...] > > > >>x=3Darray([1,2,5,8]) # row index > >>y=3Darray([0,3,7]) # column index > >>a=3Ddata[x,y] # returns an array of size [4,3], 'data' is a pytable= s 2-d > >>array > >>data[x,y]=3D0 # set the submatrix defined by x rows and y columns t= o 0 > >>data[x,y]=3Drand([4,3]) # update of a submatrix > >>data[x,y]=3Ddata2[x,y] # copy of a submatrix from one array into an= other [...] > ------------------------------------------------------- > SF.Net email is sponsored by: Discover Easy Linux Migration Strategies > from IBM. Find simple to follow Roadmaps, straightforward articles, > informative Webcasts and more! Get everything you need to get up to > speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id=16492&op,ick > _______________________________________________ > Pytables-users mailing list > Pytables-users@... > https://lists.sourceforge.net/lists/listinfo/pytables-users --=20 Antonio Valentino ```
 Re: [Pytables-users] subarray extraction and update From: Francesc Altet - 2005-07-26 10:21:05 ```A Monday 25 July 2005 16:44, Bernard KAPLAN va escriure: > In hdf5 you can define progressively a selected region (adding elements > one by one) by use of either "H5Sselect_elements" with H5S_SELECT_APPEND > or "H5Sselect_hyperslab" with H5S_SELECT_OR. Then you can read or write > the selected elements at once with "H5Dread" or "H5Dwrite". That's correct. In fact, H5Sselect_elements is already used by Table.readCoordinates. Applying it to support indexing in your suggested way was already proposed by Pete in this list a few days ago, but just for Table objects but expanding its use to *Array objects would be nice (although not high priority for us right now). Cheers, =2D-=20 >0,0< Francesc Altet =A0 =A0 http://www.carabos.com/ V V C=E1rabos Coop. V. =A0=A0Enjoy Data "-" ```
 Re: [Pytables-users] subarray extraction and update From: travlr - 2005-07-27 11:51:47 Attachments: Message as HTML ```On 7/26/05, Francesc Altet wrote: >=20 > A Monday 25 July 2005 16:44, Bernard KAPLAN va escriure: > > In hdf5 you can define progressively a selected region (adding elements > > one by one) by use of either "H5Sselect_elements" with H5S_SELECT_APPEN= D > > or "H5Sselect_hyperslab" with H5S_SELECT_OR. Then you can read or write > > the selected elements at once with "H5Dread" or "H5Dwrite". >=20 > That's correct. In fact, H5Sselect_elements is already used by > Table.readCoordinates. Applying it to support indexing in your > suggested way was already proposed by Pete in this list a few days > ago, but just for Table objects but expanding its use to *Array > objects would be nice (although not high priority for us right now). >=20 > Cheers, Although my coding experience is still fairly green, and time is pretty=20 tight, It'd be a pleasure to contribute to pytables. I'll try to see if I= =20 can muster up the know how to implement this for us. Much is appreciated of= =20 the Carabos crew. regards, Pete -- > >0,0< Francesc Altet http://www.carabos.com/ > V V C=E1rabos Coop. V. Enjoy Data > "-" >=20 >=20 >=20 > ------------------------------------------------------- > SF.Net email is sponsored by: Discover Easy Linux Migration Strategies > from IBM. Find simple to follow Roadmaps, straightforward articles, > informative Webcasts and more! Get everything you need to get up to > speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id=16492&opclick > _______________________________________________ > Pytables-users mailing list > Pytables-users@... > https://lists.sourceforge.net/lists/listinfo/pytables-users > ```
 Re: [Pytables-users] subarray extraction and update From: Francesc Altet - 2005-07-28 11:52:34 ```A Wednesday 27 July 2005 13:51, travlr va escriure: > Although my coding experience is still fairly green, and time is pretty > tight, It'd be a pleasure to contribute to pytables. I'll try to see if I > can muster up the know how to implement this for us. Much is appreciated = of > the Carabos crew. Yes, no problem. My personal advice is that you look at how readCoordinates makes use of the H5Sselect_elements, and apply the same to arrays. Read the excellent HDF5 docs available in: http://hdf.ncsa.uiuc.edu/HDF5/doc/ Do not hesitate asking on this list if you get stuck. The C=E1rabos crew will in vacancies for most of the month of August, but some of us will check the e-mail from time to time. Also, I know that there are quite a few other people in the list having the necessary skills to answer to your questions. Cheers, =2D-=20 >0,0< Francesc Altet =A0 =A0 http://www.carabos.com/ V V C=E1rabos Coop. V. =A0=A0Enjoy Data "-" ```