Menu
▾
▴
pytables-users — PyTables users discussion list
You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(5) |
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
|
Feb
(2) |
Mar
|
Apr
(5) |
May
(11) |
Jun
(7) |
Jul
(18) |
Aug
(5) |
Sep
(15) |
Oct
(4) |
Nov
(1) |
Dec
(4) |
| 2004 |
Jan
(5) |
Feb
(2) |
Mar
(5) |
Apr
(8) |
May
(8) |
Jun
(10) |
Jul
(4) |
Aug
(4) |
Sep
(20) |
Oct
(11) |
Nov
(31) |
Dec
(41) |
| 2005 |
Jan
(79) |
Feb
(22) |
Mar
(14) |
Apr
(17) |
May
(35) |
Jun
(24) |
Jul
(26) |
Aug
(9) |
Sep
(57) |
Oct
(64) |
Nov
(25) |
Dec
(37) |
| 2006 |
Jan
(76) |
Feb
(24) |
Mar
(79) |
Apr
(44) |
May
(33) |
Jun
(12) |
Jul
(15) |
Aug
(40) |
Sep
(17) |
Oct
(21) |
Nov
(46) |
Dec
(23) |
| 2007 |
Jan
(18) |
Feb
(25) |
Mar
(41) |
Apr
(66) |
May
(18) |
Jun
(29) |
Jul
(40) |
Aug
(32) |
Sep
(34) |
Oct
(17) |
Nov
(46) |
Dec
(17) |
| 2008 |
Jan
(17) |
Feb
(42) |
Mar
(23) |
Apr
(11) |
May
(65) |
Jun
(28) |
Jul
(28) |
Aug
(16) |
Sep
(24) |
Oct
(33) |
Nov
(16) |
Dec
(5) |
| 2009 |
Jan
(19) |
Feb
(25) |
Mar
(11) |
Apr
(32) |
May
(62) |
Jun
(28) |
Jul
(61) |
Aug
(20) |
Sep
(61) |
Oct
(11) |
Nov
(14) |
Dec
(53) |
| 2010 |
Jan
(17) |
Feb
(31) |
Mar
(39) |
Apr
(43) |
May
(49) |
Jun
(47) |
Jul
(35) |
Aug
(58) |
Sep
(55) |
Oct
(91) |
Nov
(77) |
Dec
(63) |
| 2011 |
Jan
(50) |
Feb
(30) |
Mar
(67) |
Apr
(31) |
May
(17) |
Jun
(83) |
Jul
(17) |
Aug
(33) |
Sep
(35) |
Oct
(19) |
Nov
(29) |
Dec
(26) |
| 2012 |
Jan
(53) |
Feb
(22) |
Mar
(118) |
Apr
(45) |
May
(28) |
Jun
(71) |
Jul
(87) |
Aug
(55) |
Sep
(30) |
Oct
(73) |
Nov
(41) |
Dec
(28) |
| 2013 |
Jan
(19) |
Feb
(30) |
Mar
(14) |
Apr
(63) |
May
(20) |
Jun
(59) |
Jul
(40) |
Aug
(33) |
Sep
(1) |
Oct
|
Nov
|
Dec
|
|
From: Antonio V. <ant...@ti...> - 2012-10-26 17:16:32
|
Hi Jason, Il 26/10/2012 18:44, Jason Moore ha scritto: > Where exactly do I submit the bug report? There doesn't seem to be a bug > option here: https://launchpad.net/ubuntu/quantal/+package/python-tables > > Jason of course you need a launchpad account, than you can follow instruction of the ReportingBugs page on the ubuntu wiki [1] [1] https://help.ubuntu.com/community/ReportingBugs ciao > On Thu, Oct 25, 2012 at 11:27 PM, Antonio Valentino < > ant...@ti...> wrote: > >> Hi Jason, >> >> Il giorno 26/ott/2012, alle ore 07:28, Jason Moore <moo...@gm...> >> ha scritto: >> >>> So it looks like python-tables in Ubuntu 12.10 requires libhdf5-7 and >> libhdf5-7 has /usr/lib/libhdf5.so.7 not libhdf5.so.6. >> >> Correct, the hdf5 package has been updated in ubuntu 12.10. >> If you use the standard python-tables package from ubuntu repositories >> please file a bug report on launchpad. >> >> >>> Jason >>> >>> On Thu, Oct 25, 2012 at 4:10 PM, Aquil Abdullah < >> aqu...@gm...> wrote: >>> Can you check to see if libhdf5.so is in your path? If not, you can add >> it to the path where it resides to your PATH variable. >>> >>> Hopefully, that helps. >>> >>> Aquil H. Abdullah >>> >>> On Oct 25, 2012, at 18:42, Jason Moore <moo...@gm...> wrote: >>> >>>> I just updated to Ubuntu 12.10 and my pytables install is broken. I >> reinstalled and it seems like I have hdf5 1.8.4 installed but I get this >> error: >>>> >>>> moorepants@moorepants-LT:BicycleDataProcessor(master)$ vitables >> InstrumentedBicycleData.h5 >>>> Traceback (most recent call last): >>>> File "/usr/bin/vitables", line 105, in <module> >>>> main(sys.argv) >>>> File "/usr/bin/vitables", line 48, in main >>>> from vitables.vtapp import VTApp >>>> File "/usr/share/vitables/vitables/vtapp.py", line 35, in <module> >>>> import tables >>>> File "/usr/local/lib/python2.7/dist-packages/tables/__init__.py", >> line 30, in <module> >>>> from tables.utilsExtension import getPyTablesVersion, getHDF5Version >>>> ImportError: libhdf5.so.6: cannot open shared object file: No such file >> or directory >>>> >>>> What am I missing? >>>> >>>> Jason >> >> >> Anyway you can also use the eotools PPA [1] that has a more updated >> version of pytables: 2.4 against 2.3.1 of the official universe repo. >> >> [1] >> https://launchpad.net/~a.valentino/+archive/eotools?field.series_filter=quantal >> >> best regards >> >> >> -- >> Antonio Valentino -- Antonio Valentino |
|
From: Jason M. <moo...@gm...> - 2012-10-26 16:44:39
|
Where exactly do I submit the bug report? There doesn't seem to be a bug option here: https://launchpad.net/ubuntu/quantal/+package/python-tables Jason On Thu, Oct 25, 2012 at 11:27 PM, Antonio Valentino < ant...@ti...> wrote: > Hi Jason, > > Il giorno 26/ott/2012, alle ore 07:28, Jason Moore <moo...@gm...> > ha scritto: > > > So it looks like python-tables in Ubuntu 12.10 requires libhdf5-7 and > libhdf5-7 has /usr/lib/libhdf5.so.7 not libhdf5.so.6. > > Correct, the hdf5 package has been updated in ubuntu 12.10. > If you use the standard python-tables package from ubuntu repositories > please file a bug report on launchpad. > > > > Jason > > > > On Thu, Oct 25, 2012 at 4:10 PM, Aquil Abdullah < > aqu...@gm...> wrote: > > Can you check to see if libhdf5.so is in your path? If not, you can add > it to the path where it resides to your PATH variable. > > > > Hopefully, that helps. > > > > Aquil H. Abdullah > > > > On Oct 25, 2012, at 18:42, Jason Moore <moo...@gm...> wrote: > > > >> I just updated to Ubuntu 12.10 and my pytables install is broken. I > reinstalled and it seems like I have hdf5 1.8.4 installed but I get this > error: > >> > >> moorepants@moorepants-LT:BicycleDataProcessor(master)$ vitables > InstrumentedBicycleData.h5 > >> Traceback (most recent call last): > >> File "/usr/bin/vitables", line 105, in <module> > >> main(sys.argv) > >> File "/usr/bin/vitables", line 48, in main > >> from vitables.vtapp import VTApp > >> File "/usr/share/vitables/vitables/vtapp.py", line 35, in <module> > >> import tables > >> File "/usr/local/lib/python2.7/dist-packages/tables/__init__.py", > line 30, in <module> > >> from tables.utilsExtension import getPyTablesVersion, getHDF5Version > >> ImportError: libhdf5.so.6: cannot open shared object file: No such file > or directory > >> > >> What am I missing? > >> > >> Jason > > > Anyway you can also use the eotools PPA [1] that has a more updated > version of pytables: 2.4 against 2.3.1 of the official universe repo. > > [1] > https://launchpad.net/~a.valentino/+archive/eotools?field.series_filter=quantal > > best regards > > > -- > Antonio Valentino > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > -- Personal Website <http://biosport.ucdavis.edu/lab-members/jason-moore> Davis Bike Collective <http://www.davisbikecollective.org> Minister, Davis, CA BikeDavis.info Google Voice: +01 530-601-9791 Home: +01 530-753-0794 |
|
From: Antonio V. <ant...@ti...> - 2012-10-26 06:27:36
|
Hi Jason, Il giorno 26/ott/2012, alle ore 07:28, Jason Moore <moo...@gm...> ha scritto: > So it looks like python-tables in Ubuntu 12.10 requires libhdf5-7 and libhdf5-7 has /usr/lib/libhdf5.so.7 not libhdf5.so.6. Correct, the hdf5 package has been updated in ubuntu 12.10. If you use the standard python-tables package from ubuntu repositories please file a bug report on launchpad. > Jason > > On Thu, Oct 25, 2012 at 4:10 PM, Aquil Abdullah <aqu...@gm...> wrote: > Can you check to see if libhdf5.so is in your path? If not, you can add it to the path where it resides to your PATH variable. > > Hopefully, that helps. > > Aquil H. Abdullah > > On Oct 25, 2012, at 18:42, Jason Moore <moo...@gm...> wrote: > >> I just updated to Ubuntu 12.10 and my pytables install is broken. I reinstalled and it seems like I have hdf5 1.8.4 installed but I get this error: >> >> moorepants@moorepants-LT:BicycleDataProcessor(master)$ vitables InstrumentedBicycleData.h5 >> Traceback (most recent call last): >> File "/usr/bin/vitables", line 105, in <module> >> main(sys.argv) >> File "/usr/bin/vitables", line 48, in main >> from vitables.vtapp import VTApp >> File "/usr/share/vitables/vitables/vtapp.py", line 35, in <module> >> import tables >> File "/usr/local/lib/python2.7/dist-packages/tables/__init__.py", line 30, in <module> >> from tables.utilsExtension import getPyTablesVersion, getHDF5Version >> ImportError: libhdf5.so.6: cannot open shared object file: No such file or directory >> >> What am I missing? >> >> Jason Anyway you can also use the eotools PPA [1] that has a more updated version of pytables: 2.4 against 2.3.1 of the official universe repo. [1] https://launchpad.net/~a.valentino/+archive/eotools?field.series_filter=quantal best regards -- Antonio Valentino |
|
From: Jason M. <moo...@gm...> - 2012-10-26 05:29:12
|
So it looks like python-tables in Ubuntu 12.10 requires libhdf5-7 and libhdf5-7 <http://packages.ubuntu.com/quantal/amd64/libhdf5-7/filelist> has /usr/lib/libhdf5.so.7 not libhdf5.so.6. Jason On Thu, Oct 25, 2012 at 4:10 PM, Aquil Abdullah <aqu...@gm...>wrote: > Can you check to see if libhdf5.so is in your path? If not, you can add it > to the path where it resides to your PATH variable. > > Hopefully, that helps. > > Aquil H. Abdullah > > On Oct 25, 2012, at 18:42, Jason Moore <moo...@gm...> wrote: > > I just updated to Ubuntu 12.10 and my pytables install is broken. I > reinstalled and it seems like I have hdf5 1.8.4 installed but I get this > error: > > moorepants@moorepants-LT:BicycleDataProcessor(master)$ vitables > InstrumentedBicycleData.h5 > Traceback (most recent call last): > File "/usr/bin/vitables", line 105, in <module> > main(sys.argv) > File "/usr/bin/vitables", line 48, in main > from vitables.vtapp import VTApp > File "/usr/share/vitables/vitables/vtapp.py", line 35, in <module> > import tables > File "/usr/local/lib/python2.7/dist-packages/tables/__init__.py", line > 30, in <module> > from tables.utilsExtension import getPyTablesVersion, getHDF5Version > ImportError: libhdf5.so.6: cannot open shared object file: No such file or > directory > > What am I missing? > > Jason > > -- > Personal Website <http://biosport.ucdavis.edu/lab-members/jason-moore> > Davis Bike Collective <http://www.davisbikecollective.org> Minister, > Davis, CA > BikeDavis.info > Google Voice: +01 530-601-9791 > Home: +01 530-753-0794 > > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > > -- Personal Website <http://biosport.ucdavis.edu/lab-members/jason-moore> Davis Bike Collective <http://www.davisbikecollective.org> Minister, Davis, CA BikeDavis.info Google Voice: +01 530-601-9791 Home: +01 530-753-0794 |
|
From: Aquil A. <aqu...@gm...> - 2012-10-25 23:11:01
|
Can you check to see if libhdf5.so is in your path? If not, you can add it
to the path where it resides to your PATH variable.
Hopefully, that helps.
Aquil H. Abdullah
On Oct 25, 2012, at 18:42, Jason Moore <moo...@gm...> wrote:
I just updated to Ubuntu 12.10 and my pytables install is broken. I
reinstalled and it seems like I have hdf5 1.8.4 installed but I get this
error:
moorepants@moorepants-LT:BicycleDataProcessor(master)$ vitables
InstrumentedBicycleData.h5
Traceback (most recent call last):
File "/usr/bin/vitables", line 105, in <module>
main(sys.argv)
File "/usr/bin/vitables", line 48, in main
from vitables.vtapp import VTApp
File "/usr/share/vitables/vitables/vtapp.py", line 35, in <module>
import tables
File "/usr/local/lib/python2.7/dist-packages/tables/__init__.py", line
30, in <module>
from tables.utilsExtension import getPyTablesVersion, getHDF5Version
ImportError: libhdf5.so.6: cannot open shared object file: No such file or
directory
What am I missing?
Jason
--
Personal Website <http://biosport.ucdavis.edu/lab-members/jason-moore>
Davis Bike Collective <http://www.davisbikecollective.org> Minister, Davis,
CA
BikeDavis.info
Google Voice: +01 530-601-9791
Home: +01 530-753-0794
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
_______________________________________________
Pytables-users mailing list
Pyt...@li...
https://lists.sourceforge.net/lists/listinfo/pytables-users
|
|
From: Jason M. <moo...@gm...> - 2012-10-25 22:42:10
|
I just updated to Ubuntu 12.10 and my pytables install is broken. I
reinstalled and it seems like I have hdf5 1.8.4 installed but I get this
error:
moorepants@moorepants-LT:BicycleDataProcessor(master)$ vitables
InstrumentedBicycleData.h5
Traceback (most recent call last):
File "/usr/bin/vitables", line 105, in <module>
main(sys.argv)
File "/usr/bin/vitables", line 48, in main
from vitables.vtapp import VTApp
File "/usr/share/vitables/vitables/vtapp.py", line 35, in <module>
import tables
File "/usr/local/lib/python2.7/dist-packages/tables/__init__.py", line
30, in <module>
from tables.utilsExtension import getPyTablesVersion, getHDF5Version
ImportError: libhdf5.so.6: cannot open shared object file: No such file or
directory
What am I missing?
Jason
--
Personal Website <http://biosport.ucdavis.edu/lab-members/jason-moore>
Davis Bike Collective <http://www.davisbikecollective.org> Minister, Davis,
CA
BikeDavis.info
Google Voice: +01 530-601-9791
Home: +01 530-753-0794
|
|
From: Anthony S. <sc...@gm...> - 2012-10-22 14:48:58
|
Hi Jernej, This is a floating point representation issue. With 32 bits, the representations of the numbers you want are as given. You can see the same issue if you look closely with numpy. Observe the last couple of digits: In [1]: import numpy as np In [2]: np.array([12.12, 12.1234567], 'f4') Out[2]: array([ 12.11999989, 12.12345695], dtype=float32) If you want to 'fix' this issue, you should bump up the precision to 64 bits by using Float64Cols. The equivalent in numpy is: In [7]: np.array([12.12, 12.1234567], 'f8') Out[7]: array([ 12.12 , 12.1234567]) Also I would recommend reading this: http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html Be Well Anthony On Mon, Oct 22, 2012 at 9:36 AM, Jernej Makovsek <jer...@gm...>wrote: > Hi. > pyTables saves wrong data to file. I read the docs, but I don't know what > I'm doing wrong. It's the first time I noticed this kind of behavior. Any > suggestions? > > *problem: * > writing items in testL = [12.12, 12.1234567] to table > data in table: col1 = 12.1199998856, col2 = 12.1234569550 > expected: col1 = 12.12, col2 = 12.1234567 > > *script:* > class MyDescrClass(IsDescription): > eventDate = Float64Col() > col1 = Float32Col() > col2 = Float32Col() > > newdb = "_misc/db/test.h5" > dbFilters = tables.Filters(complevel=9, complib='blosc', shuffle=1) > h5file = tables.openFile(db, filters=dbFilters, mode="a") > > testL = [12.12, 12.1234567] > print("data in list: %.10f, %.10f" % (testL[0], testL[1])) > > newh5file = tables.openFile(newdb, filters=dbFilters, mode="a") > newh5file.createTable(newh5file.root, "myTable", MyDescrClass, > "someTitle", filters=dbFilters, expectedrows=1000000000) > > testT = newh5file.root.myTable > testT.row["col1"] = testL[0] > testT.row["col2"] = testL[1] > testT.row["eventDate"] = 1311530521.215325 > testT.row.append() > testT.flush() > > for row in testT: > print("data in table: col1 = %.10f, col2 = %.10f" % (row["col1"], > row["col2"])) > newh5file.close() > > *output:* > data in list: 12.1200000000, 12.1234567000 > data in table: col1 = 12.1199998856, col2 = 12.1234569550 > > > tables.print_versions() > > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > PyTables version: 2.4.0 > HDF5 version: 1.8.4-patch1 > NumPy version: 1.6.1 > Numexpr version: 1.4.2 (not using Intel's VML/MKL) > Zlib version: 1.2.3.4 (in Python interpreter) > BZIP2 version: 1.0.6 (6-Sept-2010) > Blosc version: 1.1.3 (2010-11-16) > Cython version: 0.15.1 > Python version: 2.7.3 (default, Aug 1 2012, 05:14:39) > [GCC 4.6.3] > Platform: linux2-x86_64 > Byte-ordering: little > Detected cores: 4 > > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > > |
|
From: Jernej M. <jer...@gm...> - 2012-10-22 14:37:06
|
Hi.
pyTables saves wrong data to file. I read the docs, but I don't know what
I'm doing wrong. It's the first time I noticed this kind of behavior. Any
suggestions?
*problem: *
writing items in testL = [12.12, 12.1234567] to table
data in table: col1 = 12.1199998856, col2 = 12.1234569550
expected: col1 = 12.12, col2 = 12.1234567
*script:*
class MyDescrClass(IsDescription):
eventDate = Float64Col()
col1 = Float32Col()
col2 = Float32Col()
newdb = "_misc/db/test.h5"
dbFilters = tables.Filters(complevel=9, complib='blosc', shuffle=1)
h5file = tables.openFile(db, filters=dbFilters, mode="a")
testL = [12.12, 12.1234567]
print("data in list: %.10f, %.10f" % (testL[0], testL[1]))
newh5file = tables.openFile(newdb, filters=dbFilters, mode="a")
newh5file.createTable(newh5file.root, "myTable", MyDescrClass, "someTitle",
filters=dbFilters, expectedrows=1000000000)
testT = newh5file.root.myTable
testT.row["col1"] = testL[0]
testT.row["col2"] = testL[1]
testT.row["eventDate"] = 1311530521.215325
testT.row.append()
testT.flush()
for row in testT:
print("data in table: col1 = %.10f, col2 = %.10f" % (row["col1"],
row["col2"]))
newh5file.close()
*output:*
data in list: 12.1200000000, 12.1234567000
data in table: col1 = 12.1199998856, col2 = 12.1234569550
tables.print_versions()
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
PyTables version: 2.4.0
HDF5 version: 1.8.4-patch1
NumPy version: 1.6.1
Numexpr version: 1.4.2 (not using Intel's VML/MKL)
Zlib version: 1.2.3.4 (in Python interpreter)
BZIP2 version: 1.0.6 (6-Sept-2010)
Blosc version: 1.1.3 (2010-11-16)
Cython version: 0.15.1
Python version: 2.7.3 (default, Aug 1 2012, 05:14:39)
[GCC 4.6.3]
Platform: linux2-x86_64
Byte-ordering: little
Detected cores: 4
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
|
|
From: Francesc A. <fa...@py...> - 2012-10-22 12:46:11
|
Hey, thanks to everybody that contributed datasets! I'll look into them and hope to be able to select something to show. Francesc On 10/21/12 10:55 PM, Jason Moore wrote: > This is a PyTables generated file with data collected from vehicle > (bicycle) dynamics measurements. Meta data are in tables and time > series are stored in array objects. > > http://mae.ucdavis.edu/~biosport/InstrumentedBicycleData/InstrumentedBicycleData.h5.bz2 > <http://mae.ucdavis.edu/%7Ebiosport/InstrumentedBicycleData/InstrumentedBicycleData.h5.bz2> > > It is about 308 mb compressed and 610 mb uncompressed. > > Jason > > On Sun, Oct 21, 2012 at 1:01 PM, Andy Wilson > <wil...@gm... <mailto:wil...@gm...>> wrote: > > On Sun, Oct 21, 2012 at 10:41 AM, Francesc Alted > <fa...@py... <mailto:fa...@py...>> wrote: > > > Hi, > > > > I'm going to give a tutorial on PyTables next Thursday during > the PyData > > conference in New York (http://nyc2012.pydata.org/) and I'd like > to use > > some real life data files. So, if you have some public > repository with > > data generated with PyTables, please tell me. I'm looking for files > > that are not very large (< 1GB), and that use the Table object > > significantly. A small description of the data included will be > more > > that welcome too! > > > > Thanks! > > > > -- > > Francesc Alted > > > > Hi Francesc. > > I've been working on a library for accessing climatology data that > uses pytables to cache data from the USGS. It could easily be used to > create a sample dataset for some area of interest. File size is > determined by how much data gets queried. > > > The general layout is: > > /usgs/sites > - the sites table contains information and metadata about a site > > > /usgs/values/<AGENCY>/<SITE_CODE>/<PARAMETER_CODE> > - a table containing all the timeseries data for each site and > parameter is created as data are queried > - parameter codes are a bit obscure but a dict with descriptive > metadata stashed at table.attrs.variable > - the datetime column has a CSIndex on it and stored as as a string > because some sites have data prior to the year 1901 > - pretty inefficient in terms of disk space (lots of large-ish string > columns) because it handles a very general class of data types > > > Here's what the code would look like to download and create the hdf5 > file for 10 random sites in New York: > > import ulmo > > # the default location for the hdf5 file is OS dependent, so provide > the path you want to use > hdf5_file_path = './usgs_data.h5' > > # get list of sites in NY > ulmo.usgs.pytables.update_site_list(state_code='NY', > path=hdf5_file_path) > sites = ulmo.usgs.pytables.get_sites(path=hdf5_file_path) > > # download data for a few random sites > for site in sites.keys()[:10]: > ulmo.usgs.pytables.update_site_data(site, path=hdf5_file_path) > > > > The project is on github: https://github.com/swtools/ulmo > and the code that does all the pytables stuff (including the table > descriptions) is here: > https://github.com/swtools/ulmo/blob/master/ulmo/usgs/pytables.py > > -andy > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > <mailto:Pyt...@li...> > https://lists.sourceforge.net/lists/listinfo/pytables-users > > > > > -- > Jason K. Moore, Ph.D. > Personal Website <http://biosport.ucdavis.edu/lab-members/jason-moore> > Sports Biomechanics Lab <http://biosport.ucdavis.edu>, UC Davis > Davis Open Science <http://daviswiki.org/Davis_Open_Science> > Google Voice: +01 530-601-9791 > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > > > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users -- Francesc Alted |
|
From: Francesc A. <fa...@py...> - 2012-10-22 10:26:42
|
Thanks Anthony. And yes, I'm really glad that PyTables continues alive and kicking so thanks to the new developers! It is a bit sad that I'm not able to contribute anymore, but that's life. Francesc On 10/21/12 8:10 PM, Anthony Scopatz wrote: > Congrats Francesc! > > It is a testament to how useful PyTables is that it is still around and > going strong! > > Personally, I know that your fast, polite, and in-depth responses on the > mailing list have made PyTables the great resources that it is. > Additionally, > it has served as a model to me for how open source projects > /should/ be run! > > I'd also really like to thank Antonio for driving new features into > the code base! > > If only we were all on the same continent, we could have a PyTables > birthday > party or something... > > Be Well > Anthony > > On Sun, Oct 21, 2012 at 10:26 AM, Francesc Alted <fa...@py... > <mailto:fa...@py...>> wrote: > > Hi!, > > This month PyTables celebrates the 10th anniversary of its first > public > release: > > http://osdir.com/ml/python.scientific.user/2002-10/msg00043.html > > There one can read that very new features of Python like > generators and > metaclasses were leveraged. Even that a nascent Pyrex (the antecessor > of Cython) was used for the extensions. Oh, what memories! > > The original text below: > > ------------------------------------------------------------- > Hi!, > > PyTables is a Python package which allows dealing with HDF5 > tables. Such > a table is defined as a collection of records whose values are > stored in > fixed-length fields. PyTables is intended to be easy-to-use, and > tries to > be a high-performance interface to HDF5. To achieve this, the newest > improvements introduced in Python 2.2 (like generators or slots and > metaclasses in new-brand classes) has been used. Pyrex creation > extension > tool has been chosen to access the HDF5 library. > > This package should be platform independent, but until now I've > tested it > only with Linux. It's the first public release (v 0.1), and it is in > alpha state. > > You can get it from: > > http://sourceforge.net/projects/pytables/ > > There is still not a project home page. Perhaps in next days. > > Feedback welcome.! > > -- > Francesc Alted PGP KeyID: 0x61C8C11F > Scientific aplications developer > Public PGP key available:http://www.openlc.org/falted_at_openlc.asc > Key fingerprint = 1518 38FE 3A3D 8BE8 24A0 3E5B 1328 32CC 61C8 C11F > > > -- > Francesc Alted > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > <mailto:Pyt...@li...> > https://lists.sourceforge.net/lists/listinfo/pytables-users > > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > > > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users -- Francesc Alted |
|
From: Ernesto P. <e.p...@un...> - 2012-10-22 10:10:12
|
Hi Francesc, thank you very much for your indispensable software and thank you to all people involved in this project for their invaluable effort. Best, Ernesto Il giorno 21/ott/2012, alle ore 17.26, Francesc Alted ha scritto: > Hi!, > > This month PyTables celebrates the 10th anniversary of its first public > release: > > http://osdir.com/ml/python.scientific.user/2002-10/msg00043.html > > There one can read that very new features of Python like generators and > metaclasses were leveraged. Even that a nascent Pyrex (the antecessor > of Cython) was used for the extensions. Oh, what memories! > > The original text below: > > ------------------------------------------------------------- > Hi!, > > PyTables is a Python package which allows dealing with HDF5 tables. Such > a table is defined as a collection of records whose values are stored in > fixed-length fields. PyTables is intended to be easy-to-use, and tries to > be a high-performance interface to HDF5. To achieve this, the newest > improvements introduced in Python 2.2 (like generators or slots and > metaclasses in new-brand classes) has been used. Pyrex creation extension > tool has been chosen to access the HDF5 library. > > This package should be platform independent, but until now I've tested it > only with Linux. It's the first public release (v 0.1), and it is in > alpha state. > > You can get it from: > > http://sourceforge.net/projects/pytables/ > > There is still not a project home page. Perhaps in next days. > > Feedback welcome.! > > -- > Francesc Alted PGP KeyID: 0x61C8C11F > Scientific aplications developer > Public PGP key available:http://www.openlc.org/falted_at_openlc.asc > Key fingerprint = 1518 38FE 3A3D 8BE8 24A0 3E5B 1328 32CC 61C8 C11F > > > -- > Francesc Alted > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users **** Riservatezza / Confidentiality **** In ottemperanza al D.Lgs. n. 196 del 30/6/2003 in materia di protezione dei dati personali, le informazioni contenute in questo messaggio sono strettamente riservate ed esclusivamente indirizzate al destinatario indicato (oppure alla persona responsabile di rimetterlo al destinatario). Vogliate tener presente che qualsiasi uso, riproduzione o divulgazione di questo messaggio e' vietato. Nel caso in cui aveste ricevuto questo messaggio per errore, vogliate cortesemente avvertire il mittente e distruggere il presente messaggio. |
|
From: Jason M. <jk...@uc...> - 2012-10-21 20:55:48
|
This is a PyTables generated file with data collected from vehicle (bicycle) dynamics measurements. Meta data are in tables and time series are stored in array objects. http://mae.ucdavis.edu/~biosport/InstrumentedBicycleData/InstrumentedBicycleData.h5.bz2 It is about 308 mb compressed and 610 mb uncompressed. Jason On Sun, Oct 21, 2012 at 1:01 PM, Andy Wilson <wil...@gm...>wrote: > On Sun, Oct 21, 2012 at 10:41 AM, Francesc Alted <fa...@py...> > wrote: > > > Hi, > > > > I'm going to give a tutorial on PyTables next Thursday during the PyData > > conference in New York (http://nyc2012.pydata.org/) and I'd like to use > > some real life data files. So, if you have some public repository with > > data generated with PyTables, please tell me. I'm looking for files > > that are not very large (< 1GB), and that use the Table object > > significantly. A small description of the data included will be more > > that welcome too! > > > > Thanks! > > > > -- > > Francesc Alted > > > > Hi Francesc. > > I've been working on a library for accessing climatology data that > uses pytables to cache data from the USGS. It could easily be used to > create a sample dataset for some area of interest. File size is > determined by how much data gets queried. > > > The general layout is: > > /usgs/sites > - the sites table contains information and metadata about a site > > > /usgs/values/<AGENCY>/<SITE_CODE>/<PARAMETER_CODE> > - a table containing all the timeseries data for each site and > parameter is created as data are queried > - parameter codes are a bit obscure but a dict with descriptive > metadata stashed at table.attrs.variable > - the datetime column has a CSIndex on it and stored as as a string > because some sites have data prior to the year 1901 > - pretty inefficient in terms of disk space (lots of large-ish string > columns) because it handles a very general class of data types > > > Here's what the code would look like to download and create the hdf5 > file for 10 random sites in New York: > > import ulmo > > # the default location for the hdf5 file is OS dependent, so provide > the path you want to use > hdf5_file_path = './usgs_data.h5' > > # get list of sites in NY > ulmo.usgs.pytables.update_site_list(state_code='NY', path=hdf5_file_path) > sites = ulmo.usgs.pytables.get_sites(path=hdf5_file_path) > > # download data for a few random sites > for site in sites.keys()[:10]: > ulmo.usgs.pytables.update_site_data(site, path=hdf5_file_path) > > > > The project is on github: https://github.com/swtools/ulmo > and the code that does all the pytables stuff (including the table > descriptions) is here: > https://github.com/swtools/ulmo/blob/master/ulmo/usgs/pytables.py > > -andy > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > -- Jason K. Moore, Ph.D. Personal Website <http://biosport.ucdavis.edu/lab-members/jason-moore> Sports Biomechanics Lab <http://biosport.ucdavis.edu>, UC Davis Davis Open Science <http://daviswiki.org/Davis_Open_Science> Google Voice: +01 530-601-9791 |
|
From: Andy W. <wil...@gm...> - 2012-10-21 20:02:26
|
On Sun, Oct 21, 2012 at 10:41 AM, Francesc Alted <fa...@py...> wrote: > Hi, > > I'm going to give a tutorial on PyTables next Thursday during the PyData > conference in New York (http://nyc2012.pydata.org/) and I'd like to use > some real life data files. So, if you have some public repository with > data generated with PyTables, please tell me. I'm looking for files > that are not very large (< 1GB), and that use the Table object > significantly. A small description of the data included will be more > that welcome too! > > Thanks! > > -- > Francesc Alted Hi Francesc. I've been working on a library for accessing climatology data that uses pytables to cache data from the USGS. It could easily be used to create a sample dataset for some area of interest. File size is determined by how much data gets queried. The general layout is: /usgs/sites - the sites table contains information and metadata about a site /usgs/values/<AGENCY>/<SITE_CODE>/<PARAMETER_CODE> - a table containing all the timeseries data for each site and parameter is created as data are queried - parameter codes are a bit obscure but a dict with descriptive metadata stashed at table.attrs.variable - the datetime column has a CSIndex on it and stored as as a string because some sites have data prior to the year 1901 - pretty inefficient in terms of disk space (lots of large-ish string columns) because it handles a very general class of data types Here's what the code would look like to download and create the hdf5 file for 10 random sites in New York: import ulmo # the default location for the hdf5 file is OS dependent, so provide the path you want to use hdf5_file_path = './usgs_data.h5' # get list of sites in NY ulmo.usgs.pytables.update_site_list(state_code='NY', path=hdf5_file_path) sites = ulmo.usgs.pytables.get_sites(path=hdf5_file_path) # download data for a few random sites for site in sites.keys()[:10]: ulmo.usgs.pytables.update_site_data(site, path=hdf5_file_path) The project is on github: https://github.com/swtools/ulmo and the code that does all the pytables stuff (including the table descriptions) is here: https://github.com/swtools/ulmo/blob/master/ulmo/usgs/pytables.py -andy |
|
From: Antonio V. <ant...@ti...> - 2012-10-21 19:04:28
|
Il 21/10/2012 17:26, Francesc Alted ha scritto: > Hi!, > > This month PyTables celebrates the 10th anniversary of its first public > release: > > http://osdir.com/ml/python.scientific.user/2002-10/msg00043.html > Happy birthday PyTables!!! :)) ciao -- Antonio Valentino |
|
From: Anthony S. <sc...@gm...> - 2012-10-21 18:11:20
|
Congrats Francesc! It is a testament to how useful PyTables is that it is still around and going strong! Personally, I know that your fast, polite, and in-depth responses on the mailing list have made PyTables the great resources that it is. Additionally, it has served as a model to me for how open source projects *should* be run! I'd also really like to thank Antonio for driving new features into the code base! If only we were all on the same continent, we could have a PyTables birthday party or something... Be Well Anthony On Sun, Oct 21, 2012 at 10:26 AM, Francesc Alted <fa...@py...>wrote: > Hi!, > > This month PyTables celebrates the 10th anniversary of its first public > release: > > http://osdir.com/ml/python.scientific.user/2002-10/msg00043.html > > There one can read that very new features of Python like generators and > metaclasses were leveraged. Even that a nascent Pyrex (the antecessor > of Cython) was used for the extensions. Oh, what memories! > > The original text below: > > ------------------------------------------------------------- > Hi!, > > PyTables is a Python package which allows dealing with HDF5 tables. Such > a table is defined as a collection of records whose values are stored in > fixed-length fields. PyTables is intended to be easy-to-use, and tries to > be a high-performance interface to HDF5. To achieve this, the newest > improvements introduced in Python 2.2 (like generators or slots and > metaclasses in new-brand classes) has been used. Pyrex creation extension > tool has been chosen to access the HDF5 library. > > This package should be platform independent, but until now I've tested it > only with Linux. It's the first public release (v 0.1), and it is in > alpha state. > > You can get it from: > > http://sourceforge.net/projects/pytables/ > > There is still not a project home page. Perhaps in next days. > > Feedback welcome.! > > -- > Francesc Alted PGP KeyID: 0x61C8C11F > Scientific aplications developer > Public PGP key available:http://www.openlc.org/falted_at_openlc.asc > Key fingerprint = 1518 38FE 3A3D 8BE8 24A0 3E5B 1328 32CC 61C8 C11F > > > -- > Francesc Alted > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > |
|
From: Anthony S. <sc...@gm...> - 2012-10-21 18:03:49
|
Hello Francesc, I look forward to your pydata hearing how your tutorial goes! Here [1] is a file that stores some basic nuclear data that is freely redistributable. It stores atomic weights, bound neutron scattering lengths, and pre-compiled neutron cross sections (xs) for 5 different energy regimes. Everything in here is a table. The file is rather (at about 165 kb). There are integer, float, and complex columns. I hope that this helps! Be Well Anthony 1. https://s3.amazonaws.com/pyne/prebuilt_nuc_data.h5 On Sun, Oct 21, 2012 at 10:41 AM, Francesc Alted <fa...@py...>wrote: > Hi, > > I'm going to give a tutorial on PyTables next Thursday during the PyData > conference in New York (http://nyc2012.pydata.org/) and I'd like to use > some real life data files. So, if you have some public repository with > data generated with PyTables, please tell me. I'm looking for files > that are not very large (< 1GB), and that use the Table object > significantly. A small description of the data included will be more > that welcome too! > > Thanks! > > -- > Francesc Alted > > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > |
|
From: Francesc A. <fa...@py...> - 2012-10-21 15:41:05
|
Hi, I'm going to give a tutorial on PyTables next Thursday during the PyData conference in New York (http://nyc2012.pydata.org/) and I'd like to use some real life data files. So, if you have some public repository with data generated with PyTables, please tell me. I'm looking for files that are not very large (< 1GB), and that use the Table object significantly. A small description of the data included will be more that welcome too! Thanks! -- Francesc Alted |
|
From: Francesc A. <fa...@py...> - 2012-10-21 15:26:42
|
Hi!, This month PyTables celebrates the 10th anniversary of its first public release: http://osdir.com/ml/python.scientific.user/2002-10/msg00043.html There one can read that very new features of Python like generators and metaclasses were leveraged. Even that a nascent Pyrex (the antecessor of Cython) was used for the extensions. Oh, what memories! The original text below: ------------------------------------------------------------- Hi!, PyTables is a Python package which allows dealing with HDF5 tables. Such a table is defined as a collection of records whose values are stored in fixed-length fields. PyTables is intended to be easy-to-use, and tries to be a high-performance interface to HDF5. To achieve this, the newest improvements introduced in Python 2.2 (like generators or slots and metaclasses in new-brand classes) has been used. Pyrex creation extension tool has been chosen to access the HDF5 library. This package should be platform independent, but until now I've tested it only with Linux. It's the first public release (v 0.1), and it is in alpha state. You can get it from: http://sourceforge.net/projects/pytables/ There is still not a project home page. Perhaps in next days. Feedback welcome.! -- Francesc Alted PGP KeyID: 0x61C8C11F Scientific aplications developer Public PGP key available:http://www.openlc.org/falted_at_openlc.asc Key fingerprint = 1518 38FE 3A3D 8BE8 24A0 3E5B 1328 32CC 61C8 C11F -- Francesc Alted |
|
From: Ernesto P. <e.p...@un...> - 2012-10-17 16:03:05
|
Hi all, > Hi Anthony, > > Il giorno 16/ott/2012, alle ore 02:04, Anthony Scopatz <sc...@gm...> ha scritto: > > > Hello Ernesto, > > > > So you are actually asking two different questions, one on reading and the other on writing. In general reading, or querying, with multiprocessing works very well. Writing to a single file with multiple processes is destined to failure though. So the strategy that many people have adopted is to have multiple processes create the data and then have a master process which acts as a queue for writing out the data. Please see the example here for more inspiration [1]. Note that we have been having problems recently with multiprocess writing out to multiple files, but that is not what you want to do. > > > > Be Well > > Anthony > > > > 1. https://github.com/PyTables/PyTables/blob/develop/examples/multiprocess_access_queues.py > > It seems that the topic PyTables + multiprocessing became very popular since some time. > > Probably we should add a FAQ entry and provide a more extended tutorial based on the example provided by Josh. > I agree. This is something that we should advertise a little more. I think that this could be very useful especially for non computer scientists as me ! Best, Ernesto > Be Well > Anthony > > > > cheers > > -- > Antonio Valentino > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct_______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users |
|
From: Anthony S. <sc...@gm...> - 2012-10-17 15:30:05
|
On Wed, Oct 17, 2012 at 12:58 AM, Antonio Valentino < ant...@ti...> wrote: > Hi Anthony, > > Il giorno 16/ott/2012, alle ore 02:04, Anthony Scopatz <sc...@gm...> > ha scritto: > > > Hello Ernesto, > > > > So you are actually asking two different questions, one on reading and > the other on writing. In general reading, or querying, with > multiprocessing works very well. Writing to a single file with multiple > processes is destined to failure though. So the strategy that many people > have adopted is to have multiple processes create the data and then have a > master process which acts as a queue for writing out the data. Please see > the example here for more inspiration [1]. Note that we have been having > problems recently with multiprocess writing out to multiple files, but that > is not what you want to do. > > > > Be Well > > Anthony > > > > 1. > https://github.com/PyTables/PyTables/blob/develop/examples/multiprocess_access_queues.py > > It seems that the topic PyTables + multiprocessing became very popular > since some time. > > Probably we should add a FAQ entry and provide a more extended tutorial > based on the example provided by Josh. > I agree. This is something that we should advertise a little more. Be Well Anthony > > > cheers > > -- > Antonio Valentino > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_sfd2d_oct > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > |
|
From: Antonio V. <ant...@ti...> - 2012-10-17 05:58:29
|
Hi Anthony, Il giorno 16/ott/2012, alle ore 02:04, Anthony Scopatz <sc...@gm...> ha scritto: > Hello Ernesto, > > So you are actually asking two different questions, one on reading and the other on writing. In general reading, or querying, with multiprocessing works very well. Writing to a single file with multiple processes is destined to failure though. So the strategy that many people have adopted is to have multiple processes create the data and then have a master process which acts as a queue for writing out the data. Please see the example here for more inspiration [1]. Note that we have been having problems recently with multiprocess writing out to multiple files, but that is not what you want to do. > > Be Well > Anthony > > 1. https://github.com/PyTables/PyTables/blob/develop/examples/multiprocess_access_queues.py It seems that the topic PyTables + multiprocessing became very popular since some time. Probably we should add a FAQ entry and provide a more extended tutorial based on the example provided by Josh. cheers -- Antonio Valentino |
|
From: Anthony S. <sc...@gm...> - 2012-10-16 00:04:40
|
Hello Ernesto, So you are actually asking two different questions, one on reading and the other on writing. In general reading, or querying, with multiprocessing works very well. Writing to a single file with multiple processes is destined to failure though. So the strategy that many people have adopted is to have multiple processes create the data and then have a master process which acts as a queue for writing out the data. Please see the example here for more inspiration [1]. Note that we have been having problems recently with multiprocess writing out to multiple files, but that is not what you want to do. Be Well Anthony 1. https://github.com/PyTables/PyTables/blob/develop/examples/multiprocess_access_queues.py On Mon, Oct 15, 2012 at 11:45 AM, Ernesto Picardi <e.p...@un...>wrote: > Dear all, > > I have a hdf5 file including several tables. To speed up the creation of > all tables, could I create each individual table by independent processes > launched by multiprocessing module? Could I employ independent processes to > query diverse tables of the same hdf5 file? > > Thank you very much in advance for whatever answer. > > Regards, > > Ernesto > > > > > > > > **** Riservatezza / Confidentiality **** > In ottemperanza al D.Lgs. n. 196 del 30/6/2003 in materia di protezione > dei dati personali, le informazioni contenute in questo messaggio sono > strettamente riservate ed esclusivamente indirizzate al destinatario > indicato (oppure alla persona responsabile di rimetterlo al destinatario). > Vogliate tener presente che qualsiasi uso, riproduzione o divulgazione di > questo messaggio e' vietato. Nel caso in cui aveste ricevuto questo > messaggio per errore, vogliate cortesemente avvertire il mittente e > distruggere il presente messaggio. > > > > ------------------------------------------------------------------------------ > Don't let slow site performance ruin your business. Deploy New Relic APM > Deploy New Relic app performance management and know exactly > what is happening inside your Ruby, Python, PHP, Java, and .NET app > Try New Relic at no cost today and get our sweet Data Nerd shirt too! > http://p.sf.net/sfu/newrelic-dev2dev > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > |
|
From: Ernesto P. <e.p...@un...> - 2012-10-15 19:04:48
|
Dear all, I have a hdf5 file including several tables. To speed up the creation of all tables, could I create each individual table by independent processes launched by multiprocessing module? Could I employ independent processes to query diverse tables of the same hdf5 file? Thank you very much in advance for whatever answer. Regards, Ernesto **** Riservatezza / Confidentiality **** In ottemperanza al D.Lgs. n. 196 del 30/6/2003 in materia di protezione dei dati personali, le informazioni contenute in questo messaggio sono strettamente riservate ed esclusivamente indirizzate al destinatario indicato (oppure alla persona responsabile di rimetterlo al destinatario). Vogliate tener presente che qualsiasi uso, riproduzione o divulgazione di questo messaggio e' vietato. Nel caso in cui aveste ricevuto questo messaggio per errore, vogliate cortesemente avvertire il mittente e distruggere il presente messaggio. |
|
From: Anthony S. <sc...@gm...> - 2012-10-12 15:26:22
|
On Fri, Oct 12, 2012 at 8:47 AM, Aquil H. Abdullah <aqu...@gm... > wrote: > I have a process that uses PyTabels and opens a bunch of HDF5 files in > read-only mode. I know that if I don't close these files that AtExit hook > will close the open files and display the message: > > "Closing remaining open files:" > > My question is simple is it possible for me to run into any corruption > issues by not explicitly closing files that have been opened in read-only > mode? > Hello Aquil, I don't think that you will have any issues with doing this. However, I would just go ahead and close all of the files anyway. The 'with' statement is great for that. Also, recall line 2 of the Zen of Python: "Explicit is better than implicit." Be Well Anthony > > -- > Aquil H. Abdullah > "I never think of the future. It comes soon enough" - Albert Einstein > > > > ------------------------------------------------------------------------------ > Don't let slow site performance ruin your business. Deploy New Relic APM > Deploy New Relic app performance management and know exactly > what is happening inside your Ruby, Python, PHP, Java, and .NET app > Try New Relic at no cost today and get our sweet Data Nerd shirt too! > http://p.sf.net/sfu/newrelic-dev2dev > _______________________________________________ > Pytables-users mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/pytables-users > > |
|
From: Aquil H. A. <aqu...@gm...> - 2012-10-12 13:47:23
|
I have a process that uses PyTabels and opens a bunch of HDF5 files in read-only mode. I know that if I don't close these files that AtExit hook will close the open files and display the message: "Closing remaining open files:" My question is simple is it possible for me to run into any corruption issues by not explicitly closing files that have been opened in read-only mode? -- Aquil H. Abdullah "I never think of the future. It comes soon enough" - Albert Einstein |
22 messages has been excluded from this view by a project administrator.
