|
From: Peter B. <bu...@gm...> - 2010-10-10 10:56:35
|
Hi, To load csv data, I use a modified version of csv2rec for which the data type of each column is specified explicitly in the data file. By removing the dtype guessing you get a speedup and you also avoid potential mess-ups. Alessio: sadly you right about it not being possible to trust Excel with data. Could you please give more details on the sqlite method you suggest ? -- >> by Alessio Civ Oct 10, 2010; 09:04am: Hi, a strong advice from someone who is using excel format with tons of data is to save them in csv and then import in Sqlite. Excel messes up the data types and gives a lot of troubles with numbers. Sqlite is fast and data are secure. The power of this system is that you can query your data and plot what you need for example. I can share with you my script to import from csv to sqlite if you want. -- thanks, peter butterworth |