I am trying to build a large table (105 columns and over a million rows). Using Access, I can use Import / External Data and then give it a big text file (tab separated). It takes barely any time at all. Now I want to do it from Java, so I looked at UCanAccess.
First, I tried calls to execute() with INSERT statements, but that took a minute for every thousand rows, which means it would take 17 hours.
Next, I tried using prepareStatement() and that halved the time, but still would need 8 hours.
Is there a bulk way of loading data that can run faster than this?
Thanks!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It is not our game, but you should setAutocommit(false) at the start, and
then commit just each 1000-2000 records... performance will improve
dramatically... or much better, do it all directly with jackcess.
I am trying to build a large table (105 columns and over a million rows).
Using Access, I can use Import / External Data and then give it a big text
file (tab separated). It takes barely any time at all. Now I want to do it
from Java, so I looked at UCanAccess.
First, I tried calls to execute() with INSERT statements, but that took a
minute for every thousand rows, which means it would take 17 hours.
Next, I tried using prepareStatement() and that halved the time, but still
would need 8 hours.
Is there a bulk way of loading data that can run faster than this?
I am trying to build a large table (105 columns and over a million rows). Using Access, I can use Import / External Data and then give it a big text file (tab separated). It takes barely any time at all. Now I want to do it from Java, so I looked at UCanAccess.
First, I tried calls to execute() with INSERT statements, but that took a minute for every thousand rows, which means it would take 17 hours.
Next, I tried using prepareStatement() and that halved the time, but still would need 8 hours.
Is there a bulk way of loading data that can run faster than this?
Thanks!
It is not our game, but you should setAutocommit(false) at the start, and
then commit just each 1000-2000 records... performance will improve
dramatically... or much better, do it all directly with jackcess.
Il Gio 23 Set 2021, 20:09 Steven Rubin strubin9@users.sourceforge.net ha
scritto: