Menu
▾
▴
[Hibernate-issues] [Hibernate-JIRA] Closed: (HB-386) Hibernate crashes while populate large tables
|
From: <leg...@at...> - 2003-10-12 17:24:21
|
Message:
The following issue has been closed.
Resolver: Gavin King
Date: Sun, 12 Oct 2003 12:23 PM
Hibernate does not have any memory leaks. Please do not report these kind of problems here. We have a user forum!
---------------------------------------------------------------------
View the issue:
http://opensource.atlassian.com/projects/hibernate/secure/ViewIssue.jspa?key=HB-386
Here is an overview of the issue:
---------------------------------------------------------------------
Key: HB-386
Summary: Hibernate crashes while populate large tables
Type: Bug
Status: Closed
Priority: Major
Resolution: REJECTED
Project: Hibernate2
Components:
core
Versions:
2.1
2.0.3
Assignee:
Reporter: Jeff Gager
Created: Wed, 8 Oct 2003 6:18 AM
Updated: Sun, 12 Oct 2003 12:23 PM
Environment: Dell 2Ghz Pentium 4 512MB RAM Running Windows XP.
Java 1.4.2
Eclipse 2.1.1
Description:
I am trying to write a simple java application that reads a large number of records from a Oracle database (using JDBC) and loads them into a mySQL database through hibernate.
The application crashes with java.lang.OutOfMemoryError.
As the application runs I can see it steadilly using up memory until it eventually falls over. I am commiting the Transaction, closing the Session and creating a new Session and Transaction every 1000 records. I think hibernate is either failing to release some kind of resource, or it is performing some action on the existing data before creating each record which is consuming more memory as more records are created.
I am not sure how to proceed, your help would be much appreciated.
Source follows;
package X
lots of import's
public class XLoader {
private static Logger log = Logger.getLogger("X.XLoader");
public static void main(String[] args) throws Throwable {
log.info("Processing starting at " + new Date());
Connection conn = null;
try {
// Create the JDBC Driver
final OracleDriver driver = new OracleDriver();
// Load the Oracle JDBC driver
DriverManager.registerDriver(driver);
// Establish Read-only JDBC Connection to the PAS database
conn = (Connection)DriverManager.getConnection("jdbc:oracle:oci8:@XXX", "XXX", "XXX");
conn.setAutoCommit(false);
// Loading...
loadPerson(conn);
log.info("Processing completed at " + new Date());
} catch (Throwable e) {
log.fatal("Unhandled Exception - processing halted\n" + e.toString());
throw e;
} finally {
conn.close();
System.exit(1);
log.info("Finished at " + new Date());
}
}
private static void loadPerson(Connection conn) throws Throwable {
log.info("Processing people at " + new Date());
// Declare statment and result set
PreparedStatement stmt = null;
ResultSet rset = null;
Session session = null;
Transaction tx = null;
SessionFactory sf = new Configuration().configure("/XX.xml").buildSessionFactory();
try {
// Load people
String sql = "BIG SELECT STATEMENT HERE";
log.info("Reading people at " + new Date());
stmt = conn.prepareStatement(sql,
ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);
rset = stmt.executeQuery();
if (rset.getFetchSize() < 1)
throw new SQLException("No People found");
session = sf.openSession();
tx = session.beginTransaction();
log.info("Loading people at " + new Date());
int counter = 0;
String lastPayroll = "";
Payroll payroll = null;
while (rset.next()) {
// Create new payrolls as we find them
String thisPayroll = rset.getString(1);
if (!thisPayroll.equals(lastPayroll)) {
payroll = new Payroll();
payroll.setName(rset.getString(1));
payroll.setFrequency(rset.getString(2));
payroll.setStatus(rset.getString(3));
payroll.setFirstPaydate(rset.getDate(4));
payroll.setNextPaydate(rset.getDate(5));
payroll.setPrevPaydate(rset.getDate(6));
payroll.setRunDate(rset.getDate(7));
payroll.setAccountName(rset.getString(8));
payroll.setAccountNumber(rset.getInt(9));
payroll.setSortcode(rset.getInt(10));
payroll.setBacsName(rset.getString(11));
payroll.setBacsNumber(rset.getInt(12));
ArrayList payrollMemberships = new ArrayList();
payroll.setMemberships(payrollMemberships);
session.save(payroll);
}
lastPayroll = thisPayroll;
// Create people
Person person = new Person();
person.setReference(rset.getString(13));
person.setSurname(rset.getString(14));
person.setSex(rset.getString(15).charAt(0));
person.setForenames(rset.getString(16));
person.setInitials(rset.getString(17));
person.setTitle(rset.getString(18));
person.setNiNumber(rset.getString(19));
person.setBirth(rset.getDate(20));
ArrayList personMemberships = new ArrayList();
person.setMemberships(personMemberships);
ArrayList personAddresses = new ArrayList();
person.setAddresses(personAddresses);
session.save(person);
// Create memberships
Membership membership = new Membership();
membership.setStart(rset.getDate(21));
membership.setStatus("Paying");
membership.setPerson(person);
membership.setPayroll(payroll);
person.addMembership(membership);
payroll.addMembership(membership);
session.save(membership);
// Create addresses
Address address = new Address();
address.setStart(new Date());
address.setLine1(rset.getString(22));
address.setLine2(rset.getString(23));
address.setTown(rset.getString(24));
address.setCounty(rset.getString(25));
address.setPostcode(rset.getString(26));
address.setPerson(person);
person.addAddress(address);
session.save(address);
if ((++counter % 1000) == 0) {
tx.commit();
long payid = payroll.getId().longValue();
session.close();
session = sf.openSession();
tx = session.beginTransaction();
payroll = (Payroll)session.load(Payroll.class, new Long(payid));
log.info("Loaded " + counter);
}
}
tx.commit();
session.close();
sf.close();
log.info("Total People Loaded " + counter);
rset.close();
stmt.close();
// Handle errors
} catch (Throwable e) {
log.fatal(e.getMessage(), e);
rset.close();
stmt.close();
if (sf != null) sf.close();
if (session != null) session.close();
if (tx != null) tx.rollback();
throw e;
}
log.info("Finished processing people at " + new Date());
}
}
---------------------------------------------------------------------
JIRA INFORMATION:
This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
http://opensource.atlassian.com/projects/hibernate/secure/Administrators.jspa
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
|
