hibernate-devel

[Hibernate] slow performance on flush

[Christoph S. <ch...@mc...>]

Hi all!

I'm trying to increase the performance of this pseudocode:

for (j=3D1;j<100;j++)
{
    for (i=3D1;i<100;i++)
    {
        session.save(class)
    }
    session.flush();
    session.connection().commit();
}


Now my problem is that flush doesnt take a constant time, but every =
invocation of flush takes more time than the last one, the more I flush =
the slower it gets. To me it looks like hibernate keeps the references =
to the flushed objects in the entries map, and doesnt remove them. Is =
there anything I can do about it?

regards
 chris

[Hibernate] Re: slow performance on flush

[Gavin K. <ga...@ap...>]

Yup, all entries are kept in the session cache until you close the =
session. Why would you use this kind of code?? Just flush() once at the =
end (outside the loop).
  ----- Original Message -----=20
  From: Christoph Sturm=20
  To: hib...@li... ; Gavin King=20
  Sent: Tuesday, November 26, 2002 12:18 AM
  Subject: slow performance on flush


  Hi all!

  I'm trying to increase the performance of this pseudocode:

  for (j=3D1;j<100;j++)
  {
      for (i=3D1;i<100;i++)
      {
          session.save(class)
      }
      session.flush();
      session.connection().commit();
  }


  Now my problem is that flush doesnt take a constant time, but every =
invocation of flush takes more time than the last one, the more I flush =
the slower it gets. To me it looks like hibernate keeps the references =
to the flushed objects in the entries map, and doesnt remove them. Is =
there anything I can do about it?

  regards
   chris

[Hibernate] Re: slow performance on flush

[Christoph S. <ch...@mc...>]

I have a really long running import, and I want to write the records to =
the db as they come in. Is it save to clear the entries Map after a =
flush? Or would you recommend a different approach?


  ----- Original Message -----=20
  From: Gavin King=20
  To: Christoph Sturm=20
  Cc: hibernate list=20
  Sent: Monday, November 25, 2002 2:23 PM
  Subject: Re: slow performance on flush


  Yup, all entries are kept in the session cache until you close the =
session. Why would you use this kind of code?? Just flush() once at the =
end (outside the loop).
    ----- Original Message -----=20
    From: Christoph Sturm=20
    To: hib...@li... ; Gavin King=20
    Sent: Tuesday, November 26, 2002 12:18 AM
    Subject: slow performance on flush


    Hi all!

    I'm trying to increase the performance of this pseudocode:

    for (j=3D1;j<100;j++)
    {
        for (i=3D1;i<100;i++)
        {
            session.save(class)
        }
        session.flush();
        session.connection().commit();
    }


    Now my problem is that flush doesnt take a constant time, but every =
invocation of flush takes more time than the last one, the more I flush =
the slower it gets. To me it looks like hibernate keeps the references =
to the flushed objects in the entries map, and doesnt remove them. Is =
there anything I can do about it?

    regards
     chris

[Hibernate] Re: slow performance on flush

[Gavin K. <ga...@ap...>]

Just use a new Session for each loop iteration. It is even possible in a =
special case like this to have multiple sessions in the same JDBC =
transaction.=20

for ( int i=3D0; i<100; i++) {
    Session s =3D sf.openSession(conn);
    for ( int j=3D0; j<100; j++) {
        s.save( ... );
    }
    s.flush();
    s.close();
}
conn.commit();

However I suspect in this case you are better off doing it in multiple =
transactions....

  ----- Original Message -----=20
  From: Christoph Sturm=20
  To: Gavin King=20
  Cc: hibernate list=20
  Sent: Tuesday, November 26, 2002 12:27 AM
  Subject: Re: slow performance on flush


  I have a really long running import, and I want to write the records =
to the db as they come in. Is it save to clear the entries Map after a =
flush? Or would you recommend a different approach?


    ----- Original Message -----=20
    From: Gavin King=20
    To: Christoph Sturm=20
    Cc: hibernate list=20
    Sent: Monday, November 25, 2002 2:23 PM
    Subject: Re: slow performance on flush


    Yup, all entries are kept in the session cache until you close the =
session. Why would you use this kind of code?? Just flush() once at the =
end (outside the loop).
      ----- Original Message -----=20
      From: Christoph Sturm=20
      To: hib...@li... ; Gavin King=20
      Sent: Tuesday, November 26, 2002 12:18 AM
      Subject: slow performance on flush


      Hi all!

      I'm trying to increase the performance of this pseudocode:

      for (j=3D1;j<100;j++)
      {
          for (i=3D1;i<100;i++)
          {
              session.save(class)
          }
          session.flush();
          session.connection().commit();
      }


      Now my problem is that flush doesnt take a constant time, but =
every invocation of flush takes more time than the last one, the more I =
flush the slower it gets. To me it looks like hibernate keeps the =
references to the flushed objects in the entries map, and doesnt remove =
them. Is there anything I can do about it?

      regards
       chris

Re: [Hibernate] Re: slow performance on flush

[Juozas B. <ba...@ce...>]

It is better to use some "native" API like "COPY" for long running import,
some RDBM's support imports without constaint
checks, SQL parsing ....  , and sometimes "long running import" becomes very
"short".

----- Original Message -----
From: "Christoph Sturm" <ch...@mc...>
To: "Gavin King" <ga...@ap...>
Cc: "hibernate list" <hib...@li...>
Sent: Monday, November 25, 2002 3:27 PM
Subject: [Hibernate] Re: slow performance on flush


I have a really long running import, and I want to write the records to the
db as they come in. Is it save to clear the entries Map after a flush? Or
would you recommend a different approach?


  ----- Original Message -----
  From: Gavin King
  To: Christoph Sturm
  Cc: hibernate list
  Sent: Monday, November 25, 2002 2:23 PM
  Subject: Re: slow performance on flush


  Yup, all entries are kept in the session cache until you close the
session. Why would you use this kind of code?? Just flush() once at the end
(outside the loop).
    ----- Original Message -----
    From: Christoph Sturm
    To: hib...@li... ; Gavin King
    Sent: Tuesday, November 26, 2002 12:18 AM
    Subject: slow performance on flush


    Hi all!

    I'm trying to increase the performance of this pseudocode:

    for (j=1;j<100;j++)
    {
        for (i=1;i<100;i++)
        {
            session.save(class)
        }
        session.flush();
        session.connection().commit();
    }


    Now my problem is that flush doesnt take a constant time, but every
invocation of flush takes more time than the last one, the more I flush the
slower it gets. To me it looks like hibernate keeps the references to the
flushed objects in the entries map, and doesnt remove them. Is there
anything I can do about it?

    regards
     chris

Re: [Hibernate] Re: slow performance on flush

[Christoph S. <ch...@mc...>]

Hey!

The stuff you mention sounds cool, and I already thought about that, but to
my knowledge there's no java api that supports stuff like that. If you know
a api that supports such functionality, please tell me, that would be really
great!

regards
 chris
----- Original Message -----
From: "Juozas Baliuka" <ba...@ce...>
To: "Christoph Sturm" <ch...@mc...>; "Gavin King"
<ga...@ap...>
Cc: "hibernate list" <hib...@li...>
Sent: Monday, November 25, 2002 6:03 PM
Subject: Re: [Hibernate] Re: slow performance on flush


>
> It is better to use some "native" API like "COPY" for long running import,
> some RDBM's support imports without constaint
> checks, SQL parsing ....  , and sometimes "long running import" becomes
very
> "short".
>
> ----- Original Message -----
> From: "Christoph Sturm" <ch...@mc...>
> To: "Gavin King" <ga...@ap...>
> Cc: "hibernate list" <hib...@li...>
> Sent: Monday, November 25, 2002 3:27 PM
> Subject: [Hibernate] Re: slow performance on flush
>
>
> I have a really long running import, and I want to write the records to
the
> db as they come in. Is it save to clear the entries Map after a flush? Or
> would you recommend a different approach?
>
>
>   ----- Original Message -----
>   From: Gavin King
>   To: Christoph Sturm
>   Cc: hibernate list
>   Sent: Monday, November 25, 2002 2:23 PM
>   Subject: Re: slow performance on flush
>
>
>   Yup, all entries are kept in the session cache until you close the
> session. Why would you use this kind of code?? Just flush() once at the
end
> (outside the loop).
>     ----- Original Message -----
>     From: Christoph Sturm
>     To: hib...@li... ; Gavin King
>     Sent: Tuesday, November 26, 2002 12:18 AM
>     Subject: slow performance on flush
>
>
>     Hi all!
>
>     I'm trying to increase the performance of this pseudocode:
>
>     for (j=1;j<100;j++)
>     {
>         for (i=1;i<100;i++)
>         {
>             session.save(class)
>         }
>         session.flush();
>         session.connection().commit();
>     }
>
>
>     Now my problem is that flush doesnt take a constant time, but every
> invocation of flush takes more time than the last one, the more I flush
the
> slower it gets. To me it looks like hibernate keeps the references to the
> flushed objects in the entries map, and doesnt remove them. Is there
> anything I can do about it?
>
>     regards
>      chris
>
>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> hibernate-devel mailing list
> hib...@li...
> https://lists.sourceforge.net/lists/listinfo/hibernate-devel
>

Re: [Hibernate] Re: slow performance on flush

[Juozas B. <ba...@ce...>]

It is no standard way to for this trick, It depends on RDBMS implementation.
It is possible upload tab separated text file to server and execute  native
SQL on postgersql like this:
"COPY mytable FROM filename "
http://www.postgresql.org/idocs/index.php?sql-copy.html


> Hey!
>
> The stuff you mention sounds cool, and I already thought about that, but
to
> my knowledge there's no java api that supports stuff like that. If you
know
> a api that supports such functionality, please tell me, that would be
really
> great!
>
> regards
>  chris
> ----- Original Message -----
> From: "Juozas Baliuka" <ba...@ce...>
> To: "Christoph Sturm" <ch...@mc...>; "Gavin King"
> <ga...@ap...>
> Cc: "hibernate list" <hib...@li...>
> Sent: Monday, November 25, 2002 6:03 PM
> Subject: Re: [Hibernate] Re: slow performance on flush
>
>
> >
> > It is better to use some "native" API like "COPY" for long running
import,
> > some RDBM's support imports without constaint
> > checks, SQL parsing ....  , and sometimes "long running import" becomes
> very
> > "short".
> >
> > ----- Original Message -----
> > From: "Christoph Sturm" <ch...@mc...>
> > To: "Gavin King" <ga...@ap...>
> > Cc: "hibernate list" <hib...@li...>
> > Sent: Monday, November 25, 2002 3:27 PM
> > Subject: [Hibernate] Re: slow performance on flush
> >
> >
> > I have a really long running import, and I want to write the records to
> the
> > db as they come in. Is it save to clear the entries Map after a flush?
Or
> > would you recommend a different approach?
> >
> >
> >   ----- Original Message -----
> >   From: Gavin King
> >   To: Christoph Sturm
> >   Cc: hibernate list
> >   Sent: Monday, November 25, 2002 2:23 PM
> >   Subject: Re: slow performance on flush
> >
> >
> >   Yup, all entries are kept in the session cache until you close the
> > session. Why would you use this kind of code?? Just flush() once at the
> end
> > (outside the loop).
> >     ----- Original Message -----
> >     From: Christoph Sturm
> >     To: hib...@li... ; Gavin King
> >     Sent: Tuesday, November 26, 2002 12:18 AM
> >     Subject: slow performance on flush
> >
> >
> >     Hi all!
> >
> >     I'm trying to increase the performance of this pseudocode:
> >
> >     for (j=1;j<100;j++)
> >     {
> >         for (i=1;i<100;i++)
> >         {
> >             session.save(class)
> >         }
> >         session.flush();
> >         session.connection().commit();
> >     }
> >
> >
> >     Now my problem is that flush doesnt take a constant time, but every
> > invocation of flush takes more time than the last one, the more I flush
> the
> > slower it gets. To me it looks like hibernate keeps the references to
the
> > flushed objects in the entries map, and doesnt remove them. Is there
> > anything I can do about it?
> >
> >     regards
> >      chris
> >
> >
> >
> >
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > hibernate-devel mailing list
> > hib...@li...
> > https://lists.sourceforge.net/lists/listinfo/hibernate-devel
> >
>