hibernate-devel — development

RE: [Hibernate-devel] RE: Pessimistic locking?

[Gavin_King/Cirrus%<CI...@ci...>]

> What are the benefits in doing a "select for update" on the PK
> after the data has been loaded? Why not just write the data and let
> the database upgrade the lock automatically?

woops. i misunderstood the sense of this question....

This would be of benefit for applications using longer-running
transactions and also for applications using "long transactions
with versioning", where you could use it to check the version number
of an object you do not necessarily plan to update.

RE: [Hibernate-devel] RE: Pessimistic locking?

[Gavin_King/Cirrus%<CI...@ci...>]

> I don't understand your claim that post-load lock upgrades are a
> fundamental requirement of pessimistic locking. In databases that
> allow reads of rows with write locks (e.g. default behavior in Oracle)
> that behavior would be considered a form of optimistic locking.

Oracle supports two isolation levels, read committed and serializable.

In ANY database, read committed would be equivalent to optimistic
locking.(Read committed is equivalent to "no read locks" which I think we
all agreed
to define as optimistic locking. It turns out that oracle actually
implements this internally using timestamps, which we know is another
feature
of optimistic locking.)

Now my understanding is that, contrary to Sybase and DB2, Oracle does not
use
read locks to implement serializable isolation. So you find out at
transaction commit time that the transaction was unserializable, rather
than having the transaction block halfway through. This does sound like
optimistic locking to me. I think postgres might also work this way.

P.S. I would personally be tempted to think of this as a misfeature of
oracle since it means that with very-high-concurrency data and a
high isolation level, the probability of transaction failure is
large. Perhaps I'm wrong on this; there certainly are performance
advantages to the non-blocking model.

> At least with Oracle, unless you obtain the write lock at select
> time the data may be altered before you update it. What are the
> benefits in doing a "select for update" on the PK after the
> data has been loaded? Why not just write the data and let the
> database upgrade the lock automatically?

Well, in other databases where there are read locks (and assuming
isolation is at least repeatable read), a "select for update" would obtain
a
write lock, instead of a read lock. The oracle situation would be a write
lock as opposed to *no* lock.....

> I'll be more specific. It is optimistic rather than pessimistic locking
> in databases like Oracle and others that (by default) allow reads
> while a row is write locked.

Fair enough.

Anyways the result of this whole discussion is everyone convinced me we
need to let the user specify "select for update" where appropriate. That
feature is now implemented.

I also agreed to stop calling "long transactions with versioning" simply
"optimistic locking". (I have often heard "optimistic locking" used to
mean this in OO-development circles, but its clearly less-than-specific.)

RE: [Hibernate-devel] RE: Pessimistic locking?

[Steve <st...@ba...>]

> -----Original Message-----
> From: Paul Szego [mailto:pau...@ne...]
> 
> Steve Bate wrote:
> > I prefer to have the pessimistic locking supported at load time --
> > either by the query language or through the persistence
> > API for loading objects. Allowing explicit lock upgrades after
> > loading would be nice for flexibility, but it also can lead to
> > deadlocks in the database if multiple objects are being loaded
> > and modified in a transaction.
> 
> Deadlocks are nothing new. Databases have deadlock detection, and 
> there's a mountain of litreature on how to avoid avoid it in the first 
> place.
> 
> It's not just "something nice" - it's a fundamental requirement if 
> you're using pessimistic locks.

I don't understand your claim that post-load lock upgrades are a
fundamental requirement of pessimistic locking. In databases that
allow reads of rows with write locks (e.g. default behavior in Oracle)
that behavior would be considered a form of optimistic locking.

See: http://www.orafaq.com/papers/locking.pdf

At least with Oracle, unless you obtain the write lock at select
time the data may be altered before you update it. What are the
benefits in doing a "select for update" on the PK after the 
data has been loaded? Why not just write the data and let the
database upgrade the lock automatically? I suppose the first approach
gives you a somewhat smaller window for the data to change before
the update. On the other hand, it requires another access to the
database. If you are developing applications with relatively long
transactions I could see behavior potentially being a benefit. 
The systems I work on are high rate transaction processing systems 
with short transactions. The deferred, explicit lock upgrades are a 
"nice to have" but not very useful in that context.

I understand that deadlocks are nothing new. As I'm sure you know,
one of the techniques for avoiding deadlock is to use pessimistic
locking and lock the objects in the same order in each client.
The immediate load-time locking is a requirement for this strategy.
A deferred lock upgrade could still lead to deadlocks. Even though
the database will detect and break these deadlocks, the customers
I work with prefer to avoid them in the first place.

> > It's also effectively optimistic
> > locking since the data initially read may have changed between
> > the time you load the object and the time you upgrade the lock.
> 
> Not necessarily true. That's what things like read locks, isolation 
> levels and cursor stability are all about.

I'll be more specific. It is optimistic rather than pessimistic locking
in databases like Oracle and others that (by default) allow reads 
while a row is write locked.

Regards,

Steve

[Hibernate-devel] pulling a few different threads together:

[Gavin_King/Cirrus%<CI...@ci...>]

As a logical extension to some of the stuff we've talked about and also
bringing in some other things people have asked about, I think I can
start to see a new approach to the whole relationship between sessions
and connections. We have already considered changing the commit()/cancel()
process to the more fine grained:

Session.flush()
Session.connection().commit()
Session.connection().rollback()
Session.close()

Now, to support a much more useable "optimistic locking with versioning"
scheme, we could add the following:

Session.disconnect()
Session.reconnect()

disconnect would return the connection to the pool and reconnect would
grab a new connection. This is (a) more efficient than and (b) more
useable than the suggested approach in the documentation which is to do
some selects with Connection.setAutocommit(true) and then start a
transaction to do the updates.

Session.disconnect() would relinquish any read locks (by releasing the
connection) and would need to be proceeded by a connection().commit()
if any updates had been performed.

Updates performed subsequent to a Session.reconnect() would depend upon
the checking of version numbers to maintain consistency. Write locks
could easily be re-obtained by using Session.lock() (which also, in
the current implementation, checks the version number).

How I envisage this stuff would be used is, for example, in a stateful
session bean where we can't keep a connection (or transaction) open
between invocations from the UI. The bean would disconnect the session
at the end of each operation and reconnect at the start. Currently you
would need a new Session each time which would force you to reload a
versioned object, even if all you were going to do was update it.

So the session would come to represent a real business transaction that
might span multiple database transactions (using optimistic locking
with versioning to enforce consistency).

I'm *really* keen on this approch, so shoot me down if necessary....

Re: [Hibernate-devel] RE: Pessimistic locking?

[Paul S. <pau...@ne...>]

Steve Bate wrote:
> I prefer to have the pessimistic locking supported at load time --
> either by the query language or through the persistence
> API for loading objects. Allowing explicit lock upgrades after
> loading would be nice for flexibility, but it also can lead to
> deadlocks in the database if multiple objects are being loaded
> and modified in a transaction.

Deadlocks are nothing new. Databases have deadlock detection, and 
there's a mountain of litreature on how to avoid avoid it in the first 
place.

It's not just "something nice" - it's a fundamental requirement if 
you're using pessimistic locks.

> It's also effectively optimistic
> locking since the data initially read may have changed between
> the time you load the object and the time you upgrade the lock.

Not necessarily true. That's what things like read locks, isolation 
levels and cursor stability are all about.

Regards, PaulS.

[Hibernate-devel] Session.lock() + Session.loadWithLock()

[Gavin_King/Cirrus%<CI...@ci...>]

I've added these two new methods to the Session interface.

Session.lock(object) - upgrade existing readlock (where supported)
Session.loadWithLock(clazz, id) - load the object using for update

when using loadWithReadlock(), hibernate reverts to its default
behaviour, hibernate.use_outer_join=false, because you can't use
"for update" with an outer join.

thanks for everyone's input.

Re: [Hibernate-devel] Questions about HiLo Keygenerator

[Gavin_King/Cirrus%<CI...@ci...>]

If this helps, I just added functionality to HiLoGenerator to let it
instantiate its own DriverManagerConnectionProvider. This still won't let
you use an application server datasource, but it will help you at least
temporarily in the other situation.

You can configure it like this:

<generator class="hilo.long">
  <param>oracle.jdbc.driver.OracleDriver</param>
  <param>jdbc:oracle:thin:@localhost:1521:test</param>
  <param>user</param>
  <param>pass</param>
  <param>tablename</param>
  <param>columname</param>
</generator>

Or simply:

<generator class="hilo.long">
  <param>oracle.jdbc.driver.OracleDriver</param>
  <param>jdbc:oracle:thin:@localhost:1521:test</param>
  <param>user</param>
  <param>pass</param>
</generator>

You need updates to the following files (from CVS)

cirrus.hibernate.id.HiLoGenerator
cirrus.hibernate.id.HiLoStringGenerator
cirrus.hibernate.map.RootClass

I hope this helps.

peace

Gavin

Re: [Hibernate-devel] Questions about HiLo Keygenerator

[Gavin_King/Cirrus%<CI...@ci...>]

> It seems to me that the HILo Key generator only works if Hibernate
> manages the jdbc connections, because it wants to open a separate
> connection. Why is that needed? Also the comments in the code say that
> it wont work with application server datasources. Why is that the case?

Yeah, sorry the code comment is wrong. It *should* read:

"only works when using DriverManagerConnectionProvider"

It doesn't work if you supply a connection + it doesn't work with a
datasource. The reason for this is that we would need HiLoGenerator
to be notified of a transaction rollback in the (rare) event that
the transaction got rolled back on the transaction we grabbed the
new hi value. Also, other sessions would be blocked from using the
generator until we knew the outcome of that transaction. All this
doesn't fit the overall architecture very well.

I actually didn't remember this earlier (even though I knew it the
last time I implemented a hi/lo generator) and earlier versions were
using the same connection as the session......

The solution is to have an alternate implementation of HiLoGenerator
that uses a stateless session bean with (TX_REQUIRES_NEW) to grab and
increment the hi value. Daniel has implemented this and will release it
when hes ready, i guess.....

> I think uuids are a bit ugly, and hibernate doesnt seem to support
> database native key generation, so I dont know whats the best way to
> generate pks right now.

My advice is:

implement your own id generator class (possibly as a stop-gap measure)
using whatever makes sense for your environment. Its _really_ easy to
implement IDGenerator .... like *one* method or something.....

there will certainly be more options in the next version.

sorry about this situation it was my dumb mistake....

[Hibernate-devel] Questions about HiLo Keygenerator

[Christoph S. <ch...@sc...>]

Hi All!

It seems to me that the HILo Key generator only works if Hibernate 
manages the jdbc connections, because it wants to open a separate 
connection. Why is that needed? Also the comments in the code say that 
it wont work with application server datasources. Why is that the case?

I think uuids are a bit ugly, and hibernate doesnt seem to support 
database native key generation, so I dont know whats the best way to 
generate pks right now.

regards
 chris

Re: [Hibernate-devel] Pessimistic locking?

[Paul S. <pau...@ne...>]

Gavin_King/Cirrus%CI...@ci... wrote:
>>It seems that way. There are formal definitions - optimistic concurrency
>>control schemes have been around for a while (and not like how you
>>describe).
>>
> 
> I went and had a good look on google to see how these terms are commonly
> used. Everyone agrees that optimistic locking means the application does
> some work by retaining a version number/timestamp and that pessimistic
> locking works by the database retaining a "lock" at read time. No-one seems
> terribly explicit as to whether a read lock constitutes a lock or not. I
> would have thought it did. Apparently you don't think so.... I would love
> to be sure of terminology here because when we come to document this I
> don't want to look clueless ;)
> 
> 
>>Check out: "Concurrency Control and Recovery in Database Systems" by
>>Bernstein, Hadzilacos & Goodman (1987).
>>
> 
> k, i will try to find it (p.s. i am a mathematician not a c.s. major so
> sometimes im missing authorative sources here)

There's better stuff out there (more recent). That one's just a good 
summary of what had happened up until that point. For more recent stuff, 
check out the Morgan Kaufmann series on Data Management Systems. The 
Transaction Processing Concepts and Techniques by Jim Gray is a classic 
(althoug almost 10 years old).

Somone else here might know better references?

>>At the very least, the lock() method on an already loaded object would
>>do the trick for starters. As long as the call makes its way through to
>>the database.
>>
> 
> Yup. After i posted that last email i wondered about this option. Its a
> nice one i think. Frees you to lock an object even sometime after you first
> retrieved it. All good. Im guessing its enough to just touch the row,
> right, ie. just select the primary key with "for update" to lock the whole
> row?

Yep. There is an "OF" operator in the FOR UPDATE clause that specifies 
columns, but it's only used to determine what tables will have rows locked.

One other thing: what about the "NOWAIT" operator? Comes in handy.

> Would probably be good to allow "for update" in query language also, since
> that would let you lock a bunch of rows in one query.

But you can't use it with the "DISTINCT" clause, so it might be an issue 
with the query stuff. Other restictions that exist in Oracle at least, 
and probably in other DB's (e.g can't use the group-by clause, not in 
sub-queries). Don't know which might be an issue here.

I have no idea, but is there any kind of "standard" out there (explicit 
or de-facto) that addresses this? SQL92 or OQL? JDO? Is it part of the 
query language or the API in the ODMG or JDO stuff? Or both?

PaulS.

Re: [Hibernate-devel] Pessimistic locking?

[Paul S. <pau...@ne...>]

Doug Currie wrote:
>>I went and had a good look on google to see how these terms are commonly
>>used. Everyone agrees that optimistic locking means the application does
>>some work by retaining a version number/timestamp and that pessimistic
>>locking works by the database retaining a "lock" at read time. No-one seems
>>terribly explicit as to whether a read lock constitutes a lock or not. I
>>would have thought it did.
>>
> 
> I agree.

Yep. The main issue I had was your description had optimistic locking 
spanning two transactions. The other minor detail is that timestamp or 
versioning is just one approach to optimistic locking.

> 
>>I would love to be sure of terminology here because when we come to
>>document this I don't want to look clueless ;)

I'm no expert - I'll be the first to admit it. And there is a lot of 
confusion, since the same terms are used to describe different things by 
different people, and different terms to describe the same thing.

Case in point: read lock vs. shared lock, or exclusive lock vs. update 
lock vs. write lock.

> I notice that Benjamin Chi started a thread in the open discussion
> called "Optimistic Locking with Versioning." I think that including
> the word "Versioning" in any description of the Hibernate optimistic
> locking feature will allay most confusion about what Hibernate is
> doing. Both "Optimistic Locking with Versioning" and "Long Transaction
> Support with Versioning" sound right to me.

Yep - optimistic locking is one thing, and long transaction support is 
another (orthogonal) thing.

PaulS.

[Hibernate-devel] RE: Pessimistic locking?

[Steve B. <ba...@te...>]

I prefer to have the pessimistic locking supported at load time --
either by the query language or through the persistence
API for loading objects. Allowing explicit lock upgrades after
loading would be nice for flexibility, but it also can lead to
deadlocks in the database if multiple objects are being loaded
and modified in a transaction. It's also effectively optimistic
locking since the data initially read may have changed between
the time you load the object and the time you upgrade the lock.

Steve

Re[2]: [Hibernate-devel] Pessimistic locking?

[Doug C. <de...@fl...>]

> I went and had a good look on google to see how these terms are commonly
> used. Everyone agrees that optimistic locking means the application does
> some work by retaining a version number/timestamp and that pessimistic
> locking works by the database retaining a "lock" at read time. No-one seems
> terribly explicit as to whether a read lock constitutes a lock or not. I
> would have thought it did.

I agree.

> I would love to be sure of terminology here because when we come to
> document this I don't want to look clueless ;)

I notice that Benjamin Chi started a thread in the open discussion
called "Optimistic Locking with Versioning." I think that including
the word "Versioning" in any description of the Hibernate optimistic
locking feature will allay most confusion about what Hibernate is
doing. Both "Optimistic Locking with Versioning" and "Long Transaction
Support with Versioning" sound right to me.

e

Re[2]: [Hibernate-devel] Pessimistic locking?

[Doug C. <de...@fl...>]

In my experience, when people say "optimistic locking" they can mean
many different things. My earliest exposure (70s) to the term was in
OS design where pessimistic locks aborted on contention, and
optimistic locks waited. It seems to have taken on slightly different
meanings in the RDB (SQL) and ODB worlds as well, as Gavin's and
Paul's definitions highlight.

Anyway, I have seem other O/R tools and ODBs call the technique Gavin
is describing (no lock, check version number on update) "long duration
transactions" and "optimistic locking."

>   * a global property for the database ie. hibernate.select_for_update=true
>   * on a class-by-class basis in the mapping file
>   * as a property of the individual session ie. Session.setSelectForUpdate

All too gross.

>   * as part of the query language ie. SELECT foo FROM CLASS f.Foo FOR UPDATE
>   * as part of the API ie. session.loadForUpdate(fooID)

These two are good. The dialect could have a new property to indicate
its support for this feature. If not available, ignore it.

e

Re: [Hibernate-devel] Pessimistic locking?

[Gavin_King/Cirrus%<CI...@ci...>]

> It seems that way. There are formal definitions - optimistic concurrency
> control schemes have been around for a while (and not like how you
> describe).

I went and had a good look on google to see how these terms are commonly
used. Everyone agrees that optimistic locking means the application does
some work by retaining a version number/timestamp and that pessimistic
locking works by the database retaining a "lock" at read time. No-one seems
terribly explicit as to whether a read lock constitutes a lock or not. I
would have thought it did. Apparently you don't think so.... I would love
to be sure of terminology here because when we come to document this I
don't want to look clueless ;)

> Check out: "Concurrency Control and Recovery in Database Systems" by
> Bernstein, Hadzilacos & Goodman (1987).

k, i will try to find it (p.s. i am a mathematician not a c.s. major so
sometimes im missing authorative sources here)

> At the very least, the lock() method on an already loaded object would
> do the trick for starters. As long as the call makes its way through to
> the database.

Yup. After i posted that last email i wondered about this option. Its a
nice one i think. Frees you to lock an object even sometime after you first
retrieved it. All good. Im guessing its enough to just touch the row,
right, ie. just select the primary key with "for update" to lock the whole
row?

Would probably be good to allow "for update" in query language also, since
that would let you lock a bunch of rows in one query.

What do we do about databases which don't support the for update syntax?
throw exception? ignore it? Is there some other way to lock a row in
sybase?

Re: [Hibernate-devel] Pessimistic locking?

[Paul S. <pau...@ne...>]

Gavin_King/Cirrus%CI...@ci... wrote:
> 
> Okay, heres what I mean by pessimistic/optimistic (you seem to be working
> from a different definition).

It seems that way. There are formal definitions - optimistic concurrency 
control schemes have been around for a while (and not like how you 
describe).

The basis of this stuff happened back in the late 70's / early 80's with 
Field Call's, which optimistic approaches are based on - the IBM guys 
had this stuff in IMS soon after then. Optimistic stuff was around in 
the mid to late 80's.

Check out: "Concurrency Control and Recovery in Database Systems" by 
Bernstein, Hadzilacos & Goodman (1987).

>>If the exclusive lock is only obtained in the second servlet that
>>handles the form being posted a possibility is that it fails. From the
>>the user's perspective this is optimistic locking behaviour.
>>
> 
>>What you need to be able to do is gain the exclusive lock in the first
>>servlet, the one that knows the user is planning to do the update.
>>
> 
> hmmmmmm.....this is not usually a recommended approach in web applications.

Extremely debatable...

> Anyways all that is beside the point.

... so I won't touch it ;)

> The point is that some people
> probably want to do a select for update (I would imagine people using MySQL
> would be very keen on it). So, how is this functionality best exposed to
> the user:
> 
>   * a global property for the database ie. hibernate.select_for_update=true
>   * on a class-by-class basis in the mapping file
>   * as a property of the individual session ie. Session.setSelectForUpdate
> ()
>   * as part of the query language ie. SELECT foo FROM CLASS f.Foo FOR
> UPDATE
>   * as part of the API ie. session.loadForUpdate(fooID)

I've had a brief look at other tools / frameworks / products for an idea 
of their approaches (e.g. JDO, ODMG, Castor, TOPLink, etc).

I don't like any global approaches - whether it's per database, class or 
session. These simply won't work. I don't want every read of my Product 
class from the product catalog to get an exclusive lock just because 
someon's reading the catalog.

What you really need is a way to lock a single object, and have that 
translate onto the correct SELECT ... FOR UPDATE call(s). For this only 
the last option you describe works. The others won't help at all.

Something like a different locking version of the find() and load() 
methods in the Session interface, and a way to lock an object already 
loaded. Also need to consider wait vs. nowait options (and timeout values?).

At the very least, the lock() method on an already loaded object would 
do the trick for starters. As long as the call makes its way through to 
the database.

I'm not proposing that Hibernate do any lock processing (which is what 
scared me about the cache discussions) - just allow the underlying 
database to do it.

What do you think? PaulS.

Re: [Hibernate-devel] caching algorithm

[Gavin_King/Cirrus%<CI...@ci...>]

> some general questions: where does this cache fit in? Is it per
> Datastore, per Session, or something else?

per session factory.

> What's the intended purpose of the cache?

Well, people asked for it......

its intended to increase performance by caching data that changes rarely
(or is never changed by the application) between transactions. Many
applications will have a set of data that it modifies, and another set of
"reference" or "static" data that it can sensibly cache between
transactions. Possibly the static data changes rarely, or it changes on a
predictable cycle (daily, for example). There is no reason to force
hibernate to re-select that data each session. On the other hand, you could
easily mount a convincing case that the application itself should be
responsible for caching this stuff...

There is also data that is mostly read and very rarely updated. Its harder
for the application to manage its own cache in this situation. For
performance reasons, we would like to be able to get some caching going
here.

I am certainly *not* trying to implement caching for the case of data which
is frequently updated. I feel this a harder problem and the caching
algorithm i implemented doesn't improve performance in this case.

> This might sounds like a
> trivial question, but I can't tell from the discussion. A lot of it
> seems to be heading away from just a lightweight persistence layer.

Well, this is the line i have drawn: if implementing caching would require
any significant added complexity to the core persistence engine, or if it
would have a negative performance impact upon the vanilla case of no
caching, I will happily drop the idea. The way i've done it at the moment
satisifies these requirements. This is added to the other requirement that
hibernate never ever uses in-memory locks upon objects.

> Scared, PaulS :|

Yeah, yeah, i know, i know ..... its against the spirit of the project, it
cant scale to a cluster, I can see neither you nor Doug like it and i'm not
that keen on it myself. Its not hurting anything by staying there for the
moment, but I also promise not to document it for the moment. I also
promise we wont head further down this path (i never intended to,
actually).

Re: [Hibernate-devel] Pessimistic locking?

[Gavin_King/Cirrus%<CI...@ci...>]

> Without doing this you effectively get optimistic lock behaviour. That
> is, you find out at commit time whether everything is okay or not.
> Because the O/R layer will defer all SQL insert/update/delete statements
> until the last second, we don't have any exclusive locks as far as the
> underlying DB is concerned until that point, which from the Hibernate
> callers perspective is commit time.

Okay, heres what I mean by pessimistic/optimistic (you seem to be working
from a different definition).

pessimistic locking:

Retrieval and update of data occurs in one transaction. the database is
solely responsible for ensuring transaction isolation.

Take sysbase, set transaction isolation to serializable. start two
transactions. select the same rows from both transactions. Then, from one
transaction, try to do an update. That transaction will block, waiting to
see what the other transaction tries to do.

Whats happened is that both transactions acquire a read lock when you issue
the select. Any attempts to acquire a write lock (eg. the escalation when
you try to update) will block waiting for the other transaction to release
its read lock.

optimistic locking:

We retrieve the data in one transaction and then update it in another. To
ensure consistency, we check version numbers/timestamps/etc when we update.
In the previous example, the first transaction would have been allowed to
proceed with the update, since there was *no* lock held by the other
transaction. The second would have failed to update because its
timestamp/version number would be stale.

So the distinction between selection with or without "FOR UPDATE" (ie.
between a read lock and a write lock is different to my distinction between
optimistic (no lock) and pessimistic locking.

clearly there are shades of optimism.... :)


> If the exclusive lock is only obtained in the second servlet that
> handles the form being posted a possibility is that it fails. From the
> the user's perspective this is optimistic locking behaviour.

> What you need to be able to do is gain the exclusive lock in the first
> servlet, the one that knows the user is planning to do the update.

hmmmmmm.....this is not usually a recommended approach in web applications.
Generally people say transactions should not span "user-think-time". Still,
its appropriate in some other situations.....

Anyways all that is beside the point. The point is that some people
probably want to do a select for update (I would imagine people using MySQL
would be very keen on it). So, how is this functionality best exposed to
the user:

  * a global property for the database ie. hibernate.select_for_update=true
  * on a class-by-class basis in the mapping file
  * as a property of the individual session ie. Session.setSelectForUpdate
()
  * as part of the query language ie. SELECT foo FROM CLASS f.Foo FOR
UPDATE
  * as part of the API ie. session.loadForUpdate(fooID)

I figure, if paul is using this, other people will also want it.....

Gavin.

Re: [Hibernate-devel] caching algorithm

[Paul S. <pau...@ne...>]

Hi,

some general questions: where does this cache fit in? Is it per 
Datastore, per Session, or something else?

What's the intended purpose of the cache? This might sounds like a 
trivial question, but I can't tell from the discussion. A lot of it 
seems to be heading away from just a lightweight persistence layer.

Scared, PaulS :|

Gavin_King/Cirrus%CI...@ci... wrote:
> 
> Alright....ima put my head on the block here. Heres the caching algorithm:
> 
> 
> When Loading Objects
> ====================
> 1. grab the current timestamp (before-select)
> 2. issue SQL selects
> 3. hydrate objects
> 4. disassemble hydrated objects
> synchronized {
> 5. check that 'before-select' is greater than 'after-update' for that
> instance
> 6. if so, add the disassembled state to the cache
> }
> 
> Before Updating or Deleting
> ===========================
> synchronized {
> 1. remove cached object from cache
> 2. mark instance uncacheable, by incrementing counter (update-count)
> }
> 3. issue SQL Update or Delete, with a check for stale state
> 
> After Transaction Completion
> ============================
> synchronized {
> 1. decrement 'update-count' for uncacheable instance
> 2. if 'update-count'==0, set 'after-update' to system time
> }
> 
> Before A Load by ID
> ===================
> 1. look for cached state in cache
> 2. assemble object from cached state
> 3. call PersistentLifecycle.load(), if necessary
> 
> (There is a seperate cache per class, the synchronized blocks
> lock the cache itself.)
> 
> Are there bugs in this??

Re: [Hibernate-devel] Pessimistic locking?

[Paul S. <pau...@ne...>]

Gavin_King/Cirrus%CI...@ci... wrote:
> 
>>Another simple question then: how do I "lock" something? If I was doing
>>plain old JDBC, I'd use a "SELECT ... FOR UPDATE" with or without a
>>NOWAIT option.
>>
> 
> Okay, I almost brought this up myself earlier. I've been concerned about
> it for a while. My understanding is that most databases will lock rows
> automatically when necessary (according to the transaction isolation
> level). Hibernate never uses FOR UPDATE because some databases (eg.
> Sybase) don't even allow this clause on a select. However, I sort of
> thought that some other platforms (eg. MySQL) might need it. Does anyone
> know more about this than me? It will be simple enough to make hibernate
> always select for update on those databases which need it, or we can add
> findForUpdate() or loadForUpdate() to the API.
> 
> Are there other times when you *need* to use a select for update? ie.
> times when you *need* to grab a write lock immediately rather than
> escalating to a write lock when we issue an UPDATE?

Without doing this you effectively get optimistic lock behaviour. That 
is, you find out at commit time whether everything is okay or not. 
Because the O/R layer will defer all SQL insert/update/delete statements 
until the last second, we don't have any exclusive locks as far as the 
underlying DB is concerned until that point, which from the Hibernate 
callers perspective is commit time.

Consider a simple web app: a user selects something to update, a servlet 
(say, or  JSP, or whatever) grabs the info from the database and 
responds with the HTML page with a form to fill in. The user submits the 
form, and another servlet updates the DB.

If the exclusive lock is only obtained in the second servlet that 
handles the form being posted a possibility is that it fails. From the 
the user's perspective this is optimistic locking behaviour.

What you need to be able to do is gain the exclusive lock in the first 
servlet, the one that knows the user is planning to do the update.

The same applies no matter what the architecture: a fat Swing client 
talking directly to the DB has the same issues.

PaulS.

Re: [Hibernate-devel] Pessimistic locking?

[Gavin_King/Cirrus%<CI...@ci...>]

> Another simple question then: how do I "lock" something? If I was doing
> plain old JDBC, I'd use a "SELECT ... FOR UPDATE" with or without a
> NOWAIT option.

Okay, I almost brought this up myself earlier. I've been concerned about
it for a while. My understanding is that most databases will lock rows
automatically when necessary (according to the transaction isolation
level). Hibernate never uses FOR UPDATE because some databases (eg.
Sybase) don't even allow this clause on a select. However, I sort of
thought that some other platforms (eg. MySQL) might need it. Does anyone
know more about this than me? It will be simple enough to make hibernate
always select for update on those databases which need it, or we can add
findForUpdate() or loadForUpdate() to the API.

Are there other times when you *need* to use a select for update? ie.
times when you *need* to grab a write lock immediately rather than
escalating to a write lock when we issue an UPDATE?

This, for sure, is something I need a really definite answer on before
we go out of beta....

thanks for raising it Paul.

Re: [Hibernate-devel] Pessimistic locking?

[Paul S. <pau...@ne...>]

Gavin_King/Cirrus%CI...@ci... wrote:
>>Is it possible? There's mention of optimistic locking stuff, but not the
>>plain old vanilla pessimistic variety. Does hibernate handle this? Even
>>just by asking the underlying database to do it?
>>
> 
> Of course - thats the default behaviour.
> 
> If you supply your own JDBC connection, you need to setAutocommit(false)
> before opening the session. If you let hibernate open a connection, it will
> do that for you. (Well, in the case of a datasource, we assume it was
> already done by the appserver.)

Got that bit working so far.

> As you imply, Hibernate doesn't implement any concurrency management
> itself, it delegates everything to the database. So the transactional
> semantics are exactly what they would be if you were doing direct JDBC.

Another simple question then: how do I "lock" something? If I was doing 
plain old JDBC, I'd use a "SELECT ... FOR UPDATE" with or without a 
NOWAIT option.

I have no idea how to do this with Hibernate.

> (Ignore this whole "cross-transactional cache" thing, its *not* part of the
> core persistence engine and is disabled by default. I might even end up
> throwing it away.....)
>
> I could argue at length for the advantages of this approach over
> implementing concurrency management inside the object-relational layer, but
> you can probably already guess what i'm going to say. ;)

I agree - I like the approach of doing what you need to (O/R mapping), 
and letting the database do what it's good at (*lots* of really hard 
stuff - like locking).

PaulS.

Re: [Hibernate-devel] version 0.9.7

[Gavin_King/Cirrus%<CI...@ci...>]

> I think that we need to document the dependancies a little better, in
> particular what version's are required. Even better, what "spec" they
> need to implement.

That kind of documentation would be great. I have added the jars i'm using
to CVS. Unfortunately, i'm not sure exactly which are the earliest versions
that work.

Re: [Hibernate-devel] Pessimistic locking?

[Gavin_King/Cirrus%<CI...@ci...>]

> Is it possible? There's mention of optimistic locking stuff, but not the
> plain old vanilla pessimistic variety. Does hibernate handle this? Even
> just by asking the underlying database to do it?

Of course - thats the default behaviour.

If you supply your own JDBC connection, you need to setAutocommit(false)
before opening the session. If you let hibernate open a connection, it will
do that for you. (Well, in the case of a datasource, we assume it was
already done by the appserver.)

As you imply, Hibernate doesn't implement any concurrency management
itself, it delegates everything to the database. So the transactional
semantics are exactly what they would be if you were doing direct JDBC.
(Ignore this whole "cross-transactional cache" thing, its *not* part of the
core persistence engine and is disabled by default. I might even end up
throwing it away.....)

I could argue at length for the advantages of this approach over
implementing concurrency management inside the object-relational layer, but
you can probably already guess what i'm going to say. ;)

[Hibernate-devel] Pessimistic locking?

[Paul S. <pau...@ne...>]

Is it possible? There's mention of optimistic locking stuff, but not the 
plain old vanilla pessimistic variety. Does hibernate handle this? Even 
just by asking the underlying database to do it?

Regards, PaulS.