<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0
Transitional//EN">
<HTML><HEAD>
<STYLE type=text/css> P, UL, OL, DL, DIR,
MENU, PRE { margin: 0 auto;}</STYLE>
<META content="MSHTML 6.00.2900.2802" name=GENERATOR></HEAD>
<BODY leftMargin=1 topMargin=1 rightMargin=1><FONT
face=Tahoma>
<DIV><FONT face=Arial size=2>Bryan-</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT size=2>>> Just to be clear,
the design that I proposed did not allow
the possibility<BR>>> one thread running
more than one transaction.</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>Right - that's
why I'm proposing a subtle change to the
approch where locks are transaction based
and not thread based.</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>Alex - In the
following discussion, notice that we are
having to do a lot of asserts in the transaction
methods - this is one of the reasons that
I was thinking we should just pass the transaction
into the calls on the lock manager...
If we do that, it gets rid of the setCurrentThreadTransaction
and releaseCurrentThreadTransaction completely,
and keeps the context of the locks straight...
I guess I'm still not entirely convinced
that using the thread local variable is a
good thing here...</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT></DIV>
<DIV><FONT face=Arial size=2>Anyway, back
to the proposed implementation using thread
local:</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>Bryan - in this
design, setCurrentThreadTransaction must
be called before the API is used for performing
locking operations on a given transaction.
</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>In extremely
rough pseudo code, here is how these methods
would be used. There may very well
be a good way to refactor this (not sure
if the lock manager reference belongs in
the transaction object itself), but it gets
the idea across:</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>void startTransaction(){</FONT></DIV>
<DIV><FONT face=Arial size=2> lockMgr.setCurrentThreadTransaction(this);
// this will throw exception if there is
already a current tx for the thread</FONT></DIV>
<DIV> <FONT face=Arial
size=2>return tx;</FONT></DIV>
<DIV><FONT face=Arial size=2>}</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>void suspendTransaction(){</FONT></DIV>
<DIV><FONT face=Arial size=2> Transaction
oldTx = lockMgr.releaseCurrentThreadTransaction();</FONT></DIV>
<DIV><FONT face=Arial size=2>
assert(oldTx == this); // if this is not
true, something bad happened</FONT></DIV>
<DIV><FONT face=Arial size=2>}</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>void resumeTransaction(){</FONT></DIV>
<DIV><FONT face=Arial size=2> lockMgr.setCurrentThreadTransaction(this);
// this will throw exception if there is
already a current tx for the thread</FONT></DIV>
<DIV><FONT face=Arial size=2>}</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>void DATA fetch(recid){</FONT></DIV>
<DIV><FONT face=Arial size=2> lockMgr.assertCurrentTransaction(this);
// this will throw an exception if the current
tx for the thread is not 'this'</FONT></DIV>
<DIV><FONT face=Arial size=2> lockMgr.lock(recid,
READ);</FONT></DIV>
<DIV> <FONT face=Arial
size=2>// do actual fetch</FONT></DIV>
<DIV><FONT face=Arial size=2>}</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>void commit(){</FONT></DIV>
<DIV><FONT face=Arial size=2> lockMgr.assertCurrentTransaction(this);</FONT></DIV>
<DIV> <FONT face=Arial
size=2>lockMgr.releaseLocks();</FONT></DIV>
<DIV> <FONT face=Arial
size=2>lockMgr.releaseCurrentThreadTransaction();</FONT></DIV>
<DIV> <FONT face=Arial
size=2>// do actual commit</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>}</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>Please note
that I'm not convinced that the above is
the way to go - but I am pretty sure that
dis-allowing overlapping transactions in
the same thread is not a design tradeoff
we want to make at this point...</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>- K</FONT></DIV>
<DIV><FONT face=Arial size=2> </FONT>
<TABLE>
<TBODY>
<TR>
<TD width=1 bgColor=blue><FONT face=Arial
size=2></FONT></TD>
<TD><FONT face=Arial size=2><FONT color=red>>
Kevin,<BR><BR>Just to be clear, the design
that I proposed did not allow the possibility<BR>of
one thread running more than one transaction.
Now, on to your proposal.<BR><BR>Can
you clarify the semantics of the following
when<BR>setCurrentThreadTransaction( Object
transaction ) has not been called? Is<BR>it
illegal to use the locking API in this case?<BR><BR>Also,
can you build out how you would use these
methods to construct the<BR>start/suspend/resume
semantics that we have been discussing?<BR><BR>-bryan<BR><BR>-----Original
Message-----<BR>From: <A href="mailto:jdbm-developer-admin@...
color=#0000ff>jdbm-developer-admin@...:
'JDBM Developer listserv '<BR>Sent: 3/28/2006
10:19 AM<BR>Subject: re[6]: [Jdbm-developer]
2PL: transactions, threads and lockin g<BR>(rese
nd!)<BR><BR>Bryan-<BR><BR>> We are straying
a little far away from this use case in our<BR>discussions
and<BR>> I would like to handle this before
getting into more rarified use<BR>cases.
We<BR>> can always refactor once
we have the basic mechanisms in place, right?<BR><BR>Actually,
I think that the current discussion is quite
important to<BR>making sure we don't get
painted into a corner that will be very<BR>difficult
to get out of. If we know up front
that the<BR>thread->transaction equivalence
in the current discussion is going to be<BR>a
problem, I think that we should consider
a different approach.<BR><BR>I'm not suggesting
that we actually design or write code for
the high<BR>concurrency, non blocking situation
- just that we don't intentionally<BR>make
a design decision that would preclude future
development in this<BR>area.<BR><BR>I think
that Alex's suggestion on one *current* transaction
per thread<BR>is a nice compromise that should
be relatively easy to include in the<BR>design.
I think this needs to be addressed
now, instead of pushing it<BR>out and attempting
to refactor later, because it will absolutely
impact<BR>the interface to the locking sub-system.<BR><BR>I
believe that this is relatively simple to
do - just have the locking<BR>subsystem have
the following additional calls:<BR><BR>setCurrentThreadTransaction(Object
transaction) - if there is already a<BR>current
transaction for the thread, this call throws
an exception.<BR>Otherwise, it sets the current
transaction for the thread in thread<BR>local
storage (<BR><A href="http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ThreadLocal.html"><FONT
color=#0000ff>http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ThreadLocal.html</FONT></A><BR><A
href="http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ThreadLocal.html"><FONT
color=#0000ff><http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ThreadLocal.html></FONT></A>
)<BR><BR>Object releaseCurrentThreadTransaction()
- Clears the thread local value<BR>and returns
the old transaction object that was stored
there. <BR><BR><BR>Once we have this in place,
all locking calls operate in the context
of<BR>the transaction stored in the thread
local variable, instad of using<BR>Thread.currentThread()
to obtain context.<BR><BR><BR>Another thing
to consider here: If we use thread->transaction<BR>equivalence,
and we do wind up with multiple overlapping
transactions<BR>running on the same thread
(for example, if we are running a unit test<BR>to
check things out), the lock system will fail.
I'd much rather see a<BR>deadlock occur
than the locking fail to operate properly
in this<BR>situation.<BR><BR>- K<BR> <BR> >
Kevin,<BR><BR>It seems to me that the core
use case for 2PL is to force serial<BR>execution<BR>of
concurrent processes when those processes
would result in<BR>incompatible<BR>access
to the same resource and to detect and abort
(or restart)<BR>transactions whose locking
requests would result in a deadlock<BR><BR>We
are straying a little far away from this
use case in our discussions<BR>and<BR>I would
like to handle this before getting into more
rarified use cases.<BR>We<BR>can always refactor
once we have the basic mechanisms in place,
right?<BR><BR>It is clear that we need to
continue what is essentially a requirements<BR>discussion
for jdbm transactions, but I would like to
take a phased<BR>approach<BR>to this so that
we do not demand up front more than we can
easily<BR>accomplish.<BR><BR>Does that make
sense?<BR><BR>-bryan<BR><BR>-----Original
Message-----<BR>From: <A href="mailto:jdbm-developer-admin@...
color=#0000ff><mailto:jdbm-developer-admin@...
href="mailto:jdbm-developer-admin@...
color=#0000ff>jdbm-developer-admin@...:
JDBM Developer listserv<BR>Sent: 3/28/2006
9:45 AM<BR>Subject: re[4]: [Jdbm-developer]
2PL: transactions, threads and lockin g<BR>(rese
nd!)<BR><BR>Bryan-<BR><BR>In the non-blocking
IO situation, the developer wouldn't make
a blocking<BR>call - that's why I explicitly
requested non-blocking lock checks. In<BR>that
situation, the lock request would fail, and
the system would queue<BR>the request for
the next cycle through - in effect, the state
machine<BR>for the request context would
switch from 'active processing' to 'try<BR>again'.<BR><BR>We
may very well find that using optimistic
locking is more appropriate<BR>for these
high performance non-blocking IO operation,
or we may find<BR>that providing a completely
separate non blocking interface to jdbm2
it<BR>the way to go. My goal here is
to make sure we don't make any design<BR>decisions
that would preclude that possibility in the
future (I<BR>definitely don't think we want
to include that in our first cut at<BR>developing
the system).<BR><BR>If we make a hard thread->transaction
1:1 mapping, this would definitely<BR>preclude
the possibility, so I want to make sure we
relax that<BR>requirement.<BR><BR><BR>Your
comments on locking vs synchronization are
dead on. These two<BR>concepts share
many attributes, but they are definitely
not the same. I<BR>think Alex's comment
regarding the concurrent package was that
if a<BR>developer *wanted* to implement synchronization
mechanisms to better<BR>manage the database
interaction, they could. There is often
information<BR>at the application level that
allows for more efficient processing of<BR>requests
than 2PL - for example, an application server
could have a<BR>business object that it synchronizes
when it makes changes to it. If<BR>you
have a ton of threads that could all be making
changes to that<BR>object at the same time,
synchronization would be a very good idea,
even<BR>though the low level locking in the
database would theoretically protect<BR>against
data corruption due to concurrent access.<BR><BR>Another
thing to consider in the locking library
is that maybe all of<BR>the methods should
be non blocking, but make a monitor object
available<BR>if a user of the framework wants
to actually block. The concurrent<BR>package
could be used for handling the details of
blocking, release,<BR>etc... and it keeps
the locking/blocking concerns separated...
just a<BR>thought...<BR><BR>In jdbm's
use of the locking library, we would almost
certainly block if<BR>a lock failed - at
least in 2PL mode.<BR><BR>Cheers!<BR><BR>-
K<BR><BR><BR> > <BR>Let
me try to summarize. There are two
reasons why start/suspend/resume<BR>might
be required: non-blocking single threaded
IO and passing<BR>responsibility for executing
a transaction down a chain of thread pools<BR>associated
with a staged processing model. The
other use case that I<BR>find interesting
is parallel processing. <BR><BR>The
staged processing use case makes the most
sense, but I wonder what a<BR>transaction
restart would look like under that framework?
It seems that<BR>not only would the
tx need to restart, but it would need to
restart at<BR>the first thread pool of the
staged processor. <BR><BR>I am not convinced
that a persistence layer that is using a
locking<BR>strategy is suitable for a non-blocking
single threaded process. If any<BR>lock
request blocks, then the entire server will
halt without a chance<BR>to resume. This
seems unworkable. My other concern
with this scenario<BR>is that the latency
involved in disk IO can be quite large and
is not<BR>necessarily predicable, e.g., an
incremental write could occur at an<BR>arbitrary
moment. Such latency could cause the
server to drop messages,<BR>which also seems
unworkable. Please let me know if I
am wrong, but I do<BR>not see how jdbm could
be used in such an environment, especially
in<BR>conjunction with a locking strategy
for managing resource access<BR>contention.
<BR><BR>However, I would also like to talk
more about the 2PL locking model. I<BR>have
been designing based on the early work by
Gray and Bayer covering<BR>2PL, intention
and implicit locking, and online deadlock
detection<BR>algorithms. This is quite
different from java.util.concurrent in two<BR>ways.
First, the lock modes are not just
READ (Share) and WRITE<BR>(Exclusive), but
also include intention and implicit lock
modes.<BR>Second, deadlocks are detected
using an induced WAITS_FOR graph among<BR>the
transactions that are blocked. There
is no comparable feature in<BR>java.util.concurrent
the classes in those packages are not
design to<BR>support deadlock detection.
<BR><BR>I have been writing up the package
semantics for the 2PL support [1].<BR>Some
feedback on this would be useful. <BR><BR>Thanks,
<BR><BR>-bryan <BR><BR>[1]<BR><BR><A href="http://cvs.sourceforge.net/viewcvs.py/cweb/concurrent/src/java/org/Cogn"><FONT
color=#0000ff><http://cvs.sourceforge.net/viewcvs.py/cweb/concurrent/src/java/org/Cogn</FONT></A><BR>><BR><A
href="http://cvs.sourceforge.net/viewcvs.py/cweb/concurrent/src/java/org/Cogn"><FONT
color=#0000ff><http://cvs.sourceforge.net/viewcvs.py/cweb/concurrent/src/java/org/Cogn</FONT></A><BR>itiveWeb/concurrent/locking/package.html?rev=1.3&view=log><BR><BR><A
href="http://cvs.sourceforge.net/viewcvs.py/cweb/concurrent/src/java/org/Cogn"><FONT
color=#0000ff><http://cvs.sourceforge.net/viewcvs.py/cweb/concurrent/src/java/org/Cogn</FONT></A><BR>i><BR><A
href="http://cvs.sourceforge.net/viewcvs.py/cweb/concurrent/src/java/org/Cogni"><FONT
color=#0000ff>http://cvs.sourceforge.net/viewcvs.py/cweb/concurrent/src/java/org/Cogni</FONT></A><BR>tiveWeb/concurrent/locking/package.html?rev=1.3&view=log
<BR><BR><BR><BR>From: <A
href="mailto:jdbm-developer-admin@...
color=#0000ff><mailto:jdbm-developer-admin@...
href="mailto:jdbm-developer-admin@...
color=#0000ff><mailto:jdbm-developer-admin@...
href="mailto:jdbm-developer-admin@...
color=#0000ff><mailto:jdbm-developer-admin@...
href="mailto:jdbm-developer-admin@...
color=#0000ff>jdbm-developer-admin@...
href="mailto:jdbm-developer-admin@...
color=#0000ff><mailto:jdbm-developer-admin@...
href="mailto:jdbm-developer-admin@...
color=#0000ff><mailto:jdbm-developer-admin@...
href="mailto:jdbm-developer-admin@...
color=#0000ff><mailto:jdbm-developer-admin@...
href="mailto:jdbm-developer-admin@...
color=#0000ff>[mailto:jdbm-developer-admin@...>
On Behalf Of Kevin<BR>Day<BR>Sent: Tuesday,
March 28, 2006 12:06 AM<BR>To: JDBM Developer
listserv<BR>Subject: re[2]: [Jdbm-developer]
2PL: transactions, threads and locking<BR>(rese
nd!) <BR><BR><BR>Alex- <BR><BR><BR><BR>OK
- I wrote a big long email explaining my
thoughts, and finally<BR>managed to work
myself around to understanding where you
are coming<BR>from... <BR><BR><BR><BR>Suspend
and resume does indeed take care of the issue.
If each thread<BR>has a single active
transaction, then that thread will block
as soon as<BR>it hits a lock that doesn't
belong to that transaction. The key
is that<BR>if a tx is blocked, it can't possibly
be switched to a different thread<BR>(this
is what the suspend call is providing for),
which ensures that we<BR>can't have multiple
threads making calls against the same tx
at the same<BR>time. At this point,
blocking becomes a matter of simple wait/notify
on<BR>the lock object - unless we want to
implement some sort of priority<BR>based
notification (which we might...). <BR><BR><BR><BR>This
means that using thread local storage for
the transaction does make<BR>sense. I
also think that the transaction itself will
need to store it's<BR>associated thread for
use in checks to ensure that the user doesn't<BR>continue
to use a tx after it suspends it. I
suppose another thing that<BR>could be done
here would be to make the resume call implicit
(we are<BR>already going to have to check
each<BR>Transaction.insert/update/delete/commit
call to ensure that the current<BR>thread
is correct). <BR><BR><BR><BR>Not sure about
the performance implications of actually
performing that<BR>check for each call...
<BR><BR><BR><BR>Thanks guys, sorry to slow
the process down. <BR><BR><BR><BR>- K <BR><BR><BR><BR><BR>>
Kevin Day wrote: <BR>Interesting. Does
this imply that there can only be one executing
Tx<BR>per thread at any given point in time?
I think this starts to break<BR>down,
though, because you can't release the locks
when you suspend a Tx<BR>(not without viloating
2PL, anyway). I don't understand. Can
you<BR>provide an example of what you mean?<BR><BR>No
matter what Bryan does, he still has to deal
with the fact that a<BR>single thread could
wind up needing to block itself (by definition,
this<BR>would be deadlock). This scenario
is pretty easy to detect, and should<BR>probably
abort one of the transactions (it implies
that the application<BR>developer has not
designed their threading/transaction handling<BR>properly).
In the scheme I was proposing, a transaction
holds locks, not<BR>a thread. If
a transaction holds a lock, it cannot block
itself. You<BR>can only get into a
deadlock situation with two or more transactions.<BR><BR>(If
you want locks for synchronizing threads,
you can use<BR>java.util.concurrency or something
alike)<BR><BR>alex<BR><<BR><<BR> <BR>-------------------------------------------------------
This SF.Net<BR>email is sponsored by xPML,
a groundbreaking scripting language that<BR>extends
applications into web and mobile media. Attend
the live webcast<BR>and join the prime developer
group breaking into this new coding<BR>territory!<BR><BR><A
href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=12164"><FONT
color=#0000ff><http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=12164</FONT></A><BR>2><BR><A
href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642"><FONT
color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642</FONT></A><BR>_______________________________________________
Jdbm-developer mailing<BR>list <A href="mailto:Jdbm-developer@...
color=#0000ff><mailto:Jdbm-developer@...
href="mailto:Jdbm-developer@...
color=#0000ff>Jdbm-developer@...
href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT
color=#0000ff><https://lists.sourceforge.net/lists/listinfo/jdbm-developer></FONT></A><BR><A
href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT
color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A><BR><BR><<BR> <BR>-------------------------------------------------------
This SF.Net<BR>email is sponsored by xPML,
a groundbreaking scripting language that<BR>extends
applications into web and mobile media. Attend
the live webcast<BR>and join the prime developer
group breaking into this new coding<BR>territory!<BR><A
href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642"><FONT
color=#0000ff>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642</FONT></A><BR>_______________________________________________
Jdbm-developer mailing<BR>list <A href="mailto:Jdbm-developer@...
color=#0000ff>Jdbm-developer@...
href="https://lists.sourceforge.net/lists/listinfo/jdbm-developer"><FONT
color=#0000ff>https://lists.sourceforge.net/lists/listinfo/jdbm-developer</FONT></A><BR><BR><<BR></FONT></FONT></TD></TR></TBODY></TABLE></DIV></FONT></BODY></HTML>
|