modeling-users

[Modeling-users] Rollback?

[Wolfgang K. <wol...@gm...>]

Hello,

just out of curiosity:

Is the possibility to rollback changes on the roadmap of Modeling? This is 
for the case where several instances of an application using modeling run in 
parallel at different sites, and a transaction needs to be distributed over 
more than one instance of the application...

And, what would happen in Modeling if a different application, which does not 
pass through Modeling, but connects directly to the underlying database, 
modifies records in this database? Does Modeling go so far and notice when a 
lock on a record is acquired by a different application, lock the 
corresponding object in memory, and then update the objects in memory with 
the changes the other applications makes to the database? Or is such a 
function totally utopian? *duck*

TIA,

Sincerely,

Wolfgang Keller

-- 
My email-address is correct.
Do NOT remove ".nospam" to reply.

Re: [Modeling-users] Rollback?

[<sbi...@us...>]

    Hi Wolgang & all,

Wolfgang Keller wrote:
> Hello,
>=20
> just out of curiosity:
>=20
> Is the possibility to rollback changes on the roadmap of Modeling? This=
 is=20
> for the case where several instances of an application using modeling r=
un in=20
> parallel at different sites, and a transaction needs to be distributed =
over=20
> more than one instance of the application...

I'm not sure to understand what you mean by "a transaction needs to be di=
stributed over more than one instance". If what follows doers not answer =
your question, or if you need further explanations, please do not hesitat=
e to ask.

For the moment being, an EditingContext (EC) is the abstraction, at the o=
bject level, of a transaction. One EC, one transaction, that can be commi=
tted (hence, producing a transaction at the database level), or rolled ba=
ck.  You can also have sub-tranbsactions at the object level (see http://=
modeling.sourceforge.net/UserGuide/nested-editing-context.html).

Now as far as I understand it, your next question is about concurrent mod=
ifications in different ECs. That can happen: in the same process, differ=
ent threads, or in different processes --see below.

> And, what would happen in Modeling if a different application, which do=
es not=20
> pass through Modeling, but connects directly to the underlying database=
,=20
> modifies records in this database? Does Modeling go so far and notice w=
hen a=20
> lock on a record is acquired by a different application, lock the=20
> corresponding object in memory, and then update the objects in memory w=
ith=20
> the changes the other applications makes to the database? Or is such a=20
> function totally utopian? *duck*

The features you ask for is partially addressed by a patch described here=
:

  http://permalink.gmane.org/gmane.comp.python.modeling/374

although that they are addressed in a different but (imho) equivalent way=
, as described below.

With this patch:

- an EditingContext saving changes broadcasts its changes to others, who =
in turn merge the committed changes with their own, uncommitted, changes.
(for now, this only works within the same process, with e.g. ECs dispatch=
ed in different threads, but NOT for ECs that are created in different pr=
ocesses)

- if, as you described, an external application changes some database val=
ues, any EC saving its changes will receive an OptimisticLockingFailure e=
xception when trying to save changes on some of the externally modified d=
atas.
In this case: external application=3D=3Dan other process; it can be anyth=
ing modifying the db, and particularly: an other application using the mo=
deling framework.

This is obviously the first step, in further development the externally m=
odified objects would be sent back to the EC so that you can choose to re=
fresh the values, examine the differences and accept or discard the chang=
es.


Explicit database-locking would be called Pessimlistic Locking, where mod=
ifying an object implies acquiring a lock on the corresponding database c=
olumn(s) until changes are saved (or discarded).  This feature is on the =
TODO list, but development has not started yet.


-- S=E9bastien.


Wolfgang Keller wrote:
> Hello,
>=20
> just out of curiosity:
>=20
> Is the possibility to rollback changes on the roadmap of Modeling? This=
 is=20
> for the case where several instances of an application using modeling r=
un in=20
> parallel at different sites, and a transaction needs to be distributed =
over=20
> more than one instance of the application...
>=20
> And, what would happen in Modeling if a different application, which do=
es not=20
> pass through Modeling, but connects directly to the underlying database=
,=20
> modifies records in this database? Does Modeling go so far and notice w=
hen a=20
> lock on a record is acquired by a different application, lock the=20
> corresponding object in memory, and then update the objects in memory w=
ith=20
> the changes the other applications makes to the database? Or is such a=20
> function totally utopian? *duck*
>=20
> TIA,
>=20
> Sincerely,
>=20
> Wolfgang Keller
>=20

Re: [Modeling-users] Rollback?

[Wolfgang K. <wol...@gm...>]

>> Is the possibility to rollback changes on the roadmap of Modeling? This is 
>> for the case where several instances of an application using modeling run 
>> in 
>> parallel at different sites, and a transaction needs to be distributed over 
>> more than one instance of the application...
> 
> I'm not sure to understand what you mean by "a transaction needs to be 
> distributed over more than one instance".

Well, a distributed transaction.

Two database servers, two applications, one application on each server, both 
applications use modeling for persistence and are connected to each other 
through RPC.

But I figured out that it will be more efficient to simply link the 
underlying databases with multi-master replication instead of doing it at the 
application level.

So don't bother for my stupid question...

> Explicit database-locking would be called Pessimlistic Locking, where 
> modifying an object implies acquiring a lock on the corresponding database 
> column(s) until changes are saved (or discarded).  This feature is on the 
> TODO list, but development has not started yet.

Any realistic estimate when it will be available?

In a transaction-intensive application, not having pessimistic locking will 
mean getting lots of transaction failures, I guess? And lots of handwork to 
clean up afterwards...

What I am in fact looking for is something that allows transactions at the 
object level to be as ACID as PostgreSQL transactions at the database level.

Am I just dreaming?

TIA,

SIncerely,

Wolfgang Keller

-- 
My email-address is correct.
Do NOT remove ".nospam" to reply.

Re: [Modeling-users] Rollback?

[<sbi...@us...>]

Wolfgang Keller wrote:
[...]
>> Explicit database-locking would be called Pessimlistic Locking, where=20
>> modifying an object implies acquiring a lock on the corresponding data=
base=20
>> column(s) until changes are saved (or discarded).  This feature is on =
the=20
>> TODO list, but development has not started yet.
>=20
> Any realistic estimate when it will be available?
>=20
> In a transaction-intensive application, not having pessimistic locking =
will=20
> mean getting lots of transaction failures, I guess? And lots of handwor=
k to=20
> clean up afterwards...

You're right, if the application is very busy, you'll get a lot of failur=
es.
However, it's really application-specific: if most conflicts can be progr=
ammatically solved your application will softly behave under optimistic l=
ocking; if not, agreed, pessimistic locking is needed.

But, given that pessimistic locking locks any data that is going to be up=
dated, if your application is transaction-intensive updating the same dat=
a, you'll get a lot of processes waiting at the same time.

To answer your question, no idea on a ETA for pessimistic locking --we've=
 not started implementing it.

> What I am in fact looking for is something that allows transactions at =
the=20
> object level to be as ACID as PostgreSQL transactions at the database l=
evel.
>=20
> Am I just dreaming?

No you're not: the framework bring the ACI properties at the object-level=
, and the Durability property on standard (non-nested) EditingContexts.

  http://modeling.sourceforge.net/UserGuide/nested-editing-context.html
  http://modeling.sourceforge.net/UserGuide/nested-ec-ACI.html

However, I understand what you mean here; you must be thinking of transac=
tions being ACID when opened at the object level "down to the database". =
 But we're not at the same level; it won't even help if the framework had=
 something like a central server on top of the db & shared among differen=
t processes since the data still can be updated directly at the database =
level by any other process.

That must be the reason for wanting pessimistic locking. The pb. with pes=
simistic is that a transaction at the object level are likely to last muc=
h longer than standard SQL transactions (following the different steps of=
 updating data through numerous web pages for example), so in the meantim=
e, all other app. are denied starting any updates.


Well, I'll stop here, but if you really need to deal with concurrent upda=
tes of the same data I'll summarize the siuation:

- no locking (for now, the default): concurrent updates override previous=
 changes, not even detecting that another process has changed the data (p=
lease note the framework DO broadcast changes between threads)

- optimistic locking: you'll get notifications when saving changes, givin=
g you the opportunity to examine the change and, e.g., automatically hand=
le them or provide feedback to the user.  As explained in a previous post=
, implementation has started: despite some limitations (w/ nested ECs and=
 the handling of deleted objects as well), it is in a sable state.

- pessimistic locking: locks rows before updating objects' attributes. No=
t started yet. You may want to bypass the lack of this feature by impl. y=
our own, simple, locking mechanism. For example, by overriding willChange=
() [ http://modeling.sourceforge.net/UserGuide/node75.html ]  and acquiri=
ng a global lock in the database  --agreed, this is simple, but locking s=
pecific rows (using info. of their GlobalID) needs a lot of care to avoid=
 deadlocks.


-- S=E9bastien.

Re: [Modeling-users] Rollback?

[Wolfgang K. <wol...@gm...>]

> But, given that pessimistic locking locks any data that is going to be 
> updated, if your application is transaction-intensive updating the same data, 

> you'll get a lot of processes waiting at the same time.

"Waiting for Godot"

Or, as users of SAP call it in German; "Sanduhr-Anzeige-Programm" - 
Hourglass-Display-Program. >:->
 
> it won't even help if the framework had something like a central server on 
> top of the db & shared among different processes since the data still can 
> be updated directly at the database level by any other process. 

Well, I can forbid that for the main database of my application.
 
> - optimistic locking: you'll get notifications when saving changes,

How does this work?

> giving you the opportunity to examine the change and, e.g., automatically 
> handle them or provide feedback to the user. 

That would be what I am looking for.

Sincerely,

Wolfgang Keller

-- 
My email-address is correct.
Do NOT remove ".nospam" to reply.

Re: [Modeling-users] Rollback?

[<sbi...@us...>]

Sorry for the very late answer, for an unknown reason I did not see your =
post.

> Well, I can forbid that for the main database of my application.
> =20
>> - optimistic locking: you'll get notifications when saving changes,
>=20
> How does this work?

I was thinking of a delegate method being called by saveChanges() when it=
 notices that changes have happened in the database since the last time t=
he DB was queried (meaning that someone else has modified the db).

The delegate will receive a full view of observed modifications: the prop=
erties that have changed (attributes+relationships), objects that have be=
en deleted.  With this information any possible processing is possible: c=
ustom fusion of local+db modifications, notifying the user of a conflict =
and asking him how to solve it, ignoring changes and overriding the db ch=
anges, any combination of the previous solutions... In any case, the dele=
gate will decide whether the transaction opened by saveChanges() should b=
e cancelled for real or re-submitted after conflicts have been solved.

-- S=E9bastien.



Wolfgang Keller wrote:
>> But, given that pessimistic locking locks any data that is going to be=
=20
>> updated, if your application is transaction-intensive updating the sam=
e data,=20
>=20
>> you'll get a lot of processes waiting at the same time.
>=20
> "Waiting for Godot"
>=20
> Or, as users of SAP call it in German; "Sanduhr-Anzeige-Programm" -=20
> Hourglass-Display-Program. >:->
> =20
>> it won't even help if the framework had something like a central serve=
r on=20
>> top of the db & shared among different processes since the data still =
can=20
>> be updated directly at the database level by any other process.=20
>=20
> Well, I can forbid that for the main database of my application.
> =20
>> - optimistic locking: you'll get notifications when saving changes,
>=20
> How does this work?
>=20
>> giving you the opportunity to examine the change and, e.g., automatica=
lly=20
>> handle them or provide feedback to the user.=20
>=20
> That would be what I am looking for.
>=20
> Sincerely,
>=20
> Wolfgang Keller
>=20