Menu
▾
▴
firebird-devel
|
From: Pavel C. <pc...@us...> - 2002-12-15 10:46:33
|
Hi, On 13 Dec 2002 at 19:01, Ann W. Harrison wrote: > The OIT, oldest interesting transaction, is the first transaction that did > not commit. It could be active, but more likely it rolled back or died and > was rolled back. The reason it is "interesting" is that other transactions > must ignore or undo its changes. Developers are often confused why rolled back transactions are "interesting". Actually, they just fall in "not committed" category, but more important are active and limbo trn. While active transactions are covered by OAT, limbo trn are not -> hence OIT (limbo trn can become committed in future!!!). I don't completely understand why rolled back trn are maintained through OIT in trn state table. FB could handle back- version made by transactions that are not in trn state table as rolled back and ignore/gc them, but FB handles all three in the same way. If I'm not mistaken, IB/FB has an optimisation that allows to advance OIT on rollback. Transactions maintain in-memory undo log of all changes made, and this log is used to unwind all changes on rollback, so OIT can advance (transaction is marked as committed). Transaction is marked as rolled back and thus interesting only when server goes up after abnormal termination (all active trn are marked as rolled back). So, the rollback command or when client dies does not freeze OIT, only server abend does. Of course, one can disable trn undo log in trn parameters. I'm also not sure if CS and SS behave the same way (but should). > Record versions created by the OAT and subsequent transactions can not > be garbage collected because they might be needed to provide a > consistent view for an active transaction. True, but that same apply for OIT transactions, because if they are limbo, they could be committed manualy. > The OIT is reset in one of two ways: a sweep or a backup and restore. True, "by the way" garbage collection can't reset OIT. To do that, all row versions made by transaction must be removed, and only sweep and gbak read whole database and thus GC all those versions. The big trn state table (due to big difference between OIT and actual transaction) only slow down a bit trn start, not the normal work. And of course, memory consumption is higher. But the real pain of stalled OIT is blocked GC and amount of back-versions that remain in database. If they run over from head-row datapage to other datapages, the performance goes south pretty quickly. > Read-only read-committed transactions do not affect the > OAT - they can go on forever without affecting garbage collection. In > this case, unlike the OIT, the transaction must be declared as read-only, > not just be de-facto read-only. I'd like to add that they are actually handled internaly as pre- committed. They should be a solution to OIT/ OAT problem caused by use of CommitRetaing in 7x24 apps. > The performance impact of the OIT, especially in the SS is probably not > significant. The impact of the OAT is quite significant, particularly > in applications that repeatedly update the same records. I think that impact of OIT is almost the same as OAT. Applications that update the same records are problem on its own, because they made exceptionally long back-version chains due to inefficient GC (we discussed that in ib-architect some time ago). > That effect could be reduced by some bookkeeping that told the server > which record versions were important so intermediate versions could be > removed. At the moment, it seems to me that the cost of that > bookkeeping and the complexity of removing intermediate record versions > is not worth the cost and risk. What do you mean by intermediate versions ? Many versions of single row that are made by one transaction ? I think that the bookkeeping is already there as undo log. It's primarily used to undo the changes made by procedures and triggers, but also at transaction level on rollback if undo log is not disabled (if it's disabled, it's still there for proc and triggers). Best regards Pavel Cisar http://www.ibphoenix.com For all your upto date Firebird and InterBase information |
|
From: Nickolay S. <sk...@bs...> - 2002-12-15 11:26:22
|
Hello, Pavel ! Sunday, December 15, 2002, 1:51:27 PM, you wrote: Just a little corrections where I have knowledge. > If I'm not mistaken, IB/FB has an optimisation that allows to advance OIT > on rollback. Transactions maintain in-memory undo log of all changes > made, and this log is used to unwind all changes on rollback, so OIT can > advance (transaction is marked as committed). Transaction is marked as > rolled back and thus interesting only when server goes up after abnormal > termination (all active trn are marked as rolled back). So, the rollback > command or when client dies does not freeze OIT, only server abend does. > Of course, one can disable trn undo log in trn parameters. I'm also not > sure if CS and SS behave the same way (but should). If amount of work done in transaction is large (~10000 rows modification or more) transaction-level savepoint is no longer used to undo transaction work on rollback. >> Record versions created by the OAT and subsequent transactions can not >> be garbage collected because they might be needed to provide a >> consistent view for an active transaction. > True, but that same apply for OIT transactions, because if they are > limbo, they could be committed manualy. >> The OIT is reset in one of two ways: a sweep or a backup and restore. > True, "by the way" garbage collection can't reset OIT. To do that, all > row versions made by transaction must be removed, and only sweep and gbak > read whole database and thus GC all those versions. > The big trn state table (due to big difference between OIT and actual > transaction) only slow down a bit trn start, not the normal work. And of > course, memory consumption is higher. But the real pain of stalled OIT is > blocked GC and amount of back-versions that remain in database. If they > run over from head-row datapage to other datapages, the performance goes > south pretty quickly. >> Read-only read-committed transactions do not affect the >> OAT - they can go on forever without affecting garbage collection. In >> this case, unlike the OIT, the transaction must be declared as read-only, >> not just be de-facto read-only. > I'd like to add that they are actually handled internaly as pre- > committed. They should be a solution to OIT/ OAT problem caused by use of > CommitRetaing in 7x24 apps. >> The performance impact of the OIT, especially in the SS is probably not >> significant. The impact of the OAT is quite significant, particularly >> in applications that repeatedly update the same records. > I think that impact of OIT is almost the same as OAT. Applications that > update the same records are problem on its own, because they made > exceptionally long back-version chains due to inefficient GC (we > discussed that in ib-architect some time ago). What prevents GC OAT or OIT is not significant for this discussion. Uncommited serializable transaction prevents GC efficiently and this is a security flaw. Malicious or simply uneducated user with no rights can cause this on a production database and shit is going to happen then (exceptionally in 24x7 usage scenario). >> That effect could be reduced by some bookkeeping that told the server >> which record versions were important so intermediate versions could be >> removed. At the moment, it seems to me that the cost of that >> bookkeeping and the complexity of removing intermediate record versions >> is not worth the cost and risk. > What do you mean by intermediate versions ? Many versions of single row > that are made by one transaction ? I think that the bookkeeping is > already there as undo log. It's primarily used to undo the changes made > by procedures and triggers, but also at transaction level on rollback if > undo log is not disabled (if it's disabled, it's still there for proc and > triggers). One transaction cannot create more then one version of a record in the database (that is the reason why explicit lock+update in one transaction makes essentialy no overhead over singular update). I think that Ann was talking about removal of intermediate records in a chain of back record versions and leave just enough versions to maintain consistency of all active snapshots. This would solve a particular problem of performance degradation when having old uncommited serializable transactions. But I don't think it is needed to implement because of following: 1. It is difficult (costly and risky if you want) 2. It is going to break internal database history consistency, i.e. you'll not be able to restore snapshot for recent committed transactions. This may be useful feature (it is implemented in Oracle 9i and is very popular among DBA). I still recommend to look at problem from security point of view and add ability of limit period of guarantied stapshot consistency (in hours, like in Oracle) and kill transaction when its snapshot is getting inconsistent. Oracle internals and operation seems very, very close to Firebird internals. Saying more, Firebird CS have better potential to scale then Oracle now. Firebird CS codebase is already cluster-ready and it should work exteremely nice in cluster because of very clever VIO code. I'm curious when first clustered solution will be implemented with Firebird... -- Best regards, Nickolay Samofatov mailto:sk...@bs... |
|
From: Ann W. H. <aha...@ib...> - 2002-12-15 18:31:16
|
At 02:24 PM 12/15/2002 +0300, Nickolay Samofatov wrote: >Oracle internals and operation seems very, very close to Firebird >internals. Saying more, Firebird CS have better potential to scale >then Oracle now. Firebird CS codebase is already cluster-ready and it >should work exteremely nice in cluster because of very clever VIO code. >I'm curious when first clustered solution will be implemented with >Firebird... You missed it by about 20 years. The architecture was designed for VAX clusters and ran there very nicely. The difference between VAX clusters and current cluster implementations is that VAXen had a distributed lock manager. With that piece, we'd run on clusters perfectly. Regards, Ann www.ibphoenix.com We have answers. |
|
From: Ann W. H. <aha...@ib...> - 2002-12-15 18:11:08
|
>On 13 Dec 2002 at 19:01, Ann W. Harrison wrote: > > > The OIT, oldest interesting transaction, is the first transaction that did > > not commit. At 11:51 AM 12/15/2002 +0100, Pavel Cisar wrote: >Hi, > >Developers are often confused why rolled back transactions are >"interesting". Actually, they just fall in "not committed" category, but >more important are active and limbo trn. Rolled back transactions are just as interesting as limbo and active transactions, well to me at least. >While active transactions are >covered by OAT, limbo trn are not -> hence OIT (limbo trn can become >committed in future!!!). Actually, what's interesting is that all three types of records require special handling. Records created by transactions that rolled back should be removed or at a minimum, ignored in favor of an earlier version, if any. Records created by transactions in limbo either cause an error or are ignored, depending on the state of the ignore_limbo transaction parameter. Records created by active transactions are ignored. >I don't completely understand why rolled back >trn are maintained through OIT in trn state table. Because that's the only way that a transaction knows that a record version was created by a transaction that failed. >FB could handle back- >version made by transactions that are not in trn state table as rolled >back and ignore/gc them, but FB handles all three in the same way. > >If I'm not mistaken, IB/FB has an optimisation that allows to advance OIT >on rollback. Not to advance it, but a rolled back transaction that made no changes is registered as committed. > Transactions maintain in-memory undo log of all changes >made, and this log is used to unwind all changes on rollback, so OIT can >advance (transaction is marked as committed). So if a transaction rolls back in a controlled manner and undoes all its changes, it too is marked as committed. > Transaction is marked as >rolled back and thus interesting only when server goes up after abnormal >termination (all active trn are marked as rolled back). Network errors that break connections, users typing control-alt-delete, any number of things can cause a transaction to fail in a way that leaves it unable to undo its changes. >So, the rollback >command or when client dies does not freeze OIT, I believe that client death does establish an OIT. > only server abend does. >Of course, one can disable trn undo log in trn parameters. I'm also not >sure if CS and SS behave the same way (but should). > > > Record versions created by the OAT and subsequent transactions can not > > be garbage collected because they might be needed to provide a > > consistent view for an active transaction. > >True, but that same apply for OIT transactions, because if they are >limbo, they could be committed manualy. > > > The OIT is reset in one of two ways: a sweep or a backup and restore. > >True, "by the way" garbage collection can't reset OIT. To do that, all >row versions made by transaction must be removed, and only sweep and gbak >read whole database and thus GC all those versions. Actually it requires both a backup and a restore. Gbak doesn't do the bookkeeping necessary to notice what transactions it has undone. Only sweep does that. >The big trn state table (due to big difference between OIT and actual >transaction) only slow down a bit trn start, not the normal work. And of >course, memory consumption is higher. But the real pain of stalled OIT is >blocked GC No. The OIT does not block garbage collection. A limbo record can not be garbage collected, of course, but neither can it be modified or deleted, so it's not going to start a long chain of back versions. > > Read-only read-committed transactions do not affect the > > OAT - they can go on forever without affecting garbage collection. In > > this case, unlike the OIT, the transaction must be declared as read-only, > > not just be de-facto read-only. > >I'd like to add that they are actually handled internaly as pre- >committed. They should be a solution to OIT/ OAT problem caused by use of >CommitRetaing in 7x24 apps. No, they aren't because read-committed transactions that update records need to register themselves as active to avoid dirty reads. > > The performance impact of the OIT, especially in the SS is probably not > > significant. The impact of the OAT is quite significant, particularly > > in applications that repeatedly update the same records. > >I think that impact of OIT is almost the same as OAT. Applications that >update the same records are problem on its own, because they made >exceptionally long back-version chains due to inefficient GC (we >discussed that in ib-architect some time ago). Right, but that effect - delayed garbage collection - is independent of the OIT. The OIT itself has no effect on garbage collection. > > That effect could be reduced by some bookkeeping that told the server > > which record versions were important so intermediate versions could be > > removed. At the moment, it seems to me that the cost of that > > bookkeeping and the complexity of removing intermediate record versions > > is not worth the cost and risk. > >What do you mean by intermediate versions ? Many versions of single row >that are made by one transaction ? I think that the bookkeeping is >already there as undo log. The intermediate versions are not created by a single transaction, but by a series of transactions. Suppose you start a consistency mode (snapshot) transaction. It stays open. The next 60 transactions all modify a single record, serially. Each starts, modifies the record and commits before the next transaction starts. At the end, you'll have 61 versions of that record. Only the oldest one and the newest are interesting. The 59 intermediate versions are just taking up space. Regards, Ann www.ibphoenix.com We have answers. |
|
From: Pavel C. <pc...@us...> - 2002-12-15 20:27:48
|
Ann, On 15 Dec 2002 at 13:10, Ann W. Harrison wrote: > >I don't completely understand why rolled back > >trn are maintained through OIT in trn state table. > > Because that's the only way that a transaction knows that > a record version was created by a transaction that failed. Yep, I had a temporary mind darkening :-) Transaction numbers that are not in transaction state table are committed ones. > > Transaction is marked as > >rolled back and thus interesting only when server goes up after abnormal > >termination (all active trn are marked as rolled back). > > Network errors that break connections, users typing control-alt-delete, > any number of things can cause a transaction to fail in a way that > leaves it unable to undo its changes. > > >So, the rollback > >command or when client dies does not freeze OIT, > > I believe that client death does establish an OIT. But when client dies and server notice, server performs the rollback, don't he ? If undo log is used on rollback to rewind changes and allow to mark transaction as committed instead, client or network failure should not affect OIT in most cases (every time undo log is used). > Actually it requires both a backup and a restore. Gbak doesn't do the > bookkeeping necessary to notice what transactions it has undone. Only > sweep does that. But when sweep is performed on backup ? I would suppose that restore is not necessary then. > >The big trn state table (due to big difference between OIT and actual > >transaction) only slow down a bit trn start, not the normal work. And of > >course, memory consumption is higher. But the real pain of stalled OIT is > >blocked GC > > No. The OIT does not block garbage collection. A limbo record can not > be garbage collected, of course, but neither can it be modified or deleted, > so it's not going to start a long chain of back versions. So you say that GC can expunge rolled back versions beyond OAT, leaving there just those that are not removable (i.e. limbo) ? > > > Read-only read-committed transactions do not affect the > > > OAT - they can go on forever without affecting garbage collection. In > > > this case, unlike the OIT, the transaction must be declared as read-only, > > > not just be de-facto read-only. > > > >I'd like to add that they are actually handled internaly as pre- > >committed. They should be a solution to OIT/ OAT problem caused by use of > >CommitRetaing in 7x24 apps. > > No, they aren't because read-committed transactions that update records > need to register themselves as active to avoid dirty reads. But I meant read-only (by definition in trn parameter block) read committed transactions you mentioned earlier. They internally acquire pre- committed status, don't they ? > The intermediate versions are not created by a single transaction, but > by a series of transactions. Suppose you start a consistency mode > (snapshot) transaction. It stays open. The next 60 transactions all > modify a single record, serially. Each starts, modifies the record and > commits before the next transaction starts. > > At the end, you'll have 61 versions of that record. Only the oldest > one and the newest are interesting. The 59 intermediate versions > are just taking up space. I see. That would really make a difference. But does this really need a bookkeeping ? How about an extension logic in VIO_chase_record_version that will notice subsequent versions from committed transactions and mark it for GC (and extended GC to handle that, of course) ? Best regards Pavel Cisar http://www.ibphoenix.com For all your upto date Firebird and InterBase information |
|
From: Ann W. H. <aha...@ib...> - 2002-12-16 03:44:26
|
At 09:32 PM 12/15/2002 +0100, Pavel Cisar wrote: > > > > I believe that client death does establish an OIT. > >But when client dies and server notice, server performs the rollback, >don't he ? The server changes the state of the transaction to rolled back, but the undo log is not available (or reliable) so the transaction can affect the OIT> > > Actually it requires both a backup and a restore. Gbak doesn't do the > > bookkeeping necessary to notice what transactions it has undone. Only > > sweep does that. > >But when sweep is performed on backup ? I would suppose that restore is >not necessary then. Sweep is a different operation than backup. Sweep moves the OIT, backup alone does not. Backup and restore does, of course, by recreating the database with all new transaction id's. >So you say that GC can expunge rolled back versions beyond OAT, leaving >there just those that are not removable (i.e. limbo) ? Yes. Unneeded record versions older than the OIT can be garbage collected. Limbo transactions don't count as unneeded. >But I meant read-only (by definition in trn parameter block) read >committed transactions you mentioned earlier. They internally acquire pre- >committed status, don't they ? yes. >I see. That would really make a difference. But does this really need a >bookkeeping ? How about an extension logic in VIO_chase_record_version >that will notice subsequent versions from committed transactions and mark >it for GC (and extended GC to handle that, of course) ? The problem, and the reason for the bookkeeping, is that we currently keep track of the oldest active. That's an over-simplification. The "oldest active" is actually the oldest transaction that was active when any currently active transaction started. In order to remove intermediate versions, we'd need to keep track of the oldest transaction that was active when each active transaction started. There's also the problem of reconstructing back versions from differences, but that's not so difficult. Regards, Ann www.ibphoenix.com We have answers. |
|
From: Claudio V. C. <cv...@us...> - 2002-12-16 04:54:08
|
> -----Original Message----- > From: fir...@li... > [mailto:fir...@li...]On Behalf Of Pavel > Cisar > > > > I believe that client death does establish an OIT. > > But when client dies and server notice, server performs the rollback, > don't he ? If undo log is used on rollback to rewind changes and allow to > mark transaction as committed instead, client or network failure should > not affect OIT in most cases (every time undo log is used). I don't know why this can't happen. If the client crashed, the server knows the changes that it received until the client crashed. So, naively, I would expect that the undo log is usable. Can you explain why not, Ann? > > But I meant read-only (by definition in trn parameter block) read > committed transactions you mentioned earlier. They internally acquire pre- > committed status, don't they ? A small change was done on this, since there was a comment in the code, inviting to do the change. Pre-committed from the point of view of OAT management maybe, but I would dare to call pre-committed only the system transaction. ;-) And of course, there are some system tables whose changes are seen immediately by every txn, regardless of its isolation level. C. |
|
From: Ann W. H. <aha...@ib...> - 2002-12-16 15:15:35
|
At 01:56 AM 12/16/2002 -0400, Claudio Valderrama C. wrote: > > > > But when client dies and server notice, server performs the rollback, > > don't he ? If undo log is used on rollback to rewind changes and allow to > > mark transaction as committed instead, client or network failure should > > not affect OIT in most cases (every time undo log is used). > >I don't know why this can't happen. If the client crashed, the server knows >the changes that it received until the client crashed. So, naively, I would >expect that the undo log is usable. Can you explain why not, Ann? No, not from the code. If I were doing it, I'd assume that a transaction that rolled back might have something severely wrong with it and not trust its undo log. However, there are two paths, TRA_release_transaction and TRA_rollback. The latter applies the undo log, the former does not. The only calls to the former are from various shutdown routines. So, apparently, the undo log is used on client death, regardless of how I thought it worked. Hard to argue with the code. >Pre-committed from the point of view of OAT management maybe, but I would >dare to call pre-committed only the system transaction. In fact, the system transaction, transaction 0, is always active... >And of course, there are some system tables whose changes are seen >immediately by every txn, regardless of its isolation level. Those changes are made by transaction 0. Regards, Ann www.ibphoenix.com We have answers. |
|
From: Nickolay S. <sk...@bs...> - 2002-12-15 18:52:38
|
Hello Ann, Sunday, December 15, 2002, 9:34:25 PM, you wrote: > At 02:24 PM 12/15/2002 +0300, Nickolay Samofatov wrote: >>Oracle internals and operation seems very, very close to Firebird >>internals. Saying more, Firebird CS have better potential to scale >>then Oracle now. Firebird CS codebase is already cluster-ready and it >>should work exteremely nice in cluster because of very clever VIO code. >>I'm curious when first clustered solution will be implemented with >>Firebird... > You missed it by about 20 years. The architecture was designed for > VAX clusters and ran there very nicely. The difference between VAX > clusters and current cluster implementations is that VAXen had a > distributed lock manager. With that piece, we'd run on clusters > perfectly. This is exactly the piece I'm going to create/adapt just after adoption of new memory manager and class library. Ann, what about contracts with hardware manufacturers ? Firebird Clustered Appliance. Sounds good, yeah ? Serveral Intel SMP Linux machines wired using gigabit ethernet, Myrinet or fibre channel network + shared disk controller or SAN. I can see only Oracle on this market. I worked with such machines for some time. One such a 8-node cluster costs a way over a million dollars with initial hardware costing less that half of that. Isn't it a good way to raise funds for Firebird development ? -- Best regards, Nickolay Samofatov mailto:sk...@bs... |
|
From: Ded <de...@hq...> - 2002-12-15 21:46:58
|
"Ann W. Harrison" wrote: > Read-only read-committed transactions do not affect the > OAT - they can go on forever without affecting garbage collection. In > this case, unlike the OIT, the transaction must be declared as read-only, > not just be de-facto read-only. I should remember that since this feature was implemented in FB more than year ago there were created many applications which used it starting one common read only read commiting transaction on application start and closing it on application stop, making changes in short write transactions. IMHO it's not wise to make this behaviour illegal, so, if decision to implement transaction-level timeout will be taken, I think it should'nt be default behaviour but explicitly forced on start of transaction. But in general I agree with those who said that network and security problems should be sold on connection level and FB have no need in transaction level timeouts to serve badly designed applications and thoughtless usage of interactive tools. Best regards, Alexander V.Nevsky. |
|
From: Ann W. H. <aha...@ib...> - 2002-12-13 16:09:36
|
At 07:16 AM 12/13/2002 -0400, Daniel Rail wrote: >example: a broken link between client and server due to network >failure, how long does the transaction stays alive and how to kill it >earlier, other than doing a backup and restore. The transaction and connection should be gone within 10 seconds. > Also, it would be a >nice feature to monitor transactions when doing the development and >testing, since it's possible to create a query or update statement >that's taking longer than expected and want to kill it in order to >make the necessary corrections and run it again, without having to do >a forced shutdown of Firebird. Yes, absolutely. Regards, Ann www.ibphoenix.com We have answers. |
|
From: Phil S. <ph...@sh...> - 2002-12-13 16:19:54
|
On Friday 13 December 2002 16:12, Ann W. Harrison wrote: Hi, > >a broken link between client and server due to network > >failure, how long does the transaction stays alive > > The transaction and connection should be gone within 10 seconds. It is consistantly about 60 secs on the machine I am working on at the moment, is there a setting that I might have altered? Phil -- Linux 2.4.4-4GB 2:49pm up 9 days, 21:16, 1 user, load average: 0.15, 0.13, 0.05 |
|
From: Ann W. H. <aha...@ib...> - 2002-12-13 17:09:33
|
At 02:51 PM 12/13/2002 +0000, Phil Shrimpton wrote: > > > > The transaction and connection should be gone within 10 seconds. > >It is consistantly about 60 secs on the machine I am working on at the >moment, is there a setting that I might have altered? No, it's me again, thinking I remember the code. Regards, Ann www.ibphoenix.com We have answers. |
|
From: Roman R. <rro...@ac...> - 2002-12-14 02:28:45
|
> It is consistantly about 60 secs on the machine I am working on at the > moment, is there a setting that I might have altered? In Java you control this by Socket.setSoTimeout(int milliseconds) (SO_TIMEOUT socket option). In one open-source project (JavaGroups, a group communication toolkit) a failure detector fails to detect physical link failure if it expects an I/O exception when a group member crashes. When you read something from a socket and it blocks in read operation (no data in the stream, but socket is open and "alive"), you do not get exception if network cable is removed, only if you close the socket on the other side. The only "fix" was to specify timeout for the socket, but this thing made that particular failure detector useless (it became a partial case of heartbeat failure detector). The only working failure detector to detect physical link failure in JavaGroups is heartbeat detector based on timeouts. Socket-based detector using exception does not detect such failures. I do not know, maybe this is Java specific... Just wanted you to know about this issue. Also, correct me if I'm wrong, but it was proven mathematically the impossibility to detect a failure reliably in an async. system and distinguish a failed node from the slow one. The question is if we need a 100% reliable failure detector or not? Best regards, Roman Rokytskyy __________________________________________________ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com |
|
From: Roman R. <rro...@ac...> - 2002-12-14 09:27:01
|
> No, just a timeout detector IMO. My basic argument is that detecting > connection failure is fine and good, but you can't rely on clients being > well behaved enough to kill their connections when they forget about their > transactions, so we need timeouts or leases or something for transactions > as well. I this case I tend to agree with others that it is a responsibility of transaction manager to kill such client. As far as I know, no database tries to solve this problem. Does Firebird send keep-alive messages even there is no requests from the client, only open connection? If yes, I think JayBird is not replying on them now... Best regards, Roman Rokytskyy __________________________________________________ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com |
|
From: David J. <dav...@di...> - 2002-12-14 14:35:45
|
On 2002.12.14 04:25:12 -0500 Roman Rokytskyy wrote: > > No, just a timeout detector IMO. My basic argument is that detecting > > connection failure is fine and good, but you can't rely on clients > being > > well behaved enough to kill their connections when they forget about > their > > transactions, so we need timeouts or leases or something for > transactions > > as well. > > I this case I tend to agree with others that it is a responsibility of > transaction manager to kill such client. As far as I know, no database > tries > to solve this problem. It is a standard part of the xa stuff, and you'd only encounter it using the xa parts of a database. For instance, in java it's available through the XAResource interface. Even though the xa spec is perhaps more "advice on how to confuse your customers" than an explanation of how its supposed to work, I'd be very surprised if most db vendors didn't implement something as fundamental to reliability as this in their xa support. I think most databases actually go much further than I am proposing and heuristically decide to rollback or commit prepared transactions that are not ended within a timeout. These transactions can theoretically be recovered when the tm comes back up: the reason for the database to end them unilaterally is to avoid problems with too many locks. I think the mga provides enough flexibility that we can wait for the tm to provide the actual disposition of these tx. david jencks > > Does Firebird send keep-alive messages even there is no requests from the > client, only open connection? If yes, I think JayBird is not replying on > them now... > > Best regards, > Roman Rokytskyy > > __________________________________________________ > Do You Yahoo!? > Everything you'll ever need on one web page > from News and Sport to Email and Music Charts > http://uk.my.yahoo.com > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > |
|
From: Marco M. <mm...@ly...> - 2002-12-11 20:19:50
|
What about a paramter for the transaction? I mean, each transaction can optionally specify the maximum duration. But I think that could be done at "client side" too, with some properties in the connection compoents. And I think that's precisely what IBO (www.ibobjects.com) native components do: from the help of TIB_Transaction.TimeoutProps ... ... ForceClosed This property determines the period in seconds IBO will wait before it forces a transaction to end, without regard for user activity. PromptUser This is the point in time that IBO will actually get intrusive about prompting the user to get on with their work and get the transaction closed. PromptUserDuration Period in seconds between IBO's prompts to the user to resolve the outstanding transaction. PromptUserRetry Amount of time in seconds that IBO waits before retrying prompting the user to resolve their transaction. ... regards Marco Menardi Christian Pradelli wrote: > Hi! > > Dealing with my application to see that any transaction keeps open during a > long time, I think, is not possible to put a parameter in the configuration > file to set a Max transaction duration?. > This would be important because may be I'm a good developer and I do the > best client-server application, but if a user open an administration tool, > put a table in edit mode and keep it open.... > With a feature like this we could guarantee the performance of Firebird > separating it from bad software design. > > I'm right? > > best regards christian > Christian > > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > |
|
From: Jason C. (JAC2) <ja...@ja...> - 2002-12-12 08:44:02
|
Just my thoughts. The main reason for this feature is to cope with software that hasn't been written in a sympathetic way towards sensible transaction length, so implementing in the client seems weaker than having a _database_ parameter. Also, units of time do give users the most consistent interface (i.e. it would become known that 24 hours and your app will be rolling back all over the place), but from a true protection perspective it would be better to do this for the difference between transaction numbers e.g. next TX - theTX >= 500,000. Could you imagine someone realising that the date on the server is a day out, adjusting it & all users get their TX's rolled back :-). Also I think that it should kill the connection and not just the transaction as then the offending application will cease, as opposed to one element of it ceasing to work (the client will probably not know the tx has been rolled back & I review plenty of code where unusual exceptions are just swallowed). Also it should put an entry in the log file. Finally, this should really be considered a bug in the client application and to add to the debugging the ultimate would be stats on Open TX's with connections and IP addresses etc. "Marco Menardi" <mm...@ly...> wrote in message news:at7p45$anb$1...@ne...... > What about a paramter for the transaction? I mean, each transaction can > optionally specify the maximum duration. But I think that could be done > at "client side" too, with some properties in the connection compoents. > And I think that's precisely what IBO (www.ibobjects.com) native > components do: > from the help of TIB_Transaction.TimeoutProps ... > ... > ForceClosed > This property determines the period in seconds IBO will wait before it > forces a transaction to end, without regard for user activity. > > PromptUser > This is the point in time that IBO will actually get intrusive about > prompting the user to get on with their work and get the transaction closed. > > PromptUserDuration > Period in seconds between IBO's prompts to the user to resolve the > outstanding transaction. > > PromptUserRetry > Amount of time in seconds that IBO waits before retrying prompting the > user to resolve their transaction. > ... > > regards > Marco Menardi > > Christian Pradelli wrote: > > Hi! > > > > Dealing with my application to see that any transaction keeps open during a > > long time, I think, is not possible to put a parameter in the configuration > > file to set a Max transaction duration?. > > This would be important because may be I'm a good developer and I do the > > best client-server application, but if a user open an administration tool, > > put a table in edit mode and keep it open.... > > With a feature like this we could guarantee the performance of Firebird > > separating it from bad software design. > > > > I'm right? > > > > best regards christian > > Christian > > > > > > > > > > ------------------------------------------------------- > > This sf.net email is sponsored by: > > With Great Power, Comes Great Responsibility > > Learn to use your power at OSDN's High Performance Computing Channel > > http://hpc.devchannel.org/ > > _______________________________________________ > > Firebird-devel mailing list > > Fir...@li... > > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > |
|
From: <psc...@in...> - 2002-12-13 00:28:15
|
On 12 Dec 2002 at 8:25, Jason Chapman (JAC2) wrote: > Just my thoughts. How about my thoughts.... > > The main reason for this feature is to cope with software that hasn't > been written in a sympathetic way towards sensible transaction length, > so implementing in the client seems weaker than having a _database_ > parameter. The software should be fixed, a maximum transaction duration, cures the symptom, and not the overall disease, a badly designed app, which probably has other problems as well. > Also, units of time do give users the most consistent interface (i.e. > it would become known that 24 hours and your app will be rolling back > all over the place), but from a true protection perspective it would > be better to do this for the difference between transaction numbers > e.g. next TX - theTX >= 500,000. Could you imagine someone realising > that the date on the server is a day out, adjusting it & all users get > their TX's rolled back :-). There are a couple of issues, any overall general transaction life routine, will either need too long a period that a transaction can live, or it's going to kill something important. For example if the year end sales report runs for 17 hours, then you need to set the timeout to at least 18 hours. This means that ANY transaction can live for 18 hours, if you have 25,000 normal transactions an hour, your still sunk. Next, whether caused by a sales report, or a duffus playing around, the performance effect is the same, that long running transaction, is going to gum up the works. Ultimately the application needs to be changed to use more shorter transactions. > Also I think that it should kill the connection and not just the > transaction as then the offending application will cease, as opposed > to one element of it ceasing to work (the client will probably not > know the tx has been rolled back & I review plenty of code where > unusual exceptions are just swallowed). Also it should put an entry in > the log file. > > Finally, this should really be considered a bug in the client > application and to add to the debugging the ultimate would be stats on > Open TX's with connections and IP addresses etc. I think a better solution, is when you create a transaction, you can identify the application, the engine then has a duration, but it doesn't kill the transaction or the connection, but it does write an entry in it's log that transaction xxxx owned by application yyy exceeded the transaction limit of zzzzzz ms. This way when transactions are causing problems by running too long, the DBA can notify the proper person and the application can be fixed. |
|
From: David J. <dav...@di...> - 2002-12-13 01:17:53
|
On 2002.12.12 19:26:10 -0500 psc...@in... wrote: > On 12 Dec 2002 at 8:25, Jason Chapman (JAC2) wrote: > > > Just my thoughts. > > How about my thoughts.... > > > > > The main reason for this feature is to cope with software that hasn't > > been written in a sympathetic way towards sensible transaction length, > > so implementing in the client seems weaker than having a _database_ > > parameter. > > The software should be fixed, a maximum transaction duration, cures the > symptom, > and not the overall disease, a badly designed app, which probably has > other > problems as well. NO!! The purpose of timeouts is not to compensate for badly written anything, it is to make it possible to write robust distributed programs that can continue operating even when some parts crash or become unavailable. If someone removes the lan cable between your well written app and the firebird server, either firecbird has to be able to recognize that your uncompleted transactions can be rolled back, or some person will have to do it. > > > Also, units of time do give users the most consistent interface (i.e. > > it would become known that 24 hours and your app will be rolling back > > all over the place), but from a true protection perspective it would > > be better to do this for the difference between transaction numbers > > e.g. next TX - theTX >= 500,000. Could you imagine someone realising > > that the date on the server is a day out, adjusting it & all users get > > their TX's rolled back :-). > > There are a couple of issues, any overall general transaction life > routine, will either > need too long a period that a transaction can live, or it's going to kill > something > important. For example if the year end sales report runs for 17 hours, > then you > need to set the timeout to at least 18 hours. This means that ANY > transaction can > live for 18 hours, if you have 25,000 normal transactions an hour, your > still sunk. > Specify the timeout per transaction. Personally I think the JINI idea of renewable leases is the most elegant, but simple timeouts are standard in the database world. > Next, whether caused by a sales report, or a duffus playing around, the > performance > effect is the same, that long running transaction, is going to gum up the > works. > Ultimately the application needs to be changed to use more shorter > transactions. > > > Also I think that it should kill the connection and not just the > > transaction as then the offending application will cease, as opposed > > to one element of it ceasing to work (the client will probably not > > know the tx has been rolled back & I review plenty of code where > > unusual exceptions are just swallowed). Also it should put an entry in > > the log file. > > > > Finally, this should really be considered a bug in the client > > application and to add to the debugging the ultimate would be stats on > > Open TX's with connections and IP addresses etc. > > I think a better solution, is when you create a transaction, you can > identify the > application, the engine then has a duration, but it doesn't kill the > transaction or the > connection, but it does write an entry in it's log that transaction xxxx > owned by > application yyy exceeded the transaction limit of zzzzzz ms. This way > when > transactions are causing problems by running too long, the DBA can notify > the > proper person and the application can be fixed. Hardware failures are not always caused by bad programming. Why make the poor dba figure out how to recover when firebird can do it just fine itself? david jencks > > > > > > > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > |
|
From: Christian P. <c_p...@ya...> - 2002-12-13 01:54:15
|
finally somebody understand which is my point!! :) "David Jencks" <dav...@di...> escribió en el mensaje news:20021212201712.Q14852@HP.home.home... > On 2002.12.12 19:26:10 -0500 psc...@in... wrote: > > On 12 Dec 2002 at 8:25, Jason Chapman (JAC2) wrote: > > > > > Just my thoughts. > > > > How about my thoughts.... > > > > > > > > The main reason for this feature is to cope with software that hasn't > > > been written in a sympathetic way towards sensible transaction length, > > > so implementing in the client seems weaker than having a _database_ > > > parameter. > > > > The software should be fixed, a maximum transaction duration, cures the > > symptom, > > and not the overall disease, a badly designed app, which probably has > > other > > problems as well. > > NO!! The purpose of timeouts is not to compensate for badly written > anything, it is to make it possible to write robust distributed programs > that can continue operating even when some parts crash or become > unavailable. If someone removes the lan cable between your well written > app and the firebird server, either firecbird has to be able to recognize > that your uncompleted transactions can be rolled back, or some person will > have to do it. > > > > > > > Also, units of time do give users the most consistent interface (i.e. > > > it would become known that 24 hours and your app will be rolling back > > > all over the place), but from a true protection perspective it would > > > be better to do this for the difference between transaction numbers > > > e.g. next TX - theTX >= 500,000. Could you imagine someone realising > > > that the date on the server is a day out, adjusting it & all users get > > > their TX's rolled back :-). > > > > There are a couple of issues, any overall general transaction life > > routine, will either > > need too long a period that a transaction can live, or it's going to kill > > something > > important. For example if the year end sales report runs for 17 hours, > > then you > > need to set the timeout to at least 18 hours. This means that ANY > > transaction can > > live for 18 hours, if you have 25,000 normal transactions an hour, your > > still sunk. > > > Specify the timeout per transaction. Personally I think the JINI idea of > renewable leases is the most elegant, but simple timeouts are standard in > the database world. > > > > Next, whether caused by a sales report, or a duffus playing around, the > > performance > > effect is the same, that long running transaction, is going to gum up the > > works. > > Ultimately the application needs to be changed to use more shorter > > transactions. > > > > > Also I think that it should kill the connection and not just the > > > transaction as then the offending application will cease, as opposed > > > to one element of it ceasing to work (the client will probably not > > > know the tx has been rolled back & I review plenty of code where > > > unusual exceptions are just swallowed). Also it should put an entry in > > > the log file. > > > > > > Finally, this should really be considered a bug in the client > > > application and to add to the debugging the ultimate would be stats on > > > Open TX's with connections and IP addresses etc. > > > > I think a better solution, is when you create a transaction, you can > > identify the > > application, the engine then has a duration, but it doesn't kill the > > transaction or the > > connection, but it does write an entry in it's log that transaction xxxx > > owned by > > application yyy exceeded the transaction limit of zzzzzz ms. This way > > when > > transactions are causing problems by running too long, the DBA can notify > > the > > proper person and the application can be fixed. > > > Hardware failures are not always caused by bad programming. Why make the > poor dba figure out how to recover when firebird can do it just fine > itself? > > david jencks > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------- > > This sf.net email is sponsored by: > > With Great Power, Comes Great Responsibility > > Learn to use your power at OSDN's High Performance Computing Channel > > http://hpc.devchannel.org/ > > _______________________________________________ > > Firebird-devel mailing list > > Fir...@li... > > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > > > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > |
|
From: Ann W. H. <aha...@ib...> - 2002-12-13 15:59:24
|
At 08:17 PM 12/12/2002 -0500, David Jencks wrote: >NO!! The purpose of timeouts is not to compensate for badly written >anything, it is to make it possible to write robust distributed programs >that can continue operating even when some parts crash or become >unavailable. If someone removes the lan cable between your well written >app and the firebird server, either firecbird has to be able to recognize >that your uncompleted transactions can be rolled back, or some person will >have to do it. If my dog chews through the LAN cable, Firebird's keep-alive check on the network connection will fail, any transactions associated with the connection will be rolled back, and the connection resources released. Regards, Ann www.ibphoenix.com We have answers. |
|
From: Martijn T. <m.t...@up...> - 2002-12-13 16:05:53
|
> >NO!! The purpose of timeouts is not to compensate for badly written > >anything, it is to make it possible to write robust distributed programs > >that can continue operating even when some parts crash or become > >unavailable. If someone removes the lan cable between your well written > >app and the firebird server, either firecbird has to be able to recognize > >that your uncompleted transactions can be rolled back, or some person will > >have to do it. > > > If my dog chews through the LAN cable, Firebird's keep-alive check on > the network connection will fail, any transactions associated with > the connection will be rolled back, and the connection resources released. And don't forget to punish your dog... it might get rolled back to it's own corner in the living room... I know I would. btw, Ann, it's Friday afternoon - stop working, get a beer. :) With regards, Martijn Tonies InterBase Workbench - the developer tool for InterBase & Firebird Firebird Workbench - the developer tool for Firebird Upscene Productions http://www.upscene.com "This is an object-oriented system. If we change anything, the users object." |
|
From: Lester C. <le...@ls...> - 2002-12-13 16:18:23
|
> If my dog chews through the LAN cable, Firebird's keep-alive check on > the network connection will fail, any transactions associated with > the connection will be rolled back, and the connection resources released. You have one of those - mine's a rabbit with a preference for mains leads - preferably live ones. Seriously, any disruption to network activity has to be handled - via a timeout - at the server. -- Lester Caine ----------------------------- L.S.Caine Electronic Services |
|
From: <psc...@in...> - 2002-12-13 22:02:43
|
On 12 Dec 2002 at 20:17, David Jencks wrote: > > The software should be fixed, a maximum transaction duration, cures > > the symptom, and not the overall disease, a badly designed app, > > which probably has other problems as well. > > NO!! The purpose of timeouts is not to compensate for badly written > anything, it is to make it possible to write robust distributed > programs that can continue operating even when some parts crash or > become unavailable. If someone removes the lan cable between your > well written app and the firebird server, either firecbird has to be > able to recognize that your uncompleted transactions can be rolled > back, or some person will have to do it. If the network connection is severed, whether because someone pulled a Lan cable, or the client machine crashed or the power failed, or a storm zapped the client, the server detects this situation, and aborts all transactions for that client, eventually based on network timeouts. It's just while waiting for the first row, when it seems to not process keep alive packets that it may not notice, this should be logged as a bug and fixed if it's true. > > There are a couple of issues, any overall general transaction life > > routine, will either need too long a period that a transaction can > > live, or it's going to kill something important. For example if the > > year end sales report runs for 17 hours, then you need to set the > > timeout to at least 18 hours. This means that ANY transaction can > > live for 18 hours, if you have 25,000 normal transactions an hour, > > your still sunk. > > > Specify the timeout per transaction. Personally I think the JINI idea > of renewable leases is the most elegant, but simple timeouts are > standard in the database world. This means that the application needs to know that it contains a longer running transaction, and it may still be a problem, whether a long running transaction is legitimate or not, it causes the same problem, server performance goes down the toilet. Ideally that 18 hour sales report, should be written in such a way, that it doesn't need an 18 hour transaction. IMNSHO we need to find a way to minimize the impact of long running transactions, rather then eliminating long running transactions. If a transaction that started in 1997 and is still running, and causes no problems then who would really care? Paul |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X