Menu
▾
▴
firebird-devel
|
From: Christian P. <c_p...@ya...> - 2002-12-11 14:57:38
|
Hi! Dealing with my application to see that any transaction keeps open during a long time, I think, is not possible to put a parameter in the configuration file to set a Max transaction duration?. This would be important because may be I'm a good developer and I do the best client-server application, but if a user open an administration tool, put a table in edit mode and keep it open.... With a feature like this we could guarantee the performance of Firebird separating it from bad software design. I'm right? best regards christian Christian |
|
From: Ann W. H. <aha...@ib...> - 2002-12-11 18:07:45
|
At 12:03 PM 12/11/2002 -0300, Christian Pradelli wrote: >Hi! > >Dealing with my application to see that any transaction keeps open during a >long time, I think, is not possible to put a parameter in the configuration >file to set a Max transaction duration?. I guess that would depend on the units used. If you can set the number of months that a transaction can be open, that seems OK. If the units are hours or minutes, the chance of killing a major report or year-end process is high enough to scare me off. Regards, Ann www.ibphoenix.com We have answers. |
|
From: Christian P. <c_p...@ya...> - 2002-12-11 18:36:45
|
Hi Ann!
> I guess that would depend on the units used. If you
> can set the
> number of months that a transaction can be open,
> that seems OK.
> If the units are hours or minutes, the chance of
> killing a major
> report or year-end process is high enough to scare
> me off.
I've a customer who have 50 concurrent users and made about 100000
transaction a day over one GDB, this week I was testing a new application
over the same database and today I see this in the static's:
Oldest transaction 1481648
Oldest active 1481649
Oldest snapshot 1481649
Next transaction 1598010
this is not good, of course, the design of the application is horrible!!!!.
:))
I'm thinking in minutes or hours, but if you don't want to use this feature
you can set it to 0 so the transaction will never be closed by the server.
I think that a feature like this will be a very usefull for critical systems
with hight number of users and for tunning new applications.
Christian
|
|
From: Daniel R. <da...@ac...> - 2002-12-11 19:43:42
|
Hi all, >> I guess that would depend on the units used. If you >> can set the >> number of months that a transaction can be open, >> that seems OK. >> If the units are hours or minutes, the chance of >> killing a major >> report or year-end process is high enough to scare >> me off. [...snip...] > I'm thinking in minutes or hours, but if you don't want to use this feature > you can set it to 0 so the transaction will never be closed by the server. > I think that a feature like this will be a very usefull for critical systems > with hight number of users and for tunning new applications. I, too, like the idea of a maximum transaction duration. Here's one way of designing it: - Default duration defined at the server level, so when creating a new database it will use that default duration. - Default duration defined at the database level to be able to override the server's default duration. - Pass the duration upon connection to override the default duration, so that in some circumstances the duration can be extended to perform i.e. the year-end processing/reports. I know that this might require some changes to the API. Just my 2 cents. -- Best regards, Daniel Rail Senior System Engineer ACCRA Group Inc. (www.accra.ca) ACCRA Med Software Inc. (www.accramed.ca) |
|
From: Phil S. <ph...@sh...> - 2002-12-11 20:09:39
|
On Wednesday 11 December 2002 19:43, Daniel Rail wrote: Hi, > I, too, like the idea of a maximum transaction duration. Personally, I think it is a bad idea, that would only encourage poorly designed applications. In a properley designed application, I can't think of a good reason to have an open, idle transaction. If you are not reading or writing to the database, there is no need to have an open transaction. This is probably the wrong list to be discusing this.. Cheers Phil -- Linux 2.4.4-4GB 6:31pm up 8 days, 57 min, 1 user, load average: 0.62, 0.21, 0.07 |
|
From: Christian P. <c_p...@ya...> - 2002-12-11 20:20:13
|
Hi > Personally, I think it is a bad idea, that would only encourage poorly > designed applications. You are right, but you are thinking that you (or you application) are the only one that access the database, and this is not right every time. In a large database sometimes you have more than one developer and different applications, you can't cover all the items. Or simple a end user discover ibconsole and start playing browsing data. Think that I'm talking about large volume transaction databases. Firebird should be safe from dummies attack.:)) Christian |
|
From: Pavel C. <pc...@us...> - 2002-12-12 09:58:04
|
Hi, On 11 Dec 2002 at 15:43, Daniel Rail wrote: > I, too, like the idea of a maximum transaction duration. No! Trying to solve the problem that majority doesn't have or can solve in different way by shifting the whole system out of balance is really a bad idea. Linus or Jim Starkey would send any such proposal promptly and colourfully to the place it deserve - a tenth circle of hell. Better solution is to provide a method to monitor transactions (for example through temporary system tables like in IB7), so anyone interested can write a small monitor app. that will kill the long transaction or far better page the sysadmin's beeper. Best regards Pavel Cisar http://www.ibphoenix.com For all your upto date Firebird and InterBase information |
|
From: Daniel R. <da...@ac...> - 2002-12-12 11:07:23
|
Hi all, > Better solution is to provide a method to monitor transactions (for > example through temporary system tables like in IB7), so anyone > interested can write a small monitor app. that will kill the long > transaction or far better page the sysadmin's beeper. I was wondering what the new monitor functionality in IB7 was for. That is a better solution. Also, being able to see what is going on in the transaction would be good too, this is just to make sure not to kill a transaction that is part of a "known" long running process(i.e. year-end processing/reporting). Unless it would be possible to flag a transaction as one that can take time to complete, compared to others. -- Best regards, Daniel Rail Senior System Engineer ACCRA Group Inc. (www.accra.ca) ACCRA Med Software Inc. (www.accramed.ca) |
|
From: David J. <dav...@di...> - 2002-12-12 14:37:18
|
On 2002.12.12 05:02:55 -0500 Pavel Cisar wrote: > Hi, > > On 11 Dec 2002 at 15:43, Daniel Rail wrote: > > > I, too, like the idea of a maximum transaction duration. > > No! Trying to solve the problem that majority doesn't have or can solve > in different way by shifting the whole system out of balance is really a > bad idea. Linus or Jim Starkey would send any such proposal promptly and > colourfully to the place it deserve - a tenth circle of hell. Umm, then they should share their wisdom on reliable distributed system design with the rest of the world, which AFAIK universally uses timeouts to model node failure in distributed systems. This can be explicit as in the XA transaction model with transaction timeouts (a fairly old example) and JINI leases (a much newer example) or implicit as in systems with heartbeats. Typically transaction managers use "presumed abort" optimization so they don't make any permanent record of a transaction until the prepare phase is successful. They don't make any cleanup attempts on transactions they haven't prepared. Therefore some kind of "forget on timeout" for unprepared transactions in the database is needed for automatic recovery. I'm considerably less enthusiasitic about heuristic completion of prepared transactions on timeout. david jencks > > Better solution is to provide a method to monitor transactions (for > example through temporary system tables like in IB7), so anyone > interested can write a small monitor app. that will kill the long > transaction or far better page the sysadmin's beeper. > > Best regards > Pavel Cisar > http://www.ibphoenix.com > For all your upto date Firebird and > InterBase information > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > |
|
From: Pavel C. <pc...@us...> - 2002-12-12 20:46:44
|
David, On 12 Dec 2002 at 9:35, David Jencks wrote: > Umm, then they should share their wisdom on reliable distributed system > design with the rest of the world, which AFAIK universally uses timeouts > to model node failure in distributed systems. This can be explicit as in > the XA transaction model with transaction timeouts (a fairly old example) > and JINI leases (a much newer example) or implicit as in systems with > heartbeats. I didn't think that transaction timeout itself is not useful or needed in some cases, but there is no need to implement it at database engine level as flat transaction behaviour. Your distributed transaction manager can easily implement this without any change in database engine. Best regards Pavel Cisar http://www.ibphoenix.com For all your upto date Firebird and InterBase information |
|
From: David J. <dav...@di...> - 2002-12-12 22:58:22
|
On 2002.12.12 15:51:27 -0500 Pavel Cisar wrote: > David, > > On 12 Dec 2002 at 9:35, David Jencks wrote: > > > Umm, then they should share their wisdom on reliable distributed system > > design with the rest of the world, which AFAIK universally uses > timeouts > > to model node failure in distributed systems. This can be explicit as > in > > the XA transaction model with transaction timeouts (a fairly old > example) > > and JINI leases (a much newer example) or implicit as in systems with > > heartbeats. > > I didn't think that transaction timeout itself is not useful or needed in > > some cases, but there is no need to implement it at database engine level > > as flat transaction behaviour. Your distributed transaction manager can > easily implement this without any change in database engine. I don't see how. Unless the resource manager can discard timed out work automatically, someone has to do it manually. If the tm crashes, it will have no record of transactions that have been started but not prepared, and will not be able to instruct the resource manager to forget them: there could be other tms that are responsible for them and are still actively using them. Please explain how the abandoned work on the resource manager will get cleaned up automatically in this case unless the rm can unilaterally discard it based only on information it knows. thanks david jencks > > Best regards > Pavel Cisar > http://www.ibphoenix.com > For all your upto date Firebird and > InterBase information > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > |
|
From: Pavel C. <pc...@us...> - 2002-12-13 09:17:08
|
Favid, On 12 Dec 2002 at 17:57, David Jencks wrote: > > I didn't think that transaction timeout itself is not useful or needed in > > some cases, but there is no need to implement it at database engine level > > as flat transaction behaviour. Your distributed transaction manager can > > easily implement this without any change in database engine. > > I don't see how. Unless the resource manager can discard timed out work > automatically, someone has to do it manually. If the tm crashes, it will > have no record of transactions that have been started but not prepared, and > will not be able to instruct the resource manager to forget them: there > could be other tms that are responsible for them and are still actively > using them. > > Please explain how the abandoned work on the resource manager will get > cleaned up automatically in this case unless the rm can unilaterally > discard it based only on information it knows. Well, I'm not completely aware about all intimate details in this design, especially about various dependencies, but... Transactions are bound to connections. If connection or connection manager crashes, transactions are rolled back automatically. You need to implement trn. timeout "controller" close to the connection, so they would crash together if at all, so db engine "takes over" to rollback the transaction. If special handling is needed, the timeout controller have to expose an API to trn. manager (if they are not the same) or other subsystem that starts transactions with timeout. If connection between timeout manager and other modules is broken, timeout manager "expires" the transaction(s). In your design, the "timeout controller" should probably belong to resource manager, and yes, it can expire transactions automatically when trn. manager goes down. Best regards Pavel Cisar http://www.ibphoenix.com For all your upto date Firebird and InterBase information |
|
From: David J. <dav...@di...> - 2002-12-13 15:03:08
|
On 2002.12.13 04:21:59 -0500 Pavel Cisar wrote: > Favid, > > On 12 Dec 2002 at 17:57, David Jencks wrote: > > > > I didn't think that transaction timeout itself is not useful or > needed in > > > some cases, but there is no need to implement it at database engine > level > > > as flat transaction behaviour. Your distributed transaction manager > can > > > easily implement this without any change in database engine. > > > > I don't see how. Unless the resource manager can discard timed out > work > > automatically, someone has to do it manually. If the tm crashes, it > will > > have no record of transactions that have been started but not prepared, > and > > will not be able to instruct the resource manager to forget them: there > > could be other tms that are responsible for them and are still actively > > using them. > > > > Please explain how the abandoned work on the resource manager will get > > cleaned up automatically in this case unless the rm can unilaterally > > discard it based only on information it knows. > > Well, I'm not completely aware about all intimate details in this design, > > especially about various dependencies, but... > > Transactions are bound to connections. If connection or connection > manager crashes, transactions are rolled back automatically. You need to > implement trn. timeout "controller" close to the connection, so they > would crash together if at all, so db engine "takes over" to rollback the > > transaction. If special handling is needed, the timeout controller have > to expose an API to trn. manager (if they are not the same) or other > subsystem that starts transactions with timeout. If connection between > timeout manager and other modules is broken, timeout manager "expires" > the transaction(s). > > In your design, the "timeout controller" should probably belong to > resource manager, and yes, it can expire transactions automatically when > trn. manager goes down. Resource manager == database == firebird engine:-) While it might be possible to put a timeout manager on the client side of the connection, I dont' think there is any way to make it as reliable as putting it on the engine/server side. I think there will always be some failure modes where the connection is not killed but the remote timeout manager stops working. I think your argument at root is that the "lost connection" detection by firebird is sufficient to detect problems. I dont' completely know how this works but think it is based on a heartbeat of some kind, pinging the clients. is this correct? I still think that transaction timeouts would be a really useful and standard feature, although they don't appear to fit in the Delphi transaction model if I remember correctly. thanks david jencks > > Best regards > Pavel Cisar > http://www.ibphoenix.com > For all your upto date Firebird and > InterBase information > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > |
|
From: Pavel C. <pc...@us...> - 2002-12-13 18:52:13
|
David, On 13 Dec 2002 at 10:02, David Jencks wrote: > Resource manager == database == firebird engine:-) Ah, I see :-) > While it might be possible to put a timeout manager on the client side of > the connection, I dont' think there is any way to make it as reliable as > putting it on the engine/server side. I think there will always be some > failure modes where the connection is not killed but the remote timeout > manager stops working. There should not be any such modes that will leave the dead connection open forever. Server should always notice, sooner or later, that connection is dead/broken. Of course, this doesn't solve the problem when client gets a connection and then goes to infinite loop. I agree that we should handle that in some way, to prevent malicious code or DOS attacks, but this is still IMO related to connections, not transaction level. > I think your argument at root is that the "lost connection" detection by > firebird is sufficient to detect problems. I dont' completely know how > this works but think it is based on a heartbeat of some kind, pinging the > clients. is this correct? Yes (see my reply to Ivan in this thread). Actually, the code is pretty tricky and depends on some assumptions about connection behaviour at OS level. I have my doubts about the quality of this code and its design, because I saw over years many different kind of troubles tracked to it in some way. > I still think that transaction timeouts would be a really useful and > standard feature, although they don't appear to fit in the Delphi > transaction model if I remember correctly. First, what's the definition of "timeout" ? Total amount of time ? Total time of inactivity ? What's more important to you, transaction or connection timeout ? Connection timeout is there already, and could be improved (for example: closing connection after some time of inactivity at FB protocol level). Best regards Pavel Cisar http://www.ibphoenix.com For all your upto date Firebird and InterBase information |
|
From: David J. <dav...@di...> - 2002-12-14 02:49:01
|
On 2002.12.13 13:57:02 -0500 Pavel Cisar wrote: > David, > > On 13 Dec 2002 at 10:02, David Jencks wrote: > > > Resource manager == database == firebird engine:-) > > Ah, I see :-) > > > While it might be possible to put a timeout manager on the client side > of > > the connection, I dont' think there is any way to make it as reliable > as > > putting it on the engine/server side. I think there will always be > some > > failure modes where the connection is not killed but the remote timeout > > manager stops working. > > There should not be any such modes that will leave the dead connection > open forever. Server should always notice, sooner or later, that > connection is dead/broken. Of course, this doesn't solve the problem when > > client gets a connection and then goes to infinite loop. I agree that we > should handle that in some way, to prevent malicious code or DOS attacks, > > but this is still IMO related to connections, not transaction level. > > > I think your argument at root is that the "lost connection" detection > by > > firebird is sufficient to detect problems. I dont' completely know > how > > this works but think it is based on a heartbeat of some kind, pinging > the > > clients. is this correct? > > Yes (see my reply to Ivan in this thread). Actually, the code is pretty > tricky and depends on some assumptions about connection behaviour at OS > level. I have my doubts about the quality of this code and its design, > because I saw over years many different kind of troubles tracked to it in > > some way. > > > I still think that transaction timeouts would be a really useful and > > standard feature, although they don't appear to fit in the Delphi > > transaction model if I remember correctly. > > First, what's the definition of "timeout" ? Total amount of time ? Total > time of inactivity ? I am to some extent still learning, but I think it is total time since transaction started. > > What's more important to you, transaction or connection timeout ? Right now, since I am working on the jboss tm, I'm thinking more about transaction timeouts. Thinking about it I think connection timeouts are useful too. How about considering the JINI renewable lease model for them: a client requests a lease for a particular length of time, and the server can grant a lease for that length or less. If the client decides it needs the resource for longer, it has to renew the lease periodically with the server, the server can again decide if and for how long it will grant the lease. I dont' know how hard it would be to implement in C++ but it seems very elegant and effective in JINI in java. BTW as I recall JINI also has leases on transactions for their timeout model. This would be fine with me also since it is even more flexible than the simple timeout model. It does seem to involve some asynchronous communication (sending lease renewals while you are waiting for a query to complete). Thanks david jencks > Connection timeout is there already, and could be improved (for example: > closing connection after some time of inactivity at FB protocol level). > > Best regards > Pavel Cisar > http://www.ibphoenix.com > For all your upto date Firebird and > InterBase information > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > |
|
From: Pavel C. <pc...@us...> - 2002-12-15 10:46:33
|
David, On 13 Dec 2002 at 21:48, David Jencks wrote: > I am to some extent still learning, but I think it is total time since > transaction started. Interesting, so server should kill requests in progress when time ticket expire ? I'd worry to introduce that. > Right now, since I am working on the jboss tm, I'm thinking more about > transaction timeouts. > > Thinking about it I think connection timeouts are useful too. How about > considering the JINI renewable lease model for them: a client requests a > lease for a particular length of time, and the server can grant a lease for > that length or less. If the client decides it needs the resource for > longer, it has to renew the lease periodically with the server, the server > can again decide if and for how long it will grant the lease. What I still don't fully understand is the reason why one needs a timeout on trn. Only one reason I can imagine is to prevent blocking of server resources from dead clients, so some sort of keep-alive protocol is more suitable than pure timeout. But we just can think about the same thing in different terms ? Well, if this is the case, then I agree with you, even on trn "timeout" (one can have live connection but dead thread with trn). So, optional keep-alive protocol controlled by client (not by server as it is now on connections) at transaction level is a valuable feature. > BTW as I recall JINI also has leases on transactions for their timeout > model. This would be fine with me also since it is even more flexible > than the simple timeout model. It does seem to involve some > asynchronous communication (sending lease renewals while you are > waiting for a query to complete). Interesting. Can you design an API concept for FB ? Best regards Pavel Cisar http://www.ibphoenix.com For all your upto date Firebird and InterBase information |
|
From: Roman R. <rro...@ac...> - 2002-12-15 12:08:28
|
Hi, Sorry for breaking into the conversation... >> BTW as I recall JINI also has leases on transactions for their timeout >> model. This would be fine with me also since it is even more flexible >> than the simple timeout model. It does seem to involve some >> asynchronous communication (sending lease renewals while you are >> waiting for a query to complete). > > Interesting. Can you design an API concept for FB ? I would suggest following (Java syntax): /** * Lease new transaction. This method tries to obtain a lease on * starting new transaction, and in case of success starts it. * * @param trHandle object that will contain transaction handle in case * of success. * @param dbHandle database handle obtained in isc_attach_database. * @param tpb transaction parameters buffer. * @param leasePeriod desired period of transaction duration in * milliseconds. * * @return granted period of transaction duration. * * @throws GDSException if lease cannot be granted. */ int isc_lease_transaction(isc_tr_handle trHandle, isc_db_handle dbHandle, byte[] tpb, int leasePeriod) throws GDSException; /** * Renew already obtained transaction lease. This method tries to * extend lease duration for the specified period of time. * * @param trHandle object that will contain transaction handle in case * of success. * @param dbHandle database handle obtained in isc_attach_database. * @param newLeasePeriod amount of time for which lease should be * extended in milliseconds starting from now. * * @return granted period of transaction duration. * * @throws GDSException if lease cannot be renewed. */ int isc_renew_lease(isc_tr_handle trHandle, isc_db_handle dbHandle, int newLeasePeriod) throws GDSException; Method isc_lease_transaction should be used instead of isc_start_transaction. Cancelling of the lease happens in isc_prepare_transaction2 or isc_rollback_transaction. If neither isc_prepare_transaction2 nor isc_rollback_transaction was called during the granted lease duration, transaction is marked "rollback only". No other statement can be executed in this transaction except isc_rollback_transaction. In this case server can perform rollback internally, but should keep transaction handle until isc_rollback_transaction is called or database connection is closed. Having this implemented you can limit serializable transactions the same way Oracle does this. Client might ask for very long transaction, but server will return only few hours duration and will deny all lease renewals. Similar thing is implemented in JavaGroups and it works. I use it as a lock manager, JBoss people thought about using it for JBoss clustering. Main drawback of this scheme is that when duration of the transaction is not known you have to start thread to renew lease before it expires. If CPU is quite busy, you might fail to renew it. Another drawback is increased network traffic when lease duration is short. Best regards, Roman Rokytskyy __________________________________________________ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com |
|
From: David J. <dav...@di...> - 2002-12-15 13:30:23
|
This looks good to me. I think doing this for connections as well would be good too. I think the main implementation problem will be making the socket usage asynchronous at both ends so you still only need one socket per connection. Adding more threads to some clients and possibly the server to handle asynchronous lease renewals may be tricky too. david jencks On 2002.12.15 07:07:05 -0500 Roman Rokytskyy wrote: > Hi, > > > I would suggest following (Java syntax): > > /** > * Lease new transaction. This method tries to obtain a lease on > * starting new transaction, and in case of success starts it. <snip> |
|
From: Ann W. H. <aha...@ib...> - 2002-12-13 15:49:28
|
At 05:57 PM 12/12/2002 -0500, David Jencks wrote: >Please explain how the abandoned work on the resource manager will get >cleaned up automatically in this case unless the rm can unilaterally >discard it based only on information it knows. I've probably got this all wrong, but if the tm tells a client connection that it's dead, one assumes that the client will disconnect, then Firebird will detect that the connection is gone and release all its resources. Regards, Ann www.ibphoenix.com We have answers. |
|
From: David J. <dav...@di...> - 2002-12-13 16:10:08
|
On 2002.12.13 10:47:47 -0500 Ann W. Harrison wrote: > At 05:57 PM 12/12/2002 -0500, David Jencks wrote: > > >Please explain how the abandoned work on the resource manager will get > >cleaned up automatically in this case unless the rm can unilaterally > >discard it based only on information it knows. > > I've probably got this all wrong, but if the tm tells a client > connection that it's dead, one assumes that the client will > disconnect, then Firebird will detect that the connection is > gone and release all its resources. > > Schemes like this put the responsibility for cleaning up failures on the client. As Nickolay points out, this leaves lots of opportunity for malicious clients. Even with friendly clients, I don't think it is possible to provide the same level of reliability as a server-controlled timeout cleanup. What if everything on a client except the tcp connection gets locked up? The dead connection detection in the server operates I think on this same principle. However, I'm not convinced that it alone provides sufficient reliability. thanks david jencks > > Regards, > > Ann > www.ibphoenix.com > We have answers. > > > > > ------------------------------------------------------- > This sf.net email is sponsored by: > With Great Power, Comes Great Responsibility > Learn to use your power at OSDN's High Performance Computing Channel > http://hpc.devchannel.org/ > _______________________________________________ > Firebird-devel mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > |
|
From: Daniel R. <da...@ac...> - 2002-12-13 11:16:34
|
Hi, On December 12, 2002 at 8:26 PM, psc...@in... wrote: > On 12 Dec 2002 at 7:06, Daniel Rail wrote: >> I was wondering what the new monitor functionality in IB7 was for. >> That is a better solution. Also, being able to see what is going on >> in the transaction would be good too, this is just to make sure not to >> kill a transaction that is part of a "known" long running process(i.e. >> year-end processing/reporting). Unless it would be possible to flag a >> transaction as one that can take time to complete, compared to others. > The problem with automatically killing a transaction, or even monitoring a > transaction and killing it manually, is that it cures the symptom, but not the under- > lying disease, which is an application that is taking resources and holding them > when it really shouldn't. It never gets fixed properly, because it becomes easier to > simply let it go, and kill the transaction on occasion. I know that a software that is well written, hardly needs a way to kill a long running transaction, unless it is totally unexpected. In example: a broken link between client and server due to network failure, how long does the transaction stays alive and how to kill it earlier, other than doing a backup and restore. Also, it would be a nice feature to monitor transactions when doing the development and testing, since it's possible to create a query or update statement that's taking longer than expected and want to kill it in order to make the necessary corrections and run it again, without having to do a forced shutdown of Firebird. > Maybe there are better ways to fix this, for example split the transaction into two > processes, one that is application oriented, and one that is engine oriented, a > commit retain would then commit and end the engine part of the transaction, and > create a new engine part of the transaction assigned to the same application part, > so that the long running transaction's process simply does a commit retain, when > and where it's logically convenient, and there becomes no need for a long running > transaction. Doesn't Nickolay's save points take care of that? But what happens if the long running transaction is a year-end report or a year-end reconciliation, which can take hours. But, if for some unforseen reason, you have to cancel that report or reconciliation and rollback everything, and your own application seems frozen, then you have to be able to kill it on the server, so that everything is rolled back. -- Best regards, Daniel Rail Senior System Engineer ACCRA Group Inc. (www.accra.ca) ACCRA Med Software Inc. (www.accramed.ca) |
|
From: Martijn T. <m.t...@up...> - 2002-12-13 11:49:12
|
Hi, > I know that a software that is well written, hardly needs a way to > kill a long running transaction, unless it is totally unexpected. In > example: a broken link between client and server due to network > failure, how long does the transaction stays alive and how to kill it > earlier, other than doing a backup and restore. Also, it would be a Firebird is already supposed to detect this, if I recall correctly. So the transaction duration thing hasn't really got anything to do with that. With regards, Martijn Tonies InterBase Workbench - the developer tool for InterBase & Firebird Firebird Workbench - the developer tool for Firebird Upscene Productions http://www.upscene.com "This is an object-oriented system. If we change anything, the users object." |
|
From: <psc...@in...> - 2002-12-13 22:02:56
|
On 13 Dec 2002 at 7:16, Daniel Rail wrote: > Hi, > > > > > On December 12, 2002 at 8:26 PM, psc...@in... wrote: > > > On 12 Dec 2002 at 7:06, Daniel Rail wrote: > >> I was wondering what the new monitor functionality in IB7 was for. > >> That is a better solution. Also, being able to see what is going > >> on in the transaction would be good too, this is just to make sure > >> not to kill a transaction that is part of a "known" long running > >> process(i.e. year-end processing/reporting). Unless it would be > >> possible to flag a transaction as one that can take time to > >> complete, compared to others. > > > The problem with automatically killing a transaction, or even > > monitoring a transaction and killing it manually, is that it cures > > the symptom, but not the under- lying disease, which is an > > application that is taking resources and holding them when it really > > shouldn't. It never gets fixed properly, because it becomes easier > > to simply let it go, and kill the transaction on occasion. > > I know that a software that is well written, hardly needs a way to > kill a long running transaction, unless it is totally unexpected. In > example: a broken link between client and server due to network > failure, how long does the transaction stays alive and how to kill it > earlier, other than doing a backup and restore. Also, it would be a > nice feature to monitor transactions when doing the development and > testing, since it's possible to create a query or update statement > that's taking longer than expected and want to kill it in order to > make the necessary corrections and run it again, without having to do > a forced shutdown of Firebird. I wonder if we are looking at the whole thing from the wrong perspective. A stalled OIT/OAT causes the engines performance to degrade; so we are looking for ways to not stall the OIT/OAT. That may not be the right approach, if we could find a way that stalling the OIT/OAT doesn't degrade performance, or at least not as quickly then it wouldn't be an issue. In the case of a broken link, then we do detect it, abort it's transactions, and roll them back, eventually. If your application takes too long and you give it the three fingered salute (windows), or issue a kill command (unix), then the transactions it owns should die with the app. I think it does this now... Paul |
|
From: <psc...@in...> - 2002-12-14 19:32:04
|
Ann: As always, when this group gets hot under the collar over an issue, you have the sensible answer to the real problem. On 13 Dec 2002 at 19:01, Ann W. Harrison wrote: > At 05:00 PM 12/13/2002 -0500, psc...@in... wrote: > > >I wonder if we are looking at the whole thing from the wrong > >perspective. A stalled > >OIT/OAT causes the engines performance to degrade; ... if we could > >find a way that stalling the OIT/OAT doesn't degrade performance, or > >at least not as quickly then it wouldn't be an issue. > > This is probably a good time for me to explain OAT, OIT, and their > impact on performance, again. > > The OIT, oldest interesting transaction, is the first transaction that > did not commit. It could be active, but more likely it rolled back or > died and was rolled back. The reason it is "interesting" is that > other transactions must ignore or undo its changes. Firebird creates > an array of bits (two bits per transaction) that represents the states > of all transactions between the current and the oldest interesting. > When that array gets huge, it can affect performance, but not a lot. > To reduce the impact of the oldest interesting transaction, all > read-only transactions commit, regardless of what their client > requested. The OIT is reset in one of two ways: a sweep or a backup > and restore. I guess this is more of a maintenance issue, if you do a backup and sweep every day, then the in most cases the OIT array will never get large enough to cause a problem. > The OAT, oldest active transaction, is the oldest transaction that was > active when any currently active transaction started. Record versions > created by the OAT and subsequent transactions can not be garbage > collected because they might be needed to provide a consistent view > for an active transaction. Read-only read-committed transactions do > not affect the OAT - they can go on forever without affecting garbage > collection. In this case, unlike the OIT, the transaction must be > declared as read-only, not just be de-facto read-only. The OIT is > reset whenever the server is restarted and whenever there are no > active connections. > > The performance impact of the OIT, especially in the SS is probably > not significant. The impact of the OAT is quite significant, > particularly in applications that repeatedly update the same records. > That effect could be reduced by some bookkeeping that told the server > which record versions were important so intermediate versions could be > removed. At the moment, it seems to me that the cost of that > bookkeeping and the complexity of removing intermediate record > versions is not worth the cost and risk. So a 17 hour month end processing program, that used a Read Only read committed transaction in it's main loop and then creates another very short running transaction that does the insert or update followed by a commit every few records or even every record, would have little OAT related performance issues. Since the OAT would not be held up by the long running (read only) transaction. Paul |
|
From: Ann W. H. <aha...@ib...> - 2002-12-13 23:55:25
|
At 05:00 PM 12/13/2002 -0500, psc...@in... wrote: >I wonder if we are looking at the whole thing from the wrong >perspective. A stalled >OIT/OAT causes the engines performance to degrade; ... if we could find a >way that stalling the OIT/OAT doesn't degrade performance, or at least not as >quickly then it wouldn't be an issue. This is probably a good time for me to explain OAT, OIT, and their impact on performance, again. The OIT, oldest interesting transaction, is the first transaction that did not commit. It could be active, but more likely it rolled back or died and was rolled back. The reason it is "interesting" is that other transactions must ignore or undo its changes. Firebird creates an array of bits (two bits per transaction) that represents the states of all transactions between the current and the oldest interesting. When that array gets huge, it can affect performance, but not a lot. To reduce the impact of the oldest interesting transaction, all read-only transactions commit, regardless of what their client requested. The OIT is reset in one of two ways: a sweep or a backup and restore. The OAT, oldest active transaction, is the oldest transaction that was active when any currently active transaction started. Record versions created by the OAT and subsequent transactions can not be garbage collected because they might be needed to provide a consistent view for an active transaction. Read-only read-committed transactions do not affect the OAT - they can go on forever without affecting garbage collection. In this case, unlike the OIT, the transaction must be declared as read-only, not just be de-facto read-only. The OIT is reset whenever the server is restarted and whenever there are no active connections. The performance impact of the OIT, especially in the SS is probably not significant. The impact of the OAT is quite significant, particularly in applications that repeatedly update the same records. That effect could be reduced by some bookkeeping that told the server which record versions were important so intermediate versions could be removed. At the moment, it seems to me that the cost of that bookkeeping and the complexity of removing intermediate record versions is not worth the cost and risk. Regards, Ann www.ibphoenix.com We have answers. |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X