Menu
▾
▴
firebird-devel
|
From: Alex P. <pes...@ma...> - 2011-03-25 12:05:11
|
On 03/24/11 20:15, Adriano dos Santos Fernandes wrote: > On 24-03-2011 12:50, Alex Peshkoff wrote: >> On 03/24/11 18:07, Adriano dos Santos Fernandes wrote: >>> On 24-03-2011 11:56, Alex Peshkoff wrote: >>>>> You see, reference counting alone don't solve the user problem. >>>> When used wrong - yes, do not. But if: >>>> >>>> T1: globalObj->addRef(); >>>> T1: start T2 /* that's using globalObj */ >>>> >>>> It's OK. BTW, it's present in MS docs - "You must also call AddRef on a >>>> pointer before passing it ....". >>>> >>> Reference counting is overloaded to make this work. >> Sorry, it's just words... >> >>> With a single atomic >>> variable user fix the problem without need a polluted API. >>> >> Reference counter is really not more than atomic counter. >> >>> It's user responsibility to use pointers before free them, and I showed >>> you can't fix user mistakes. >> Certainly not. I never pretended to fix user mistakes. >> What can be done - is help to avoid them. And providing functions, >> familiar to many people in other products, is good way to do it. >> >> I do not understand what do you want to show. You say that use of >> refcounted API is polluted and overloaded. But it is used in IT world, >> and it is used actively. Even forgetting OLE2 - OOO is also using it. >> Before saying "I want to suggest something new" it's very good idea to >> look around - what do other people do? And before calling everything you >> do not like for some reason 'ugly, polluted and overloaded' try to >> explain - why? >> > This is from wikipedia: > > "One primary motivation for reference counting in COM is to enable > interoperability across different programming languages and runtime > systems. This is what we also want in API. > A client need only know how to invoke object methods in order > to manage object life cycle; thus, the client is completely abstracted > from whatever memory allocator the implementation of the COM object > uses. As a typical example, a Visual Basic program using a COM object is > agnostic towards whether that object was allocated (and must later be > deallocated) by a C++ allocator or another Visual Basic component. > However, this support for heterogeneity has a major cost: it requires > correct reference count management by all parties involved. While > high-level languages like Visual Basic manage reference counts > automatically, C/C++ programmers are entrusted to increment and > decrement reference counts at the appropriate time. C++ programs can and > should avoid the task of managing reference counts manually by using > smart pointers. Bugs caused by incorrect reference counting in COM > systems are notoriously hard to resolve, This is really main disadvantage of refCounted. > especially because the error > may occur in an opaque, third-party component. > Microsoft has abandoned reference counting in favor of tracing garbage > collection for the .NET Framework." Very strange phrase - .NET is framework of another level, Visual Basic based. Certainly, there is no use in letting VB programmers work with reference counters - this is high-level language, and it never used to work with reference counters manually. > I would add that COM dates 1990 decade, and reference counting is > abandoned in new environments. Do I suggest we add garbage collection to > Firebird API? Of course, no. BTW, smart pointers is a kind of trivial GC mechanism, are not them? > I'm just saying we should add nothing there > as there are nothing to care about. And we known the first paragraph is > solvable by simple "disposable" approach. > >> I agree that there may be other methods to make pointers stable in MT >> world. One of them is to say - we do not care about this, it's user >> responsibility. As for me it's much better to have required support (and >> pay attention - traditional for interfaces). Certainly, if you can >> suggest other approach, which does not let people make errors - let's go >> with it. But please make it not limited with C++ only. >> > As also said in the wikipedia snippet, it's difficult to make reference > count work manually, in languages like C and non-C++ languages expected > to use our API. You also said about user don't need to call it, as then > only a single release is going to destroy the object. So, we're adding > nothing except a complication. I prefer much more things that consistent > fail in dev environment than ones that works in dev and may fail in > production. It's hard not to agree with last phrase - but how is it related with reference counted objects? > Do you like counters and they solve internal problems in good way, go on > and use them. But this is not necessary and is evil in our API. > >> Currently I prefer to stay with traditional reference counters because: >> 1. They are familiar to a lot of programmers - just say 'please work >> with fb3 handles like OLE2 objects when MT safety is needed', and at >> least 70% of people will understand what you mean. >> 2. They do what they are supposed to do - help people work with our API >> in MT safe way. >> 3. They are very simple to implement. >> > You list only the "pros". What about the cons: they are obsolete (1990), Or may be better say 'well tested since 1990 with know pros and cons? > complicated (without smart pointers), If some language does not support smart pointers in any form, it's certainly hard to use it for MT programming. Most popular for today (C++ and Delphi) support them. All that's needed is to have a trivial wrapper around refCounted object. > bugged (reference cycles) You've told many times about that cycles - did they ever hurt us when working with refCounted objects? > and > unnecessary (as I showed in my example, user still need to care when an > object is live). And this is just wrong statement. Without reference counters in API objects support for MT safety in user program (which as far as I remember you suggest people implement themself) becomes much more complicated. Let's suppose that user implemented some helper around IAttachment*, which is not refCounted as you wish. Please explain, how must behave that object when detach is requested, but there are some other threads, using that object. |
|
From: Adriano d. S. F. <adr...@gm...> - 2011-03-25 18:16:52
|
On 25-03-2011 09:05, Alex Peshkoff wrote: > > BTW, smart pointers is a kind of trivial GC mechanism, are not them? > C++ users don't need you adding addRef/release to let them control object life via smart pointers. Boost has a lot of smart pointer classes, and one would just use it (or write one) if needed. > > If some language does not support smart pointers in any form, it's > certainly hard to use it for MT programming. > Most popular for today (C++ and Delphi) support them. All that's needed > is to have a trivial wrapper around refCounted object. > And the wrapper could be the smart pointer itself. No need to have it on our API. > > And this is just wrong statement. Without reference counters in API > objects support for MT safety in user program (which as far as I > remember you suggest people implement themself) becomes much more > complicated. > And you're not fixing MT problems at the necessary levels, because it need user attention. (my example again) > Let's suppose that user implemented some helper around IAttachment*, > which is not refCounted as you wish. Please explain, how must behave > that object when detach is requested, but there are some other threads, > using that object. > Let me first say something about your refcounted approach: Current, when an user detachs and attachment, all handles related to it and not-yet-freed are closed. Let say you did the same when calling IAttachment::detach. Then your refcount don't work, cause it will wipe pointers to objects with references. Let say it don't free the subobjects. Then you're making a common situation (close a parent -> close children) to leak memory. Now about your question, more directly. If this thread is doing something inside the yvalve, the detach may (*internally) decrement a refcounter and don't free the object, so we don't see a crash inside the yvalve. As soon the yvalve release this reference, the object is wiped. Note we don't need any effort to support this. As soon we remove addRef/release from IInterface, we can make YAttachment and friends to inherit from RefCounted and IAttachment and friends. So I insist, moving addRef/release to the API fixes nothing and only complicate things. Transactions has no release, it's freed by commit/rollback. Attachments has no release, it's freed by detach/drop. There is no reason to insert a release function in them that sometimes do a thing (decrement a refcounter) and sometimes do another (rollback or detach). Adriano |
|
From: Alex P. <pes...@ma...> - 2011-03-27 12:57:42
|
On 03/25/11 21:16, Adriano dos Santos Fernandes wrote: > >> Let's suppose that user implemented some helper around IAttachment*, >> which is not refCounted as you wish. Please explain, how must behave >> that object when detach is requested, but there are some other threads, >> using that object. >> > Let me first say something about your refcounted approach: > > Current, when an user detachs and attachment, all handles related to it > and not-yet-freed are closed. > > Let say you did the same when calling IAttachment::detach. Then your > refcount don't work, cause it will wipe pointers to objects with references. > > Let say it don't free the subobjects. Then you're making a common > situation (close a parent -> close children) to leak memory. > > Now about your question, more directly. If this thread is doing > something inside the yvalve, the detach may (*internally) decrement a > refcounter and don't free the object, so we don't see a crash inside the > yvalve. As soon the yvalve release this reference, the object is wiped. This is wrong suggestion. We must return correct status value from detach, but what can we return if we just decrement refcounter? And what should we do when we try to do real job, but something goes wrong? This is specially well seen on a sample with commit and some error thrown when doing DFW. This is the place where presence of reference counters in API objects obviously helps. Certainly, a more or less cunning combination of smart pointer to refcounted helper object, containing lock and a flag, will work (at least I do not see reason why should it not work). But if it's OK for yValve (if we forget about extra performance loss due to the lock), I think that suggesting users to implement something similar themselves is not good idea. Another approach that will sooner of all work is when detach/drop/commit/etc do not dispose/release an object. This must be done explicitly, i.e. the sequence of calls is: ITransaction* tra = att->startTransaction(); // do some job with that transaction tra->commit(); tra->dispose(); This works, but looks bad for ST appliactions. > Note we don't need any effort to support this. As soon we remove > addRef/release from IInterface, we can make YAttachment and friends to > inherit from RefCounted and IAttachment and friends. > > So I insist, moving addRef/release to the API fixes nothing and only > complicate things. Transactions has no release, it's freed by > commit/rollback. Attachments has no release, it's freed by detach/drop. > There is no reason to insert a release function in them that sometimes > do a thing (decrement a refcounter) and sometimes do another (rollback > or detach). What to do if release/dispose is called for not detached/committed object is really hard question. The most logical way is to treat death of the object from release/dispose exactly like from process exit. And this is in accordance with at least OLE DB standards. |
|
From: Adriano d. S. F. <adr...@gm...> - 2011-03-27 15:55:27
|
On 27-03-2011 09:57, Alex Peshkoff wrote: > On 03/25/11 21:16, Adriano dos Santos Fernandes wrote: >> >>> Let's suppose that user implemented some helper around IAttachment*, >>> which is not refCounted as you wish. Please explain, how must behave >>> that object when detach is requested, but there are some other threads, >>> using that object. >>> >> Let me first say something about your refcounted approach: >> >> Current, when an user detachs and attachment, all handles related to it >> and not-yet-freed are closed. >> >> Let say you did the same when calling IAttachment::detach. Then your >> refcount don't work, cause it will wipe pointers to objects with references. >> >> Let say it don't free the subobjects. Then you're making a common >> situation (close a parent -> close children) to leak memory. >> >> Now about your question, more directly. If this thread is doing >> something inside the yvalve, the detach may (*internally) decrement a >> refcounter and don't free the object, so we don't see a crash inside the >> yvalve. As soon the yvalve release this reference, the object is wiped. > > This is wrong suggestion. We must return correct status value from > detach, but what can we return if we just decrement refcounter? And what > should we do when we try to do real job, but something goes wrong? > This is specially well seen on a sample with commit and some error > thrown when doing DFW. > No, Alex. We don't delay the detach in the provider object. We delay "delete this" on the yvalve object. And since the user access always via the yvalve object, the refcounting on the provider object becomes unnecessary too. > Another approach that will sooner of all work is when > detach/drop/commit/etc do not dispose/release an object. This must be > done explicitly, i.e. the sequence of calls is: > > ITransaction* tra = att->startTransaction(); > // do some job with that transaction > tra->commit(); > tra->dispose(); > > This works, but looks bad for ST appliactions. > Don't think this is ok. We need dispose for things who only need dispose, and we don't need it in objects that must be freed by another ways. >> Note we don't need any effort to support this. As soon we remove >> addRef/release from IInterface, we can make YAttachment and friends to >> inherit from RefCounted and IAttachment and friends. >> >> So I insist, moving addRef/release to the API fixes nothing and only >> complicate things. Transactions has no release, it's freed by >> commit/rollback. Attachments has no release, it's freed by detach/drop. >> There is no reason to insert a release function in them that sometimes >> do a thing (decrement a refcounter) and sometimes do another (rollback >> or detach). > > What to do if release/dispose is called for not detached/committed > object is really hard question. > The most logical way is to treat death of the object from > release/dispose exactly like from process exit. > And this is in accordance with at least OLE DB standards. > It's hard question but has simple answer. Objects which has commit/rollback or detach/drop should not have release. Adriano |
|
From: Alex P. <pes...@ma...> - 2011-03-28 07:36:58
|
On 03/27/11 19:55, Adriano dos Santos Fernandes wrote: > On 27-03-2011 09:57, Alex Peshkoff wrote: >> On 03/25/11 21:16, Adriano dos Santos Fernandes wrote: >>>> Let's suppose that user implemented some helper around IAttachment*, >>>> which is not refCounted as you wish. Please explain, how must behave >>>> that object when detach is requested, but there are some other threads, >>>> using that object. >>>> >>> Let me first say something about your refcounted approach: >>> >>> Current, when an user detachs and attachment, all handles related to it >>> and not-yet-freed are closed. >>> >>> Let say you did the same when calling IAttachment::detach. Then your >>> refcount don't work, cause it will wipe pointers to objects with references. >>> >>> Let say it don't free the subobjects. Then you're making a common >>> situation (close a parent -> close children) to leak memory. >>> >>> Now about your question, more directly. If this thread is doing >>> something inside the yvalve, the detach may (*internally) decrement a >>> refcounter and don't free the object, so we don't see a crash inside the >>> yvalve. As soon the yvalve release this reference, the object is wiped. >> This is wrong suggestion. We must return correct status value from >> detach, but what can we return if we just decrement refcounter? And what >> should we do when we try to do real job, but something goes wrong? >> This is specially well seen on a sample with commit and some error >> thrown when doing DFW. >> > No, Alex. We don't delay the detach in the provider object. We delay > "delete this" on the yvalve object. And since the user access always via > the yvalve object, the refcounting on the provider object becomes > unnecessary too. > But such behavior can cause races. Suppose attachment helper is used by 2 threads. One wants to access it, checks for object's presence, and copies pointer to IAttachment into the register (it's needed at least to access VTable). At this moment thread gets swapped from CPU by OS scheduler. Another thread performs detach and the pointer (already copied into the register) becomes invalid. But when first thread gets control it does not know that pointer became invalid - and segfault happens. Be sure - this is not just theory. It happens in 2.5 when it's attempted to be shutdown under very high load, and races in yvalve take more time than unallocated block stays in memory pool cache. IMO we should fix this at least in FB3. Certainly, more clever helper object can solve a problem. But I do not see a way to build it without lock. And I claim that this is too high price for removing reference counters from API. >> Another approach that will sooner of all work is when >> detach/drop/commit/etc do not dispose/release an object. This must be >> done explicitly, i.e. the sequence of calls is: >> >> ITransaction* tra = att->startTransaction(); >> // do some job with that transaction >> tra->commit(); >> tra->dispose(); >> >> This works, but looks bad for ST appliactions. >> > Don't think this is ok. We need dispose for things who only need > dispose, and we don't need it in objects that must be freed by another ways. > Having dispose is also a way to avoid races when detach() happens. But I prefer reference counters. |
|
From: Adriano d. S. F. <adr...@gm...> - 2011-03-28 14:19:05
|
On 28-03-2011 04:36, Alex Peshkoff wrote: > On 03/27/11 19:55, Adriano dos Santos Fernandes wrote: >> On 27-03-2011 09:57, Alex Peshkoff wrote: >>> On 03/25/11 21:16, Adriano dos Santos Fernandes wrote: >>>>> Let's suppose that user implemented some helper around IAttachment*, >>>>> which is not refCounted as you wish. Please explain, how must behave >>>>> that object when detach is requested, but there are some other threads, >>>>> using that object. >>>>> >>>> Let me first say something about your refcounted approach: >>>> >>>> Current, when an user detachs and attachment, all handles related to it >>>> and not-yet-freed are closed. >>>> >>>> Let say you did the same when calling IAttachment::detach. Then your >>>> refcount don't work, cause it will wipe pointers to objects with references. >>>> >>>> Let say it don't free the subobjects. Then you're making a common >>>> situation (close a parent -> close children) to leak memory. >>>> >>>> Now about your question, more directly. If this thread is doing >>>> something inside the yvalve, the detach may (*internally) decrement a >>>> refcounter and don't free the object, so we don't see a crash inside the >>>> yvalve. As soon the yvalve release this reference, the object is wiped. >>> This is wrong suggestion. We must return correct status value from >>> detach, but what can we return if we just decrement refcounter? And what >>> should we do when we try to do real job, but something goes wrong? >>> This is specially well seen on a sample with commit and some error >>> thrown when doing DFW. >>> >> No, Alex. We don't delay the detach in the provider object. We delay >> "delete this" on the yvalve object. And since the user access always via >> the yvalve object, the refcounting on the provider object becomes >> unnecessary too. >> > > But such behavior can cause races. Suppose attachment helper is used by > 2 threads. One wants to access it, checks for object's presence, and > copies pointer to IAttachment into the register (it's needed at least to > access VTable). At this moment thread gets swapped from CPU by OS > scheduler. Another thread performs detach and the pointer (already > copied into the register) becomes invalid. But when first thread gets > control it does not know that pointer became invalid - and segfault > happens. Be sure - this is not just theory. It happens in 2.5 when it's > attempted to be shutdown under very high load, and races in yvalve take > more time than unallocated block stays in memory pool cache. IMO we > should fix this at least in FB3. > > Certainly, more clever helper object can solve a problem. But I do not > see a way to build it without lock. And I claim that this is too high > price for removing reference counters from API. > The user should not call detach and another methods simultaneously, never. Why the heck they would do it? This is user application problem. Users are supposed to know basic how-to-work with pointers, otherwise they don't even can get a pointer to the object. If they want to crash the local system you can do nothing about it. What I said about internal refcounter helping here is about the legacy handles. Adriano |
|
From: Alex P. <pes...@ma...> - 2011-03-28 14:47:37
|
On 03/28/11 18:18, Adriano dos Santos Fernandes wrote: > The user should not call detach and another methods simultaneously, > never. Why the heck they would do it? This is user application problem. Why not? User has background thread which on the regular basis performs some actions with database. When database is close, that thread must close. I think this is not user application problem, but a problem of API, suggested not taking into an account MT specifics. > Users are supposed to know basic how-to-work with pointers, otherwise > they don't even can get a pointer to the object. If they want to crash > the local system you can do nothing about it. Sometimes I can. With reference counted objects. At least I can help them to not crash when it can be avoided. > What I said about internal refcounter helping here is about the legacy > handles. Leave them in peace for a while - let's first understand about new API. |
|
From: Adriano d. S. F. <adr...@gm...> - 2011-03-28 14:56:29
|
On 28-03-2011 11:47, Alex Peshkoff wrote: > On 03/28/11 18:18, Adriano dos Santos Fernandes wrote: > >> The user should not call detach and another methods simultaneously, >> never. Why the heck they would do it? This is user application problem. > > Why not? > User has background thread which on the regular basis performs some > actions with database. When database is close, that thread must close. I > think this is not user application problem, but a problem of API, > suggested not taking into an account MT specifics. > >> Users are supposed to know basic how-to-work with pointers, otherwise >> they don't even can get a pointer to the object. If they want to crash >> the local system you can do nothing about it. > > Sometimes I can. With reference counted objects. At least I can help > them to not crash when it can be avoided. > Yes, you can. We can do everything, that don't mean it's the right choice. Support (which the cost of spoil an API) a user mistake about detaching an attachment and having concurrent threads calling functions on it and hiding or throwing another errors in destructors is sure a wrong choice. Adriano |
|
From: Vlad K. <hv...@us...> - 2011-03-28 18:28:27
|
>>> The user should not call detach and another methods simultaneously,
>>> never. Why the heck they would do it? This is user application problem.
Do you offer us to undo thread-safety changes made in v2.5 client library ? :)
>> Why not?
>> User has background thread which on the regular basis performs some
>> actions with database. When database is close, that thread must close. I
>> think this is not user application problem, but a problem of API,
>> suggested not taking into an account MT specifics.
>>
>>> Users are supposed to know basic how-to-work with pointers, otherwise
>>> they don't even can get a pointer to the object. If they want to crash
>>> the local system you can do nothing about it.
We can't prevent crash in user code, yes. But we must prevent crash in our code, always.
Even if we got crazy wrong input parameters. And in reality we already have all necessary
syncronization around handles, sockets, etc. So, why force user to repeat the same code
making applications less efficient ?
>> Sometimes I can. With reference counted objects. At least I can help
>> them to not crash when it can be avoided.
>>
> Yes, you can. We can do everything, that don't mean it's the right choice.
>
> Support (which the cost of spoil an API) a user mistake about detaching
> an attachment and having concurrent threads calling functions on it and
> hiding or throwing another errors in destructors is sure a wrong choice.
Nobody going to hide errors. With refcounting at proxy objects concurrent thread will receive
isc_bad_XXX_handle error, instead of crash. Should i said that destructor will not throw ? :)
Regards,
Vlad
|
|
From: Adriano d. S. F. <adr...@gm...> - 2011-03-28 18:45:40
|
On 28-03-2011 15:28, Vlad Khorsun wrote: >>>> The user should not call detach and another methods simultaneously, >>>> never. Why the heck they would do it? This is user application problem. > > Do you offer us to undo thread-safety changes made in v2.5 client library ? :) > This has *nothing* related, as 2.5 has no public objects, only handles, and nothing in 3.0 changes about it. I'm just saying that we should not protect users from using unallocated objects at a price of adding confusing and non-needed functions in our API. >>> Why not? >>> User has background thread which on the regular basis performs some >>> actions with database. When database is close, that thread must close. I >>> think this is not user application problem, but a problem of API, >>> suggested not taking into an account MT specifics. >>> >>>> Users are supposed to know basic how-to-work with pointers, otherwise >>>> they don't even can get a pointer to the object. If they want to crash >>>> the local system you can do nothing about it. > > We can't prevent crash in user code, yes. But we must prevent crash in our code, always. > Even if we got crazy wrong input parameters. And in reality we already have all necessary > syncronization around handles, sockets, etc. So, why force user to repeat the same code > making applications less efficient ? > >>> Sometimes I can. With reference counted objects. At least I can help >>> them to not crash when it can be avoided. >>> >> Yes, you can. We can do everything, that don't mean it's the right choice. >> >> Support (which the cost of spoil an API) a user mistake about detaching >> an attachment and having concurrent threads calling functions on it and >> hiding or throwing another errors in destructors is sure a wrong choice. > > Nobody going to hide errors. With refcounting at proxy objects concurrent thread will receive > isc_bad_XXX_handle error, instead of crash. Should i said that destructor will not throw ? :) > User should *not* request a detachment and then request actions on the objects. This *is* user application problem, no mater it being single or multi thread. You and Alex can still use yours refcounting, and in fact I'm still using them on my new why.cpp to fix existing problems. But don't add it to our API, and we don't need it even if you want to use proxy objects in the engine. It's just a mater of using it internally. Yvalve do refcounting and don't call methods on unallocated objects. Engine proxies do whatever they need and don't call methods on unallocated objects. But, please, add addRef/release to public API to prevent user from do mistakes at the price of do even more evil is not correct. Adriano |
|
From: Alex P. <pes...@ma...> - 2011-03-29 13:45:24
|
We have too many problems in single thread. I try to divide it to smaller parts. The first one - here we at least all seem to talk about same thing:-) - is to use reference counters or not. On 03/28/11 22:45, Adriano dos Santos Fernandes wrote: > User should *not* request a detachment and then request actions on the > objects. This *is* user application problem, no mater it being single or > multi thread. In that case our new API is regression compared with old ISC API with handles. Pre FB3 an attempt to request actions on the objects after detachment cause correct reply from client library - invalid handle. Without reference count support we will get segfault in FB3. I treat this as serious regression. > You and Alex can still use yours refcounting, They are not our :-) Could one of us be an authors of such great solution! > and in fact I'm still > using them on my new why.cpp to fix existing problems. > > But don't add it to our API, As one of users replied: Object-based interfaces should use the refcount. I and Vlad totally agree with him. Let's look at use of refcount in API from 3 different points of view: languages that support smart pointers (C++, Delphi), languages that perform GC themselves (.Net platform is a good sample) and old-stylers like plain-C. In the first case people are free to make a choice. If they wish, they use smart pointers to API objects. Also they may forget about reference counters, and write as if there is no such functions at all, synchronizing access to API objects themself. IMHO manual control (calling addRef/release when needed) is not good idea for such languages. What about high-level case - end-users just never see raw API. Solution how to work with API will be taken by people, who write connectors/providers/etc for that platforms. We know what IB .net provider author thinks about it. In the third case (though telling true it's too brave to start MT project on such language) people can synchronize access to API objects themselves, as if there is no reference counters at all. Or use them - it's there choice. For example: method to call a kind of dtor when thread terminates do exist, calling addRef when is also not too hard task, therefore who knows what will be more reliable. Reference counters really have one big problem - when used incorrectly, related bugs are hard to fix. But missing them can sometimes cause even more serious problems. Therefore we (I and Vlad) suggest to give people freedom of choice regarding this problem. If someone wants to use them - we must provide that support. If not - everyone is free to provide his own synchronization. |
|
From: Dimitry S. <sd...@ib...> - 2011-03-29 14:12:26
|
29.03.2011 15:45, Alex Peshkoff wrote: > On 03/28/11 22:45, Adriano dos Santos Fernandes wrote: >> User should *not* request a detachment and then request actions on the >> objects. This *is* user application problem, no mater it being single or >> multi thread. > > In that case our new API is regression compared with old ISC API with > handles. Pre FB3 an attempt to request actions on the objects after > detachment cause correct reply from client library - invalid handle. > Without reference count support we will get segfault in FB3. I treat > this as serious regression. You can't avoid segfaults if you are going to reveal pointers in API at all. No matter whether this is pointers to objects (as in proposed API) or structures (as in original GDS design). No way for refcounters can prevent use of invalid pointers. Call of (IAttachment*)(123456)->AnyRoutine() will crash. -- SY, SD. |
|
From: Alex P. <pes...@ma...> - 2011-03-30 09:56:41
|
On 03/29/11 18:11, Dimitry Sibiryakov wrote: > 29.03.2011 15:45, Alex Peshkoff wrote: >> On 03/28/11 22:45, Adriano dos Santos Fernandes wrote: >>> User should *not* request a detachment and then request actions on the >>> objects. This *is* user application problem, no mater it being single or >>> multi thread. >> In that case our new API is regression compared with old ISC API with >> handles. Pre FB3 an attempt to request actions on the objects after >> detachment cause correct reply from client library - invalid handle. >> Without reference count support we will get segfault in FB3. I treat >> this as serious regression. > You can't avoid segfaults if you are going to reveal pointers in API at all. No matter > whether this is pointers to objects (as in proposed API) or structures (as in original GDS > design). No way for refcounters can prevent use of invalid pointers. Call of > (IAttachment*)(123456)->AnyRoutine() will crash. Dmitry, please do not mix artificially created pointers (BTW, why so complex example with 123456? 0 will cause crash too.) with pointers, that were correct, but were destroyed in another thread. Your sample shows an error in the program. On the other hand, access to a pointer that was correct, but became invalid, is rather typical case when dealing with MT program. I must say that this is not artificial theoretical problem. We had that kind of races in yValve in firebird 2.5 before internal objects (CAttachment, CTransaction, etc.) were not made reference counted and pointers to them were not made smart. Dmitry Kovalenko watched 'invalid handle value' error in his IB provider. What else arguments do you need to let people use API with protected from disappearance pointers? |
|
From: Dimitry S. <sd...@ib...> - 2011-03-30 10:13:36
|
30.03.2011 11:56, Alex Peshkoff wrote: > Dmitry, please do not mix artificially created pointers (BTW, why so > complex example with 123456? 0 will cause crash too.) with pointers, > that were correct, but were destroyed in another thread. Your sample > shows an error in the program. On the other hand, access to a pointer > that was correct, but became invalid, is rather typical case when > dealing with MT program. There is no difference. Whether pointer was invalid from the beginning or become, result is the same. AFAIU, you want to protect API latter with reference counter, but it will work only in correctly written programs. But such programs work correctly without reference counters. So, what's the point? > Dmitry Kovalenko watched 'invalid > handle value' error in his IB provider. What else arguments do you need > to let people use API with protected from disappearance pointers? I'd like to be sure that such protection will have any advantage over current abstract handles. So far I don't see in proposals any difference from current design, so I don't understand the purpose of changes. -- SY, SD. |
|
From: Vlad K. <hv...@us...> - 2011-03-30 10:32:27
|
> 30.03.2011 11:56, Alex Peshkoff wrote:
>> Dmitry, please do not mix artificially created pointers (BTW, why so
>> complex example with 123456? 0 will cause crash too.) with pointers,
>> that were correct, but were destroyed in another thread. Your sample
>> shows an error in the program. On the other hand, access to a pointer
>> that was correct, but became invalid, is rather typical case when
>> dealing with MT program.
>
> There is no difference. Whether pointer was invalid from the beginning or become,
> result is the same.
Your example is a crash in *user* code, not in ours.
> AFAIU, you want to protect API latter with reference counter, but it
> will work only in correctly written programs. But such programs work correctly without
> reference counters. So, what's the point?
Reference counting prevents MT races in most cases, it is already used by our code
internally and allows user to not add additional locking layer for MT safety.
>> Dmitry Kovalenko watched 'invalid
>> handle value' error in his IB provider. What else arguments do you need
>> to let people use API with protected from disappearance pointers?
>
> I'd like to be sure that such protection will have any advantage over current abstract
> handles. So far I don't see in proposals any difference from current design, so I don't
> understand the purpose of changes.
Handles always checked before dereferencing. It have its cost both in memory usage and
CPU instructions.
Interface pointers needs no implicit checks by our code because it will crash in user code
when VMT is accessed.
Do you see the difference ?
Regards,
Vlad
|
|
From: Dimitry S. <sd...@ib...> - 2011-03-30 10:46:21
|
30.03.2011 12:32, Vlad Khorsun wrote: >> There is no difference. Whether pointer was invalid from the beginning or become, >> result is the same. > > Your example is a crash in *user* code, not in ours. Only if called function is virtual and is called via reference to base class. Otherwise crash will occur in _your_ code as soon as it try to access any member variable. > Reference counting prevents MT races in most cases, it is already used by our code > internally and allows user to not add additional locking layer for MT safety. Ok, here is a code: Interface* a = new RefCountedClass; Interface* b = NULL; b = a; delete a; b->Anything(); I don't understand how reference counting works here. Could you explain? -- SY, SD. |
|
From: Vlad K. <hv...@us...> - 2011-03-30 11:04:21
|
> 30.03.2011 12:32, Vlad Khorsun wrote:
>>> There is no difference. Whether pointer was invalid from the beginning or become,
>>> result is the same.
>>
>> Your example is a crash in *user* code, not in ours.
>
> Only if called function is virtual and is called via reference to base class. Otherwise
> crash will occur in _your_ code as soon as it try to access any member variable.
Do you know what is interfaces ? It is pure virtual classes and all methods is virtual.
>> Reference counting prevents MT races in most cases, it is already used by our code
>> internally and allows user to not add additional locking layer for MT safety.
>
> Ok, here is a code:
>
> Interface* a = new RefCountedClass;
> Interface* b = NULL;
> b = a;
> delete a;
> b->Anything();
>
> I don't understand how reference counting works here. Could you explain?
This code can't be used as example because user physically can't call new\delete for
interface pointer obtained from our code. See below (simplificated):
SmartPtr<IAttachment*> att = fb->attachDatabase(...); // refcount == 1
SmartPtr<IAttachment*> b = att; // recount = 2
b = NULL; // refcount = 1
a->detach(); // refcount = 1 but detach really happens !!!
a->XXX // refcount = 1, returns isc_bad_db_handle
a = NULL; // refcount = 0, YValve-s object freed
With Delphi you will just use IAttachment, without SmartPtr, as Delphi's "interface" have
built-in compiler magic to work with reference counts.
Regards,
Vlad
|
|
From: Adriano d. S. F. <adr...@gm...> - 2011-03-30 15:33:55
|
On 30-03-2011 08:04, Vlad Khorsun wrote: > > With Delphi you will just use IAttachment, without SmartPtr, as Delphi's "interface" have > built-in compiler magic to work with reference counts. > AFAIK, this is not how it works. Delphi "interface" has nothing directly related with refcount. Refcount comes on when IInterface is used, but IInterface is about COM (has queryInterface). I don't think you can make our interfaces to work with IInterface, so you won't get RAII with Delphi and our interfaces. Adriano |
|
From: Daniel R. <da...@ac...> - 2011-03-30 17:20:12
|
X-UCS- Spam Mail 2 Hi, Just to clarify a few things with relations to Delphi. At March-30-11, 12:33 PM, Adriano dos Santos Fernandes wrote: > On 30-03-2011 08:04, Vlad Khorsun wrote: >> >> With Delphi you will just use IAttachment, without SmartPtr, as Delphi's "interface" have >> built-in compiler magic to work with reference counts. >> > AFAIK, this is not how it works. > Delphi "interface" has nothing directly related with refcount. Actually, Vlad is correct. Here's what is written in Delphi's help for IInterface: "IInterface introduces the QueryInterface method, which is useful for discovering and using other interfaces implemented by the same object. IInterface also introduces the reference counting methods, _AddRef and _Release. The Delphi compiler automatically provides calls to these methods when interfaces are used. The TInterfacedObject class is provided as a base class for objects that implement interfaces. Use of TInterfacedObject is not mandatory, but will often be easier than designing an implementer class from scratch." > Refcount comes on when IInterface is used, but IInterface is about COM > (has queryInterface). I don't think you can make our interfaces to work > with IInterface, so you won't get RAII with Delphi and our interfaces. You are correct when you say "Refcount comes on when IInterface is used". But, Delphi's IInterface has nothing to do with COM, as I can use IInterface in an application, without ever using COM. And, there are some that was able to implement RAII in Delphi, with the use of IInterface (I did a search on Google). -- Best regards, Daniel Rail Senior Software Developer ACCRA Solutions Inc. (www.accra.ca) ACCRA Med Software Inc. (www.filopto.com) |
|
From: Adriano d. S. F. <adr...@gm...> - 2011-03-30 17:33:43
|
On 30-03-2011 14:16, Daniel Rail wrote: > X-UCS- Spam Mail 2 > > Hi, > > Just to clarify a few things with relations to Delphi. > > At March-30-11, 12:33 PM, Adriano dos Santos Fernandes wrote: > >> On 30-03-2011 08:04, Vlad Khorsun wrote: >>> >>> With Delphi you will just use IAttachment, without SmartPtr, as Delphi's "interface" have >>> built-in compiler magic to work with reference counts. >>> >> AFAIK, this is not how it works. > >> Delphi "interface" has nothing directly related with refcount. > > Actually, Vlad is correct. > Here's what is written in Delphi's help for IInterface: > IInterface is a "interface" with three methods. This interface is understood by the Delphi compiler and makes RAII works. > "IInterface introduces the QueryInterface method, which is useful > for discovering and using other interfaces implemented by the same > object. IInterface also introduces the reference counting methods, > _AddRef and _Release. The Delphi compiler automatically provides > calls to these methods when interfaces are used. The > TInterfacedObject class is provided as a base class for objects that > implement interfaces. Use of TInterfacedObject is not mandatory, but > will often be easier than designing an implementer class from > scratch." > >> Refcount comes on when IInterface is used, but IInterface is about COM >> (has queryInterface). I don't think you can make our interfaces to work >> with IInterface, so you won't get RAII with Delphi and our interfaces. > > You are correct when you say "Refcount comes on when IInterface is > used". But, Delphi's IInterface has nothing to do with COM, as I can > use IInterface in an application, without ever using COM. And, there > are some that was able to implement RAII in Delphi, with the use of > IInterface (I did a search on Google). > And IInterface is incompatible with our Interface class, cause our don't have QueryInterface. So our Interface must be declared as: Interface = interface ... end; and not: Interface = interface(IInterface) ... end; Hence you can't make the Firebird objects uses Delphi's RAII. Adriano |
|
From: Vlad K. <hv...@us...> - 2011-03-30 17:43:16
|
> And IInterface is incompatible with our Interface class, cause our don't
> have QueryInterface.
Yes, and this is sad but don't prevent Delphi to work with our Interface.
> So our Interface must be declared as:
> Interface = interface
> ...
> end;
>
> and not:
> Interface = interface(IInterface)
> ...
> end;
This is he same ;)
----
The declaration of an interface can specify an ancestor interface. If no ancestor is specified,
the interface is a direct descendant of IInterface, which is defined in the System unit and is
the ultimate ancestor of all other interfaces.
----
> Hence you can't make the Firebird objects uses Delphi's RAII.
Not directly, unfortunately.
Regards,
Vlad
|
|
From: Vlad K. <hv...@us...> - 2011-03-30 17:40:49
|
> On 30-03-2011 08:04, Vlad Khorsun wrote:
>>
>> With Delphi you will just use IAttachment, without SmartPtr, as Delphi's "interface" have
>> built-in compiler magic to work with reference counts.
>>
> AFAIK, this is not how it works.
>
> Delphi "interface" has nothing directly related with refcount.
>
> Refcount comes on when IInterface is used, but IInterface is about COM
> (has queryInterface). I don't think you can make our interfaces to work
> with IInterface, so you won't get RAII with Delphi and our interfaces.
Therefore it will be very desirable to add queryInterface to the our base interface,
even empty or raising notImplemented error. IUnknown *is* industry standard, despite
of our wish to accept it.
Simple Delphi wrappers could be written to deal with reference counters but
i prefer to add queryInterface to the our interfaces.
Regards,
Vlad
|
|
From: Adriano d. S. F. <adr...@gm...> - 2011-03-30 19:13:06
|
On 30-03-2011 14:40, Vlad Khorsun wrote: >> On 30-03-2011 08:04, Vlad Khorsun wrote: >>> >>> With Delphi you will just use IAttachment, without SmartPtr, as Delphi's "interface" have >>> built-in compiler magic to work with reference counts. >>> >> AFAIK, this is not how it works. >> >> Delphi "interface" has nothing directly related with refcount. >> >> Refcount comes on when IInterface is used, but IInterface is about COM >> (has queryInterface). I don't think you can make our interfaces to work >> with IInterface, so you won't get RAII with Delphi and our interfaces. > > Therefore it will be very desirable to add queryInterface to the our base interface, > even empty or raising notImplemented error. IUnknown *is* industry standard, despite > of our wish to accept it. > > Simple Delphi wrappers could be written to deal with reference counters but > i prefer to add queryInterface to the our interfaces. > No, it's not desirable to add bogus method to our interface. We also use different calling convention than COM, otherwise our versioning system don't work. Also: - Interfaces are something completely unknown for most Delphi developers. It's used only for interact with COM and somethings like shell objects (also COM) in the Windows API. It's completely unused in Delphi visual components AFAIK. - It's sad that (if) Delphi creators still don't put in the language a way to support RAII decently. - We're create an API that almost no Delphi user will use directly. It's probably even many years to component creators starts using it. So it's okay to let them explicitly call a method to release an object, like they need to destroy any other object, cause they language don't support RAII. Adriano |
|
From: Daniel R. <da...@ac...> - 2011-03-30 19:45:45
|
X-UCS- Spam Mail 2 Hi, At March-30-11, 4:12 PM, Adriano dos Santos Fernandes wrote: > Also: > - Interfaces are something completely unknown for most Delphi > developers. It's used only for interact with COM and somethings like > shell objects (also COM) in the Windows API. It's completely unused in > Delphi visual components AFAIK. And, also to help develop an application with plugins(without using COM). Also, interfaces are used in the VCL(although not everywhere), and the data connectivity components. > - It's sad that (if) Delphi creators still don't put in the language a > way to support RAII decently. I do agree here. Yet, there is a way to implement it in a Delphi interface. > - We're create an API that almost no Delphi user will use directly. It's > probably even many years to component creators starts using it. So it's > okay to let them explicitly call a method to release an object, like > they need to destroy any other object, cause they language don't support > RAII. Most Delphi developers wouldn't be using the Firebird API directly(at least client side), they will be using third-party components that are wrappers(i.e.: IB Objects, FIB Plus, UIB, etc...). The authors of those components are most likely the ones that will be using the Firebird API. For the server side, I know that I most likely will be creating a plugin for the security(authentication) that will be related to our software, and most likely it will be developed using C++. And, the main reason for using C++ in this case is that Delphi doesn't currently support 64-bit development(although it is planned for later this year, but I'm not holding my breath since it was on the roadmap for the last 4 versions). -- Best regards, Daniel Rail Senior Software Developer ACCRA Solutions Inc. (www.accra.ca) ACCRA Med Software Inc. (www.filopto.com) |
|
From: Vlad K. <hv...@us...> - 2011-03-30 20:09:22
|
>> Therefore it will be very desirable to add queryInterface to the our base interface,
>> even empty or raising notImplemented error. IUnknown *is* industry standard, despite
>> of our wish to accept it.
>>
>> Simple Delphi wrappers could be written to deal with reference counters but
>> i prefer to add queryInterface to the our interfaces.
>>
> No, it's not desirable to add bogus method to our interface.
Make it not bogus
> We also use
> different calling convention than COM, otherwise our versioning system
> don't work.
Here i don't understand you. Also, please, don't use wors "COM" everywhere, this just
wrong and annoying.
> Also:
> - Interfaces are something completely unknown for most Delphi
> developers.
You are terrible wrong here
> It's used only for interact with COM and somethings like
> shell objects (also COM) in the Windows API.
and here.
> It's completely unused in
> Delphi visual components AFAIK.
and here too.
> - It's sad that (if) Delphi creators still don't put in the language a
> way to support RAII decently.
There is no stack allocated objects, yes. But if you ask google, you'll find a samples of
how to use interfaces to implement RAII. It is not as natural as in C++, but it is exists.
> - We're create an API that almost no Delphi user will use directly.
Again wrong assumption (based on zero knowledge of Delphi ?)
> It's probably even many years to component creators starts using it.
We have contacts with vendors of most used component libraries. It is more than enough
to have support of new API from them. Moreover, it will be not too hard to create Delphi unit
with necessary declarations (and wrappers, if needed) to me.
Regards,
Vlad
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X