|
From: Michael C. <mic...@gm...> - 2008-06-28 09:03:15
Attachments:
dynamicDispatch.patch
|
Hi all, I am one of the core developers for the NVDA screen reader at http://www.nvda-project.org/ We use comtypes very heavily in our project, mostly for access to MSAA, RichEdit, MSHTML, and SAPI speech synthesizers. However, for access to such apps as MS Word, MS Excel etc, we tend to use pywin32, for the single reason that comtypes interfaces for these apps can take so long to generate, plus pywin32 has very good dynamic IDispatch support. Pywin32 is great, but it is annoying switching between python COM implementations, we would rather just use one or the other. This is the reason why I have created a patch for latest comtypes trunk, which improves its support for dynamic IDispatch, making it now possible to fully depend only on dynamic IDispatch, with out having to generate any interfaces when going from one IDispatch object to another etc. Firstly, a variable called 'disableCodeGeneration' has been added to the base comtypes package. If you do not wish for any COM interfaces to be generated by comtypes if it has to, set this to True. It is False by default. comtypes.client._generate._createWrapper now throws an exception, if disableCodeGeneration is True and it finally gets to the point where it must generate some interfaces (they didn't exist already or they are out of date). comtypes.client.GetBestInterface now tries returning a dynamic.Dispatch object if at any time there is an error. e.g. the interface couldn't be found or created etc. So with just these changes, if you have no already generated interfaces, and you make a dynamic Dispatch object some how, now if any of its methods or properties return IDispatch objects, rather than them being smartly wrapped in real generated interfaces, dynamic Dispatch objects are just returned instead. So the rule is: with disableCodeGeneration set to True, if the interface already exists it will use it, but if it doesn't and it supports IDispatch, it will use dynamic Dispatch, rather than trying to generate proper interfaces. I also made some changes to comtypes.client.dynamic._Dispatch, so that it works nicer with properties and methods. It now makes use of ITypeComp, using Bind to get IDs and invKind info from a name. It also caches as much information as it can to make sure its as fast as possible. Note that information is cached by its guid and name, on the actual dynamic Dispatch class, so that multiple uses of the same IDispatch type stay just as fast. So now, the only time an IDispatch property is treeted as a property (i.e. doesn't have to be called like a function) is if its invkind is propget and it has no params or optional params. Otherwise the method caller is used instead. examples in MS Word are: range.text is a property range.collapse() is a method examples in MSAA are: pacc.accName() or pacc.accName(0) The property has an optional argument so therefore it has to be treeted as a method. pacc.accFocus is a property with no arguments at all, so it can be treeted as a property. __call__ has also been added to _Dispatch, so that the IDispatch's default method can be called, by calling the dynamic Dispatch object like a function. An example of this is an MS Word paragraphs object, calling paragraphs(1) will now work. Also __enum now tries querying directly to IEnumVARIANT, if it fails to call _enum. This now means that MSAA objects that have an IEnumVARIANT as as one of their interfaces will now work pythonicly as a list/iterater. This patch makes dynamic._Dispatch rely on the fact that the IDispatch object has ITypeInfo via GetTypeInfo(0). The question i have is is this true for all IDispatch objects? or should we still support the more basic GetIdsOfNames and then just treet all properties as methods, if ITypeComp can't be found? A completely different approach to this could be where if a dynamic Dispatch object calls invoke, and invoke returns an IDispatch, then we should wrap it as a dynamic Dispatch object rather than getting its best interface. However, I am not sure if this could currently be at all possible in comtypes as we would need to be able to detect the fact that invoke was called from a dynamic Dispatch object, as its result gets wrapped... at a very low level. I think for now, allowing the user to refuse code generation better suits the pattern of use. What are people's thoughts on this? Mick |
|
From: Michael C. <mic...@gm...> - 2008-07-01 19:48:11
Attachments:
dynamicDispatch.patch
|
Hi all, I have attached a new version of my dynamic dispatch patch. This time, if an IDispatch interface does not support ITypeInfo, then dynamic dispatch falls back to using the old GetIDsOfNames etc. This is so far mainly there for the Mozilla Gecko toolkit, as its IDispatch with MSAA is hardly complete. Mick Michael Curran wrote: > Hi all, > > I am one of the core developers for the NVDA screen reader at > http://www.nvda-project.org/ > > We use comtypes very heavily in our project, mostly for access to MSAA, > RichEdit, MSHTML, and SAPI speech synthesizers. > > However, for access to such apps as MS Word, MS Excel etc, we tend to > use pywin32, for the single reason that comtypes interfaces for these > apps can take so long to generate, plus pywin32 has very good dynamic > IDispatch support. > > Pywin32 is great, but it is annoying switching between python COM > implementations, we would rather just use one or the other. > > This is the reason why I have created a patch for latest comtypes trunk, > which improves its support for dynamic IDispatch, making it now possible > to fully depend only on dynamic IDispatch, with out having to generate > any interfaces when going from one IDispatch object to another etc. > > Firstly, a variable called 'disableCodeGeneration' has been added to the > base comtypes package. If you do not wish for any COM interfaces to be > generated by comtypes if it has to, set this to True. It is False by > default. > > comtypes.client._generate._createWrapper now throws an exception, if > disableCodeGeneration is True and it finally gets to the point where it > must generate some interfaces (they didn't exist already or they are out > of date). > > comtypes.client.GetBestInterface now tries returning a dynamic.Dispatch > object if at any time there is an error. e.g. the interface couldn't be > found or created etc. > > So with just these changes, if you have no already generated interfaces, > and you make a dynamic Dispatch object some how, now if any of its > methods or properties return IDispatch objects, rather than them being > smartly wrapped in real generated interfaces, dynamic Dispatch objects > are just returned instead. > > So the rule is: with disableCodeGeneration set to True, if the interface > already exists it will use it, but if it doesn't and it supports > IDispatch, it will use dynamic Dispatch, rather than trying to generate > proper interfaces. > > I also made some changes to comtypes.client.dynamic._Dispatch, so that > it works nicer with properties and methods. > > It now makes use of ITypeComp, using Bind to get IDs and invKind info > from a name. It also caches as much information as it can to make sure > its as fast as possible. > > Note that information is cached by its guid and name, on the actual > dynamic Dispatch class, so that multiple uses of the same IDispatch type > stay just as fast. > > So now, the only time an IDispatch property is treeted as a property > (i.e. doesn't have to be called like a function) is if its invkind is > propget and it has no params or optional params. Otherwise the method > caller is used instead. > > examples in MS Word are: > range.text is a property > range.collapse() is a method > examples in MSAA are: > pacc.accName() or pacc.accName(0) The property has an optional argument > so therefore it has to be treeted as a method. > pacc.accFocus is a property with no arguments at all, so it can be > treeted as a property. > > __call__ has also been added to _Dispatch, so that the IDispatch's > default method can be called, by calling the dynamic Dispatch object > like a function. > An example of this is an MS Word paragraphs object, calling > paragraphs(1) will now work. > > Also __enum now tries querying directly to IEnumVARIANT, if it fails to > call _enum. This now means that MSAA objects that have an IEnumVARIANT > as as one of their interfaces will now work pythonicly as a list/iterater. > > This patch makes dynamic._Dispatch rely on the fact that the IDispatch > object has ITypeInfo via GetTypeInfo(0). The question i have is is this > true for all IDispatch objects? or should we still support the more > basic GetIdsOfNames and then just treet all properties as methods, if > ITypeComp can't be found? > > A completely different approach to this could be where if a dynamic > Dispatch object calls invoke, and invoke returns an IDispatch, then we > should wrap it as a dynamic Dispatch object rather than getting its best > interface. However, I am not sure if this could currently be at all > possible in comtypes as we would need to be able to detect the fact that > invoke was called from a dynamic Dispatch object, as its result gets > wrapped... at a very low level. I think for now, allowing the user to > refuse code generation better suits the pattern of use. > > What are people's thoughts on this? > > Mick > > |
|
From: Tim G. <ma...@ti...> - 2008-07-01 19:57:01
|
Michael Curran wrote: > Hi all, > > I have attached a new version of my dynamic dispatch patch. > > This time, if an IDispatch interface does not support ITypeInfo, then > dynamic dispatch falls back to using the old GetIDsOfNames etc. > > This is so far mainly there for the Mozilla Gecko toolkit, as its > IDispatch with MSAA is hardly complete. Just so you don't think everyone's ignoring you... I'm certainly interested in the patches, altho' without any specific use-case in mind. Are you aware, tho', that Thomas Heller posted a few days back to say he was away and incommunicado for two or three weeks? He's obviously best placed to comment on things. TJG |
|
From: Michael C. <mic...@gm...> - 2008-07-05 07:41:31
Attachments:
dynamicDispatch.patch
|
Hi Tim, I wasn't aware Thomas was away, thanks for letting me know. He just suggested to me the other week that I should join and start a discussion on here about the patch. In regards to use cases, I guess the reason why I want this functionality is for the main project I am working on, which is the NVDA screen reader, at http://www.nvda-project.org/ In that project we use comtypes very heavily, for MSAA, SAPI, RichEdit controls, etc. But for our support for MS Word and Excel etc, we currently use pywin32. Reason being that with comtypes the COM interfaces are just way too large and take way too long to generate. Pywin32 does a great job with dynamic dispatch, though its just annoying having to use two Python COM implementations in the same project. So this is why I would like to improve comtypes dynamic dispatch support. I have also updated the patch, again, now fixing a few little issues with the non-typeinfo dynamic dispatch support, and more importantly, I have added some more tests to test_dyndispatch.py which tests many of my code changes using MSAA objects. Mick Tim Golden wrote: > Michael Curran wrote: >> Hi all, >> >> I have attached a new version of my dynamic dispatch patch. >> >> This time, if an IDispatch interface does not support ITypeInfo, then >> dynamic dispatch falls back to using the old GetIDsOfNames etc. >> >> This is so far mainly there for the Mozilla Gecko toolkit, as its >> IDispatch with MSAA is hardly complete. > > Just so you don't think everyone's ignoring you... I'm certainly > interested in the patches, altho' without any specific use-case > in mind. Are you aware, tho', that Thomas Heller posted a few > days back to say he was away and incommunicado for two or three > weeks? He's obviously best placed to comment on things. > > TJG > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > comtypes-users mailing list > com...@li... > https://lists.sourceforge.net/lists/listinfo/comtypes-users |
|
From: Thomas H. <th...@ct...> - 2008-07-17 19:57:57
|
Michael Curran schrieb: > Hi Tim, > > I wasn't aware Thomas was away, thanks for letting me know. He just > suggested to me the other week that I should join and start a discussion > on here about the patch. > > In regards to use cases, I guess the reason why I want this > functionality is for the main project I am working on, which is the NVDA > screen reader, at > http://www.nvda-project.org/ > > In that project we use comtypes very heavily, for MSAA, SAPI, RichEdit > controls, etc. But for our support for MS Word and Excel etc, we > currently use pywin32. Reason being that with comtypes the COM > interfaces are just way too large and take way too long to generate. Michael, I can understand that you feel like this, however let me play devils advocate. The code generation takes long, sure, but it is only done once. Also, you *could* generate when building the distribution and distribute them with your own code anyway. And what's the problem with the size? The NVDA zipfile is 11 MB, and the installer is 8.4 MB already, are a few MB more such a big deal? The problem with (parts of) your patch is that I fear there will be different behaviour in code using comtypes depending on what the 'disableCodeGeneration' current value is (and the *user* of our code is controlling the value of this variable), and additionally different behaviour depending on whether generated code is present or not even if the code generation is disabled. I can imagine several ways to fix that, possibly allowing the programmer to choose between dynamic dispatch and the custom part of dual interfaces: 1. It may be possible to explicitely specify that you want a dynamic object on COM object creation. Maybe introducing a comtypes.client.Dispatch() function in addition to comtypes.client.CreateObject(), similarly for the other creation function comtypes.client.GetActiveObject(). The returned *dynamic* dispatch objects/interfaces would probably have an attribute that prevented code generation for anything retrived from them. 2. Another approach would be to always use the automation interface if the object supports it. There may be other ways that escape me at the moment. > Pywin32 does a great job with dynamic dispatch, though its just annoying > having to use two Python COM implementations in the same project. > > So this is why I would like to improve comtypes dynamic dispatch support. > > I have also updated the patch, again, now fixing a few little issues > with the non-typeinfo dynamic dispatch support, and more importantly, I > have added some more tests to test_dyndispatch.py which tests many of my > code changes using MSAA objects. > Thomas |
|
From: Thomas H. <th...@ct...> - 2008-07-17 20:05:15
|
Michael Curran schrieb: > Hi Tim, > > I wasn't aware Thomas was away, thanks for letting me know. He just > suggested to me the other week that I should join and start a discussion > on here about the patch. I thought that other users (there aren't so many, yet) would have opinions on this. Thomas |
|
From: Thomas H. <th...@ct...> - 2008-08-07 19:26:40
|
Michael Curran schrieb: > > In regards to use cases, I guess the reason why I want this > functionality is for the main project I am working on, which is the NVDA > screen reader, at > http://www.nvda-project.org/ > > In that project we use comtypes very heavily, for MSAA, SAPI, RichEdit > controls, etc. But for our support for MS Word and Excel etc, we > currently use pywin32. Reason being that with comtypes the COM > interfaces are just way too large and take way too long to generate. > > Pywin32 does a great job with dynamic dispatch, though its just annoying > having to use two Python COM implementations in the same project. > > So this is why I would like to improve comtypes dynamic dispatch support. > > I have also updated the patch, again, now fixing a few little issues > with the non-typeinfo dynamic dispatch support, and more importantly, I > have added some more tests to test_dyndispatch.py which tests many of my > code changes using MSAA objects. Michael, thanks for the patch and sorry for the late reply. I have now made up my mind and have a plan how I want to integrate this into comtypes. 1. I will add new named parameter to the object creation functions in comtypes.client, CreateObject, GetActiveObject, and CoGetObject. This parameter will be named 'dynamic' and will default to False. If one of these functions is called with 'dynamic = True', then a dynamic dispatch object will be returned (an instance of comtypes.client.dynamic._Dispatch). No wrapper code will be generated, and even if wrapper code is already there it will not be used. 2. It must be made sure that such an object will not trigger the code generation when an attribute is accessed or a method is called. Since all attribute and method access goes through the 'IDispatch.Invoke' implementation in comtypes.automation, the last line in this method (which now is ' return result.value') this is the place where it can be implemented. 'result' is a VARIANT instance, and I will add a new property - maybe result.dyn_value - that will return a dynamic dispatch object. 3. There are some good ideas in your patch for comtypes.client.dynamic - thanks for that. But it seems there are still also a few problems with it which need to be fixed. I will release the current code ASAP as comtypes-0.5.1 before starting with these changes, hopefully it is not too late for you. -- Thanks, Thomas |
|
From: Thomas H. <th...@ct...> - 2008-08-20 19:35:40
|
Michael Curran schrieb: > Hi Thomas, > > I really appreciate you taking the time to look at my patch, many thanks. > > The idea about adding a dynamic keyword arg to createObject etc sounds > fine to me. I had talked about this idea with another developer on my > project, but we couldn't work out the best way to have invoke not > generate code. Your idea about having a dyn_value on a variant sounds good. I have started to implement this idea in a branch now: https://comtypes.svn.sourceforge.net/svnroot/comtypes/branches/dyndispatch > I'm sure this will all work with the way you have proposed, but I would > like to note, just so it is taken in to consideration, that when we use > comtypes in NVDA, we generally don't make use of CreateObject etc, but > rather we create a NULL interface pointer for the interface we are > interested in, and then using byref, we use particular win32 api > functions such as AccessibleObjectFromEvent in oleacc.dll to place the > real pointer value in our pointer. This is because much of the time we > retreave COM pointers from particular Windows etc. > > So all I guess I'm saying here is that one of our use patterns with > comtypes would be starting straight from an IDispatch which was > arbitrarily retreaved, not necessarily through createObject etc. So as > long as we can create a dynamic dispatch object with what ever means we > see fit, and what ever attributes we ask for no code is generated, than > this is fine. Well, it all depends on what you do with the IDispatch pointer you received. You can pass it to comtypes.client.GetBestInterface(pdisp), and comtypes will try to generate the code if it find a typelib, or you pass it to comtypes.client.dynamic._Dispatch(pdisp) which will then use dynamic dispatch. That is also what comtypes.client.CreateObject() does, depending on the 'dynamic=...' parameter. -- Thanks, Thomas |
|
From: Michael C. <mic...@gm...> - 2008-08-22 11:51:45
|
Hi Thomas, I have checked out the dyndispatch branch, and have given it a quick test run. It is working great so far :) at least comtypes.client.CreateObject(blablabla,dynamic=True) is giving back a dynamic dispatch, and any methods I call on that dynamic dispatch are giving back dynamic dispatches (when they support it). I understand work still needs to be completed for how dynamic dispatch properties and methods are handled, but so far this all looks really good from my end. Thanks again for looking in to this for me Mick Thomas Heller wrote: > Michael Curran schrieb: > >> Hi Thomas, >> >> I really appreciate you taking the time to look at my patch, many thanks. >> >> The idea about adding a dynamic keyword arg to createObject etc sounds >> fine to me. I had talked about this idea with another developer on my >> project, but we couldn't work out the best way to have invoke not >> generate code. Your idea about having a dyn_value on a variant sounds good. >> > > I have started to implement this idea in a branch now: > > https://comtypes.svn.sourceforge.net/svnroot/comtypes/branches/dyndispatch > > >> I'm sure this will all work with the way you have proposed, but I would >> like to note, just so it is taken in to consideration, that when we use >> comtypes in NVDA, we generally don't make use of CreateObject etc, but >> rather we create a NULL interface pointer for the interface we are >> interested in, and then using byref, we use particular win32 api >> functions such as AccessibleObjectFromEvent in oleacc.dll to place the >> real pointer value in our pointer. This is because much of the time we >> retreave COM pointers from particular Windows etc. >> >> So all I guess I'm saying here is that one of our use patterns with >> comtypes would be starting straight from an IDispatch which was >> arbitrarily retreaved, not necessarily through createObject etc. So as >> long as we can create a dynamic dispatch object with what ever means we >> see fit, and what ever attributes we ask for no code is generated, than >> this is fine. >> > > Well, it all depends on what you do with the IDispatch pointer you received. > You can pass it to comtypes.client.GetBestInterface(pdisp), and comtypes > will try to generate the code if it find a typelib, or you pass it to > comtypes.client.dynamic._Dispatch(pdisp) which will then use dynamic > dispatch. That is also what comtypes.client.CreateObject() does, depending > on the 'dynamic=...' parameter. > > |
|
From: Michael C. <mic...@gm...> - 2008-08-08 00:19:14
|
Hi Thomas, I really appreciate you taking the time to look at my patch, many thanks. The idea about adding a dynamic keyword arg to createObject etc sounds fine to me. I had talked about this idea with another developer on my project, but we couldn't work out the best way to have invoke not generate code. Your idea about having a dyn_value on a variant sounds good. I'm sure this will all work with the way you have proposed, but I would like to note, just so it is taken in to consideration, that when we use comtypes in NVDA, we generally don't make use of CreateObject etc, but rather we create a NULL interface pointer for the interface we are interested in, and then using byref, we use particular win32 api functions such as AccessibleObjectFromEvent in oleacc.dll to place the real pointer value in our pointer. This is because much of the time we retreave COM pointers from particular Windows etc. So all I guess I'm saying here is that one of our use patterns with comtypes would be starting straight from an IDispatch which was arbitrarily retreaved, not necessarily through createObject etc. So as long as we can create a dynamic dispatch object with what ever means we see fit, and what ever attributes we ask for no code is generated, than this is fine. However, your solution fits this model ok, I just wanted to point that out for future reference that is all. Thanks again Mick Thomas Heller wrote: > Michael Curran schrieb: >> In regards to use cases, I guess the reason why I want this >> functionality is for the main project I am working on, which is the NVDA >> screen reader, at >> http://www.nvda-project.org/ >> >> In that project we use comtypes very heavily, for MSAA, SAPI, RichEdit >> controls, etc. But for our support for MS Word and Excel etc, we >> currently use pywin32. Reason being that with comtypes the COM >> interfaces are just way too large and take way too long to generate. >> >> Pywin32 does a great job with dynamic dispatch, though its just annoying >> having to use two Python COM implementations in the same project. >> >> So this is why I would like to improve comtypes dynamic dispatch support. >> >> I have also updated the patch, again, now fixing a few little issues >> with the non-typeinfo dynamic dispatch support, and more importantly, I >> have added some more tests to test_dyndispatch.py which tests many of my >> code changes using MSAA objects. > > Michael, > > thanks for the patch and sorry for the late reply. I have now made up my mind > and have a plan how I want to integrate this into comtypes. > > 1. I will add new named parameter to the object creation functions in comtypes.client, > CreateObject, GetActiveObject, and CoGetObject. This parameter will be named > 'dynamic' and will default to False. If one of these functions is called with > 'dynamic = True', then a dynamic dispatch object will be returned (an instance > of comtypes.client.dynamic._Dispatch). No wrapper code will be generated, and > even if wrapper code is already there it will not be used. > > 2. It must be made sure that such an object will not trigger the code generation > when an attribute is accessed or a method is called. Since all attribute and > method access goes through the 'IDispatch.Invoke' implementation in comtypes.automation, > the last line in this method (which now is ' return result.value') this is the place > where it can be implemented. 'result' is a VARIANT instance, and I will add a new > property - maybe result.dyn_value - that will return a dynamic dispatch object. > > 3. There are some good ideas in your patch for comtypes.client.dynamic - thanks for that. > But it seems there are still also a few problems with it which need to be fixed. > > I will release the current code ASAP as comtypes-0.5.1 before starting with these changes, > hopefully it is not too late for you. > |