From: Mike P. <mp...@pi...> - 2009-08-31 14:47:56
Attachments:
ajax.zip
|
Hi David, I also have made an attempt to "ajaxify" the web gui of misterhouse. I didn't know "Comet" but now I know that I have implemented the "XMLHttpRequest long polling" way of Comet. Please find attached a patch against a misterhouse-2.105 installation. After that the normal ia5 web gui with mh.ini gd=1 is changed that requests to switch items on/off are sent via XMLHttpRequest, therefore there is no page reloading. The answer of this request is ignored. Via "Comet" the states of the displayed items are tracked, i.e. if you change item states via wall switch this change is displayed "immediately" (you will have data transfer and processing times). Please have a look at it and tell me what you think. On the other hand if you have something in work for "Comet" I am interested to see how you implemented this. My implementation works as follows: When the web page is generated also a vector is created. This vector contains the item names and states which are shown in the web page. After loading the page the OnLoad-event triggers the refreshing of the states. For that the vector is sent back to misterhouse. In misterhouse an object is created which contains all the items and the states. This object is tied to the items (via tie_event). This object also holds the socket for answering the request. As soon as one of the events is changed the state of the waiting object goes to changed. In a Post-Mainloop-hook the states of all waiting objects is checked. If a waiting object is in state "changed" some xml is generated which contains all the changes and is sent to the web gui. There the url of the graphics is updated to contain the new state and the vector is updated. Then the process starts again. There is also an ongoing work to create a new gui for misterhouse (in german only, latest version: http://knx-user-forum.de/61299-post82.html). This gui is using the same service. So I think the service is universally usable. Regards Mike |
From: David N. <dno...@ya...> - 2009-09-02 04:34:14
|
I applied your patch and got it working after I installed GD. It doesn't work in IE, probably due to the different way IE does xml calls. Let me look at your code more. I want to answer a few questions before we inflict this code on others. Will it break the web interface on any other browsers, assuming we fix IE? I'm not crazy about GD mode, can we make it work without it? Can we make the long polling feature more generalized so it works on all data? Possibly integrated with the xml server code? Are you forking the long poll server process, or using threads? Will either work on windows? David ----- Original Message ----- From: "Mike Pieper" <mp...@pi...> To: <mis...@li...> Sent: Monday, August 31, 2009 7:47 AM Subject: Re: [mh] AJAX examples committed to svn > Hi David, > > I also have made an attempt to "ajaxify" the web gui of misterhouse. I > didn't know "Comet" but now I know that I have implemented the > "XMLHttpRequest long polling" way of Comet. Please find attached a patch > against a misterhouse-2.105 installation. After that the normal ia5 web > gui > with mh.ini gd=1 is changed that requests to switch items on/off are sent > via XMLHttpRequest, therefore there is no page reloading. The answer of > this request is ignored. Via "Comet" the states of the displayed items are > tracked, i.e. if you change item states via wall switch this change is > displayed "immediately" (you will have data transfer and processing > times). > > Please have a look at it and tell me what you think. > On the other hand if you have something in work for "Comet" I am > interested > to see how you implemented this. > > My implementation works as follows: > When the web page is generated also a vector is created. This vector > contains the item names and states which are shown in the web page. After > loading the page the OnLoad-event triggers the refreshing of the states. > For that the vector is sent back to misterhouse. In misterhouse an object > is created which contains all the items and the states. This object is > tied > to the items (via tie_event). This object also holds the socket for > answering the request. As soon as one of the events is changed the state > of > the waiting object goes to changed. In a Post-Mainloop-hook the states of > all waiting objects is checked. If a waiting object is in state "changed" > some xml is generated which contains all the changes and is sent to the > web > gui. There the url of the graphics is updated to contain the new state and > the vector is updated. Then the process starts again. > > There is also an ongoing work to create a new gui for misterhouse (in > german only, latest version: http://knx-user-forum.de/61299-post82.html). > This gui is using the same service. So I think the service is universally > usable. > > Regards > Mike -------------------------------------------------------------------------------- > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 > 30-Day > trial. Simplify your report design, integration and deployment - and focus > on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july -------------------------------------------------------------------------------- > ________________________________________________________ > To unsubscribe from this list, go to: > http://sourceforge.net/mail/?group_id=1365 > > |
From: Mike P. <mp...@pi...> - 2009-09-02 07:52:11
|
Hi David, yes sorry I forgot to mention that I tested only with firefox. There should be nothing special, so it should be possible to work with it on IE and firefox. > I'm not crazy about GD mode, can we make it work without it? The reason why I used GD mode is because the graphics are generated by misterhouse with respect to the current state. When the state is changed I only need to create a new url with the new state in it. The rest is handled on misterhouse. So generally: the web gui should only be interested in the changed state of the item. The web gui of the german guy what I mentioned in my former post is written in javascript. All "graphics" is handled on the client side. The client needs only the state. Without GD or javascript, the web gui must know which icon to load for which state. This is not so easy to change. An alternative would be that the client tells the long polling service which data it's interested in. > Can we make the long polling feature more generalized so it works on all data? I tried to avoid real polling. With tie_events there is no polling on item level. The only object what is polled is the container object. Where only one state is checked. So if you have 20 items on the web gui there is only one object what is polled. Any change on any of the 20 items signals its change event driven. This can't work on data like the weather hash, because there is no tie_event feature. It would be better to encapsulate the weather hash by an object (or item). Ore more generally: all what should be long polled must be derived from Generic_Item. Having global data without accessor methods is bad coding style anyway (in my opinion). Do you know a way to easily track the state of data? > Possibly integrated with the xml server code? Are you forking the long poll server process, or using threads? I've used http_server because there I can "end" the server call with leaving the socket open, i.e. with leaving the call open. I tried to integrate the service in the SOAP interface, but there I didn't found a comparable feature. The same at xml_server. There is no extra process or thread for the long polling feature. It's simply that the socket is left open. The socket is stored in the container object (the waiting object). The container object is polled via PostMainloop-hook, i.e. it's part of the mainloop. But checking the container object is cheap, because only the state (changed or not) is evaluated. If the container object is in state "changed" it generates the xml, writes it into the socket and closes the socket. To not leak container objects, there is a timeout (currently hard coded 60 secs) where the container object is also seen as changed. Then an "empty" xml answer is sent and the web gui has to call again ... Because it uses no process and no thread it should run on windows too. Mike |
From: David N. <dno...@ya...> - 2009-09-08 08:27:15
|
I just committed this update to svn: I have modified Mike Pieper's server side implementation of the ajax (or comet) long polling technique to work with the existing xml subroutine in Misterhouse (actually, it will work with any subroutine that is written as discussed below). I also modified the groups and weather output of the xml subroutine to work with the new web service. When you use a url like this http://localhost:8080/long_poll?subroutine(args), the web server will add a call to that subroutine to the main loop. That subroutine should return an empty string each time it's run until the data its monitoring has changed. At that point, whatever it returns is passed back to the client. This is normally, but not necessarily, XML. The following url is what i used for testing: http://localhost:8080/long_poll?xml('groups=$All_Lights','updates_only,fields=state') If you browse to this url, your browser should wait up to a minute for one of the items in the All_Lights group to change state. Once that happens, the xml describing the new state will be returned. I haven't written the client side code yet to work with the new service, and the server code needs a lot more work. Please look it over and comment. David ----- Original Message ----- From: "Mike Pieper" <mp...@pi...> To: "The main list for the MisterHouse home automation program" <mis...@li...> Sent: Wednesday, September 02, 2009 12:51 AM Subject: Re: [mh] AJAX examples committed to svn > Hi David, > > yes sorry I forgot to mention that I tested only with firefox. There > should > be nothing special, so it should be possible to work with it on IE and > firefox. > >> I'm not crazy about GD mode, can we make it work without it? > The reason why I used GD mode is because the graphics are generated by > misterhouse with respect to the current state. When the state is changed I > only need to create a new url with the new state in it. The rest is > handled > on misterhouse. > So generally: the web gui should only be interested in the changed state > of > the item. The web gui of the german guy what I mentioned in my former post > is written in javascript. All "graphics" is handled on the client side. > The > client needs only the state. > Without GD or javascript, the web gui must know which icon to load for > which state. This is not so easy to change. > An alternative would be that the client tells the long polling service > which data it's interested in. > >> Can we make the long polling feature more generalized so it works on all > data? > I tried to avoid real polling. With tie_events there is no polling on item > level. The only object what is polled is the container object. Where only > one state is checked. So if you have 20 items on the web gui there is only > one object what is polled. Any change on any of the 20 items signals its > change event driven. > This can't work on data like the weather hash, because there is no > tie_event feature. It would be better to encapsulate the weather hash by > an > object (or item). Ore more generally: all what should be long polled must > be derived from Generic_Item. Having global data without accessor methods > is bad coding style anyway (in my opinion). > Do you know a way to easily track the state of data? > >> Possibly integrated with the xml server code? Are you forking the long > poll server process, or using threads? > I've used http_server because there I can "end" the server call with > leaving the socket open, i.e. with leaving the call open. I tried to > integrate the service in the SOAP interface, but there I didn't found a > comparable feature. The same at xml_server. > There is no extra process or thread for the long polling feature. It's > simply that the socket is left open. The socket is stored in the container > object (the waiting object). The container object is polled via > PostMainloop-hook, i.e. it's part of the mainloop. But checking the > container object is cheap, because only the state (changed or not) is > evaluated. If the container object is in state "changed" it generates the > xml, writes it into the socket and closes the socket. > To not leak container objects, there is a timeout (currently hard coded 60 > secs) where the container object is also seen as changed. Then an "empty" > xml answer is sent and the web gui has to call again ... > Because it uses no process and no thread it should run on windows too. > > Mike > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 > 30-Day > trial. Simplify your report design, integration and deployment - and focus > on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > ________________________________________________________ > To unsubscribe from this list, go to: > http://sourceforge.net/mail/?group_id=1365 > |
From: Mike P. <mp...@pi...> - 2009-10-02 09:25:13
|
On Tue, 8 Sep 2009 01:26:59 -0700, "David Norwood" <dno...@ya...> wrote: > I just committed this update to svn: Ooops ... In my opinion it was not yet ready for checking in. > I haven't written the client side code yet to work with the new service, > and > the server code needs a lot more work. Please look it over and comment. My code was already used for an experimental javascript web gui. It had already some features which didn't make it into the svn. I see two main problems: - cpu usage because of heavy polling - race conditions, which may bring the web gui out of sync with mh cpu usage My idle mh uses 5% cpu. When I start the example (... $AllLights ...) the cpu usage goes up to 10%. That's not good. What happens if there are 10 clients? In the initial code, there was only very less code running in the main loop. The waiting object or monitor was only checked for a boolean state. The change of the monitor state happened event driven. Of course we all know that polling is bad. So we should try to use events wherever we can. If we can't easily monitor changes (for instance to global data), we could at least try to "eventize" this. For instance we could add some code to the main loop which polls the weather hash for changes (but slowly). If we see a change we "send an event", which could be used by the long polling stuff (or other interested items). Then we would have only one polling loop, even if we have 10 waiting clients. race conditions The current implementation has the problem that the long polling call waits for any change in misterhouse. If the web gui shows LampA and LampB as "Off" and calls for updates. Then it may happen, that LampA is switched "On". In this moment, this change is sent to the web gui. But in the next main loop LampB is also switched "On". This happens while the web gui is still processing the change of LampA. After changing the state of LampA in the web gui to "On", the long polling call is started again. Because LampB is already "On" nothing happens now. The LampB is "On" but the web gui shows "Off". The web gui is out of sync with mh. This is only fixed if the state of LampB changes again. In the initial code, the web gui sends the presented state of all items to mh. Therefore after the change of LampA is processed, it makes a new long polling call and give it the information that LampB is "Off". Because the initial code is event driven and the LampB event is long gone, it would have the same problem like the current implementation. But while creating the monitor object the current state of the object is checked against the state from the web gui. If there is a difference the monitor is set immediately to "changed" and it will send the new state. Because mh is single threaded it may not happen that the state is changed in the meantime. It's race condition free. If an item has many states, like the weather hash, we could use a "state version" number, i.e. every time we see a change of the weather hash we increment a version number in a monitor object. The web gui than only have to give the currently presented version number and if the number in mh is different we know that a change happened without having to compare every value. I'll try to make some changes to the current implementation to make it less cpu using, race condition free and not limited to "items". It will be independent of the external interface. That means it will be usable by SOAP, xml_server and whatever else. But currently I don't have the time to work on this. Mike |
From: David N. <dno...@ya...> - 2009-10-04 05:25:32
|
Mike Pieper wrote: > On Tue, 8 Sep 2009 01:26:59 -0700, "David Norwood" <dno...@ya...> > wrote: >> I just committed this update to svn: > Ooops ... In my opinion it was not yet ready for checking in. Release early and often is my motto. :) >> I haven't written the client side code yet to work with the new service, >> and >> the server code needs a lot more work. Please look it over and comment. > My code was already used for an experimental javascript web gui. It > had already some features which didn't make it into the svn. Noted. > I see two main problems: > - cpu usage because of heavy polling > - race conditions, which may bring the web gui out of sync with > mh [snip] First, thank you for your input. I really appreciate it. I have read your comments and plan to reimplement the long_poll feature more closely to your code. I found this piece of perl magic (as you put it) that will allow us to monitor variables without adding to the main loop: http://search.cpan.org/dist/Tie-Watch/Watch.pm I first need to create a Variable_Item.pm similar to Weather_Item.pm, but using the above module. Then modify your code to create a watcher for any global variables passed in. I like your idea about a revision number for hashes like %Weather. Do you know what the limit is for url length? David |
From: Mike P. <mp...@pi...> - 2009-10-04 11:51:25
Attachments:
namesfix.patch
|
On Sat, 3 Oct 2009 22:25:17 -0700, "David Norwood" <dno...@ya...> wrote: > Release early and often is my motto. :) That's a good motto. Please find attached a patch which changes only some names of objects. The names I used before were bad english (if it was english at all) :). > I found this piece of perl magic (as you put it) that will allow us > to monitor variables without adding to the main loop: > > http://search.cpan.org/dist/Tie-Watch/Watch.pm That's cool. > Do you know what the limit is for url length? http://www.boutell.com/newfaq/misc/urllength.html But you have right, it's not a good idea to sent all via url. I think the interface I've used is not nice. What is the preferred interface for javascript programs? Do you know the SOAP interface? code/examples/SOAP_examples/ Mike |