Menu
▾
▴
[ naviserver-Feature Requests-1156141 ] Add ns_conn channel command
|
From: SourceForge.net <no...@so...> - 2005-03-17 18:26:23
|
Feature Requests item #1156141, was opened at 2005-03-03 21:02 Message generated for change (Comment added) made by vasiljevic You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-17 19:26 Message: Logged In: YES user_id=95086 Added to CVS head ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-17 15:20 Message: Logged In: YES user_id=184124 i do not ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-17 13:10 Message: Logged In: YES user_id=95086 OK, If you do not mind, I'll add ns_conn channel It will return the Tcl channel you can use for talking to the client. When you've finished, you just "close $sock" and finish the proc. The connection thread is then ready for other users/requests. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-17 12:41 Message: Logged In: YES user_id=87254 Yes, I get it :-) My only point was that HTTP is not a short lived connection protocol, if you don't want it to be. Lengthy connections with multiple messages seems to be one of the key features here. The *only* thing HTTP can't do is have the server initiate a message to the client. For the kinds of cases you're talking about here with multi-hour long connections, I'd be wanting some kind of 'ping' request from the client every so often just to make sure the server was idle and not broken. The ping becomes a 'what messages do you have for me?' request, the server's response is itself a request to the client. Anyway, I was talking pure nonsense about the keepalive stuff; the client doen't have to be so broken after all. I'm fine with you adding this feature. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-05 10:03 Message: Logged In: YES user_id=95086 "still don't understand what it buys you that a modern HTTP 1.1 implemnetation doesn't. If there was no other way to do Stephen, I do not think you understood my point. What this command buys me is to be able to initiate an http connection and reuse the socket-to-socket comm channel for other purposes. For example we make "GET" request to a certain URL and then when the connection gets established, the server, takes the conn socket, wraps a Tcl channel around it and passes this channel to my (arbitraty) Tcl proc. The client, instead of getting the response body, parses only the returned http headers to see that the connection succeeded and then passes this socket to my custom Tcl proc. Now, on the sever and on the client side you have a channel and both sides can use it to establish whatever protocol they want. They have a socket pipe between. The http is just used to handshake the peers, clear? What we do with this? We have two server instances on two different hosts. We open a channel (using the above trick) from one to another and then use this to generate a stream of preformatted packets (our protocol) on one side which does the backup of data from to the other side ultimately to the tape drive(s) attached to that host. We shuffle TB of data this way daily in about 500 installations worldwide, so I know this works fine. Now, see, this has nothing to do with HTTP1.1. or 1.0 alone. And yes, if you look in the implementation, you will see that it is a simple wrapper to the Ns_ConnSock. It simply wraps a Tcl channel around the the connection socket, nothing more and nothing less. And, I would not say that it requires a broken client. It just requires a special client-processing. This is not ment for browser-clients. It has nothing to do with serving pages or any other short lived connections. Once we setup such a connection, we use it for hours and maybe days. We could have achieved this with a simple socket connection from client to a specific port on the server. But we are not able to use any arbitrary port. All we have is port 80 (the web server port). So we must play by the http rules initially when accessing the server. When we passed this http handshake stuff, we are left with a socket-to-socket comm-channel which is transparent and can be used for any purpose whatsoever. Now, is this little bit more clear now? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-05 04:32 Message: Logged In: YES user_id=184124 What is broken about it? It is the Tcl interface to Ns_ConnSock(), just Tcl way, using channel. The merit is to use sync inter-exchange between client and server using established connection. Client is not broken, it is designed for this exchange, it is the tool to build messaging, not general purpose protocol. It make naviserver more flexible platform for implementing different ppas on top of it using Tcl, that's all. Btw, when are you going to commit your protocol extensions? ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-05 01:20 Message: Logged In: YES user_id=87254 This command isn't a good idea because it creates a broken interface which requires a broken client to work, and I still don't understand what it buys you that a modern HTTP 1.1 implemnetation doesn't. If there was no other way to do it, then it may still have some merit. But Ns_ConnSock() is trivial and gives you all you need to create a Tcl channel. We've already decided to do this the right way, whatever that turns out to be, so why should we simultaneously also do it the wrong way? In fact you seem to agree, noting that this sucks because it's not UDP and the overhead of HTTP is just too much for you. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 16:36 Message: Logged In: YES user_id=184124 And that's why i want to add simple UDP support to the server: for implementing simple messaging between servers in the cluster, using UDP without HTTP overhead will make it simple and faster. Using Tcl channel is intermediate solution for such messaging because it eliminates making multiple HTTP requests to the server but still uses heavy TCP/HTTP overhead. Still i think this command is very usefull and will make server more versatile provided that we will supply usage examples with the server as well. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 16:26 Message: Logged In: YES user_id=95086 "Second, this has nothing to do with non-HTTP protocol, it is HTTP communication protocol, what's inside the body has nothing to do with it again." This is true. It is a regular http request, just for the body part, the peers are engaging in a bidirectional communication and can do whatever they pleased. The request is considered done when one of the peers close the socket. Strictly speaking, I'm using the http body and this trick to implement arbitrary client-server comm channel. This will not work for browsers of course. It has nothing to do with browsers at all in fact. It arose out of the necessity to support alternate protocols given what we had in AS, that is pure http machinery. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 15:38 Message: Logged In: YES user_id=184124 On the closing note, keepalive has nothing to do with this and should not be considered in this case. Even if you connect using HTTP 1.1 and specify keepalive, sever has the right to issue Connection: close header which will disable keepalive. So, it WILL work for all connections. Second, this has nothing to do with non-HTTP protocol, it is HTTP communication protocol, what's inside the body has nothing to do with it again. And pipelining is a bad example here, the point it to send the request and wait for the answer and base on that send another request. Pipelining will allow you just to submit all requests and then later receive replies, it was designed for speeding browser's connection especially retrieving images. It looks to me Stephen you are all for HTTP core and around-the-HTTP-driver solutions only. This will make simple solutions complex because of getting around/workaround HTTP based core in the server. What is wrong with one simpe function which will give the server more options without breaking anything? ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 10:45 Message: Logged In: YES user_id=95086 I see. The gotcha's are: o. it's yet another implementation of non-HTTP protocol o. (HTTP 1.0 only, and no Keepalive header) be used to get o. function in the docs might reasonably expect it to work for all HTTP connections That's pretty clear. Well, I think you are right. This is our attempt to work around http-only caps of the server. Agreed. It will be better to get it done properly and we will then internally switch to the new implementation when it's ready. At the moment I have all this in a separate module and this can stay so until the multi-protocol caps are agreed upon and implemented. I will close this RFE then. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 10:36 Message: Logged In: YES user_id=87254 I don't understand why you're not using standard HTTP pipelining. It would seem to be so much easier than having to write custom client and server code. This is a small change, but I'm not excited about it because it's yet another implementation of non-HTTP protocol support, which makes 3 so far... :-( It also seems to require that a deliberately broken client (HTTP 1.0 only, and no Keepalive header) be used to get around the server's io read-ahead. Someone seeing this function in the docs might reasonably expect it to work for all HTTP connections. Also, I don't think this requires core support. Ns_ConnSock() will return the underlying file descriptor for a connection. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 09:36 Message: Logged In: YES user_id=95086 OK. Maybe I should explain little bit more what I'm using this for. Imagine you'd like to make a client-server connection and then exchange arbitrary data on this connection. Imagine something like telnet connection. You telnet from client to server, get a comm-pipe betwen them and then you can shuffle whatever you like across this comm-pipe. What we have is a http server responding only to http requests (as it is now, w/o multiprotocol support). So, what I do is to use http to setup the connection, have a registered procedure on the server grab the socket and install my custom protocol handler on it. The http is only used to establish the communication. All the rest is handled by my custom code on the server and on the client. Once the connection is setup, we have a point-to-point socket communication link between server and client and we can use it for just about anything. All you need for this is a client-side http-library (we have ours written in Tcl) which will give you the socket back once the connection is established) and serve-side proc implementing your protocol of choice. So, the entire server http-machinery stays in place. I do not see any problem there because we have been using this trick for about 4 years in production very successfully. What I do not know is wether there are any gotcha's with that approach (we haven't hit any yet). I also do not know what will happen when we add alternate multiprotocol caps to server. Maybe this will become obsolete. But, as I see this now, the change is trivial and does not hurt anybody. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 00:21 Message: Logged In: YES user_id=87254 HTTP also has (although our server doesn't, but I want to add this) pipelining, which is sending multiple requests at once without first waiting for the reply to previous requests. That sounds like your "i do it when I send multiple http requests". ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
