Menu
▾
▴
naviserver-tracker — SourceForge Tracker Notifications
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
(61) |
Mar
(153) |
Apr
(39) |
May
(10) |
Jun
(15) |
Jul
(15) |
Aug
(2) |
Sep
|
Oct
(17) |
Nov
(2) |
Dec
(13) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
(18) |
Feb
(9) |
Mar
|
Apr
|
May
|
Jun
|
Jul
(7) |
Aug
(1) |
Sep
(2) |
Oct
|
Nov
(1) |
Dec
|
| 2007 |
Jan
(8) |
Feb
(3) |
Mar
|
Apr
|
May
(2) |
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2008 |
Jan
|
Feb
|
Mar
|
Apr
(6) |
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: SourceForge.net <no...@so...> - 2005-03-04 23:46:59
|
Feature Requests item #1156875, was opened at 2005-03-04 12:03 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add server watchdog process Initial Comment: We have been using this for quite some time and it proved extremely useful. We doublefork the nsd process and make the first forked instance control the second. The first one (the watchdog) reacts on exit codes and signals caught during the watch and correspondingly restarts the second instance (the worker). Also, we have added the the "-restart" option to the "ns_shutdown" command. This just sends the SIGINT to the worker process. The watchdog is handling this signal and respawns the worker automatically. During operation, the watchdog logs events and their cause into the system log file. This looks like: Feb 28 04:00:05 Develop nsd[19400]: worker: started. Mar 1 04:00:13 Develop nsd[4475]: watchdog: worker 19400 exited (2). Mar 1 04:00:15 Develop nsd[21290]: worker: started. Mar 1 04:00:18 Develop nsd[14705]: watchdog: worker 19399 exited (2). Mar 1 04:00:20 Develop nsd[21300]: worker: started. We have done all the changes with "--enable-watchdog" so anybody who needs this feature will have to compile with this option. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 16:46 Message: Logged In: YES user_id=87254 I don't think this has to hide behind a config option. It's either a good idea or it's not. Sounds good to me. Is there a patch? I'm wondering about some of the implementation details... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 23:30:24
|
Feature Requests item #1156899, was opened at 2005-03-04 12:30 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156899&group_id=130646 Category: Tcl-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add optional "numbytes" arg to ns_conncptofp Initial Comment: The current ns_conncptofp will copy the content to the file. The entire content. What is missing is to tell to the command to copy only part of the content. The proposed API would be: ns_conncptofp channel ?numbytes? The numbytes can be: ommited (procedure does what it did before) >0 (will copy somany bytes up to contentLength or end-of-data whatever comes first) 0 (will do nothing) -1 (will behave as when argument is ommited) This command is one of those which support the old-compat connId argument. So the actual API is as of today: ns_conncptofp ?connId? channel I would vote to scrap the connId parsing. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 16:30 Message: Logged In: YES user_id=87254 This seems reasonable. Should it look like this though? ns_conncptofp ?-bytes tocopy? channel The reason it is attractive to drop the connid arg is that with two optional args it becomes a pain to parse. I'm also thinking of how ns_sendmail has been extended over time by appending optional args. The C version of this command returns an error if you ask for more bytes than are available. Should the Tcl version silently truncate? Maybe it should return the number of bytes actually copied? Does it make sense to ask for 0 (zero) bytes? Maybe bytes <= 0 should revert to default behaviour which is to copy all that's available? The existing error message "could not copy content (likely client disconnect)" is wrong. This might have been the case in 3.x, but not 4.x. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156899&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 19:30:43
|
Feature Requests item #1156899, was opened at 2005-03-04 20:30 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156899&group_id=130646 Category: Tcl-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add optional "numbytes" arg to ns_conncptofp Initial Comment: The current ns_conncptofp will copy the content to the file. The entire content. What is missing is to tell to the command to copy only part of the content. The proposed API would be: ns_conncptofp channel ?numbytes? The numbytes can be: ommited (procedure does what it did before) >0 (will copy somany bytes up to contentLength or end-of-data whatever comes first) 0 (will do nothing) -1 (will behave as when argument is ommited) This command is one of those which support the old-compat connId argument. So the actual API is as of today: ns_conncptofp ?connId? channel I would vote to scrap the connId parsing. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156899&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 19:03:56
|
Feature Requests item #1156875, was opened at 2005-03-04 20:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add server watchdog process Initial Comment: We have been using this for quite some time and it proved extremely useful. We doublefork the nsd process and make the first forked instance control the second. The first one (the watchdog) reacts on exit codes and signals caught during the watch and correspondingly restarts the second instance (the worker). Also, we have added the the "-restart" option to the "ns_shutdown" command. This just sends the SIGINT to the worker process. The watchdog is handling this signal and respawns the worker automatically. During operation, the watchdog logs events and their cause into the system log file. This looks like: Feb 28 04:00:05 Develop nsd[19400]: worker: started. Mar 1 04:00:13 Develop nsd[4475]: watchdog: worker 19400 exited (2). Mar 1 04:00:15 Develop nsd[21290]: worker: started. Mar 1 04:00:18 Develop nsd[14705]: watchdog: worker 19399 exited (2). Mar 1 04:00:20 Develop nsd[21300]: worker: started. We have done all the changes with "--enable-watchdog" so anybody who needs this feature will have to compile with this option. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 16:44:03
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 16:44 Message: Logged In: YES user_id=184124 And again, i have nothing against it, i said it before, what i am offering is a littl ebit different, for network based protocols driver stuff is kay, i just want to be able to submit conn threads from within the server, not through the main driver. If i have already request data, through shared memory, file or other way, i want to submit wroker thread directly, not by connecting to myself and submitting data using HTTP or other protocol i will have to implement just to queue te request. This is for other applications, not HTTP/Web based, more like messaging/authentication servers and alike. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 14:41 Message: Logged In: YES user_id=184124 Can you provide what you have to modify in driver/nssock, i think that will not be easy and will require too many modifications. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 10:08 Message: Logged In: YES user_id=87254 What do you think of my suggestion that the nssock module and nsd/driver.c could be easily modified to handle udp and unix domain sockets when specified in the config file? A change like that combined with the work you've already done to the binder would mean no development is required to handle all socket types, and would be transparent to the upper layers. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-26 17:10 Message: Logged In: YES user_id=184124 Stephan, I am suggesting a compromise, let's put ParseProc and DriverSockQueu patches, they can coexists easily but will offer two way to add new drivers. Once we have new drivers working we will see what can be modified/adjusted, for now we do not have anything except my several drivers i use in the production and i use my own loop for smtp driver to keep processing and state machine in one place. Again, i have nothing against your method, i used to do similar in my previous versions but now i need to be completely free to implement aditional driver and i think we should have this ability. Zoran, what do you think? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 22:57 Message: Logged In: YES user_id=184124 Try to implement HTTP over UDP as i provided in the patch, you will need to modify core driver again, this is one example. Next, if i want to implement small driver that do not interact with web part of aolserver, no filters, it just need to do one small thing, it should be fast but will handle very many requests. I do not need filters, callbacks, i need my own main loop. I do not understand why you are insisting that all drivers should go through http driver's main loop. If i implement dns server and web interface to it, why they should compete in the same driver's main loop and go through registered filters if they share only backend database structure? ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 22:22 Message: Logged In: YES user_id=87254 You don't have to use registered filters or procs. It's an option for protocol implementations where the initial accept and parsing take place in the driver thread, and processing takes place in a conn thread. Once you submit to the queue however, either a filter, a registered proc, or a registered proxy proc must handle the request. It makes no difference whether you submit explicitly with Ns_DriverSockQueue or implicitly after your Ns_ParseProc completes. How much of the request structure you fill in is up to you, it's all optional. If you choose not to use it there is no overhead. It doesn't have to be a text based protocol for this to make sense. You might fill in the request structure if you wanted to enable people to handle different request types via C or Tcl. Or, you could store binary data in Ns_Cls storage and access it from registered procs etc. In the case where you want to handle connection accept/parse/reponse processing in the same thread, nothing needs to be added. You can do that today with Ns_SockCallback (or by placing all your code in an Ns_ParseProc and returning the response from there) Can you be more specific about the overhead of using an Ns_ParseProc? Exactly what memory is allocated, what code is run that should not..? What specifically can you not do with the Ns_ParseProc interface that you need the Ns_Driver* routines for? I'm looking for concrete examples. As far as I can tell, with Ns_ParseProc you write less code, get more options, and it takes advantage of infrastructure to give you more speed. I must be missing something, but you'll have to explain it to me in more detail because I don't get it. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 20:43 Message: Logged In: YES user_id=184124 Okay, the biggest problem with your way i see that i am enforced to go through http driver, even with my parse proc, i need to know how it works to implement my driver so correct hooks/filters will be called. If i want skip some parts i am not able to do it, http driver works only one way. Another issue is if something will not fit into current driver, additional hook needs to be introduced and core driver needs to be modified again. If my driver fits http-like/text mode paradigm, reusing http driver is the easiest way, but if i need something specific, all extra step to mimic the connection as http request, so all other parts of the http driver will not fail is just unnecesssary extra efforts. And i you mentioned, using Ns_DriverSockQueue is low level function and requires from the developer full attention for bulding the driver, but if this is what i want, why not. I can spawn my own thread if some callbacks are long running Tcl scripts and queue connection from my own loop. If i omly want to received packets, decode them and submit connection, i do not need all http driver infrusturcture, why to enforce using it? So we can have at least 3 ways for supporting multiprotocols: - standard callbacks, completely different threads, no connection pooling (Exists now) - Ns_DriverSockQueue, new thread or callback thread, reusing connection pooling. All filters/traces are reused as well - driver Parse proc in the http driver to reuse drivers thread and connection pooling with all filters/traces All 3 methiods can co-exists and do not interfere with each other, that's my point, and all they add just couple extra functions. I would implement all methods, i have 3 drivers i need to run: smtp, dns, snmp, can add imap in the future ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 20:31 Message: Logged In: YES user_id=87254 "Each way has its own drawbacks..." That's what I'm asking. What are the drawbacks for each approach we've come up with so far? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 15:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 08:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 15:36:06
|
Feature Requests item #1156141, was opened at 2005-03-03 20:02 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Closed Resolution: Rejected Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 15:36 Message: Logged In: YES user_id=184124 And that's why i want to add simple UDP support to the server: for implementing simple messaging between servers in the cluster, using UDP without HTTP overhead will make it simple and faster. Using Tcl channel is intermediate solution for such messaging because it eliminates making multiple HTTP requests to the server but still uses heavy TCP/HTTP overhead. Still i think this command is very usefull and will make server more versatile provided that we will supply usage examples with the server as well. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 15:26 Message: Logged In: YES user_id=95086 "Second, this has nothing to do with non-HTTP protocol, it is HTTP communication protocol, what's inside the body has nothing to do with it again." This is true. It is a regular http request, just for the body part, the peers are engaging in a bidirectional communication and can do whatever they pleased. The request is considered done when one of the peers close the socket. Strictly speaking, I'm using the http body and this trick to implement arbitrary client-server comm channel. This will not work for browsers of course. It has nothing to do with browsers at all in fact. It arose out of the necessity to support alternate protocols given what we had in AS, that is pure http machinery. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 14:38 Message: Logged In: YES user_id=184124 On the closing note, keepalive has nothing to do with this and should not be considered in this case. Even if you connect using HTTP 1.1 and specify keepalive, sever has the right to issue Connection: close header which will disable keepalive. So, it WILL work for all connections. Second, this has nothing to do with non-HTTP protocol, it is HTTP communication protocol, what's inside the body has nothing to do with it again. And pipelining is a bad example here, the point it to send the request and wait for the answer and base on that send another request. Pipelining will allow you just to submit all requests and then later receive replies, it was designed for speeding browser's connection especially retrieving images. It looks to me Stephen you are all for HTTP core and around-the-HTTP-driver solutions only. This will make simple solutions complex because of getting around/workaround HTTP based core in the server. What is wrong with one simpe function which will give the server more options without breaking anything? ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 09:45 Message: Logged In: YES user_id=95086 I see. The gotcha's are: o. it's yet another implementation of non-HTTP protocol o. (HTTP 1.0 only, and no Keepalive header) be used to get o. function in the docs might reasonably expect it to work for all HTTP connections That's pretty clear. Well, I think you are right. This is our attempt to work around http-only caps of the server. Agreed. It will be better to get it done properly and we will then internally switch to the new implementation when it's ready. At the moment I have all this in a separate module and this can stay so until the multi-protocol caps are agreed upon and implemented. I will close this RFE then. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 09:36 Message: Logged In: YES user_id=87254 I don't understand why you're not using standard HTTP pipelining. It would seem to be so much easier than having to write custom client and server code. This is a small change, but I'm not excited about it because it's yet another implementation of non-HTTP protocol support, which makes 3 so far... :-( It also seems to require that a deliberately broken client (HTTP 1.0 only, and no Keepalive header) be used to get around the server's io read-ahead. Someone seeing this function in the docs might reasonably expect it to work for all HTTP connections. Also, I don't think this requires core support. Ns_ConnSock() will return the underlying file descriptor for a connection. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 08:36 Message: Logged In: YES user_id=95086 OK. Maybe I should explain little bit more what I'm using this for. Imagine you'd like to make a client-server connection and then exchange arbitrary data on this connection. Imagine something like telnet connection. You telnet from client to server, get a comm-pipe betwen them and then you can shuffle whatever you like across this comm-pipe. What we have is a http server responding only to http requests (as it is now, w/o multiprotocol support). So, what I do is to use http to setup the connection, have a registered procedure on the server grab the socket and install my custom protocol handler on it. The http is only used to establish the communication. All the rest is handled by my custom code on the server and on the client. Once the connection is setup, we have a point-to-point socket communication link between server and client and we can use it for just about anything. All you need for this is a client-side http-library (we have ours written in Tcl) which will give you the socket back once the connection is established) and serve-side proc implementing your protocol of choice. So, the entire server http-machinery stays in place. I do not see any problem there because we have been using this trick for about 4 years in production very successfully. What I do not know is wether there are any gotcha's with that approach (we haven't hit any yet). I also do not know what will happen when we add alternate multiprotocol caps to server. Maybe this will become obsolete. But, as I see this now, the change is trivial and does not hurt anybody. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:21 Message: Logged In: YES user_id=87254 HTTP also has (although our server doesn't, but I want to add this) pipelining, which is sending multiple requests at once without first waiting for the reply to previous requests. That sounds like your "i do it when I send multiple http requests". ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 22:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 22:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 15:26:27
|
Feature Requests item #1156141, was opened at 2005-03-03 21:02 Message generated for change (Comment added) made by vasiljevic You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Closed Resolution: Rejected Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 16:26 Message: Logged In: YES user_id=95086 "Second, this has nothing to do with non-HTTP protocol, it is HTTP communication protocol, what's inside the body has nothing to do with it again." This is true. It is a regular http request, just for the body part, the peers are engaging in a bidirectional communication and can do whatever they pleased. The request is considered done when one of the peers close the socket. Strictly speaking, I'm using the http body and this trick to implement arbitrary client-server comm channel. This will not work for browsers of course. It has nothing to do with browsers at all in fact. It arose out of the necessity to support alternate protocols given what we had in AS, that is pure http machinery. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 15:38 Message: Logged In: YES user_id=184124 On the closing note, keepalive has nothing to do with this and should not be considered in this case. Even if you connect using HTTP 1.1 and specify keepalive, sever has the right to issue Connection: close header which will disable keepalive. So, it WILL work for all connections. Second, this has nothing to do with non-HTTP protocol, it is HTTP communication protocol, what's inside the body has nothing to do with it again. And pipelining is a bad example here, the point it to send the request and wait for the answer and base on that send another request. Pipelining will allow you just to submit all requests and then later receive replies, it was designed for speeding browser's connection especially retrieving images. It looks to me Stephen you are all for HTTP core and around-the-HTTP-driver solutions only. This will make simple solutions complex because of getting around/workaround HTTP based core in the server. What is wrong with one simpe function which will give the server more options without breaking anything? ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 10:45 Message: Logged In: YES user_id=95086 I see. The gotcha's are: o. it's yet another implementation of non-HTTP protocol o. (HTTP 1.0 only, and no Keepalive header) be used to get o. function in the docs might reasonably expect it to work for all HTTP connections That's pretty clear. Well, I think you are right. This is our attempt to work around http-only caps of the server. Agreed. It will be better to get it done properly and we will then internally switch to the new implementation when it's ready. At the moment I have all this in a separate module and this can stay so until the multi-protocol caps are agreed upon and implemented. I will close this RFE then. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 10:36 Message: Logged In: YES user_id=87254 I don't understand why you're not using standard HTTP pipelining. It would seem to be so much easier than having to write custom client and server code. This is a small change, but I'm not excited about it because it's yet another implementation of non-HTTP protocol support, which makes 3 so far... :-( It also seems to require that a deliberately broken client (HTTP 1.0 only, and no Keepalive header) be used to get around the server's io read-ahead. Someone seeing this function in the docs might reasonably expect it to work for all HTTP connections. Also, I don't think this requires core support. Ns_ConnSock() will return the underlying file descriptor for a connection. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 09:36 Message: Logged In: YES user_id=95086 OK. Maybe I should explain little bit more what I'm using this for. Imagine you'd like to make a client-server connection and then exchange arbitrary data on this connection. Imagine something like telnet connection. You telnet from client to server, get a comm-pipe betwen them and then you can shuffle whatever you like across this comm-pipe. What we have is a http server responding only to http requests (as it is now, w/o multiprotocol support). So, what I do is to use http to setup the connection, have a registered procedure on the server grab the socket and install my custom protocol handler on it. The http is only used to establish the communication. All the rest is handled by my custom code on the server and on the client. Once the connection is setup, we have a point-to-point socket communication link between server and client and we can use it for just about anything. All you need for this is a client-side http-library (we have ours written in Tcl) which will give you the socket back once the connection is established) and serve-side proc implementing your protocol of choice. So, the entire server http-machinery stays in place. I do not see any problem there because we have been using this trick for about 4 years in production very successfully. What I do not know is wether there are any gotcha's with that approach (we haven't hit any yet). I also do not know what will happen when we add alternate multiprotocol caps to server. Maybe this will become obsolete. But, as I see this now, the change is trivial and does not hurt anybody. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 00:21 Message: Logged In: YES user_id=87254 HTTP also has (although our server doesn't, but I want to add this) pipelining, which is sending multiple requests at once without first waiting for the reply to previous requests. That sounds like your "i do it when I send multiple http requests". ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 14:41:50
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 14:41 Message: Logged In: YES user_id=184124 Can you provide what you have to modify in driver/nssock, i think that will not be easy and will require too many modifications. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 10:08 Message: Logged In: YES user_id=87254 What do you think of my suggestion that the nssock module and nsd/driver.c could be easily modified to handle udp and unix domain sockets when specified in the config file? A change like that combined with the work you've already done to the binder would mean no development is required to handle all socket types, and would be transparent to the upper layers. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-26 17:10 Message: Logged In: YES user_id=184124 Stephan, I am suggesting a compromise, let's put ParseProc and DriverSockQueu patches, they can coexists easily but will offer two way to add new drivers. Once we have new drivers working we will see what can be modified/adjusted, for now we do not have anything except my several drivers i use in the production and i use my own loop for smtp driver to keep processing and state machine in one place. Again, i have nothing against your method, i used to do similar in my previous versions but now i need to be completely free to implement aditional driver and i think we should have this ability. Zoran, what do you think? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 22:57 Message: Logged In: YES user_id=184124 Try to implement HTTP over UDP as i provided in the patch, you will need to modify core driver again, this is one example. Next, if i want to implement small driver that do not interact with web part of aolserver, no filters, it just need to do one small thing, it should be fast but will handle very many requests. I do not need filters, callbacks, i need my own main loop. I do not understand why you are insisting that all drivers should go through http driver's main loop. If i implement dns server and web interface to it, why they should compete in the same driver's main loop and go through registered filters if they share only backend database structure? ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 22:22 Message: Logged In: YES user_id=87254 You don't have to use registered filters or procs. It's an option for protocol implementations where the initial accept and parsing take place in the driver thread, and processing takes place in a conn thread. Once you submit to the queue however, either a filter, a registered proc, or a registered proxy proc must handle the request. It makes no difference whether you submit explicitly with Ns_DriverSockQueue or implicitly after your Ns_ParseProc completes. How much of the request structure you fill in is up to you, it's all optional. If you choose not to use it there is no overhead. It doesn't have to be a text based protocol for this to make sense. You might fill in the request structure if you wanted to enable people to handle different request types via C or Tcl. Or, you could store binary data in Ns_Cls storage and access it from registered procs etc. In the case where you want to handle connection accept/parse/reponse processing in the same thread, nothing needs to be added. You can do that today with Ns_SockCallback (or by placing all your code in an Ns_ParseProc and returning the response from there) Can you be more specific about the overhead of using an Ns_ParseProc? Exactly what memory is allocated, what code is run that should not..? What specifically can you not do with the Ns_ParseProc interface that you need the Ns_Driver* routines for? I'm looking for concrete examples. As far as I can tell, with Ns_ParseProc you write less code, get more options, and it takes advantage of infrastructure to give you more speed. I must be missing something, but you'll have to explain it to me in more detail because I don't get it. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 20:43 Message: Logged In: YES user_id=184124 Okay, the biggest problem with your way i see that i am enforced to go through http driver, even with my parse proc, i need to know how it works to implement my driver so correct hooks/filters will be called. If i want skip some parts i am not able to do it, http driver works only one way. Another issue is if something will not fit into current driver, additional hook needs to be introduced and core driver needs to be modified again. If my driver fits http-like/text mode paradigm, reusing http driver is the easiest way, but if i need something specific, all extra step to mimic the connection as http request, so all other parts of the http driver will not fail is just unnecesssary extra efforts. And i you mentioned, using Ns_DriverSockQueue is low level function and requires from the developer full attention for bulding the driver, but if this is what i want, why not. I can spawn my own thread if some callbacks are long running Tcl scripts and queue connection from my own loop. If i omly want to received packets, decode them and submit connection, i do not need all http driver infrusturcture, why to enforce using it? So we can have at least 3 ways for supporting multiprotocols: - standard callbacks, completely different threads, no connection pooling (Exists now) - Ns_DriverSockQueue, new thread or callback thread, reusing connection pooling. All filters/traces are reused as well - driver Parse proc in the http driver to reuse drivers thread and connection pooling with all filters/traces All 3 methiods can co-exists and do not interfere with each other, that's my point, and all they add just couple extra functions. I would implement all methods, i have 3 drivers i need to run: smtp, dns, snmp, can add imap in the future ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 20:31 Message: Logged In: YES user_id=87254 "Each way has its own drawbacks..." That's what I'm asking. What are the drawbacks for each approach we've come up with so far? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 15:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 08:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 14:38:27
|
Feature Requests item #1156141, was opened at 2005-03-03 20:02 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Closed Resolution: Rejected Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-03-04 14:38 Message: Logged In: YES user_id=184124 On the closing note, keepalive has nothing to do with this and should not be considered in this case. Even if you connect using HTTP 1.1 and specify keepalive, sever has the right to issue Connection: close header which will disable keepalive. So, it WILL work for all connections. Second, this has nothing to do with non-HTTP protocol, it is HTTP communication protocol, what's inside the body has nothing to do with it again. And pipelining is a bad example here, the point it to send the request and wait for the answer and base on that send another request. Pipelining will allow you just to submit all requests and then later receive replies, it was designed for speeding browser's connection especially retrieving images. It looks to me Stephen you are all for HTTP core and around-the-HTTP-driver solutions only. This will make simple solutions complex because of getting around/workaround HTTP based core in the server. What is wrong with one simpe function which will give the server more options without breaking anything? ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 09:45 Message: Logged In: YES user_id=95086 I see. The gotcha's are: o. it's yet another implementation of non-HTTP protocol o. (HTTP 1.0 only, and no Keepalive header) be used to get o. function in the docs might reasonably expect it to work for all HTTP connections That's pretty clear. Well, I think you are right. This is our attempt to work around http-only caps of the server. Agreed. It will be better to get it done properly and we will then internally switch to the new implementation when it's ready. At the moment I have all this in a separate module and this can stay so until the multi-protocol caps are agreed upon and implemented. I will close this RFE then. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 09:36 Message: Logged In: YES user_id=87254 I don't understand why you're not using standard HTTP pipelining. It would seem to be so much easier than having to write custom client and server code. This is a small change, but I'm not excited about it because it's yet another implementation of non-HTTP protocol support, which makes 3 so far... :-( It also seems to require that a deliberately broken client (HTTP 1.0 only, and no Keepalive header) be used to get around the server's io read-ahead. Someone seeing this function in the docs might reasonably expect it to work for all HTTP connections. Also, I don't think this requires core support. Ns_ConnSock() will return the underlying file descriptor for a connection. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 08:36 Message: Logged In: YES user_id=95086 OK. Maybe I should explain little bit more what I'm using this for. Imagine you'd like to make a client-server connection and then exchange arbitrary data on this connection. Imagine something like telnet connection. You telnet from client to server, get a comm-pipe betwen them and then you can shuffle whatever you like across this comm-pipe. What we have is a http server responding only to http requests (as it is now, w/o multiprotocol support). So, what I do is to use http to setup the connection, have a registered procedure on the server grab the socket and install my custom protocol handler on it. The http is only used to establish the communication. All the rest is handled by my custom code on the server and on the client. Once the connection is setup, we have a point-to-point socket communication link between server and client and we can use it for just about anything. All you need for this is a client-side http-library (we have ours written in Tcl) which will give you the socket back once the connection is established) and serve-side proc implementing your protocol of choice. So, the entire server http-machinery stays in place. I do not see any problem there because we have been using this trick for about 4 years in production very successfully. What I do not know is wether there are any gotcha's with that approach (we haven't hit any yet). I also do not know what will happen when we add alternate multiprotocol caps to server. Maybe this will become obsolete. But, as I see this now, the change is trivial and does not hurt anybody. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:21 Message: Logged In: YES user_id=87254 HTTP also has (although our server doesn't, but I want to add this) pipelining, which is sending multiple requests at once without first waiting for the reply to previous requests. That sounds like your "i do it when I send multiple http requests". ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 22:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 22:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 10:08:15
|
Feature Requests item #1151137, was opened at 2005-02-24 09:01 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 03:08 Message: Logged In: YES user_id=87254 What do you think of my suggestion that the nssock module and nsd/driver.c could be easily modified to handle udp and unix domain sockets when specified in the config file? A change like that combined with the work you've already done to the binder would mean no development is required to handle all socket types, and would be transparent to the upper layers. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-26 10:10 Message: Logged In: YES user_id=184124 Stephan, I am suggesting a compromise, let's put ParseProc and DriverSockQueu patches, they can coexists easily but will offer two way to add new drivers. Once we have new drivers working we will see what can be modified/adjusted, for now we do not have anything except my several drivers i use in the production and i use my own loop for smtp driver to keep processing and state machine in one place. Again, i have nothing against your method, i used to do similar in my previous versions but now i need to be completely free to implement aditional driver and i think we should have this ability. Zoran, what do you think? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 15:57 Message: Logged In: YES user_id=184124 Try to implement HTTP over UDP as i provided in the patch, you will need to modify core driver again, this is one example. Next, if i want to implement small driver that do not interact with web part of aolserver, no filters, it just need to do one small thing, it should be fast but will handle very many requests. I do not need filters, callbacks, i need my own main loop. I do not understand why you are insisting that all drivers should go through http driver's main loop. If i implement dns server and web interface to it, why they should compete in the same driver's main loop and go through registered filters if they share only backend database structure? ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 15:22 Message: Logged In: YES user_id=87254 You don't have to use registered filters or procs. It's an option for protocol implementations where the initial accept and parsing take place in the driver thread, and processing takes place in a conn thread. Once you submit to the queue however, either a filter, a registered proc, or a registered proxy proc must handle the request. It makes no difference whether you submit explicitly with Ns_DriverSockQueue or implicitly after your Ns_ParseProc completes. How much of the request structure you fill in is up to you, it's all optional. If you choose not to use it there is no overhead. It doesn't have to be a text based protocol for this to make sense. You might fill in the request structure if you wanted to enable people to handle different request types via C or Tcl. Or, you could store binary data in Ns_Cls storage and access it from registered procs etc. In the case where you want to handle connection accept/parse/reponse processing in the same thread, nothing needs to be added. You can do that today with Ns_SockCallback (or by placing all your code in an Ns_ParseProc and returning the response from there) Can you be more specific about the overhead of using an Ns_ParseProc? Exactly what memory is allocated, what code is run that should not..? What specifically can you not do with the Ns_ParseProc interface that you need the Ns_Driver* routines for? I'm looking for concrete examples. As far as I can tell, with Ns_ParseProc you write less code, get more options, and it takes advantage of infrastructure to give you more speed. I must be missing something, but you'll have to explain it to me in more detail because I don't get it. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 13:43 Message: Logged In: YES user_id=184124 Okay, the biggest problem with your way i see that i am enforced to go through http driver, even with my parse proc, i need to know how it works to implement my driver so correct hooks/filters will be called. If i want skip some parts i am not able to do it, http driver works only one way. Another issue is if something will not fit into current driver, additional hook needs to be introduced and core driver needs to be modified again. If my driver fits http-like/text mode paradigm, reusing http driver is the easiest way, but if i need something specific, all extra step to mimic the connection as http request, so all other parts of the http driver will not fail is just unnecesssary extra efforts. And i you mentioned, using Ns_DriverSockQueue is low level function and requires from the developer full attention for bulding the driver, but if this is what i want, why not. I can spawn my own thread if some callbacks are long running Tcl scripts and queue connection from my own loop. If i omly want to received packets, decode them and submit connection, i do not need all http driver infrusturcture, why to enforce using it? So we can have at least 3 ways for supporting multiprotocols: - standard callbacks, completely different threads, no connection pooling (Exists now) - Ns_DriverSockQueue, new thread or callback thread, reusing connection pooling. All filters/traces are reused as well - driver Parse proc in the http driver to reuse drivers thread and connection pooling with all filters/traces All 3 methiods can co-exists and do not interfere with each other, that's my point, and all they add just couple extra functions. I would implement all methods, i have 3 drivers i need to run: smtp, dns, snmp, can add imap in the future ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 13:31 Message: Logged In: YES user_id=87254 "Each way has its own drawbacks..." That's what I'm asking. What are the drawbacks for each approach we've come up with so far? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 08:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 01:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 14:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 09:45:46
|
Feature Requests item #1156141, was opened at 2005-03-03 21:02 Message generated for change (Comment added) made by vasiljevic You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 10:45 Message: Logged In: YES user_id=95086 I see. The gotcha's are: o. it's yet another implementation of non-HTTP protocol o. (HTTP 1.0 only, and no Keepalive header) be used to get o. function in the docs might reasonably expect it to work for all HTTP connections That's pretty clear. Well, I think you are right. This is our attempt to work around http-only caps of the server. Agreed. It will be better to get it done properly and we will then internally switch to the new implementation when it's ready. At the moment I have all this in a separate module and this can stay so until the multi-protocol caps are agreed upon and implemented. I will close this RFE then. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 10:36 Message: Logged In: YES user_id=87254 I don't understand why you're not using standard HTTP pipelining. It would seem to be so much easier than having to write custom client and server code. This is a small change, but I'm not excited about it because it's yet another implementation of non-HTTP protocol support, which makes 3 so far... :-( It also seems to require that a deliberately broken client (HTTP 1.0 only, and no Keepalive header) be used to get around the server's io read-ahead. Someone seeing this function in the docs might reasonably expect it to work for all HTTP connections. Also, I don't think this requires core support. Ns_ConnSock() will return the underlying file descriptor for a connection. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 09:36 Message: Logged In: YES user_id=95086 OK. Maybe I should explain little bit more what I'm using this for. Imagine you'd like to make a client-server connection and then exchange arbitrary data on this connection. Imagine something like telnet connection. You telnet from client to server, get a comm-pipe betwen them and then you can shuffle whatever you like across this comm-pipe. What we have is a http server responding only to http requests (as it is now, w/o multiprotocol support). So, what I do is to use http to setup the connection, have a registered procedure on the server grab the socket and install my custom protocol handler on it. The http is only used to establish the communication. All the rest is handled by my custom code on the server and on the client. Once the connection is setup, we have a point-to-point socket communication link between server and client and we can use it for just about anything. All you need for this is a client-side http-library (we have ours written in Tcl) which will give you the socket back once the connection is established) and serve-side proc implementing your protocol of choice. So, the entire server http-machinery stays in place. I do not see any problem there because we have been using this trick for about 4 years in production very successfully. What I do not know is wether there are any gotcha's with that approach (we haven't hit any yet). I also do not know what will happen when we add alternate multiprotocol caps to server. Maybe this will become obsolete. But, as I see this now, the change is trivial and does not hurt anybody. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 00:21 Message: Logged In: YES user_id=87254 HTTP also has (although our server doesn't, but I want to add this) pipelining, which is sending multiple requests at once without first waiting for the reply to previous requests. That sounds like your "i do it when I send multiple http requests". ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 09:36:35
|
Feature Requests item #1156141, was opened at 2005-03-03 13:02 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 02:36 Message: Logged In: YES user_id=87254 I don't understand why you're not using standard HTTP pipelining. It would seem to be so much easier than having to write custom client and server code. This is a small change, but I'm not excited about it because it's yet another implementation of non-HTTP protocol support, which makes 3 so far... :-( It also seems to require that a deliberately broken client (HTTP 1.0 only, and no Keepalive header) be used to get around the server's io read-ahead. Someone seeing this function in the docs might reasonably expect it to work for all HTTP connections. Also, I don't think this requires core support. Ns_ConnSock() will return the underlying file descriptor for a connection. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 01:36 Message: Logged In: YES user_id=95086 OK. Maybe I should explain little bit more what I'm using this for. Imagine you'd like to make a client-server connection and then exchange arbitrary data on this connection. Imagine something like telnet connection. You telnet from client to server, get a comm-pipe betwen them and then you can shuffle whatever you like across this comm-pipe. What we have is a http server responding only to http requests (as it is now, w/o multiprotocol support). So, what I do is to use http to setup the connection, have a registered procedure on the server grab the socket and install my custom protocol handler on it. The http is only used to establish the communication. All the rest is handled by my custom code on the server and on the client. Once the connection is setup, we have a point-to-point socket communication link between server and client and we can use it for just about anything. All you need for this is a client-side http-library (we have ours written in Tcl) which will give you the socket back once the connection is established) and serve-side proc implementing your protocol of choice. So, the entire server http-machinery stays in place. I do not see any problem there because we have been using this trick for about 4 years in production very successfully. What I do not know is wether there are any gotcha's with that approach (we haven't hit any yet). I also do not know what will happen when we add alternate multiprotocol caps to server. Maybe this will become obsolete. But, as I see this now, the change is trivial and does not hurt anybody. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 16:21 Message: Logged In: YES user_id=87254 HTTP also has (although our server doesn't, but I want to add this) pipelining, which is sending multiple requests at once without first waiting for the reply to previous requests. That sounds like your "i do it when I send multiple http requests". ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 15:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 15:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 08:38:26
|
Feature Requests item #1156107, was opened at 2005-03-03 20:04 Message generated for change (Comment added) made by vasiljevic You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156107&group_id=130646 Category: Tcl-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Scrap optional connid parameter from Tcl API Initial Comment: Some Tcl-API commands still accept optional connection-id parameter which is left as the compatibility trace from older 2.x nsd servers. This complicates argument processing and serves no other obvious purpose. The idea is to scrap this optional connid parsing from the Tcl-API calls making them simpler to document and implement. I never used any code running for the 2.x servers hence for me it is OK to remove those. Any thoughts on that? ---------------------------------------------------------------------- >Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 09:38 Message: Logged In: YES user_id=95086 ns_conn ns_register_filter ns_return (some of them) They all accept a weird syntax: <proc> ?connid? args which is PITA to parse and maintain. Yet, it is not used any more... It is only there to be able to run 2.x server-side Tcl code ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:08 Message: Logged In: YES user_id=184124 Which commands are you talking about? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156107&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-04 08:36:07
|
Feature Requests item #1156141, was opened at 2005-03-03 21:02 Message generated for change (Comment added) made by vasiljevic You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-03-04 09:36 Message: Logged In: YES user_id=95086 OK. Maybe I should explain little bit more what I'm using this for. Imagine you'd like to make a client-server connection and then exchange arbitrary data on this connection. Imagine something like telnet connection. You telnet from client to server, get a comm-pipe betwen them and then you can shuffle whatever you like across this comm-pipe. What we have is a http server responding only to http requests (as it is now, w/o multiprotocol support). So, what I do is to use http to setup the connection, have a registered procedure on the server grab the socket and install my custom protocol handler on it. The http is only used to establish the communication. All the rest is handled by my custom code on the server and on the client. Once the connection is setup, we have a point-to-point socket communication link between server and client and we can use it for just about anything. All you need for this is a client-side http-library (we have ours written in Tcl) which will give you the socket back once the connection is established) and serve-side proc implementing your protocol of choice. So, the entire server http-machinery stays in place. I do not see any problem there because we have been using this trick for about 4 years in production very successfully. What I do not know is wether there are any gotcha's with that approach (we haven't hit any yet). I also do not know what will happen when we add alternate multiprotocol caps to server. Maybe this will become obsolete. But, as I see this now, the change is trivial and does not hurt anybody. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-04 00:21 Message: Logged In: YES user_id=87254 HTTP also has (although our server doesn't, but I want to add this) pipelining, which is sending multiple requests at once without first waiting for the reply to previous requests. That sounds like your "i do it when I send multiple http requests". ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 23:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 23:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 23:25:42
|
Feature Requests item #1145957, was opened at 2005-02-21 20:13 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1145957&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen Deasey (sdeasey) Assigned to: Stephen Deasey (sdeasey) Summary: Cookie support for C and Tcl Initial Comment: Atached is a patch which implements a simple cookie API for both C and Tcl. Here's what it looks like (the formating will probably get mangled, sorry): void Ns_ConnSetCookie(Ns_Conn *conn, char *name, char *value, int maxage); void Ns_ConnSetSecureCookie(Ns_Conn *conn, char *name, char *value, int maxage); void Ns_ConnSetCookieEx(Ns_Conn *conn, char *name, char *value, int maxage, char *domain, char *path, int secure); void Ns_ConnDeleteCookie(Ns_Conn *conn, char *name, char *domain, char *path); void Ns_ConnDeleteSecureCookie(Ns_Conn *conn, char *name, char *domain, char *path); char *Ns_ConnGetCookie(Ns_DString *dest, Ns_Conn *conn, char *name); ns_setcookie ?-secure bool? ?-domain d? ?-path p? ?-maxage secs? name data ns_getcookie name ?default? ns_deletecookie ?-secure bool? ?-domain d? ?-path p? name As you can see it only tackles the basics, no comment or version field. Although I've never personaly had to use those features. Cookies are a pre-requisite for the authentication module which exercises the new autht/authz API I haven't written yet... Obviously, the string parsing in the GetCookie code will have to be very carefully looked at before this sees the Internet. But in general, does this look like it's on the right track? ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 16:25 Message: Logged In: YES user_id=87254 Good call. I've been using a simillar algorithm in Tcl only and it works fine, but I guess that's because I've only been storing session keys etc. which are quite small. I'll have to examine all the cookie headers. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 09:05 Message: Logged In: YES user_id=184124 Looks good, the only issue i have, some browser can send more than one Cookie header and you are looking only in the first header. Also, this is new API, no problem adding it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1145957&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 23:21:55
|
Feature Requests item #1156141, was opened at 2005-03-03 13:02 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 16:21 Message: Logged In: YES user_id=87254 HTTP also has (although our server doesn't, but I want to add this) pipelining, which is sending multiple requests at once without first waiting for the reply to previous requests. That sounds like your "i do it when I send multiple http requests". ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 15:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 15:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 22:45:44
|
Feature Requests item #1156141, was opened at 2005-03-03 20:02 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:45 Message: Logged In: YES user_id=184124 Keepalive works when i close my connection, i.e. when i exited from conn thread, but the point here is to do it from the conn thread, i received request from my other server sending me some info, i send my info back and get ack. Everything inside one connection, before keepalive. And yes, i send messages in my format, anyway i do it when i send multiple http reaqests, i am interested in message body only. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 22:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 22:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 22:32:29
|
Feature Requests item #1156141, was opened at 2005-03-03 13:02 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 15:32 Message: Logged In: YES user_id=87254 HTTP already handles sending multiple requests accross one socket with keepalives. It might be more useful to add keepalives to our HTTP client code. In the case of HTTP keepalives, the socket is returned to the driver thread after the first request, still open, and the conn thread goes on to do other useful work. If a new request arrives before the connection times out it is passed to the next free conn thread. If you don't usee HTTP with keepalives you will have to invent some kind of message chunking syntax and some kind of time-out protocol. But you will still end up wasting conn threads as they idle waiting for possible extra messages. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 15:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 22:20:15
|
Feature Requests item #1156141, was opened at 2005-03-03 20:02 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:20 Message: Logged In: YES user_id=184124 I could use it in my apps where i have cluster of several servers exchanging http requests between each other, sometimes one session needs 2 or more exchanges. With this channel i can exchange using same connection and i need to send/receive data, not headers, so this can save and improve my operations. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 22:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 22:12:25
|
Feature Requests item #1156141, was opened at 2005-03-03 13:02 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-03-03 15:12 Message: Logged In: YES user_id=87254 Usually, by the time your script runs and you call [ns_conn channel], the data from the client will already have been read and buffered. You can access this with [ns_conn content]. Does reading work? What's the objective? Do you want to hadle streaming data, or make reading and writing more Tcl-like? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 15:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 22:08:09
|
Feature Requests item #1156107, was opened at 2005-03-03 19:04 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156107&group_id=130646 Category: Tcl-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Scrap optional connid parameter from Tcl API Initial Comment: Some Tcl-API commands still accept optional connection-id parameter which is left as the compatibility trace from older 2.x nsd servers. This complicates argument processing and serves no other obvious purpose. The idea is to scrap this optional connid parsing from the Tcl-API calls making them simpler to document and implement. I never used any code running for the 2.x servers hence for me it is OK to remove those. Any thoughts on that? ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:08 Message: Logged In: YES user_id=184124 Which commands are you talking about? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156107&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 22:07:21
|
Feature Requests item #1156141, was opened at 2005-03-03 20:02 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-03-03 22:07 Message: Logged In: YES user_id=184124 Very usefull, commit it ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 22:04:05
|
Feature Requests item #1156239, was opened at 2005-03-03 22:04 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: nslog restructure Initial Comment: Post submit for nslog restructure, no functionality changed really, just added new commands and options to control nslog on the fly. The source code changed to be more flexible of extending nslog functionality with more options. The only possible bug fix is checking for unknown in the X-Forwarded-For header. Sorry for ignoring/forgetting the procedure. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 20:02:51
|
Feature Requests item #1156141, was opened at 2005-03-03 21:02 Message generated for change (Settings changed) made by vasiljevic You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) >Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-03-03 20:02:27
|
Feature Requests item #1156141, was opened at 2005-03-03 21:02 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Nobody/Anonymous (nobody) Summary: Add ns_conn channel command Initial Comment: Extend the ns_conn command to allow wrapping an Tcl channel arround it. This channel can be used to talk to the remote client afterwards. The idea is simple. The client uses standard http protocol to open up a connection to the server. Server accepts the connection and obtains the channel from it. The client also gets the channel to the server and from this point on, both can exchange arbitrary data. Example implementation is given as a patch against the current CVS state. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646 |
