Menu
▾
▴
[ naviserver-Feature Requests-1151137 ] New driver API and Udp module
|
From: SourceForge.net <no...@so...> - 2005-02-25 05:51:03
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
