Menu
▾
▴
naviserver-tracker — SourceForge Tracker Notifications
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
(61) |
Mar
(153) |
Apr
(39) |
May
(10) |
Jun
(15) |
Jul
(15) |
Aug
(2) |
Sep
|
Oct
(17) |
Nov
(2) |
Dec
(13) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
(18) |
Feb
(9) |
Mar
|
Apr
|
May
|
Jun
|
Jul
(7) |
Aug
(1) |
Sep
(2) |
Oct
|
Nov
(1) |
Dec
|
| 2007 |
Jan
(8) |
Feb
(3) |
Mar
|
Apr
|
May
(2) |
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2008 |
Jan
|
Feb
|
Mar
|
Apr
(6) |
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: SourceForge.net <no...@so...> - 2005-03-03 19:04:45
|
Feature Requests item #1156107, was opened at 2005-03-03 20:04 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156107&group_id=130646 Category: Tcl-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Scrap optional connid parameter from Tcl API Initial Comment: Some Tcl-API commands still accept optional connection-id parameter which is left as the compatibility trace from older 2.x nsd servers. This complicates argument processing and serves no other obvious purpose. The idea is to scrap this optional connid parsing from the Tcl-API calls making them simpler to document and implement. I never used any code running for the 2.x servers hence for me it is OK to remove those. Any thoughts on that? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156107&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-28 05:07:22
|
Feature Requests item #1122940, was opened at 2005-02-15 01:05 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1122940&group_id=130646 Category: C-API Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: Stephen Deasey (sdeasey) Assigned to: Stephen Deasey (sdeasey) Summary: Tcl command switch parsing routines Initial Comment: Parsing the switches and arguments for a Tcl command can be a real pain: it's very easy to make off-by-one errors, and it often involves a lot of repetitive typing which obscures the real intent of the command. The attached patch provides the routine Ns_ParseObjv. It parses switches and arguments, checks types, and generates error/usage messages automatically. Switches are handled efficiently via indexed lookups. The intent is not only to reduce the number of errors in switch handling code but to make switch handling such an easy task that developers are more likely to design easy to use, tcl-friendly APIs rather than APIs that are convenient to implement. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-27 22:07 Message: Logged In: YES user_id=87254 Added to cvs. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-15 01:07 Message: Logged In: YES user_id=87254 Here's an example: int NsTclCacheCreateObjCmd(ClientData arg, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[]) { Ns_Cache *cache; Ns_DString ds; int scope = CACHE_SERVER; int size = 0, objsize = 0, entries = 0, timeout = 0; char *name = NULL; Ns_ObjvTable scopes[] = { {"global", CACHE_GLOBAL}, {"server", CACHE_SERVER}, {"interp", CACHE_INTERP}, NULL, NULL }; Ns_ObjvSpec opts[] = { {"-scope", Ns_ObjvIndex, &scope, &scopes}, {"-size", Ns_ObjvInt, &size, NULL}, {"-objsize", Ns_ObjvInt, &objsize, NULL}, {"-entries", Ns_ObjvInt, &entries, NULL}, {"-timeout", Ns_ObjvInt, &timeout, NULL}, {"--", Ns_ObjvBreak, NULL, NULL}, {NULL, NULL, NULL, NULL} }; Ns_ObjvSpec args[] = { {"cache", Ns_ObjvString, &name, NULL}, {NULL, NULL, NULL, NULL} }; if (Ns_ParseObjv(opts, args, interp, 1, objc, objv) != NS_OK) { return TCL_ERROR; } /* . . . */ return TCL_OK; } ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1122940&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-28 05:05:55
|
Bugs item #1145277, was opened at 2005-02-21 02:28 Message generated for change (Settings changed) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1145277&group_id=130646 Category: None Group: Current >Status: Closed Resolution: None Priority: 6 Submitted By: Stephen Deasey (sdeasey) Assigned to: Stephen Deasey (sdeasey) Summary: Encoding URL path and query components Initial Comment: The current code follows (mostly) the rules for encoding/decoding segments of a URL query component. But the rules for segments of the path component are different, and there is no code to handle this. One of those differences is the treatment of + characters. In the query component this should be decoded into an ASCII space. In the path component however it should be treated literaly. The end result is that clients which correctly request the file /some+file are dissapointed with a 404 error. This patch deprecates the old C API and introduces the following new routines: Tcl_Encoding Ns_GetUrlEncoding(char *charset); char *Ns_UrlPathEncode(Ns_DString *, char *, Tcl_Encoding); char *Ns_UrlPathDecode(Ns_DString *, char *, Tcl_Encoding); char *Ns_UrlQueryEncode(Ns_DString *, char *, Tcl_Encoding); char *Ns_UrlQueryDecode(Ns_DString *, char *, Tcl_Encoding); request.c is modified to call Ns_UrlPathDecode() and form.c is modified to call Ns_UrlQueryDecode(). The updated Tcl commands look like this: ns_urldecode ?-part path|query? ?--? segment ns_urlencode ?-part path|query? ?--? segment ?segment ...? If the URL component part is not specified, the default is 'query'. Multiple segments will be joined with / or & when encoding. The '-charset' switch has been dropped, it seems broken considering Tcl's use of UTF8 encoded strings internaly. I added a set of tests to exercise the Tcl commands. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-27 22:05 Message: Logged In: YES user_id=87254 Fixed in cvs. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1145277&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-28 04:35:13
|
Bugs item #1145927, was opened at 2005-02-21 19:09 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1145927&group_id=130646 Category: None Group: Current >Status: Closed Resolution: None Priority: 4 Submitted By: Stephen Deasey (sdeasey) Assigned to: Stephen Deasey (sdeasey) Summary: application/xml not text/xml Initial Comment: Currently, xml-like files are served up as text/xml, but the correct mime type is application/xml. Various 'bad things' happen when you get this wrong: http://www.xml.com/pub/a/2004/07/21/dive.html http://www.xml.com/pub/a/2004/08/04/deviant.html I'm going to change the .xml and .xsl mime types to application/xml. I'll also add in a few other xml types such as .rdf, rss and .xslt. And various missing media types like .mp4, .swf. .torrent etc. Should be no problems. ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-27 21:35 Message: Logged In: YES user_id=87254 Fixed in cvs. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1145927&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-26 17:10:07
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-26 17:10 Message: Logged In: YES user_id=184124 Stephan, I am suggesting a compromise, let's put ParseProc and DriverSockQueu patches, they can coexists easily but will offer two way to add new drivers. Once we have new drivers working we will see what can be modified/adjusted, for now we do not have anything except my several drivers i use in the production and i use my own loop for smtp driver to keep processing and state machine in one place. Again, i have nothing against your method, i used to do similar in my previous versions but now i need to be completely free to implement aditional driver and i think we should have this ability. Zoran, what do you think? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 22:57 Message: Logged In: YES user_id=184124 Try to implement HTTP over UDP as i provided in the patch, you will need to modify core driver again, this is one example. Next, if i want to implement small driver that do not interact with web part of aolserver, no filters, it just need to do one small thing, it should be fast but will handle very many requests. I do not need filters, callbacks, i need my own main loop. I do not understand why you are insisting that all drivers should go through http driver's main loop. If i implement dns server and web interface to it, why they should compete in the same driver's main loop and go through registered filters if they share only backend database structure? ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 22:22 Message: Logged In: YES user_id=87254 You don't have to use registered filters or procs. It's an option for protocol implementations where the initial accept and parsing take place in the driver thread, and processing takes place in a conn thread. Once you submit to the queue however, either a filter, a registered proc, or a registered proxy proc must handle the request. It makes no difference whether you submit explicitly with Ns_DriverSockQueue or implicitly after your Ns_ParseProc completes. How much of the request structure you fill in is up to you, it's all optional. If you choose not to use it there is no overhead. It doesn't have to be a text based protocol for this to make sense. You might fill in the request structure if you wanted to enable people to handle different request types via C or Tcl. Or, you could store binary data in Ns_Cls storage and access it from registered procs etc. In the case where you want to handle connection accept/parse/reponse processing in the same thread, nothing needs to be added. You can do that today with Ns_SockCallback (or by placing all your code in an Ns_ParseProc and returning the response from there) Can you be more specific about the overhead of using an Ns_ParseProc? Exactly what memory is allocated, what code is run that should not..? What specifically can you not do with the Ns_ParseProc interface that you need the Ns_Driver* routines for? I'm looking for concrete examples. As far as I can tell, with Ns_ParseProc you write less code, get more options, and it takes advantage of infrastructure to give you more speed. I must be missing something, but you'll have to explain it to me in more detail because I don't get it. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 20:43 Message: Logged In: YES user_id=184124 Okay, the biggest problem with your way i see that i am enforced to go through http driver, even with my parse proc, i need to know how it works to implement my driver so correct hooks/filters will be called. If i want skip some parts i am not able to do it, http driver works only one way. Another issue is if something will not fit into current driver, additional hook needs to be introduced and core driver needs to be modified again. If my driver fits http-like/text mode paradigm, reusing http driver is the easiest way, but if i need something specific, all extra step to mimic the connection as http request, so all other parts of the http driver will not fail is just unnecesssary extra efforts. And i you mentioned, using Ns_DriverSockQueue is low level function and requires from the developer full attention for bulding the driver, but if this is what i want, why not. I can spawn my own thread if some callbacks are long running Tcl scripts and queue connection from my own loop. If i omly want to received packets, decode them and submit connection, i do not need all http driver infrusturcture, why to enforce using it? So we can have at least 3 ways for supporting multiprotocols: - standard callbacks, completely different threads, no connection pooling (Exists now) - Ns_DriverSockQueue, new thread or callback thread, reusing connection pooling. All filters/traces are reused as well - driver Parse proc in the http driver to reuse drivers thread and connection pooling with all filters/traces All 3 methiods can co-exists and do not interfere with each other, that's my point, and all they add just couple extra functions. I would implement all methods, i have 3 drivers i need to run: smtp, dns, snmp, can add imap in the future ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 20:31 Message: Logged In: YES user_id=87254 "Each way has its own drawbacks..." That's what I'm asking. What are the drawbacks for each approach we've come up with so far? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 15:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 08:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-26 13:26:51
|
Feature Requests item #1120023, was opened at 2005-02-10 14:30 Message generated for change (Comment added) made by vasiljevic You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1120023&group_id=130646 Category: None Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) Assigned to: Zoran Vasiljevic (vasiljevic) Summary: Allow to start server under root privilges Initial Comment: Before all of you freak out, read please :-) Currently the server will just complain if you attempt to start it as root. However, we do need to do this. Our app is actually a backup/archive and server-synchronize product and it needs to access all files on the server and do something with them. For this to work, I had to manually disable checking for user/group of 0/0 during the startup. Now, my idea is to allow compile-time option which would control this, like for example --enable-rootuser or similar. I think Apache build allows you to do this (at least it did when I worked with Apache for the last time, about 6 years ago). Default would be to *not* allow server to be started as root, which is what we have now. This can be #ifdef ALLOW_ROOT_USER #endif at one place in the nsmain.c. Any thoughts? ---------------------------------------------------------------------- >Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-02-26 14:26 Message: Logged In: YES user_id=95086 Added in CVS. Closing this RFE. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-02-11 21:50 Message: Logged In: YES user_id=95086 Allright for me. As soon as cvs is in place I will add this. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-11 15:02 Message: Logged In: YES user_id=184124 That's okay with me ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-11 04:50 Message: Logged In: YES user_id=87254 I think thttpd works like this. A lot of it's config settings are compile time only options (you actually have to hack the makefile). But Vlad makes a good point: If some one asks to run as root maybe we should just respect that. I'm not sure about running as user 'nobody' as default though. I think that is an apache specific user, created when you install that package. It might not always exist. Also, if all daemons run as 'nobody' then you're really not increasing security, because a flaw in one exposes the data of all the others rather than contain it. I would suggest the following: You *have* to supply the user to switch to on the command line. If you don't, the server complains and exits. You may specify root, in which case we log a warning message but respect your wishes and continue. Specifying any other user works as before. I would be happy with something like that. Sound OK? ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-02-10 18:18 Message: Logged In: YES user_id=95086 You mean: no user given -> run as nobody (65534 or whatever uid nobody is) user given -> run as this user, whatever the user (might be root, uid=0) OK. No problems for me. This is even simpler. If nobody objects, I will make this change. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-10 16:25 Message: Logged In: YES user_id=184124 I have no problem with that, if want NS to be more versatile than just webserver we will have to run it as root for some tasks, so i would suggest event more radical solution, if root given as id on the command line, run it as root, if nothing given via command line, switch to default nobody if root. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1120023&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 22:57:41
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 22:57 Message: Logged In: YES user_id=184124 Try to implement HTTP over UDP as i provided in the patch, you will need to modify core driver again, this is one example. Next, if i want to implement small driver that do not interact with web part of aolserver, no filters, it just need to do one small thing, it should be fast but will handle very many requests. I do not need filters, callbacks, i need my own main loop. I do not understand why you are insisting that all drivers should go through http driver's main loop. If i implement dns server and web interface to it, why they should compete in the same driver's main loop and go through registered filters if they share only backend database structure? ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 22:22 Message: Logged In: YES user_id=87254 You don't have to use registered filters or procs. It's an option for protocol implementations where the initial accept and parsing take place in the driver thread, and processing takes place in a conn thread. Once you submit to the queue however, either a filter, a registered proc, or a registered proxy proc must handle the request. It makes no difference whether you submit explicitly with Ns_DriverSockQueue or implicitly after your Ns_ParseProc completes. How much of the request structure you fill in is up to you, it's all optional. If you choose not to use it there is no overhead. It doesn't have to be a text based protocol for this to make sense. You might fill in the request structure if you wanted to enable people to handle different request types via C or Tcl. Or, you could store binary data in Ns_Cls storage and access it from registered procs etc. In the case where you want to handle connection accept/parse/reponse processing in the same thread, nothing needs to be added. You can do that today with Ns_SockCallback (or by placing all your code in an Ns_ParseProc and returning the response from there) Can you be more specific about the overhead of using an Ns_ParseProc? Exactly what memory is allocated, what code is run that should not..? What specifically can you not do with the Ns_ParseProc interface that you need the Ns_Driver* routines for? I'm looking for concrete examples. As far as I can tell, with Ns_ParseProc you write less code, get more options, and it takes advantage of infrastructure to give you more speed. I must be missing something, but you'll have to explain it to me in more detail because I don't get it. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 20:43 Message: Logged In: YES user_id=184124 Okay, the biggest problem with your way i see that i am enforced to go through http driver, even with my parse proc, i need to know how it works to implement my driver so correct hooks/filters will be called. If i want skip some parts i am not able to do it, http driver works only one way. Another issue is if something will not fit into current driver, additional hook needs to be introduced and core driver needs to be modified again. If my driver fits http-like/text mode paradigm, reusing http driver is the easiest way, but if i need something specific, all extra step to mimic the connection as http request, so all other parts of the http driver will not fail is just unnecesssary extra efforts. And i you mentioned, using Ns_DriverSockQueue is low level function and requires from the developer full attention for bulding the driver, but if this is what i want, why not. I can spawn my own thread if some callbacks are long running Tcl scripts and queue connection from my own loop. If i omly want to received packets, decode them and submit connection, i do not need all http driver infrusturcture, why to enforce using it? So we can have at least 3 ways for supporting multiprotocols: - standard callbacks, completely different threads, no connection pooling (Exists now) - Ns_DriverSockQueue, new thread or callback thread, reusing connection pooling. All filters/traces are reused as well - driver Parse proc in the http driver to reuse drivers thread and connection pooling with all filters/traces All 3 methiods can co-exists and do not interfere with each other, that's my point, and all they add just couple extra functions. I would implement all methods, i have 3 drivers i need to run: smtp, dns, snmp, can add imap in the future ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 20:31 Message: Logged In: YES user_id=87254 "Each way has its own drawbacks..." That's what I'm asking. What are the drawbacks for each approach we've come up with so far? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 15:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 08:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 22:22:30
|
Feature Requests item #1151137, was opened at 2005-02-24 09:01 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 15:22 Message: Logged In: YES user_id=87254 You don't have to use registered filters or procs. It's an option for protocol implementations where the initial accept and parsing take place in the driver thread, and processing takes place in a conn thread. Once you submit to the queue however, either a filter, a registered proc, or a registered proxy proc must handle the request. It makes no difference whether you submit explicitly with Ns_DriverSockQueue or implicitly after your Ns_ParseProc completes. How much of the request structure you fill in is up to you, it's all optional. If you choose not to use it there is no overhead. It doesn't have to be a text based protocol for this to make sense. You might fill in the request structure if you wanted to enable people to handle different request types via C or Tcl. Or, you could store binary data in Ns_Cls storage and access it from registered procs etc. In the case where you want to handle connection accept/parse/reponse processing in the same thread, nothing needs to be added. You can do that today with Ns_SockCallback (or by placing all your code in an Ns_ParseProc and returning the response from there) Can you be more specific about the overhead of using an Ns_ParseProc? Exactly what memory is allocated, what code is run that should not..? What specifically can you not do with the Ns_ParseProc interface that you need the Ns_Driver* routines for? I'm looking for concrete examples. As far as I can tell, with Ns_ParseProc you write less code, get more options, and it takes advantage of infrastructure to give you more speed. I must be missing something, but you'll have to explain it to me in more detail because I don't get it. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 13:43 Message: Logged In: YES user_id=184124 Okay, the biggest problem with your way i see that i am enforced to go through http driver, even with my parse proc, i need to know how it works to implement my driver so correct hooks/filters will be called. If i want skip some parts i am not able to do it, http driver works only one way. Another issue is if something will not fit into current driver, additional hook needs to be introduced and core driver needs to be modified again. If my driver fits http-like/text mode paradigm, reusing http driver is the easiest way, but if i need something specific, all extra step to mimic the connection as http request, so all other parts of the http driver will not fail is just unnecesssary extra efforts. And i you mentioned, using Ns_DriverSockQueue is low level function and requires from the developer full attention for bulding the driver, but if this is what i want, why not. I can spawn my own thread if some callbacks are long running Tcl scripts and queue connection from my own loop. If i omly want to received packets, decode them and submit connection, i do not need all http driver infrusturcture, why to enforce using it? So we can have at least 3 ways for supporting multiprotocols: - standard callbacks, completely different threads, no connection pooling (Exists now) - Ns_DriverSockQueue, new thread or callback thread, reusing connection pooling. All filters/traces are reused as well - driver Parse proc in the http driver to reuse drivers thread and connection pooling with all filters/traces All 3 methiods can co-exists and do not interfere with each other, that's my point, and all they add just couple extra functions. I would implement all methods, i have 3 drivers i need to run: smtp, dns, snmp, can add imap in the future ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 13:31 Message: Logged In: YES user_id=87254 "Each way has its own drawbacks..." That's what I'm asking. What are the drawbacks for each approach we've come up with so far? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 08:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 01:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 14:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 20:43:01
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 20:43 Message: Logged In: YES user_id=184124 Okay, the biggest problem with your way i see that i am enforced to go through http driver, even with my parse proc, i need to know how it works to implement my driver so correct hooks/filters will be called. If i want skip some parts i am not able to do it, http driver works only one way. Another issue is if something will not fit into current driver, additional hook needs to be introduced and core driver needs to be modified again. If my driver fits http-like/text mode paradigm, reusing http driver is the easiest way, but if i need something specific, all extra step to mimic the connection as http request, so all other parts of the http driver will not fail is just unnecesssary extra efforts. And i you mentioned, using Ns_DriverSockQueue is low level function and requires from the developer full attention for bulding the driver, but if this is what i want, why not. I can spawn my own thread if some callbacks are long running Tcl scripts and queue connection from my own loop. If i omly want to received packets, decode them and submit connection, i do not need all http driver infrusturcture, why to enforce using it? So we can have at least 3 ways for supporting multiprotocols: - standard callbacks, completely different threads, no connection pooling (Exists now) - Ns_DriverSockQueue, new thread or callback thread, reusing connection pooling. All filters/traces are reused as well - driver Parse proc in the http driver to reuse drivers thread and connection pooling with all filters/traces All 3 methiods can co-exists and do not interfere with each other, that's my point, and all they add just couple extra functions. I would implement all methods, i have 3 drivers i need to run: smtp, dns, snmp, can add imap in the future ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 20:31 Message: Logged In: YES user_id=87254 "Each way has its own drawbacks..." That's what I'm asking. What are the drawbacks for each approach we've come up with so far? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 15:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 08:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 20:31:57
|
Feature Requests item #1151137, was opened at 2005-02-24 09:01 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 13:31 Message: Logged In: YES user_id=87254 "Each way has its own drawbacks..." That's what I'm asking. What are the drawbacks for each approach we've come up with so far? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 08:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 01:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 14:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 15:23:51
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 15:23 Message: Logged In: YES user_id=184124 I am not arguing with you about your ParseProc driver extension, it is very usefull and makes driver more flexible. I just want to add couple of new API functions that will allow developer to queue connections from any place, that's it. How drivers will be written and how developer will decide to handle it is up to each particular developer. But nobody will be locked up in only one way of doing it. Each way has its own drawbacks so we have multiple choices and developer will decide for itself how it should be. Both our ways do not mutually exclusive. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 08:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 08:28:40
|
Feature Requests item #1151137, was opened at 2005-02-24 09:01 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 01:28 Message: Logged In: YES user_id=87254 Ah I see you're right, an extra thread is not created for each incoming request. 'udpThread' is perhaps not the best name for the socket callback though... :-) I still don't see the advantage of using Ns_SockCalback. A single thread is created by the server to handle all registered callbacks, including those from ns_sockcallback. At runtime, while this single thread is running tcl code to handle one callback, your dirver callback cannot run, no new connections will be accepted, etc. Why is it better to use the Ns_SockCallback thread rather than a driver thread? Maybe I'm reading this wrong, but how do you handle the case where the request arrives as more than one packet? Ns_DriverSockRead() is called from the 'udpThread' socket callback. The only return value checked is NS_OK, but couldn't this also be SOCK_MORE? How would you handle things like keepalive? I think you'd have to reimplement that in the Ns_SockCallback thread. Re the proxy stuff, Ns_RegisterRequest() and Ns_RegisterProxyRequest() seem very simillar. With Ns_RegisterRequest(), filters are run and you get the choice of using C or Tcl. Ns_RegisterProxyRequest() offers no advantage that I can see -- you still need a complete Request structure, even if you just ignore it. The comm driver initialization should probably be changed to automatically handle unix domain sockets. You add /foo rather that 127.0.0.1 in the config file and it knows to create the correct type of socket. nssock and nsopenssl wouldn't have to be modified at all. UDP is different. There are two types of protocols: single packet, such as DNS or RADIUS; multi packet, as used in some streaming media and p2p protocols. I think for the single packet case we again might want to modify the driver code to automatically handle it. No read-ahead is necessary, there's only one packet per request, so it's placed in the request buffer and passed on to the next stage, which is parsing. Here, a return code of SOCK_MORE would be illegal. Every multi-packet UDP protocol will require custom framing/sequencing and the developer will have to create a new socket driver. Taking RADIUS as an example, which is a single packet UDP protocol, you'd create a very simple Ns_ParseProc who's only job is to check that the number of bytes specified in the header actually arrived, and return SOCK_READY. A default request structure is created for you so the only other thing you have to do is Ns_RegisterRequest() for the '/' URL. Within your request proc, call Ns_ConnContent() and parse the buffer. Now, you do have a number of other options to make this more flexible. You could parse the request in your Ns_ParseProc and then fill out the request structure. e.g. the different RADIUS message types could be expressed as HTTP verbs. This buys you flexibility. Now you can Ns_RegisterRequest() a different routine for each RADIUS message type, and someone reusing your code can override your default implementation. You can also handle some message types in C, and others in Tcl. You might decide to put some useful information about the RADIUS request in the URL line. Now you get logging for free. You might decide to parse the key/value pairs from the RADIUS request into query vars or headers. Now you don't have to write a bunch of RADIUS specific Tcl commands to access the data, should you want to handle that from Tcl. ns_queryget or ns_conn headers will work. How far you want to go is up to you, the writer of the protocol module. But at the low end all you need is an Ns_ParseProc and usually a single registered request proc, which is a lot less code to write than the Ns_Driver* stuff, I think. I'm sure it's not perfect! So what's wrong with it? Why would it suck to write a RADIUS protocol module as I've just described, to take one example? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 14:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 05:51:03
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:51 Message: Logged In: YES user_id=184124 I think we need to collect all solutions and then see where are going, i am holding to port all my old/running modules because i do not know how naviserver will handle foreign protocols. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 05:48:39
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:48 Message: Logged In: YES user_id=184124 yes, using proxy can solve immediate requirements without hacking NsConnProc by adding hooks to call driver specific C functions. If i need my smtp server and main loop is in C, i need somehow call it in the connection thread. Using registered proxy function i can do it now, i do not need filter/traces. This is for completely new modules. I can implement main loop in the module as Tcl command and then call it in the connection filter, it is possible, it will just require many different parts to be in place and still filter should be registered as Tcl proc which will call another driver main loop. sometimes low level stuff makes things easier and simpler:-))) ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 05:45:22
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:45 Message: Logged In: YES user_id=184124 In all my installations i need sha module but it wasn't in the distribution, so i need to download it or repackage aolserver to include nssha1. That is my point, it is just one simple function use more often than something like ns_jpegsize or so. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 05:42:02
|
Feature Requests item #1151137, was opened at 2005-02-24 09:01 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:42 Message: Logged In: YES user_id=87254 Well I don't know about the wisdom of adding sha1 to the core at this stage :-) I see your point though, encryption and hashing functions are almost universally required for systems/server work. Maybe we need to consider adding a new encryption module to the core distribution. Like nsdb it would export a C API via libnscypher.so (or whatever) as well as the Tcl module nscypher.so. Times have changed and things like the openssl libraries are common on all platforms so it's not the big deal it used to be to add such dependiencies. Such a module should make implementing SSL in nsopenssl easier. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 22:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 14:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 05:41:35
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-25 05:41 Message: Logged In: YES user_id=184124 No, i do not spawn new thread, i use callback feature of the server, when socket is ready, server calls provided callback ad that callback just submits the socket to connection pool. if pool is full, Ns_DriverSockQueue will return NS_TIMEOUT, you can retry. Yes, it is low-level, for high level, HTTP driver provides a lot of functionality, it could be extended like you did, but still it is HTTP driver hacked. If i want completely new driver, like RADIUS server, http driver will not help me, i need low level stuff, and it is there already. To reuse resource limiting, i added Ns_DriverSockQueue function, so new conections can be queue instead of creating new threads. I do not have anything against your patch for extending current driver, it is very usefull. i am offering new API for low level drivers. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-25 05:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 05:36:31
|
Feature Requests item #1151137, was opened at 2005-02-24 09:01 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:36 Message: Logged In: YES user_id=87254 The proxy stuff is too late in the cycle to do much. The request has to be fully parsed by then (for read-ahead). If a proxy function is available, then all filters, the auth phase, registered procs, cleanup procs etc. are bypassed. A lot of that stuff can be very useful for non-HTTP protocols. I don't think handling stuff via a proxy function buys you much. ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 14:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-25 05:32:32
|
Feature Requests item #1151137, was opened at 2005-02-24 09:01 Message generated for change (Comment added) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Stephen Deasey (sdeasey) Date: 2005-02-24 22:32 Message: Logged In: YES user_id=87254 I don't understand what you're trying to acheive here (well, apart from multi-protocol... :-). The newly exposed Ns_Driver* entry points are quite low level, and so the implementor of the new protocol is left to do a lot of the heavy lifting. For example, the way I read it you have to create your own listen socket and register a callback. Every time a new reaquest comes in a thread is spawned to handle it. From that thread you then submit the parsed request to one of the conn threads. Excessive thread creation and message passing between threads is not going to perform well. And it seems you have to write more code than e.g. the example POP3 driver I posted some time ago. You're also not taking advantage of the other facilities that the server offers. What happens if 1000 connections arrive, do you spawn 1000 threads? You could of course code up some limit checks, but this already exists. What if a client sends you a continuous stream of data, 2GB... etc. By using the driver hooks to provide the new protocol parser, you deny yourself the opportunity to use something like the nsopenssl module. This should work just fine for protocols like SMTP, IMAP, POP3 and probably others. Anyway, I think one of the most carefully coded aspects of the server is it's attention to resource usage. That goes for IO, context switching, memory, etc. It's espescialy nice that most of the time you're not even aware that all this work is being done for you. I'd like new protocol drivers to be able to transparently take advantage of that. Could you take a look at my old POP3 demo driver? It's the attachment nspopd-0.3.tar.bz2 over here: http://sourceforge.net/tracker/index.php?func=detail&aid=973010&group_id=3152&atid=353152 It's not obvious that anything interesting is going on, so it's not much to look at. But actually, conn socket read-ahead is happening eficiently in one thread with async IO, the conn threads are treated as a precious resource (heavy-weight Tcl interps) and are allocated at the last minute, there's an easy API in C and Tcl to implement the actual reading of data from the INBOX (could be from the file system, db etc.). You've got a lot of experience writing servers, what do you think is wrong with this model? What can it not do, or what could it do better? ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 14:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-24 21:29:11
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 21:29 Message: Logged In: YES user_id=184124 Another thing, once we can submit connections from any place, no need to build any drivers, even in C, i can register new proxy proc and set protocol field in my request, so when submitted, connection will run registered proxy proc. for example: in my smtp driver/module, i create driver, register proxy for smtp: protocol, register callback for the socket. Once connection accepted, in my module i submit that connection to the queue with request-protocl set to smtp:. queue.c will call my proxy handler, which is C function. No need to add anything else. This way even standard aolserver can be extended without touching precious http driver thread. Sorry for sarcasm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-24 16:07:05
|
Feature Requests item #1120167, was opened at 2005-02-10 17:13 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1120167&group_id=130646 Category: None Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: Cron like scheduler Initial Comment: I have working cron-like scheduer as Tcl command for other project and it makes real difference in doing background tasks. Current AS schedule_proc functionality is fine but not flexible, having it as unix cron would give real opportunities to run tasks from AS, not using UNIX cron in addition. I have code, it is very small and simple, i was surpised myself when i have it done. I am just not sure is it better be new command or extend ns_schedule_proc with additional switches. Currently, syntax is somethign like this: ns_cron script -id id -enable 0|1 -once 0|1 -thread 0|1 -min N -hour N -day N -mon N -year N -dow N -interval N -min|hour|mon|day|year|dow can be repeated. ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 16:07 Message: Logged In: YES user_id=184124 It is duplication of current schedule, no need to add at this time ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-11 14:12 Message: Logged In: YES user_id=184124 This is Tcl command, parsing isn't that hard assuming that everything comes as switch value pairs. It is C+++, but converting to use Tcl hases instead of queue is simple. ------------------------------------------------------------- case cmdSchedule: { if(objc < 3) { Tcl_WrongNumArgs(interp,2,objv,"script ?-id id? ?-min mins? ?-hour hrs? ?-day day? ?-mon mon? ?-year year? ?-dow dow? ?-interval secs? ?-thread 0|1? ?-once 0|1? ?-enable 0|1? ?-remove 1?"); return TCL_ERROR; } Schedule *sched = 0; for(int i = 3;i < objc - 1;i += 2) { if(!strcmp(Tcl_GetString(objv[i]),"-min")) { int v = atoi(Tcl_GetString(objv[i+1])); if(v < 0 || v >= 60) continue; if(!sched) sched = new Schedule(); sched->min[v] = 1; sched->min[sizeof(sched->min)-1] = 1; } else if(!strcmp(Tcl_GetString(objv[i]),"-hour")) { int v = atoi(Tcl_GetString(objv[i+1])); if(v < 0 ||v >= 24) continue; if(!sched) sched = new Schedule(); sched->hour[v] = 1; sched->hour[sizeof(sched->hour)-1] = 1; } else if(!strcmp(Tcl_GetString(objv[i]),"-day")) { int v = atoi(Tcl_GetString(objv[i+1])); if(v < 0 || v >= 32) continue; if(!sched) sched = new Schedule(); sched->mday[v] = 1; sched->mday[sizeof(sched->mday)-1] = 1; } else if(!strcmp(Tcl_GetString(objv[i]),"-mon")) { int v = atoi(Tcl_GetString(objv[i+1])); if(v < 1 || v > 12) continue; if(!sched) sched = new Schedule(); sched->mon[v-1] = 1; sched->mon[sizeof(sched->mon)-1] = 1; } else if(!strcmp(Tcl_GetString(objv[i]),"-year")) { int v = atoi(Tcl_GetString(objv[i+1])); if(v < 2005 || v >= 2015) continue; if(!sched) sched = new Schedule(); sched->year[v-2005] = 1; sched->year[sizeof(sched->year)-1] = 1; } else if(!strcmp(Tcl_GetString(objv[i]),"-dow")) { int v = atoi(Tcl_GetString(objv[i+1])); if(v < 0 || v >= 7) continue; if(!sched) sched = new Schedule(); sched->wday[v] = 1; sched->wday[sizeof(sched->wday)-1] = 1; } else if(!strcmp(Tcl_GetString(objv[i]),"-interval")) { if(!sched) sched = new Schedule(); sched->interval = atoi(Tcl_GetString(objv[i+1])); } else if(!strcmp(Tcl_GetString(objv[i]),"-thread")) { if(!sched) sched = new Schedule(); sched->thread_flag = atoi(Tcl_GetString(objv[i+1])); } else if(!strcmp(Tcl_GetString(objv[i]),"-once")) { if(!sched) sched = new Schedule(); sched->once_flag = atoi(Tcl_GetString(objv[i+1])); } else if(!strcmp(Tcl_GetString(objv[i]),"-enable")) { if(!sched) sched = new Schedule(); sched->enable_flag = atoi(Tcl_GetString(objv[i+1])); } else if(!strcmp(Tcl_GetString(objv[i]),"-remove")) { if(sched) delete sched; sched = 0; return TCL_OK; } else if(!strcmp(Tcl_GetString(objv[i]),"-id")) { if(sched) delete sched; int id = atoi(Tcl_GetString(objv[i+1])); vector<Schedule*>::iterator iter; pthread_mutex_lock(&schedmutex); for(iter = schedqueue.begin();iter != schedqueue.end();iter++) { sched = *iter; if(sched->id == id) { schedqueue.erase(iter); break; } } pthread_mutex_unlock(&schedmutex); if(iter == schedqueue.end()) { Tcl_AppendResult(interp,"schedule id not found",0); return TCL_ERROR; } } } if(!sched || sched->Setup()) { Tcl_AppendResult(interp,"unable to setup schedule, no date/time given",0); return TCL_ERROR; } sched->script = Tcl_GetString(objv[2]); pthread_mutex_lock(&schedmutex); schedqueue.push_back(sched); pthread_mutex_unlock(&schedmutex); Tcl_SetObjResult(interp,Tcl_NewIntObj(sched->id)); } ---------------------------------------------------------------------- Comment By: Stephen Deasey (sdeasey) Date: 2005-02-11 04:05 Message: Logged In: YES user_id=87254 That's a lot of switches to parse, you should check out my new Ns_ParseObjv (when I get it posted...) ! Hmm, not sure what the best api would be. It would be nice to have everything integrated with the sched stuff that's already there. The switch parsing for NsTclSchedCmd looks scary though :-) You've both already tackled this though, so looks like a good candidate for inclusion. Sounds good to me. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-10 17:27 Message: Logged In: YES user_id=184124 No, it is C module for use in Tcl, not pure Tcl implementation, i use it in C application with embedded Tcl for scripting. Once i get to my home and reincarnate dialup or bring it on laptop tomorrow, it sucks to be without Internet at home. ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-02-10 17:23 Message: Logged In: YES user_id=95086 All Tcl? Care to attach the implementation? I also had the need for this and did it myself i Tcl as well (hey, we're masters in reinventing the wheel!) I think the reason to implement this in core server as C-code was the idea to be able to use it from the C-API as well w/o being dependent on some Tcl code from outside. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1120167&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-24 16:05:56
|
Feature Requests item #1145957, was opened at 2005-02-22 03:13 Message generated for change (Comment added) made by seryakov You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1145957&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen Deasey (sdeasey) Assigned to: Stephen Deasey (sdeasey) Summary: Cookie support for C and Tcl Initial Comment: Atached is a patch which implements a simple cookie API for both C and Tcl. Here's what it looks like (the formating will probably get mangled, sorry): void Ns_ConnSetCookie(Ns_Conn *conn, char *name, char *value, int maxage); void Ns_ConnSetSecureCookie(Ns_Conn *conn, char *name, char *value, int maxage); void Ns_ConnSetCookieEx(Ns_Conn *conn, char *name, char *value, int maxage, char *domain, char *path, int secure); void Ns_ConnDeleteCookie(Ns_Conn *conn, char *name, char *domain, char *path); void Ns_ConnDeleteSecureCookie(Ns_Conn *conn, char *name, char *domain, char *path); char *Ns_ConnGetCookie(Ns_DString *dest, Ns_Conn *conn, char *name); ns_setcookie ?-secure bool? ?-domain d? ?-path p? ?-maxage secs? name data ns_getcookie name ?default? ns_deletecookie ?-secure bool? ?-domain d? ?-path p? name As you can see it only tackles the basics, no comment or version field. Although I've never personaly had to use those features. Cookies are a pre-requisite for the authentication module which exercises the new autht/authz API I haven't written yet... Obviously, the string parsing in the GetCookie code will have to be very carefully looked at before this sees the Internet. But in general, does this look like it's on the right track? ---------------------------------------------------------------------- >Comment By: Vlad Seryakov (seryakov) Date: 2005-02-24 16:05 Message: Logged In: YES user_id=184124 Looks good, the only issue i have, some browser can send more than one Cookie header and you are looking only in the first header. Also, this is new API, no problem adding it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1145957&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-24 16:01:19
|
Feature Requests item #1151137, was opened at 2005-02-24 16:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 Category: C-API Group: None Status: Open Resolution: None Priority: 5 Submitted By: Vlad Seryakov (seryakov) Assigned to: Vlad Seryakov (seryakov) Summary: New driver API and Udp module Initial Comment: Hi guys, Attached is minor driver extensions which do not change existing drivers but add new functionality. There are some cosmetic changes, like moving some fields in the Ns_Sock/Ns_Driver structres so they can be accessed publically and made some private functions public but functionality preserved as before. I included udp driver as an example of new API, and also added ns_sha1 command in the tclmisc.c, it is just one command and it is uses practically everywhere. ------------------------------------------------------------------------- To test udp driver i use new ns_udp command: ossweb:nscp 8> ns_udp send 127.0.0.1 5060 "GET / HTTP/1.0\n\n" HTTP/1.0 200 OK MIME-Version: 1.0 Date: Thu, 24 Feb 2005 05:39:50 GMT Server: NaviServer/4.0.10 Content-Type: text/html; charset=iso-8859-1 Content-Length: 661 Connection: close <HEAD><TITLE>Seryakov's Family Intranet</TITLE></HEAD> ..... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1151137&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-22 03:15:48
|
Feature Requests item #1119257, was opened at 2005-02-09 05:32 Message generated for change (Settings changed) made by sdeasey You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1119257&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zoran Vasiljevic (vasiljevic) >Assigned to: Stephen Deasey (sdeasey) Summary: Better interface to cache facility Initial Comment: Those commands are in the server itself: ns_cache_flush ns_cache_stats ns_cache_size ns_cache_names ns_cache_keys I would not touch them for the compatibility reasons. However, they are pretty limited (introspection/management). What is missing is type of functionality added by the nscache module which I believe should have be done long time ago already. Suggestion: Include nscache into the core Tcl commands. Stephen, you mentioned you've benn working on alternate cache implementation. Can we use this and add better Tcl interface? ---------------------------------------------------------------------- Comment By: Zoran Vasiljevic (vasiljevic) Date: 2005-02-09 10:25 Message: Logged In: YES user_id=95086 We can synthesize your changes and Stephens rewrite of the cache guts. I see no problem there. ---------------------------------------------------------------------- Comment By: Vlad Seryakov (seryakov) Date: 2005-02-09 10:20 Message: Logged In: YES user_id=184124 I added ns_cache incr command aslo some time ago, i think it is inthe CVS version of nscache. Also i used to play with core cache to add more prices size calculation incuding overhead. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1119257&group_id=130646 |
|
From: SourceForge.net <no...@so...> - 2005-02-22 03:13:21
|
Feature Requests item #1145957, was opened at 2005-02-21 20:13 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1145957&group_id=130646 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Stephen Deasey (sdeasey) Assigned to: Stephen Deasey (sdeasey) Summary: Cookie support for C and Tcl Initial Comment: Atached is a patch which implements a simple cookie API for both C and Tcl. Here's what it looks like (the formating will probably get mangled, sorry): void Ns_ConnSetCookie(Ns_Conn *conn, char *name, char *value, int maxage); void Ns_ConnSetSecureCookie(Ns_Conn *conn, char *name, char *value, int maxage); void Ns_ConnSetCookieEx(Ns_Conn *conn, char *name, char *value, int maxage, char *domain, char *path, int secure); void Ns_ConnDeleteCookie(Ns_Conn *conn, char *name, char *domain, char *path); void Ns_ConnDeleteSecureCookie(Ns_Conn *conn, char *name, char *domain, char *path); char *Ns_ConnGetCookie(Ns_DString *dest, Ns_Conn *conn, char *name); ns_setcookie ?-secure bool? ?-domain d? ?-path p? ?-maxage secs? name data ns_getcookie name ?default? ns_deletecookie ?-secure bool? ?-domain d? ?-path p? name As you can see it only tackles the basics, no comment or version field. Although I've never personaly had to use those features. Cookies are a pre-requisite for the authentication module which exercises the new autht/authz API I haven't written yet... Obviously, the string parsing in the GetCookie code will have to be very carefully looked at before this sees the Internet. But in general, does this look like it's on the right track? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1145957&group_id=130646 |
