Thread: [Boa-devel] Dynamic Content
Brought to you by:
jnelson
From: <all...@li...> - 2003-02-03 06:00:18
|
Has anyone thought about a faster alternative to CGI for serving dynamic content with boa? I am aware of the general approach of embedding an interpreter in the web server, and there is of course fastcgi which is scary in a different way. Neither seem compatible to small is good philosophy. What would be? /Allan --=20 Allan Wind P.O. Box 2022 Woburn, MA 01888-0022 USA |
From: <la...@do...> - 2003-02-04 04:04:42
|
Allan - > Has anyone thought about a faster alternative to CGI for serving dynamic > content with boa? Yes. Oh, you want an answer, too? ;-) > I am aware of the general approach of embedding an > interpreter in the web server, and there is of course fastcgi which is > scary in a different way. Neither seem compatible to small is good > philosophy. What would be? IMHO, fastcgi merely postpones the problem. Instead of embedding an interpreter, you could instead simply "plug-in" the application program, with a dynamic loader. That latter possibility places complete control in the hands of the application to never stall. It might be better to keep control, in the form of a properly designed and debugged interpreter. Any candidate interpreter has to be able to perform direct (in-memory, not network transaction based) database queries, using something like gdbm. The output is obviously HTML. It's OK to have "source code" compiled down to "byte code" that the server "interprets". The interpreter has to keep "per-thread" state explicitly (not just in the Boa program counter), so when a buffer fills, Boa's main file-descriptor loop can take over. Like I said, I have thought about this off-and-on for years, and never put anything in. I have studied the available code base, e.g., php and other scripting languages, and never found anything that was programming-model compatible with Boa. Some things that come close, like php, are license-incompatible. I'm not scared to code something up, if I think it makes sense, could do real-world work, and people would use it. - Larry |
From: <all...@li...> - 2003-02-04 04:47:14
|
On 2003-02-03 20:04:07, la...@do... wrote: > Instead of embedding an interpreter, you could instead simply "plug-in" > the application program, with a dynamic loader. I was thinking along the same lines with a plugin "manager" to load and unload such modules. Stability, and maybe even scalability, may dictate one or more seperate such manager processes (similiar in philosophy to fastcgi or even postfix's Local Mail Transfer Protocol). Is there anything along these lines already, which then would reduce the readers exercise to simply connect the dots? CORBA (as in IIOP) might be a suitable transport, but it's big fat and possible heavy dependency to slap on. The application side of any interface has to be web server independent. /Allan --=20 Allan Wind P.O. Box 2022 Woburn, MA 01888-0022 USA |
From: Jon N. <jn...@ja...> - 2003-02-04 14:16:37
|
Another possibility that I've been playing with recently is SCGI: http://www.mems-exchange.org/software/scgi/ It's like FastCGI but easier to impliment. I also have laying around some code to do loadable module support, which worked well, but needed improvement. The loadable module support was able to hook into Boa at various times and stages: 1. When Boa starts up 2. When Boa shuts down 3. When a request is first accepted 4. When a request is shutdown 5. several stages during the reading and parsing of the request At the time, there were loadable modules for a. directory generation (pulled out of Boa proper) b. ssl c. virtualhosting (no long req'd.) d. some other stuff. I forget. ;-) -- Democracy is two wolves and a sheep voting on what to have for dinner. Liberty is two wolves attempting to have a sheep for dinner and finding a well-informed, well-armed sheep. Jon Nelson <jn...@ja...> C and Python Code Gardener |
From: <all...@li...> - 2003-02-04 15:43:07
|
On 2003-02-04 07:52:49, Jon Nelson wrote: > Another possibility that I've been playing with recently is SCGI: > http://www.mems-exchange.org/software/scgi/ I read the SCGI specification eariler, and found the pre-pending of string size a little odd, it leaves them with only \0 to seperate header field and value. Is it _that_ time consuming to parse http headers twice? Isn't the header names a fixed set (possible extended from HTTP to provide the information the environment does in CGI)? What about SOAP (which I know hardly anything about, btw)? /Allan --=20 Allan Wind P.O. Box 2022 Woburn, MA 01888-0022 USA |
From: Jon N. <jn...@ja...> - 2003-02-04 17:07:03
|
On Tue, 4 Feb 2003, Allan Wind wrote: > On 2003-02-04 07:52:49, Jon Nelson wrote: > > Another possibility that I've been playing with recently is SCGI: > > http://www.mems-exchange.org/software/scgi/ > > I read the SCGI specification eariler, and found the pre-pending of > string size a little odd, it leaves them with only \0 to seperate header > field and value. Is it _that_ time consuming to parse http headers > twice? Isn't the header names a fixed set (possible extended from > HTTP to provide the information the environment does in CGI)? What > about SOAP (which I know hardly anything about, btw)? SOAP is expensive. Basically SCGI uses the netstrings specification, which when boiled down means that every "string" is prefixed by its length, and \0 is the seperator (versus \n, \r\n, \r, tab, and others). Header names are not from a fixed set. There are some environment variables that should always be present, others that are usually present, and yet others that are purely optional. If you send to a CGI the header "foo: bar" the CGI should get HTTP_FOO=bar in its environment. All "unrecognized" header values get prefixed with HTTP_. However the mechanism, it's an unimportant detail really. It's far easier IMO that the FastCGI spec which is quite a bit more complex. Furthermore, if I read it properly, I don't think the httpd even needs to re-parse the output, making SCGI a "smarter" NPH. Another bonus of SCGI is that the httpd has its choice if reliable transports (tcp/ip or fifo/socket), thus the SCGI "server" could be on a different machine than the httpd, the bonus there being able to sustain far higher load and possibly even load balancing *right at the network layer*. -- Democracy is two wolves and a sheep voting on what to have for dinner. Liberty is two wolves attempting to have a sheep for dinner and finding a well-informed, well-armed sheep. Jon Nelson <jn...@ja...> C and Python Code Gardener |
From: <all...@li...> - 2003-02-04 18:22:29
|
On 2003-02-04 10:43:15, Jon Nelson wrote: > SOAP is expensive. That was my gut feeling as well. > Basically SCGI uses the netstrings specification, as in http://python.ca/nas/scgi/protocol.txt? > which when boiled down means that every "string" is prefixed by its > length, and \0 is the seperator (versus \n, \r\n, \r, tab, and others). Why is the header and body two different such strings? Opposed to one such string per header "line" and one for the body? > Header names are not from a fixed set. Ok. > There are some environment variables that should always be present, > others that are usually present, and yet others that are purely > optional. If you send to a CGI the header "foo: bar" the CGI should > get HTTP_FOO=3Dbar in its environment. All "unrecognized" header values > get prefixed with HTTP_. Ok, I was aware of the how that worked, but not if CGI just lagged behind time. > However the mechanism, it's an unimportant detail really. It's far > easier IMO that the FastCGI spec which is quite a bit more complex. > Furthermore, if I read it properly, I don't think the httpd even needs > to re-parse the output, making SCGI a "smarter" NPH. Lost me there. Unless the httpd already parses things as netstrings it would have to at least generate this special format, no? What about just passing the whole message through, possible decorated with a header containing meta information (ip, server root and maybe even an index to parse the message itself) so you end up doing something like: send(header(request)) send(message) where request is assumed to contain meta information. > Another bonus of SCGI is that the httpd has its choice if reliable > transports (tcp/ip or fifo/socket), thus the SCGI "server" could be on a > different machine than the httpd, the bonus there being able to sustain > far higher load and possibly even load balancing *right at the network > layer*. Yes, I am with totally with you there. /Allan --=20 Allan Wind P.O. Box 2022 Woburn, MA 01888-0022 USA |
From: Jon N. <jn...@ja...> - 2003-02-04 18:40:49
|
On Tue, 4 Feb 2003, Allan Wind wrote: > > However the mechanism, it's an unimportant detail really. It's far > > easier IMO that the FastCGI spec which is quite a bit more complex. > > Furthermore, if I read it properly, I don't think the httpd even needs > > to re-parse the output, making SCGI a "smarter" NPH. > > Lost me there. Unless the httpd already parses things as netstrings it > would have to at least generate this special format, no? What about > just passing the whole message through, possible decorated with a header > containing meta information (ip, server root and maybe even an index to > parse the message itself) so you end up doing something like: > > send(header(request)) > send(message) > > where request is assumed to contain meta information. The httpd *still* has to parse all of the headers, and convert it to *something*. Right now, they are simply added to an array (fixed size) of environment variables for the CGI. I wrote a quick proof-of-concept scgi.c file that took that information and converted it. However, if Boa *were* to support more than just CGI, then the logical thing to do is have an intermediate format that is much less expensive, probably a key/value struct, from which either format could be generated inexpensively. -- Democracy is two wolves and a sheep voting on what to have for dinner. Liberty is two wolves attempting to have a sheep for dinner and finding a well-informed, well-armed sheep. Jon Nelson <jn...@ja...> C and Python Code Gardener |
From: <all...@li...> - 2003-02-05 19:29:55
|
On 2003-02-04 12:17:02, Jon Nelson wrote: > The httpd *still* has to parse all of the headers, and convert it to > *something*. I understand, but I was really thinking a 2nd time on the client side (raw vs partially or fully digested messages). > However, if Boa *were* to support more than just CGI, then the logical > thing to do is have an intermediate format that is much less > expensive, probably a key/value struct, from which either format could > be generated inexpensively. Interesting, if cgi was just another module one might choose to rip it out in favor of a better mechanism (however you define "best"). /Allan --=20 Allan Wind P.O. Box 2022 Woburn, MA 01888-0022 USA |