tclhttpd-users — Discussion and general Q&A for TclHttpd

[Tclhttpd-users] http1.5 : bug at pkIndex.tcl : windows NT

[Moisei R. <mra...@il...>]

Hello, please find below bug report : (sorry for non-standard way to assign
it)

package require httpd 1.5 failed when there is a space in the package's
path (D:/program files/tcl/lib/tclhttpd3.3)

Bug reason :

not well formed path when :

from ...tcl/lib/tclhttpd3.3/lib/pkIndex.html

.......
package ifneeded httpd 1.5 "
     source [file join [list $dir] httpd.tcl]
.......


Bug fix :
replace
     source [file join [list $dir] httpd.tcl]
by
     source \"[file join $dir httpd.tcl]\"


Best Regards,
Moisei.

Re: [Tclhttpd-users] Tcl Conference coming up

[Donald G P. <dgp...@er...>]

"Larry W. Virden" wrote:
> Just a note for those of you who may not be aware, the 8th Tcl Conference
> is being held July 23-27 in San Diego, CA, as part of the O'Reilly Open Source
> Convention.  This represents a continuation of the Tcl Conference that USENIX
> has sponsored in the past.

It should also be noted that the deadline for "Early Bird" fees
(save up to $400) and conference rate hotel reservations is coming
soon -- June 22.  Even if you're a last-minute guy like me, time to
start your travel plans.

-- 
| Don Porter          Mathematical and Computational Sciences Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

[Tclhttpd-users] Tcl Conference coming up

[Larry W. V. <lv...@ca...>]

Just a note for those of you who may not be aware, the 8th Tcl Conference
is being held July 23-27 in San Diego, CA, as part of the O'Reilly Open Source
Convention.  This represents a continuation of the Tcl Conference that USENIX
has sponsored in the past.

If you have ever been to a Tcl conference in the past, you'll know it's
both fun and enlightening to share experiences with other Tcl enthusiasts.
All skill levels are welcome; the tutorials presented this year cover both
beginner and advanced levels.
Check out the full schedule at the web site:

    http://conferences.oreilly.com/oscon/
-- 
Never apply a Star Trek solution to a Babylon 5 problem.
Larry W. Virden <mailto:lv...@ca...> <URL: http://www.purl.org/NET/lvirden/>
Even if explicitly stated to the contrary, nothing in this posting should 
be construed as representing my employer's opinions.
-><-

RE: [Tclhttpd-users] Does Tcl WebServer support Server-side Control?

[Harmon K. <cra...@ho...>]

I do this type of thing by setting a form's target to a named frame in the 
currently displayed page. For example, if the screen is split by frames into 
a left and right side, the form in the left frame sends its output to the 
frame on the right. The CGI program's output is displayed as it arrives in 
the right frame. Not exactly server push, just turning the browser into a 
console window. You can still mark up the program's output with html to make 
a decent display.

Michael (Harmon)

>From: "Dan O'Brien" <dmo...@lu...>
>To: David LeBlanc <wh...@oz...>
>CC: Bhaven Bid <Bha...@in...>, tcl...@li...
>Subject: RE: [Tclhttpd-users] Does Tcl WebServer support Server-side 
>Control?
>Date: Thu, 31 May 2001 06:55:43 -0400 (EDT)
>
>My two cents...
>
>On Thu, 31 May 2001, David LeBlanc wrote:
>
> > comments interspersed.
> >
> > > We are looking at using the TCL Webserver (tclhttpd) to web-enable the
> > > application and we have a query regarding the same:
> > > Is there a way of controlling the display on a browser from the
> > > web-server side? I mean, as the test runs, and data is available for
> > > display, how can we dynamically update the same on the web-browser?
> > > Asking the browser to auto-refreshing the page every second or so is 
>not
> > > a clean solution. Is there some facility available in tclhttpd that
> > > allows the server to drive the browser (push data), rather than the
> > > browser asking for it (pull)?
> >
> > I don't believe (but am not sure) that TclHttpd supports server push (or 
>in
> > fact, anyone but Netscape servers). The alternative might be to either 
>run
> > javascript from the browser or install the tcl plug-in on the browser 
>and
> > use one of the rpc or soap type protocols to communicate.Using the Tcl
> > plug-in would give you the greatest flexibility in remoting an 
>application's
> > UI imho.
>
>This is a limitation of browser technology.  We re-engineered our
>application moving from Windows appliction to browser and lost async
>update in the process (we only use Dynamic HTML).
>
> > > I guess what we need is something like ASP (Active Server Pages) or 
>JSP
> > > (Java Server Pages). Does the Tcl Webserver have any facility to
> > > interface with either of these?
>
>These won't help.  Browser is a pull technology.  Netscape browser
>supports server-side push, but IE doesn't.
>
>If you really want this kind of dynamic update, you have to rely on Java
>Applets in the browser.  We wanted to avoid Applets for the weightiness
>(downloads over dial up are a killer), but will eventually be forced to
>use them for async display update.
>
>The applet can use all kinds of techniques to get data to display, even
>relying on HTTP calls (hiding the polling).  If you want true server side
>push, then the applet will have to open some kind of socket on the some
>server host application and then read data.
>
>Cheers,
>
>Dan O'Brien  <mailto:dmo...@lu...>
>Work: 614-860-2392  Home: 740-927-2178
>Cell: 614-783-4859
>----------------------------------------
>
>
>_______________________________________________
>TclHttpd-users mailing list
>Tcl...@li...
>http://lists.sourceforge.net/lists/listinfo/tclhttpd-users

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

[Tclhttpd-users] Warning: e-mail worm / virus

[Erik L. <e.l...@hc...>]

During the last 24 hours you, this mailing list or news group may have
received medium intelligable mail with attachments from "me" during the
last 24 hours.

Please discard, and under no circumstance open any attachments.

I have to inform you that a virus / e-mail worm has been using my e-mail
adress books to propagate itself. You, this mailing list or news group
were in one of the the adress books.

The worm has been identified as: W32/Magistr@MM.

More information about its nature can be found on:

   http://vil.mcafee.com/dispVirus.asp?virus_k=99040&


I regret having played a role in the transmssion,


Sincerely,

Erik Leunissen.

RE: [Tclhttpd-users] Does Tcl WebServer support Server-side Control?

[Dan O'B. <dmo...@lu...>]

My two cents...

On Thu, 31 May 2001, David LeBlanc wrote:

> comments interspersed.
> 
> > We are looking at using the TCL Webserver (tclhttpd) to web-enable the
> > application and we have a query regarding the same:
> > Is there a way of controlling the display on a browser from the
> > web-server side? I mean, as the test runs, and data is available for
> > display, how can we dynamically update the same on the web-browser?
> > Asking the browser to auto-refreshing the page every second or so is not
> > a clean solution. Is there some facility available in tclhttpd that
> > allows the server to drive the browser (push data), rather than the
> > browser asking for it (pull)?
> 
> I don't believe (but am not sure) that TclHttpd supports server push (or in
> fact, anyone but Netscape servers). The alternative might be to either run
> javascript from the browser or install the tcl plug-in on the browser and
> use one of the rpc or soap type protocols to communicate.Using the Tcl
> plug-in would give you the greatest flexibility in remoting an application's
> UI imho.

This is a limitation of browser technology.  We re-engineered our
application moving from Windows appliction to browser and lost async
update in the process (we only use Dynamic HTML).

> > I guess what we need is something like ASP (Active Server Pages) or JSP
> > (Java Server Pages). Does the Tcl Webserver have any facility to
> > interface with either of these?

These won't help.  Browser is a pull technology.  Netscape browser
supports server-side push, but IE doesn't.  

If you really want this kind of dynamic update, you have to rely on Java
Applets in the browser.  We wanted to avoid Applets for the weightiness
(downloads over dial up are a killer), but will eventually be forced to
use them for async display update.

The applet can use all kinds of techniques to get data to display, even
relying on HTTP calls (hiding the polling).  If you want true server side
push, then the applet will have to open some kind of socket on the some
server host application and then read data.

Cheers,

Dan O'Brien  <mailto:dmo...@lu...>
Work: 614-860-2392  Home: 740-927-2178
Cell: 614-783-4859  
----------------------------------------

RE: [Tclhttpd-users] Does Tcl WebServer support Server-side Control?

[David L. <wh...@oz...>]

comments interspersed.

> -----Original Message-----
> From: tcl...@li...
> [mailto:tcl...@li...]On Behalf Of Bhaven Bid
> Sent: Wednesday, May 30, 2001 10:26 PM
> To: tcl...@li...
> Subject: [Tclhttpd-users] Does Tcl WebServer support Server-side Control?
>
>
> Hi,
>
> We have a Tcl + Tk + Expect application, which we need to Web-Enable.
> The application is a automation test-tool, which runs some tests on some
> devices, and displays results in a text widget, as well as a Status Bar.
> The application may update the GUI/console at any time, based on
> information available as the test progresses.
>
> We are looking at using the TCL Webserver (tclhttpd) to web-enable the
> application and we have a query regarding the same:
> Is there a way of controlling the display on a browser from the
> web-server side? I mean, as the test runs, and data is available for
> display, how can we dynamically update the same on the web-browser?
> Asking the browser to auto-refreshing the page every second or so is not
> a clean solution. Is there some facility available in tclhttpd that
> allows the server to drive the browser (push data), rather than the
> browser asking for it (pull)?

I don't believe (but am not sure) that TclHttpd supports server push (or in
fact, anyone but Netscape servers). The alternative might be to either run
javascript from the browser or install the tcl plug-in on the browser and
use one of the rpc or soap type protocols to communicate.Using the Tcl
plug-in would give you the greatest flexibility in remoting an application's
UI imho.

> I guess what we need is something like ASP (Active Server Pages) or JSP
> (Java Server Pages). Does the Tcl Webserver have any facility to
> interface with either of these?

TclHttpd uses "Tcl Markup Language" (TML) in the same way as ASP or JSP.

> Other information: Presently, the application runs on Linux.
>
> Please let us know if you need more information.
>
> Regards,
> Bhaven.
>
> Bhaven Bid
> Programmer Analyst
> Infosys Technologies Limited
> Powered by intellect. Driven by values.
>
>
> _______________________________________________
> TclHttpd-users mailing list
> Tcl...@li...
> http://lists.sourceforge.net/lists/listinfo/tclhttpd-users
>

[Tclhttpd-users] Does Tcl WebServer support Server-side Control?

[Bhaven B. <Bha...@in...>]

Hi,

We have a Tcl + Tk + Expect application, which we need to Web-Enable.
The application is a automation test-tool, which runs some tests on some
devices, and displays results in a text widget, as well as a Status Bar.
The application may update the GUI/console at any time, based on
information available as the test progresses.

We are looking at using the TCL Webserver (tclhttpd) to web-enable the
application and we have a query regarding the same:
Is there a way of controlling the display on a browser from the
web-server side? I mean, as the test runs, and data is available for
display, how can we dynamically update the same on the web-browser?
Asking the browser to auto-refreshing the page every second or so is not
a clean solution. Is there some facility available in tclhttpd that
allows the server to drive the browser (push data), rather than the
browser asking for it (pull)?

I guess what we need is something like ASP (Active Server Pages) or JSP
(Java Server Pages). Does the Tcl Webserver have any facility to
interface with either of these?

Other information: Presently, the application runs on Linux.

Please let us know if you need more information.

Regards,
Bhaven.

Bhaven Bid
Programmer Analyst
Infosys Technologies Limited
Powered by intellect. Driven by values.

Re: [Tclhttpd-users] INFO: CGI Programming with Tclhttpd3.3

[Ralf W. <wp...@ep...>]

I agree with CGIs using CR-LF ( probably both ways ?).
 
To me, adding a "fconfigure $fd -translation auto crlf" at line 3 
in procedure CgiRead() would force the code to do the desired
trick as it then works in both worlds (UNIX and NT) detecting the
"empty" line. (and regardless wether the cgi really uses \r\n or just
\n)
As CgiCopy switches back to binary and everything else just somehow
shuts down the socket, there should be no additional side effects.   
 
Brent Welch wrote:
> 
> CGI programs use CR-LF, or network line ending conventions.
> I also wonder if the UNIX version of the script has the correct
> 
> #!/usr/local/tclsh8.3
> 
> initial line.  Or you can also try
> 
> #!/bin/sh
> # \
> exec /this/is/the/path/to/tclsh8.3
> 
> >>>Jacob Levy said:
>  > There are two possibilities:
>  >
>  > (a) The CGI protocol is defined in terms of "\n" delineated lines only. In that
>  > case the bug is (was) in the CGI program, and it should proactively set the
>  > output channel into lf mode.
>  >
>  > (b) The CGI protocol is defined in terms of platform specific end of line
>  > conventions -- whatever is the convention on each platform, that's what the
>  > server should expect and accept. In that case the bug is in tclHTTPd.
>  >
>  > Somehow I suspect (a) is the right choice between these two, but it *does* make
>  > it harder to use arbitrary platform specific programs as a CGI program...
>  >
>  > Brent, what's the right answer?
>  >
>  > --JYL
>  >
>  > Vloet Petrus wrote:
>  >
>  > > Dear Lady, Dear Sir,
>  > >
>  > > This mail provides information that might be helpfull to people who like to
>  > > do CGI Programming with tclhttpd3.3.
>  > >
>  > > The set up on which is based my experience was the following:
>  > >
>  > > Tclhttpd: 3.3
>  > > Tclsh: protclsh83
>  > > OS: Windows NT Service pack 5 & Solaris 2.6
>  > >
>  > > The CGI program is entirely written in tcl.
>  > >
>  > > On Tuesday I copied the CGI Program from Solaris to Windows NT in the hope
>  > > that it would be running in minutes.
>  > > To my surprise the only response on the web client was "Document contained
>  > > no data". I checked the CGI code once, twice,
>  > > implemented traces,  renamed the .tcl script to .cgi, but the executable in
>  > > the cgi-bin, etc, etc, but "Document contained no data" did not disappear.
>  > >
>  > > The next step was to debug tclhttpd with TclPro.
>  > >
>  > > In the CGI Module I learned that on windows files with the extensions  .tcl
>  > > and .cgi
>  > > are carried out by Tcl.
>  > >
>  > > At a certain point in the operation CgiSpawn calls CgiExec. This function
>  > > starts the CGI Program and returns a
>  > > filedescriptor by which the CGI-Module communicates with the CGI-Program. A
>  > > channel has been set up.
>  > >
>  > > The "POST" data from the Web Client is read by the CGI-Module and sent to
>  > > the CGI-Program, this all in binary mode.
>  > > At the end CgiSpawn calls CgiCopyDone with the aim to install a call back
>  > > function to be able to read data from the
>  > > channel. The installed call back function is CgiRead.
>  > >
>  > > A CGI Program can send Http Header Data (optionally) as well as Http User
>  > > Data
>  > > but has to separate the data by an "empty line", so CgiRead can distinguish
>  > > the data sets.
>  > >
>  > > CgiRead checks for an empty line in the following way:
>  > >
>  > > set n [ gets $fd line]
>  > > if { $n == 0 }   {
>  > >   #  End of Header reached
>  > >  ..
>  > >
>  > > and the channel, represented by $fd, is set up in binary mode. In this case
>  > > the end of line is just "line feed".
>  > >
>  > > The CGI Program started in the following way "open protclsh83 scriptname",
>  > > did not know anything about the
>  > > binary mode and sent the default end of line on Windows "carriage return /
>  > > line feed".
>  > >
>  > > What happened now was that CgiRead never detected the "empty line" as
>  > > separator between http header and http user data.
>  > > (gets on carriage return returns 1 and not 0). The user data became "part of
>  > > the header" and in the end the
>  > > web client returns "Document contained no data".
>  > >
>  > > Conclusion:
>  > >
>  > > Be sure that on Windows the CGI-Program sends only a line feed as end of
>  > > line.
>  > >
>  > > In tclsh you can "provocate" this by confguring the stdout channel in the
>  > > CGI Program in the following way
>  > > fconfigure stdout -configure binary
>  > >
>  > > This is the explanation to what Anders Nillson already reported in the
>  > > beginning of April
>  > >
>  > > _______________________________________________
>  > > TclHttpd-users mailing list
>  > > Tcl...@li...
>  > > http://lists.sourceforge.net/lists/listinfo/tclhttpd-users
>  >
>  >
>  > _______________________________________________
>  > TclHttpd-users mailing list
>  > Tcl...@li...
>  > http://lists.sourceforge.net/lists/listinfo/tclhttpd-users
> 
> --
> Brent Welch
> Software Architect, Panasas Inc
> Pioneering Object-Based Network Storage (ONS)
> 
> www.panasas.com
> we...@pa...
> 
> _______________________________________________
> TclHttpd-users mailing list
> Tcl...@li...
> http://lists.sourceforge.net/lists/listinfo/tclhttpd-users

Ralf Weishaupt 
                 
E-Mail: wp...@ep...                  EIGNER + PARTNER AG
Senior  Developer                      Ruschgraben 133
Manager Installation Environment       D-76139 Karlsruhe
Phone: +49 (721) 6291 - 0                 
Fax:     +49 (721) 6291 - 88           http://www.ep-ag.com
---------------------------------------------------------

Re: [Tclhttpd-users] INFO: CGI Programming with Tclhttpd3.3

[Brent W. <we...@pa...>]

CGI programs use CR-LF, or network line ending conventions.
I also wonder if the UNIX version of the script has the correct

#!/usr/local/tclsh8.3

initial line.  Or you can also try

#!/bin/sh
# \
exec /this/is/the/path/to/tclsh8.3

>>>Jacob Levy said:
 > There are two possibilities:
 > 
 > (a) The CGI protocol is defined in terms of "\n" delineated lines only. In that
 > case the bug is (was) in the CGI program, and it should proactively set the
 > output channel into lf mode.
 > 
 > (b) The CGI protocol is defined in terms of platform specific end of line
 > conventions -- whatever is the convention on each platform, that's what the
 > server should expect and accept. In that case the bug is in tclHTTPd.
 > 
 > Somehow I suspect (a) is the right choice between these two, but it *does* make
 > it harder to use arbitrary platform specific programs as a CGI program...
 > 
 > Brent, what's the right answer?
 > 
 > --JYL
 > 
 > Vloet Petrus wrote:
 > 
 > > Dear Lady, Dear Sir,
 > >
 > > This mail provides information that might be helpfull to people who like to
 > > do CGI Programming with tclhttpd3.3.
 > >
 > > The set up on which is based my experience was the following:
 > >
 > > Tclhttpd: 3.3
 > > Tclsh: protclsh83
 > > OS: Windows NT Service pack 5 & Solaris 2.6
 > >
 > > The CGI program is entirely written in tcl.
 > >
 > > On Tuesday I copied the CGI Program from Solaris to Windows NT in the hope
 > > that it would be running in minutes.
 > > To my surprise the only response on the web client was "Document contained
 > > no data". I checked the CGI code once, twice,
 > > implemented traces,  renamed the .tcl script to .cgi, but the executable in
 > > the cgi-bin, etc, etc, but "Document contained no data" did not disappear.
 > >
 > > The next step was to debug tclhttpd with TclPro.
 > >
 > > In the CGI Module I learned that on windows files with the extensions  .tcl
 > > and .cgi
 > > are carried out by Tcl.
 > >
 > > At a certain point in the operation CgiSpawn calls CgiExec. This function
 > > starts the CGI Program and returns a
 > > filedescriptor by which the CGI-Module communicates with the CGI-Program. A
 > > channel has been set up.
 > >
 > > The "POST" data from the Web Client is read by the CGI-Module and sent to
 > > the CGI-Program, this all in binary mode.
 > > At the end CgiSpawn calls CgiCopyDone with the aim to install a call back
 > > function to be able to read data from the
 > > channel. The installed call back function is CgiRead.
 > >
 > > A CGI Program can send Http Header Data (optionally) as well as Http User
 > > Data
 > > but has to separate the data by an "empty line", so CgiRead can distinguish
 > > the data sets.
 > >
 > > CgiRead checks for an empty line in the following way:
 > >
 > > set n [ gets $fd line]
 > > if { $n == 0 }   {
 > >   #  End of Header reached
 > >  ..
 > >
 > > and the channel, represented by $fd, is set up in binary mode. In this case
 > > the end of line is just "line feed".
 > >
 > > The CGI Program started in the following way "open protclsh83 scriptname",
 > > did not know anything about the
 > > binary mode and sent the default end of line on Windows "carriage return /
 > > line feed".
 > >
 > > What happened now was that CgiRead never detected the "empty line" as
 > > separator between http header and http user data.
 > > (gets on carriage return returns 1 and not 0). The user data became "part of
 > > the header" and in the end the
 > > web client returns "Document contained no data".
 > >
 > > Conclusion:
 > >
 > > Be sure that on Windows the CGI-Program sends only a line feed as end of
 > > line.
 > >
 > > In tclsh you can "provocate" this by confguring the stdout channel in the
 > > CGI Program in the following way
 > > fconfigure stdout -configure binary
 > >
 > > This is the explanation to what Anders Nillson already reported in the
 > > beginning of April
 > >
 > > _______________________________________________
 > > TclHttpd-users mailing list
 > > Tcl...@li...
 > > http://lists.sourceforge.net/lists/listinfo/tclhttpd-users
 > 
 > 
 > _______________________________________________
 > TclHttpd-users mailing list
 > Tcl...@li...
 > http://lists.sourceforge.net/lists/listinfo/tclhttpd-users

--
Brent Welch
Software Architect, Panasas Inc
Pioneering Object-Based Network Storage (ONS)

www.panasas.com
we...@pa...

Re: [Tclhttpd-users] INFO: CGI Programming with Tclhttpd3.3

[Jacob L. <jy...@be...>]

There are two possibilities:

(a) The CGI protocol is defined in terms of "\n" delineated lines only. In that
case the bug is (was) in the CGI program, and it should proactively set the
output channel into lf mode.

(b) The CGI protocol is defined in terms of platform specific end of line
conventions -- whatever is the convention on each platform, that's what the
server should expect and accept. In that case the bug is in tclHTTPd.

Somehow I suspect (a) is the right choice between these two, but it *does* make
it harder to use arbitrary platform specific programs as a CGI program...

Brent, what's the right answer?

--JYL

Vloet Petrus wrote:

> Dear Lady, Dear Sir,
>
> This mail provides information that might be helpfull to people who like to
> do CGI Programming with tclhttpd3.3.
>
> The set up on which is based my experience was the following:
>
> Tclhttpd: 3.3
> Tclsh: protclsh83
> OS: Windows NT Service pack 5 & Solaris 2.6
>
> The CGI program is entirely written in tcl.
>
> On Tuesday I copied the CGI Program from Solaris to Windows NT in the hope
> that it would be running in minutes.
> To my surprise the only response on the web client was "Document contained
> no data". I checked the CGI code once, twice,
> implemented traces,  renamed the .tcl script to .cgi, but the executable in
> the cgi-bin, etc, etc, but "Document contained no data" did not disappear.
>
> The next step was to debug tclhttpd with TclPro.
>
> In the CGI Module I learned that on windows files with the extensions  .tcl
> and .cgi
> are carried out by Tcl.
>
> At a certain point in the operation CgiSpawn calls CgiExec. This function
> starts the CGI Program and returns a
> filedescriptor by which the CGI-Module communicates with the CGI-Program. A
> channel has been set up.
>
> The "POST" data from the Web Client is read by the CGI-Module and sent to
> the CGI-Program, this all in binary mode.
> At the end CgiSpawn calls CgiCopyDone with the aim to install a call back
> function to be able to read data from the
> channel. The installed call back function is CgiRead.
>
> A CGI Program can send Http Header Data (optionally) as well as Http User
> Data
> but has to separate the data by an "empty line", so CgiRead can distinguish
> the data sets.
>
> CgiRead checks for an empty line in the following way:
>
> set n [ gets $fd line]
> if { $n == 0 }   {
>   #  End of Header reached
>  ..
>
> and the channel, represented by $fd, is set up in binary mode. In this case
> the end of line is just "line feed".
>
> The CGI Program started in the following way "open protclsh83 scriptname",
> did not know anything about the
> binary mode and sent the default end of line on Windows "carriage return /
> line feed".
>
> What happened now was that CgiRead never detected the "empty line" as
> separator between http header and http user data.
> (gets on carriage return returns 1 and not 0). The user data became "part of
> the header" and in the end the
> web client returns "Document contained no data".
>
> Conclusion:
>
> Be sure that on Windows the CGI-Program sends only a line feed as end of
> line.
>
> In tclsh you can "provocate" this by confguring the stdout channel in the
> CGI Program in the following way
> fconfigure stdout -configure binary
>
> This is the explanation to what Anders Nillson already reported in the
> beginning of April
>
> _______________________________________________
> TclHttpd-users mailing list
> Tcl...@li...
> http://lists.sourceforge.net/lists/listinfo/tclhttpd-users

[Tclhttpd-users] INFO: CGI Programming with Tclhttpd3.3

[Vloet P. <pet...@si...>]

Dear Lady, Dear Sir,

This mail provides information that might be helpfull to people who like to
do CGI Programming with tclhttpd3.3.

The set up on which is based my experience was the following:

Tclhttpd: 3.3
Tclsh: protclsh83
OS: Windows NT Service pack 5 & Solaris 2.6

The CGI program is entirely written in tcl. 

On Tuesday I copied the CGI Program from Solaris to Windows NT in the hope
that it would be running in minutes.
To my surprise the only response on the web client was "Document contained
no data". I checked the CGI code once, twice, 
implemented traces,  renamed the .tcl script to .cgi, but the executable in
the cgi-bin, etc, etc, but "Document contained no data" did not disappear.

The next step was to debug tclhttpd with TclPro.

In the CGI Module I learned that on windows files with the extensions  .tcl
and .cgi  
are carried out by Tcl.

At a certain point in the operation CgiSpawn calls CgiExec. This function
starts the CGI Program and returns a
filedescriptor by which the CGI-Module communicates with the CGI-Program. A
channel has been set up.

The "POST" data from the Web Client is read by the CGI-Module and sent to
the CGI-Program, this all in binary mode.
At the end CgiSpawn calls CgiCopyDone with the aim to install a call back
function to be able to read data from the 
channel. The installed call back function is CgiRead.

A CGI Program can send Http Header Data (optionally) as well as Http User
Data
but has to separate the data by an "empty line", so CgiRead can distinguish
the data sets.

CgiRead checks for an empty line in the following way: 

set n [ gets $fd line]
if { $n == 0 }   {
  #  End of Header reached
 ..

and the channel, represented by $fd, is set up in binary mode. In this case
the end of line is just "line feed".

The CGI Program started in the following way "open protclsh83 scriptname",
did not know anything about the 
binary mode and sent the default end of line on Windows "carriage return /
line feed". 

What happened now was that CgiRead never detected the "empty line" as
separator between http header and http user data.
(gets on carriage return returns 1 and not 0). The user data became "part of
the header" and in the end the
web client returns "Document contained no data".

Conclusion:

Be sure that on Windows the CGI-Program sends only a line feed as end of
line. 

In tclsh you can "provocate" this by confguring the stdout channel in the
CGI Program in the following way
fconfigure stdout -configure binary


This is the explanation to what Anders Nillson already reported in the
beginning of April

Re: [Tclhttpd-users] Re: Non-blocking subst?

[Brent W. <we...@pa...>]

I'm interested in the FreeBSD details.

>>>Don Porter said:
 > On 17 May, Brent Welch wrote:
 > > Right - that's what the threaded option in TclHttpd is all about.
 > 
 > Thanks everyone for all the followups that confirmed my expectations.
 > Template substitutions that make long running queries to a database
 > need to be handled with threads, or a specialized event-driven template
 > processor that would need to be written.
 > 
 > Looks like threads is the right answer for me then.  Two problems with
 > that:
 > 
 > 1) It takes away my excuse to play with the asynchronous PostgreSQL
 >    interface.  With threads, I don't think I have a reason to use it. :(
 > 
 > 2) It takes me back to problem number one.  I haven't been able to
 >    configure and make a --enable-threads Tcl on FreeBSD 4.3.  Anyone
 >    else had that trouble?  Anyone solved it?  More details on request,
 >    or perhaps I should take that issue to comp.lang.tcl.
 > 
 > -- 
 > | Don Porter                                       dgp...@er...  |  
 > |           "Some days you just can't get rid of a bomb!"              |
 > |                       -- Adam West as BATMAN                         | 
 > |______________________________________________________________________|
 > 

--
Brent Welch
Software Architect, Panasas Inc
Pioneering Object-Based Network Storage (ONS)

www.panasas.com
we...@pa...

Re: [Tclhttpd-users] Re: Non-blocking subst?

[Don P. <dgp...@er...>]

On 17 May, Brent Welch wrote:
> Right - that's what the threaded option in TclHttpd is all about.

Thanks everyone for all the followups that confirmed my expectations.
Template substitutions that make long running queries to a database
need to be handled with threads, or a specialized event-driven template
processor that would need to be written.

Looks like threads is the right answer for me then.  Two problems with
that:

1) It takes away my excuse to play with the asynchronous PostgreSQL
   interface.  With threads, I don't think I have a reason to use it. :(

2) It takes me back to problem number one.  I haven't been able to
   configure and make a --enable-threads Tcl on FreeBSD 4.3.  Anyone
   else had that trouble?  Anyone solved it?  More details on request,
   or perhaps I should take that issue to comp.lang.tcl.

-- 
| Don Porter                                       dgp...@er...  |  
|           "Some days you just can't get rid of a bomb!"              |
|                       -- Adam West as BATMAN                         | 
|______________________________________________________________________|

Re: [Tclhttpd-users] Re: Non-blocking subst?

[Brent W. <we...@pa...>]

Right - that's what the threaded option in TclHttpd is all about.

>>>Ted Dunning said:
 > 
 > 
 > Events won't work for what was originally required.  The issue is that in
 > the line
 > 
 >      ReturnToClient [subst $template]
 > 
 > if the return from subst is delayed by entering the event loop, additional
 > events that cause the same thing will wind up pushing additional stack
 > frames that cannot complete until THEIR version ReturnToClient is called.
 > This means that regardless of how TCL handles events, tclhttpd will still be
 > screwed and stack overflow is likely to result since the oldest events will
 > never be cleared until the newest event is completed and the newest event
 > can be blocked by an even newer event arriving.
 > 
 > The simplest (by far) way to do this is to build a thread that handles each
 > subst.  It needn't handle anything more if you want global state up to that
 > point although that entails moving everything that the template might
 > reference into the child thread.  If you want to avoid that complexity, then
 > you should just process the entire query in a thread.  If you need access to
 > structures in the main thread, you can always post events to the main thread
 > to ask for the pieces that you need.  In fact, if database handles can't be
 > shared, then this can be taken a step further by putting database access
 > stubs into the child thread that post scripts back to the database thread.
 > These can then be handled in an event-driven fashion and when the results
 > are available the original child thread can just call the threaded
 > ReturnToClient to send results back to the requestor.  This approach gives
 > you the best of event driven DB access along with the best aspects of
 > threads (i.e. independent stack and state).
 > 
 > 
 > >>>co...@fi... said:
 >  > I just had a quick look at the tcl implementation of events.
 >  > 
 >  > Events are processed in the order they occur on the event queue.  Quick 
 >  > perusal of the unix and generic uses of event queue shows that all events
 > to
 >       
 >  > be processed are placed at the end of the queue, and therefore will be 
 >  > serviced in order of arrival.
 >  > 
 >  > This implies, doesn't it, that all current events will be serviced before
 > an
 >      y 
 >  > new event, and therefore that no event will starve.
 >  > 
 >  > I think that it does what's required.
 >  > 
 >  > Colin.
 >  > 
 > 
 > _______________________________________________
 > TclHttpd-users mailing list
 > Tcl...@li...
 > http://lists.sourceforge.net/lists/listinfo/tclhttpd-users

--
Brent Welch
Software Architect, Panasas Inc
Pioneering Object-Based Network Storage (ONS)

www.panasas.com
we...@pa...

[Tclhttpd-users] Re: Non-blocking subst?

[Ted D. <tdu...@mu...>]

Events won't work for what was originally required.  The issue is that in
the line

     ReturnToClient [subst $template]

if the return from subst is delayed by entering the event loop, additional
events that cause the same thing will wind up pushing additional stack
frames that cannot complete until THEIR version ReturnToClient is called.
This means that regardless of how TCL handles events, tclhttpd will still be
screwed and stack overflow is likely to result since the oldest events will
never be cleared until the newest event is completed and the newest event
can be blocked by an even newer event arriving.

The simplest (by far) way to do this is to build a thread that handles each
subst.  It needn't handle anything more if you want global state up to that
point although that entails moving everything that the template might
reference into the child thread.  If you want to avoid that complexity, then
you should just process the entire query in a thread.  If you need access to
structures in the main thread, you can always post events to the main thread
to ask for the pieces that you need.  In fact, if database handles can't be
shared, then this can be taken a step further by putting database access
stubs into the child thread that post scripts back to the database thread.
These can then be handled in an event-driven fashion and when the results
are available the original child thread can just call the threaded
ReturnToClient to send results back to the requestor.  This approach gives
you the best of event driven DB access along with the best aspects of
threads (i.e. independent stack and state).


>>>co...@fi... said:
 > I just had a quick look at the tcl implementation of events.
 > 
 > Events are processed in the order they occur on the event queue.  Quick 
 > perusal of the unix and generic uses of event queue shows that all events
to
      
 > be processed are placed at the end of the queue, and therefore will be 
 > serviced in order of arrival.
 > 
 > This implies, doesn't it, that all current events will be serviced before
an
     y 
 > new event, and therefore that no event will starve.
 > 
 > I think that it does what's required.
 > 
 > Colin.
 > 

Re: [Tclhttpd-users] Non-blocking subst?

[Brent W. <we...@pa...>]

The Tcl event queue is careful not to involve starvation.
There are, however, some quirky uses of the event queue by
Tk to help do modal things, so there is actually somewhat
sophisticated control over where you can insert events.
But again, that is only exploited by the window system.

>>>co...@fi... said:
 > I just had a quick look at the tcl implementation of events.
 > 
 > Events are processed in the order they occur on the event queue.  Quick 
 > perusal of the unix and generic uses of event queue shows that all events to
      
 > be processed are placed at the end of the queue, and therefore will be 
 > serviced in order of arrival.
 > 
 > This implies, doesn't it, that all current events will be serviced before an
     y 
 > new event, and therefore that no event will starve.
 > 
 > I think that it does what's required.
 > 
 > Colin.
 > 
 > 
 > 
 > _______________________________________________
 > TclHttpd-users mailing list
 > Tcl...@li...
 > http://lists.sourceforge.net/lists/listinfo/tclhttpd-users

--
Brent Welch
Software Architect, Panasas Inc
Pioneering Object-Based Network Storage (ONS)

www.panasas.com
we...@pa...

Re: [Tclhttpd-users] Non-blocking subst?

[<co...@fi...>]

I just had a quick look at the tcl implementation of events.

Events are processed in the order they occur on the event queue.  Quick 
perusal of the unix and generic uses of event queue shows that all events to 
be processed are placed at the end of the queue, and therefore will be 
serviced in order of arrival.

This implies, doesn't it, that all current events will be serviced before any 
new event, and therefore that no event will starve.

I think that it does what's required.

Colin.

Re: [Tclhttpd-users] Non-blocking subst?

[Brent W. <we...@pa...>]

The only plausible way to get templates to work is to somehow split
the page processing into two phases.  The first uses the non-blocking
postgres interface, and the second uses a subst-style template processing
using procedures that presumably don't block.

Or, use threads :-)

>>>Colin McCormack said:
 > > 
 > > I'm working on some integration with tclhttpd and an asynchronous
 > > interface to PostgreSQL, as described at
 > > 
 > > http://dev.scriptics.com/software/tclhttpd/technotes.html
 > 
 > I've just put the libraries into cvs under http://sourceforge.net/projects/l
     ibt
 > clpq with some minor bugfixes which've arisen from use at $work.
 > 
 > I should get together a release sometime.
 > 
 > > So far the pieces look good, and I'm hopeful that I'll be able
 > > to handle multiple "concurrent" requests without having to resort
 > > to threads.
 > 
 > Great.  Yeah, there's no need for threads, everything's event-driven by the 
 > connection to postgresql.
 > 
 > > Trouble arises when I think about making use of templates, though.
 > > Essentially, use of templates boils down to
 > > 
 > > 	ReturnToClient [subst $template]
 > 
 > Good point.  Hadn't thought about that.
 > 
 > > However, this will block while [subst] is evaluating $template.
 > > And it expects the job to be done when [subst] completes.  This
 > > doesn't match with a callback style of an asynchronous interface.
 > 
 > I guess the original conception of subst was that it'd be mainly 
 > processor-bound, and not give many opportunities for background processing.
 > 
 > > I can present a blocking interface while continuing to process
 > > events inside the [subst] by use of a [vwait] in the
 > > template, but that leads down the treacherous path of nested
 > > [vwait]s and its LIFO processing implications.
 > 
 > Interesting that you run into what's essentially process scheduling.
 > 
 > You don't actually want LIFO, do you?  If a job is going to take a long time
      
 > to complete, while another which came in later is already completed, you don
     't 
 > want to make the second one wait until the first one's done, merely because 
     it 
 > arrived first.
 > 
 > Perhaps what you actually need to be concerned about is process starvation -
      
 > guaranteeing that all tasks are served eventually, and scheduling fairness -
      
 > guaranteeing that all tasks get roughly the same probability of gaining the 
 > CPU when they're all ready to run.
 > 
 > > Has anyone else confronted this issue?  How did you resolve
 > > it?  It's looking to me like the asynchronous interface to
 > > PostgreSQL, nice as it is, really can't be used with templates.
 > > I'd love to be wrong about that.
 > 
 > No, this is entirely a new consideration, to me.  Nested vwaits - perhaps wh
     at 
 > you need is an array element per connection which would contain the result o
     f 
 > the subst, by connection id - that'd help with the housekeeping.
 > 
 > It seems that to guarantee fairness, it would be sufficient to ensure that i
     f 
 > there are several events waiting to be processed at the same time, at the ti
     me 
 > a choice is being made as to which to event is serviced first, choosing one 
     at 
 > random would guarantee both fairness and lack of starvation.
 > 
 > Failing that, placing the ready-to-be-serviced events in a FIFO would be 
 > reasonably fair.
 > 
 > If we kick this around a bit, and decide on a good approach, I'll dig into t
     he 
 > tcl and see how best to implement, if you like.
 > 
 > Colin.
 > 
 > 
 > 
 > _______________________________________________
 > TclHttpd-users mailing list
 > Tcl...@li...
 > http://lists.sourceforge.net/lists/listinfo/tclhttpd-users

--
Brent Welch
Software Architect, Panasas Inc
Pioneering Object-Based Network Storage (ONS)

www.panasas.com
we...@pa...

Re: [Tclhttpd-users] Non-blocking subst?

[Colin M. <co...@fi...>]

> 
> I'm working on some integration with tclhttpd and an asynchronous
> interface to PostgreSQL, as described at
> 
> http://dev.scriptics.com/software/tclhttpd/technotes.html

I've just put the libraries into cvs under http://sourceforge.net/projects/libt
clpq with some minor bugfixes which've arisen from use at $work.

I should get together a release sometime.

> So far the pieces look good, and I'm hopeful that I'll be able
> to handle multiple "concurrent" requests without having to resort
> to threads.

Great.  Yeah, there's no need for threads, everything's event-driven by the 
connection to postgresql.

> Trouble arises when I think about making use of templates, though.
> Essentially, use of templates boils down to
> 
> 	ReturnToClient [subst $template]

Good point.  Hadn't thought about that.

> However, this will block while [subst] is evaluating $template.
> And it expects the job to be done when [subst] completes.  This
> doesn't match with a callback style of an asynchronous interface.

I guess the original conception of subst was that it'd be mainly 
processor-bound, and not give many opportunities for background processing.

> I can present a blocking interface while continuing to process
> events inside the [subst] by use of a [vwait] in the
> template, but that leads down the treacherous path of nested
> [vwait]s and its LIFO processing implications.

Interesting that you run into what's essentially process scheduling.

You don't actually want LIFO, do you?  If a job is going to take a long time 
to complete, while another which came in later is already completed, you don't 
want to make the second one wait until the first one's done, merely because it 
arrived first.

Perhaps what you actually need to be concerned about is process starvation - 
guaranteeing that all tasks are served eventually, and scheduling fairness - 
guaranteeing that all tasks get roughly the same probability of gaining the 
CPU when they're all ready to run.

> Has anyone else confronted this issue?  How did you resolve
> it?  It's looking to me like the asynchronous interface to
> PostgreSQL, nice as it is, really can't be used with templates.
> I'd love to be wrong about that.

No, this is entirely a new consideration, to me.  Nested vwaits - perhaps what 
you need is an array element per connection which would contain the result of 
the subst, by connection id - that'd help with the housekeeping.

It seems that to guarantee fairness, it would be sufficient to ensure that if 
there are several events waiting to be processed at the same time, at the time 
a choice is being made as to which to event is serviced first, choosing one at 
random would guarantee both fairness and lack of starvation.

Failing that, placing the ready-to-be-serviced events in a FIFO would be 
reasonably fair.

If we kick this around a bit, and decide on a good approach, I'll dig into the 
tcl and see how best to implement, if you like.

Colin.

[Tclhttpd-users] Non-blocking subst?

[Donald G P. <dgp...@er...>]

I'm working on some integration with tclhttpd and an asynchronous
interface to PostgreSQL, as described at

http://dev.scriptics.com/software/tclhttpd/technotes.html

So far the pieces look good, and I'm hopeful that I'll be able
to handle multiple "concurrent" requests without having to resort
to threads.

Trouble arises when I think about making use of templates, though.
Essentially, use of templates boils down to

	ReturnToClient [subst $template]

However, this will block while [subst] is evaluating $template.
And it expects the job to be done when [subst] completes.  This
doesn't match with a callback style of an asynchronous interface.

I can present a blocking interface while continuing to process
events inside the [subst] by use of a [vwait] in the
template, but that leads down the treacherous path of nested
[vwait]s and its LIFO processing implications.

Has anyone else confronted this issue?  How did you resolve
it?  It's looking to me like the asynchronous interface to
PostgreSQL, nice as it is, really can't be used with templates.
I'd love to be wrong about that.

-- 
| Don Porter          Mathematical and Computational Sciences Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

[Tclhttpd-users] Re: help from tclhttpd to aolserver

[Brent W. <we...@pa...>]

wg...@ge... said:
> Hi! I try to move my web server from tclhttpd to aolserver, since
> dev.scriptics.com also had this problem, so if you give me some advice
> is very usefully.  Please help me. Thank you very much!

I have some code that I used to migrate .tml files to AOLserver.
It's probably in a fairly raw state, however.  I'll attach a tar
file with a bunch of pieces you can sort through, OK?

tclhttpd_compat is a set of Tcl files I put in the AOLservers
Tcl module area.

acs-tcl is a set of Tcl files I put into the ArsDigita (ACS or OpenACS)
tcl directory.


--
Brent Welch
Software Architect, Panasas Inc
Pioneering Object-Based Network Storage (ONS)

www.panasas.com
we...@pa...

Re: [Tclhttpd-users] CGI POST mixed form data problems: additional linefeeds

[Brent W. <we...@pa...>]

Right, - this is fixed in the current release (3.3)
From the ChangeLog:

2001-01-31  Brent Welch <we...@aj...>

        * lib/upload.tcl: Added a file upload domain handler.
        Check out the definition of Upload_Url, which registers
        the domain, and UploadTest, which is the sample callback.
        * lib/cgi.tcl: Added binary translation to the pipe used
        to send POST data to a CGI script.

>>>Ralf Weishaupt said:
 > Hi,
 > using POST and mixed form data with tclhttpd 3.1.0 the called cgi
 > received additional ^M's ( corresponding the number of \n in the
 > original data.) Regardless on which system ( some UNIX/NT4/Win2000 ) the
 > server was running. As a result uploading files to the cgi was not
 > possible.
 > 
 > I tracked it down to a desired (?) side effect of the output channel
 > (stdin of the cgi) being configured to crlf translation.
 > 
 > To get rid of this I modified the POST block in CgisSpawn (cgi.tcl) to
 > look like this: 
 > 
 >     if {$data(proto) == "POST"} {
 > 
 >         # We are not using fcopy here so that the Httpd module
 >         # can keep track of how much post data is left.
 >         fconfigure $fd -translation {binary binary}
 >         set more 1
 >         set buffer ""
 >         while {$more > 0} {
 >             set more [Httpd_GetPostData $sock buffer 8192]
 >             puts -nonewline $fd $buffer
 >             set buffer {}
 >         }
 > 
 >         # Errors appear here because of the non-blocking writes.
 > 
 >         catch {flush $fd}
 >         fconfigure $fd -translation {binary crlf}
 >     }
 > 
 > By Adding lines 5 and 17 (fconfigure ...) in this block (switch to
 > binary and back to crlf) the additional ^Ms disappeared and the
 > uploading works fine now.
 > 
 > Wpt
 > 
 > _______________________________________________
 > TclHttpd-users mailing list
 > Tcl...@li...
 > http://lists.sourceforge.net/lists/listinfo/tclhttpd-users

--
Brent Welch
Software Architect, Panasas Inc
Pioneering Object-Based Network Storage (ONS)

www.panasas.com
we...@pa...

[Tclhttpd-users] CGI POST mixed form data problems: additional linefeeds

[Ralf W. <wp...@ep...>]

Hi,
using POST and mixed form data with tclhttpd 3.1.0 the called cgi
received additional ^M's ( corresponding the number of \n in the
original data.) Regardless on which system ( some UNIX/NT4/Win2000 ) the
server was running. As a result uploading files to the cgi was not
possible.

I tracked it down to a desired (?) side effect of the output channel
(stdin of the cgi) being configured to crlf translation.

To get rid of this I modified the POST block in CgisSpawn (cgi.tcl) to
look like this: 

    if {$data(proto) == "POST"} {

        # We are not using fcopy here so that the Httpd module
        # can keep track of how much post data is left.
        fconfigure $fd -translation {binary binary}
        set more 1
        set buffer ""
        while {$more > 0} {
            set more [Httpd_GetPostData $sock buffer 8192]
            puts -nonewline $fd $buffer
            set buffer {}
        }

        # Errors appear here because of the non-blocking writes.

        catch {flush $fd}
        fconfigure $fd -translation {binary crlf}
    }

By Adding lines 5 and 17 (fconfigure ...) in this block (switch to
binary and back to crlf) the additional ^Ms disappeared and the
uploading works fine now.

Wpt

Re: [Tclhttpd-users] Unexpected behaviour of Url_DecodeQuery

[Erik L. <e.l...@hc...>]

Thanks for your help,

Erik Leunissen.
======================

Brent Welch wrote:
> 
> format query should not take a list - try
> http::geturl $url -query [http::formatQuery key1 value1 key2 value2]
>