naviserver-tracker — SourceForge Tracker Notifications

[ naviserver-Feature Requests-1159307 ] Get rid of nsext/nspd

[SourceForge.net <no...@so...>]

Feature Requests item #1159307, was opened at 2005-03-08 19:41
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1159307&group_id=130646

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: Get rid of nsext/nspd

Initial Comment:
Last time i used those modules was when i used sybase
driver, actually those things were created for sybase
drivers only. Except that, all other DB modules uses
native access and even for sybase FreeTSD exists with
working module(i use it for SQL Server access).

No point keeping this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1159307&group_id=130646

[ naviserver-Feature Requests-1156107 ] Scrap optional connid parameter from Tcl API

[SourceForge.net <no...@so...>]

Feature Requests item #1156107, was opened at 2005-03-03 20:04
Message generated for change (Comment added) made by vasiljevic
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156107&group_id=130646

Category: Tcl-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Scrap optional connid parameter from Tcl API

Initial Comment:
Some Tcl-API commands still accept optional connection-id
parameter which is left as the compatibility trace from
older
2.x nsd servers. This complicates argument processing and
serves no other obvious purpose.
The idea is to scrap this optional connid parsing from the
Tcl-API calls making them simpler to document and
implement.

I never used any code running for the 2.x servers hence 
for me it is OK to remove those. 

Any thoughts on that?


----------------------------------------------------------------------

>Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-08 19:34

Message:
Logged In: YES 
user_id=95086

I will do this on as-encountered basis. If others are about to
rewrite some command that still parses this ugly argument,
I suggest to remove it entirely. Furthermore, we should 
gradually start to use Stephens new tclobjv arg parsing.
Stephen, you mind checkin in some example command
rewrite so we can all see and use this as a guidance while
rewriting or implementing new parsers? 

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 09:38

Message:
Logged In: YES 
user_id=95086

ns_conn  
ns_register_filter 
ns_return (some of them) 
 
They all accept a weird syntax: 
 
  <proc> ?connid? args 
 
which is PITA to parse and maintain. Yet, it is not used any 
more... It is only there to be able to run 2.x server-side Tcl 
code  

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 23:08

Message:
Logged In: YES 
user_id=184124

Which commands are you talking about?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156107&group_id=130646

[ naviserver-Feature Requests-1159259 ] Add nscache module functionality

[SourceForge.net <no...@so...>]

Feature Requests item #1159259, was opened at 2005-03-08 19:30
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1159259&group_id=130646

Category: Tcl-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Stephen Deasey (sdeasey)
Summary: Add nscache module functionality

Initial Comment:
This is really what needs to be done sine the server itself
does support the caches and what lacks is the Tcl access
to them. The nscache module (or its equivalent) deserves
to be part of the server distribution. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1159259&group_id=130646

[ naviserver-Feature Requests-1156875 ] Add server watchdog process

[SourceForge.net <no...@so...>]

Feature Requests item #1156875, was opened at 2005-03-04 20:03
Message generated for change (Comment added) made by vasiljevic
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add server watchdog process

Initial Comment:
We have been using this for quite some time and it 
proved extremely useful. 
We doublefork the nsd process and make the first 
forked instance control the second. The first one 
(the watchdog) reacts on exit codes and signals 
caught during the watch and correspondingly restarts 
the second instance (the worker). 
 
Also, we have added the the "-restart" option to 
the "ns_shutdown" command. This just sends the 
SIGINT to the worker process. The watchdog is 
handling this signal and respawns the worker 
automatically. 
 
During operation, the watchdog logs events and their 
cause  into the system log file. This looks like: 
Feb 28 04:00:05 Develop nsd[19400]: worker: started. 
Mar  1 04:00:13 Develop nsd[4475]: watchdog: worker 
19400 exited (2). 
Mar  1 04:00:15 Develop nsd[21290]: worker: started. 
Mar  1 04:00:18 Develop nsd[14705]: watchdog: worker 
19399 exited (2). 
Mar  1 04:00:20 Develop nsd[21300]: worker: started. 
 
We have done all the changes with "--enable-watchdog" 
so anybody who needs this feature will have to compile 
with this option.  
 

----------------------------------------------------------------------

>Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-08 17:19

Message:
Logged In: YES 
user_id=95086

small glich: 
After applying the patch, change nsmain.c 
from: 
    nsconf.pid = serverPid: 
to 
    nsconf.pid = getpid(); 
 
otherwise the pid file will contain the bogus 
server pid if the watchdog restarted it later. 
   

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-08 16:26

Message:
Logged In: YES 
user_id=95086

Another try... 
Important to note: SIGKILL (signal 9) cannot be handled 
hence if somebody kills the watchdog with SIGKILL, the 
server will be left lingering w/o the watchdog. This is  
important to know. I do not see any possibility how to 
recover in such cases (i.e. how to stop the server). 
 
Apart from that, all objections from Stephen are taken 
into account. Please try again. A new copy of watchdog.patch 
file is attached. 

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-08 06:20

Message:
Logged In: YES 
user_id=184124

I just found this code in my old server sources, i just
chnaged internal name to Ns/NS, used to run pretty stable.

-------------------------------------------------------------------

#define NS_EXIT 99

static void NsTerminate(int sig)
{
    printf("NS[%d] signal %d received...",getpid(),sig);
    // Kill the server with the same signal
    if(nsPid > 0) kill(nsPid,sig);

    // Exit in case of fatal signal
    if(sig != SIGHUP) {
      printf("NS[%d] terminating...",getpid());
      exit(0);
    }
    // Reassign signal handler
    signal(sig,NsTerminate);
}

void NsWatchdog(int argc, char *argv[], char *envp[])
{
    int failcount = 0;
    time_t start;
    int status;
    pid_t pid;

    signal(SIGTTOU,SIG_IGN);
    signal(SIGTTIN,SIG_IGN);
    signal(SIGTSTP,SIG_IGN);
    signal(SIGPIPE,SIG_IGN);
    signal(SIGQUIT,SIG_IGN);
    signal(SIGHUP,NsTerminate);
    signal(SIGINT,NsTerminate);
    signal(SIGTERM,NsTerminate);

    // Go background
    if((pid = fork())) {
      if(pid < 0) err_logger("warpConfigure: fork:
%s",strerror(errno));
       exit(0);
    }
    setsid();
       
    for(;;) {
               
        // Execute the real server
        nsPid = fork();
        // Child, continue as server
        if(nsPid == 0) {
          exit(nsMain(argc, argv, ServerInit));
        }
        
        /* parent, behaves like a guardian */
        time(&start);
        printf("NS[%d] server process started",getpid());
        pid = waitpid(-1, &status, 0);
        if(WIFEXITED(status))
          printf("NS[%d] child process exited with status
%d",pid,WEXITSTATUS(status));
        else
        if(WIFSIGNALED(status))
          printf("NS[%d] child process exited due to signal
%d",pid,WTERMSIG(status));
        else
          printf("NS[%d] child process exited", pid);

        // Special exit code
        if(WIFEXITED(status)) {
          if(WEXITSTATUS(status) == NS_EXIT) {
            printf("NS[%d] child configuration error,
exiting",getpid());
            exit(0);
          } else
          if(WEXITSTATUS(status) == SIGHUP) {
          } else {
            if(time(0) - start < 10) failcount++;
            if(failcount == 10) {
              printf("Exiting due to repeated, frequent
failures");
              exit(1);
            }
          }
        }
        sleep(3);
    }
}


----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-08 05:23

Message:
Logged In: YES 
user_id=87254


NsWatchdog() is called after the server drops root privs, so
both the watchdog and the server run as the defined user. 
What happens if the server dies and on restart needs to
rebind privileged ports?

A ps listing shows two running processes, parent and child.
  If I kill either one, the watchdog dies, the server
continues to run.
  If I kill -9 the parent, the child continues to run.
  If I kill -9 the child, the server is restarted.
Something seems not quite right here...

I'm a bit confused about how the code works.  For example,
NsWatchdog() seems to ignore all of it's arguments?  Here's
the code which calls it:

if (mode == 0) {
        i = ns_fork();
        if (i < 0) {
            Ns_Fatal("nsmain: fork() failed: '%s'",
strerror(errno));
        }
        if (i > 0) {
            return 0;
        }
        setsid();
        i = NsWatchdog(argc, argv, initProc);
        if (i < 0) {
            Ns_Fatal("nsmain: watchdog failed: '%s'",
strerror(errno));
        }
        if (i > 0) {
            return 0;
        }
        nsconf.pid = getpid();
    }

NsWatchdog() says it returns the worker pid, it also sets
the global variable wpid.  The code above ignores the
returned value, and the global variable, and instead calls
getpid()...  The variable 'i' is existing code, but still
somehow doesn't seem suitable...

Could that Ns_Fatal() above be moved into the NsWatchdog()
function?  I think maybe some comment is needed here.  The
code structure is very like that just above where ns_fork()
is called, but this function will return *multiple* times,
right?  This is kind of suprising, and an extra twist on the
already confusing fork() semantics (or maybe it's just me
who gets confused by fork...).  A return value of 0 here is
a 'request for orderly shutdown', right?

How about some more logging to syslog?  For example,
distinguish between start and restart.  Mention when the
MAX_SLEEP_PERIOD has been reached, etc.


Couple of small things:

Can we refer to 'the server', rather than 'the worker'? 
Worker and Watchdog begin with 'w', and so does the global
variable wpid...  Maybe serverPid?

NsWatchdog is a static function, it doesn't need the Ns prefix.

In NsWatchdog(), the variable 'run' should be something like
'nretries', 'nap' should be something more like
'retrySeconds' and MAX_SLEEP_PERIOD should be MAX_RETRY_SECONDS.

The comment for WaitForWorker() is misslabeled.

Should SigHandler() be called WatchdogSigHandler()?






----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-07 00:03

Message:
Logged In: YES 
user_id=184124

Looks good to me

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 13:06

Message:
Logged In: YES 
user_id=95086

Ah, correction: 
The restart option sends SIGINT to the worker process 
which causes the watchdog to restart it.  
And, the patchfile is now attached! 

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 13:03

Message:
Logged In: YES 
user_id=95086

Here is the patch. 
I have added "-restart" option to "ns_shutdown". It is rather
clumsy to parse but should do. We should rewrite this with
your args parsing routine. 
The restart option sends SIGTERM to the worker process
which causes the watchdog to restart it. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-05 00:46

Message:
Logged In: YES 
user_id=87254


I don't think this has to hide behind a config option.  It's
either a good idea or it's not.  Sounds good to me.  Is
there a patch?  I'm wondering about some of the
implementation details...



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

[ naviserver-Feature Requests-1156875 ] Add server watchdog process

[SourceForge.net <no...@so...>]

Feature Requests item #1156875, was opened at 2005-03-04 20:03
Message generated for change (Comment added) made by vasiljevic
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add server watchdog process

Initial Comment:
We have been using this for quite some time and it 
proved extremely useful. 
We doublefork the nsd process and make the first 
forked instance control the second. The first one 
(the watchdog) reacts on exit codes and signals 
caught during the watch and correspondingly restarts 
the second instance (the worker). 
 
Also, we have added the the "-restart" option to 
the "ns_shutdown" command. This just sends the 
SIGINT to the worker process. The watchdog is 
handling this signal and respawns the worker 
automatically. 
 
During operation, the watchdog logs events and their 
cause  into the system log file. This looks like: 
Feb 28 04:00:05 Develop nsd[19400]: worker: started. 
Mar  1 04:00:13 Develop nsd[4475]: watchdog: worker 
19400 exited (2). 
Mar  1 04:00:15 Develop nsd[21290]: worker: started. 
Mar  1 04:00:18 Develop nsd[14705]: watchdog: worker 
19399 exited (2). 
Mar  1 04:00:20 Develop nsd[21300]: worker: started. 
 
We have done all the changes with "--enable-watchdog" 
so anybody who needs this feature will have to compile 
with this option.  
 

----------------------------------------------------------------------

>Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-08 16:26

Message:
Logged In: YES 
user_id=95086

Another try... 
Important to note: SIGKILL (signal 9) cannot be handled 
hence if somebody kills the watchdog with SIGKILL, the 
server will be left lingering w/o the watchdog. This is  
important to know. I do not see any possibility how to 
recover in such cases (i.e. how to stop the server). 
 
Apart from that, all objections from Stephen are taken 
into account. Please try again. A new copy of watchdog.patch 
file is attached. 

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-08 06:20

Message:
Logged In: YES 
user_id=184124

I just found this code in my old server sources, i just
chnaged internal name to Ns/NS, used to run pretty stable.

-------------------------------------------------------------------

#define NS_EXIT 99

static void NsTerminate(int sig)
{
    printf("NS[%d] signal %d received...",getpid(),sig);
    // Kill the server with the same signal
    if(nsPid > 0) kill(nsPid,sig);

    // Exit in case of fatal signal
    if(sig != SIGHUP) {
      printf("NS[%d] terminating...",getpid());
      exit(0);
    }
    // Reassign signal handler
    signal(sig,NsTerminate);
}

void NsWatchdog(int argc, char *argv[], char *envp[])
{
    int failcount = 0;
    time_t start;
    int status;
    pid_t pid;

    signal(SIGTTOU,SIG_IGN);
    signal(SIGTTIN,SIG_IGN);
    signal(SIGTSTP,SIG_IGN);
    signal(SIGPIPE,SIG_IGN);
    signal(SIGQUIT,SIG_IGN);
    signal(SIGHUP,NsTerminate);
    signal(SIGINT,NsTerminate);
    signal(SIGTERM,NsTerminate);

    // Go background
    if((pid = fork())) {
      if(pid < 0) err_logger("warpConfigure: fork:
%s",strerror(errno));
       exit(0);
    }
    setsid();
       
    for(;;) {
               
        // Execute the real server
        nsPid = fork();
        // Child, continue as server
        if(nsPid == 0) {
          exit(nsMain(argc, argv, ServerInit));
        }
        
        /* parent, behaves like a guardian */
        time(&start);
        printf("NS[%d] server process started",getpid());
        pid = waitpid(-1, &status, 0);
        if(WIFEXITED(status))
          printf("NS[%d] child process exited with status
%d",pid,WEXITSTATUS(status));
        else
        if(WIFSIGNALED(status))
          printf("NS[%d] child process exited due to signal
%d",pid,WTERMSIG(status));
        else
          printf("NS[%d] child process exited", pid);

        // Special exit code
        if(WIFEXITED(status)) {
          if(WEXITSTATUS(status) == NS_EXIT) {
            printf("NS[%d] child configuration error,
exiting",getpid());
            exit(0);
          } else
          if(WEXITSTATUS(status) == SIGHUP) {
          } else {
            if(time(0) - start < 10) failcount++;
            if(failcount == 10) {
              printf("Exiting due to repeated, frequent
failures");
              exit(1);
            }
          }
        }
        sleep(3);
    }
}


----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-08 05:23

Message:
Logged In: YES 
user_id=87254


NsWatchdog() is called after the server drops root privs, so
both the watchdog and the server run as the defined user. 
What happens if the server dies and on restart needs to
rebind privileged ports?

A ps listing shows two running processes, parent and child.
  If I kill either one, the watchdog dies, the server
continues to run.
  If I kill -9 the parent, the child continues to run.
  If I kill -9 the child, the server is restarted.
Something seems not quite right here...

I'm a bit confused about how the code works.  For example,
NsWatchdog() seems to ignore all of it's arguments?  Here's
the code which calls it:

if (mode == 0) {
        i = ns_fork();
        if (i < 0) {
            Ns_Fatal("nsmain: fork() failed: '%s'",
strerror(errno));
        }
        if (i > 0) {
            return 0;
        }
        setsid();
        i = NsWatchdog(argc, argv, initProc);
        if (i < 0) {
            Ns_Fatal("nsmain: watchdog failed: '%s'",
strerror(errno));
        }
        if (i > 0) {
            return 0;
        }
        nsconf.pid = getpid();
    }

NsWatchdog() says it returns the worker pid, it also sets
the global variable wpid.  The code above ignores the
returned value, and the global variable, and instead calls
getpid()...  The variable 'i' is existing code, but still
somehow doesn't seem suitable...

Could that Ns_Fatal() above be moved into the NsWatchdog()
function?  I think maybe some comment is needed here.  The
code structure is very like that just above where ns_fork()
is called, but this function will return *multiple* times,
right?  This is kind of suprising, and an extra twist on the
already confusing fork() semantics (or maybe it's just me
who gets confused by fork...).  A return value of 0 here is
a 'request for orderly shutdown', right?

How about some more logging to syslog?  For example,
distinguish between start and restart.  Mention when the
MAX_SLEEP_PERIOD has been reached, etc.


Couple of small things:

Can we refer to 'the server', rather than 'the worker'? 
Worker and Watchdog begin with 'w', and so does the global
variable wpid...  Maybe serverPid?

NsWatchdog is a static function, it doesn't need the Ns prefix.

In NsWatchdog(), the variable 'run' should be something like
'nretries', 'nap' should be something more like
'retrySeconds' and MAX_SLEEP_PERIOD should be MAX_RETRY_SECONDS.

The comment for WaitForWorker() is misslabeled.

Should SigHandler() be called WatchdogSigHandler()?






----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-07 00:03

Message:
Logged In: YES 
user_id=184124

Looks good to me

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 13:06

Message:
Logged In: YES 
user_id=95086

Ah, correction: 
The restart option sends SIGINT to the worker process 
which causes the watchdog to restart it.  
And, the patchfile is now attached! 

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 13:03

Message:
Logged In: YES 
user_id=95086

Here is the patch. 
I have added "-restart" option to "ns_shutdown". It is rather
clumsy to parse but should do. We should rewrite this with
your args parsing routine. 
The restart option sends SIGTERM to the worker process
which causes the watchdog to restart it. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-05 00:46

Message:
Logged In: YES 
user_id=87254


I don't think this has to hide behind a config option.  It's
either a good idea or it's not.  Sounds good to me.  Is
there a patch?  I'm wondering about some of the
implementation details...



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

[ naviserver-Feature Requests-1156239 ] nslog restructure

[SourceForge.net <no...@so...>]

Feature Requests item #1156239, was opened at 2005-03-03 22:04
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646

Category: C-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: nslog restructure

Initial Comment:
Post submit for nslog restructure, no functionality
changed really, just added new commands and options to
control nslog on the fly. The source code changed to be
more flexible of extending nslog functionality with
more options.

The only possible bug fix is checking for unknown in
the X-Forwarded-For header.

Sorry for ignoring/forgetting the procedure.

----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-08 14:39

Message:
Logged In: YES 
user_id=184124

Another approach is to have another lock for accessing
extheaders only. Extra calls for every log write but it wlll
be more safely.

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-08 05:04

Message:
Logged In: YES 
user_id=184124

The chance is very small, i did not want to put locks around
extendedheaders in the LogTrace routine, setting
extendedheaders will not happen very often(if happen at all)
and assignment of any pointer is one cpu operation.

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-08 04:54

Message:
Logged In: YES 
user_id=87254


This worries me:

Ns_MutexLock(&logPtr->lock);
/* Wait for others to finish logging extheaders */
sleep(1);
logPtr->extheaders = h1;
 Ns_MutexUnlock(&logPtr->lock);

Is 1 second enough?  Does the server crash if it's not?



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 22:55

Message:
Logged In: YES 
user_id=184124

Syntax has been changed according to NS coding style

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646

[ naviserver-Bugs-1158382 ] fastpath virtual hosting location

[SourceForge.net <no...@so...>]

Bugs item #1158382, was opened at 2005-03-07 16:20
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1158382&group_id=130646

Category: Tcl-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: fastpath virtual hosting location

Initial Comment:
I use vhost module (something like that should be
included into Ns as well, but that's for RFE) and when
using fastpath directory listing, the proc returns
driver's location which is always the same.

fastpath.tcl :152 

    set hidedot [nsv_get _ns_fastpath hidedot]
    set location [ns_conn location]

    # For virtual hosts we should use Host: header and
not driver's location
    if { [set host [ns_set iget [ns_conn headers]
host]] != "" } { 
      # Extract protocol from the location, is there
any other way?
      set location [lindex [split $location :] 0]://$host
    }


----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-08 14:08

Message:
Logged In: YES 
user_id=184124

No, this is just quick fix for fastpath.tcl, location is
still driver's problem. This is particular for browsing
server's directry with fastpath, nothing else.
I also think virtual hosting in AS 4 is
broken/not-completed, that's why i use vhost module

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-08 06:48

Message:
Logged In: YES 
user_id=87254


Ns_ConnLocation() is still called to make the notice at the
bottom of the page.  I think in any virtual hosting
situation, Ns_ConnLocation() needs to be fixed.

I think the signature of Ns_ConnLocation() is broken.  It
returns a pointer to static storage, but for virtual hosts
you really need to be able to build the location string on
the fly.  So you'd really like it to append to a dstring so
that the caller can free the memory.

I created an interface like this and made the location proc
pluggable: Ns_SetLocationProc(), also allowing Tcl callbacks
to be registered.  Looking at it now though I think it's
over the top.  I'm leaning more towards the idea that it
would be simpler and less code to just teach the core server
how to do mass virtual hosting.

So, this doesn't look like the right solution to me.  Do
your virtual hosting patches fix Ns_ConnLocation()?  Can we
just fix it correctly the first time?



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1158382&group_id=130646

[ naviserver-Bugs-1158382 ] fastpath virtual hosting location

[SourceForge.net <no...@so...>]

Bugs item #1158382, was opened at 2005-03-07 09:20
Message generated for change (Comment added) made by sdeasey
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1158382&group_id=130646

Category: Tcl-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: fastpath virtual hosting location

Initial Comment:
I use vhost module (something like that should be
included into Ns as well, but that's for RFE) and when
using fastpath directory listing, the proc returns
driver's location which is always the same.

fastpath.tcl :152 

    set hidedot [nsv_get _ns_fastpath hidedot]
    set location [ns_conn location]

    # For virtual hosts we should use Host: header and
not driver's location
    if { [set host [ns_set iget [ns_conn headers]
host]] != "" } { 
      # Extract protocol from the location, is there
any other way?
      set location [lindex [split $location :] 0]://$host
    }


----------------------------------------------------------------------

>Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-07 23:48

Message:
Logged In: YES 
user_id=87254


Ns_ConnLocation() is still called to make the notice at the
bottom of the page.  I think in any virtual hosting
situation, Ns_ConnLocation() needs to be fixed.

I think the signature of Ns_ConnLocation() is broken.  It
returns a pointer to static storage, but for virtual hosts
you really need to be able to build the location string on
the fly.  So you'd really like it to append to a dstring so
that the caller can free the memory.

I created an interface like this and made the location proc
pluggable: Ns_SetLocationProc(), also allowing Tcl callbacks
to be registered.  Looking at it now though I think it's
over the top.  I'm leaning more towards the idea that it
would be simpler and less code to just teach the core server
how to do mass virtual hosting.

So, this doesn't look like the right solution to me.  Do
your virtual hosting patches fix Ns_ConnLocation()?  Can we
just fix it correctly the first time?



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1158382&group_id=130646

[ naviserver-Feature Requests-1156875 ] Add server watchdog process

[SourceForge.net <no...@so...>]

Feature Requests item #1156875, was opened at 2005-03-04 19:03
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add server watchdog process

Initial Comment:
We have been using this for quite some time and it 
proved extremely useful. 
We doublefork the nsd process and make the first 
forked instance control the second. The first one 
(the watchdog) reacts on exit codes and signals 
caught during the watch and correspondingly restarts 
the second instance (the worker). 
 
Also, we have added the the "-restart" option to 
the "ns_shutdown" command. This just sends the 
SIGINT to the worker process. The watchdog is 
handling this signal and respawns the worker 
automatically. 
 
During operation, the watchdog logs events and their 
cause  into the system log file. This looks like: 
Feb 28 04:00:05 Develop nsd[19400]: worker: started. 
Mar  1 04:00:13 Develop nsd[4475]: watchdog: worker 
19400 exited (2). 
Mar  1 04:00:15 Develop nsd[21290]: worker: started. 
Mar  1 04:00:18 Develop nsd[14705]: watchdog: worker 
19399 exited (2). 
Mar  1 04:00:20 Develop nsd[21300]: worker: started. 
 
We have done all the changes with "--enable-watchdog" 
so anybody who needs this feature will have to compile 
with this option.  
 

----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-08 05:20

Message:
Logged In: YES 
user_id=184124

I just found this code in my old server sources, i just
chnaged internal name to Ns/NS, used to run pretty stable.

-------------------------------------------------------------------

#define NS_EXIT 99

static void NsTerminate(int sig)
{
    printf("NS[%d] signal %d received...",getpid(),sig);
    // Kill the server with the same signal
    if(nsPid > 0) kill(nsPid,sig);

    // Exit in case of fatal signal
    if(sig != SIGHUP) {
      printf("NS[%d] terminating...",getpid());
      exit(0);
    }
    // Reassign signal handler
    signal(sig,NsTerminate);
}

void NsWatchdog(int argc, char *argv[], char *envp[])
{
    int failcount = 0;
    time_t start;
    int status;
    pid_t pid;

    signal(SIGTTOU,SIG_IGN);
    signal(SIGTTIN,SIG_IGN);
    signal(SIGTSTP,SIG_IGN);
    signal(SIGPIPE,SIG_IGN);
    signal(SIGQUIT,SIG_IGN);
    signal(SIGHUP,NsTerminate);
    signal(SIGINT,NsTerminate);
    signal(SIGTERM,NsTerminate);

    // Go background
    if((pid = fork())) {
      if(pid < 0) err_logger("warpConfigure: fork:
%s",strerror(errno));
       exit(0);
    }
    setsid();
       
    for(;;) {
               
        // Execute the real server
        nsPid = fork();
        // Child, continue as server
        if(nsPid == 0) {
          exit(nsMain(argc, argv, ServerInit));
        }
        
        /* parent, behaves like a guardian */
        time(&start);
        printf("NS[%d] server process started",getpid());
        pid = waitpid(-1, &status, 0);
        if(WIFEXITED(status))
          printf("NS[%d] child process exited with status
%d",pid,WEXITSTATUS(status));
        else
        if(WIFSIGNALED(status))
          printf("NS[%d] child process exited due to signal
%d",pid,WTERMSIG(status));
        else
          printf("NS[%d] child process exited", pid);

        // Special exit code
        if(WIFEXITED(status)) {
          if(WEXITSTATUS(status) == NS_EXIT) {
            printf("NS[%d] child configuration error,
exiting",getpid());
            exit(0);
          } else
          if(WEXITSTATUS(status) == SIGHUP) {
          } else {
            if(time(0) - start < 10) failcount++;
            if(failcount == 10) {
              printf("Exiting due to repeated, frequent
failures");
              exit(1);
            }
          }
        }
        sleep(3);
    }
}


----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-08 04:23

Message:
Logged In: YES 
user_id=87254


NsWatchdog() is called after the server drops root privs, so
both the watchdog and the server run as the defined user. 
What happens if the server dies and on restart needs to
rebind privileged ports?

A ps listing shows two running processes, parent and child.
  If I kill either one, the watchdog dies, the server
continues to run.
  If I kill -9 the parent, the child continues to run.
  If I kill -9 the child, the server is restarted.
Something seems not quite right here...

I'm a bit confused about how the code works.  For example,
NsWatchdog() seems to ignore all of it's arguments?  Here's
the code which calls it:

if (mode == 0) {
        i = ns_fork();
        if (i < 0) {
            Ns_Fatal("nsmain: fork() failed: '%s'",
strerror(errno));
        }
        if (i > 0) {
            return 0;
        }
        setsid();
        i = NsWatchdog(argc, argv, initProc);
        if (i < 0) {
            Ns_Fatal("nsmain: watchdog failed: '%s'",
strerror(errno));
        }
        if (i > 0) {
            return 0;
        }
        nsconf.pid = getpid();
    }

NsWatchdog() says it returns the worker pid, it also sets
the global variable wpid.  The code above ignores the
returned value, and the global variable, and instead calls
getpid()...  The variable 'i' is existing code, but still
somehow doesn't seem suitable...

Could that Ns_Fatal() above be moved into the NsWatchdog()
function?  I think maybe some comment is needed here.  The
code structure is very like that just above where ns_fork()
is called, but this function will return *multiple* times,
right?  This is kind of suprising, and an extra twist on the
already confusing fork() semantics (or maybe it's just me
who gets confused by fork...).  A return value of 0 here is
a 'request for orderly shutdown', right?

How about some more logging to syslog?  For example,
distinguish between start and restart.  Mention when the
MAX_SLEEP_PERIOD has been reached, etc.


Couple of small things:

Can we refer to 'the server', rather than 'the worker'? 
Worker and Watchdog begin with 'w', and so does the global
variable wpid...  Maybe serverPid?

NsWatchdog is a static function, it doesn't need the Ns prefix.

In NsWatchdog(), the variable 'run' should be something like
'nretries', 'nap' should be something more like
'retrySeconds' and MAX_SLEEP_PERIOD should be MAX_RETRY_SECONDS.

The comment for WaitForWorker() is misslabeled.

Should SigHandler() be called WatchdogSigHandler()?






----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 23:03

Message:
Logged In: YES 
user_id=184124

Looks good to me

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 12:06

Message:
Logged In: YES 
user_id=95086

Ah, correction: 
The restart option sends SIGINT to the worker process 
which causes the watchdog to restart it.  
And, the patchfile is now attached! 

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 12:03

Message:
Logged In: YES 
user_id=95086

Here is the patch. 
I have added "-restart" option to "ns_shutdown". It is rather
clumsy to parse but should do. We should rewrite this with
your args parsing routine. 
The restart option sends SIGTERM to the worker process
which causes the watchdog to restart it. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-04 23:46

Message:
Logged In: YES 
user_id=87254


I don't think this has to hide behind a config option.  It's
either a good idea or it's not.  Sounds good to me.  Is
there a patch?  I'm wondering about some of the
implementation details...



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

[ naviserver-Feature Requests-1156239 ] nslog restructure

[SourceForge.net <no...@so...>]

Feature Requests item #1156239, was opened at 2005-03-03 22:04
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646

Category: C-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: nslog restructure

Initial Comment:
Post submit for nslog restructure, no functionality
changed really, just added new commands and options to
control nslog on the fly. The source code changed to be
more flexible of extending nslog functionality with
more options.

The only possible bug fix is checking for unknown in
the X-Forwarded-For header.

Sorry for ignoring/forgetting the procedure.

----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-08 05:04

Message:
Logged In: YES 
user_id=184124

The chance is very small, i did not want to put locks around
extendedheaders in the LogTrace routine, setting
extendedheaders will not happen very often(if happen at all)
and assignment of any pointer is one cpu operation.

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-08 04:54

Message:
Logged In: YES 
user_id=87254


This worries me:

Ns_MutexLock(&logPtr->lock);
/* Wait for others to finish logging extheaders */
sleep(1);
logPtr->extheaders = h1;
 Ns_MutexUnlock(&logPtr->lock);

Is 1 second enough?  Does the server crash if it's not?



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 22:55

Message:
Logged In: YES 
user_id=184124

Syntax has been changed according to NS coding style

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646

[ naviserver-Feature Requests-1156239 ] nslog restructure

[SourceForge.net <no...@so...>]

Feature Requests item #1156239, was opened at 2005-03-03 15:04
Message generated for change (Comment added) made by sdeasey
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646

Category: C-API
Group: None
>Status: Open
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: nslog restructure

Initial Comment:
Post submit for nslog restructure, no functionality
changed really, just added new commands and options to
control nslog on the fly. The source code changed to be
more flexible of extending nslog functionality with
more options.

The only possible bug fix is checking for unknown in
the X-Forwarded-For header.

Sorry for ignoring/forgetting the procedure.

----------------------------------------------------------------------

>Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-07 21:54

Message:
Logged In: YES 
user_id=87254


This worries me:

Ns_MutexLock(&logPtr->lock);
/* Wait for others to finish logging extheaders */
sleep(1);
logPtr->extheaders = h1;
 Ns_MutexUnlock(&logPtr->lock);

Is 1 second enough?  Does the server crash if it's not?



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 15:55

Message:
Logged In: YES 
user_id=184124

Syntax has been changed according to NS coding style

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646

[ naviserver-Feature Requests-1156875 ] Add server watchdog process

[SourceForge.net <no...@so...>]

Feature Requests item #1156875, was opened at 2005-03-04 12:03
Message generated for change (Comment added) made by sdeasey
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add server watchdog process

Initial Comment:
We have been using this for quite some time and it 
proved extremely useful. 
We doublefork the nsd process and make the first 
forked instance control the second. The first one 
(the watchdog) reacts on exit codes and signals 
caught during the watch and correspondingly restarts 
the second instance (the worker). 
 
Also, we have added the the "-restart" option to 
the "ns_shutdown" command. This just sends the 
SIGINT to the worker process. The watchdog is 
handling this signal and respawns the worker 
automatically. 
 
During operation, the watchdog logs events and their 
cause  into the system log file. This looks like: 
Feb 28 04:00:05 Develop nsd[19400]: worker: started. 
Mar  1 04:00:13 Develop nsd[4475]: watchdog: worker 
19400 exited (2). 
Mar  1 04:00:15 Develop nsd[21290]: worker: started. 
Mar  1 04:00:18 Develop nsd[14705]: watchdog: worker 
19399 exited (2). 
Mar  1 04:00:20 Develop nsd[21300]: worker: started. 
 
We have done all the changes with "--enable-watchdog" 
so anybody who needs this feature will have to compile 
with this option.  
 

----------------------------------------------------------------------

>Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-07 21:23

Message:
Logged In: YES 
user_id=87254


NsWatchdog() is called after the server drops root privs, so
both the watchdog and the server run as the defined user. 
What happens if the server dies and on restart needs to
rebind privileged ports?

A ps listing shows two running processes, parent and child.
  If I kill either one, the watchdog dies, the server
continues to run.
  If I kill -9 the parent, the child continues to run.
  If I kill -9 the child, the server is restarted.
Something seems not quite right here...

I'm a bit confused about how the code works.  For example,
NsWatchdog() seems to ignore all of it's arguments?  Here's
the code which calls it:

if (mode == 0) {
        i = ns_fork();
        if (i < 0) {
            Ns_Fatal("nsmain: fork() failed: '%s'",
strerror(errno));
        }
        if (i > 0) {
            return 0;
        }
        setsid();
        i = NsWatchdog(argc, argv, initProc);
        if (i < 0) {
            Ns_Fatal("nsmain: watchdog failed: '%s'",
strerror(errno));
        }
        if (i > 0) {
            return 0;
        }
        nsconf.pid = getpid();
    }

NsWatchdog() says it returns the worker pid, it also sets
the global variable wpid.  The code above ignores the
returned value, and the global variable, and instead calls
getpid()...  The variable 'i' is existing code, but still
somehow doesn't seem suitable...

Could that Ns_Fatal() above be moved into the NsWatchdog()
function?  I think maybe some comment is needed here.  The
code structure is very like that just above where ns_fork()
is called, but this function will return *multiple* times,
right?  This is kind of suprising, and an extra twist on the
already confusing fork() semantics (or maybe it's just me
who gets confused by fork...).  A return value of 0 here is
a 'request for orderly shutdown', right?

How about some more logging to syslog?  For example,
distinguish between start and restart.  Mention when the
MAX_SLEEP_PERIOD has been reached, etc.


Couple of small things:

Can we refer to 'the server', rather than 'the worker'? 
Worker and Watchdog begin with 'w', and so does the global
variable wpid...  Maybe serverPid?

NsWatchdog is a static function, it doesn't need the Ns prefix.

In NsWatchdog(), the variable 'run' should be something like
'nretries', 'nap' should be something more like
'retrySeconds' and MAX_SLEEP_PERIOD should be MAX_RETRY_SECONDS.

The comment for WaitForWorker() is misslabeled.

Should SigHandler() be called WatchdogSigHandler()?






----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 16:03

Message:
Logged In: YES 
user_id=184124

Looks good to me

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 05:06

Message:
Logged In: YES 
user_id=95086

Ah, correction: 
The restart option sends SIGINT to the worker process 
which causes the watchdog to restart it.  
And, the patchfile is now attached! 

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 05:03

Message:
Logged In: YES 
user_id=95086

Here is the patch. 
I have added "-restart" option to "ns_shutdown". It is rather
clumsy to parse but should do. We should rewrite this with
your args parsing routine. 
The restart option sends SIGTERM to the worker process
which causes the watchdog to restart it. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-04 16:46

Message:
Logged In: YES 
user_id=87254


I don't think this has to hide behind a config option.  It's
either a good idea or it's not.  Sounds good to me.  Is
there a patch?  I'm wondering about some of the
implementation details...



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

[ naviserver-Bugs-1158382 ] fastpath virtual hosting location

[SourceForge.net <no...@so...>]

Bugs item #1158382, was opened at 2005-03-07 16:20
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1158382&group_id=130646

Category: Tcl-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: fastpath virtual hosting location

Initial Comment:
I use vhost module (something like that should be
included into Ns as well, but that's for RFE) and when
using fastpath directory listing, the proc returns
driver's location which is always the same.

fastpath.tcl :152 

    set hidedot [nsv_get _ns_fastpath hidedot]
    set location [ns_conn location]

    # For virtual hosts we should use Host: header and
not driver's location
    if { [set host [ns_set iget [ns_conn headers]
host]] != "" } { 
      # Extract protocol from the location, is there
any other way?
      set location [lindex [split $location :] 0]://$host
    }


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719006&aid=1158382&group_id=130646

[ naviserver-Feature Requests-1156875 ] Add server watchdog process

[SourceForge.net <no...@so...>]

Feature Requests item #1156875, was opened at 2005-03-04 19:03
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add server watchdog process

Initial Comment:
We have been using this for quite some time and it 
proved extremely useful. 
We doublefork the nsd process and make the first 
forked instance control the second. The first one 
(the watchdog) reacts on exit codes and signals 
caught during the watch and correspondingly restarts 
the second instance (the worker). 
 
Also, we have added the the "-restart" option to 
the "ns_shutdown" command. This just sends the 
SIGINT to the worker process. The watchdog is 
handling this signal and respawns the worker 
automatically. 
 
During operation, the watchdog logs events and their 
cause  into the system log file. This looks like: 
Feb 28 04:00:05 Develop nsd[19400]: worker: started. 
Mar  1 04:00:13 Develop nsd[4475]: watchdog: worker 
19400 exited (2). 
Mar  1 04:00:15 Develop nsd[21290]: worker: started. 
Mar  1 04:00:18 Develop nsd[14705]: watchdog: worker 
19399 exited (2). 
Mar  1 04:00:20 Develop nsd[21300]: worker: started. 
 
We have done all the changes with "--enable-watchdog" 
so anybody who needs this feature will have to compile 
with this option.  
 

----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 23:03

Message:
Logged In: YES 
user_id=184124

Looks good to me

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 12:06

Message:
Logged In: YES 
user_id=95086

Ah, correction: 
The restart option sends SIGINT to the worker process 
which causes the watchdog to restart it.  
And, the patchfile is now attached! 

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 12:03

Message:
Logged In: YES 
user_id=95086

Here is the patch. 
I have added "-restart" option to "ns_shutdown". It is rather
clumsy to parse but should do. We should rewrite this with
your args parsing routine. 
The restart option sends SIGTERM to the worker process
which causes the watchdog to restart it. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-04 23:46

Message:
Logged In: YES 
user_id=87254


I don't think this has to hide behind a config option.  It's
either a good idea or it's not.  Sounds good to me.  Is
there a patch?  I'm wondering about some of the
implementation details...



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

[ naviserver-Feature Requests-1156239 ] nslog restructure

[SourceForge.net <no...@so...>]

Feature Requests item #1156239, was opened at 2005-03-03 22:04
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646

Category: C-API
Group: None
>Status: Closed
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: nslog restructure

Initial Comment:
Post submit for nslog restructure, no functionality
changed really, just added new commands and options to
control nslog on the fly. The source code changed to be
more flexible of extending nslog functionality with
more options.

The only possible bug fix is checking for unknown in
the X-Forwarded-For header.

Sorry for ignoring/forgetting the procedure.

----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 22:55

Message:
Logged In: YES 
user_id=184124

Syntax has been changed according to NS coding style

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156239&group_id=130646

[ naviserver-Feature Requests-1157968 ] Add ns_sha1 command

[SourceForge.net <no...@so...>]

Feature Requests item #1157968, was opened at 2005-03-06 22:52
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157968&group_id=130646

Category: Tcl-API
Group: None
>Status: Closed
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
>Assigned to: Vlad Seryakov (seryakov)
Summary: Add ns_sha1 command

Initial Comment:
Move ns_sha1 command from optional third-party module
int the core, almost all web and other applicztion have
need for digest hashing function.

----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 22:53

Message:
Logged In: YES 
user_id=184124

added to CVS

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157968&group_id=130646

[ naviserver-Feature Requests-1157968 ] Add ns_sha1 command

[SourceForge.net <no...@so...>]

Feature Requests item #1157968, was opened at 2005-03-06 22:52
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157968&group_id=130646

Category: Tcl-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add ns_sha1 command

Initial Comment:
Move ns_sha1 command from optional third-party module
int the core, almost all web and other applicztion have
need for digest hashing function.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157968&group_id=130646

[ naviserver-Feature Requests-1157192 ] Add object-based keylist implementation

[SourceForge.net <no...@so...>]

Feature Requests item #1157192, was opened at 2005-03-05 10:56
Message generated for change (Comment added) made by vasiljevic
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157192&group_id=130646

Category: Tcl-API
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add object-based keylist implementation

Initial Comment:
I did this already for 4.1 aolserver. I would make this for
naviserver as well. The Tcl API is the same just the
underlying
implementation of keylists is object based, instead of
strings.
This is much faster.

----------------------------------------------------------------------

>Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-06 19:42

Message:
Logged In: YES 
user_id=95086

Added into CVS head branch.

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 05:14

Message:
Logged In: YES 
user_id=184124

Sure

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157192&group_id=130646

[ naviserver-Feature Requests-1157192 ] Add object-based keylist implementation

[SourceForge.net <no...@so...>]

Feature Requests item #1157192, was opened at 2005-03-05 09:56
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157192&group_id=130646

Category: Tcl-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add object-based keylist implementation

Initial Comment:
I did this already for 4.1 aolserver. I would make this for
naviserver as well. The Tcl API is the same just the
underlying
implementation of keylists is object based, instead of
strings.
This is much faster.

----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-06 04:14

Message:
Logged In: YES 
user_id=184124

Sure

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157192&group_id=130646

[ naviserver-Feature Requests-1156875 ] Add server watchdog process

[SourceForge.net <no...@so...>]

Feature Requests item #1156875, was opened at 2005-03-04 20:03
Message generated for change (Comment added) made by vasiljevic
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add server watchdog process

Initial Comment:
We have been using this for quite some time and it 
proved extremely useful. 
We doublefork the nsd process and make the first 
forked instance control the second. The first one 
(the watchdog) reacts on exit codes and signals 
caught during the watch and correspondingly restarts 
the second instance (the worker). 
 
Also, we have added the the "-restart" option to 
the "ns_shutdown" command. This just sends the 
SIGINT to the worker process. The watchdog is 
handling this signal and respawns the worker 
automatically. 
 
During operation, the watchdog logs events and their 
cause  into the system log file. This looks like: 
Feb 28 04:00:05 Develop nsd[19400]: worker: started. 
Mar  1 04:00:13 Develop nsd[4475]: watchdog: worker 
19400 exited (2). 
Mar  1 04:00:15 Develop nsd[21290]: worker: started. 
Mar  1 04:00:18 Develop nsd[14705]: watchdog: worker 
19399 exited (2). 
Mar  1 04:00:20 Develop nsd[21300]: worker: started. 
 
We have done all the changes with "--enable-watchdog" 
so anybody who needs this feature will have to compile 
with this option.  
 

----------------------------------------------------------------------

>Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 13:06

Message:
Logged In: YES 
user_id=95086

Ah, correction: 
The restart option sends SIGINT to the worker process 
which causes the watchdog to restart it.  
And, the patchfile is now attached! 

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 13:03

Message:
Logged In: YES 
user_id=95086

Here is the patch. 
I have added "-restart" option to "ns_shutdown". It is rather
clumsy to parse but should do. We should rewrite this with
your args parsing routine. 
The restart option sends SIGTERM to the worker process
which causes the watchdog to restart it. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-05 00:46

Message:
Logged In: YES 
user_id=87254


I don't think this has to hide behind a config option.  It's
either a good idea or it's not.  Sounds good to me.  Is
there a patch?  I'm wondering about some of the
implementation details...



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

[ naviserver-Feature Requests-1156875 ] Add server watchdog process

[SourceForge.net <no...@so...>]

Feature Requests item #1156875, was opened at 2005-03-04 20:03
Message generated for change (Comment added) made by vasiljevic
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add server watchdog process

Initial Comment:
We have been using this for quite some time and it 
proved extremely useful. 
We doublefork the nsd process and make the first 
forked instance control the second. The first one 
(the watchdog) reacts on exit codes and signals 
caught during the watch and correspondingly restarts 
the second instance (the worker). 
 
Also, we have added the the "-restart" option to 
the "ns_shutdown" command. This just sends the 
SIGINT to the worker process. The watchdog is 
handling this signal and respawns the worker 
automatically. 
 
During operation, the watchdog logs events and their 
cause  into the system log file. This looks like: 
Feb 28 04:00:05 Develop nsd[19400]: worker: started. 
Mar  1 04:00:13 Develop nsd[4475]: watchdog: worker 
19400 exited (2). 
Mar  1 04:00:15 Develop nsd[21290]: worker: started. 
Mar  1 04:00:18 Develop nsd[14705]: watchdog: worker 
19399 exited (2). 
Mar  1 04:00:20 Develop nsd[21300]: worker: started. 
 
We have done all the changes with "--enable-watchdog" 
so anybody who needs this feature will have to compile 
with this option.  
 

----------------------------------------------------------------------

>Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 13:03

Message:
Logged In: YES 
user_id=95086

Here is the patch. 
I have added "-restart" option to "ns_shutdown". It is rather
clumsy to parse but should do. We should rewrite this with
your args parsing routine. 
The restart option sends SIGTERM to the worker process
which causes the watchdog to restart it. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-05 00:46

Message:
Logged In: YES 
user_id=87254


I don't think this has to hide behind a config option.  It's
either a good idea or it's not.  Sounds good to me.  Is
there a patch?  I'm wondering about some of the
implementation details...



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156875&group_id=130646

[ naviserver-Feature Requests-1157192 ] Add object-based keylist implementation

[SourceForge.net <no...@so...>]

Feature Requests item #1157192, was opened at 2005-03-05 10:56
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157192&group_id=130646

Category: Tcl-API
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add object-based keylist implementation

Initial Comment:
I did this already for 4.1 aolserver. I would make this for
naviserver as well. The Tcl API is the same just the
underlying
implementation of keylists is object based, instead of
strings.
This is much faster.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1157192&group_id=130646

[ naviserver-Feature Requests-1156141 ] Add ns_conn channel command

[SourceForge.net <no...@so...>]

Feature Requests item #1156141, was opened at 2005-03-03 21:02
Message generated for change (Comment added) made by vasiljevic
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646

Category: None
Group: None
Status: Closed
Resolution: Rejected
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add ns_conn channel command

Initial Comment:
Extend the ns_conn command to allow wrapping an Tcl 
channel arround it. This channel can be used to talk to
the remote client afterwards.

The idea is simple. The client uses standard http protocol
to open up a connection to the server. Server accepts the
connection and obtains the channel from it. The client 
also gets the channel to the server and from this point
on, 
both can exchange arbitrary data.

Example implementation is given as a patch against
the current CVS state.

----------------------------------------------------------------------

>Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-05 10:03

Message:
Logged In: YES 
user_id=95086

   "still don't understand what it buys you that a modern HTTP 
   1.1 implemnetation doesn't.  If there was no other way to do 
 
Stephen, I do not think you understood my point. What this 
command buys me is to be able to initiate an http connection 
and reuse the socket-to-socket comm channel for other 
purposes. 
 
For example we make "GET" request to a certain URL and  
then when the connection gets established, the server, 
takes the conn socket,  wraps a Tcl channel around it and  
passes this channel to my (arbitraty) Tcl proc. 
The client, instead of getting the response body, parses  
only the returned http headers to see that the connection 
succeeded and then passes this socket to my custom 
Tcl proc. Now, on the sever and on the client side you have  
a channel and both sides can use it to establish whatever  
protocol they want. They have a socket pipe between. 
The http is just used to handshake the peers, clear? 
 
What we do with this? We have two server instances on two 
different hosts. We open a channel (using the above trick) 
from one to another and then use this to generate a  
stream of preformatted packets (our protocol) on one side 
which does the backup of data from to the other side 
ultimately to the tape drive(s) attached to that host.  
We shuffle TB of data this way daily in about 500  
installations worldwide, so I  know this works fine.  
 
Now, see, this has nothing to do with HTTP1.1. or 1.0 alone.  
And yes, if you look in the implementation, you will see that  
it is a simple wrapper to the Ns_ConnSock. It simply wraps  
a Tcl channel around the the connection socket, nothing  
more and nothing less. 
 
And, I would not say that it requires a broken client.  It just  
requires a special client-processing. This is not ment for 
browser-clients. It has nothing to do with serving pages 
or any other short lived connections. Once we setup such 
a connection, we use it for hours and maybe days. 
 
We could have achieved this with a simple socket connection 
from client to a specific port  on the server. But we are not 
able to use any arbitrary port. All we have is port 80 (the web 
server port). So we must play by the http rules initially when 
accessing the server. When we passed this http handshake 
stuff, we are left with a socket-to-socket comm-channel  
which is transparent and can be used for any purpose 
whatsoever. 
 
Now, is this little bit more clear now? 

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-05 04:32

Message:
Logged In: YES 
user_id=184124

What is broken about it? It is the Tcl interface to
Ns_ConnSock(), just Tcl way, using channel. The merit is to
use sync inter-exchange between client and server using
established connection. Client is not broken, it is designed
for this exchange, it is the tool to build messaging, not
general purpose protocol. It make naviserver more flexible
platform for implementing different ppas on top of it using
Tcl, that's all.
Btw, when are you going to commit your protocol extensions?

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-05 01:20

Message:
Logged In: YES 
user_id=87254


This command isn't a good idea because it creates a broken
interface which requires a broken client to work, and I
still don't understand what it buys you that a modern HTTP
1.1 implemnetation doesn't.  If there was no other way to do
it, then it may still have some merit.  But Ns_ConnSock() is
trivial and gives you all you need to create a Tcl channel.

We've already decided to do this the right way, whatever
that turns out to be, so why should we simultaneously also
do it the wrong way?

In fact you seem to agree, noting that this sucks because
it's not UDP and the overhead of HTTP is just too much for you.



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-04 16:36

Message:
Logged In: YES 
user_id=184124

And that's why i want to add simple UDP support to the
server: for implementing simple messaging between servers in
the cluster, using UDP without HTTP overhead will make it
simple and faster. Using Tcl channel is intermediate
solution for such messaging because it eliminates making
multiple HTTP requests to the server but still uses heavy
TCP/HTTP overhead. Still i think this command is very
usefull and will make server more versatile provided that we
will supply usage examples with the server as well.

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 16:26

Message:
Logged In: YES 
user_id=95086

  "Second, this has nothing to do with non-HTTP protocol, it is 
  HTTP communication protocol, what's inside the body has 
  nothing to do with it again." 
 
This is true. It is a regular http request, just for the body 
part, the peers are engaging in a bidirectional communication 
and can do whatever they pleased.  The request is considered 
done when one of the peers close the socket. 
 
Strictly speaking, I'm using the http body and this trick 
to implement arbitrary client-server comm channel. This will 
not work for browsers of course. It has nothing to do with 
browsers at all in fact. It arose out of the necessity to  
support alternate protocols given what we had in AS, 
that is pure http machinery. 
 

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-04 15:38

Message:
Logged In: YES 
user_id=184124

On the closing note, keepalive has nothing to do with this
and should not be considered in this case. Even if you
connect using HTTP 1.1 and specify keepalive, sever has the
right to issue Connection: close header which will disable
keepalive.
So, it WILL work for all connections.
Second, this has nothing to do with non-HTTP protocol, it is
HTTP communication protocol, what's inside the body has
nothing to do with it again.
And pipelining is a bad example here, the point it to send
the request and wait for the answer and base on that send
another request. Pipelining will allow you just to submit
all requests and then later receive replies, it was designed
for speeding browser's connection especially retrieving images.

It looks to me Stephen you are all for HTTP core and
around-the-HTTP-driver solutions only. This will make simple
solutions complex because of getting around/workaround HTTP
based core in the server.

What is wrong with one simpe function which will give the
server more options without breaking anything?

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 10:45

Message:
Logged In: YES 
user_id=95086

I see. The gotcha's are: 
 
  o. it's yet another implementation of non-HTTP protocol 
  o. (HTTP 1.0 only, and no Keepalive header) be used to get 
  o. function in the docs might reasonably expect it to work for 
     all HTTP connections 
 
That's pretty clear. Well, I think you are right. This is our  
attempt to work around http-only caps of the server.  
Agreed. It will be better to get it done properly and we will 
then internally switch to the new implementation when 
it's ready. At the moment I have all this in a separate module 
and this can stay so until the multi-protocol caps are agreed 
upon and implemented. 
 
I will close this RFE then. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-04 10:36

Message:
Logged In: YES 
user_id=87254


I don't understand why you're not using standard HTTP
pipelining.  It would seem to be so much easier than having
to write custom client and server code.

This is a small change, but I'm not excited about it because
it's yet another implementation of non-HTTP protocol
support, which makes 3 so far...  :-(

It also seems to require that a deliberately broken client
(HTTP 1.0 only, and no Keepalive header) be used to get
around the server's io read-ahead.  Someone seeing this
function in the docs might reasonably expect it to work for
all HTTP connections.

Also, I don't think this requires core support. 
Ns_ConnSock() will return the underlying file descriptor for
a connection.



----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 09:36

Message:
Logged In: YES 
user_id=95086

OK. Maybe I should explain little bit more what I'm using this 
for. 
Imagine you'd like to make a client-server connection and 
then exchange arbitrary data on this connection. Imagine 
something like telnet connection. You telnet from client to 
server, get a comm-pipe betwen them and then you can 
shuffle whatever you like across this comm-pipe.  
What we have is a http server responding only to http  
requests (as it is now, w/o multiprotocol support). So, what 
I do is to use http to setup the connection, have a registered 
procedure on the server grab the socket and install my custom 
protocol handler on it. The http is only used to establish the 
communication. All the rest is handled by my custom code 
on the server and on the client.  
Once the connection is setup, we have a point-to-point  
socket communication link between server and client and we 
can use it for just about anything. All you need for this is a  
client-side http-library (we have ours written in Tcl) which  
will give you the socket back once the connection is  
established) and serve-side proc implementing your protocol 
of choice. So, the entire server http-machinery stays in place. 
I do not see any problem there because we have been using 
this trick for about 4 years in production very successfully. 
What I do not know is wether there are any gotcha's with that 
approach (we haven't hit any yet). 
I also do not know what will happen when we add alternate 
multiprotocol caps to server. Maybe this will become obsolete. 
But, as I see this now, the change is trivial and does not hurt 
anybody. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-04 00:21

Message:
Logged In: YES 
user_id=87254


HTTP also has (although our server doesn't, but I want to
add this) pipelining, which is sending multiple requests at
once without first waiting for the reply to previous requests.

That sounds like your "i do it when I send multiple http
requests".



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 23:45

Message:
Logged In: YES 
user_id=184124

Keepalive works when i close my connection, i.e. when i
exited from conn thread, but the point here is to do it from
the conn thread, i received request from my other server
sending me some info, i send my info back and get ack.
Everything inside one connection, before keepalive.
And yes, i send messages in my format, anyway i do it when i
send multiple http reaqests, i am interested in message body
only.

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-03 23:32

Message:
Logged In: YES 
user_id=87254


HTTP already handles sending multiple requests accross one
socket with keepalives.  It might be more useful to add
keepalives to our HTTP client code.

In the case of HTTP keepalives, the socket is returned to
the driver thread after the first request, still open, and
the conn thread goes on to do other useful work.  If a new
request arrives before the connection times out it is passed
to the next free conn thread.

If you don't usee HTTP with keepalives you will have to
invent some kind of message chunking syntax and some kind of
time-out protocol.  But you will still end up wasting conn
threads as they idle waiting for possible extra messages.



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 23:20

Message:
Logged In: YES 
user_id=184124

I could use it in my apps where i have cluster of several
servers exchanging http requests between each other,
sometimes one session needs 2 or more exchanges.
With this channel i can exchange using same connection and 
i need to send/receive data, not headers, so this can save
and improve my operations.

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-03 23:12

Message:
Logged In: YES 
user_id=87254


Usually, by the time your script runs and you call [ns_conn
channel], the data from the client will already have been
read and buffered.  You can access this with [ns_conn content].

Does reading work?

What's the objective?  Do you want to hadle streaming data,
or make reading and writing more Tcl-like?


----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 23:07

Message:
Logged In: YES 
user_id=184124

Very usefull, commit it

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646

[ naviserver-Feature Requests-1156141 ] Add ns_conn channel command

[SourceForge.net <no...@so...>]

Feature Requests item #1156141, was opened at 2005-03-03 20:02
Message generated for change (Comment added) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646

Category: None
Group: None
Status: Closed
Resolution: Rejected
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add ns_conn channel command

Initial Comment:
Extend the ns_conn command to allow wrapping an Tcl 
channel arround it. This channel can be used to talk to
the remote client afterwards.

The idea is simple. The client uses standard http protocol
to open up a connection to the server. Server accepts the
connection and obtains the channel from it. The client 
also gets the channel to the server and from this point
on, 
both can exchange arbitrary data.

Example implementation is given as a patch against
the current CVS state.

----------------------------------------------------------------------

>Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-05 03:32

Message:
Logged In: YES 
user_id=184124

What is broken about it? It is the Tcl interface to
Ns_ConnSock(), just Tcl way, using channel. The merit is to
use sync inter-exchange between client and server using
established connection. Client is not broken, it is designed
for this exchange, it is the tool to build messaging, not
general purpose protocol. It make naviserver more flexible
platform for implementing different ppas on top of it using
Tcl, that's all.
Btw, when are you going to commit your protocol extensions?

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-05 00:20

Message:
Logged In: YES 
user_id=87254


This command isn't a good idea because it creates a broken
interface which requires a broken client to work, and I
still don't understand what it buys you that a modern HTTP
1.1 implemnetation doesn't.  If there was no other way to do
it, then it may still have some merit.  But Ns_ConnSock() is
trivial and gives you all you need to create a Tcl channel.

We've already decided to do this the right way, whatever
that turns out to be, so why should we simultaneously also
do it the wrong way?

In fact you seem to agree, noting that this sucks because
it's not UDP and the overhead of HTTP is just too much for you.



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-04 15:36

Message:
Logged In: YES 
user_id=184124

And that's why i want to add simple UDP support to the
server: for implementing simple messaging between servers in
the cluster, using UDP without HTTP overhead will make it
simple and faster. Using Tcl channel is intermediate
solution for such messaging because it eliminates making
multiple HTTP requests to the server but still uses heavy
TCP/HTTP overhead. Still i think this command is very
usefull and will make server more versatile provided that we
will supply usage examples with the server as well.

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 15:26

Message:
Logged In: YES 
user_id=95086

  "Second, this has nothing to do with non-HTTP protocol, it is 
  HTTP communication protocol, what's inside the body has 
  nothing to do with it again." 
 
This is true. It is a regular http request, just for the body 
part, the peers are engaging in a bidirectional communication 
and can do whatever they pleased.  The request is considered 
done when one of the peers close the socket. 
 
Strictly speaking, I'm using the http body and this trick 
to implement arbitrary client-server comm channel. This will 
not work for browsers of course. It has nothing to do with 
browsers at all in fact. It arose out of the necessity to  
support alternate protocols given what we had in AS, 
that is pure http machinery. 
 

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-04 14:38

Message:
Logged In: YES 
user_id=184124

On the closing note, keepalive has nothing to do with this
and should not be considered in this case. Even if you
connect using HTTP 1.1 and specify keepalive, sever has the
right to issue Connection: close header which will disable
keepalive.
So, it WILL work for all connections.
Second, this has nothing to do with non-HTTP protocol, it is
HTTP communication protocol, what's inside the body has
nothing to do with it again.
And pipelining is a bad example here, the point it to send
the request and wait for the answer and base on that send
another request. Pipelining will allow you just to submit
all requests and then later receive replies, it was designed
for speeding browser's connection especially retrieving images.

It looks to me Stephen you are all for HTTP core and
around-the-HTTP-driver solutions only. This will make simple
solutions complex because of getting around/workaround HTTP
based core in the server.

What is wrong with one simpe function which will give the
server more options without breaking anything?

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 09:45

Message:
Logged In: YES 
user_id=95086

I see. The gotcha's are: 
 
  o. it's yet another implementation of non-HTTP protocol 
  o. (HTTP 1.0 only, and no Keepalive header) be used to get 
  o. function in the docs might reasonably expect it to work for 
     all HTTP connections 
 
That's pretty clear. Well, I think you are right. This is our  
attempt to work around http-only caps of the server.  
Agreed. It will be better to get it done properly and we will 
then internally switch to the new implementation when 
it's ready. At the moment I have all this in a separate module 
and this can stay so until the multi-protocol caps are agreed 
upon and implemented. 
 
I will close this RFE then. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-04 09:36

Message:
Logged In: YES 
user_id=87254


I don't understand why you're not using standard HTTP
pipelining.  It would seem to be so much easier than having
to write custom client and server code.

This is a small change, but I'm not excited about it because
it's yet another implementation of non-HTTP protocol
support, which makes 3 so far...  :-(

It also seems to require that a deliberately broken client
(HTTP 1.0 only, and no Keepalive header) be used to get
around the server's io read-ahead.  Someone seeing this
function in the docs might reasonably expect it to work for
all HTTP connections.

Also, I don't think this requires core support. 
Ns_ConnSock() will return the underlying file descriptor for
a connection.



----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 08:36

Message:
Logged In: YES 
user_id=95086

OK. Maybe I should explain little bit more what I'm using this 
for. 
Imagine you'd like to make a client-server connection and 
then exchange arbitrary data on this connection. Imagine 
something like telnet connection. You telnet from client to 
server, get a comm-pipe betwen them and then you can 
shuffle whatever you like across this comm-pipe.  
What we have is a http server responding only to http  
requests (as it is now, w/o multiprotocol support). So, what 
I do is to use http to setup the connection, have a registered 
procedure on the server grab the socket and install my custom 
protocol handler on it. The http is only used to establish the 
communication. All the rest is handled by my custom code 
on the server and on the client.  
Once the connection is setup, we have a point-to-point  
socket communication link between server and client and we 
can use it for just about anything. All you need for this is a  
client-side http-library (we have ours written in Tcl) which  
will give you the socket back once the connection is  
established) and serve-side proc implementing your protocol 
of choice. So, the entire server http-machinery stays in place. 
I do not see any problem there because we have been using 
this trick for about 4 years in production very successfully. 
What I do not know is wether there are any gotcha's with that 
approach (we haven't hit any yet). 
I also do not know what will happen when we add alternate 
multiprotocol caps to server. Maybe this will become obsolete. 
But, as I see this now, the change is trivial and does not hurt 
anybody. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-03 23:21

Message:
Logged In: YES 
user_id=87254


HTTP also has (although our server doesn't, but I want to
add this) pipelining, which is sending multiple requests at
once without first waiting for the reply to previous requests.

That sounds like your "i do it when I send multiple http
requests".



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 22:45

Message:
Logged In: YES 
user_id=184124

Keepalive works when i close my connection, i.e. when i
exited from conn thread, but the point here is to do it from
the conn thread, i received request from my other server
sending me some info, i send my info back and get ack.
Everything inside one connection, before keepalive.
And yes, i send messages in my format, anyway i do it when i
send multiple http reaqests, i am interested in message body
only.

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-03 22:32

Message:
Logged In: YES 
user_id=87254


HTTP already handles sending multiple requests accross one
socket with keepalives.  It might be more useful to add
keepalives to our HTTP client code.

In the case of HTTP keepalives, the socket is returned to
the driver thread after the first request, still open, and
the conn thread goes on to do other useful work.  If a new
request arrives before the connection times out it is passed
to the next free conn thread.

If you don't usee HTTP with keepalives you will have to
invent some kind of message chunking syntax and some kind of
time-out protocol.  But you will still end up wasting conn
threads as they idle waiting for possible extra messages.



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 22:20

Message:
Logged In: YES 
user_id=184124

I could use it in my apps where i have cluster of several
servers exchanging http requests between each other,
sometimes one session needs 2 or more exchanges.
With this channel i can exchange using same connection and 
i need to send/receive data, not headers, so this can save
and improve my operations.

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-03 22:12

Message:
Logged In: YES 
user_id=87254


Usually, by the time your script runs and you call [ns_conn
channel], the data from the client will already have been
read and buffered.  You can access this with [ns_conn content].

Does reading work?

What's the objective?  Do you want to hadle streaming data,
or make reading and writing more Tcl-like?


----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 22:07

Message:
Logged In: YES 
user_id=184124

Very usefull, commit it

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646

[ naviserver-Feature Requests-1156141 ] Add ns_conn channel command

[SourceForge.net <no...@so...>]

Feature Requests item #1156141, was opened at 2005-03-03 13:02
Message generated for change (Comment added) made by sdeasey
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646

Category: None
Group: None
Status: Closed
Resolution: Rejected
Priority: 5
Submitted By: Zoran Vasiljevic (vasiljevic)
Assigned to: Zoran Vasiljevic (vasiljevic)
Summary: Add ns_conn channel command

Initial Comment:
Extend the ns_conn command to allow wrapping an Tcl 
channel arround it. This channel can be used to talk to
the remote client afterwards.

The idea is simple. The client uses standard http protocol
to open up a connection to the server. Server accepts the
connection and obtains the channel from it. The client 
also gets the channel to the server and from this point
on, 
both can exchange arbitrary data.

Example implementation is given as a patch against
the current CVS state.

----------------------------------------------------------------------

>Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-04 17:20

Message:
Logged In: YES 
user_id=87254


This command isn't a good idea because it creates a broken
interface which requires a broken client to work, and I
still don't understand what it buys you that a modern HTTP
1.1 implemnetation doesn't.  If there was no other way to do
it, then it may still have some merit.  But Ns_ConnSock() is
trivial and gives you all you need to create a Tcl channel.

We've already decided to do this the right way, whatever
that turns out to be, so why should we simultaneously also
do it the wrong way?

In fact you seem to agree, noting that this sucks because
it's not UDP and the overhead of HTTP is just too much for you.



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-04 08:36

Message:
Logged In: YES 
user_id=184124

And that's why i want to add simple UDP support to the
server: for implementing simple messaging between servers in
the cluster, using UDP without HTTP overhead will make it
simple and faster. Using Tcl channel is intermediate
solution for such messaging because it eliminates making
multiple HTTP requests to the server but still uses heavy
TCP/HTTP overhead. Still i think this command is very
usefull and will make server more versatile provided that we
will supply usage examples with the server as well.

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 08:26

Message:
Logged In: YES 
user_id=95086

  "Second, this has nothing to do with non-HTTP protocol, it is 
  HTTP communication protocol, what's inside the body has 
  nothing to do with it again." 
 
This is true. It is a regular http request, just for the body 
part, the peers are engaging in a bidirectional communication 
and can do whatever they pleased.  The request is considered 
done when one of the peers close the socket. 
 
Strictly speaking, I'm using the http body and this trick 
to implement arbitrary client-server comm channel. This will 
not work for browsers of course. It has nothing to do with 
browsers at all in fact. It arose out of the necessity to  
support alternate protocols given what we had in AS, 
that is pure http machinery. 
 

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-04 07:38

Message:
Logged In: YES 
user_id=184124

On the closing note, keepalive has nothing to do with this
and should not be considered in this case. Even if you
connect using HTTP 1.1 and specify keepalive, sever has the
right to issue Connection: close header which will disable
keepalive.
So, it WILL work for all connections.
Second, this has nothing to do with non-HTTP protocol, it is
HTTP communication protocol, what's inside the body has
nothing to do with it again.
And pipelining is a bad example here, the point it to send
the request and wait for the answer and base on that send
another request. Pipelining will allow you just to submit
all requests and then later receive replies, it was designed
for speeding browser's connection especially retrieving images.

It looks to me Stephen you are all for HTTP core and
around-the-HTTP-driver solutions only. This will make simple
solutions complex because of getting around/workaround HTTP
based core in the server.

What is wrong with one simpe function which will give the
server more options without breaking anything?

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 02:45

Message:
Logged In: YES 
user_id=95086

I see. The gotcha's are: 
 
  o. it's yet another implementation of non-HTTP protocol 
  o. (HTTP 1.0 only, and no Keepalive header) be used to get 
  o. function in the docs might reasonably expect it to work for 
     all HTTP connections 
 
That's pretty clear. Well, I think you are right. This is our  
attempt to work around http-only caps of the server.  
Agreed. It will be better to get it done properly and we will 
then internally switch to the new implementation when 
it's ready. At the moment I have all this in a separate module 
and this can stay so until the multi-protocol caps are agreed 
upon and implemented. 
 
I will close this RFE then. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-04 02:36

Message:
Logged In: YES 
user_id=87254


I don't understand why you're not using standard HTTP
pipelining.  It would seem to be so much easier than having
to write custom client and server code.

This is a small change, but I'm not excited about it because
it's yet another implementation of non-HTTP protocol
support, which makes 3 so far...  :-(

It also seems to require that a deliberately broken client
(HTTP 1.0 only, and no Keepalive header) be used to get
around the server's io read-ahead.  Someone seeing this
function in the docs might reasonably expect it to work for
all HTTP connections.

Also, I don't think this requires core support. 
Ns_ConnSock() will return the underlying file descriptor for
a connection.



----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2005-03-04 01:36

Message:
Logged In: YES 
user_id=95086

OK. Maybe I should explain little bit more what I'm using this 
for. 
Imagine you'd like to make a client-server connection and 
then exchange arbitrary data on this connection. Imagine 
something like telnet connection. You telnet from client to 
server, get a comm-pipe betwen them and then you can 
shuffle whatever you like across this comm-pipe.  
What we have is a http server responding only to http  
requests (as it is now, w/o multiprotocol support). So, what 
I do is to use http to setup the connection, have a registered 
procedure on the server grab the socket and install my custom 
protocol handler on it. The http is only used to establish the 
communication. All the rest is handled by my custom code 
on the server and on the client.  
Once the connection is setup, we have a point-to-point  
socket communication link between server and client and we 
can use it for just about anything. All you need for this is a  
client-side http-library (we have ours written in Tcl) which  
will give you the socket back once the connection is  
established) and serve-side proc implementing your protocol 
of choice. So, the entire server http-machinery stays in place. 
I do not see any problem there because we have been using 
this trick for about 4 years in production very successfully. 
What I do not know is wether there are any gotcha's with that 
approach (we haven't hit any yet). 
I also do not know what will happen when we add alternate 
multiprotocol caps to server. Maybe this will become obsolete. 
But, as I see this now, the change is trivial and does not hurt 
anybody. 

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-03 16:21

Message:
Logged In: YES 
user_id=87254


HTTP also has (although our server doesn't, but I want to
add this) pipelining, which is sending multiple requests at
once without first waiting for the reply to previous requests.

That sounds like your "i do it when I send multiple http
requests".



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 15:45

Message:
Logged In: YES 
user_id=184124

Keepalive works when i close my connection, i.e. when i
exited from conn thread, but the point here is to do it from
the conn thread, i received request from my other server
sending me some info, i send my info back and get ack.
Everything inside one connection, before keepalive.
And yes, i send messages in my format, anyway i do it when i
send multiple http reaqests, i am interested in message body
only.

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-03 15:32

Message:
Logged In: YES 
user_id=87254


HTTP already handles sending multiple requests accross one
socket with keepalives.  It might be more useful to add
keepalives to our HTTP client code.

In the case of HTTP keepalives, the socket is returned to
the driver thread after the first request, still open, and
the conn thread goes on to do other useful work.  If a new
request arrives before the connection times out it is passed
to the next free conn thread.

If you don't usee HTTP with keepalives you will have to
invent some kind of message chunking syntax and some kind of
time-out protocol.  But you will still end up wasting conn
threads as they idle waiting for possible extra messages.



----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 15:20

Message:
Logged In: YES 
user_id=184124

I could use it in my apps where i have cluster of several
servers exchanging http requests between each other,
sometimes one session needs 2 or more exchanges.
With this channel i can exchange using same connection and 
i need to send/receive data, not headers, so this can save
and improve my operations.

----------------------------------------------------------------------

Comment By: Stephen Deasey (sdeasey)
Date: 2005-03-03 15:12

Message:
Logged In: YES 
user_id=87254


Usually, by the time your script runs and you call [ns_conn
channel], the data from the client will already have been
read and buffered.  You can access this with [ns_conn content].

Does reading work?

What's the objective?  Do you want to hadle streaming data,
or make reading and writing more Tcl-like?


----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2005-03-03 15:07

Message:
Logged In: YES 
user_id=184124

Very usefull, commit it

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1156141&group_id=130646