naviserver-devel — Developer discussion

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Stephen D. <sd...@gm...>]

On 6/19/06, Vlad Seryakov <vl...@cr...> wrote:
> Stephen,
>
> I understand you have the "right" vision how to do things but after long
> period of time just removing Tcl API functions (cache set/get) is not
> very polite.
> It breaks a lot of Tcl code and i am not sure why do you think they are
> racey and i need to emulate them in Tcl instead of C.
>
> Very frustrating.


We talked about this before.  If I remember correctly, you didn't
disagree with me that the commands were racy, you said you code base
was large and you didn't have time to figure out what you really
needed.

More and more unnecessary code is being added.  Things are getting out
of control.  However, you'll notice that I did not just remove
everything.  With the new -force switch to ns_cache_eval, _set, _get,
and _info can be trivially emulated in a couple of lines of Tcl, if
you want that kind of racy behaviour.

Everything is of course still available via the C api, and I'd
recommend taking a look at what your real requirements are and
building accordingly.  If it can be generalised, we can get it into
the core.


Sorry this is hurried.  I don't have 'net access.  I'll be travelling
for the next couple of weeks.


> Stephen Deasey wrote:
> > Update of /cvsroot/naviserver/naviserver
> > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv27993
> >
> > Modified Files:
> >       ChangeLog
> > Log Message:
> >         * nsd/fastpath.c:
> >         * nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
> >         Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> >
> >         * tcl/cache.tcl:
> >         * nsd/nsconf.c:
> >         * nsd/dns.c: Update to new API.
> >
> >         * nsd/nsd.h:
> >         * nsd/server.c: Remove tcl cache timeout param in anticipation of new
> >         limits API.
> >
> >         * nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
> >         commands. The first three are racey and can be emulated with _eval
> >         and the new -force switch.  The new -contents switch to the _stats
> >         command replaces _info, discouraging racey check-and-set usage.
> >
> >         * nsd/tcltime.c:
> >         * nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
> >         type. Handy for passing absolute time deadlines to routines which
> >         timeout.
> >
> >         * include/ns.h:
> >         * nsd/cache.c: New Ns_CacheResetStats routine.
> >
> >         Ns_CacheGetConfig removed -- caches are immutable; the current
> >         settings can be looked up in the config file.
> >
> >         Directly expire entries from within all routines which return one,
> >         but but only if the value is not null, indicating no concurrent
> >         update is in progress.
> >
> >         Use Ns_CacheDeleteEntry to remove an entry from the cache without
> >         updating the stats. Flush stats should reflect explicit flushes
> >         only.
> >
> >         Log stats when a cache is destroyed.
> >
> >         * nsd/tclcache.c: ns_cache_create now takes a -maxentry option
> >         which is the maximum size of an entry allowed in the cache. This
> >         prevents one large entry completely emptying an otherwise usefully
> >         full cache.
> >
> >         The ns_cache_create -timeout switch has been removed in
> >         anticipation of a general purpose limits scheme.
> >
> >         The -timeout option is now expected to be an absolute time in the
> >         future, not an offset from the current time. This is awkward to
> >         use manually, but should not be needed with the above mentioned
> >         limits scheme. The -ttl option has been renames -expires and is
> >         also an absolute time.
> >
> >         ns_cache_eval now takes a -force switch which will unconditioanly
> >         replace any exiting entry, whether it has expired or not. Replaces
> >         _set.
> >
> >         ns_cache_stats -content dumps the size and expirey for each
> >         entry in the cache. Adding the -reset switch to either mode resets
> >         the stats to zero.
> >
> >         * tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
> >         harder with some more flush tests.
> >
> >
> >
> > Index: ChangeLog
> > ===================================================================
> > RCS file: /cvsroot/naviserver/naviserver/ChangeLog,v
> > retrieving revision 1.405
> > retrieving revision 1.406
> > diff -C2 -d -r1.405 -r1.406
> > *** ChangeLog 19 Jun 2006 19:05:47 -0000      1.405
> > --- ChangeLog 19 Jun 2006 21:16:38 -0000      1.406
> > ***************
> > *** 1,2 ****
> > --- 1,71 ----
> > + 2006-06-19  Stephen Deasey  <sd...@us...>
> > +
> > +     * nsd/fastpath.c:
> > +     * nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
> > +     Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> > +
> > +     * tcl/cache.tcl:
> > +     * nsd/nsconf.c:
> > +     * nsd/dns.c: Update to new API.
> > +
> > +     * nsd/nsd.h:
> > +     * nsd/server.c: Remove tcl cache timeout param in anticipation of new
> > +     limits API.
> > +
> > +     * nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
> > +     commands. The first three are racey and can be emulated with _eval
> > +     and the new -force switch.  The new -contents switch to the _stats
> > +     command replaces _info, discouraging racey check-and-set usage.
> > +
> > +     * nsd/tcltime.c:
> > +     * nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
> > +     type. Handy for passing absolute time deadlines to routines which
> > +     timeout.
> > +
> > +     * include/ns.h:
> > +     * nsd/cache.c: New Ns_CacheResetStats routine.
> > +
> > +     Ns_CacheGetConfig removed -- caches are immutable; the current
> > +     settings can be looked up in the config file.
> > +
> > +     Directly expire entries from within all routines which return one,
> > +     but but only if the value is not null, indicating no concurrent
> > +     update is in progress.
> > +
> > +     Use Ns_CacheDeleteEntry to remove an entry from the cache without
> > +     updating the stats. Flush stats should reflect explicit flushes
> > +     only.
> > +
> > +     Log stats when a cache is destroyed.
> > +
> > +     * nsd/tclcache.c: ns_cache_create now takes a -maxentry option
> > +     which is the maximum size of an entry allowed in the cache. This
> > +     prevents one large entry completely emptying an otherwise usefully
> > +     full cache.
> > +
> > +     The ns_cache_create -timeout switch has been removed in
> > +     anticipation of a general purpose limits scheme.
> > +
> > +     The -timeout option is now expected to be an absolute time in the
> > +     future, not an offset from the current time. This is awkward to
> > +     use manually, but should not be needed with the above mentioned
> > +     limits scheme. The -ttl option has been renames -expires and is
> > +     also an absolute time.
> > +
> > +     ns_cache_eval now takes a -force switch which will unconditioanly
> > +     replace any exiting entry, whether it has expired or not. Replaces
> > +     _set.
> > +
> > +     ns_cache_stats -content dumps the size and expirey for each
> > +     entry in the cache. Adding the -reset switch to either mode resets
> > +     the stats to zero.
> > +
> > +
> > +
> > +
> > +
> > +     * tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
> > +     harder with some more flush tests.
> > +
> >   2006-05-19  Vlad Seryakov  <ser...@us...>
> >
> >
> >
> >
> > _______________________________________________
> > naviserver-commits mailing list
> > nav...@li...
> > https://lists.sourceforge.net/lists/listinfo/naviserver-commits
> >
>
> --
> Vlad Seryakov
> 571 262-8608 office
> vl...@cr...
> http://www.crystalballinc.com/vlad/
>
>
>
> _______________________________________________
> naviserver-commits mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-commits
>

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Stephen D. <sd...@gm...>]

On 6/19/06, Vlad Seryakov <vl...@cr...> wrote:
> Also, what is wrong with per-cache expiration policy, nc_cache_create
> -timeout parameter.
> How i can create 2 caches with different expiration settings?


Nothing particularly, but there's a better way to do it:

Look at the aolserver head, at nsd/limits.c.  A 'limits' is a
collection of setting such as max upload size, timeout etc.  You
create one or more of these then bind them to a URL, just like
ns_register_proc.

The timeout in the aolserver implementation refers to just the time
waiting for a conn thread.  I'm going to extend that to mean total
time allowed for the conn to run.

So, you might set timeout to be 30 seconds.  You bind this to /*, i.e.
every URL.  Now, when you use a cache, or ns_proxy, or ns_db, or in
fact anything where a timeout is needed, and you haven't specified
anything expliciltly using a  -timeout switch, the limits are used.

This is actually more what you want -- you're defining the quality of
service you expect to give.  I made some changes recently so that Tcl
code which times out in this way throws a particular error code. This
is picked up by the server and turned into the appropriate HTTP
'server busy' code.

To get this working, Ns_Conn structs need to be allocated in driver.c,
not queue.c as they are now.  driver.c is looking a little scary...



>  >Ns_CacheGetConfig removed -- caches are immutable; the current
>  > settings can be looked up in the config file.
>
> This is not absolutely true if nothing in the config and i need to find
> out default compiled-in value. GetConfig gives runtime value regardless
> of what is set or not set in the config file. With per-cache timeout
> this makes sense. How limits are supposed to work?


Don't compile in defaults, use Ns_ConfigGet* etc.


> Stephen Deasey wrote:
> > Update of /cvsroot/naviserver/naviserver
> > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv27993
> >
> > Modified Files:
> >       ChangeLog
> > Log Message:
> >         * nsd/fastpath.c:
> >         * nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
> >         Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> >
> >         * tcl/cache.tcl:
> >         * nsd/nsconf.c:
> >         * nsd/dns.c: Update to new API.
> >
> >         * nsd/nsd.h:
> >         * nsd/server.c: Remove tcl cache timeout param in anticipation of new
> >         limits API.
> >
> >         * nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
> >         commands. The first three are racey and can be emulated with _eval
> >         and the new -force switch.  The new -contents switch to the _stats
> >         command replaces _info, discouraging racey check-and-set usage.
> >
> >         * nsd/tcltime.c:
> >         * nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
> >         type. Handy for passing absolute time deadlines to routines which
> >         timeout.
> >
> >         * include/ns.h:
> >         * nsd/cache.c: New Ns_CacheResetStats routine.
> >
> >         Ns_CacheGetConfig removed -- caches are immutable; the current
> >         settings can be looked up in the config file.
> >
> >         Directly expire entries from within all routines which return one,
> >         but but only if the value is not null, indicating no concurrent
> >         update is in progress.
> >
> >         Use Ns_CacheDeleteEntry to remove an entry from the cache without
> >         updating the stats. Flush stats should reflect explicit flushes
> >         only.
> >
> >         Log stats when a cache is destroyed.
> >
> >         * nsd/tclcache.c: ns_cache_create now takes a -maxentry option
> >         which is the maximum size of an entry allowed in the cache. This
> >         prevents one large entry completely emptying an otherwise usefully
> >         full cache.
> >
> >         The ns_cache_create -timeout switch has been removed in
> >         anticipation of a general purpose limits scheme.
> >
> >         The -timeout option is now expected to be an absolute time in the
> >         future, not an offset from the current time. This is awkward to
> >         use manually, but should not be needed with the above mentioned
> >         limits scheme. The -ttl option has been renames -expires and is
> >         also an absolute time.
> >
> >         ns_cache_eval now takes a -force switch which will unconditioanly
> >         replace any exiting entry, whether it has expired or not. Replaces
> >         _set.
> >
> >         ns_cache_stats -content dumps the size and expirey for each
> >         entry in the cache. Adding the -reset switch to either mode resets
> >         the stats to zero.
> >
> >         * tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
> >         harder with some more flush tests.
> >
> >
> >
> > Index: ChangeLog
> > ===================================================================
> > RCS file: /cvsroot/naviserver/naviserver/ChangeLog,v
> > retrieving revision 1.405
> > retrieving revision 1.406
> > diff -C2 -d -r1.405 -r1.406
> > *** ChangeLog 19 Jun 2006 19:05:47 -0000      1.405
> > --- ChangeLog 19 Jun 2006 21:16:38 -0000      1.406
> > ***************
> > *** 1,2 ****
> > --- 1,71 ----
> > + 2006-06-19  Stephen Deasey  <sd...@us...>
> > +
> > +     * nsd/fastpath.c:
> > +     * nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
> > +     Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> > +
> > +     * tcl/cache.tcl:
> > +     * nsd/nsconf.c:
> > +     * nsd/dns.c: Update to new API.
> > +
> > +     * nsd/nsd.h:
> > +     * nsd/server.c: Remove tcl cache timeout param in anticipation of new
> > +     limits API.
> > +
> > +     * nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
> > +     commands. The first three are racey and can be emulated with _eval
> > +     and the new -force switch.  The new -contents switch to the _stats
> > +     command replaces _info, discouraging racey check-and-set usage.
> > +
> > +     * nsd/tcltime.c:
> > +     * nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
> > +     type. Handy for passing absolute time deadlines to routines which
> > +     timeout.
> > +
> > +     * include/ns.h:
> > +     * nsd/cache.c: New Ns_CacheResetStats routine.
> > +
> > +     Ns_CacheGetConfig removed -- caches are immutable; the current
> > +     settings can be looked up in the config file.
> > +
> > +     Directly expire entries from within all routines which return one,
> > +     but but only if the value is not null, indicating no concurrent
> > +     update is in progress.
> > +
> > +     Use Ns_CacheDeleteEntry to remove an entry from the cache without
> > +     updating the stats. Flush stats should reflect explicit flushes
> > +     only.
> > +
> > +     Log stats when a cache is destroyed.
> > +
> > +     * nsd/tclcache.c: ns_cache_create now takes a -maxentry option
> > +     which is the maximum size of an entry allowed in the cache. This
> > +     prevents one large entry completely emptying an otherwise usefully
> > +     full cache.
> > +
> > +     The ns_cache_create -timeout switch has been removed in
> > +     anticipation of a general purpose limits scheme.
> > +
> > +     The -timeout option is now expected to be an absolute time in the
> > +     future, not an offset from the current time. This is awkward to
> > +     use manually, but should not be needed with the above mentioned
> > +     limits scheme. The -ttl option has been renames -expires and is
> > +     also an absolute time.
> > +
> > +     ns_cache_eval now takes a -force switch which will unconditioanly
> > +     replace any exiting entry, whether it has expired or not. Replaces
> > +     _set.
> > +
> > +     ns_cache_stats -content dumps the size and expirey for each
> > +     entry in the cache. Adding the -reset switch to either mode resets
> > +     the stats to zero.
> > +
> > +
> > +
> > +
> > +
> > +     * tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
> > +     harder with some more flush tests.
> > +
> >   2006-05-19  Vlad Seryakov  <ser...@us...>
> >
> >
> >
> >
> > _______________________________________________
> > naviserver-commits mailing list
> > nav...@li...
> > https://lists.sourceforge.net/lists/listinfo/naviserver-commits
> >
>
> --
> Vlad Seryakov
> 571 262-8608 office
> vl...@cr...
> http://www.crystalballinc.com/vlad/
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

Re: [naviserver-devel] Wiki

[Rick C. <rc...@Kn...>]

I just asked for one using the self-registration page.

-----Original Message-----
From: nav...@li...
[mailto:nav...@li...] On Behalf Of
Bernd Eidenschink
Sent: Wednesday, June 21, 2006 8:55 AM
To: nav...@li...
Subject: Re: [naviserver-devel] Wiki


Rick,

who gave you initially an account (cobbr2) on the Wiki? The Wiki did not
allow=20
changes for people without an account.=20

Bernd.

> Looks like you managed to fix it back to normal, Vlad. Great.  Who is
this
> person "P" (previously "Push" and "Search" who's screwing around?).
>
> Oh -- I'm cobbr2 in case you hadn't guessed --



_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Wiki

[Bernd E. <eid...@we...>]

Rick,

who gave you initially an account (cobbr2) on the Wiki? The Wiki did not allow 
changes for people without an account. 

Bernd.

> Looks like you managed to fix it back to normal, Vlad. Great.  Who is this
> person "P" (previously "Push" and "Search" who's screwing around?).
>
> Oh -- I'm cobbr2 in case you hadn't guessed --

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Zoran V. <zv...@ar...>]

Am 20.06.2006 um 16:21 schrieb Vlad Seryakov:

> On the other
> hand, as we talked a year ago, whoever introduces a feature and nobody
> object in the reasonable time(i remember we agreed on 1 week) feature
> remains in the code until something new/better comes up but that will
> need discussion why to remove it because somebody may use it already
> once introduced. We all added a lot of things that everyone of us need
> and that was one of the reasons of this fork, we needed features  
> and we
> added them.

Allight! Well, you objected immediately... Lets see what Stephen
can add to that.
I yet have to digest the change but will definitely object if
I'm not able to give alternate timeouts for different caches.
Also, I'd keep the cache API as simple as possible.

>
> As for CVS HEAD using in the production, i am comfortable with it and
> that is the reason i am adding new things, i have it running on
> pre-production first, and once the feature stable and working it moves
> to production, because i needed it in the production in the first  
> place
> so i added it.

Good. So there is no so big harm at the moment for you. For ourselves,
we must go and get this stabilized at some point as I'm planning
to get our 2.3 release out and for that I need stable nscache and
nsproxy.

Cheers
Zoran

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Vlad S. <vl...@cr...>]

I agree overall, RFE would help especially in case of large API/internal 
changes so everybody would be prepared of what to expect. On the other 
hand, as we talked a year ago, whoever introduces a feature and nobody 
object in the reasonable time(i remember we agreed on 1 week) feature 
remains in the code until something new/better comes up but that will 
need discussion why to remove it because somebody may use it already 
once introduced. We all added a lot of things that everyone of us need 
and that was one of the reasons of this fork, we needed features and we 
added them.

As for CVS HEAD using in the production, i am comfortable with it and 
that is the reason i am adding new things, i have it running on 
pre-production first, and once the feature stable and working it moves 
to production, because i needed it in the production in the first place 
so i added it.

Of course it does not excuse bad decisions which happen from time to 
time but we have mailing list or RFE or CVS to keep the architecture and 
structure sane.

Zoran Vasiljevic wrote:
> Am 19.06.2006 um 23:30 schrieb Vlad Seryakov:
> 
>> Stephen,
>>
>> I understand you have the "right" vision how to do things but after  
>> long
>> period of time just removing Tcl API functions (cache set/get) is not
>> very polite.
>> It breaks a lot of Tcl code and i am not sure why do you think they  
>> are
>> racey and i need to emulate them in Tcl instead of C.
>>
>> Very frustrating.
>>
>>
> 
> I might add couple of words to the above...
> 
> Normally, the CVS HEAD is not meant to be used for any production work.
> Having said that, I must admit that we do exactly the opposite, i.e.
> we take the HEAD and splice it into our product, as-is. This is not a
> very good idea as this places extreme (stability) burden for new
> developments and things to try out. So I guess we should be doing
> tags/minor releases or whatever more frequently. This way we can leave
> the HEAD volatile and stick to last known workable state.
> A "positive" side effect of using HEAD for production is that we get  
> much
> better "testing"  but this is also not very good, i.e. to test the code
> on the customer sites... Therefore the test-suite should be more  
> elaborate
> and we are adding more and more there, so I believe this is the right  
> way
> to go.
> 
> Over the time I tried to maintain RFE->Discussion->Implementation
> path where one would first inform everybody about what one is going to
> do, and why (the RFE), then go see if there are any voices against or
> if there are some better ideas, especially if the existing API is being
> changed (the Discussion) and finally go implement what is agreed upon
> (the Implementation).
> 
> Now, I understand that this is a kind of buerocracy but it really  
> *helps* resolve
> such issues. In later time I could observe that we all neglected the  
> RFE/Discussion
> part (myself included) which then obviously leads to collisions. So I  
> would suggest
> that we take the RFE's seriously and start to work on some  
> implementation when we
> all come to some reasonable consensus.  This will save lots of our  
> time and efforts.
> 
> The *last* thing I would like to do is to discourage people adding  
> new things and
> ideas to the code by placing excessive organizational burden to it!
> OTOH, adding an RFE allows for others to adjust and comment before  
> the code
> gets comminted and maintained which will be helpful on the mid/long  
> term.
> 
> So, afer all this prose, lets see how we will proceed in this  
> particular case...
> I have no problem in changing our code to adopt to new (better) API.  
> I might have
> problems if I will have to do that every now and then OR if the API  
> does less than
> we're used to w/o offering acceptable workarounds.
> 
> Cheers,
> Zoran
> 
> 
> 
> 
> 
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Zoran V. <zv...@ar...>]

Am 19.06.2006 um 23:30 schrieb Vlad Seryakov:

> Stephen,
>
> I understand you have the "right" vision how to do things but after  
> long
> period of time just removing Tcl API functions (cache set/get) is not
> very polite.
> It breaks a lot of Tcl code and i am not sure why do you think they  
> are
> racey and i need to emulate them in Tcl instead of C.
>
> Very frustrating.
>
>

I might add couple of words to the above...

Normally, the CVS HEAD is not meant to be used for any production work.
Having said that, I must admit that we do exactly the opposite, i.e.
we take the HEAD and splice it into our product, as-is. This is not a
very good idea as this places extreme (stability) burden for new
developments and things to try out. So I guess we should be doing
tags/minor releases or whatever more frequently. This way we can leave
the HEAD volatile and stick to last known workable state.
A "positive" side effect of using HEAD for production is that we get  
much
better "testing"  but this is also not very good, i.e. to test the code
on the customer sites... Therefore the test-suite should be more  
elaborate
and we are adding more and more there, so I believe this is the right  
way
to go.

Over the time I tried to maintain RFE->Discussion->Implementation
path where one would first inform everybody about what one is going to
do, and why (the RFE), then go see if there are any voices against or
if there are some better ideas, especially if the existing API is being
changed (the Discussion) and finally go implement what is agreed upon
(the Implementation).

Now, I understand that this is a kind of buerocracy but it really  
*helps* resolve
such issues. In later time I could observe that we all neglected the  
RFE/Discussion
part (myself included) which then obviously leads to collisions. So I  
would suggest
that we take the RFE's seriously and start to work on some  
implementation when we
all come to some reasonable consensus.  This will save lots of our  
time and efforts.

The *last* thing I would like to do is to discourage people adding  
new things and
ideas to the code by placing excessive organizational burden to it!
OTOH, adding an RFE allows for others to adjust and comment before  
the code
gets comminted and maintained which will be helpful on the mid/long  
term.

So, afer all this prose, lets see how we will proceed in this  
particular case...
I have no problem in changing our code to adopt to new (better) API.  
I might have
problems if I will have to do that every now and then OR if the API  
does less than
we're used to w/o offering acceptable workarounds.

Cheers,
Zoran

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Vlad S. <vl...@cr...>]

Also, what is wrong with per-cache expiration policy, nc_cache_create 
-timeout parameter.
How i can create 2 caches with different expiration settings?

 >Ns_CacheGetConfig removed -- caches are immutable; the current
 > settings can be looked up in the config file.

This is not absolutely true if nothing in the config and i need to find 
out default compiled-in value. GetConfig gives runtime value regardless 
of what is set or not set in the config file. With per-cache timeout 
this makes sense. How limits are supposed to work?

Stephen Deasey wrote:
> Update of /cvsroot/naviserver/naviserver
> In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv27993
> 
> Modified Files:
> 	ChangeLog 
> Log Message:
>         * nsd/fastpath.c:
>         * nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
>         Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> 
>         * tcl/cache.tcl:
>         * nsd/nsconf.c:
>         * nsd/dns.c: Update to new API.
> 
>         * nsd/nsd.h:
>         * nsd/server.c: Remove tcl cache timeout param in anticipation of new
>         limits API.
> 
>         * nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
>         commands. The first three are racey and can be emulated with _eval
>         and the new -force switch.  The new -contents switch to the _stats
>         command replaces _info, discouraging racey check-and-set usage.
> 
>         * nsd/tcltime.c:
>         * nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
>         type. Handy for passing absolute time deadlines to routines which
>         timeout.
> 
>         * include/ns.h:
>         * nsd/cache.c: New Ns_CacheResetStats routine.
> 
>         Ns_CacheGetConfig removed -- caches are immutable; the current
>         settings can be looked up in the config file.
> 
>         Directly expire entries from within all routines which return one,
>         but but only if the value is not null, indicating no concurrent
>         update is in progress.
> 
>         Use Ns_CacheDeleteEntry to remove an entry from the cache without
>         updating the stats. Flush stats should reflect explicit flushes
>         only.
> 
>         Log stats when a cache is destroyed.
> 
>         * nsd/tclcache.c: ns_cache_create now takes a -maxentry option
>         which is the maximum size of an entry allowed in the cache. This
>         prevents one large entry completely emptying an otherwise usefully
>         full cache.
> 
>         The ns_cache_create -timeout switch has been removed in
>         anticipation of a general purpose limits scheme.
> 
>         The -timeout option is now expected to be an absolute time in the
>         future, not an offset from the current time. This is awkward to
>         use manually, but should not be needed with the above mentioned
>         limits scheme. The -ttl option has been renames -expires and is
>         also an absolute time.
> 
>         ns_cache_eval now takes a -force switch which will unconditioanly
>         replace any exiting entry, whether it has expired or not. Replaces
>         _set.
> 
>         ns_cache_stats -content dumps the size and expirey for each
>         entry in the cache. Adding the -reset switch to either mode resets
>         the stats to zero.
> 
>         * tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
>         harder with some more flush tests.
> 
> 
> 
> Index: ChangeLog
> ===================================================================
> RCS file: /cvsroot/naviserver/naviserver/ChangeLog,v
> retrieving revision 1.405
> retrieving revision 1.406
> diff -C2 -d -r1.405 -r1.406
> *** ChangeLog	19 Jun 2006 19:05:47 -0000	1.405
> --- ChangeLog	19 Jun 2006 21:16:38 -0000	1.406
> ***************
> *** 1,2 ****
> --- 1,71 ----
> + 2006-06-19  Stephen Deasey  <sd...@us...>
> + 
> + 	* nsd/fastpath.c:
> + 	* nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
> + 	Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> + 
> + 	* tcl/cache.tcl:
> + 	* nsd/nsconf.c:
> + 	* nsd/dns.c: Update to new API.
> + 
> + 	* nsd/nsd.h:
> + 	* nsd/server.c: Remove tcl cache timeout param in anticipation of new
> + 	limits API.
> + 
> + 	* nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
> + 	commands. The first three are racey and can be emulated with _eval
> + 	and the new -force switch.  The new -contents switch to the _stats
> + 	command replaces _info, discouraging racey check-and-set usage.
> + 
> + 	* nsd/tcltime.c:
> + 	* nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
> + 	type. Handy for passing absolute time deadlines to routines which
> + 	timeout.
> + 
> + 	* include/ns.h:
> + 	* nsd/cache.c: New Ns_CacheResetStats routine.
> + 
> + 	Ns_CacheGetConfig removed -- caches are immutable; the current
> + 	settings can be looked up in the config file.
> + 
> + 	Directly expire entries from within all routines which return one,
> + 	but but only if the value is not null, indicating no concurrent
> + 	update is in progress.
> + 
> + 	Use Ns_CacheDeleteEntry to remove an entry from the cache without
> + 	updating the stats. Flush stats should reflect explicit flushes
> + 	only.
> + 
> + 	Log stats when a cache is destroyed.
> + 
> + 	* nsd/tclcache.c: ns_cache_create now takes a -maxentry option
> + 	which is the maximum size of an entry allowed in the cache. This
> + 	prevents one large entry completely emptying an otherwise usefully
> + 	full cache.
> + 
> + 	The ns_cache_create -timeout switch has been removed in
> + 	anticipation of a general purpose limits scheme.
> + 
> + 	The -timeout option is now expected to be an absolute time in the
> + 	future, not an offset from the current time. This is awkward to
> + 	use manually, but should not be needed with the above mentioned
> + 	limits scheme. The -ttl option has been renames -expires and is
> + 	also an absolute time.
> + 
> + 	ns_cache_eval now takes a -force switch which will unconditioanly
> + 	replace any exiting entry, whether it has expired or not. Replaces
> + 	_set.
> + 
> + 	ns_cache_stats -content dumps the size and expirey for each
> + 	entry in the cache. Adding the -reset switch to either mode resets
> + 	the stats to zero.
> + 
> + 	
> + 
> + 	
> + 
> + 	* tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
> + 	harder with some more flush tests.
> + 
>   2006-05-19  Vlad Seryakov  <ser...@us...>
>   
> 
> 
> 
> _______________________________________________
> naviserver-commits mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-commits
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Vlad S. <vl...@cr...>]

Stephen,

I understand you have the "right" vision how to do things but after long 
period of time just removing Tcl API functions (cache set/get) is not 
very polite.
It breaks a lot of Tcl code and i am not sure why do you think they are 
racey and i need to emulate them in Tcl instead of C.

Very frustrating.


Stephen Deasey wrote:
> Update of /cvsroot/naviserver/naviserver
> In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv27993
> 
> Modified Files:
> 	ChangeLog 
> Log Message:
>         * nsd/fastpath.c:
>         * nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
>         Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> 
>         * tcl/cache.tcl:
>         * nsd/nsconf.c:
>         * nsd/dns.c: Update to new API.
> 
>         * nsd/nsd.h:
>         * nsd/server.c: Remove tcl cache timeout param in anticipation of new
>         limits API.
> 
>         * nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
>         commands. The first three are racey and can be emulated with _eval
>         and the new -force switch.  The new -contents switch to the _stats
>         command replaces _info, discouraging racey check-and-set usage.
> 
>         * nsd/tcltime.c:
>         * nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
>         type. Handy for passing absolute time deadlines to routines which
>         timeout.
> 
>         * include/ns.h:
>         * nsd/cache.c: New Ns_CacheResetStats routine.
> 
>         Ns_CacheGetConfig removed -- caches are immutable; the current
>         settings can be looked up in the config file.
> 
>         Directly expire entries from within all routines which return one,
>         but but only if the value is not null, indicating no concurrent
>         update is in progress.
> 
>         Use Ns_CacheDeleteEntry to remove an entry from the cache without
>         updating the stats. Flush stats should reflect explicit flushes
>         only.
> 
>         Log stats when a cache is destroyed.
> 
>         * nsd/tclcache.c: ns_cache_create now takes a -maxentry option
>         which is the maximum size of an entry allowed in the cache. This
>         prevents one large entry completely emptying an otherwise usefully
>         full cache.
> 
>         The ns_cache_create -timeout switch has been removed in
>         anticipation of a general purpose limits scheme.
> 
>         The -timeout option is now expected to be an absolute time in the
>         future, not an offset from the current time. This is awkward to
>         use manually, but should not be needed with the above mentioned
>         limits scheme. The -ttl option has been renames -expires and is
>         also an absolute time.
> 
>         ns_cache_eval now takes a -force switch which will unconditioanly
>         replace any exiting entry, whether it has expired or not. Replaces
>         _set.
> 
>         ns_cache_stats -content dumps the size and expirey for each
>         entry in the cache. Adding the -reset switch to either mode resets
>         the stats to zero.
> 
>         * tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
>         harder with some more flush tests.
> 
> 
> 
> Index: ChangeLog
> ===================================================================
> RCS file: /cvsroot/naviserver/naviserver/ChangeLog,v
> retrieving revision 1.405
> retrieving revision 1.406
> diff -C2 -d -r1.405 -r1.406
> *** ChangeLog	19 Jun 2006 19:05:47 -0000	1.405
> --- ChangeLog	19 Jun 2006 21:16:38 -0000	1.406
> ***************
> *** 1,2 ****
> --- 1,71 ----
> + 2006-06-19  Stephen Deasey  <sd...@us...>
> + 
> + 	* nsd/fastpath.c:
> + 	* nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
> + 	Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> + 
> + 	* tcl/cache.tcl:
> + 	* nsd/nsconf.c:
> + 	* nsd/dns.c: Update to new API.
> + 
> + 	* nsd/nsd.h:
> + 	* nsd/server.c: Remove tcl cache timeout param in anticipation of new
> + 	limits API.
> + 
> + 	* nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
> + 	commands. The first three are racey and can be emulated with _eval
> + 	and the new -force switch.  The new -contents switch to the _stats
> + 	command replaces _info, discouraging racey check-and-set usage.
> + 
> + 	* nsd/tcltime.c:
> + 	* nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
> + 	type. Handy for passing absolute time deadlines to routines which
> + 	timeout.
> + 
> + 	* include/ns.h:
> + 	* nsd/cache.c: New Ns_CacheResetStats routine.
> + 
> + 	Ns_CacheGetConfig removed -- caches are immutable; the current
> + 	settings can be looked up in the config file.
> + 
> + 	Directly expire entries from within all routines which return one,
> + 	but but only if the value is not null, indicating no concurrent
> + 	update is in progress.
> + 
> + 	Use Ns_CacheDeleteEntry to remove an entry from the cache without
> + 	updating the stats. Flush stats should reflect explicit flushes
> + 	only.
> + 
> + 	Log stats when a cache is destroyed.
> + 
> + 	* nsd/tclcache.c: ns_cache_create now takes a -maxentry option
> + 	which is the maximum size of an entry allowed in the cache. This
> + 	prevents one large entry completely emptying an otherwise usefully
> + 	full cache.
> + 
> + 	The ns_cache_create -timeout switch has been removed in
> + 	anticipation of a general purpose limits scheme.
> + 
> + 	The -timeout option is now expected to be an absolute time in the
> + 	future, not an offset from the current time. This is awkward to
> + 	use manually, but should not be needed with the above mentioned
> + 	limits scheme. The -ttl option has been renames -expires and is
> + 	also an absolute time.
> + 
> + 	ns_cache_eval now takes a -force switch which will unconditioanly
> + 	replace any exiting entry, whether it has expired or not. Replaces
> + 	_set.
> + 
> + 	ns_cache_stats -content dumps the size and expirey for each
> + 	entry in the cache. Adding the -reset switch to either mode resets
> + 	the stats to zero.
> + 
> + 	
> + 
> + 	
> + 
> + 	* tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
> + 	harder with some more flush tests.
> + 
>   2006-05-19  Vlad Seryakov  <ser...@us...>
>   
> 
> 
> 
> _______________________________________________
> naviserver-commits mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-commits
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Cache is broken

[Zoran V. <zv...@ar...>]

Am 14.06.2006 um 12:12 schrieb Zoran Vasiljevic:

> One solution would be to really delete expired entry in
> Ns_CacheFindEntry()
> instead of just ExpireEntry().  This would salvage logic in
> Ns_CacheWaitCreateEntry().
>
> Another solution would be to add new bit in the Entry structure  
> marking
> the structure as expired. Then add new logic in the
> Ns_CacheWaitCreateEntry()
> which would check that bit and act accordingly.

Third solution is NOT to ExpireEntry from within the Ns_CacheFindEntry.
Instead, just record the event (update counters) and return NULL.
This will have a side effect that such entries cannot be directly
flushed (i.e. deleted) until they are either set to a new value
OR until the entire cache is purged.

I have checked-in this change and am satisfied with it, as it
solves my problem. We could invest some more time and perhaps
find more optimal or elegant solution but I will leave this to
Stephen.

Cheers,
Zoran

Re: [naviserver-devel] Cache is broken

[Zoran V. <zv...@ar...>]

Am 07.06.2006 um 16:29 schrieb Stephen Deasey:

>
> I don't have time to try and reproduce the problems from scratch, but
> I'd be happy to fix it if there's a couple of concise test cases.

But I was forced to do so... (customer support)...

Well, I have fixed that spurious "timeout waiting for update:.."
problems I was experiencing. The problem was rather trivial and
could be ranked as "omission" in the Ns_CacheSetValueExpires() code
(look for diffs between 1.8 and 1.9 to see the changes).

Another problem was: ns_cache_exists which returned true
for values that were internally expired. I corrected that
as well (see also below).

Stephen, there is an *architectural* problem between the  
Ns_CacheFindEntry()
call which internally calls ExpireEntry() and Ns_CacheWaitCreateEntry()
which checks the value to non-NULL and waits forever (or for some
time, eventually aborting with timeout).

The API sequence for that is simple:

   call Ns_CacheFindEntry()
     this one will check the entry and eventualy call ExpireEntry()
     which will unset the entry value but NOT delete the entry itself

   call Ns_CacheWaitCreateEntry which will find the entry (as it is
     not deleted) but will wait forever (or timeout) because the entry
     value is empty and nobody is going to set it any more

This is what really happened in my case even before the 1.8 version
of the file, but it was just harder to trigger. In 1.8 it is trivial
to trigger and our app breaks very early there.

Now the question is: how can we fix that?

One solution would be to really delete expired entry in  
Ns_CacheFindEntry()
instead of just ExpireEntry().  This would salvage logic in  
Ns_CacheWaitCreateEntry().

Another solution would be to add new bit in the Entry structure marking
the structure as expired. Then add new logic in the  
Ns_CacheWaitCreateEntry()
which would check that bit and act accordingly.

Please tell me what do you think. If possible "as soon as possible"
as I have some very angry customers chasing me. I'm OK with any of
the proposed changes or with any other as well, as long as it fixes
the problem.

What I COULD NOT verify was any memory-related problems as Vlad is
experiencing. I can take the code thru Purify once more but this will
take me another two days of work.

Cheers,
Zoran

Re: [naviserver-devel] Cache is broken

[Stephen D. <sd...@gm...>]

On 6/7/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 07.06.2006 um 16:29 schrieb Stephen Deasey:
>
> > that some random installed version of the libraries are not used.
> > Otherwise, you wont be testing what you think you are. And the tests
> > get busted, as they are now...
>
> True. This is but the case with all tests since the beginning
> *except* on Linux. On Mac it always tests the installed libs
> instead of the development ones because Mac OSX does not define
> LD_LIBRARY_PATH. Instead it defines DYLD_LIBRARY_PATH!
> So there is more that just nsproxy libs not being correctly tested.
> So we must fix more than anticipated...


Yeah, this has always been the case.  It's not just an nsproxy problem.

LD_LIBRARY_PATH does not override the rpath which is burnt into the
libraries and programs. I think there is a way around this, somehow...

Re: [naviserver-devel] Cache is broken

[Zoran V. <zv...@ar...>]

Am 07.06.2006 um 16:29 schrieb Stephen Deasey:

> that some random installed version of the libraries are not used.
> Otherwise, you wont be testing what you think you are. And the tests
> get busted, as they are now...

True. This is but the case with all tests since the beginning
*except* on Linux. On Mac it always tests the installed libs
instead of the development ones because Mac OSX does not define
LD_LIBRARY_PATH. Instead it defines DYLD_LIBRARY_PATH!
So there is more that just nsproxy libs not being correctly tested.
So we must fix more than anticipated...

I have tried to get a reproducible case of nscache problem outside
of our application but could not. This will take me some more time.

Cheers
Zoran

Re: [naviserver-devel] Cache is broken

[Stephen D. <sd...@gm...>]

On 6/5/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 05.06.2006 um 23:22 schrieb Vlad Seryakov:
>
> > Definitely, i just recompiled NS on working server with recent version
> > and started crashing on ns_cache. This is the showstopper.
>
> At the moment I had to comment-out certain (recent) changes to get
> it working. But this is not a solution. I think Stephen
> would need a reproducible case to be able to nail this one down.
> I have been looking in the sources but I could not see any
> problem there :-(
>
> I will not have time to look into that until the end of this week
> unfortunately.
>
> Cheers
> Zoran


I don't have time to try and reproduce the problems from scratch, but
I'd be happy to fix it if there's a couple of concise test cases.

btw. The tests are busted at the moment -- the nsproxy tests all fail
because they cannot pickup the uninstalled libraries.

Remember, if you configure the server with a non-standard --prefix
then that gets burned into the binaries as a search path for
libraries.  When you run the tests it will link against any installed
libraries, not the ones you think your testing in the source
directory.

This is a bug.  I don't know how to fix it yet :-(  When you're
testing, *always* configure with a bogus prefix such as /tmp/ns so
that some random installed version of the libraries are not used.
Otherwise, you wont be testing what you think you are. And the tests
get busted, as they are now...

Re: [naviserver-devel] Cache is broken

[Zoran V. <zv...@ar...>]

Am 05.06.2006 um 23:22 schrieb Vlad Seryakov:

> Definitely, i just recompiled NS on working server with recent version
> and started crashing on ns_cache. This is the showstopper.

At the moment I had to comment-out certain (recent) changes to get
it working. But this is not a solution. I think Stephen
would need a reproducible case to be able to nail this one down.
I have been looking in the sources but I could not see any
problem there :-(

I will not have time to look into that until the end of this week
unfortunately.

Cheers
Zoran

Re: [naviserver-devel] Cache is broken

[Vlad S. <vl...@cr...>]

Definitely, i just recompiled NS on working server with recent version 
and started crashing on ns_cache. This is the showstopper.

Zoran Vasiljevic wrote:
> Am 05.06.2006 um 04:36 schrieb Vlad Seryakov:
> 
>> Current cache implementation in naviserver is somewhat broken because
>> with recent version i cannot run my app under ns_cache_ and event with
>> old ns_cache module as well, i have crashes inside C level cache API,
>> but i use only Tcl api. Something was changed that influenced even
>> ns_cache because it used underlying C level cache of the core.
>>
>> My app uses simple caches, just ns_cache eval, the difference that  
>> this
>> is Network management system and it runs many many threads at the same
>> time using ns_job, so i am guessing that concurrency is what kills. It
>> works on older versions of NS 4.99.0|1 and AOL 4.0 using ns_cache, i
>> tried to convert it to internal ns_cache_family but it does not  
>> work, it
>> hangs indefinitely, but works with ns_cache up until CVS version.
> 
> I also think it is the concurrency as I regulary have problems
> (which I already described) and cannot point my finger at any
> place. Simulating trouble-cases is also not simple (haven't got
> the chance to do that), yet my application regulary breaks.
> 
> In my case, it is not crashing, rather error due to timeouts
> waiting for lock which normally should not exist.
> 
> I think this will not be easy...
> 
> Cheers
> Zoran
> 
> 
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Cache is broken

[Zoran V. <zv...@ar...>]

Am 05.06.2006 um 04:36 schrieb Vlad Seryakov:

>
> Current cache implementation in naviserver is somewhat broken because
> with recent version i cannot run my app under ns_cache_ and event with
> old ns_cache module as well, i have crashes inside C level cache API,
> but i use only Tcl api. Something was changed that influenced even
> ns_cache because it used underlying C level cache of the core.
>
> My app uses simple caches, just ns_cache eval, the difference that  
> this
> is Network management system and it runs many many threads at the same
> time using ns_job, so i am guessing that concurrency is what kills. It
> works on older versions of NS 4.99.0|1 and AOL 4.0 using ns_cache, i
> tried to convert it to internal ns_cache_family but it does not  
> work, it
> hangs indefinitely, but works with ns_cache up until CVS version.

I also think it is the concurrency as I regulary have problems
(which I already described) and cannot point my finger at any
place. Simulating trouble-cases is also not simple (haven't got
the chance to do that), yet my application regulary breaks.

In my case, it is not crashing, rather error due to timeouts
waiting for lock which normally should not exist.

I think this will not be easy...

Cheers
Zoran

[naviserver-devel] Cache is broken

[Vlad S. <vl...@cr...>]

Hello,

Current cache implementation in naviserver is somewhat broken because 
with recent version i cannot run my app under ns_cache_ and event with 
old ns_cache module as well, i have crashes inside C level cache API, 
but i use only Tcl api. Something was changed that influenced even 
ns_cache because it used underlying C level cache of the core.

My app uses simple caches, just ns_cache eval, the difference that this 
is Network management system and it runs many many threads at the same 
time using ns_job, so i am guessing that concurrency is what kills. It 
works on older versions of NS 4.99.0|1 and AOL 4.0 using ns_cache, i 
tried to convert it to internal ns_cache_family but it does not work, it 
hangs indefinitely, but works with ns_cache up until CVS version.

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] New nsproxy module

[Bernd E. <eid...@we...>]

>> > There's no aolserver module, as far as I know, but there's been
>> > interest.  http://fastcgi.com has details.
>> >
>>
>> I know FastCGI since 1996 already and have done quite a few
>> projects with it. The problem is: i do not know what you can
>> use it for if you have naviserver as-is.


> You could run Ruby on Rails...

PHP?

http://www.caucho.com/resin-3.0/thirdparty/php.xtp
http://phplens.com/phpeverywhere/fastcgi-php
http://www.lighttpd.net/documentation/fastcgi.html

Re: [naviserver-devel] New nsproxy module

[Stephen D. <sd...@gm...>]

On 5/24/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 24.05.2006 um 23:36 schrieb Stephen Deasey:
>
> >
> > There's no aolserver module, as far as I know, but there's been
> > interest.  http://fastcgi.com has details.
> >
>
> I know FastCGI since 1996 already and have done quite a few
> projects with it. The problem is: i do not know what you can
> use it for if you have naviserver as-is.


You could run Ruby on Rails...

Re: [naviserver-devel] New nsproxy module

[Zoran V. <zv...@ar...>]

Am 24.05.2006 um 23:36 schrieb Stephen Deasey:

>
> There's no aolserver module, as far as I know, but there's been
> interest.  http://fastcgi.com has details.
>

I know FastCGI since 1996 already and have done quite a few
projects with it. The problem is: i do not know what you can
use it for if you have naviserver as-is.

>
> Noticed a couple of things in nsproxy:  the name space for Tcl handle
> is global, should be per virtual server.  Could the handle code be
> split out?  It's similar to the stuff in nsd/tclobj.c, although for
> different situations.  Would be good as library code.

I will look into that.

Cheers,
Zoran

Re: [naviserver-devel] New nsproxy module

[Stephen D. <sd...@gm...>]

On 5/20/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 20.05.2006 um 21:41 schrieb Stephen Deasey:
>
> > This is almost a Fast CGI module.  Differences are this uses pipes,
> > while Fast CGI uses sockets (although the code mentions a move to
> > sockets).  Fast CGI produces a complete response for the browser,
> > where as this returns a script result to the calling proc.  But the
> > differences are small.
> >
> > I'm not sure 'proxy' is the best name for this, bundled with a web
> > server as it is...
>
> Well, to be honest, I also was not sure about the "proxy" name.
> How would you name it?


You could call it nsfastcgi  :-)


> About the resemblence to FastCGI I can't say much but this might be.
> I never looked at that code. Is that something bad, I mean the
> resemblence to fastcgi? BTW, where is this module?


There's no aolserver module, as far as I know, but there's been
interest.  http://fastcgi.com has details.


Noticed a couple of things in nsproxy:  the name space for Tcl handle
is global, should be per virtual server.  Could the handle code be
split out?  It's similar to the stuff in nsd/tclobj.c, although for
different situations.  Would be good as library code.

Re: [naviserver-devel] New nsproxy module

[Zoran V. <zv...@ar...>]

Am 20.05.2006 um 21:41 schrieb Stephen Deasey:

> This is almost a Fast CGI module.  Differences are this uses pipes,
> while Fast CGI uses sockets (although the code mentions a move to
> sockets).  Fast CGI produces a complete response for the browser,
> where as this returns a script result to the calling proc.  But the
> differences are small.
>
> I'm not sure 'proxy' is the best name for this, bundled with a web
> server as it is...

Well, to be honest, I also was not sure about the "proxy" name.
How would you name it?

About the resemblence to FastCGI I can't say much but this might be.
I never looked at that code. Is that something bad, I mean the
resemblence to fastcgi? BTW, where is this module?

Cheers
Zoran

Re: [naviserver-devel] New nsproxy module

[Stephen D. <sd...@gm...>]

This is almost a Fast CGI module.  Differences are this uses pipes,
while Fast CGI uses sockets (although the code mentions a move to
sockets).  Fast CGI produces a complete response for the browser,
where as this returns a script result to the calling proc.  But the
differences are small.

I'm not sure 'proxy' is the best name for this, bundled with a web
server as it is...



On 5/13/06, Zoran Vasiljevic <zv...@ar...> wrote:
> Hi!
>
> I have concluded the work of getting the nsproxy module
> from the AOLserver repository into ours.
>
> There have been *numerous* changes and improvements to the
> initial code. See ChangeLog for more details.
> There is also a test-suite and a man-page written in doctools.
> All is under the nsproxy directory which is new so you may
> need to checkout your sandbox.
>
> To get a html or nroff file from ns_proxy.man, please see
> the docs/src/README.doc file.
>
> The sample-config.tcl now includes the nsproxy module.
> Also the test server used for "make test" loads and
> exercises the module. As of my experience, it works fine
> on Mac OSX, Linux and Solaris OS'es.
>
> Please checkoout the new nsproxy and tell me if you get
> any problems with it. We plan to deploy it in one of the next
> releases of our product. Our main motivation to work on that
> was the ability to execute Tcl code in a controllable manner
> outside the server and under different user/group than
> the server itself. Our app runs the nsd as root but at some
> places we need to execute some Tcl code as different user.
> Now, under Unix threading model it is not possible to "impersonate"
> a thread, which is possible only under Windows. For Unix we need
> to get the code executed in a different process, hence I was very
> pleased to see Jim adding this nsproxy module to AOLserver.
> I believe this was just a small rewrite of the db-proxy module.
> But it is in itself a very helpful thing for us. It will hopefully
> be of use for the rest of the Naviserver users.
>
>
> Off-topic:
> At this point it is clear that we need to invest more time in
> getting the docs written. We now have so much new functionality
> built into the server that I start loosing the overview.
> I would encourage everybody to look into the files in doctools
> provided so far and start adding content. In one of the next
> days I will add a template page for each Tcl-API command under
> the docs/src.
>
> Cheers
> Zoran

Re: [naviserver-devel] New nsproxy module

[Zoran V. <zv...@ar...>]

Am 16.05.2006 um 16:00 schrieb Vlad Seryakov:

> Recently i was looking into nsproxy to accomplish an opposite task,  
> my nsd runs as nobody but i need ability to execute some scripts as  
> root but it looks like nsproxy is not designed for this task  
> without hacking nsd: i.e. run nsd as root, spawn nsproxy then  
> manually switch nsd to nobody.

NOW I got it. Seems that SF email delivery is busted...
Anyhow, I belive you can setuid the nsproxy program
which would then execute under root and be able to
lower the privilige to some other user or stay as
root. I never tried that but this should be OK.

Cheers
Zoran