naviserver-devel — Developer discussion

Re: [naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Stephen D. <sd...@gm...>]

On 6/25/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 25.06.2006 um 12:48 schrieb Stephen Deasey:
>
> > What happens if you push but don't pop, maybe an uncaught error?
>
> I was also thinking about:
>
>    ns_logctl register callback ?arg...?
>    ns_logctl unregister callback
>
> This way the push/pop is avoided and the problem of non-symetric
> usage is gone, as callbacks will be added/removed to/from a list
> not a stack.
>
> The callbacks themselves will be called in succession by the underlying
> implementation. The very first (default, always present) callback will
> be a call to write() as in the current implementation. This guarantees
> that log entries will be written into the log file, as today.
> Other callbacks may be called in FIFO or LIFO order (there are some
> thoughts about that below...)
>
> >
> > The very first change in aolserver cvs after being opened was to
> > remove the modlog api. Like most logging implementations, it took both
> > a facility and  a level.  Ns_Log only takes a level.
> >
> > How about adding a 'facility' arg back?
>
> Would that mean ns_log will be something like:
>
>     ns_log ?-facility whatever? severity arg ?arg ...?
>
> ??
>
> Actually, at the moment I was not thinking about facilities although
> it might
> be considered as well. My primary motivation behind this is to get
> better
> *organization* of the existing log information (severity, message,
> timestamp
> process-id/thread-id).
>
> At the moment we have everything line-wise in one log file. This is
> good as
> you always know where to look, and there is a time-related order
> there as well.
> But, disadvantage is: one can easily get lost if you have 100'000 of
> lines of
> log info generated by different modules all doing logs for the same
> "job".
> By adding some additional log "sinks" I can channel log information into
> appropriate "per-job" log files. This retains the global log file but
> adds
> more *order* by separating "job-related" logs into corresponding per-
> job files.
>
> For example:
>
>    proc joblog {level timestamp message jobId} {...}
>    set id the_job_id_or_handle
>    ns_logctl register joblog $id
>
> So when somebody says:
>
>    ns_log info "This is the info"
>
> the log facility will emit a log line into the log file
> (as it does now), but will in addition call:
>
>     joblog info 1151234419:0000000 "This is the Info=10"
> the_job_id_or_handle
>
> Depending on the passed user argument(s) (in the above case
> the_job_id_or_handle) the "joblog" will know what to do, i.e.
> where and how to produce log entry.
> The "timestamp" argument will include both seconds and microseconds
> (we already have Tcl_Obj type for that).
>
> When the callback is not needed any more, it can be removed
> by calling:
>
>    ns_logctl unregister joblog
>
> One can have any number of callbacks registered. All of them
> will be entered in a list and invoked one after another
> with the same level, timestamp and message arguments, but with
> different user argument(s).
>
> One additional thing we MAY do is to invent a callback return
> value to signalize "continue" or "break" in the similar sense
> as filters are doing. This way any callback can forbid lower
> level callbacks to be invoked. This is just an idea and it
> needs careful thinking though. For example: what does this mean
> for the standard logfile sink? If an late callback returns break
> would that cut-off the standard log sink or just registered
> callbacks? Of course, this will require us to defined order of
> invocation i.e. LIFO or FIFO as well...
>
> I hope I managed to convey the *general* idea to you...


Got it.

It might still be a good idea to add 'facility' as a parameter.  e.g.
your registering a callback and passing an arg 'id'.  Your callback is
distinguishing what to do depending on the arg -- it get's called no
matter what the level is.

If you add facility, you could:

  ns_logctl register $facility $priority joblog ?arg ...?

The filtering would be done by the server, rather than the callback.
You're focusing on sinks here, but you need a way to distinguish what
goes where, and facility levels seem like such a standard thing.  It
would enable you to turn on verbose logging for just one area of code,
which would reduce log spam.

Take a look at log4j by the apache project.  This is very popular and
seems to have inspired a lot of imitators such as log4py, log4c etc.
Maybe log4c has a better API than the old Ns_ModLog stuff.

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Stephen D. <sd...@gm...>]

On 6/22/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 21.06.2006 um 19:01 schrieb Stephen Deasey:
>
> > Look at the aolserver head, at nsd/limits.c.  A 'limits' is a
> > collection of setting such as max upload size, timeout etc.  You
> > create one or more of these then bind them to a URL, just like
> > ns_register_proc.
>
> But Stepehn, lets look this from another perspective...
>
> We use NS to about 80-85% as a general purpose application
> server and to the rest as webserver. For us, binding a
> limit to an URL does not mean much. Instead we have divided
> our code in multiple C and Tcl modules we plug into the server
> and each module has its own configuration section where we draw
> defaults/limits etc, falling back to defaults built into the server.
>
> WFor example, we use -timoeut option for generating cache to be
> able to create a cache per-module with per-module defaults.
> This is of course different for every module. How are we to solve
> this with the scheme you are proposing or with the a'la AOLserver
> limits infrastructure (per URL based) ???
> Can we "bind" those limits to whatever we want? How is this working
> in detail? Do you have a executive summary (in a nutshell) for me
> or do I have to go to AS sources and learn all that from there?


You chopped out the description from your quote above  :-)

The idea is to use the idea of 'limits', as aolserver does, but to use
the timeout limit in a broader way.

The aolserver timeout limit refers only to the amount of time
connections spend waiting for conn threads before being dropped.  The
idea here is to extend that and say the timeout is the total amount of
time the connection should spend running.

A single page request can often have more than one call to ns_cache_,
ns_httpget, ns_prcxy, ns_job, and so on.  You can set the timeout for
each call to ns_cache to be 30 seconds.  If the first call completes
immediately, but the second takes 31 seconds, your page fails.
However, by definition you were quite prepared to wait 58 seconds for
the page to complete.

When a connection is accepted the time is logged.  Depending on the
URL, a particular limit is selected, say 30 seconds.  This is added to
the accept time to get a time in the future beyond which you're not
prepared to wait on a condition variable used by any of the calls
mentioned above.

You mentioned modules with different timeout values.  It depends how
you use them.  If your accessing a cache from a scheduled proc, then
you don't have a URL to get a limit, so perhaps you want some
per-cache settings.  I guess you might insist that even with the URL
limit available, one cache absolutely needs to wait significantly
longer than another and you're prepared to accept the limitations
mentioned above that happen when you add a series of timeouts
together.

Either way, per-cache timeouts seem reasonable.  But this is a Tcl
thing.  There's a new Tcl cache structure for storing this info.  Tcl
and C are so different that it's just a limitation to force this into
the C cache routines.

Here's a question though: if you have URL limits and per cache limits,
which takes precedence?

Re: [naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Zoran V. <zv...@ar...>]

Am 25.06.2006 um 11:30 schrieb Gustaf Neumann:

> Well, i would not base the whole design decision on this fact, since
> also from the c-level one could call the tcl/xotcl log functions

Well, this is not the only reason (see below). I also believe it is
cleaner to have C-code not dependent (if possible) to any Tcl code.
By getting apropriate "hooks" into the log facility, one can handle
just about any situation with relative ease being it just C or just
Tcl or a combination.

At the moment, the way of expanding the log facility is pretty
inflexible. Currently you can overload internal log and flush
calls only. If one module redefines those log calls, another module
can junk its modifications by plugging into the same hook. Also, the
"default" operation of the log-facility is from that point, entirely
dependent on the overloading module. This is not very nice and the
idea is to improve on that as well. I will however not change that
behaviour as I do not really know wether somebody is using it or not.
What I'd do is to change the default way of handling the log entries
by adding a list of callbacks one can add and/or remote to/from.
The callbacks themselves can be written in C or in Tcl similar to the
filter callbacks we already use. This way, the current log processing
(a call to write(2...) will be implemented as yet-another-callback so
the entire beast will be completely generic. By addind more callbacks
one will not influence/replace the default processing, rather expand it!

I hope to get some code pretty soon and I will post diffs to
the list. This will then be much easier to understand.

Cheers
Zoran


  

Re: [naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Zoran V. <zv...@ar...>]

Am 25.06.2006 um 12:48 schrieb Stephen Deasey:

> What happens if you push but don't pop, maybe an uncaught error?

I was also thinking about:

   ns_logctl register callback ?arg...?
   ns_logctl unregister callback

This way the push/pop is avoided and the problem of non-symetric
usage is gone, as callbacks will be added/removed to/from a list
not a stack.

The callbacks themselves will be called in succession by the underlying
implementation. The very first (default, always present) callback will
be a call to write() as in the current implementation. This guarantees
that log entries will be written into the log file, as today.
Other callbacks may be called in FIFO or LIFO order (there are some
thoughts about that below...)

>
> The very first change in aolserver cvs after being opened was to
> remove the modlog api. Like most logging implementations, it took both
> a facility and  a level.  Ns_Log only takes a level.
>
> How about adding a 'facility' arg back?

Would that mean ns_log will be something like:

    ns_log ?-facility whatever? severity arg ?arg ...?

??

Actually, at the moment I was not thinking about facilities although =20
it might
be considered as well. My primary motivation behind this is to get =20
better
*organization* of the existing log information (severity, message, =20
timestamp
process-id/thread-id).

At the moment we have everything line-wise in one log file. This is =20
good as
you always know where to look, and there is a time-related order =20
there as well.
But, disadvantage is: one can easily get lost if you have 100'000 of =20
lines of
log info generated by different modules all doing logs for the same =20
"job".
By adding some additional log "sinks" I can channel log information into
appropriate "per-job" log files. This retains the global log file but =20=

adds
more *order* by separating "job-related" logs into corresponding per-=20
job files.

For example:

   proc joblog {level timestamp message jobId} {...}
   set id the_job_id_or_handle
   ns_logctl register joblog $id

So when somebody says:

   ns_log info "This is the info"

the log facility will emit a log line into the log file
(as it does now), but will in addition call:

    joblog info 1151234419:0000000 "This is the Info=10" =20
the_job_id_or_handle

Depending on the passed user argument(s) (in the above case
the_job_id_or_handle) the "joblog" will know what to do, i.e.
where and how to produce log entry.
The "timestamp" argument will include both seconds and microseconds
(we already have Tcl_Obj type for that).

When the callback is not needed any more, it can be removed
by calling:

   ns_logctl unregister joblog

One can have any number of callbacks registered. All of them
will be entered in a list and invoked one after another
with the same level, timestamp and message arguments, but with
different user argument(s).

One additional thing we MAY do is to invent a callback return
value to signalize "continue" or "break" in the similar sense
as filters are doing. This way any callback can forbid lower
level callbacks to be invoked. This is just an idea and it
needs careful thinking though. For example: what does this mean
for the standard logfile sink? If an late callback returns break
would that cut-off the standard log sink or just registered
callbacks? Of course, this will require us to defined order of
invocation i.e. LIFO or FIFO as well...

I hope I managed to convey the *general* idea to you...

Cheers,
Zoran

Re: [naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Stephen D. <sd...@gm...>]

What happens if you push but don't pop, maybe an uncaught error?

The very first change in aolserver cvs after being opened was to
remove the modlog api. Like most logging implementations, it took both
a facility and  a level.  Ns_Log only takes a level.

How about adding a 'facility' arg back?


On 6/22/06, Zoran Vasiljevic <zv...@ar...> wrote:
> Hi!
>
> I have a need of adding more features to the existing
> ns_log facility.
>
> We have tons of code emitting ns_log messages into the
> server logfile. But as everything ends up in one single
> file, it is somethimes extremely difficult to analyze it
> as you may end up in staring at 100'000s of lines.
>
> Our application generates jobs. Each jobs uses modules
> and modules log things (i.e calls ns_log). What we'd
> need is a way to "teach" the ns_log command that it is
> emitting log in the context of the job X and perhaps
> route log data to a specific log file. This way we could
> group the log entries that correspond to one job into one
> per-job file. This is far easier to analyze and debug, of course.
>
> What I have in mind is a kind of push/pop mechansim where
> I could do something like:
>
>     ns_logctl push <mylogprocedure1>
>     ...
>     ns_logctl push <mylogprocedure2>
>     ...
>     ns_logctl pop
>     ...
>     ns_logctl pop
>
> This would instruct the ns_log facility to invoke registered
> callbacks one by one. Each callback will do something and
> all will eventually end up at the low-level common-denominator
> being the current log file. Every callback might/could signalize
> two states: continue or break. According to that, the next
> registered call will be called (continue) or not (break).
>
> The callbacks themselves would be just Tcl code which will be
> prepared by the caller and "ns_logctl push"'ed on the stack
> of registered handlers.
>
> Now, in order to do that I will have to modify ns_logctl command
> and modify internal operation of ns_log to walk in LIFO fashion
> list of registered handlers and invoke each one of them in
> succession.
>
> Are there any better ideas how to achieve this? If not, is everybody
> OK with the proposed change?
>
> Cheers,
> Zoran
>
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

Re: [naviserver-devel] PHP module

[Bernd E. <eid...@we...>]

Am Samstag, 24. Juni 2006 17:36 schrieb Vlad Seryakov:
> I just commited PHP module, initially it was based on old aolserver
> phpmodule but i had to rewrite it completely for Naviserver and new PHP.
> I tested it with Drupal, and couple of webmail packages, works fine with
> PostgreSQL and sqlite.
> Just letting all know if anyone is interested.

That's really great news, Vlad! I'll give it a try with our PHP CMS!

Bernd.

Re: [naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Gustaf N. <ne...@wu...>]

Zoran Vasiljevic schrieb:
> Am 22.06.2006 um 22:10 schrieb Zoran Vasiljevic:
>
>   
>> But as Gustaf already suggested, one can wrap the ns_log
>> in Tcl and do whatever one will from the Tcl level anyway.
>> I will have to see if this will be sufficient.
>>     
>
> It's not :-(
>
> The problem is: we also have whole lotta C-modules
> calling Ns_Log() C-API hence pure wrapping of the
> Tcl ns_log call will just partly help.
>   
Well, i would not base the whole design decision on this fact, since
also from the c-level one could call the tcl/xotcl log functions. As you
said, speed should not be a major issue, having a flexible log environment
with easy filtering for sessions/users/modules/... and different levels of
detail is very conveniant. This should not be an argument against your
proposal, since it won't prohibit to have the tcl-layer-flexibility on top
of it as well. My point was not to rush to early to c....

-gustaf

Re: [naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Zoran V. <zv...@ar...>]

Am 22.06.2006 um 22:10 schrieb Zoran Vasiljevic:

> But as Gustaf already suggested, one can wrap the ns_log
> in Tcl and do whatever one will from the Tcl level anyway.
> I will have to see if this will be sufficient.

It's not :-(

The problem is: we also have whole lotta C-modules
calling Ns_Log() C-API hence pure wrapping of the
Tcl ns_log call will just partly help.

I have been looking at the current code and indeed
there are hooks in the log machinery to overload the
log and log-flush calls. But it is either/or stuff.
So you either use your own version OR you use the built-in
version. What I would like to do is to expand this to
push/pop any number of calls and execute them in LIFO
fashion. The ns_logctl can be used to push the log and
flush functions implemented as Tcl callbacks and there
should be equivalent C-API for that as well.

I will examine this in detail today and see if I can
have a working version to check out. This is of very
high importance to us hence you may count on seeing
some results pretty soon...

Cheers
Zoran

Re: [naviserver-devel] PHP module

[Zoran V. <zv...@ar...>]

Am 24.06.2006 um 17:36 schrieb Vlad Seryakov:

> I just commited PHP module, initially it was based on old aolserver
> phpmodule but i had to rewrite it completely for Naviserver and new  
> PHP.
> I tested it with Drupal, and couple of webmail packages, works fine  
> with
> PostgreSQL and sqlite.
> Just letting all know if anyone is interested.
>

Great! I've been also thinking about adding FCGI by rewriting
the nsproxy module (as Stephen suggested). As we do not really
need it, it would be just a gift to anybody who might have usage
for such thing.

Cheers,
Zoran

[naviserver-devel] PHP module

[Vlad S. <vl...@cr...>]

I just commited PHP module, initially it was based on old aolserver 
phpmodule but i had to rewrite it completely for Naviserver and new PHP. 
I tested it with Drupal, and couple of webmail packages, works fine with 
PostgreSQL and sqlite.
Just letting all know if anyone is interested.


-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Runtime config load over HTTP

[Vlad S. <vl...@cr...>]

Good point

Bernd Eidenschink wrote:
> Hi Vlad,
> 
> I like the idea but I would abstain from voting, because I think I personally 
> would write a shellscript for it and make use of "wget" or "curl". Those 
> tools offer tons of options, e.g.
> - http, https, ftp (active/passive mode) support
> - certificates, passwords; http digest authentication
> - http proxy support
> - tls, ssl
> - different retry and timeout parameters
> - server response can be logged etc.
> So this would not have to be maintained in the core. You need some logic 
> anyway that deals with failed connections, so this is - in my opinion - the 
> part of the startup script.
> 
> BTW, such a script could be placed into the contrib section (Would be nice 
> anyway to have examples for different distros, I will soon add one for SuSE)
> 
> No vote against it from me, but I would implement it with external tools.
> 
>> In case of multiple servers sometimes it it is easier to maintain and
>> provision when config files are stored locally. I used to have server
>> software that load config file on startup over HTTP and i would see that
>> a usefull feature for naviserver as well. Image that you start it as
>>
>> nsd -t http://server:80/nsd.tcl
>>
>> and it will get the config file from the central server that will
>> provide what is necessary, even on-the fly generating some config
>> parameters. We have HTTP facility in the core, so it is easy to call it
>> to retrieve the config and continue as usual.
>>
>> Of course there is a posibility that it will not be able to connect and
>> thus will not be able to run but this can be handled by keeping last
>> copy of config and load it in case of network issues.
> 
> 
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Runtime config load over HTTP

[Bernd E. <eid...@we...>]

Hi Vlad,

I like the idea but I would abstain from voting, because I think I personally 
would write a shellscript for it and make use of "wget" or "curl". Those 
tools offer tons of options, e.g.
- http, https, ftp (active/passive mode) support
- certificates, passwords; http digest authentication
- http proxy support
- tls, ssl
- different retry and timeout parameters
- server response can be logged etc.
So this would not have to be maintained in the core. You need some logic 
anyway that deals with failed connections, so this is - in my opinion - the 
part of the startup script.

BTW, such a script could be placed into the contrib section (Would be nice 
anyway to have examples for different distros, I will soon add one for SuSE)

No vote against it from me, but I would implement it with external tools.

> In case of multiple servers sometimes it it is easier to maintain and
> provision when config files are stored locally. I used to have server
> software that load config file on startup over HTTP and i would see that
> a usefull feature for naviserver as well. Image that you start it as
>
> nsd -t http://server:80/nsd.tcl
>
> and it will get the config file from the central server that will
> provide what is necessary, even on-the fly generating some config
> parameters. We have HTTP facility in the core, so it is easy to call it
> to retrieve the config and continue as usual.
>
> Of course there is a posibility that it will not be able to connect and
> thus will not be able to run but this can be handled by keeping last
> copy of config and load it in case of network issues.

[naviserver-devel] Runtime config load over HTTP

[Vlad S. <vl...@cr...>]

I have one more proposition to make:

In case of multiple servers sometimes it it is easier to maintain and 
provision when config files are stored locally. I used to have server 
software that load config file on startup over HTTP and i would see that 
a usefull feature for naviserver as well. Image that you start it as

nsd -t http://server:80/nsd.tcl

and it will get the config file from the central server that will 
provide what is necessary, even on-the fly generating some config 
parameters. We have HTTP facility in the core, so it is easy to call it 
to retrieve the config and continue as usual.

Of course there is a posibility that it will not be able to connect and 
thus will not be able to run but this can be handled by keeping last 
copy of config and load it in case of network issues.


-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Zoran V. <zv...@ar...>]

Am 21.06.2006 um 19:01 schrieb Stephen Deasey:

> Look at the aolserver head, at nsd/limits.c.  A 'limits' is a
> collection of setting such as max upload size, timeout etc.  You
> create one or more of these then bind them to a URL, just like
> ns_register_proc.

But Stepehn, lets look this from another perspective...

We use NS to about 80-85% as a general purpose application
server and to the rest as webserver. For us, binding a
limit to an URL does not mean much. Instead we have divided
our code in multiple C and Tcl modules we plug into the server
and each module has its own configuration section where we draw
defaults/limits etc, falling back to defaults built into the server.

WFor example, we use -timoeut option for generating cache to be
able to create a cache per-module with per-module defaults.
This is of course different for every module. How are we to solve
this with the scheme you are proposing or with the a'la AOLserver
limits infrastructure (per URL based) ???
Can we "bind" those limits to whatever we want? How is this working
in detail? Do you have a executive summary (in a nutshell) for me
or do I have to go to AS sources and learn all that from there?

Cheers,
Zoran

Re: [naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Zoran V. <zv...@ar...>]

Am 22.06.2006 um 20:26 schrieb Vlad Seryakov:

> Go ahead if it will not break "simple" logging and ns_log behaviour

It wont! But as Gustaf already suggested, one can wrap the ns_log
in Tcl and do whatever one will from the Tcl level anyway.
I will have to see if this will be sufficient. If yes, then I'll
need not touch the C-level code, which is great as it won't destabilize
the code. OTOH, I may find it not feasible in which case I will have
to dig inside. In that case, I will add to and not remove nor change
the default behaviour that everybody is using This is clear.

Cheers,
Zoran

Re: [naviserver-devel] [Fwd: RFE: Expanding ns_log/ns_logctl]

[Zoran V. <zv...@ar...>]

Am 22.06.2006 um 20:08 schrieb Gustaf Neumann:

> Hi Zoran,
>
> I use the following methods such i can put into every object/class
> messages
>
>    my log "-- item_id = $item_id"
>
> and it is prefixed with object method etc. oacs has as well session  
> ids, which
> can be used...

Actually not a bad idea indeed! I would have to
redefine the ns_log at the Tcl level... This might
be even simpler and performance is not the issue.
Well, I think I will give this a serious thought!

Cheers,
Zoran

Re: [naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Vlad S. <vl...@cr...>]

Go ahead if it will not break "simple" logging and ns_log behaviour

Zoran Vasiljevic wrote:
> Hi!
> 
> I have a need of adding more features to the existing
> ns_log facility.
> 
> We have tons of code emitting ns_log messages into the
> server logfile. But as everything ends up in one single
> file, it is somethimes extremely difficult to analyze it
> as you may end up in staring at 100'000s of lines.
> 
> Our application generates jobs. Each jobs uses modules
> and modules log things (i.e calls ns_log). What we'd
> need is a way to "teach" the ns_log command that it is
> emitting log in the context of the job X and perhaps
> route log data to a specific log file. This way we could
> group the log entries that correspond to one job into one
> per-job file. This is far easier to analyze and debug, of course.
> 
> What I have in mind is a kind of push/pop mechansim where
> I could do something like:
> 
>     ns_logctl push <mylogprocedure1>
>     ...
>     ns_logctl push <mylogprocedure2>
>     ...
>     ns_logctl pop
>     ...
>     ns_logctl pop
> 
> This would instruct the ns_log facility to invoke registered
> callbacks one by one. Each callback will do something and
> all will eventually end up at the low-level common-denominator
> being the current log file. Every callback might/could signalize
> two states: continue or break. According to that, the next
> registered call will be called (continue) or not (break).
> 
> The callbacks themselves would be just Tcl code which will be
> prepared by the caller and "ns_logctl push"'ed on the stack
> of registered handlers.
> 
> Now, in order to do that I will have to modify ns_logctl command
> and modify internal operation of ns_log to walk in LIFO fashion
> list of registered handlers and invoke each one of them in
> succession.
> 
> Are there any better ideas how to achieve this? If not, is everybody
> OK with the proposed change?
> 
> Cheers,
> Zoran
> 
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

[naviserver-devel] RFE: Expanding ns_log/ns_logctl

[Zoran V. <zv...@ar...>]

Hi!

I have a need of adding more features to the existing
ns_log facility.

We have tons of code emitting ns_log messages into the
server logfile. But as everything ends up in one single
file, it is somethimes extremely difficult to analyze it
as you may end up in staring at 100'000s of lines.

Our application generates jobs. Each jobs uses modules
and modules log things (i.e calls ns_log). What we'd
need is a way to "teach" the ns_log command that it is
emitting log in the context of the job X and perhaps
route log data to a specific log file. This way we could
group the log entries that correspond to one job into one
per-job file. This is far easier to analyze and debug, of course.

What I have in mind is a kind of push/pop mechansim where
I could do something like:

    ns_logctl push <mylogprocedure1>
    ...
    ns_logctl push <mylogprocedure2>
    ...
    ns_logctl pop
    ...
    ns_logctl pop

This would instruct the ns_log facility to invoke registered
callbacks one by one. Each callback will do something and
all will eventually end up at the low-level common-denominator
being the current log file. Every callback might/could signalize
two states: continue or break. According to that, the next
registered call will be called (continue) or not (break).

The callbacks themselves would be just Tcl code which will be
prepared by the caller and "ns_logctl push"'ed on the stack
of registered handlers.

Now, in order to do that I will have to modify ns_logctl command
and modify internal operation of ns_log to walk in LIFO fashion
list of registered handlers and invoke each one of them in
succession.

Are there any better ideas how to achieve this? If not, is everybody
OK with the proposed change?

Cheers,
Zoran

Re: [naviserver-devel] Wiki

[Rick C. <rc...@Kn...>]

I'll do what I can.  We use AOLServer 3.4.2 as a publish/subscribe app =
server, coding our services in TCL and C++. We haven't moved to the 4.x =
platform because we wrote some of the things we needed against the 3.4.2 =
conn interface (enhancing them some), so it's a bit painful to take the =
upgrade.  I've desperately wanted the new stuff, but haven't had the =
luxury of getting going with it.

Right now, I haven't even taken the head revision. I just stay on the =
lists and drool. All I can commit to now is occasionally checking the =
wiki for interlopers in skimpy Unterw=E4sche :-)

-- ReC


-----Original Message-----
From: nav...@li... =
[mailto:nav...@li...] On Behalf Of =
Zoran Vasiljevic
Sent: Wednesday, June 21, 2006 12:31 PM
To: nav...@li...
Subject: Re: [naviserver-devel] Wiki


Am 21.06.2006 um 17:57 schrieb Rick Cobb:

> I just asked for one using the self-registration page.

Do you have any interest in joining the development team?
We have forked from AOLserver in order to get a more
versatile high-performance web and application server
using Tcl as scripting language. Currently we have 4
active commiters.

Cheers,
Zoran

All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications =
in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D107521&bid=3D248729&dat=3D=
121642
_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Wiki

[Zoran V. <zv...@ar...>]

Am 21.06.2006 um 17:57 schrieb Rick Cobb:

> I just asked for one using the self-registration page.

Do you have any interest in joining the development team?
We have forked from AOLserver in order to get a more
versatile high-performance web and application server
using Tcl as scripting language. Currently we have 4
active commiters.

Cheers,
Zoran

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Zoran V. <zv...@ar...>]

Am 21.06.2006 um 20:06 schrieb Vlad Seryakov:

>
> By reading this i got an impression that i should expect in the future
> one day that CVS HEAD is completely re-written, all parts that you  
> think
> are
> unnecessary or not correspond to your requirements are removed. And  
> this
> can happen even without warning. Looks like you have big plans for all
> parts already. You should have asked us before removing parts that are
> in the source for several months already.
>
> I would suggest for all of us to define how we do development and
> respect each others rights and ability to extend server.

Slowly, slowly...

The fact that Stephen has plans and ideas is the *best* that we can get!
You also contribute *a lot* and all is going fine actually!
This is the motor. So we should keep that motor running under all
circumstances! What you are suggesting, I believe, is to keep the power
"controllable" and I'm sure everybody thinks the same way.

I love to see additions/alternations of the existing code. If nothing
changes then everbody looses. What I would only require from anybody
contributing his valuable time and knowledge to the project is to be
little bit more *expressive* in terms of communicating what/when/why...

I personally do not care if the entire code is rewriten upside down
as long as it serves the purpose.

Cheers,
Zoran

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Vlad S. <vl...@cr...>]

> We talked about this before.  If I remember correctly, you didn't
> disagree with me that the commands were racy, you said you code base
> was large and you didn't have time to figure out what you really
> needed.
> More and more unnecessary code is being added.  Things are getting out
> of control.  However, you'll notice that I did not just remove
> everything.  With the new -force switch to ns_cache_eval, _set, _get,
> and _info can be trivially emulated in a couple of lines of Tcl, if
> you want that kind of racy behaviour.

> Everything is of course still available via the C api, and I'd
> recommend taking a look at what your real requirements are and
> building accordingly.  If it can be generalised, we can get it into
> the core.
> 
> The timeout in the aolserver implementation refers to just the time
> waiting for a conn thread.  I'm going to extend that to mean total
> time allowed for the conn to run.
> 
> To get this working, Ns_Conn structs need to be allocated in driver.c,
> not queue.c as they are now.  driver.c is looking a little scary...
> 

By reading this i got an impression that i should expect in the future 
one day that CVS HEAD is completely re-written, all parts that you think 
are
unnecessary or not correspond to your requirements are removed. And this 
can happen even without warning. Looks like you have big plans for all 
parts already. You should have asked us before removing parts that are 
in the source for several months already.

I would suggest for all of us to define how we do development and 
respect each others rights and ability to extend server.

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Vlad S. <vl...@cr...>]

> Nothing particularly, but there's a better way to do it:


There are multiple ways to do it but it does not mean that server should 
allow only one way for developer. This is the reason why projects fork, 
if it is not flexible enough and not willing to be extended enough.

 >
>Look at the aolserver head, at nsd/limits.c.  A 'limits' is a
>collection of setting such as max upload size, timeout etc.  You
>create one or more of these then bind them to a URL, just like
>ns_register_proc.


May be limits is a good idea for Url based requests but for server side 
applications how i am supposed to use ns_cache with different timeout 
settings, Do not tell me emulate this via url procs. Cache is generic 
API which should not be bound to URL processing at all. AOLserver does 
it but it looks like the aolserver server is dedicated for URL 
processing only, as webserver only, that was the reason of forking to 
make naviserver more generic and feature-full. I will not agree to 
follow Aolserver path.


PS: Also, if you are not be at the computer for along time do not commit 
big patches or big changes, now everybody is waiting for you even to 
discuss things and we are all stuck.

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Vlad S. <vl...@cr...>]

> 
> We talked about this before.  If I remember correctly, you didn't
> disagree with me that the commands were racy, you said you code base
> was large and you didn't have time to figure out what you really
> needed.

That was about tls not cache

> More and more unnecessary code is being added.  Things are getting out
> of control.  However, you'll notice that I did not just remove
> everything.  With the new -force switch to ns_cache_eval, _set, _get,
> and _info can be trivially emulated in a couple of lines of Tcl, if
> you want that kind of racy behaviour.

Here is the thing, if i need to test for entry existence in the cache i 
need touse ns_cache_eval which will create this entry, same with
ns_cache_get, why do i have to create it when i want to test existence 
only, info exists does not create Tcl variables

And why ns_cache_eval -force is not racy but than ns_cache_set is if 
they do the same thing, why cryptic API flags instead of clear 
functions. If have them for nsv_ for example.

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] [naviserver-commits] naviserver ChangeLog, 1.405, 1.406

[Stephen D. <sd...@gm...>]

On 6/19/06, Vlad Seryakov <vl...@cr...> wrote:
> Stephen,
>
> I understand you have the "right" vision how to do things but after long
> period of time just removing Tcl API functions (cache set/get) is not
> very polite.
> It breaks a lot of Tcl code and i am not sure why do you think they are
> racey and i need to emulate them in Tcl instead of C.
>
> Very frustrating.


We talked about this before.  If I remember correctly, you didn't
disagree with me that the commands were racy, you said you code base
was large and you didn't have time to figure out what you really
needed.

More and more unnecessary code is being added.  Things are getting out
of control.  However, you'll notice that I did not just remove
everything.  With the new -force switch to ns_cache_eval, _set, _get,
and _info can be trivially emulated in a couple of lines of Tcl, if
you want that kind of racy behaviour.

Everything is of course still available via the C api, and I'd
recommend taking a look at what your real requirements are and
building accordingly.  If it can be generalised, we can get it into
the core.


Sorry this is hurried.  I don't have 'net access.  I'll be travelling
for the next couple of weeks.


> Stephen Deasey wrote:
> > Update of /cvsroot/naviserver/naviserver
> > In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv27993
> >
> > Modified Files:
> >       ChangeLog
> > Log Message:
> >         * nsd/fastpath.c:
> >         * nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
> >         Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> >
> >         * tcl/cache.tcl:
> >         * nsd/nsconf.c:
> >         * nsd/dns.c: Update to new API.
> >
> >         * nsd/nsd.h:
> >         * nsd/server.c: Remove tcl cache timeout param in anticipation of new
> >         limits API.
> >
> >         * nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
> >         commands. The first three are racey and can be emulated with _eval
> >         and the new -force switch.  The new -contents switch to the _stats
> >         command replaces _info, discouraging racey check-and-set usage.
> >
> >         * nsd/tcltime.c:
> >         * nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
> >         type. Handy for passing absolute time deadlines to routines which
> >         timeout.
> >
> >         * include/ns.h:
> >         * nsd/cache.c: New Ns_CacheResetStats routine.
> >
> >         Ns_CacheGetConfig removed -- caches are immutable; the current
> >         settings can be looked up in the config file.
> >
> >         Directly expire entries from within all routines which return one,
> >         but but only if the value is not null, indicating no concurrent
> >         update is in progress.
> >
> >         Use Ns_CacheDeleteEntry to remove an entry from the cache without
> >         updating the stats. Flush stats should reflect explicit flushes
> >         only.
> >
> >         Log stats when a cache is destroyed.
> >
> >         * nsd/tclcache.c: ns_cache_create now takes a -maxentry option
> >         which is the maximum size of an entry allowed in the cache. This
> >         prevents one large entry completely emptying an otherwise usefully
> >         full cache.
> >
> >         The ns_cache_create -timeout switch has been removed in
> >         anticipation of a general purpose limits scheme.
> >
> >         The -timeout option is now expected to be an absolute time in the
> >         future, not an offset from the current time. This is awkward to
> >         use manually, but should not be needed with the above mentioned
> >         limits scheme. The -ttl option has been renames -expires and is
> >         also an absolute time.
> >
> >         ns_cache_eval now takes a -force switch which will unconditioanly
> >         replace any exiting entry, whether it has expired or not. Replaces
> >         _set.
> >
> >         ns_cache_stats -content dumps the size and expirey for each
> >         entry in the cache. Adding the -reset switch to either mode resets
> >         the stats to zero.
> >
> >         * tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
> >         harder with some more flush tests.
> >
> >
> >
> > Index: ChangeLog
> > ===================================================================
> > RCS file: /cvsroot/naviserver/naviserver/ChangeLog,v
> > retrieving revision 1.405
> > retrieving revision 1.406
> > diff -C2 -d -r1.405 -r1.406
> > *** ChangeLog 19 Jun 2006 19:05:47 -0000      1.405
> > --- ChangeLog 19 Jun 2006 21:16:38 -0000      1.406
> > ***************
> > *** 1,2 ****
> > --- 1,71 ----
> > + 2006-06-19  Stephen Deasey  <sd...@us...>
> > +
> > +     * nsd/fastpath.c:
> > +     * nsd/adpeval.c: Use Ns_CacheDeleteEntry instead of
> > +     Ns_CacheFlushEntry to prevent genuine flush stats being inflated.
> > +
> > +     * tcl/cache.tcl:
> > +     * nsd/nsconf.c:
> > +     * nsd/dns.c: Update to new API.
> > +
> > +     * nsd/nsd.h:
> > +     * nsd/server.c: Remove tcl cache timeout param in anticipation of new
> > +     limits API.
> > +
> > +     * nsd/tclcmds.c: Remove ns_cache _get _set _exists _info
> > +     commands. The first three are racey and can be emulated with _eval
> > +     and the new -force switch.  The new -contents switch to the _stats
> > +     command replaces _info, discouraging racey check-and-set usage.
> > +
> > +     * nsd/tcltime.c:
> > +     * nsd/tclobjv.c: New Ns_ObjvTime parse callback for the Ns_Time
> > +     type. Handy for passing absolute time deadlines to routines which
> > +     timeout.
> > +
> > +     * include/ns.h:
> > +     * nsd/cache.c: New Ns_CacheResetStats routine.
> > +
> > +     Ns_CacheGetConfig removed -- caches are immutable; the current
> > +     settings can be looked up in the config file.
> > +
> > +     Directly expire entries from within all routines which return one,
> > +     but but only if the value is not null, indicating no concurrent
> > +     update is in progress.
> > +
> > +     Use Ns_CacheDeleteEntry to remove an entry from the cache without
> > +     updating the stats. Flush stats should reflect explicit flushes
> > +     only.
> > +
> > +     Log stats when a cache is destroyed.
> > +
> > +     * nsd/tclcache.c: ns_cache_create now takes a -maxentry option
> > +     which is the maximum size of an entry allowed in the cache. This
> > +     prevents one large entry completely emptying an otherwise usefully
> > +     full cache.
> > +
> > +     The ns_cache_create -timeout switch has been removed in
> > +     anticipation of a general purpose limits scheme.
> > +
> > +     The -timeout option is now expected to be an absolute time in the
> > +     future, not an offset from the current time. This is awkward to
> > +     use manually, but should not be needed with the above mentioned
> > +     limits scheme. The -ttl option has been renames -expires and is
> > +     also an absolute time.
> > +
> > +     ns_cache_eval now takes a -force switch which will unconditioanly
> > +     replace any exiting entry, whether it has expired or not. Replaces
> > +     _set.
> > +
> > +     ns_cache_stats -content dumps the size and expirey for each
> > +     entry in the cache. Adding the -reset switch to either mode resets
> > +     the stats to zero.
> > +
> > +
> > +
> > +
> > +
> > +     * tests/ns_cache.test: Try to exercise the underlying Ns_CacheFind
> > +     harder with some more flush tests.
> > +
> >   2006-05-19  Vlad Seryakov  <ser...@us...>
> >
> >
> >
> >
> > _______________________________________________
> > naviserver-commits mailing list
> > nav...@li...
> > https://lists.sourceforge.net/lists/listinfo/naviserver-commits
> >
>
> --
> Vlad Seryakov
> 571 262-8608 office
> vl...@cr...
> http://www.crystalballinc.com/vlad/
>
>
>
> _______________________________________________
> naviserver-commits mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-commits
>