naviserver-devel — Developer discussion

Re: [naviserver-devel] Tcl inits

[Neophytos D. <k2...@ph...>]

Hi Zoran,

I've spend the weekend compiling and re-compiling the kernel and
different versions of gcc/glib on two machines (with different
CFLAGS/CHOST settings).

NS "make test" worked only once on one of the two machines (with CHOST
set to "i386-pc-linux-gnu" - GCC 4.2 no longer supports that). The rest
of the time it was just hanging on the first test (adp.test).

> For me, the Tcl-trace or ns_ictl trace are the way to go.
> Looking at those two, the latter is more appealing since of the
> simplicity, but it may hit us hard speedwise. OTOH, the former
> is faster and uses far less resources but is pretty complex in
> implementation (this might be done simpler, yes, yet it is always
> going to be more complex than ns_ictl trace).
> 
> Now, what you all think? Should we do something there or should
> we just stay with the old AS introspective script?

I'm all for Tcl-trace and ns_ictl trace but I gather it would take a
considerable amount of time to prepare and test those scripts. Please
feel free to let me know what I can do to help --- I'm a bit short on
time this month but things should get better after September 8 - getting
married :).

IMHO, I would prefer mostly a quick migration path from AS (i.e. the old
AS introspective script). I have already ported/adapted my code to play
nicely with most important changes introduced recently in NS but I'm not
quite confident to work with the modified init.tcl from 4.99.1. This is
due to the fact that scheduled threads are no longer triggered after
init (they seem to be queued successfully but no -sched- thread is created).


Best wishes,
Neophytos

[naviserver-devel] Tcl inits

[Zoran V. <zv...@ar...>]

Hi!

After spenging a weekend over this, I know less than
before...

The question is how we are going to improve the Tcl
initialization so people could benefit from that better
than now?

As of today, what we have is:

a.
    introspective script based on the "old" way as 4.0 AS
    did it (init the interp by loading all Tcl files then
    run a script to pull-out namespaces, variables etc and
    create a script that get saved with [ns_ictl save]
    and run in each new interp by [ns_ictl update])

b.
    the ttrace way of setting Tcl traces on selected commands
    and have the "things" setup by those commands record in
    shared nsv arrays. Then use Tcl [unknown] to pull out the
    "things" on as-needed basis from shared arrays

Well, none of those is really good. The a. is limited in
what it "introspects". The b. is pretty complex and in some
situations it does not work as expected (although there is
no design problem there, more probably just a bug or lack a
feature).

Now, we do have [ns_ictl trace] where one can register things
to execute at interp creation, allocation etc.
So, I tried to do a simple thing: for each script executed
during the startup of the server I create
    [=10ns_ictl trace [list source $file]]
create-trace.

This "mostly" works, but has two negative effects:
   1. all scripts needs to be re-adjusted as they are going to
      be sourced more than once (for each new interp)
   2. in case 1000's of files are needed for the startup this may
      pose significant delay in interp creation

I think that old AS approach is the worst as it is really limited.
The Tcl-trace based approach is better but is known to have some
problems (I do not know which ones), yet, it is pretty powerful
and has some other nice side-effects (less memory footprint).
Lastly, the [ns_ictl trace] is simplest to understand and deploy
(this is also very important) but requires rewriting of existing
scripts and most probably needs somekind of caching to speedup
the load process when 1000's of startup files are involved.

For me, the Tcl-trace or ns_ictl trace are the way to go.
Looking at those two, the latter is more appealing since of the
simplicity, but it may hit us hard speedwise. OTOH, the former
is faster and uses far less resources but is pretty complex in
implementation (this might be done simpler, yes, yet it is always
going to be more complex than ns_ictl trace).

Now, what you all think? Should we do something there or should
we just stay with the old AS introspective script?

Cheers
Zoran

Re: [naviserver-devel] Fatal: received fatal signal 11

[Stephen D. <sd...@gm...>]

On 8/2/07, Neophytos Demetriou <k2...@ph...> wrote:
> >> Zoran, just to make sure I got that right: xotcl does not work with the
> >> nstrace/lazyloader at all (true or false). That's the reason I'm using
> >> the modified init.tcl file from 4.99.1.
> >
> > I wonder if this is xotcl specific, or if xotcl just tickles the right
> > (wrong) bit of code. Xotcl seems to use a lot of deeply namespaced
> > commands, who's names are stored in variables...
>
> Stephen, there is no need for deeply namespaced commands (that's just
> part of some code I wrote a couple of years back).
>
> > It would be really helpful if a minimal example of the error could be
> > isolated in standard Tcl and dropped into a file:
> > tests/testserver/modules/state.tcl
> >
> > Hard to write a test for it, but at least it will throw an error at
> > startup when the tests are run and someone will see it.  If it really
> > is an xotcl-only error, I guess it can be package required in a catch
> > or something?
>
> The problem with lazyloader=false should be easier to overcome (i.e.
> serializing xotcl code + "ns_ictl trace allocate [list source
> module-file]").


You'd be doing me a favor if you looked over these instructions for
running tests I just put on the wiki and validated that they run a
machine other than mine and are basically understandable:

http://naviserver.sourceforge.net/wiki/index.php/Running_Tests


Also, here's a start at testing the init sequence:

http://naviserver.cvs.sourceforge.net/naviserver/naviserver/tests/init.test?view=markup

If anyone can think of any expectations they have of the
initialisation code that isn't already covered, just add them onto the
end.


Also also, it's worth mentioning that the nsd/init.tcl file can be
substituted for another one using the config parameter
ns/server/$server/tcl:initfile. We do that in the test suite config
just to make sure the fresh source gets used:

http://naviserver.cvs.sourceforge.net/naviserver/naviserver/tests/test.nscfg?view=markup#l_130

That should make it convenient to isolate differences between a
complex installation and the basic upstream test environment.

Re: [naviserver-devel] Release

[Stephen D. <sd...@gm...>]

On 8/3/07, Stephen Deasey <sd...@gm...> wrote:
> The *.test file are in utf-8, and are sourced by Tcl, and I think Tcl
> it's picking up the default encoding to use from the env. So no real
> surprise there. You'd expect encoding.test:1.1, which is completely
> defined within that file, to fail.
>
> But why are the ADP tests failing? The ADP's should be read off disk
> in the correct encoding (and they seem to be). Tracing through the
> rest of it, the server seems to be spitting out the correct bytes (I
> think).
>
> So it's down to the test harness itself. It does this:
>
> proc nstest_http {args} {
>     ns_parseargs {
>         {-encoding "utf-8"} ...
>     } $args
>
>     ...
>
>         #
>         # Force a specific encoding (utf-8 default).
>         #
>
>         fconfigure $rfd -encoding $encoding
>         fconfigure $wfd -encoding $encoding
>
>
> Which looks fair enough to me.
>
> Why isn't this working?


Well, I'm reasonably happy this is not a bug in the server but some
cruftyness in the test harness.

If you were relying on environment variables to set the Tcl system
encoding before, you'll want to use the new
ns/server/$server:systemencoding parameter.

Re: [naviserver-devel] Release

[Stephen D. <sd...@gm...>]

The *.test file are in utf-8, and are sourced by Tcl, and I think Tcl
it's picking up the default encoding to use from the env. So no real
surprise there. You'd expect encoding.test:1.1, which is completely
defined within that file, to fail.

But why are the ADP tests failing? The ADP's should be read off disk
in the correct encoding (and they seem to be). Tracing through the
rest of it, the server seems to be spitting out the correct bytes (I
think).

So it's down to the test harness itself. It does this:

proc nstest_http {args} {
    ns_parseargs {
        {-encoding "utf-8"} ...
    } $args

    ...

	#
	# Force a specific encoding (utf-8 default).
	#

	fconfigure $rfd -encoding $encoding
	fconfigure $wfd -encoding $encoding


Which looks fair enough to me.

Why isn't this working?



On 8/3/07, Vlad Seryakov <vl...@cr...> wrote:
> I could have not correct environment, let me check with fresh install,
> it is happening everytime, we may put big fat warning in make tst to
> make sure it is running on fresh install :-)))
>
> Stephen Deasey wrote:
> > On 8/3/07, Vlad Seryakov <vl...@cr...> wrote:
> >> ... there are still errors in encoding tests.
> >>
> >> Stephen, i was quite busy lately and lost track, is it still broken or
> >> tests are just to up to date? The reason is, we cannot release with
> >> broken encoding support.
> >
> > Oh. I didn't know about this. Works for me.
> >
> > This fails though:
> >
> > LANG=en_US.iso-8859-1 make test TCLTESTARGS="-file encoding.test"
> >
> > (my default is UTF-8)
> >
> > I'll take a look...
> >

Re: [naviserver-devel] Release

[Vlad S. <vl...@cr...>]

I could have not correct environment, let me check with fresh install, 
it is happening everytime, we may put big fat warning in make tst to 
make sure it is running on fresh install :-)))

Stephen Deasey wrote:
> On 8/3/07, Vlad Seryakov <vl...@cr...> wrote:
>> ... there are still errors in encoding tests.
>>
>> Stephen, i was quite busy lately and lost track, is it still broken or
>> tests are just to up to date? The reason is, we cannot release with
>> broken encoding support.
> 
> Oh. I didn't know about this. Works for me.
> 
> This fails though:
> 
> LANG=en_US.iso-8859-1 make test TCLTESTARGS="-file encoding.test"
> 
> (my default is UTF-8)
> 
> I'll take a look...
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

Re: [naviserver-devel] Release

[Stephen D. <sd...@gm...>]

On 8/3/07, Vlad Seryakov <vl...@cr...> wrote:
>
> ... there are still errors in encoding tests.
>
> Stephen, i was quite busy lately and lost track, is it still broken or
> tests are just to up to date? The reason is, we cannot release with
> broken encoding support.

Oh. I didn't know about this. Works for me.

This fails though:

LANG=en_US.iso-8859-1 make test TCLTESTARGS="-file encoding.test"

(my default is UTF-8)

I'll take a look...

[naviserver-devel] Release

[Vlad S. <vl...@cr...>]

OpenACS project is considering us for use along with AOlserver and i 
think this is very good timing for release. using CVS had may not be 
good thing for people to get introduced with the project.

I noticed along with long-standing writer/spooler bug which i am 
responsible to fix (and i use writer all the time without problems, 
that's why so long delay) there are still errors in encoding tests.

Stephen, i was quite busy lately and lost track, is it still broken or 
tests are just to up to date? The reason is, we cannot release with 
broken encoding support.

Re: [naviserver-devel] Fatal: received fatal signal 11

[Neophytos D. <k2...@ph...>]

>> Zoran, just to make sure I got that right: xotcl does not work with the
>> nstrace/lazyloader at all (true or false). That's the reason I'm using
>> the modified init.tcl file from 4.99.1.
> 
> I wonder if this is xotcl specific, or if xotcl just tickles the right
> (wrong) bit of code. Xotcl seems to use a lot of deeply namespaced
> commands, who's names are stored in variables...

Stephen, there is no need for deeply namespaced commands (that's just
part of some code I wrote a couple of years back).

> It would be really helpful if a minimal example of the error could be
> isolated in standard Tcl and dropped into a file:
> tests/testserver/modules/state.tcl
> 
> Hard to write a test for it, but at least it will throw an error at
> startup when the tests are run and someone will see it.  If it really
> is an xotcl-only error, I guess it can be package required in a catch
> or something?

The problem with lazyloader=false should be easier to overcome (i.e.
serializing xotcl code + "ns_ictl trace allocate [list source
module-file]"). The lazyloader=true case should be harder and IIRC Zoran
already mentioned that it would require some changes on the xotcl-side
to work. Still, getting lazyloader=false to work with xotcl is a first step.

Best,
Neophytos

Re: [naviserver-devel] Fatal: received fatal signal 11

[Stephen D. <sd...@gm...>]

On 8/2/07, Neophytos Demetriou <k2...@ph...> wrote:
> Zoran Vasiljevic wrote:
> > Am 02.08.2007 um 19:05 schrieb Neophytos Demetriou:
> >
> >> Having said this, if Zoran comes up with a generic solution to the
> >> nstrace+xotcl problem I will be an even happier NS user :) but I guess
> >> it should be fine for now.
> >
> > Hey... the Xotcl loader MIGHT be busted with lazyloader option set.
> > I would need to re-check that. Other (non-xotcl) stuff
> > should work fine though.
>
> Zoran, just to make sure I got that right: xotcl does not work with the
> nstrace/lazyloader at all (true or false). That's the reason I'm using
> the modified init.tcl file from 4.99.1.
>


I wonder if this is xotcl specific, or if xotcl just tickles the right
(wrong) bit of code. Xotcl seems to use a lot of deeply namespaced
commands, who's names are stored in variables...

It would be really helpful if a minimal example of the error could be
isolated in standard Tcl and dropped into a file:
tests/testserver/modules/state.tcl

Hard to write a test for it, but at least it will throw an error at
startup when the tests are run and someone will see it.  If it really
is an xotcl-only error, I guess it can be package required in a catch
or something?

Re: [naviserver-devel] Fatal: received fatal signal 11

[Zoran V. <zv...@ar...>]

Am 02.08.2007 um 20:27 schrieb Neophytos Demetriou:

> Zoran, just to make sure I got that right: xotcl does not work with  
> the
> nstrace/lazyloader at all (true or false). That's the reason I'm using
> the modified init.tcl file from 4.99.1.
>

Yes. The Xotcl loader might have some problems.
But, anyways, let me fix that Tcl init stuff
altogether, as this will make Xotcl loading also
work again.

Cheers
Zoran

Re: [naviserver-devel] Fatal: received fatal signal 11

[Neophytos D. <k2...@ph...>]

Zoran Vasiljevic wrote:
> Am 02.08.2007 um 19:05 schrieb Neophytos Demetriou:
> 
>> Having said this, if Zoran comes up with a generic solution to the
>> nstrace+xotcl problem I will be an even happier NS user :) but I guess
>> it should be fine for now.
> 
> Hey... the Xotcl loader MIGHT be busted with lazyloader option set.
> I would need to re-check that. Other (non-xotcl) stuff
> should work fine though.

Zoran, just to make sure I got that right: xotcl does not work with the
nstrace/lazyloader at all (true or false). That's the reason I'm using
the modified init.tcl file from 4.99.1.

Best wishes,
Neophytos

Re: [naviserver-devel] Fatal: received fatal signal 11

[Zoran V. <zv...@ar...>]

Am 02.08.2007 um 19:05 schrieb Neophytos Demetriou:

> Having said this, if Zoran comes up with a generic solution to the
> nstrace+xotcl problem I will be an even happier NS user :) but I guess
> it should be fine for now.

Hey... the Xotcl loader MIGHT be busted with lazyloader option set.
I would need to re-check that. Other (non-xotcl) stuff
should work fine though.

Cheers
Zoran

Re: [naviserver-devel] Fatal: received fatal signal 11

[Neophytos D. <k2...@ph...>]

Stephen Deasey wrote:
> On 8/1/07, Neophytos Demetriou <k2...@ph...> wrote:
>> * For lazyloader=false, initialization seems ok but on first request I
>> get the usual problems mentioned in a previous thread (I had tried once
>> to debug those errors but it took me ages and it did not work out in the
>> end).
> 
> 
> Can you repost the errors you get?

* For lazyloader=true, everything xotcl fails during init. For instance:

invalid command name "Class"
    while executing
"Class ::xo::base::NodeLabelVisitor -parameter {
    {nodeCount "0"}
}"
    (file
"/web/service-phgt-0/packages/kernel/tcl/05-base/base-procs.tcl" line 4)
    invoked from within
"source $__file "


* For lazyloader=false, server init mostly works:

Error: invalid command name
"::namespace123::Class444::namespace789::object555"
invalid command name "::namespace123::Class444::namespace789::object555"
    while executing
"namespace origin $pn"
    (procedure "_serializensp" line 14)
    invoked from within
"_serializensp $n"
    (procedure "nstrace::statescript" line 36)
    invoked from within
"nstrace::statescript"
    invoked from within
"if {$use_trace_inits} {

    #
    # ns_eval --
    #
    # Used to eval Tcl code which is then
    # known in all threads.
    #

    proc ns_eval {..."
    (file "/opt/naviserver-4.99.2/bin/init.tcl" line 355)

Please note that the modified init.tcl from 4.99.1 seems to be working
fine. I just need to go through the two versions (4.99.1 and 4.99.2) and
make sure that I'm not skipping anything important.

Having said this, if Zoran comes up with a generic solution to the
nstrace+xotcl problem I will be an even happier NS user :) but I guess
it should be fine for now.


Thanks again and best wishes,
Neophytos

Re: [naviserver-devel] Fatal: received fatal signal 11

[Stephen D. <sd...@gm...>]

On 8/1/07, Neophytos Demetriou <k2...@ph...> wrote:
>
> * For lazyloader=false, initialization seems ok but on first request I
> get the usual problems mentioned in a previous thread (I had tried once
> to debug those errors but it took me ages and it did not work out in the
> end).


Can you repost the errors you get?

Re: [naviserver-devel] Fatal: received fatal signal 11

[Stephen D. <sd...@gm...>]

On 8/1/07, Neophytos Demetriou <k2...@ph...> wrote:
>
> I have thus removed nstrace.tcl and modified init.tcl from 4.99.1 by
> adding the following snippet in the end of the file:
>
> ns_ictl trace allocate ns_init
> ns_ictl trace deallocate ns_cleanup
>
> It seems to be working ok.


The problem here was an Ns_Set containing query variables, which is
allocated by and belongs to the Ns_Conn code, is made available to an
interp when you call ns_conn getform.

After the connection, the query-set is freed by the Ns_Conn code but
unless you call ns_set cleanup explicitly, a pointer to it remains in
the interp and is reused for the next connection.

Don't do that :-)

Seems maybe a little fragile... Not sure if set-cleanup should be
triggered from C code or not. I guess you really want to make sure
that sets owned by other code always get cleaned up, whereas sets
owned by Tcl usually get cleaned up, by the call to ns_set cleanup in
init.tcl.

Re: [naviserver-devel] Fatal: received fatal signal 11

[Neophytos D. <k2...@ph...>]

Zoran, thank you very much for your message.

Zoran Vasiljevic wrote:
> I would not go
> in much detail there as I think that even that is a UGLY
> solution (rather complicated and error-prone, although with some
> nice side-effects regarding memory footprints...).

I would be happy if nstrace was working for me as well (I mean, I see
the benefits) but, currently, the modified init.tcl file from 4.99.1 was
the only thing that made things work fine for me with the latest
versions of NS.

 > I cannot promise but I may try to see if this will work OK for you.
> Unfortunately I can't make that before this weekend.

I have no hard deadline. It would be nice, however, if it was ready in
the next couple of weeks. If not, I can live with the modified init.tcl
from 4.99.1 as long as someone can confirm that it is more or less ok
(or if it's missing something obvious, just point it out).

Best wishes,
Neophytos

Re: [naviserver-devel] Fatal: received fatal signal 11

[Zoran V. <zv...@ar...>]

Am 01.08.2007 um 09:57 schrieb Neophytos Demetriou:

>  I would prefer mostly if NS was distributed with an alternative
> init.tcl (based on the 4.99.1) so that it works for all of us. We  
> would
> then just change the initfile parameter.

Ah... as I said, the interp inits are really ugly.

The 4.0 AS and 4.99.1 NS both use a crude and ugly
introspective script that runs against a initialized
interp and pull-out definitions of procedures, variables
and namespaces found there. All is stuffed in a large
"init" script and this script is applied to every new
Tcl interpreter. Idea was to improve the speed of interp
initialization. That was perhaps necessary 7 years ago
but I believe today it is of less importance.

This is UGLY.

I have modified that to use command traces and build the
script "on-the-fly" as the first Tcl interp is being initialized
but this obviously has some side effects, who knows where
(as this works very fine for us, strangely). I would not go
in much detail there as I think that even that is a UGLY
solution (rather complicated and error-prone, although with some
nice side-effects regarding memory footprints...).

The real solution is to use [ns_ictl trace]. I promised to do
that some time ago but I'm lagging behind... (eh, work, work)
We need to rewrite the Tcl startup so that for every Tcl module
file that gets sourced in the startup interpreter we do something
like:

    ns_ictl trace allocate [list source module-file]

where the module-file is the file containing Tcl code to initialize
the interpreter. All of this is pulled from bin/init.tcl or wherever
you setup your virtual-server init script to be.

I cannot promise but I may try to see if this will work OK for you.
Unfortunately I can't make that before this weekend.

Cheers
Zoran

Re: [naviserver-devel] Fatal: received fatal signal 11

[Neophytos D. <k2...@ph...>]

Stephen Deasey wrote:
> You're using cvs HEAD, right?  (Last change was 8 days ago).
> 
> But you're using nsd/init.tcl from 4.99.1?  There's been quite a few
> changes here. I don't think the current version forces you to use
> nstrace. But if there are specific problems with the way
> initialisation is happening, let us know.

The good news is that I managed to get init.tcl from 4.99.1 to work
(based on your suggestions below). The bad news is that init.tcl 4.99.2
fails whenever nstrace is used:

* For lazyloader=true, server initialization fails.

* For lazyloader=false, initialization seems ok but on first request I
get the usual problems mentioned in a previous thread (I had tried once
to debug those errors but it took me ages and it did not work out in the
end).

> The current version, for example, calls ns_set cleanup as an interp
> trace. It should never be necessary to call this yourself.
> 
> % join [ns_ictl gettraces deallocate] \n
> 
> ns:tcltrace ns_cleanup
> nsdb:releasehandles a:(nil)
> nsproxy:cleanup a:(nil)
> 
> ns_cleanup looks like this:
> 
> proc ns_cleanup {} {
>     ns_cleanupchans;  # Close files
>     ns_cleanupvars;   # Destroy global variables
>     ns_set  cleanup;  # Destroy non-shared sets
>     ns_http cleanup;  # Abort any http requests
>     ns_ictl cleanup;  # Run depreciated 1-shot Ns_TclRegisterDefer's.
> }
> 
> 
> 'deallocate' traces get run when some C code is finished running some
> Tcl code in an interp. Interps are cached per-thread.  So we can see
> from above that set's are per-interp and the interp is cleaned after
> each request.

I have thus removed nstrace.tcl and modified init.tcl from 4.99.1 by
adding the following snippet in the end of the file:

ns_ictl trace allocate ns_init
ns_ictl trace deallocate ns_cleanup

It seems to be working ok. The problem, however, is that I'm not sure if
anything else is still missing (I cannot trust it enough yet to deploy
it). I would prefer mostly if NS was distributed with an alternative
init.tcl (based on the 4.99.1) so that it works for all of us. We would
then just change the initfile parameter.

> Do any of the tests fail for you? Run: make test.
> 
> If you build with --enable-symbols you'll be able to run 'info locals'
> and 'backtrace' in gdb and get some more information.
> 
> 
> Try this:
> 
> ns_section "ns/server/server1/module/nssock"
> ns_param  keepwait 0
> Does it crash immediately (or sooner) than num-threads-requests?

Yes,this fails on the second page request (i.e., the dependent files are
served via a different thread and thus the 40 threads are consumed earlier).

> Try this:
> 
> ns_section "ns/server/server1/module/nssock"
> ns_param  writersize 0
> Does that prevent the crash?

For a while, but it still fails after a number of requests.

Would it be possible to have an alternative init.tcl (based on the one
from 4.99.1) that has all changes applied except nstrace as an interim
solution (until nstrace is updated as described in a previous message by
ZV)?

Let me know if I can be of any help with this.

Best wishes,
Neophytos

Re: [naviserver-devel] Has anybody tried to port mod_auth_kerb to AOLServer / Naviserver?

[Vlad S. <vl...@cr...>]

On Linux we are using pam_smb and it works great, i compiled it in 
non-daemon mode and Naviserver has ns_pam module.

Rick Cobb wrote:
> We’re getting a lot of requests for real Windows “single-sign-on”. That 
> is, no sign on at all if the user’s already logged into their Windows 
> domain.  This is for corporate deployments, obviously. The Apache 
> community apparently has a module known as “mod_auth_kerb” for this. Has 
> anybody worked on porting it to AOLServer? NaviServer?  We’ve already 
> done LDAP deployments; that’s not sufficient for this community (since 
> it still requires you to log in to the web server).
> 
>  
> 
> Thanks –
> 
> -- ReC
> 
> 
> ------------------------------------------------------------------------
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] Has anybody tried to port mod_auth_kerb to AOLServer / Naviserver?

[Rick C. <rc...@Kn...>]

We're getting a lot of requests for real Windows "single-sign-on". That
is, no sign on at all if the user's already logged into their Windows
domain.  This is for corporate deployments, obviously. The Apache
community apparently has a module known as "mod_auth_kerb" for this. Has
anybody worked on porting it to AOLServer? NaviServer?  We've already
done LDAP deployments; that's not sufficient for this community (since
it still requires you to log in to the web server).

=20

Thanks -

-- ReC

Re: [naviserver-devel] Fatal: received fatal signal 11

[Neophytos D. <k2...@ph...>]

Dear Stephen,

Thank you very much for your message.

Stephen Deasey wrote:
> You're using cvs HEAD, right?  (Last change was 8 days ago).

Yes, I'm using CVS HEAD.

I have to head out of the office/town for the next few hours but I plan
to work on it again when I'm back. I'll post a reply to your message, in
detail, this evening.

Thanks again.

Best wishes,
Neophytos






> But you're using nsd/init.tcl from 4.99.1?  There's been quite a few
> changes here. I don't think the current version forces you to use
> nstrace. But if there are specific problems with the way
> initialisation is happening, let us know.
> 
> The current version, for example, calls ns_set cleanup as an interp
> trace. It should never be necessary to call this yourself.
> 
> % join [ns_ictl gettraces deallocate] \n
> 
> ns:tcltrace ns_cleanup
> nsdb:releasehandles a:(nil)
> nsproxy:cleanup a:(nil)
> 
> ns_cleanup looks like this:
> 
> proc ns_cleanup {} {
>     ns_cleanupchans;  # Close files
>     ns_cleanupvars;   # Destroy global variables
>     ns_set  cleanup;  # Destroy non-shared sets
>     ns_http cleanup;  # Abort any http requests
>     ns_ictl cleanup;  # Run depreciated 1-shot Ns_TclRegisterDefer's.
> }
> 
> 
> 'deallocate' traces get run when some C code is finished running some
> Tcl code in an interp. Interps are cached per-thread.  So we can see
> from above that set's are per-interp and the interp is cleaned after
> each request.
> 
> More here:
> 
> http://www.crystalballinc.com/vlad/software/naviserver/files/mann/ns_ictl.html
> 
> 
> 
> Anyway, regardless of all that it still shouldn't crash :-)
> 
> This works for me (no xotcl):
> 
> $ ./config --prefix=/tmp/ns --enable-symbols --with-tcl=...
> $ make install
> $ make gdbruntest
> 
> % ns_eval {{proc ::defaultRequestHandler {args} {
>     ns_return 200 text/plain yo
>     ns_set cleanup
>     return
> }}}
> 
> % ns_register_proc GET /* ::defaultRequestHandler
> 
> % time {nstest_http -getbody 1 GET /} 1000
> 
> 
> Do any of the tests fail for you? Run: make test.
> 
> If you build with --enable-symbols you'll be able to run 'info locals'
> and 'backtrace' in gdb and get some more information.
> 
> 
> Try this:
> 
> ns_section "ns/server/server1/module/nssock"
> ns_param  keepwait 0
> 
> Does it crash immediately (or sooner) than num-threads-requests?
> 
> 
> Try this:
> 
> ns_section "ns/server/server1/module/nssock"
> ns_param  writersize 0
> 
> 
> Does that prevent the crash?
> 
> 
> 
> On 7/31/07, Neophytos Demetriou <k2...@ph...> wrote:
>> Here is the data I have at the moment:
>>
>> * Server crashes every 40 page requests
>>   (times 39 dependent files, i.e. js, css, ajax, etc)
>>
>> * Only one thread per page request is used (background delivery
>> mechanism). Hence, the number 40 matches the allowable
>> maxthreads/minthreads (see configuration below).
>>
>> * Crashes with/without zippy allocator.
>>
>> * Crashes with/without HackContentEncoding.
>>
>> * gdb output:
>>
>>  Program received signal SIGSEGV, Segmentation fault.
>>  [Switching to Thread 49156 (LWP 29203)]
>>  0xb7f433cc in Ns_SetFindCmp () from
>>  /opt/naviserver-4.99.2/lib/libnsd.so
>>
>>  (gdb) continue
>>  Continuing.
>>
>>  Program received signal SIGABRT, Aborted.
>>  0xb7d30c51 in kill () from /lib/libc.so.6
>>
>> * However, the Ns_SetFindCmp code does not seem to be the culprit.
>>
>> * Request handler is as follows:
>>
>>   proc ::defaultRequestHandler {args} {
>>     set o [::xo::RequestHandler new]
>>     $o respond
>>     $o destroy
>>     #ns_set cleanup
>>     return
>>   }
>>
>>   foreach method {GET HEAD POST} {
>>     ns_register_proc $method / ::defaultRequestHandler
>>   }
>>
>> * If you uncomment the "ns_set cleanup" line, no fatal signal is
>> received. However, a server error occurs at the threshold (40 requests
>> times 39 dependent files) due to the fact that the form-conn ns_set is
>> not found (I haven't checked how NS uses sets per thread (my guess is
>> that it creates a persistent ns_set per thread).
>>
>> * NS configuration synopsis:
>>   - 4.99.1 init.tcl (see previous thread, i.e. xotcl)
>>   - without nstrace
>>   - progressminsize/uploadsize 10000 bytes
>>   - 4 spooler writer/reader
>>   - MaxThreads        40
>>   - MinThreads        40
>>   - MaxConnections    120
>>   - MaxDropped        0
>>
>>
>> I plan to investigate this further in the following days. If you have
>> any suggestions/comments/help please do not hesitate to post (or contact
>> me).
>>
>> Best wishes,
>> Neophytos
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Splunk Inc.
>> Still grepping through log files to find problems?  Stop.
>> Now Search log events and configuration files using AJAX and a browser.
>> Download your FREE copy of Splunk now >>  http://get.splunk.com/
>> _______________________________________________
>> naviserver-devel mailing list
>> nav...@li...
>> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>>
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Fatal: received fatal signal 11

[Stephen D. <sd...@gm...>]

You're using cvs HEAD, right?  (Last change was 8 days ago).

But you're using nsd/init.tcl from 4.99.1?  There's been quite a few
changes here. I don't think the current version forces you to use
nstrace. But if there are specific problems with the way
initialisation is happening, let us know.

The current version, for example, calls ns_set cleanup as an interp
trace. It should never be necessary to call this yourself.

% join [ns_ictl gettraces deallocate] \n

ns:tcltrace ns_cleanup
nsdb:releasehandles a:(nil)
nsproxy:cleanup a:(nil)

ns_cleanup looks like this:

proc ns_cleanup {} {
    ns_cleanupchans;  # Close files
    ns_cleanupvars;   # Destroy global variables
    ns_set  cleanup;  # Destroy non-shared sets
    ns_http cleanup;  # Abort any http requests
    ns_ictl cleanup;  # Run depreciated 1-shot Ns_TclRegisterDefer's.
}


'deallocate' traces get run when some C code is finished running some
Tcl code in an interp. Interps are cached per-thread.  So we can see
from above that set's are per-interp and the interp is cleaned after
each request.

More here:

http://www.crystalballinc.com/vlad/software/naviserver/files/mann/ns_ictl.html



Anyway, regardless of all that it still shouldn't crash :-)

This works for me (no xotcl):

$ ./config --prefix=/tmp/ns --enable-symbols --with-tcl=...
$ make install
$ make gdbruntest

% ns_eval {{proc ::defaultRequestHandler {args} {
    ns_return 200 text/plain yo
    ns_set cleanup
    return
}}}

% ns_register_proc GET /* ::defaultRequestHandler

% time {nstest_http -getbody 1 GET /} 1000


Do any of the tests fail for you? Run: make test.

If you build with --enable-symbols you'll be able to run 'info locals'
and 'backtrace' in gdb and get some more information.


Try this:

ns_section "ns/server/server1/module/nssock"
ns_param  keepwait 0

Does it crash immediately (or sooner) than num-threads-requests?


Try this:

ns_section "ns/server/server1/module/nssock"
ns_param  writersize 0


Does that prevent the crash?



On 7/31/07, Neophytos Demetriou <k2...@ph...> wrote:
> Here is the data I have at the moment:
>
> * Server crashes every 40 page requests
>   (times 39 dependent files, i.e. js, css, ajax, etc)
>
> * Only one thread per page request is used (background delivery
> mechanism). Hence, the number 40 matches the allowable
> maxthreads/minthreads (see configuration below).
>
> * Crashes with/without zippy allocator.
>
> * Crashes with/without HackContentEncoding.
>
> * gdb output:
>
>  Program received signal SIGSEGV, Segmentation fault.
>  [Switching to Thread 49156 (LWP 29203)]
>  0xb7f433cc in Ns_SetFindCmp () from
>  /opt/naviserver-4.99.2/lib/libnsd.so
>
>  (gdb) continue
>  Continuing.
>
>  Program received signal SIGABRT, Aborted.
>  0xb7d30c51 in kill () from /lib/libc.so.6
>
> * However, the Ns_SetFindCmp code does not seem to be the culprit.
>
> * Request handler is as follows:
>
>   proc ::defaultRequestHandler {args} {
>     set o [::xo::RequestHandler new]
>     $o respond
>     $o destroy
>     #ns_set cleanup
>     return
>   }
>
>   foreach method {GET HEAD POST} {
>     ns_register_proc $method / ::defaultRequestHandler
>   }
>
> * If you uncomment the "ns_set cleanup" line, no fatal signal is
> received. However, a server error occurs at the threshold (40 requests
> times 39 dependent files) due to the fact that the form-conn ns_set is
> not found (I haven't checked how NS uses sets per thread (my guess is
> that it creates a persistent ns_set per thread).
>
> * NS configuration synopsis:
>   - 4.99.1 init.tcl (see previous thread, i.e. xotcl)
>   - without nstrace
>   - progressminsize/uploadsize 10000 bytes
>   - 4 spooler writer/reader
>   - MaxThreads        40
>   - MinThreads        40
>   - MaxConnections    120
>   - MaxDropped        0
>
>
> I plan to investigate this further in the following days. If you have
> any suggestions/comments/help please do not hesitate to post (or contact
> me).
>
> Best wishes,
> Neophytos
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

[naviserver-devel] Fatal: received fatal signal 11

[Neophytos D. <k2...@ph...>]

Here is the data I have at the moment:

* Server crashes every 40 page requests
  (times 39 dependent files, i.e. js, css, ajax, etc)

* Only one thread per page request is used (background delivery
mechanism). Hence, the number 40 matches the allowable
maxthreads/minthreads (see configuration below).

* Crashes with/without zippy allocator.

* Crashes with/without HackContentEncoding.

* gdb output:

 Program received signal SIGSEGV, Segmentation fault.
 [Switching to Thread 49156 (LWP 29203)]
 0xb7f433cc in Ns_SetFindCmp () from
 /opt/naviserver-4.99.2/lib/libnsd.so

 (gdb) continue
 Continuing.

 Program received signal SIGABRT, Aborted.
 0xb7d30c51 in kill () from /lib/libc.so.6

* However, the Ns_SetFindCmp code does not seem to be the culprit.

* Request handler is as follows:

  proc ::defaultRequestHandler {args} {
    set o [::xo::RequestHandler new]
    $o respond
    $o destroy
    #ns_set cleanup
    return
  }

  foreach method {GET HEAD POST} {
    ns_register_proc $method / ::defaultRequestHandler
  }

* If you uncomment the "ns_set cleanup" line, no fatal signal is
received. However, a server error occurs at the threshold (40 requests
times 39 dependent files) due to the fact that the form-conn ns_set is
not found (I haven't checked how NS uses sets per thread (my guess is
that it creates a persistent ns_set per thread).

* NS configuration synopsis:
  - 4.99.1 init.tcl (see previous thread, i.e. xotcl)
  - without nstrace
  - progressminsize/uploadsize 10000 bytes
  - 4 spooler writer/reader
  - MaxThreads        40
  - MinThreads        40
  - MaxConnections    120
  - MaxDropped        0


I plan to investigate this further in the following days. If you have
any suggestions/comments/help please do not hesitate to post (or contact
me).

Best wishes,
Neophytos

Re: [naviserver-devel] Share This

[Stephen D. <sd...@gm...>]

On 7/5/07, Andrew Piskorski <at...@pi...> wrote:
> On Thu, Jul 05, 2007 at 08:18:02PM +0100, Stephen Deasey wrote:
> > http://sharethis.com/teamBio?tbio=3Djd
> >
> > No more AOLserver updates from Jim?
>
> You should just ask him?


I would just be being nosy. It makes no difference either way -- it's
not as if we ever had any idea of what is coming next.  it's just
kinda sad...


> He could conceivably be using AOLserver,
> nsdci, or related open source code at the new company.
>
>   http://code.google.com/p/nsdci/


Hasn't been touched in 5 months. Still haven't managed to hook-up the
commits list..

  http://groups.google.com/group/nsdci-commits (empty)


Jacob Rosenberg, also in Oct 2006, says:

  Over the course of 2006, nearly every single architect of the ADP platfor=
m
  and the successful technologies that made AOL.com Portal a reality have
  moved on to other things.

  ...

  Until we're able to set aside our sentimentality for the ways of the
  past, and truly embrace the future, we will fall short in what we hope to
  achieve.  I'm ready for the future =97 how about you?

  http://jacobrosenberg.net/2006/10/31/aolserver-adp-and-the-web/