naviserver-devel — Developer discussion

Re: [naviserver-devel] ns_cache

[Vlad S. <vl...@cr...>]

Looks okay, API has changed but if we accept it, i suppose onetime code 
change is acceptable

Stephen Deasey wrote:
> On 10/19/05, Zoran Vasiljevic <zv...@ar...> wrote:
> 
>>Stephen,
>>
>>How is this going? Did you have any time to do some
>>work on your improvements to the caching code?
> 
> 
> 
> I did, but still didn't finish  :-(
> 
> I've uploaded a tarball.  It builds and installs as a module so you
> don't have to patch the server to try it.  Hopefully the tests and
> comments are enough to get you going.
> 
> The idea is that if this seems at all promising, we import it into
> cvs, work on it, then incorporate it into the core.
> 
> I didn't get a chance to check it over since the last rewrite so
> there's sure to be some obvious mistakes...
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] ns_cache

[Stephen D. <sd...@gm...>]

On 10/19/05, Zoran Vasiljevic <zv...@ar...> wrote:
> Stephen,
>
> How is this going? Did you have any time to do some
> work on your improvements to the caching code?


I did, but still didn't finish  :-(

I've uploaded a tarball.  It builds and installs as a module so you
don't have to patch the server to try it.  Hopefully the tests and
comments are enough to get you going.

The idea is that if this seems at all promising, we import it into
cvs, work on it, then incorporate it into the core.

I didn't get a chance to check it over since the last rewrite so
there's sure to be some obvious mistakes...

Re: [naviserver-devel] checkin binaries

[Bernd E. <eid...@we...>]

> You should be checking the binaries with -kb option
> otherwise CVS will/might damage them.

yes, done that now, thanks for the reminder. I usually place those options and 
file extensions in cvswrappers and forget about it, so I forgot here...

Re: [naviserver-devel] ns_moduleload confusion?

[Zoran V. <zv...@ar...>]

Am 21.10.2005 um 07:49 schrieb Stephen Deasey:

> Does this work for you?
>

It does.
Thanks for the fix.
Zoran

Re: [naviserver-devel] DB drivers loading broken

[Stephen D. <sd...@gm...>]

On 10/20/05, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 21.10.2005 um 07:40 schrieb Stephen Deasey:
>
> >
> > Here's the error shown at startup:
> >
> > Notice: modload: loading
> > /home/sd/naviserver-nsdbtest/tests/../nsdbtest/nsdbtest.so
> > Error: modload: invalid server name: 'nsdbtest'
> > Error: dbdrv: failed to load driver 'nsdbtest'
> > Error: dbinit: no such default pool 'a'
> >
>
> Right. The "server" argument is kind-of misued in
> call to Ns_ModuleLoad from nsdb.c.
>
> The easiest fix to that is to use Ns_TclAllocateInterp(NULL)
> in Ns_ModuleLoad. We do not need the interp there really for
> any other purpose but getting the error message if the module
> does not load. I believe this is most non-intrusive and simple
> fix for that.


Hmm, now that I look at the code path for Ns_TclAllocateInterp with a
NULL server argument, it doesn't seem to be handled correctly.

But, does this really make sense?  The idea behind AllocateInterp is
that it caches them per thread, one for each virtual server.  But what
interp state do you need to cache for a non-virtual server interp?=20
And which threads are creating them?  I think Ns_TclAllocateInterp
should disallow a NULL server arg.  What do you think.


I seem to remember that it was me that changed your original code to
use the AllocateInterp call.  Oops...  :-)

I'll change it back.

Re: [naviserver-devel] nsdbtest compile

[Zoran V. <zv...@ar...>]

Am 21.10.2005 um 08:10 schrieb Stephen Deasey:

> Hmm..  I really don't know anything about Darwin or bundles.  Is this
> something you could fix?  I'd appreciate it.

OK. Lets see...

Zoran

Re: [naviserver-devel] nsdbtest compile

[Stephen D. <sd...@gm...>]

On 10/21/05, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Hi Stephen,
>
> This will not work on Darwin. You need to link against nsdb.so and
> this is not
> a shared library but a bundle. Hence the compile fails.
> In order to use this, the nsdbd should either be a shared library or
> the Ns_DbRegisterDriver
> must be moved into libnsd library.
>
> /bin/rm -Rf nsdbtest.so
> cc -bundle -L../nsthread -L../nsd -o nsdbtest.so nsdbtest.o  -L/Users/
> zoran/sf/tcl/unix -ltcl8.4g  -lnsthread -lnsd   -prebind -
> headerpad_max_install_names -Wl,-search_paths_first
> ld: warning -prebind has no effect with -bundle
> ld: Undefined symbols:
> _Ns_DbRegisterDriver
> make[1]: *** [nsdbtest.so] Error 1
> make: *** [all] Error 1


Hmm..  I really don't know anything about Darwin or bundles.  Is this
something you could fix?  I'd appreciate it.

[naviserver-devel] nsdbtest compile

[Zoran V. <zv...@ar...>]

Hi Stephen,

This will not work on Darwin. You need to link against nsdb.so and  
this is not
a shared library but a bundle. Hence the compile fails.
In order to use this, the nsdbd should either be a shared library or  
the Ns_DbRegisterDriver
must be moved into libnsd library.

/bin/rm -Rf nsdbtest.so
cc -bundle -L../nsthread -L../nsd -o nsdbtest.so nsdbtest.o  -L/Users/ 
zoran/sf/tcl/unix -ltcl8.4g  -lnsthread -lnsd   -prebind - 
headerpad_max_install_names -Wl,-search_paths_first
ld: warning -prebind has no effect with -bundle
ld: Undefined symbols:
_Ns_DbRegisterDriver
make[1]: *** [nsdbtest.so] Error 1
make: *** [all] Error 1


Cheers
Zoran

Re: [naviserver-devel] DB drivers loading broken

[Zoran V. <zv...@ar...>]

Am 21.10.2005 um 07:40 schrieb Stephen Deasey:

>
> Here's the error shown at startup:
>
> Notice: modload: loading
> /home/sd/naviserver-nsdbtest/tests/../nsdbtest/nsdbtest.so
> Error: modload: invalid server name: 'nsdbtest'
> Error: dbdrv: failed to load driver 'nsdbtest'
> Error: dbinit: no such default pool 'a'
>

Right. The "server" argument is kind-of misued in
call to Ns_ModuleLoad from nsdb.c.

The easiest fix to that is to use Ns_TclAllocateInterp(NULL)
in Ns_ModuleLoad. We do not need the interp there really for
any other purpose but getting the error message if the module
does not load. I believe this is most non-intrusive and simple
fix for that.

Zoran

Re: [naviserver-devel] ns_atstartup and order of execution?

[Zoran V. <zv...@ar...>]

Am 21.10.2005 um 07:42 schrieb Stephen Deasey:

>
> Not sure about LIFO for shutdown callbacks though.  Does it make
> sense?  It's one extra thing to remember...
>


Well, I was thinking about that ... The fact is, it run as that
before so I left it.
OTOH, if you do A->B->C and start it is logical to do
C->B->A at stop. Therefore I left the inverse order of
stop scripts. If this proves problematic, it can be changed
simply by flipping one argument to RunCallbacks.

Zoran

Re: [naviserver-devel] ns_moduleload confusion?

[Stephen D. <sd...@gm...>]

On 10/19/05, Zoran Vasiljevic <zv...@ar...> wrote:
> Hi everybody!
>
> The new ns_moduleload command seems to add some confusion
> to what is being done on the virtual server/interp
> initialization, unfortunately.
>
> In the way it works now, modules are loaded from the
> virtual server initialization script. At that time the new
> ns_moduleload is used to load the module (and invoke its
> Ns_ModuleInit call).
>
> In the way it worked before, modules were loaded by the core
> *before* the script got executed.
>
> This has a very fine, yet problematic side effect (which is
> BTW hitting  me very hard):
>
>    If I place some Tcl commands in a module and would
>    like to use those commands in the server init script
>    I'm out of luck :-(
>
> By coding my modules to use Ns_TclInitInterps() I was
> simply adding a callback to be run each time an interp
> was created for the thread. Now, during the startup when
> the virtual server scripts are executed, the modules were
> loaded by the server, and hence available in the interp
> which was runnign the init script. This is  because a new
> interp was created and inited before the script was run.
>
> Now, the init script itself loads the module, the module does
> register that callback but this is activated way later,
> (for other interps, threads) hence my init scripts break.
>
> I would not like to recompile all my modules. Even more,
> it might be even impossible to do that because of the
> existing field installations. So I have a serious backwards
> compatibility problem.
>
> Ah... just of curiousity, I inspected the nslog module and
> there you go: the same thing is happening. This module
> adds new Tcl command [ns_accesslog] which is inaccessible
> during the evaluation of the virtual interp init script,
> ALTHOUGH the module has been loaded with the ns_moduleload
> BEFORE this command is used for the first time.
>
> I believe we will have to reconsider the usage of ns_moduleload
> mechanism ...
>
> Otherwise, what are my options?


My fault, sorry.

Looks like I assumed caling a module's init proc was enough to fully
initialize.  It's not.  The commands are added from interp traces
(CREATE and/or ALLOCATE).

I've changed the code to run the callback immediately (just CREATE and
ALLOCATE traces).  Because the tracing API is available only before
startup, and because the module is loaded from ns_moduleload, which
necessarily requires an interp, this should add the modules commands
and other data to the current bootstrap interp.

This is relevant to the bootstrap init.tcl file and to module which
have both a binary *.so file and *.tcl files when the *.tcl files call
commands registered from the binary.


Does this work for you?

Re: [naviserver-devel] ns_atstartup and order of execution?

[Stephen D. <sd...@gm...>]

On 10/20/05, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 19.10.2005 um 18:16 schrieb Zoran Vasiljevic:
>
> > Hi !
> >
> > We have a pretty elaborate (and sometimes confusing) mechanism
> > for running various atXXX scripts.
> >
> > So, if I would like to:
> >
> >   ns_atstartup X
> >   ns_atstartup Y
> >   ns_atstartup Z
> >
> > then the server will execute:
> >
> >   Z
> >   Y
> >   X
> >
> > i.e. in the *reverse* order. This is because all of those
> > are just entered in a single-linked list and the head of
> > the list points always to the *last* registered item.
> >
> > This is suboptimal.
> >
> > In many real world scenarios, I will like to use the ns_atstartup
> > to schedule couple of scripts to do various initialization things.
> > It is often so that Y will depend on X and Z on Y to be successfull.
> > At the moment, I'm (mis)using the ns_job with a single thread and
> > post the jobs there, because it will then exectue them in FIFO fashion
> > (which is what I need). BUT: this is of course not optimal as it runs
> > parallely with the server startup and I have to be carefull to
> > accomodate
> > for that.
> >
> > So, I'm all for reversing the logic of script execution i.e. doing the
> > more natural way of runing them in FIFO fashion (now we have LIFO).
> > But, I'm afraid of the potential compatibility problems. What I would
> > like to see is if any of you would have objections to reversing the
> > logic, and if yes, ideas how we could make this compatible?
> > One idea would be:
> >
> >    ns_atXXX ?-tail? $theScript
> >
> > I'd rather go after FIFO list walk then adding yet-another-option,
> > but I'm open to suggestions...
>
> I've done the coding.
>
> All ns_atXXX EXCEPT ns_atshutdown will now execute callbacks in FIFO
> fashion. The ns_atshutdown will execute the callbacks in LIFO fashion.
> This seems more natural to me. If nobody has anyhing against
> (Vlad already expressed himself) I will commit this.


FIFO makes sense to me.  Thanks for the change.

Not sure about LIFO for shutdown callbacks though.  Does it make
sense?  It's one extra thing to remember...

Re: [naviserver-devel] DB drivers loading broken

[Stephen D. <sd...@gm...>]

On 10/20/05, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 20.10.2005 um 22:23 schrieb Vlad Seryakov:
>
> > I just recompiled new nsd.
> >
> > [20/Oct/2005:16:19:54][6153.3083728576][-main-] Notice: modload:
> > loading /usr/local/ns/bin/nspostgres.so
> > [20/Oct/2005:16:19:54][6153.3083728576][-main-] Fatal: received
> > fatal signal 11
>
> Can you use gdb to step thru Ns_ModuleLoad and see where it breaks?
> I have no postgress to compile against.


I added a new module: nsdbtest.  It implements a null db driver module
which can be used for testing.  Obviously lots more that could be done
with it, but it already flushed out a bug with calling
Ns_TclRegisterTrace.


Here's the error shown at startup:

Notice: modload: loading
/home/sd/naviserver-nsdbtest/tests/../nsdbtest/nsdbtest.so
Error: modload: invalid server name: 'nsdbtest'
Error: dbdrv: failed to load driver 'nsdbtest'
Error: dbinit: no such default pool 'a'

Re: [naviserver-devel] Tcl module loading

[Stephen D. <sd...@gm...>]

On 10/20/05, Vlad Seryakov <vl...@cr...> wrote:
> It looks like new modload broke loading Tcl based modules.
>
> I have the following line in config
>
> ns_param        oss2            Tcl
>
> which loads Tcl module, now it complaints.
> oss2 is a sibdir with .tcl files


Thanks for the fix!


<         if {![string equal [string tolower $module] tcl]} {
---
>         if {![string equal [string tolower $file] tcl]} {

Re: [naviserver-devel] DB drivers loading broken

[Zoran V. <zv...@ar...>]

Am 20.10.2005 um 22:23 schrieb Vlad Seryakov:

> I just recompiled new nsd.
>
> [20/Oct/2005:16:19:54][6153.3083728576][-main-] Notice: modload:  
> loading /usr/local/ns/bin/nspostgres.so
> [20/Oct/2005:16:19:54][6153.3083728576][-main-] Fatal: received  
> fatal signal 11

Can you use gdb to step thru Ns_ModuleLoad and see where it breaks?
I have no postgress to compile against.

Zoran

Re: [naviserver-devel] DB drivers loading broken

[Vlad S. <vl...@cr...>]

I just recompiled new nsd.

[20/Oct/2005:16:19:54][6153.3083728576][-main-] Notice: modload: loading 
/usr/local/ns/bin/nspostgres.so
[20/Oct/2005:16:19:54][6153.3083728576][-main-] Fatal: received fatal 
signal 11


First difference, if (Ns_ModuleLoad(driver, path, module, "Ns_DbDriverInit")

driver is in place of server and load fails because of unknown server. 
when i changed driver to NULL, it started segfaulting, so something 
changed internally. Old Ns_ModuleLoad ignored server argument.

Zoran Vasiljevic wrote:
> 
> Am 20.10.2005 um 20:23 schrieb Vlad Seryakov:
> 
>> Now i cannot load any Db drivers, modload calls init proc with  
>> server, module arguments, but DB driver loading is slightly different.
>>
> 
> Can you be more specific? What is different? The Ns_ModuleLoad did  not 
> change
> from the API, only internals is different (uses Tcl_FSLoadFile)...
> 
> Cheers
> Zoran
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] DB drivers loading broken

[Zoran V. <zv...@ar...>]

Am 20.10.2005 um 20:23 schrieb Vlad Seryakov:

> Now i cannot load any Db drivers, modload calls init proc with  
> server, module arguments, but DB driver loading is slightly different.
>

Can you be more specific? What is different? The Ns_ModuleLoad did  
not change
from the API, only internals is different (uses Tcl_FSLoadFile)...

Cheers
Zoran

[naviserver-devel] DB drivers loading broken

[Vlad S. <vl...@cr...>]

Now i cannot load any Db drivers, modload calls init proc with server, 
module arguments, but DB driver loading is slightly different.

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] uploading large files

[Vlad S. <vl...@cr...>]

We are using it, working stable, no crashes

Zoran Vasiljevic wrote:
> Vlad,
> 
> You have added spooling large uploads into temp file and
> said you will be using it on some of your servers.
> What are your experiences with that?
> I'm asking this because we will have to switch to the newest
> NS code in about 2 months....
> 
> Cheers
> Zoran
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

[naviserver-devel] checkin binaries

[Zoran V. <zv...@ar...>]

Bernd,

You should be checking the binaries with -kb option
otherwise CVS will/might damage them.

zoran:~/sf/ns-vfs/naviserver/tests/images zoran$ cvs stat broken.gif
vas...@cv...'s password:
===================================================================
File: broken.gif        Status: Up-to-date

    Working revision:    1.1
    Repository revision: 1.1     /cvsroot/naviserver/naviserver/tests/ 
images/broken.gif,v
    Sticky Tag:          (none)
    Sticky Date:         (none)
    Sticky Options:      (none)


The "Sticky Options" should read:

    Sticky Options:      -kb

So you do:

    cvs import -kb ....

Cheres
Zoran

[naviserver-devel] uploading large files

[Zoran V. <zv...@ar...>]

Vlad,

You have added spooling large uploads into temp file and
said you will be using it on some of your servers.
What are your experiences with that?
I'm asking this because we will have to switch to the newest
NS code in about 2 months....

Cheers
Zoran

[naviserver-devel] Tcl module loading

[Vlad S. <vl...@cr...>]

It looks like new modload broke loading Tcl based modules.

I have the following line in config

ns_param        oss2            Tcl

which loads Tcl module, now it complaints.
oss2 is a sibdir with .tcl files

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] ns_atstartup and order of execution?

[Bernd E. <eid...@we...>]

> I've done the coding.
>
> All ns_atXXX EXCEPT ns_atshutdown will now execute callbacks in FIFO
> fashion. The ns_atshutdown will execute the callbacks in LIFO fashion.
> This seems more natural to me. If nobody has anyhing against
> (Vlad already expressed himself) I will commit this.

And no objections from us as we only use ns_atshutdown and FIFO/LIFO is both 
working for us.

Re: [naviserver-devel] ns_atstartup and order of execution?

[Zoran V. <zv...@ar...>]

Am 19.10.2005 um 18:16 schrieb Zoran Vasiljevic:

> Hi !
>
> We have a pretty elaborate (and sometimes confusing) mechanism
> for running various atXXX scripts.
>
> So, if I would like to:
>
>   ns_atstartup X
>   ns_atstartup Y
>   ns_atstartup Z
>
> then the server will execute:
>
>   Z
>   Y
>   X
>
> i.e. in the *reverse* order. This is because all of those
> are just entered in a single-linked list and the head of
> the list points always to the *last* registered item.
>
> This is suboptimal.
>
> In many real world scenarios, I will like to use the ns_atstartup
> to schedule couple of scripts to do various initialization things.
> It is often so that Y will depend on X and Z on Y to be successfull.
> At the moment, I'm (mis)using the ns_job with a single thread and
> post the jobs there, because it will then exectue them in FIFO fashion
> (which is what I need). BUT: this is of course not optimal as it runs
> parallely with the server startup and I have to be carefull to  
> accomodate
> for that.
>
> So, I'm all for reversing the logic of script execution i.e. doing the
> more natural way of runing them in FIFO fashion (now we have LIFO).
> But, I'm afraid of the potential compatibility problems. What I would
> like to see is if any of you would have objections to reversing the
> logic, and if yes, ideas how we could make this compatible?
> One idea would be:
>
>    ns_atXXX ?-tail? $theScript
>
> I'd rather go after FIFO list walk then adding yet-another-option,
> but I'm open to suggestions...

I've done the coding.

All ns_atXXX EXCEPT ns_atshutdown will now execute callbacks in FIFO
fashion. The ns_atshutdown will execute the callbacks in LIFO fashion.
This seems more natural to me. If nobody has anyhing against
(Vlad already expressed himself) I will commit this.

Cheers
Zoran

Re: [naviserver-devel] please verify

[Zoran V. <zv...@ar...>]

Am 20.10.2005 um 13:12 schrieb Bernd Eidenschink:

> ns_uuencode @

It does. Damn.
Zoran