naviserver-devel — Developer discussion

Re: [naviserver-devel] Stats question

[Bernd E. <eid...@we...>]

> If you do a:
>
>   ps axf
>
> does the process actually have > 150 threads? I'm wondering if this is
> a problem with thread cleanup or with reporting.

I have to do a 
"ps axH"

"H" is "Show threads as if they were processes".

Then I currently count 23, which is about the maxthreads (15) number plus some 
writer etc. threads.

Re: [naviserver-devel] Stats question

[Stephen D. <sd...@gm...>]

On 9/27/07, Bernd Eidenschink <eid...@we...> wrote:
> > > i activated the statistics on one server and on the
> > > Main Menu > Threads
> > > page I see, amongst others, a huge list of threads with the
> > > NS_THREAD_DETACHED flag and no PROC, no ARG, with ones having creation
> > > times going back more than 10 hours.
> > >
> > > Something to worry?
> >
> > What is the name of the threads (the first column) ?
>
> The "Thread" column is empty...
> And:
> "Proc:" p:(nil)
> "Arg:" a:(nil)
> (By "huge list" i mean: about 150+)
>
> The other usual suspects names are:
> writer, sched, main, driver, conn:server1:nnnn


If you do a:

  ps axf

does the process actually have > 150 threads? I'm wondering if this is
a problem with thread cleanup or with reporting.

Re: [naviserver-devel] Stats question

[Bernd E. <eid...@we...>]

> > i activated the statistics on one server and on the
> > Main Menu > Threads
> > page I see, amongst others, a huge list of threads with the
> > NS_THREAD_DETACHED flag and no PROC, no ARG, with ones having creation
> > times going back more than 10 hours.
> >
> > Something to worry?
>
> What is the name of the threads (the first column) ?

The "Thread" column is empty...
And:
"Proc:" p:(nil)
"Arg:" a:(nil)
(By "huge list" i mean: about 150+)

The other usual suspects names are:
writer, sched, main, driver, conn:server1:nnnn

Bernd.

Re: [naviserver-devel] Stats question

[Stephen D. <sd...@gm...>]

On 9/26/07, Bernd Eidenschink <eid...@we...> wrote:
>
> Hi there,
>
> i activated the statistics on one server and on the
> Main Menu > Threads
> page I see, amongst others, a huge list of threads with the NS_THREAD_DETACHED
> flag and no PROC, no ARG, with ones having creation times going back more
> than 10 hours.
>
> Something to worry?


What is the name of the threads (the first column) ?

Re: [naviserver-devel] Stats question

[Vlad S. <vl...@cr...>]

Than could be idle conn threads

Bernd Eidenschink wrote:
> Hi there,
> 
> i activated the statistics on one server and on the 
> Main Menu > Threads
> page I see, amongst others, a huge list of threads with the NS_THREAD_DETACHED 
> flag and no PROC, no ARG, with ones having creation times going back more 
> than 10 hours.
> 
> Something to worry?
> 
> Bernd.
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

[naviserver-devel] Stats question

[Bernd E. <eid...@we...>]

Hi there,

i activated the statistics on one server and on the 
Main Menu > Threads
page I see, amongst others, a huge list of threads with the NS_THREAD_DETACHED 
flag and no PROC, no ARG, with ones having creation times going back more 
than 10 hours.

Something to worry?

Bernd.

Re: [naviserver-devel] Ns_Fatal exits cleanly. Why?

[Stephen D. <sd...@gm...>]

On 9/11/07, Vasiljevic Zoran <zv...@ar...> wrote:
>
> Am 11.09.2007 um 12:38 schrieb Stephen Deasey:
>
> > If a memory routine failed, I would expect the process to abort...
> >
>
> Me too. That's why I was confused. If however some package
> does not use Tcl allocator but uses direct calls to malloc,
> we could have this situation. I will need to check this.
>
> >
> > (Hmm, maybe there should be a flush after that LogAdd()...?)
>
> Strictly speaking, yes. But this is not helping here.
> We never got to that abort() call. This is the problem.
>
> Allright, I know now what was the idea. The "Fatal" log
> is yet another log level and should not burn down the server.
> The Panic() should burn down the server. But, this Panic
> is called only within the Tcl and there is no means of
> calling it from other modules. So, now should other modules
> (beside) Tcl initiate server dump when they recognize some
> inconsistent state?


Tcl_Panic is public. I think it's the right thing call if you want to
abort with message from some other module.

We call Tcl_Panic ourselves a few times.

Re: [naviserver-devel] Ns_Fatal exits cleanly. Why?

[Vasiljevic Z. <zv...@ar...>]

Am 11.09.2007 um 12:38 schrieb Stephen Deasey:

> If a memory routine failed, I would expect the process to abort...
>

Me too. That's why I was confused. If however some package
does not use Tcl allocator but uses direct calls to malloc,
we could have this situation. I will need to check this.

>
> (Hmm, maybe there should be a flush after that LogAdd()...?)

Strictly speaking, yes. But this is not helping here.
We never got to that abort() call. This is the problem.

Allright, I know now what was the idea. The "Fatal" log
is yet another log level and should not burn down the server.
The Panic() should burn down the server. But, this Panic
is called only within the Tcl and there is no means of
calling it from other modules. So, now should other modules
(beside) Tcl initiate server dump when they recognize some
inconsistent state?

Re: [naviserver-devel] Ns_Fatal exits cleanly. Why?

[Stephen D. <sd...@gm...>]

On 9/11/07, Vasiljevic Zoran <zv...@ar...> wrote:
> void
> Ns_Fatal(CONST char *fmt, ...)
> {
>      va_list ap;
>
>      va_start(ap, fmt);
>      LogAdd(Fatal, fmt, ap);
>      va_end(ap);
>
>      _exit(1);   /****   <<<<<  WHY THIS ????? */
> }
>
> Why not abort()?


It's called from many different places and a clean-shutdown is often a
reasonable thing to do..?


> This creates lots of troubles for us
> to nail-down such problems:
>
> nsd(17299,0xa000ed88) malloc: *** vm_allocate(size=471502848) failed
> (error code=3)
> nsd(17299,0xa000ed88) malloc: *** error: can't allocate region
> nsd(17299,0xa000ed88) malloc: *** set a breakpoint in szone_error to
> debug
> [11/Sep/2007:09:28:43][17299.2684415368][-main-] Notice: unable to alloc
> 471499774
> [11/Sep/2007:09:33:56][17299.2684415368][-main-] Fatal: unable to alloc
> 471499774 bytes
>
> I recall the Ns_Fatal bringing down the server and producing coredump.
> Now it just exists and we have no clue how to see where the fatal
> message is happening....
>


void *
ns_malloc(size_t size)
{
    return ckalloc(size);
}


Tcl handles the checking for allocation failure and calls Tcl_Panic().


void
NsInitLog(void)
{
    Ns_MutexSetName(&lock, "ns:log");
    Ns_TlsAlloc(&tls, FreeCache);
    AddClbk(LogToFile, (void*)STDERR_FILENO, NULL);

    Tcl_SetPanicProc(Panic);
}

static void
Panic(CONST char *fmt, ...)
{
    va_list ap;

    va_start(ap, fmt);
    LogAdd(Fatal, fmt, ap);
    va_end(ap);

    abort();
}


If a memory routine failed, I would expect the process to abort...


(Hmm, maybe there should be a flush after that LogAdd()...?)

Re: [naviserver-devel] SLES 10

[Gustaf N. <ne...@wu...>]

Stephen Deasey schrieb:
> Install autoconf >= 2.59c
>
> Suse 10.2 ships with autoconf-2.60
>
>
> Hmm, maybe we just need to copy those macros into ./m4 for now. I
> think Zoran had this problem as well...
>   

For the record: the same problem exists on Mac OS X 10.4.10.

autoconf --version
autoconf (GNU Autoconf) 2.59

-gustaf

[naviserver-devel] Ns_Fatal exits cleanly. Why?

[Vasiljevic Z. <zv...@ar...>]

void
Ns_Fatal(CONST char *fmt, ...)
{
     va_list ap;

     va_start(ap, fmt);
     LogAdd(Fatal, fmt, ap);
     va_end(ap);

     _exit(1);   /****   <<<<<  WHY THIS ????? */
}

Why not abort()?
This creates lots of troubles for us
to nail-down such problems:

nsd(17299,0xa000ed88) malloc: *** vm_allocate(size=471502848) failed
(error code=3)
nsd(17299,0xa000ed88) malloc: *** error: can't allocate region
nsd(17299,0xa000ed88) malloc: *** set a breakpoint in szone_error to
debug
[11/Sep/2007:09:28:43][17299.2684415368][-main-] Notice: unable to alloc
471499774
[11/Sep/2007:09:33:56][17299.2684415368][-main-] Fatal: unable to alloc
471499774 bytes

I recall the Ns_Fatal bringing down the server and producing coredump.
Now it just exists and we have no clue how to see where the fatal
message is happening....

Re: [naviserver-devel] nslog

[Bernd E. <eid...@we...>]

> Try to add this in the Makefile
>
>
> ifdef POSTGRESQL
> CFLAGS     = -I$(POSTGRESQL)/include
> MODLIBS    = -L$(POSTGRESQL)/lib -lnsdb -lpq -Wl,-rpath,$(POSTGRESQL)/lib
> endif
>
> include  $(NAVISERVER)/include/Makefile.module
>
>
> and then try to compile as
>
> make POSTGRESQL=/u01/pgsql8.2.4-dev
>
> see what happens

Looks good:

ldd...
linux-gate.so.1 =>  (0xffffe000)
libnsdb.so => /usr/local/nsroot_production/lib/libnsdb.so (0xb7fe6000)
libpq.so.5 => /u01/pgsql8.2.4-dev/lib/libpq.so.5 (0xb7fcd000)
libnsthread.so => /usr/local/nsroot_production/lib/libnsthread.so (0xb7fc7000)
libnsd.so => /usr/local/nsroot_production/lib/libnsd.so (0xb7f53000)
libtcl8.4.so => /usr/local/nsroot_production/lib/libtcl8.4.so (0xb7ea2000)
libdl.so.2 => /lib/libdl.so.2 (0xb7e95000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7e8a000)
libm.so.6 => /lib/libm.so.6 (0xb7e65000)
libc.so.6 => /lib/libc.so.6 (0xb7d44000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0xb7d11000)
libpthread.so.0 => /lib/libpthread.so.0 (0xb7cfd000)
libz.so.1 => /lib/libz.so.1 (0xb7ceb000)
/lib/ld-linux.so.2 (0x80000000)

...and the server starts!

Big thanks!

Bernd.

Re: [naviserver-devel] nslog

[Stephen D. <sd...@gm...>]

On 9/10/07, Vlad Seryakov <vl...@cr...> wrote:
> Try to add this in the Makefile
>
>
> ifdef POSTGRESQL
> CFLAGS     = -I$(POSTGRESQL)/include
> MODLIBS    = -L$(POSTGRESQL)/lib -lnsdb -lpq -Wl,-rpath,$(POSTGRESQL)/lib
> endif
>
> include  $(NAVISERVER)/include/Makefile.module
>
>
> and then try to compile as
>
> make POSTGRESQL=/u01/pgsql8.2.4-dev
>
> see what happens
>


For 4.99.3 (yeah, I know...) I'd like to convert to automake.

Something like this:

http://naviserver.sourceforge.net/snapshots/naviserver-autotooled-4.99.2.tar.gz

Modules would then not piggy-back on Makefile.module, but have their
own configure scripts, and problems like this wouldn't be so common,
hopefully.

Feedback on the *-autotooled above appreciated...

Re: [naviserver-devel] nslog

[Bernd E. <eid...@we...>]

>If you have PG installed in /u01/... why you link against library in 
>/usr/lib?

I don't do that at will - the problem is this is the driver that works and I 
can't tell in what way I compiled it different, I simply can't reproduce it.

If you look at the way I compiled the non-working-one: What is wrong there? I 
guess I can't be more specific with the given paths?

Bernd.

Re: [naviserver-devel] nslog

[Vlad S. <vl...@cr...>]

Try to add this in the Makefile


ifdef POSTGRESQL
CFLAGS     = -I$(POSTGRESQL)/include
MODLIBS    = -L$(POSTGRESQL)/lib -lnsdb -lpq -Wl,-rpath,$(POSTGRESQL)/lib
endif

include  $(NAVISERVER)/include/Makefile.module


and then try to compile as

make POSTGRESQL=/u01/pgsql8.2.4-dev

see what happens

Bernd Eidenschink wrote:
>> Ah well, we found some new bugs anyway...
> 
> Apropos :-)
> 
> I have plenty of bugs that break down to (my) human failure in fact, here's 
> another... My best friend is the compiling-the-postgres-driver-situation.
> Vlad knows, he often helps me with that but it's a neverending story.
> 
> The base of the problem is we always compile Postgres from source and have no 
> default installation directories.
> 
> Example:
> Naviserver in /usr/local/nsroot_production
> Postgres in /u01/pgsql8.2.4-dev
> 
> Say I checkout the modules/nsdbpg and try to compile, giving only the 
> NAVISERVER variable:
> 
> gmake NAVISERVER=/usr/local/nsroot_production/
> gcc -pipe -O2 -fomit-frame-pointer -Wall -Wno-implicit-int -fPIC  -pipe  -I/usr/local/nsroot_production//include -I"/usr/local/nsroot_production/include"  -DHAVE_CONFIG_H   -c -o 
> nsdbpg.o nsdbpg.c
> In file included from nsdbpg.c:37:
> dbpg.h:44:22: error: libpq-fe.h: No such file or directory
> In file included from nsdbpg.c:37:
> dbpg.h:52: error: expected specifier-qualifier-list before ‘PGconn’
> [... lots of errors following]
> 
> I try to be more specific:
> gmake NAVISERVER=/usr/local/nsroot_production \
> CFLAGS="-I/u01/pgsql8.2.4-dev/lib \
> -I/u01/pgsql8.2.4-dev/include \
> -I/usr/local/nsroot_staging/include"
> 
> (Some warnings, but finally:)
> gcc -pipe -shared -nostartfiles -L/usr/local/nsroot_production/lib -o 
> nsdbpg.so nsdbpg.o 
> tclcmds.o -lnsdb -lpq -lnsthread -lnsd -L/usr/local/nsroot_production/lib -ltcl8.4 -ldl -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/nsroot_production/lib -Wl,-rpath,/usr/local/nsroot_production/lib
> /usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/bin/ld: cannot 
> find -lpq
> collect2: ld returned 1 exit status
> make: *** [nsdbpg.so] Error 1
> 
> So, after trying various things, I edited the Makefile and changed the line:
> MODLIBS    = -lnsdb -lpq
> 
> to
> MODLIBS
> = -L/usr/local/nsroot_production/include/nsdb.h -L/u01/pgsql8.2.4-dev/lib/libpq.a
> (or)
> = -lnsdb -L/u01/pgsql8.2.4-dev/lib/libpq.a
> 
> 
> and hey! that does it - theres the nsdbpg.so
> 
> But:
> ldd nsdbpg.so
>         linux-gate.so.1 => 
>         libnsdb.so => /usr/local/nsroot_production/lib/libnsdb.so 
>         libnsthread.so => /usr/local/nsroot_production/lib/libnsthread.so 
>         libnsd.so => /usr/local/nsroot_production/lib/libnsd.so 
>         libtcl8.4.so => /usr/local/nsroot_production/lib/libtcl8.4.so 
>         libdl.so.2 => /lib/libdl.so.2 
>         libgcc_s.so.1 => /lib/libgcc_s.so.1 
>         libm.so.6 => /lib/libm.so.6 
>         libc.so.6 => /lib/libc.so.6 
>         libz.so.1 => /lib/libz.so.1 
>         libcrypt.so.1 => /lib/libcrypt.so.1 
>         libpthread.so.0 => /lib/libpthread.so.0 
>         /lib/ld-linux.so.2 
> 
> This one does not work (" undefined symbol: PQsetdbLogin"), because of libpq 
> et. al. missing.
> 
> If I compare it with this working driver (and I simpy can't tell anymore  what 
> I made different with that one):
> (ignore the "not found" messages as I don't set the LD_LIBRARY_PATH, but this 
> driver works then):
>         linux-gate.so.1 =>  
>         libnsdb.so => not found
>         libpq.so.4 => /usr/lib/libpq.so.4 
>         libnsthread.so => not found
>         libnsd.so => not found
>         libtcl8.4.so => /usr/lib/libtcl8.4.so 
>         libdl.so.2 => /lib/libdl.so.2 
>         libcrypt.so.1 => /lib/libcrypt.so.1 
>         libz.so.1 => /lib/libz.so.1 
>         libgcc_s.so.1 => /lib/libgcc_s.so.1 
>         libm.so.6 => /lib/libm.so.6 
>         libc.so.6 => /lib/libc.so.6 
>         libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 
>         libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 
>         libkrb5.so.3 => /usr/lib/libkrb5.so.3 
>         libresolv.so.2 => /lib/libresolv.so.2 
>         libnsl.so.1 => /lib/libnsl.so.1 
>         libpthread.so.0 => /lib/libpthread.so.0 
>         /lib/ld-linux.so.2 
>         libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 
>         libcom_err.so.2 => /lib/libcom_err.so.2 
>         libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 
> 
> Maybe it's something really simple to change? Something obvious?
> 
> Bernd.
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] nslog

[Vlad S. <vl...@cr...>]

I think this is the problem,

libpq.so.4 => /usr/lib/libpq.so.4

If you have PG installed in /u01/... why you link against library in 
/usr/lib?

also, 8.2.x libpq is .5 already, not .4

Bernd Eidenschink wrote:
>> Ah well, we found some new bugs anyway...
> 
> Apropos :-)
> 
> I have plenty of bugs that break down to (my) human failure in fact, here's 
> another... My best friend is the compiling-the-postgres-driver-situation.
> Vlad knows, he often helps me with that but it's a neverending story.
> 
> The base of the problem is we always compile Postgres from source and have no 
> default installation directories.
> 
> Example:
> Naviserver in /usr/local/nsroot_production
> Postgres in /u01/pgsql8.2.4-dev
> 
> Say I checkout the modules/nsdbpg and try to compile, giving only the 
> NAVISERVER variable:
> 
> gmake NAVISERVER=/usr/local/nsroot_production/
> gcc -pipe -O2 -fomit-frame-pointer -Wall -Wno-implicit-int -fPIC  -pipe  -I/usr/local/nsroot_production//include -I"/usr/local/nsroot_production/include"  -DHAVE_CONFIG_H   -c -o 
> nsdbpg.o nsdbpg.c
> In file included from nsdbpg.c:37:
> dbpg.h:44:22: error: libpq-fe.h: No such file or directory
> In file included from nsdbpg.c:37:
> dbpg.h:52: error: expected specifier-qualifier-list before ‘PGconn’
> [... lots of errors following]
> 
> I try to be more specific:
> gmake NAVISERVER=/usr/local/nsroot_production \
> CFLAGS="-I/u01/pgsql8.2.4-dev/lib \
> -I/u01/pgsql8.2.4-dev/include \
> -I/usr/local/nsroot_staging/include"
> 
> (Some warnings, but finally:)
> gcc -pipe -shared -nostartfiles -L/usr/local/nsroot_production/lib -o 
> nsdbpg.so nsdbpg.o 
> tclcmds.o -lnsdb -lpq -lnsthread -lnsd -L/usr/local/nsroot_production/lib -ltcl8.4 -ldl -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/nsroot_production/lib -Wl,-rpath,/usr/local/nsroot_production/lib
> /usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/bin/ld: cannot 
> find -lpq
> collect2: ld returned 1 exit status
> make: *** [nsdbpg.so] Error 1
> 
> So, after trying various things, I edited the Makefile and changed the line:
> MODLIBS    = -lnsdb -lpq
> 
> to
> MODLIBS
> = -L/usr/local/nsroot_production/include/nsdb.h -L/u01/pgsql8.2.4-dev/lib/libpq.a
> (or)
> = -lnsdb -L/u01/pgsql8.2.4-dev/lib/libpq.a
> 
> 
> and hey! that does it - theres the nsdbpg.so
> 
> But:
> ldd nsdbpg.so
>         linux-gate.so.1 => 
>         libnsdb.so => /usr/local/nsroot_production/lib/libnsdb.so 
>         libnsthread.so => /usr/local/nsroot_production/lib/libnsthread.so 
>         libnsd.so => /usr/local/nsroot_production/lib/libnsd.so 
>         libtcl8.4.so => /usr/local/nsroot_production/lib/libtcl8.4.so 
>         libdl.so.2 => /lib/libdl.so.2 
>         libgcc_s.so.1 => /lib/libgcc_s.so.1 
>         libm.so.6 => /lib/libm.so.6 
>         libc.so.6 => /lib/libc.so.6 
>         libz.so.1 => /lib/libz.so.1 
>         libcrypt.so.1 => /lib/libcrypt.so.1 
>         libpthread.so.0 => /lib/libpthread.so.0 
>         /lib/ld-linux.so.2 
> 
> This one does not work (" undefined symbol: PQsetdbLogin"), because of libpq 
> et. al. missing.
> 
> If I compare it with this working driver (and I simpy can't tell anymore  what 
> I made different with that one):
> (ignore the "not found" messages as I don't set the LD_LIBRARY_PATH, but this 
> driver works then):
>         linux-gate.so.1 =>  
>         libnsdb.so => not found
>         libpq.so.4 => /usr/lib/libpq.so.4 
>         libnsthread.so => not found
>         libnsd.so => not found
>         libtcl8.4.so => /usr/lib/libtcl8.4.so 
>         libdl.so.2 => /lib/libdl.so.2 
>         libcrypt.so.1 => /lib/libcrypt.so.1 
>         libz.so.1 => /lib/libz.so.1 
>         libgcc_s.so.1 => /lib/libgcc_s.so.1 
>         libm.so.6 => /lib/libm.so.6 
>         libc.so.6 => /lib/libc.so.6 
>         libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 
>         libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 
>         libkrb5.so.3 => /usr/lib/libkrb5.so.3 
>         libresolv.so.2 => /lib/libresolv.so.2 
>         libnsl.so.1 => /lib/libnsl.so.1 
>         libpthread.so.0 => /lib/libpthread.so.0 
>         /lib/ld-linux.so.2 
>         libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 
>         libcom_err.so.2 => /lib/libcom_err.so.2 
>         libkrb5support.so.0 => /usr/lib/libkrb5support.so.0 
> 
> Maybe it's something really simple to change? Something obvious?
> 
> Bernd.
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] nslog

[Bernd E. <eid...@we...>]

> Ah well, we found some new bugs anyway...

Apropos :-)

I have plenty of bugs that break down to (my) human failure in fact, here's=
=20
another... My best friend is the compiling-the-postgres-driver-situation.
Vlad knows, he often helps me with that but it's a neverending story.

The base of the problem is we always compile Postgres from source and have =
no=20
default installation directories.

Example:
Naviserver in /usr/local/nsroot_production
Postgres in /u01/pgsql8.2.4-dev

Say I checkout the modules/nsdbpg and try to compile, giving only the=20
NAVISERVER variable:

gmake NAVISERVER=3D/usr/local/nsroot_production/
gcc -pipe -O2 -fomit-frame-pointer -Wall -Wno-implicit-int -fPIC  -pipe  -I=
/usr/local/nsroot_production//include -I"/usr/local/nsroot_production/inclu=
de"  -DHAVE_CONFIG_H   -c -o=20
nsdbpg.o nsdbpg.c
In file included from nsdbpg.c:37:
dbpg.h:44:22: error: libpq-fe.h: No such file or directory
In file included from nsdbpg.c:37:
dbpg.h:52: error: expected specifier-qualifier-list before =E2=80=98PGconn=
=E2=80=99
[... lots of errors following]

I try to be more specific:
gmake NAVISERVER=3D/usr/local/nsroot_production \
CFLAGS=3D"-I/u01/pgsql8.2.4-dev/lib \
=2DI/u01/pgsql8.2.4-dev/include \
=2DI/usr/local/nsroot_staging/include"

(Some warnings, but finally:)
gcc -pipe -shared -nostartfiles -L/usr/local/nsroot_production/lib -o=20
nsdbpg.so nsdbpg.o=20
tclcmds.o -lnsdb -lpq -lnsthread -lnsd -L/usr/local/nsroot_production/lib -=
ltcl8.4 -ldl -lgcc_s  -lieee -lm  -Wl,--export-dynamic  -L/usr/local/nsroot=
_production/lib -Wl,-rpath,/usr/local/nsroot_production/lib
/usr/lib/gcc/i586-suse-linux/4.1.0/../../../../i586-suse-linux/bin/ld: cann=
ot=20
find -lpq
collect2: ld returned 1 exit status
make: *** [nsdbpg.so] Error 1

So, after trying various things, I edited the Makefile and changed the line:
MODLIBS    =3D -lnsdb -lpq

to
MODLIBS
=3D -L/usr/local/nsroot_production/include/nsdb.h -L/u01/pgsql8.2.4-dev/lib=
/libpq.a
(or)
=3D -lnsdb -L/u01/pgsql8.2.4-dev/lib/libpq.a


and hey! that does it - theres the nsdbpg.so

But:
ldd nsdbpg.so
        linux-gate.so.1 =3D>=20
        libnsdb.so =3D> /usr/local/nsroot_production/lib/libnsdb.so=20
        libnsthread.so =3D> /usr/local/nsroot_production/lib/libnsthread.so=
=20
        libnsd.so =3D> /usr/local/nsroot_production/lib/libnsd.so=20
        libtcl8.4.so =3D> /usr/local/nsroot_production/lib/libtcl8.4.so=20
        libdl.so.2 =3D> /lib/libdl.so.2=20
        libgcc_s.so.1 =3D> /lib/libgcc_s.so.1=20
        libm.so.6 =3D> /lib/libm.so.6=20
        libc.so.6 =3D> /lib/libc.so.6=20
        libz.so.1 =3D> /lib/libz.so.1=20
        libcrypt.so.1 =3D> /lib/libcrypt.so.1=20
        libpthread.so.0 =3D> /lib/libpthread.so.0=20
        /lib/ld-linux.so.2=20

This one does not work (" undefined symbol: PQsetdbLogin"), because of libp=
q=20
et. al. missing.

If I compare it with this working driver (and I simpy can't tell anymore  w=
hat=20
I made different with that one):
(ignore the "not found" messages as I don't set the LD_LIBRARY_PATH, but th=
is=20
driver works then):
        linux-gate.so.1 =3D> =20
        libnsdb.so =3D> not found
        libpq.so.4 =3D> /usr/lib/libpq.so.4=20
        libnsthread.so =3D> not found
        libnsd.so =3D> not found
        libtcl8.4.so =3D> /usr/lib/libtcl8.4.so=20
        libdl.so.2 =3D> /lib/libdl.so.2=20
        libcrypt.so.1 =3D> /lib/libcrypt.so.1=20
        libz.so.1 =3D> /lib/libz.so.1=20
        libgcc_s.so.1 =3D> /lib/libgcc_s.so.1=20
        libm.so.6 =3D> /lib/libm.so.6=20
        libc.so.6 =3D> /lib/libc.so.6=20
        libssl.so.0.9.8 =3D> /usr/lib/libssl.so.0.9.8=20
        libcrypto.so.0.9.8 =3D> /usr/lib/libcrypto.so.0.9.8=20
        libkrb5.so.3 =3D> /usr/lib/libkrb5.so.3=20
        libresolv.so.2 =3D> /lib/libresolv.so.2=20
        libnsl.so.1 =3D> /lib/libnsl.so.1=20
        libpthread.so.0 =3D> /lib/libpthread.so.0=20
        /lib/ld-linux.so.2=20
        libk5crypto.so.3 =3D> /usr/lib/libk5crypto.so.3=20
        libcom_err.so.2 =3D> /lib/libcom_err.so.2=20
        libkrb5support.so.0 =3D> /usr/lib/libkrb5support.so.0=20

Maybe it's something really simple to change? Something obvious?

Bernd.

Re: [naviserver-devel] nslog

[Stephen D. <sd...@gm...>]

On 9/10/07, Bernd Eidenschink <eid...@we...> wrote:
> > Here's the problem:
> > [09/Sep/2007:00:00:00][-sched-] Notice: nslog: closed
> > '/usr/local/nsroot/logs/access-server1.log'
> >
> > [09/Sep/2007:00:00:00][-sched-] Error: nslog: failed: roll
> > '/usr/local/nsroot/logs/access-server1.log': 'No such file or
> > directory'
> >
> >
> > So, access log rolling is enabled, and it failed this past midnight.
> >
> > Are you sure 'ls -l /usr/local/nsroot/logs' looks good?
>
> 4 drwxrwx---  2 root nsgroup 4096 Sep 10 16:34 logs
> -rw-r--r-- 1 nsrun nsgroup   8962291 Sep  9 23:04 access-server1.log
>
> Hell.
>
> Where's a scapegoat when you need one?


Ah well, we found some new bugs anyway...

Re: [naviserver-devel] nslog

[Bernd E. <eid...@we...>]

> Here's the problem:
> [09/Sep/2007:00:00:00][-sched-] Notice: nslog: closed
> '/usr/local/nsroot/logs/access-server1.log'
>
> [09/Sep/2007:00:00:00][-sched-] Error: nslog: failed: roll
> '/usr/local/nsroot/logs/access-server1.log': 'No such file or
> directory'
>
>
> So, access log rolling is enabled, and it failed this past midnight.
>
> Are you sure 'ls -l /usr/local/nsroot/logs' looks good?

4 drwxrwx---  2 root nsgroup 4096 Sep 10 16:34 logs
-rw-r--r-- 1 nsrun nsgroup   8962291 Sep  9 23:04 access-server1.log

Hell.

Where's a scapegoat when you need one?

Thanks, Stephen.

Bernd.

Re: [naviserver-devel] nslog

[Stephen D. <sd...@gm...>]

On 9/10/07, Bernd Eidenschink <eid...@we...> wrote:
> > grep -C1 'nslog:' server1.log
>
> Sent to your private mail.


Last correct shutdown and startup:

[08/Sep/2007:19:02:47] Notice: nslog: closed
'/usr/local/nsroot/logs/access-server1.log'

[08/Sep/2007:21:14:24] Notice: nslog: opened
'/usr/local/nsroot/logs/access-server1.log'


Here's the problem:


[09/Sep/2007:00:00:00][-sched-] Notice: nslog: closed
'/usr/local/nsroot/logs/access-server1.log'

[09/Sep/2007:00:00:00][-sched-] Error: nslog: failed: roll
'/usr/local/nsroot/logs/access-server1.log': 'No such file or
directory'


So, access log rolling is enabled, and it failed this past midnight.

Are you sure 'ls -l /usr/local/nsroot/logs' looks good?

Search your error log near '09/Sep/2007:00:00:00' and there may be
more info, from rollfile etc.

Re: [naviserver-devel] nslog

[Bernd E. <eid...@we...>]

> grep -C1 'nslog:' server1.log

Sent to your private mail.

Bernd.

Re: [naviserver-devel] nslog

[Stephen D. <sd...@gm...>]

On 9/10/07, Bernd Eidenschink <eid...@we...> wrote:
>
> Hi Stephen,
>
> > There's no error messages that match this pattern?
> >     "nslog: logging disabled: write() failed: '%s'"
> > You're saying you only nslog error message you get is:
> >     "nslog: flush failed: '%s'"
> >
> > Are you sure? This doesn't make sense to me...
>
> Hm, what I have is this:
>
> # grep nslog: server1.log*| grep -v "'Success'"| \
>   grep -v "Inappropriate ioctl"| \
>   grep -v "No such file or dir"| \
>   grep -v "Resource temporarily unavailable"| \
>   grep -v closed | grep -v opened



Oh, you're ignoring all the interesting messages...  'flush failed' is bogus.

grep -C1 'nslog:' server1.log, from the time the server last started.
Attach it, or post it to a website if it's > 100k.

Re: [naviserver-devel] nslog

[Bernd E. <eid...@we...>]

Hi Stephen,

> There's no error messages that match this pattern?
>     "nslog: logging disabled: write() failed: '%s'"
> You're saying you only nslog error message you get is:
>     "nslog: flush failed: '%s'"
>
> Are you sure? This doesn't make sense to me...

Hm, what I have is this:

# grep nslog: server1.log*| grep -v "'Success'"| \
  grep -v "Inappropriate ioctl"| \
  grep -v "No such file or dir"| \
  grep -v "Resource temporarily unavailable"| \
  grep -v closed | grep -v opened

Error: nslog: flush failed: 'Broken pipe'
Error: nslog: flush failed: 'Broken pipe'
Error: nslog: flush failed: 'Is a directory'
Error: nslog: flush failed: 'Connection reset by peer'
Error: nslog: flush failed: 'Is a directory'
Error: nslog: flush failed: 'Connection reset by peer'
Error: nslog: flush failed: 'Is a directory'

But what I found is a long forgotten

proc ns_log {priority args} {
}

sourced after startup that simply "puts" out extended information that shows
up in the server.log.

So I looked up this "Is a directory" message, e.g.:

[20/Jul/2007:08:10:53][15853.1257366624][dispatch_caching {
}] Warning: could'nt cache file ( /www/servers/net/pages/hlr ) because: 
couldn't open "/www/servers/net/pages/hlr
": illegal operation on a directory
[20/Jul/2007:08:10:53][15853.3037600672][-conn:server1:3] Error: nslog: flush 
failed: 'Is a directory'


Bernd.

Re: [naviserver-devel] nslog

[Stephen D. <sd...@gm...>]

On 9/10/07, Bernd Eidenschink <eid...@we...> wrote:
> > > The last log entry is from yesterday 23:04 (when the last developer went
> > > home), no new access file since then.
> >
> > Did the partition fill up?
> >
> >   $ df -h
> >
> > Or maybe the directory where the log files live was deleted/moved/had
> > it's permissions changed...?
>
> No, enough space; no logrotate stuff that could interfere, no cronjobs that
> change permissions, no rsync from other servers or things like that...
> The access-log is in the same directory as the server.log and the pidfile
> (logs/) on this server.


There's no error messages that match this pattern?

    "nslog: logging disabled: write() failed: '%s'"

You're saying you only nslog error message you get is:

    "nslog: flush failed: '%s'"

Are you sure? This doesn't make sense to me...

As far as I can see "nslog: flush failed: '%s'" is logged from
LogTrace() when LogFlush() returns NS_ERROR.

LogFlush() returns NS_ERROR when:

  1)  logPtr->fd == -1 anyway (log file closed)
  2)  write() fails within LogFlush()

In the second case, logPtr->fd is set to -1 and an error is logged.
The error is then logged again in LogTrace(), which is wrong because
firstly the error is already logged and secondly errno will be
overwritten by the call to close() so the second message will be
'Success'.

But you would expect to see a log message of type: "nslog: logging
disabled: write() failed: '%s'".

In the first case the extra log message in LogTrace( ) is also wrong,
because there's been no previous failing system call to set errno.
But for fd to equal -1 somewhere along the line there should have been
another log message to say what failed.

Even though the error logging in LogTrace() is wrong, I don't see how
it could happen without your log file actually being closed, which it
shouldn't be.

Scan the error.log for 'nslog:' messages....

Things which could have caused the log file to be closed are:

 - write() failed: disk full, file too large, ...
 - log roll failed: either scheduled or on signal
 - someone changed the logfile with a call to ns_accesslog

LogOpen() should have caught most of those with:

    "nslog: error '%s' opening '%s'"

Re: [naviserver-devel] SLES 10

[Bernd E. <eid...@we...>]

> On 9/10/07, Bernd Eidenschink <eid...@we...> wrote:
> > > Install autoconf >= 2.59c
> > >
> > > Suse 10.2 ships with autoconf-2.60
> >
> > This is a SuSE 10 Enterprise Server numbered 'autoconf-2.59-92.2'.
> > It will not be that easy to update, but i'll try.

Ok, it was easy - downloading the packages from an opensuse 10.2 mirror and 
doing:

rpm -e autoconf
rpm -Uhv m4-1.4.6-19.i586.rpm
rpm -Uhv autoconf-2.60-21.i586.rpm
rpm -Uhv autogen-5.8.7-13.i586.rpm

That did it!

Thanks,
Bernd.