naviserver-devel — Developer discussion

Re: [naviserver-devel] patch for ns_sockcallback

[Gustaf N. <ne...@wu...>]

Am 09.10.13 08:47, schrieb Wolfgang Winkler:
> Hello!
>
> We are using websockets on naviserver. After connections are 
> established, we create a callback for the read event with 
> ns_sockcallback and push them in the background with ns_chan. We hit a 
> limit of exactly 100 connections, which went away, when we commented 
> out the ns_sockcallback call.
>
> The problem is an invalid memory allocation. I've attached a patch, 
> which allows us to open much more connections, although we still get a 
> memory corruption at around 1000 connections, somtimes more, sometimes 
> less. 

Dear Wolfgang,

many thanks for the patch. The allocation unit was clearly wrong
(number of entries vs. number of bytes); the same problem exists
as well in aolserver.

The proposed change has from my understanding 2 flaws
(a memory leak, and an underallocation when max is low
(e.g. 100) and the number of entries in the hash table is
high (e.g. 1000), then only 200 entries are allocated).
The second problem might be related to your still
existing problem,

Please check your code again with the following patch:
https://bitbucket.org/naviserver/naviserver/commits/c35cd3d2394e61dc55d4c3118d7a14a7a774cd52

best regards
-gustaf neumann

> TCL is compiled with Gustaf Neumanns memory allocator path. This is 
> the backtrace with the linux standard memory allocator
>
>  *** glibc detected *** /usr/local/naviserver/bin/nsd: malloc(): 
> memory corruption: 0x000000000b371b60 ***
> ======= Backtrace: =========
> /lib/libc.so.6(+0x71e16)[0x7f9de5cf5e16]
> /lib/libc.so.6(+0x74ead)[0x7f9de5cf8ead]
> /lib/libc.so.6(__libc_malloc+0x70)[0x7f9de5cfac70]
> /usr/local/lib/libtcl8.5.so(Tcl_Alloc+0x15)[0x7f9de64b8475]
> /usr/local/lib/libtcl8.5.so(+0xafdf2)[0x7f9de652ddf2]
> /usr/local/lib/libtcl8.5.so(+0x330de)[0x7f9de64b10de]
> /usr/local/lib/libtcl8.5.so(Tcl_EvalEx+0x16)[0x7f9de64b1aa6]
> /usr/local/naviserver/lib/libnsd.so(Ns_TclEvalCallback+0x12b)[0x7f9de6e4682b]
> /usr/local/naviserver/lib/libnsd.so(NsTclTraceProc+0x1c)[0x7f9de6e4a60c]
> /usr/local/naviserver/lib/libnsd.so(+0x5984a)[0x7f9de6e4a84a]
> /usr/local/naviserver/lib/libnsd.so(+0x59c45)[0x7f9de6e4ac45]
> /usr/local/naviserver/lib/libnsd.so(Ns_TclAllocateInterp+0x15)[0x7f9de6e4aef5]
> /usr/local/naviserver/lib/libnsd.so(NsTclSockProc+0x42)[0x7f9de6e56eb2]
> /usr/local/naviserver/lib/libnsd.so(+0x5250f)[0x7f9de6e4350f]
> /usr/local/naviserver/lib/libnsthread.so(NsThreadMain+0x7e)[0x7f9de67a067e]
> /usr/local/naviserver/lib/libnsthread.so(+0x5029)[0x7f9de67a1029]
> /lib/libpthread.so.0(+0x68ca)[0x7f9de586a8ca]
> /lib/libc.so.6(clone+0x6d)[0x7f9de5d53b6d]
>
> We are running some test with the google allocator, but the results 
> are similar.
>
> Wolfgang
>
> -- 
>
> *Wolfgang Winkler*
> Geschäftsführung
> wol...@di...
> mobil +43.699.19971172
>

[naviserver-devel] patch for ns_sockcallback

[Wolfgang W. <wol...@di...>]

Hello!

We are using websockets on naviserver. After connections are 
established, we create a callback for the read event with 
ns_sockcallback and push them in the background with ns_chan. We hit a 
limit of exactly 100 connections, which went away, when we commented out 
the ns_sockcallback call.

The problem is an invalid memory allocation. I've attached a patch, 
which allows us to open much more connections, although we still get a 
memory corruption at around 1000 connections, somtimes more, sometimes 
less. TCL is compiled with Gustaf Neumanns memory allocator path. This 
is the backtrace with the linux standard memory allocator

  *** glibc detected *** /usr/local/naviserver/bin/nsd: malloc(): memory 
corruption: 0x000000000b371b60 ***
======= Backtrace: =========
/lib/libc.so.6(+0x71e16)[0x7f9de5cf5e16]
/lib/libc.so.6(+0x74ead)[0x7f9de5cf8ead]
/lib/libc.so.6(__libc_malloc+0x70)[0x7f9de5cfac70]
/usr/local/lib/libtcl8.5.so(Tcl_Alloc+0x15)[0x7f9de64b8475]
/usr/local/lib/libtcl8.5.so(+0xafdf2)[0x7f9de652ddf2]
/usr/local/lib/libtcl8.5.so(+0x330de)[0x7f9de64b10de]
/usr/local/lib/libtcl8.5.so(Tcl_EvalEx+0x16)[0x7f9de64b1aa6]
/usr/local/naviserver/lib/libnsd.so(Ns_TclEvalCallback+0x12b)[0x7f9de6e4682b]
/usr/local/naviserver/lib/libnsd.so(NsTclTraceProc+0x1c)[0x7f9de6e4a60c]
/usr/local/naviserver/lib/libnsd.so(+0x5984a)[0x7f9de6e4a84a]
/usr/local/naviserver/lib/libnsd.so(+0x59c45)[0x7f9de6e4ac45]
/usr/local/naviserver/lib/libnsd.so(Ns_TclAllocateInterp+0x15)[0x7f9de6e4aef5]
/usr/local/naviserver/lib/libnsd.so(NsTclSockProc+0x42)[0x7f9de6e56eb2]
/usr/local/naviserver/lib/libnsd.so(+0x5250f)[0x7f9de6e4350f]
/usr/local/naviserver/lib/libnsthread.so(NsThreadMain+0x7e)[0x7f9de67a067e]
/usr/local/naviserver/lib/libnsthread.so(+0x5029)[0x7f9de67a1029]
/lib/libpthread.so.0(+0x68ca)[0x7f9de586a8ca]
/lib/libc.so.6(clone+0x6d)[0x7f9de5d53b6d]

We are running some test with the google allocator, but the results are 
similar.

Wolfgang

-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.72

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz



PS: BESUCHEN SIE UNSERE NEUE SHOP INFO SEITE: www.shop-info.at

Re: [naviserver-devel] Stack smashing on Wheezy

[Gustaf N. <ne...@wu...>]

Am 03.10.13 11:08, schrieb David Osborne:
>
> Can you foresee any problems with this?
>
no. the core modules we use are quite well debugged and stress-tested 
(we do not use nsperm). Various debugger / fortifyer / analyzer find 
different things, so we can't exclude something showing up.... and if it 
shows up, this is good and easy to fix - much better than random crashes.

-gustaf

Re: [naviserver-devel] Stack smashing on Wheezy

[David O. <da...@qc...>]

Thanks Gustaf. That did the trick.

When we build a Wheezy package using debhelper v9, the default build flags
will automatically be included at build time, which are the following:

david@wheezybuild:~/naviserver$ dpkg-buildflags
CFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat
-Werror=format-security
CPPFLAGS=-D_FORTIFY_SOURCE=2
CXXFLAGS=-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat
-Werror=format-security
FFLAGS=-g -O2
LDFLAGS=-Wl,-z,relro

Can you foresee any problems with this?
I will let you know how it goes.



On 2 October 2013 23:30, Gustaf Neumann <ne...@wu...> wrote:

>  Hi David,
>
> many thanks for the good bug report. i was able to reproduce the problem
> on FC,
> The bug looks like a cut and paste problem, where the SHA context was used
> for
> sizing an MD5 entry and is several years old.
>
>
> https://bitbucket.org/naviserver/naviserver/commits/4e205d1a1332dfa59c594adf1589e2d436096721
>
> best regards
> -gustaf neumann
>
>
> Am 02.10.13 18:02, schrieb David Osborne:
>
>     Hi again,
>
>  I've just been attempting to package Naviserver for Debian Wheezy.
>
>  Using tip (and I think also with v4.99.5 tag), I get a stack overflow
> when I try to load the nsperm module.
>
>  To reproduce, I built on Wheezy (7.1) with:
>  ./autogen.sh --with-tcl=/usr/lib/tcl8.5 --enable-symbols
>  make
>  make install
>
>  Where Tcl8.5-dev is 8.5.11-2.
> Then I just add nsperm.so to the module list to load in the default
> nsd-config.tcl and start:
>  /usr/local/ns/bin/nsd -c -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl
>
>  (Error at end of email)
>  I ran a gdb backtrace which seems to point to something in Ns_ModuleInit:
>
> #5  0x00007ffff2887fd0 in Ns_ModuleInit (server=0x6589e0 "default",
> module=<optimized out>) at nsperm.c:194
>
>  This doesn't happen when I build on squeeze.
> I think this may be because of the stricter hardening defaults in Wheezy:
> http://lists.debian.org/debian-devel-announce/2012/02/msg00016.html
>
>  In wheezy, tcl8.5-dev's TCL_EXTRA_CFLAGS in tclConfig.sh default to:
> TCL_EXTRA_CFLAGS='-g -O2 -fstack-protector --param=ssp-buffer-size=4
> -Wformat -Werror=format-security -fno-unit-at-a-time -pipe
> -D_FORTIFY_SOURCE=2'
>  Looks like they then end up in Naviserver's include/MakeFile.global
>
>  In Squeeze with tcl8.5-dev at 8.5.8-2 they were:
>  TCL_EXTRA_CFLAGS='-g -O2 -fno-unit-at-a-time -pipe '
>
>  The full error output is included at the end (with the gdb backtrace at
> the very end)
>
>  Can anyone suggest how to narrow down the root cause here? Is there
> further debugging that would be useful?
>
>  Thanks yet again..
>
>  Regards,
> --
> David
>
>
>  root@naviserverwheezy:~/naviserver# gdb /usr/local/ns/bin/nsd
> GNU gdb (GDB) 7.4.1-debian
> Copyright (C) 2012 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from /usr/local/ns/bin/nsd...done.
> (gdb) run -c -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl
> Starting program: /usr/local/ns/bin/nsd -c -u nsadmin -t
> /usr/local/ns/conf/nsd-config.tcl
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> [New Thread 0x7ffff637b700 (LWP 25105)]
> [New Thread 0x7ffff5b7a700 (LWP 25106)]
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: binder: started
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: enable
> progess statistics for uploads >= 1048576 bytes
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain:
> NaviServer/4.99.6 starting
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain:
> security info: uid=1001, euid=1001, gid=1001, egid=1001
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: Tcl
> version: 8.5.11
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: max
> files: FD_SETSIZE = 1024, rl_cur = 4096, rl_max = 4096
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Warning: nsmain: rl_cur
> > FD_SETSIZE
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: pidfile: failed
> to open pid file 'nsd.pid': 'Permission denied'
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: queueLength 0
> low water 0 high water 0
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice:
> nsd/init.tcl[default]: booting virtual server:  tcl system encoding: "utf-8"
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload:
> loading nscp.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nscp: listening
> on 127.0.0.1:4080
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nscp: added
> user:
> [New Thread 0x7ffff35a2700 (LWP 25109)]
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload:
> loading nssock.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nssock: enable
> 0 spooler thread(s)
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nssock: enable
> 0 writer thread(s)
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload:
> loading nslog.so
> [New Thread 0x7ffff311a700 (LWP 25110)]
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nslog: opened
> '/usr/local/ns/logs/access.log'
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload:
> loading nscgi.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Warning: nscgi: no such
> interps section: ns/interps/interps
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: nscgi: invalid
> directory: /cgi-bin
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: nscgi: invalid
> directory: /cgi-bin
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload:
> loading nsdb.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload:
> loading nsperm.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: random:
> generating 1 seed
> [New Thread 0x7ffff2885700 (LWP 25111)]
> *** stack smashing detected ***: /usr/local/ns/bin/nsd terminated
> [Thread 0x7ffff2885700 (LWP 25111) exited]
> ======= Backtrace: =========
> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7ffff6a972a7]
> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x0)[0x7ffff6a97270]
> /usr/local/ns/bin/nsperm.so(+0x1fd0)[0x7ffff2887fd0]
> /usr/local/ns/lib/libnsd.so(Ns_ModuleLoad+0x130)[0x7ffff7b865a0]
> /usr/local/ns/lib/libnsd.so(NsTclModuleLoadObjCmd+0x1ca)[0x7ffff7b868fa]
> /usr/lib/libtcl8.5.so.0(+0x33dbe)[0x7ffff71fddbe]
> /usr/lib/libtcl8.5.so.0(+0x764be)[0x7ffff72404be]
> /usr/lib/libtcl8.5.so.0(TclObjInterpProcCore+0x2fb)[0x7ffff728227b]
> /usr/lib/libtcl8.5.so.0(+0x33dbe)[0x7ffff71fddbe]
> /usr/lib/libtcl8.5.so.0(+0x349f5)[0x7ffff71fe9f5]
> /usr/lib/libtcl8.5.so.0(Tcl_EvalEx+0x16)[0x7ffff71fe546]
> /usr/lib/libtcl8.5.so.0(Tcl_FSEvalFileEx+0x227)[0x7ffff7264077]
> /usr/lib/libtcl8.5.so.0(Tcl_EvalFile+0x2f)[0x7ffff7262b6f]
> /usr/local/ns/lib/libnsd.so(NsTclInitServer+0x38)[0x7ffff7b9c8b8]
> /usr/local/ns/lib/libnsd.so(NsInitServer+0x27d)[0x7ffff7b917cd]
> /usr/local/ns/lib/libnsd.so(Ns_Main+0x7e9)[0x7ffff7b87599]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7ffff69c6ead]
> /usr/local/ns/bin/nsd[0x400795]
> ======= Memory map: ========
> 00400000-00401000 r-xp 00000000 08:01 22241
> /usr/local/ns/bin/nsd
> 00600000-00601000 rw-p 00000000 08:01 22241
> /usr/local/ns/bin/nsd
> 00601000-01797000 rw-p 00000000 00:00 0
> [heap]
> 7ffff2805000-7ffff2806000 ---p 00000000 00:00 0
> 7ffff2806000-7ffff2886000 rw-p 00000000 00:00 0
> 7ffff2886000-7ffff288b000 r-xp 00000000 08:01 22246
> /usr/local/ns/bin/nsperm.so
> 7ffff288b000-7ffff2a8a000 ---p 00005000 08:01 22246
> /usr/local/ns/bin/nsperm.so
> 7ffff2a8a000-7ffff2a8b000 rw-p 00004000 08:01 22246
> /usr/local/ns/bin/nsperm.so
> 7ffff2a8b000-7ffff2a94000 r-xp 00000000 08:01 22259
> /usr/local/ns/lib/libnsdb.so
> 7ffff2a94000-7ffff2c93000 ---p 00009000 08:01 22259
> /usr/local/ns/lib/libnsdb.so
> 7ffff2c93000-7ffff2c94000 rw-p 00008000 08:01 22259
> /usr/local/ns/lib/libnsdb.so
> 7ffff2c94000-7ffff2c95000 r-xp 00000000 08:01 22257
> /usr/local/ns/bin/nsdb.so
> 7ffff2c95000-7ffff2e94000 ---p 00001000 08:01 22257
> /usr/local/ns/bin/nsdb.so
> 7ffff2e94000-7ffff2e95000 rw-p 00000000 08:01 22257
> /usr/local/ns/bin/nsdb.so
> 7ffff2e95000-7ffff2e9a000 r-xp 00000000 08:01 22243
> /usr/local/ns/bin/nscgi.so
> 7ffff2e9a000-7ffff3099000 ---p 00005000 08:01 22243
> /usr/local/ns/bin/nscgi.so
> 7ffff3099000-7ffff309a000 rw-p 00004000 08:01 22243
> /usr/local/ns/bin/nscgi.so
> 7ffff309a000-7ffff309b000 ---p 00000000 00:00 0
> 7ffff309b000-7ffff311b000 rw-p 00000000 00:00 0
> 7ffff311b000-7ffff3120000 r-xp 00000000 08:01 22245
> /usr/local/ns/bin/nslog.so
> 7ffff3120000-7ffff331f000 ---p 00005000 08:01 22245
> /usr/local/ns/bin/nslog.so
> 7ffff331f000-7ffff3320000 rw-p 00004000 08:01 22245
> /usr/local/ns/bin/nslog.so
> 7ffff3320000-7ffff3322000 r-xp 00000000 08:01 22242
> /usr/local/ns/bin/nssock.so
> 7ffff3322000-7ffff3521000 ---p 00002000 08:01 22242
> /usr/local/ns/bin/nssock.so
> 7ffff3521000-7ffff3522000 rw-p 00001000 08:01 22242
> /usr/local/ns/bin/nssock.so
> 7ffff3522000-7ffff3523000 ---p 00000000 00:00 0
> 7ffff3523000-7ffff35a3000 rw-p 00000000 00:00 0
> 7ffff35a3000-7ffff35a6000 r-xp 00000000 08:01 22244
> /usr/local/ns/bin/nscp.so
> 7ffff35a6000-7ffff37a6000 ---p 00003000 08:01 22244
> /usr/local/ns/bin/nscp.so
> 7ffff37a6000-7ffff37a7000 rw-p 00003000 08:01 22244
> /usr/local/ns/bin/nscp.so
> 7ffff37a7000-7ffff4d4f000 rw-p 00000000 00:00 0
> 7ffff4d4f000-7ffff4d59000 r-xp 00000000 08:01 262831
> /lib/x86_64-linux-gnu/libnss_nis-2.13.so
> 7ffff4d59000-7ffff4f58000 ---p 0000a000 08:01 262831
> /lib/x86_64-linux-gnu/libnss_nis-2.13.so
> 7ffff4f58000-7ffff4f59000 r--p 00009000 08:01 262831
> /lib/x86_64-linux-gnu/libnss_nis-2.13.so
> 7ffff4f59000-7ffff4f5a000 rw-p 0000a000 08:01 262831
> /lib/x86_64-linux-gnu/libnss_nis-2.13.so
> 7ffff4f5a000-7ffff4f6f000 r-xp 00000000 08:01 262829
> /lib/x86_64-linux-gnu/libnsl-2.13.so
> 7ffff4f6f000-7ffff516e000 ---p 00015000 08:01 262829
> /lib/x86_64-linux-gnu/libnsl-2.13.so
> 7ffff516e000-7ffff516f000 r--p 00014000 08:01 262829
> /lib/x86_64-linux-gnu/libnsl-2.13.so
> 7ffff516f000-7ffff5170000 rw-p 00015000 08:01 262829
> /lib/x86_64-linux-gnu/libnsl-2.13.so
> 7ffff5170000-7ffff5172000 rw-p 00000000 00:00 0
> 7ffff5172000-7ffff5179000 r-xp 00000000 08:01 262821
> /lib/x86_64-linux-gnu/libnss_compat-2.13.so
> 7ffff5179000-7ffff5378000 ---p 00007000 08:01 262821
> /lib/x86_64-linux-gnu/libnss_compat-2.13.so
> 7ffff5378000-7ffff5379000 r--p 00006000 08:01 262821
> /lib/x86_64-linux-gnu/libnss_compat-2.13.so
> 7ffff5379000-7ffff537a000 rw-p 00007000 08:01 262821
> /lib/x86_64-linux-gnu/libnss_compat-2.13.so
> 7ffff537a000-7ffff537b000 ---p 00000000 00:00 0
> 7ffff537b000-7ffff5b7b000 rw-p 00000000 00:00 0
> 7ffff5b7b000-7ffff5b7c000 ---p 00000000 00:00 0
> 7ffff5b7c000-7ffff637c000 rw-p 00000000 00:00 0
> 7ffff637c000-7ffff6387000 r-xp 00000000 08:01 262819
> /lib/x86_64-linux-gnu/libnss_files-2.13.so
> 7ffff6387000-7ffff6586000 ---p 0000b000 08:01 262819
> /lib/x86_64-linux-gnu/libnss_files-2.13.so
> 7ffff6586000-7ffff6587000 r--p 0000a000 08:01 262819
> /lib/x86_64-linux-gnu/libnss_files-2.13.so
> 7ffff6587000-7ffff6588000 rw-p 0000b000 08:01 262819
> /lib/x86_64-linux-gnu/libnss_files-2.13.so
> 7ffff6588000-7ffff659f000 r-xp 00000000 08:01 262811
> /lib/x86_64-linux-gnu/libpthread-2.13.so
> 7ffff659f000-7ffff679e000 ---p 00017000 08:01 262811
> /lib/x86_64-linux-gnu/libpthread-2.13.so
> 7ffff679e000-7ffff679f000 r--p 00016000 08:01 262811
> /lib/x86_64-linux-gnu/libpthread-2.13.so
> 7ffff679f000-7ffff67a0000 rw-p 00017000 08:01 262811
> /lib/x86_64-linux-gnu/libpthread-2.13.so
> 7ffff67a0000-7ffff67a4000 rw-p 00000000 00:00 0
> 7ffff67a4000-7ffff67a6000 r-xp 00000000 08:01 262817
> /lib/x86_64-linux-gnu/libdl-2.13.so
> 7ffff67a6000-7ffff69a6000 ---p 00002000 08:01 262817
> /lib/x86_64-linux-gnu/libdl-2.13.so
> 7ffff69a6000-7ffff69a7000 r--p 00002000 08:01 262817
> /lib/x86_64-linux-gnu/libdl-2.13.so
> Program received signal SIGABRT, Aborted.
> 0x00007ffff69da475 in *__GI_raise (sig=<optimized out>) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> 64    ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
>
> (gdb) backtrace
>
> #0  0x00007ffff69da475 in *__GI_raise (sig=<optimized out>) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> #1  0x00007ffff69dd6f0 in *__GI_abort () at abort.c:92
> #2  0x00007ffff6a1552b in __libc_message (do_abort=<optimized out>,
> fmt=<optimized out>) at ../sysdeps/unix/sysv/linux/libc_fatal.c:189
> #3  0x00007ffff6a972a7 in *__GI___fortify_fail (msg=0x7ffff6af50b4 "stack
> smashing detected") at fortify_fail.c:32
> #4  0x00007ffff6a97270 in __stack_chk_fail () at stack_chk_fail.c:29
> #5  0x00007ffff2887fd0 in Ns_ModuleInit (server=0x6589e0 "default",
> module=<optimized out>) at nsperm.c:194
> #6  0x00007ffff7b865a0 in Ns_ModuleLoad (interp=interp@entry=0x630f20,
> server=0x6589e0 "default", module=0x1725200 "nsperm",
>     file=0x7fffffffe180 "/usr/local/ns/bin/nsperm.so", init=0x7ffff7bb9656
> "Ns_ModuleInit") at modload.c:163
> #7  0x00007ffff7b868fa in NsTclModuleLoadObjCmd (arg=0x1718d70,
> interp=0x630f20, objc=<optimized out>, objv=<optimized out>) at
> modload.c:224
> #8  0x00007ffff71fddbe in ?? () from /usr/lib/libtcl8.5.so.0
> #9  0x00007ffff72404be in ?? () from /usr/lib/libtcl8.5.so.0
> #10 0x00007ffff728227b in TclObjInterpProcCore () from
> /usr/lib/libtcl8.5.so.0
> #11 0x00007ffff71fddbe in ?? () from /usr/lib/libtcl8.5.so.0
> #12 0x00007ffff71fe9f5 in ?? () from /usr/lib/libtcl8.5.so.0
> #13 0x00007ffff71fe546 in Tcl_EvalEx () from /usr/lib/libtcl8.5.so.0
> #14 0x00007ffff7264077 in Tcl_FSEvalFileEx () from /usr/lib/libtcl8.5.so.0
> #15 0x00007ffff7262b6f in Tcl_EvalFile () from /usr/lib/libtcl8.5.so.0
> #16 0x00007ffff7b9c8b8 in NsTclInitServer (server=<optimized out>) at
> tclinit.c:1214
> #17 0x00007ffff7b917cd in NsInitServer (server=server@entry=0x6589e0
> "default", staticInitProc=staticInitProc@entry=0x400870 <ServerInit>) at
> server.c:295
> #18 0x00007ffff7b87599 in Ns_Main (argc=<optimized out>, argv=<optimized
> out>, initProc=0x400870 <ServerInit>) at nsmain.c:626
> #19 0x00007ffff69c6ead in __libc_start_main (main=<optimized out>,
> argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>,
> fini=<optimized out>,
>     rtld_fini=<optimized out>, stack_end=0x7fffffffec98) at
> libc-start.c:228
> #20 0x0000000000400795 in _start ()
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> naviserver-devel mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
>
> --
> Univ.Prof. Dr. Gustaf Neumann
> Institute of Information Systems and New Media
> WU Vienna
> Augasse 2-6, A-1090 Vienna, AUSTRIA
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>


-- 
David Osborne
Qcode Software Limited
http://www.qcode.co.uk
T: +44 (0)131 208 0151

Re: [naviserver-devel] Stack smashing on Wheezy

[Gustaf N. <ne...@wu...>]

Hi David,

many thanks for the good bug report. i was able to reproduce the problem 
on FC,
The bug looks like a cut and paste problem, where the SHA context was 
used for
sizing an MD5 entry and is several years old.

https://bitbucket.org/naviserver/naviserver/commits/4e205d1a1332dfa59c594adf1589e2d436096721

best regards
-gustaf neumann


Am 02.10.13 18:02, schrieb David Osborne:
> Hi again,
>
> I've just been attempting to package Naviserver for Debian Wheezy.
>
> Using tip (and I think also with v4.99.5 tag), I get a stack overflow 
> when I try to load the nsperm module.
>
> To reproduce, I built on Wheezy (7.1) with:
> ./autogen.sh --with-tcl=/usr/lib/tcl8.5 --enable-symbols
> make
> make install
>
> Where Tcl8.5-dev is 8.5.11-2.
> Then I just add nsperm.so to the module list to load in the default 
> nsd-config.tcl and start:
> /usr/local/ns/bin/nsd -c -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl
>
> (Error at end of email)
> I ran a gdb backtrace which seems to point to something in Ns_ModuleInit:
>
> #5 0x00007ffff2887fd0 in Ns_ModuleInit (server=0x6589e0 "default", 
> module=<optimized out>) at nsperm.c:194
>
> This doesn't happen when I build on squeeze.
> I think this may be because of the stricter hardening defaults in Wheezy:
> http://lists.debian.org/debian-devel-announce/2012/02/msg00016.html
>
> In wheezy, tcl8.5-dev'sTCL_EXTRA_CFLAGS in tclConfig.sh default to:
> TCL_EXTRA_CFLAGS='-g -O2 -fstack-protector --param=ssp-buffer-size=4 
> -Wformat -Werror=format-security -fno-unit-at-a-time -pipe 
> -D_FORTIFY_SOURCE=2'
> Looks like they then end up in Naviserver'sinclude/MakeFile.global
>
> In Squeeze with tcl8.5-dev at 8.5.8-2 they were:
> TCL_EXTRA_CFLAGS='-g -O2 -fno-unit-at-a-time -pipe '
>
> The full error output is included at the end (with the gdb backtrace 
> at the very end)
>
> Can anyone suggest how to narrow down the root cause here? Is there 
> further debugging that would be useful?
>
> Thanks yet again..
>
> Regards,
> -- 
> David
>
>
> root@naviserverwheezy:~/naviserver# gdb /usr/local/ns/bin/nsd
> GNU gdb (GDB) 7.4.1-debian
> Copyright (C) 2012 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later 
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from /usr/local/ns/bin/nsd...done.
> (gdb) run -c -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl
> Starting program: /usr/local/ns/bin/nsd -c -u nsadmin -t 
> /usr/local/ns/conf/nsd-config.tcl
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> [New Thread 0x7ffff637b700 (LWP 25105)]
> [New Thread 0x7ffff5b7a700 (LWP 25106)]
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: binder: started
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: 
> enable progess statistics for uploads >= 1048576 bytes
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: 
> NaviServer/4.99.6 starting
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: 
> security info: uid=1001, euid=1001, gid=1001, egid=1001
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: Tcl 
> version: 8.5.11
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: max 
> files: FD_SETSIZE = 1024, rl_cur = 4096, rl_max = 4096
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Warning: nsmain: 
> rl_cur > FD_SETSIZE
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: pidfile: 
> failed to open pid file 'nsd.pid': 'Permission denied'
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: queueLength 
> 0 low water 0 high water 0
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: 
> nsd/init.tcl[default]: booting virtual server:  tcl system encoding: 
> "utf-8"
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: 
> loading nscp.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nscp: 
> listening on 127.0.0.1:4080 <http://127.0.0.1:4080>
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nscp: added 
> user:
> [New Thread 0x7ffff35a2700 (LWP 25109)]
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: 
> loading nssock.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nssock: 
> enable 0 spooler thread(s)
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nssock: 
> enable 0 writer thread(s)
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: 
> loading nslog.so
> [New Thread 0x7ffff311a700 (LWP 25110)]
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nslog: 
> opened '/usr/local/ns/logs/access.log'
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: 
> loading nscgi.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Warning: nscgi: no 
> such interps section: ns/interps/interps
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: nscgi: 
> invalid directory: /cgi-bin
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: nscgi: 
> invalid directory: /cgi-bin
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: 
> loading nsdb.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: 
> loading nsperm.so
> [02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: random: 
> generating 1 seed
> [New Thread 0x7ffff2885700 (LWP 25111)]
> *** stack smashing detected ***: /usr/local/ns/bin/nsd terminated
> [Thread 0x7ffff2885700 (LWP 25111) exited]
> ======= Backtrace: =========
> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7ffff6a972a7]
> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x0)[0x7ffff6a97270]
> /usr/local/ns/bin/nsperm.so(+0x1fd0)[0x7ffff2887fd0]
> /usr/local/ns/lib/libnsd.so(Ns_ModuleLoad+0x130)[0x7ffff7b865a0]
> /usr/local/ns/lib/libnsd.so(NsTclModuleLoadObjCmd+0x1ca)[0x7ffff7b868fa]
> /usr/lib/libtcl8.5.so.0(+0x33dbe)[0x7ffff71fddbe]
> /usr/lib/libtcl8.5.so.0(+0x764be)[0x7ffff72404be]
> /usr/lib/libtcl8.5.so.0(TclObjInterpProcCore+0x2fb)[0x7ffff728227b]
> /usr/lib/libtcl8.5.so.0(+0x33dbe)[0x7ffff71fddbe]
> /usr/lib/libtcl8.5.so.0(+0x349f5)[0x7ffff71fe9f5]
> /usr/lib/libtcl8.5.so.0(Tcl_EvalEx+0x16)[0x7ffff71fe546]
> /usr/lib/libtcl8.5.so.0(Tcl_FSEvalFileEx+0x227)[0x7ffff7264077]
> /usr/lib/libtcl8.5.so.0(Tcl_EvalFile+0x2f)[0x7ffff7262b6f]
> /usr/local/ns/lib/libnsd.so(NsTclInitServer+0x38)[0x7ffff7b9c8b8]
> /usr/local/ns/lib/libnsd.so(NsInitServer+0x27d)[0x7ffff7b917cd]
> /usr/local/ns/lib/libnsd.so(Ns_Main+0x7e9)[0x7ffff7b87599]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7ffff69c6ead]
> /usr/local/ns/bin/nsd[0x400795]
> ======= Memory map: ========
> 00400000-00401000 r-xp 00000000 08:01 22241 /usr/local/ns/bin/nsd
> 00600000-00601000 rw-p 00000000 08:01 22241 /usr/local/ns/bin/nsd
> 00601000-01797000 rw-p 00000000 00:00 
> 0                                  [heap]
> 7ffff2805000-7ffff2806000 ---p 00000000 00:00 0
> 7ffff2806000-7ffff2886000 rw-p 00000000 00:00 0
> 7ffff2886000-7ffff288b000 r-xp 00000000 08:01 22246 
> /usr/local/ns/bin/nsperm.so
> 7ffff288b000-7ffff2a8a000 ---p 00005000 08:01 22246 
> /usr/local/ns/bin/nsperm.so
> 7ffff2a8a000-7ffff2a8b000 rw-p 00004000 08:01 22246 
> /usr/local/ns/bin/nsperm.so
> 7ffff2a8b000-7ffff2a94000 r-xp 00000000 08:01 22259 
> /usr/local/ns/lib/libnsdb.so
> 7ffff2a94000-7ffff2c93000 ---p 00009000 08:01 22259 
> /usr/local/ns/lib/libnsdb.so
> 7ffff2c93000-7ffff2c94000 rw-p 00008000 08:01 22259 
> /usr/local/ns/lib/libnsdb.so
> 7ffff2c94000-7ffff2c95000 r-xp 00000000 08:01 22257 
> /usr/local/ns/bin/nsdb.so
> 7ffff2c95000-7ffff2e94000 ---p 00001000 08:01 22257 
> /usr/local/ns/bin/nsdb.so
> 7ffff2e94000-7ffff2e95000 rw-p 00000000 08:01 22257 
> /usr/local/ns/bin/nsdb.so
> 7ffff2e95000-7ffff2e9a000 r-xp 00000000 08:01 22243 
> /usr/local/ns/bin/nscgi.so
> 7ffff2e9a000-7ffff3099000 ---p 00005000 08:01 22243 
> /usr/local/ns/bin/nscgi.so
> 7ffff3099000-7ffff309a000 rw-p 00004000 08:01 22243 
> /usr/local/ns/bin/nscgi.so
> 7ffff309a000-7ffff309b000 ---p 00000000 00:00 0
> 7ffff309b000-7ffff311b000 rw-p 00000000 00:00 0
> 7ffff311b000-7ffff3120000 r-xp 00000000 08:01 22245 
> /usr/local/ns/bin/nslog.so
> 7ffff3120000-7ffff331f000 ---p 00005000 08:01 22245 
> /usr/local/ns/bin/nslog.so
> 7ffff331f000-7ffff3320000 rw-p 00004000 08:01 22245 
> /usr/local/ns/bin/nslog.so
> 7ffff3320000-7ffff3322000 r-xp 00000000 08:01 22242 
> /usr/local/ns/bin/nssock.so
> 7ffff3322000-7ffff3521000 ---p 00002000 08:01 22242 
> /usr/local/ns/bin/nssock.so
> 7ffff3521000-7ffff3522000 rw-p 00001000 08:01 22242 
> /usr/local/ns/bin/nssock.so
> 7ffff3522000-7ffff3523000 ---p 00000000 00:00 0
> 7ffff3523000-7ffff35a3000 rw-p 00000000 00:00 0
> 7ffff35a3000-7ffff35a6000 r-xp 00000000 08:01 22244 
> /usr/local/ns/bin/nscp.so
> 7ffff35a6000-7ffff37a6000 ---p 00003000 08:01 22244 
> /usr/local/ns/bin/nscp.so
> 7ffff37a6000-7ffff37a7000 rw-p 00003000 08:01 22244 
> /usr/local/ns/bin/nscp.so
> 7ffff37a7000-7ffff4d4f000 rw-p 00000000 00:00 0
> 7ffff4d4f000-7ffff4d59000 r-xp 00000000 08:01 262831 
> /lib/x86_64-linux-gnu/libnss_nis-2.13.so <http://libnss_nis-2.13.so>
> 7ffff4d59000-7ffff4f58000 ---p 0000a000 08:01 262831 
> /lib/x86_64-linux-gnu/libnss_nis-2.13.so <http://libnss_nis-2.13.so>
> 7ffff4f58000-7ffff4f59000 r--p 00009000 08:01 262831 
> /lib/x86_64-linux-gnu/libnss_nis-2.13.so <http://libnss_nis-2.13.so>
> 7ffff4f59000-7ffff4f5a000 rw-p 0000a000 08:01 262831 
> /lib/x86_64-linux-gnu/libnss_nis-2.13.so <http://libnss_nis-2.13.so>
> 7ffff4f5a000-7ffff4f6f000 r-xp 00000000 08:01 262829 
> /lib/x86_64-linux-gnu/libnsl-2.13.so <http://libnsl-2.13.so>
> 7ffff4f6f000-7ffff516e000 ---p 00015000 08:01 262829 
> /lib/x86_64-linux-gnu/libnsl-2.13.so <http://libnsl-2.13.so>
> 7ffff516e000-7ffff516f000 r--p 00014000 08:01 262829 
> /lib/x86_64-linux-gnu/libnsl-2.13.so <http://libnsl-2.13.so>
> 7ffff516f000-7ffff5170000 rw-p 00015000 08:01 262829 
> /lib/x86_64-linux-gnu/libnsl-2.13.so <http://libnsl-2.13.so>
> 7ffff5170000-7ffff5172000 rw-p 00000000 00:00 0
> 7ffff5172000-7ffff5179000 r-xp 00000000 08:01 262821 
> /lib/x86_64-linux-gnu/libnss_compat-2.13.so <http://libnss_compat-2.13.so>
> 7ffff5179000-7ffff5378000 ---p 00007000 08:01 262821 
> /lib/x86_64-linux-gnu/libnss_compat-2.13.so <http://libnss_compat-2.13.so>
> 7ffff5378000-7ffff5379000 r--p 00006000 08:01 262821 
> /lib/x86_64-linux-gnu/libnss_compat-2.13.so <http://libnss_compat-2.13.so>
> 7ffff5379000-7ffff537a000 rw-p 00007000 08:01 262821 
> /lib/x86_64-linux-gnu/libnss_compat-2.13.so <http://libnss_compat-2.13.so>
> 7ffff537a000-7ffff537b000 ---p 00000000 00:00 0
> 7ffff537b000-7ffff5b7b000 rw-p 00000000 00:00 0
> 7ffff5b7b000-7ffff5b7c000 ---p 00000000 00:00 0
> 7ffff5b7c000-7ffff637c000 rw-p 00000000 00:00 0
> 7ffff637c000-7ffff6387000 r-xp 00000000 08:01 262819 
> /lib/x86_64-linux-gnu/libnss_files-2.13.so <http://libnss_files-2.13.so>
> 7ffff6387000-7ffff6586000 ---p 0000b000 08:01 262819 
> /lib/x86_64-linux-gnu/libnss_files-2.13.so <http://libnss_files-2.13.so>
> 7ffff6586000-7ffff6587000 r--p 0000a000 08:01 262819 
> /lib/x86_64-linux-gnu/libnss_files-2.13.so <http://libnss_files-2.13.so>
> 7ffff6587000-7ffff6588000 rw-p 0000b000 08:01 262819 
> /lib/x86_64-linux-gnu/libnss_files-2.13.so <http://libnss_files-2.13.so>
> 7ffff6588000-7ffff659f000 r-xp 00000000 08:01 262811 
> /lib/x86_64-linux-gnu/libpthread-2.13.so <http://libpthread-2.13.so>
> 7ffff659f000-7ffff679e000 ---p 00017000 08:01 262811 
> /lib/x86_64-linux-gnu/libpthread-2.13.so <http://libpthread-2.13.so>
> 7ffff679e000-7ffff679f000 r--p 00016000 08:01 262811 
> /lib/x86_64-linux-gnu/libpthread-2.13.so <http://libpthread-2.13.so>
> 7ffff679f000-7ffff67a0000 rw-p 00017000 08:01 262811 
> /lib/x86_64-linux-gnu/libpthread-2.13.so <http://libpthread-2.13.so>
> 7ffff67a0000-7ffff67a4000 rw-p 00000000 00:00 0
> 7ffff67a4000-7ffff67a6000 r-xp 00000000 08:01 262817 
> /lib/x86_64-linux-gnu/libdl-2.13.so <http://libdl-2.13.so>
> 7ffff67a6000-7ffff69a6000 ---p 00002000 08:01 262817 
> /lib/x86_64-linux-gnu/libdl-2.13.so <http://libdl-2.13.so>
> 7ffff69a6000-7ffff69a7000 r--p 00002000 08:01 262817 
> /lib/x86_64-linux-gnu/libdl-2.13.so <http://libdl-2.13.so>
> Program received signal SIGABRT, Aborted.
> 0x00007ffff69da475 in *__GI_raise (sig=<optimized out>) at 
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
>
> (gdb) backtrace
>
> #0  0x00007ffff69da475 in *__GI_raise (sig=<optimized out>) at 
> ../nptl/sysdeps/unix/sysv/linux/raise.c:64
> #1  0x00007ffff69dd6f0 in *__GI_abort () at abort.c:92
> #2  0x00007ffff6a1552b in __libc_message (do_abort=<optimized out>, 
> fmt=<optimized out>) at ../sysdeps/unix/sysv/linux/libc_fatal.c:189
> #3  0x00007ffff6a972a7 in *__GI___fortify_fail (msg=0x7ffff6af50b4 
> "stack smashing detected") at fortify_fail.c:32
> #4  0x00007ffff6a97270 in __stack_chk_fail () at stack_chk_fail.c:29
> #5  0x00007ffff2887fd0 in Ns_ModuleInit (server=0x6589e0 "default", 
> module=<optimized out>) at nsperm.c:194
> #6  0x00007ffff7b865a0 in Ns_ModuleLoad (interp=interp@entry=0x630f20, 
> server=0x6589e0 "default", module=0x1725200 "nsperm",
>     file=0x7fffffffe180 "/usr/local/ns/bin/nsperm.so", 
> init=0x7ffff7bb9656 "Ns_ModuleInit") at modload.c:163
> #7  0x00007ffff7b868fa in NsTclModuleLoadObjCmd (arg=0x1718d70, 
> interp=0x630f20, objc=<optimized out>, objv=<optimized out>) at 
> modload.c:224
> #8  0x00007ffff71fddbe in ?? () from /usr/lib/libtcl8.5.so.0
> #9  0x00007ffff72404be in ?? () from /usr/lib/libtcl8.5.so.0
> #10 0x00007ffff728227b in TclObjInterpProcCore () from 
> /usr/lib/libtcl8.5.so.0
> #11 0x00007ffff71fddbe in ?? () from /usr/lib/libtcl8.5.so.0
> #12 0x00007ffff71fe9f5 in ?? () from /usr/lib/libtcl8.5.so.0
> #13 0x00007ffff71fe546 in Tcl_EvalEx () from /usr/lib/libtcl8.5.so.0
> #14 0x00007ffff7264077 in Tcl_FSEvalFileEx () from /usr/lib/libtcl8.5.so.0
> #15 0x00007ffff7262b6f in Tcl_EvalFile () from /usr/lib/libtcl8.5.so.0
> #16 0x00007ffff7b9c8b8 in NsTclInitServer (server=<optimized out>) at 
> tclinit.c:1214
> #17 0x00007ffff7b917cd in NsInitServer (server=server@entry=0x6589e0 
> "default", staticInitProc=staticInitProc@entry=0x400870 <ServerInit>) 
> at server.c:295
> #18 0x00007ffff7b87599 in Ns_Main (argc=<optimized out>, 
> argv=<optimized out>, initProc=0x400870 <ServerInit>) at nsmain.c:626
> #19 0x00007ffff69c6ead in __libc_start_main (main=<optimized out>, 
> argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, 
> fini=<optimized out>,
>     rtld_fini=<optimized out>, stack_end=0x7fffffffec98) at 
> libc-start.c:228
> #20 0x0000000000400795 in _start ()
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA

[naviserver-devel] Stack smashing on Wheezy

[David O. <da...@qc...>]

Hi again,

I've just been attempting to package Naviserver for Debian Wheezy.

Using tip (and I think also with v4.99.5 tag), I get a stack overflow when
I try to load the nsperm module.

To reproduce, I built on Wheezy (7.1) with:
./autogen.sh --with-tcl=/usr/lib/tcl8.5 --enable-symbols
make
make install

Where Tcl8.5-dev is 8.5.11-2.
Then I just add nsperm.so to the module list to load in the default
nsd-config.tcl and start:
/usr/local/ns/bin/nsd -c -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl

(Error at end of email)
I ran a gdb backtrace which seems to point to something in Ns_ModuleInit:

#5  0x00007ffff2887fd0 in Ns_ModuleInit (server=0x6589e0 "default",
module=<optimized out>) at nsperm.c:194

This doesn't happen when I build on squeeze.
I think this may be because of the stricter hardening defaults in Wheezy:
http://lists.debian.org/debian-devel-announce/2012/02/msg00016.html

In wheezy, tcl8.5-dev's TCL_EXTRA_CFLAGS in tclConfig.sh default to:
TCL_EXTRA_CFLAGS='-g -O2 -fstack-protector --param=ssp-buffer-size=4
-Wformat -Werror=format-security -fno-unit-at-a-time -pipe
-D_FORTIFY_SOURCE=2'
Looks like they then end up in Naviserver's include/MakeFile.global

In Squeeze with tcl8.5-dev at 8.5.8-2 they were:
TCL_EXTRA_CFLAGS='-g -O2 -fno-unit-at-a-time -pipe '

The full error output is included at the end (with the gdb backtrace at the
very end)

Can anyone suggest how to narrow down the root cause here? Is there further
debugging that would be useful?

Thanks yet again..

Regards,
-- 
David


root@naviserverwheezy:~/naviserver# gdb /usr/local/ns/bin/nsd
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/ns/bin/nsd...done.
(gdb) run -c -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl
Starting program: /usr/local/ns/bin/nsd -c -u nsadmin -t
/usr/local/ns/conf/nsd-config.tcl
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff637b700 (LWP 25105)]
[New Thread 0x7ffff5b7a700 (LWP 25106)]
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: binder: started
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: enable
progess statistics for uploads >= 1048576 bytes
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain:
NaviServer/4.99.6 starting
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: security
info: uid=1001, euid=1001, gid=1001, egid=1001
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: Tcl
version: 8.5.11
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nsmain: max
files: FD_SETSIZE = 1024, rl_cur = 4096, rl_max = 4096
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Warning: nsmain: rl_cur
> FD_SETSIZE
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: pidfile: failed
to open pid file 'nsd.pid': 'Permission denied'
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: queueLength 0
low water 0 high water 0
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice:
nsd/init.tcl[default]: booting virtual server:  tcl system encoding: "utf-8"
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: loading
nscp.so
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nscp: listening
on 127.0.0.1:4080
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nscp: added
user:
[New Thread 0x7ffff35a2700 (LWP 25109)]
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: loading
nssock.so
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nssock: enable 0
spooler thread(s)
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nssock: enable 0
writer thread(s)
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: loading
nslog.so
[New Thread 0x7ffff311a700 (LWP 25110)]
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: nslog: opened
'/usr/local/ns/logs/access.log'
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: loading
nscgi.so
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Warning: nscgi: no such
interps section: ns/interps/interps
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: nscgi: invalid
directory: /cgi-bin
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Error: nscgi: invalid
directory: /cgi-bin
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: loading
nsdb.so
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: modload: loading
nsperm.so
[02/Oct/2013:15:09:32][25102.7ffff7fef700][-main-] Notice: random:
generating 1 seed
[New Thread 0x7ffff2885700 (LWP 25111)]
*** stack smashing detected ***: /usr/local/ns/bin/nsd terminated
[Thread 0x7ffff2885700 (LWP 25111) exited]
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7ffff6a972a7]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x0)[0x7ffff6a97270]
/usr/local/ns/bin/nsperm.so(+0x1fd0)[0x7ffff2887fd0]
/usr/local/ns/lib/libnsd.so(Ns_ModuleLoad+0x130)[0x7ffff7b865a0]
/usr/local/ns/lib/libnsd.so(NsTclModuleLoadObjCmd+0x1ca)[0x7ffff7b868fa]
/usr/lib/libtcl8.5.so.0(+0x33dbe)[0x7ffff71fddbe]
/usr/lib/libtcl8.5.so.0(+0x764be)[0x7ffff72404be]
/usr/lib/libtcl8.5.so.0(TclObjInterpProcCore+0x2fb)[0x7ffff728227b]
/usr/lib/libtcl8.5.so.0(+0x33dbe)[0x7ffff71fddbe]
/usr/lib/libtcl8.5.so.0(+0x349f5)[0x7ffff71fe9f5]
/usr/lib/libtcl8.5.so.0(Tcl_EvalEx+0x16)[0x7ffff71fe546]
/usr/lib/libtcl8.5.so.0(Tcl_FSEvalFileEx+0x227)[0x7ffff7264077]
/usr/lib/libtcl8.5.so.0(Tcl_EvalFile+0x2f)[0x7ffff7262b6f]
/usr/local/ns/lib/libnsd.so(NsTclInitServer+0x38)[0x7ffff7b9c8b8]
/usr/local/ns/lib/libnsd.so(NsInitServer+0x27d)[0x7ffff7b917cd]
/usr/local/ns/lib/libnsd.so(Ns_Main+0x7e9)[0x7ffff7b87599]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7ffff69c6ead]
/usr/local/ns/bin/nsd[0x400795]
======= Memory map: ========
00400000-00401000 r-xp 00000000 08:01 22241
/usr/local/ns/bin/nsd
00600000-00601000 rw-p 00000000 08:01 22241
/usr/local/ns/bin/nsd
00601000-01797000 rw-p 00000000 00:00 0
[heap]
7ffff2805000-7ffff2806000 ---p 00000000 00:00 0
7ffff2806000-7ffff2886000 rw-p 00000000 00:00 0
7ffff2886000-7ffff288b000 r-xp 00000000 08:01 22246
/usr/local/ns/bin/nsperm.so
7ffff288b000-7ffff2a8a000 ---p 00005000 08:01 22246
/usr/local/ns/bin/nsperm.so
7ffff2a8a000-7ffff2a8b000 rw-p 00004000 08:01 22246
/usr/local/ns/bin/nsperm.so
7ffff2a8b000-7ffff2a94000 r-xp 00000000 08:01 22259
/usr/local/ns/lib/libnsdb.so
7ffff2a94000-7ffff2c93000 ---p 00009000 08:01 22259
/usr/local/ns/lib/libnsdb.so
7ffff2c93000-7ffff2c94000 rw-p 00008000 08:01 22259
/usr/local/ns/lib/libnsdb.so
7ffff2c94000-7ffff2c95000 r-xp 00000000 08:01 22257
/usr/local/ns/bin/nsdb.so
7ffff2c95000-7ffff2e94000 ---p 00001000 08:01 22257
/usr/local/ns/bin/nsdb.so
7ffff2e94000-7ffff2e95000 rw-p 00000000 08:01 22257
/usr/local/ns/bin/nsdb.so
7ffff2e95000-7ffff2e9a000 r-xp 00000000 08:01 22243
/usr/local/ns/bin/nscgi.so
7ffff2e9a000-7ffff3099000 ---p 00005000 08:01 22243
/usr/local/ns/bin/nscgi.so
7ffff3099000-7ffff309a000 rw-p 00004000 08:01 22243
/usr/local/ns/bin/nscgi.so
7ffff309a000-7ffff309b000 ---p 00000000 00:00 0
7ffff309b000-7ffff311b000 rw-p 00000000 00:00 0
7ffff311b000-7ffff3120000 r-xp 00000000 08:01 22245
/usr/local/ns/bin/nslog.so
7ffff3120000-7ffff331f000 ---p 00005000 08:01 22245
/usr/local/ns/bin/nslog.so
7ffff331f000-7ffff3320000 rw-p 00004000 08:01 22245
/usr/local/ns/bin/nslog.so
7ffff3320000-7ffff3322000 r-xp 00000000 08:01 22242
/usr/local/ns/bin/nssock.so
7ffff3322000-7ffff3521000 ---p 00002000 08:01 22242
/usr/local/ns/bin/nssock.so
7ffff3521000-7ffff3522000 rw-p 00001000 08:01 22242
/usr/local/ns/bin/nssock.so
7ffff3522000-7ffff3523000 ---p 00000000 00:00 0
7ffff3523000-7ffff35a3000 rw-p 00000000 00:00 0
7ffff35a3000-7ffff35a6000 r-xp 00000000 08:01 22244
/usr/local/ns/bin/nscp.so
7ffff35a6000-7ffff37a6000 ---p 00003000 08:01 22244
/usr/local/ns/bin/nscp.so
7ffff37a6000-7ffff37a7000 rw-p 00003000 08:01 22244
/usr/local/ns/bin/nscp.so
7ffff37a7000-7ffff4d4f000 rw-p 00000000 00:00 0
7ffff4d4f000-7ffff4d59000 r-xp 00000000 08:01 262831
/lib/x86_64-linux-gnu/libnss_nis-2.13.so
7ffff4d59000-7ffff4f58000 ---p 0000a000 08:01 262831
/lib/x86_64-linux-gnu/libnss_nis-2.13.so
7ffff4f58000-7ffff4f59000 r--p 00009000 08:01 262831
/lib/x86_64-linux-gnu/libnss_nis-2.13.so
7ffff4f59000-7ffff4f5a000 rw-p 0000a000 08:01 262831
/lib/x86_64-linux-gnu/libnss_nis-2.13.so
7ffff4f5a000-7ffff4f6f000 r-xp 00000000 08:01 262829
/lib/x86_64-linux-gnu/libnsl-2.13.so
7ffff4f6f000-7ffff516e000 ---p 00015000 08:01 262829
/lib/x86_64-linux-gnu/libnsl-2.13.so
7ffff516e000-7ffff516f000 r--p 00014000 08:01 262829
/lib/x86_64-linux-gnu/libnsl-2.13.so
7ffff516f000-7ffff5170000 rw-p 00015000 08:01 262829
/lib/x86_64-linux-gnu/libnsl-2.13.so
7ffff5170000-7ffff5172000 rw-p 00000000 00:00 0
7ffff5172000-7ffff5179000 r-xp 00000000 08:01 262821
/lib/x86_64-linux-gnu/libnss_compat-2.13.so
7ffff5179000-7ffff5378000 ---p 00007000 08:01 262821
/lib/x86_64-linux-gnu/libnss_compat-2.13.so
7ffff5378000-7ffff5379000 r--p 00006000 08:01 262821
/lib/x86_64-linux-gnu/libnss_compat-2.13.so
7ffff5379000-7ffff537a000 rw-p 00007000 08:01 262821
/lib/x86_64-linux-gnu/libnss_compat-2.13.so
7ffff537a000-7ffff537b000 ---p 00000000 00:00 0
7ffff537b000-7ffff5b7b000 rw-p 00000000 00:00 0
7ffff5b7b000-7ffff5b7c000 ---p 00000000 00:00 0
7ffff5b7c000-7ffff637c000 rw-p 00000000 00:00 0
7ffff637c000-7ffff6387000 r-xp 00000000 08:01 262819
/lib/x86_64-linux-gnu/libnss_files-2.13.so
7ffff6387000-7ffff6586000 ---p 0000b000 08:01 262819
/lib/x86_64-linux-gnu/libnss_files-2.13.so
7ffff6586000-7ffff6587000 r--p 0000a000 08:01 262819
/lib/x86_64-linux-gnu/libnss_files-2.13.so
7ffff6587000-7ffff6588000 rw-p 0000b000 08:01 262819
/lib/x86_64-linux-gnu/libnss_files-2.13.so
7ffff6588000-7ffff659f000 r-xp 00000000 08:01 262811
/lib/x86_64-linux-gnu/libpthread-2.13.so
7ffff659f000-7ffff679e000 ---p 00017000 08:01 262811
/lib/x86_64-linux-gnu/libpthread-2.13.so
7ffff679e000-7ffff679f000 r--p 00016000 08:01 262811
/lib/x86_64-linux-gnu/libpthread-2.13.so
7ffff679f000-7ffff67a0000 rw-p 00017000 08:01 262811
/lib/x86_64-linux-gnu/libpthread-2.13.so
7ffff67a0000-7ffff67a4000 rw-p 00000000 00:00 0
7ffff67a4000-7ffff67a6000 r-xp 00000000 08:01 262817
/lib/x86_64-linux-gnu/libdl-2.13.so
7ffff67a6000-7ffff69a6000 ---p 00002000 08:01 262817
/lib/x86_64-linux-gnu/libdl-2.13.so
7ffff69a6000-7ffff69a7000 r--p 00002000 08:01 262817
/lib/x86_64-linux-gnu/libdl-2.13.so
Program received signal SIGABRT, Aborted.
0x00007ffff69da475 in *__GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
64    ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.

(gdb) backtrace

#0  0x00007ffff69da475 in *__GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007ffff69dd6f0 in *__GI_abort () at abort.c:92
#2  0x00007ffff6a1552b in __libc_message (do_abort=<optimized out>,
fmt=<optimized out>) at ../sysdeps/unix/sysv/linux/libc_fatal.c:189
#3  0x00007ffff6a972a7 in *__GI___fortify_fail (msg=0x7ffff6af50b4 "stack
smashing detected") at fortify_fail.c:32
#4  0x00007ffff6a97270 in __stack_chk_fail () at stack_chk_fail.c:29
#5  0x00007ffff2887fd0 in Ns_ModuleInit (server=0x6589e0 "default",
module=<optimized out>) at nsperm.c:194
#6  0x00007ffff7b865a0 in Ns_ModuleLoad (interp=interp@entry=0x630f20,
server=0x6589e0 "default", module=0x1725200 "nsperm",
    file=0x7fffffffe180 "/usr/local/ns/bin/nsperm.so", init=0x7ffff7bb9656
"Ns_ModuleInit") at modload.c:163
#7  0x00007ffff7b868fa in NsTclModuleLoadObjCmd (arg=0x1718d70,
interp=0x630f20, objc=<optimized out>, objv=<optimized out>) at
modload.c:224
#8  0x00007ffff71fddbe in ?? () from /usr/lib/libtcl8.5.so.0
#9  0x00007ffff72404be in ?? () from /usr/lib/libtcl8.5.so.0
#10 0x00007ffff728227b in TclObjInterpProcCore () from
/usr/lib/libtcl8.5.so.0
#11 0x00007ffff71fddbe in ?? () from /usr/lib/libtcl8.5.so.0
#12 0x00007ffff71fe9f5 in ?? () from /usr/lib/libtcl8.5.so.0
#13 0x00007ffff71fe546 in Tcl_EvalEx () from /usr/lib/libtcl8.5.so.0
#14 0x00007ffff7264077 in Tcl_FSEvalFileEx () from /usr/lib/libtcl8.5.so.0
#15 0x00007ffff7262b6f in Tcl_EvalFile () from /usr/lib/libtcl8.5.so.0
#16 0x00007ffff7b9c8b8 in NsTclInitServer (server=<optimized out>) at
tclinit.c:1214
#17 0x00007ffff7b917cd in NsInitServer (server=server@entry=0x6589e0
"default", staticInitProc=staticInitProc@entry=0x400870 <ServerInit>) at
server.c:295
#18 0x00007ffff7b87599 in Ns_Main (argc=<optimized out>, argv=<optimized
out>, initProc=0x400870 <ServerInit>) at nsmain.c:626
#19 0x00007ffff69c6ead in __libc_start_main (main=<optimized out>,
argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>,
fini=<optimized out>,
    rtld_fini=<optimized out>, stack_end=0x7fffffffec98) at libc-start.c:228
#20 0x0000000000400795 in _start ()

Re: [naviserver-devel] Losing interp alias

[Gustaf N. <ne...@wu...>]

Dear David,

I've added a similar approach to nstrace on the bitbucket tip.

There are known deficiencies in the tcl introspection concerning aliases
http://wiki.tcl.tk/21294
but let's hope we have not to go into this.

For my tests, everything seems ok with these changes.

The base question is, do we have to save all these module internal 
information
in the blueprint, and wouldn't it be better to do package requires in 
the right order
at interp startup instead of producing a huge blueprint....
but actually independent from this question, we have to support "interp 
aliases"
in the same way we support tcl procs, otherwise we limit what packages 
we can use
in naviserver.

-gustaf

Am 27.09.13 10:42, schrieb David Osborne:
>
> Thanks for the reply. I suspected it would be something to do with the 
> tracescript.
>
> As an experiment I tried adding the following to the end of 
> _serializensp in nstrace.tcl
>
> # Add aliases
>             foreach alias [interp aliases {}] {
>                 if { [string match "$nsp*" $alias] } {
>                     append script [_aliasscript $alias]
>                 }
>             }
>
> where _aliasscript is
>
>         proc _aliasscript {cmd} {
>             return "interp alias {} $cmd {} [interp alias {} $cmd] \n"
>         }
>
>
> And it did indeed start working.  Not sure where the best way/best 
> place to do this for the naviserver code base. Any comments?

Re: [naviserver-devel] Losing interp alias

[David O. <da...@qc...>]

Thanks for the reply. I suspected it would be something to do with the
tracescript.

As an experiment I tried adding the following to the end of _serializensp
in nstrace.tcl

            # Add aliases
            foreach alias [interp aliases {}] {
                if { [string match "$nsp*" $alias] } {
                    append script [_aliasscript $alias]
                }
            }

where _aliasscript is

        proc _aliasscript {cmd} {
            return "interp alias {} $cmd {} [interp alias {} $cmd] \n"
        }


And it did indeed start working.  Not sure where the best way/best place to
do this for the naviserver code base. Any comments?


On 26 September 2013 19:09, Jeff Rogers <dv...@di...> wrote:

> Hi David,
>
> This is a known deficiency - the introspection script that creates the
> tcl initialization script doesn't capture interp aliases.  I don't think
> it's difficult to add, just hasn't been done yet.
>
> -J
>
>
>

Re: [naviserver-devel] Losing interp alias

[Jeff R. <dv...@di...>]

Hi David,

This is a known deficiency - the introspection script that creates the 
tcl initialization script doesn't capture interp aliases.  I don't think 
it's difficult to add, just hasn't been done yet.

-J

David Osborne wrote:
> Hi,
>
> Wondering if you can help with a problem.
>
> I am attempting to preload the trf accelerated version of tcllib's
> ::md5::md5 in our naviserver config.
>
> As a simplified testcase to demonstrate what's happening I can do the
> following:
>
> - use a default configuration of naviserver, e.g. in /usr/local/ns
> - copy osborne-procs.tcl to/usr/local/ns/modules/tcl/osborne-procs.tcl
> where osborne-procs.tcl contains:
>
> package require Trf
> package require md5
>
> - use the sample config as distributed with naviserver
> /usr/local/ns/bin/nsd -c -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl
>
> I get the following result:
>
> % ::md5::md5 -hex teststring
> invalid command name "Hex"
>
> This seems to be because the tcllib md5x.tcl code, checks for the Trf
> package, and if present sets up aninterp alias for the::md5::Hex
> command. And this seems to be getting lost somewhere.
>
> line 502:
> http://tcllib.cvs.sourceforge.net/viewvc/tcllib/tcllib/modules/md5/md5x.tcl?revision=1.19&view=markup
>
> If I subsequently add the alias manually everything is fine:
> % interp alias {} ::md5::Hex {} ::hex -mode encode --
> ::md5::Hex
> % ::md5::md5 -hex teststring
> D67C5CBF5B01C9F91932E3B8DEF5E5F8
>
> Can anyone shed any light as to what's going on here?
>
>
> # info patchlevel
> 8.5.8
>
> tcllib:
>    Installed: 1.12-dfsg-2
> tcl-trf:
>    Installed: 2.1.4-dfsg-2
>
> # /usr/local/ns/bin/nsd -V
> NaviServer/4.99.6
>     Tag:             16f91bafc825+ default tip
>     Built:           Aug 19 2013 at 11:26:14
>     Tcl version:     8.5
>     Platform:        linux
>
>
> --
> David
>

[naviserver-devel] Losing interp alias

[David O. <da...@qc...>]

Hi,

Wondering if you can help with a problem.

I am attempting to preload the trf accelerated version of tcllib's
::md5::md5 in our naviserver config.

As a simplified testcase to demonstrate what's happening I can do the
following:

- use a default configuration of naviserver, e.g. in /usr/local/ns
- copy osborne-procs.tcl to
/usr/local/ns/modules/tcl/osborne-procs.tclwhere osborne-procs.tcl
contains:

package require Trf
package require md5

- use the sample config as distributed with naviserver
/usr/local/ns/bin/nsd -c -u nsadmin -t /usr/local/ns/conf/nsd-config.tcl

I get the following result:

% ::md5::md5 -hex teststring
invalid command name "Hex"

This seems to be because the tcllib md5x.tcl code, checks for the Trf
package, and if present sets up an interp alias for the ::md5::Hex command.
And this seems to be getting lost somewhere.

line 502:
http://tcllib.cvs.sourceforge.net/viewvc/tcllib/tcllib/modules/md5/md5x.tcl?revision=1.19&view=markup

If I subsequently add the alias manually everything is fine:
% interp alias {} ::md5::Hex {} ::hex -mode encode --
::md5::Hex
% ::md5::md5 -hex teststring
D67C5CBF5B01C9F91932E3B8DEF5E5F8

Can anyone shed any light as to what's going on here?


# info patchlevel
8.5.8

tcllib:
  Installed: 1.12-dfsg-2
tcl-trf:
  Installed: 2.1.4-dfsg-2

# /usr/local/ns/bin/nsd -V
NaviServer/4.99.6
   Tag:             16f91bafc825+ default tip
   Built:           Aug 19 2013 at 11:26:14
   Tcl version:     8.5
   Platform:        linux


-- 
David

[naviserver-devel] Announce: OpenACS 5.8.0 released

[Gustaf N. <ne...@wu...>]

Dear all,

We are proud to announce the release of OpenACS 5.8.0.
The new release differs from OpenACS 5.7.0 in the following points:

  - Compatibility with PostgreSQL 9.2:
    The new version installs without any need for special
    parameter settings in new PostgreSQL versions.
    This makes it easier to use e.g. shared or packaged
    PostgreSQL installations.
  - Compatibility with NaviServer 4.99.5 or newer
  - Various performance and scalability improvements
  - Various bug fixes
  - Altogether, OpenACS 5.8.0 differs from OpenACS 5.7.0
    in more than 18.000 modifications (781 commits)
    contributed by 7 committers.

Many thanks to all the help!

Release Announcement: http://openacs.org/Announce-OpenACS-5.8.0
Details and Download: http://openacs.org/

Re: [naviserver-devel] Clock.tcl repeat initialisation after ns_eval

[Gustaf N. <ne...@wu...>]

Am 19.08.13 15:55, schrieb David Osborne:
> It still looks to me like something unintended is happening 
> internally. However, the clock code handles the error and falls back 
> to returning the correct result. So it may well be deemed as working 
> as designed!
i guess we to assume this. we don't have to be concerned what is happen 
internally in tcl, as long the documented interface is fine.

many thanks and all the best
-gustaf

Re: [naviserver-devel] Clock.tcl repeat initialisation after ns_eval

[David O. <da...@qc...>]

Hi Gustaf,

Ah yes, as we discussed before, using :localtime will not throw a hard
error. But the same error is occurring under the covers - but but the clock
code catches it, and falls back to using :localtime.

If I change clock_test to examine and output ::errorInfo and
::tcl::clock::CachedSystemTimeZone after the calls to clock (using
-timezone :localtime) you can see the same thing happened:

ns_schedule_proc -once 0 clock_test
1
% 2013-08-19 14:51:00
CachedSystemTimezone = :Tcl/Localtime
ns_eval {expr 1+1}
2
% [19/Aug/2013:14:51:32][16319.7fad07e38700][-ns_job_0-] Notice: Starting
thread: -ns_job_0-
ns_schedule_proc -once 0 clock_test
2
% 2013-08-19 14:51:37
CachedSystemTimezone = :Tcl/Localtime
time zone ":Tcl/Localtime" not found
    while executing
"SetupTimeZone $timezone"

It still looks to me like something unintended is happening internally.
However, the clock code handles the error and falls back to returning the
correct result. So it may well be deemed as working as designed!



On 19 August 2013 13:03, Gustaf Neumann <ne...@wu...> wrote:

>  David,
> Hmm, i guess you are using in clock_test still
>
>     ns_log Notice [clock format [clock scan now] -format "%Y-%m-%d %H:%M:%S" -timezone :Tcl/Localtime]
>
>
> ...rather than
>
>     ns_log Notice [clock format [clock scan now] -format "%Y-%m-%d %H:%M:%S" -timezone :localtime]
>
>
> The first one is not documented, the second one is.
> http://www.tcl.tk/man/tcl8.5/TclCmd/clock.htm
>
> At least, i can't reproduce the problem with :localtime, all the
> invocation patterns work fine for me.
> Do you still get same errors, when using ":localtime"?
>
>
> all the best
> -gustaf neumann
>
>
> Am 19.08.13 12:57, schrieb David Osborne:
>
>  Hi Gustaf,
>
>  I made a build from current tip and retested as per the testing scenario
> you specified previously (/usr/local/ns/bin/nsd -c -u nsadmin -t
> /usr/local/ns/conf/nsd-config.tcl with the clock_test proc included as a
> module), and still see the error:
>
> % ns_schedule_proc -once 0 clock_test
> 1
> % 2013-08-19 11:35:33
> ns_eval expr {1+1}
> 2
> % [19/Aug/2013:11:35:40][16030.7fb130f43700][-ns_job_0-] Notice: Starting
> thread: -ns_job_0-
> ns_schedule_proc -once 0 clock_test
> 2
> % [19/Aug/2013:11:35:46][16030.7fb1318d9700][-sched-] Error: time zone
> ":Tcl/Localtime" not found
> time zone ":Tcl/Localtime" not found
>     while executing
> "return -options $opts $retval"
>     (procedure "::tcl::clock::format" line 18)
>     invoked from within
> "clock format [clock scan now] -format "%Y-%m-%d %H:%M:%S" -timezone
> :Tcl/Localtime"
>     (procedure "clock_test" line 2)
>     invoked from within
> "clock_test"
>     while executing callback
> ns:tclschedproc clock_test
>
> % info patchlevel
> 8.5.8
> % ns_info patchlevel
> 4.99.6
>
>
>
>
> --
> David
>
>
> On 16 August 2013 18:51, Gustaf Neumann <ne...@wu...> wrote:
>
>>  Hi David,
>>
>> Jeffs commit with the ensemble-serializer has changed the situation of
>> ::clock
>>
>> https://bitbucket.org/naviserver/naviserver/commits/10bf51a04b2fd746a871d2ef13b75a87a7101f6f
>>
>> This change adds an ensembles-create-commands  to the blueprint for
>> commands, which might refer to the ::tcl::* namespace. This can happen for
>> ensembles in arbitrary namespaces, such as for ::info, ::string etc. If one
>> of the implementation subcommands is in ::tcl::*, we have a problem, when
>> at the time of the ensemble-recreation (in the blueprint) the subcommand is
>> not defined.
>>
>> It had now a similar problem as in your case with the current code, while
>> testing with a relative small blueprint, where suddenly "clock scan"
>> stopped working. The ::clock ensemble was loaded correctly, but the
>> underlying ::tcl::clock::scan not. It seems, at least for ::clock, we have
>> either to omit both, ::clock (the proc or ensemble) and ::tcl::* namespace,
>> or include both. Just omitting just ::tcl::* can cause harm.
>>
>> By including the ::tcl::* namespace again in the blueprint, these
>> problems are gone (commited just now).
>> Can you please recheck your problem case with clock with the current
>> code; in my testing, it works fine.
>>
>> Many thanks and all the best
>> -gustaf
>>
>>
>
>
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> naviserver-devel mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>
>
> --
> Univ.Prof. Dr. Gustaf Neumann
> Institute of Information Systems and New Media
> WU Vienna
> Augasse 2-6, A-1090 Vienna, AUSTRIA
>
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>


-- 
David Osborne
Qcode Software Limited
http://www.qcode.co.uk
T: +44 (0)131 208 0151

Re: [naviserver-devel] Clock.tcl repeat initialisation after ns_eval

[Gustaf N. <ne...@wu...>]

David,
Hmm, i guess you are using in clock_test still

     ns_log Notice [clock format [clock scan now] -format "%Y-%m-%d %H:%M:%S" -timezone :Tcl/Localtime]

...rather than

     ns_log Notice [clock format [clock scan now] -format "%Y-%m-%d %H:%M:%S" -timezone :localtime]

The first one is not documented, the second one is.
http://www.tcl.tk/man/tcl8.5/TclCmd/clock.htm

At least, i can't reproduce the problem with :localtime, all the 
invocation patterns work fine for me.
Do you still get same errors, when using ":localtime"?

all the best
-gustaf neumann


Am 19.08.13 12:57, schrieb David Osborne:
> Hi Gustaf,
>
> I made a build from current tip and retested as per the testing 
> scenario you specified previously (/usr/local/ns/bin/nsd -c -u nsadmin 
> -t /usr/local/ns/conf/nsd-config.tcl with the clock_test proc included 
> as a module), and still see the error:
>
> % ns_schedule_proc -once 0 clock_test
> 1
> % 2013-08-19 11:35:33
> ns_eval expr {1+1}
> 2
> % [19/Aug/2013:11:35:40][16030.7fb130f43700][-ns_job_0-] Notice: 
> Starting thread: -ns_job_0-
> ns_schedule_proc -once 0 clock_test
> 2
> % [19/Aug/2013:11:35:46][16030.7fb1318d9700][-sched-] Error: time zone 
> ":Tcl/Localtime" not found
> time zone ":Tcl/Localtime" not found
>     while executing
> "return -options $opts $retval"
>     (procedure "::tcl::clock::format" line 18)
>     invoked from within
> "clock format [clock scan now] -format "%Y-%m-%d %H:%M:%S" -timezone 
> :Tcl/Localtime"
>     (procedure "clock_test" line 2)
>     invoked from within
> "clock_test"
>     while executing callback
> ns:tclschedproc clock_test
>
> % info patchlevel
> 8.5.8
> % ns_info patchlevel
> 4.99.6
>
>
>
>
> -- 
> David
>
>
> On 16 August 2013 18:51, Gustaf Neumann <ne...@wu... 
> <mailto:ne...@wu...>> wrote:
>
>     Hi David,
>
>     Jeffs commit with the ensemble-serializer has changed the
>     situation of ::clock
>     https://bitbucket.org/naviserver/naviserver/commits/10bf51a04b2fd746a871d2ef13b75a87a7101f6f
>
>     This change adds an ensembles-create-commands  to the blueprint
>     for commands, which might refer to the ::tcl::* namespace. This
>     can happen for ensembles in arbitrary namespaces, such as for
>     ::info, ::string etc. If one of the implementation subcommands is
>     in ::tcl::*, we have a problem, when at the time of the
>     ensemble-recreation (in the blueprint) the subcommand is not defined.
>
>     It had now a similar problem as in your case with the current
>     code, while testing with a relative small blueprint, where
>     suddenly "clock scan" stopped working. The ::clock ensemble was
>     loaded correctly, but the underlying ::tcl::clock::scan not. It
>     seems, at least for ::clock, we have either to omit both, ::clock
>     (the proc or ensemble) and ::tcl::* namespace, or include both.
>     Just omitting just ::tcl::* can cause harm.
>
>     By including the ::tcl::* namespace again in the blueprint, these
>     problems are gone (commited just now).
>     Can you please recheck your problem case with clock with the
>     current code; in my testing, it works fine.
>
>     Many thanks and all the best
>     -gustaf
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA

Re: [naviserver-devel] Clock.tcl repeat initialisation after ns_eval

[David O. <da...@qc...>]

Hi Gustaf,

I made a build from current tip and retested as per the testing scenario
you specified previously (/usr/local/ns/bin/nsd -c -u nsadmin -t
/usr/local/ns/conf/nsd-config.tcl with the clock_test proc included as a
module), and still see the error:

% ns_schedule_proc -once 0 clock_test
1
% 2013-08-19 11:35:33
ns_eval expr {1+1}
2
% [19/Aug/2013:11:35:40][16030.7fb130f43700][-ns_job_0-] Notice: Starting
thread: -ns_job_0-
ns_schedule_proc -once 0 clock_test
2
% [19/Aug/2013:11:35:46][16030.7fb1318d9700][-sched-] Error: time zone
":Tcl/Localtime" not found
time zone ":Tcl/Localtime" not found
    while executing
"return -options $opts $retval"
    (procedure "::tcl::clock::format" line 18)
    invoked from within
"clock format [clock scan now] -format "%Y-%m-%d %H:%M:%S" -timezone
:Tcl/Localtime"
    (procedure "clock_test" line 2)
    invoked from within
"clock_test"
    while executing callback
ns:tclschedproc clock_test

% info patchlevel
8.5.8
% ns_info patchlevel
4.99.6




-- 
David


On 16 August 2013 18:51, Gustaf Neumann <ne...@wu...> wrote:

>  Hi David,
>
> Jeffs commit with the ensemble-serializer has changed the situation of
> ::clock
>
> https://bitbucket.org/naviserver/naviserver/commits/10bf51a04b2fd746a871d2ef13b75a87a7101f6f
>
> This change adds an ensembles-create-commands  to the blueprint for
> commands, which might refer to the ::tcl::* namespace. This can happen for
> ensembles in arbitrary namespaces, such as for ::info, ::string etc. If one
> of the implementation subcommands is in ::tcl::*, we have a problem, when
> at the time of the ensemble-recreation (in the blueprint) the subcommand is
> not defined.
>
> It had now a similar problem as in your case with the current code, while
> testing with a relative small blueprint, where suddenly "clock scan"
> stopped working. The ::clock ensemble was loaded correctly, but the
> underlying ::tcl::clock::scan not. It seems, at least for ::clock, we have
> either to omit both, ::clock (the proc or ensemble) and ::tcl::* namespace,
> or include both. Just omitting just ::tcl::* can cause harm.
>
> By including the ::tcl::* namespace again in the blueprint, these problems
> are gone (commited just now).
> Can you please recheck your problem case with clock with the current code;
> in my testing, it works fine.
>
> Many thanks and all the best
> -gustaf
>
>

Re: [naviserver-devel] Clock.tcl repeat initialisation after ns_eval

[Gustaf N. <ne...@wu...>]

Hi David,

Jeffs commit with the ensemble-serializer has changed the situation of 
::clock
https://bitbucket.org/naviserver/naviserver/commits/10bf51a04b2fd746a871d2ef13b75a87a7101f6f

This change adds an ensembles-create-commands  to the blueprint for 
commands, which might refer to the ::tcl::* namespace. This can happen 
for ensembles in arbitrary namespaces, such as for ::info, ::string etc. 
If one of the implementation subcommands is in ::tcl::*, we have a 
problem, when at the time of the ensemble-recreation (in the blueprint) 
the subcommand is not defined.

It had now a similar problem as in your case with the current code, 
while testing with a relative small blueprint, where suddenly "clock 
scan" stopped working. The ::clock ensemble was loaded correctly, but 
the underlying ::tcl::clock::scan not. It seems, at least for ::clock, 
we have either to omit both, ::clock (the proc or ensemble) and ::tcl::* 
namespace, or include both. Just omitting just ::tcl::* can cause harm.

By including the ::tcl::* namespace again in the blueprint, these 
problems are gone (commited just now).
Can you please recheck your problem case with clock with the current 
code; in my testing, it works fine.

Many thanks and all the best
-gustaf

Am 10.07.13 15:01, schrieb David Osborne:
>
> On 8 July 2013 17:16, Gustaf Neumann <ne...@wu... 
> <mailto:ne...@wu...>> wrote:
>
>     The problem is not only the caches (otherwise it would have been
>     sufficient to exclude the ::tcl::* namespaces from the blueprint), but
>     also the definition of proc "clock", which redefines itself in
>     terms of
>     an ensemble. This makes it sensitive to the definition order.
>
>
> So should the exclusion of ::tcl from the blueprint mean there should 
> be no ::tcl::clock variables defined from the previous invocations of 
> ::tcl::clock commands in the interpreter after an ns_eval?
>
>
>     Btw, my impression is that when ":localtime" is used instead of
>     ":Tcl/Localtime" ....
>
>     proc clock_test {} {
>          ns_log Notice [clock format [clock scan now] -format
>     "%Y-%m-%d %H:%M:%S" -timezone :localtime]
>          return true
>     }
>
>     ... then everything is fine even when we do not exclude "proc clock"
>     from the blueprint. Can you check this?
>
>
> Yes. It's very easy to work around the error.
>
> If you call "clock format" with no timezone option at all, the same 
> error happens but is wrapped in a "catch".
> The timezone is defaulted to :Tcl/Localtime. This is set as 
> CachedSystemTimeZone. After a ns_eval, the clock code tries to lookup 
> :Tcl/Localtime in TZData, it's not there so it marks it as a 
> BadTimeZone. The difference is that it then proceeds to fall back to 
> using :localtime instead. But :Tcl/Localtime is still flagged as a 
> BadTimeZone.
>
> So we can operationally get round this. We were just concerned it was 
> exposing a bug.
> Whether or not it needs fixing, is definitely debatable!
>
> -- 
> David Osborne
>

[naviserver-devel] Naviserver Debian package

[David O. <da...@qc...>]

Hi,

In case anyone may find it of use, we have a Debian packaged version of
Naviserver which is available for general use on Squeeze.

The latest version, 4.99.5-4 is based the bitbucket v4.99.5 tag.
The package should be treated as experimental.

To install, add the following line to /etc/apt/sources.list :

deb http://debian.qcode.co.uk squeeze main

Then:
> apt-get update
> apt-get install naviserver

And if required,
> apt-get install naviserver-nsdbpg
> apt-get install naviserver-nsssl

We would welcome comments/contributions regarding the Debian packaging code
so if you're interested the repositories are as follows:

https://github.com/qcode-software/naviserver
https://github.com/qcode-software/naviserver-nsssl-0.2
https://github.com/qcode-software/naviserver-nsdbpg

It's built using the
git-buildpackage<http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.html>tool
.

Regards,
-- 
David

[naviserver-devel] figures from using naviserver on openacs.org

[Gustaf N. <ne...@wu...>]

Dear all,

here some statics about moving openacs.org  from aolserver to naviserver.
Well, we did not only change the change the server, but as well

- pg 8.3->9.2
- xotcl 1.6.7 -> nsf/xotcl 2.0
- OpenACS 5.5 -> 5.8
- tcl 8.5.11 -> tcl 8.5.14

it is not easy to say, how much which part contributed in total,
but in average on the same machine, under the same OS release,
the new site is significantly faster.

http://openacs.org/forums/message-view?message_id=4074774

-gustaf neumann

Re: [naviserver-devel] "::errorInfo": no such variable

[David O. <da...@qc...>]

That looks great Gustaf. Thanks very much for looking at this.

Regards,

-- 
David


On 5 August 2013 09:11, Gustaf Neumann <ne...@wu...> wrote:

> Dear David,
>
> My suspicion that one should not rely on the universal existence of
> ::errorInfo and ::errorCode is backed by the newer definitions of
> init.tcl in tcl 8.5 and 8.6. In these releases, the code checks for the
> existence of these variables before reading its content. I've commit a
> patch, that is quite similar to this.
>
>
> https://bitbucket.org/naviserver/naviserver/commits/d118a87689afe116ee3e618809a4b34f66a75379
>
> all the best
> -gustaf neumann
>
> Am 03.08.13 12:07, schrieb Gustaf Neumann:
> > Am 01.08.13 15:00, schrieb David Osborne:
> >> Hi,
> >>
> >> We intermittently encounter the above error when we call a ns_eval
> >> from our naviserver codebase (with Tcl 8.5.11 ).
> > The code is in "proc getentry" in nstrace.
> >
> > Actually, one should only access the value ::errorInfo and friends
> > immediately after an error occured, that is typically within a catch
> > clause. "proc getentry" seems to try to preserve the previous
> > ::errorInfo and ::errorCode settings (not sure why). I think the access
> > to ::errorInfo should be guarded by an "info exists".
> >
> > Will try this later and do some testing.
> > Greetings from dubai.
> > -g
> >
>
>
>

Re: [naviserver-devel] "::errorInfo": no such variable

[Gustaf N. <ne...@wu...>]

Dear David,

My suspicion that one should not rely on the universal existence of 
::errorInfo and ::errorCode is backed by the newer definitions of 
init.tcl in tcl 8.5 and 8.6. In these releases, the code checks for the 
existence of these variables before reading its content. I've commit a 
patch, that is quite similar to this.

https://bitbucket.org/naviserver/naviserver/commits/d118a87689afe116ee3e618809a4b34f66a75379

all the best
-gustaf neumann

Am 03.08.13 12:07, schrieb Gustaf Neumann:
> Am 01.08.13 15:00, schrieb David Osborne:
>> Hi,
>>
>> We intermittently encounter the above error when we call a ns_eval
>> from our naviserver codebase (with Tcl 8.5.11 ).
> The code is in "proc getentry" in nstrace.
>
> Actually, one should only access the value ::errorInfo and friends
> immediately after an error occured, that is typically within a catch
> clause. "proc getentry" seems to try to preserve the previous
> ::errorInfo and ::errorCode settings (not sure why). I think the access
> to ::errorInfo should be guarded by an "info exists".
>
> Will try this later and do some testing.
> Greetings from dubai.
> -g
>

Re: [naviserver-devel] "::errorInfo": no such variable

[Gustaf N. <ne...@wu...>]

Am 01.08.13 15:00, schrieb David Osborne:
> Hi,
>
> We intermittently encounter the above error when we call a ns_eval 
> from our naviserver codebase (with Tcl 8.5.11 ).
The code is in "proc getentry" in nstrace.

Actually, one should only access the value ::errorInfo and friends 
immediately after an error occured, that is typically within a catch 
clause. "proc getentry" seems to try to preserve the previous 
::errorInfo and ::errorCode settings (not sure why). I think the access 
to ::errorInfo should be guarded by an "info exists".

Will try this later and do some testing.
Greetings from dubai.
-g

[naviserver-devel] "::errorInfo": no such variable

[David O. <da...@qc...>]

Hi,

We intermittently encounter the above error when we call a ns_eval from our
naviserver codebase (with Tcl 8.5.11 ).

I can't reliably reproduce it, but it seems when we call a ns_eval AND a
new thread is created, getentry will attempt to access ::errorInfo (and
::errorCode) when they do not exist yet. (getentry is called to load
tclcurl and tdom shared libraries in our environment)

There is at least some discussion as to whether it is a Tcl bug that these
globals don't exist by default:
https://core.tcl.tk/tcl/tktview?name=3480249fff

I remember there was talk about dropping support for Tcl 8.4 in
Naviserver.. if that was the case could potentially avoid this issue using
something like:

        proc getentry {store var} {            variable epoch
  if {[catch {nsv_set nstrace-${store}-${epoch} $var} val opts]} {
           set ::errorInfo [dict get $opts -errorinfo]
set ::errorCode [dict get $opts -errorcode]                set val ""
          }            return $val        }

Otherwise just an existence check on the globals?
Or do you think there may be another issue at play here?

Thanks.

Error below:

[01/Aug/2013:11:01:02][31152.7f4b6e44c700][-ns_job_1-] Notice: Starting
thread: -ns_job_1-
[01/Aug/2013:11:01:02][31152.7f4b6e44c700][-ns_job_1-] Debug:
ns:interptrace[srv]: create nslog:initinterp
/var/log/naviserver/tlc.access.log
[01/Aug/2013:11:01:02][31152.7f4b6e44c700][-ns_job_1-] Debug:
ns:interptrace[srv]: create p:0x7f4b72096ec0 a:(nil)
[01/Aug/2013:11:01:02][31152.7f4b6e44c700][-ns_job_1-] Debug:
ns:interptrace[srv]: create nsdb:initinterp a:0x20ee560
[01/Aug/2013:11:01:02][31152.7f4b6e44c700][-ns_job_1-] Debug:
ns:interptrace[srv]: create nsproxy:initinterp a:0x2b07380
[01/Aug/2013:11:01:02][31152.7f4b6e44c700][-ns_job_1-] Debug:
ns:interptrace[srv]: allocate ns:tcltrace ns_init
getentry: store = load
getentry: var = /usr/lib/tcltk/TclCurl7.19.6/libTclCurl7.19.6.so
[01/Aug/2013:11:01:02][31152.7f4b6e44c700][-ns_job_1-] Error: can't read
"::errorInfo": no such variable
can't read "::errorInfo": no such variable
    while executing
"set ei $::errorInfo"
    (procedure "nstrace::getentry" line 9)
    invoked from within
"nstrace::getentry load $image"
    (procedure "script::_load" line 5)
    invoked from within
"script::_$cmd"
    (procedure "nstrace::statescript" line 14)
    invoked from within
"nstrace::statescript"
    (procedure "_ns_eval" line 15)
    invoked from within
"_ns_eval { expr {1+1}}"


-- 
David Osborne

Re: [naviserver-devel] Clock.tcl repeat initialisation after ns_eval

[Gustaf N. <ne...@wu...>]

Am 10.07.13 15:01, schrieb David Osborne:
>
> On 8 July 2013 17:16, Gustaf Neumann <ne...@wu... 
> <mailto:ne...@wu...>> wrote:
>
>     The problem is not only the caches (otherwise it would have been
>     sufficient to exclude the ::tcl::* namespaces from the blueprint), but
>     also the definition of proc "clock", which redefines itself in
>     terms of
>     an ensemble. This makes it sensitive to the definition order.
>
>
> So should the exclusion of ::tcl from the blueprint mean there should 
> be no ::tcl::clock variables defined from the previous invocations of 
> ::tcl::clock commands in the interpreter after an ns_eval?
it means that the blueprint does never contain ::tcl::* variables
(and procs), so it will not contain e.g. ::tcl::clock::* variables.
The blueprint will be loaded into an interpreter which might or
might not have ::tcl::* variables already defined. It simply says,
that naviserver will not touch the ::tcl stuff.

In practice this means, when the interp has already
an initialized ::tcl::clock state, this clock state will survive
the update unchanged (e.g. via "ns_ictl update";
an update happens as well in PopInterp(),
when an initialized  interp is provided to a thread).

As long as a naviserver app does not add itself content
into the ::tcl namespace, and expects that this content
is included in the blueprint, excluding is fine.

I would argue, that a naviserver app should not modify
the ::tcl namespace, therefore the change is fine.

-gustaf neumann

Re: [naviserver-devel] Clock.tcl repeat initialisation after ns_eval

[David O. <da...@qc...>]

On 8 July 2013 17:16, Gustaf Neumann <ne...@wu...> wrote:

> The problem is not only the caches (otherwise it would have been
> sufficient to exclude the ::tcl::* namespaces from the blueprint), but
> also the definition of proc "clock", which redefines itself in terms of
> an ensemble. This makes it sensitive to the definition order.
>

So should the exclusion of ::tcl from the blueprint mean there should be no
::tcl::clock variables defined from the previous invocations of
::tcl::clock commands in the interpreter after an ns_eval?


> Btw, my impression is that when ":localtime" is used instead of
> ":Tcl/Localtime" ....
>
> proc clock_test {} {
>      ns_log Notice [clock format [clock scan now] -format "%Y-%m-%d
> %H:%M:%S" -timezone :localtime]
>      return true
> }
>
> ... then everything is fine even when we do not exclude "proc clock"
> from the blueprint. Can you check this?
>

Yes. It's very easy to work around the error.

If you call "clock format" with no timezone option at all, the same error
happens but is wrapped in a "catch".
The timezone is defaulted to :Tcl/Localtime. This is set as
CachedSystemTimeZone. After a ns_eval, the clock code tries to lookup
:Tcl/Localtime in TZData, it's not there so it marks it as a BadTimeZone.
The difference is that it then proceeds to fall back to using :localtime
instead. But :Tcl/Localtime is still flagged as a BadTimeZone.

So we can operationally get round this. We were just concerned it was
exposing a bug.
Whether or not it needs fixing, is definitely debatable!

-- 
David Osborne

Re: [naviserver-devel] Clock.tcl repeat initialisation after ns_eval

[Gustaf N. <ne...@wu...>]

Am 08.07.13 21:12, schrieb Jeff Rogers:
> Sheer luck is definitely a possibility,
what i called "luck" was the random order of entries as returned from the
hash tables during serialization.
> but also naviserver is missing
> the ensemble serialization that aolserver has, so things defined (or
> redefined) as ensembles would get lost.
good catch. this is certainly needed regardless of the clock discussion.

in the clock case, proc clock is defining the ensemble on the fly,

     proc clock args {
	namespace eval ::tcl::clock [list namespace ensemble create -command \
		[uplevel 1 [list namespace origin [lindex [info level 0] 0]]] \
		-subcommands {
		    add clicks format microseconds milliseconds scan seconds
		}]
	
	# Auto-loading stubs for 'clock.tcl'
	
	foreach cmd {add format scan} {
	    proc ::tcl::clock::$cmd args {
		variable TclLibDir
		source -encoding utf-8 [file join $TclLibDir clock.tcl]
		return [uplevel 1 [info level 0]]
	    }
	}

	return [uplevel 1 [info level 0]]
     }

so a missing ensemble is created by proc clock on the fly.
but without proc clock, everything is fine, since tcl-init creates
the ensemble and naviserver does not touch it...
so the situation is somewhere between tricky and wierd.

> I've copied this code over from aolserver.  I don't know if it will help
> the clock problem, although it is possible clock was the motivating
> issue for that code in the first place.
>
> In aolserver the code is wrapped to prevent it from executing on pre-8.5
> tcl.  Is that still needed, or is 8.4 dead enough to have 8.5 as a
> default requirement?

so far, i have tried to keep non-optional stuff in naviserver tcl 8.4 
compatible, but
i have stopped testing against 8.4. so, i would not mind if we would 
require tcl 8.5,
but i know there are naviserver + tcl 8.4 installations out there....

-g