naviserver-devel — Developer discussion

Re: [naviserver-devel] Compilation error NaviServer 4.99.8

[Gustaf N. <ne...@wu...>]

Dear Andrei,

Thanks for the compile-log. The problem is that the compiler flags that are
picked up in your environment include a " -fvisibility=hidden", most likely
picked up from tcl 8.6.1. Using  "-fvisibility=hidden" should work 
certainly,
an was fixed in the tip versions.

The compilation works well with 4.99.8, if you either compile against 
Tcl 8.5.* or
when you use

    make "CFLAGS_EXTRA=-fPIC -pipe"

It might be, that the problems you are experiencing have to do with tcl8.6.
We use on all our production systems tcl 8.5.* (e.g. on openacs.org: Tcl 
8.5.15).
If you have the chance to use tcl 8.5, please try.

all the best
-g


Am 25.09.15 um 10:43 schrieb Clinciu Andrei:
> Hi
>
> I've tried compiling the 4.99.8 and I remembered why I even went to 
> get the bitbucket version, because of failure to compile.
>
> Doing:
> *./configure --prefix=/opt/ns --enable-symbols --enable-threads*
> *make -j8 *
> *or simply 'make' *
> Also trying with autoconf (maybe the configure missed something?)
> This however was not an issue for the bitbucket version at that time
>
> Error:
> libnsd.so: undefined reference to `NS_finalshutdown'
> collect2: error: ld returned 1 exit status
> make[1]: *** [nsd] Error 1
> make[1]: Leaving directory 
> `/home/lostone/naviserver/naviserver-4.99.8/nsd'
> make: *** [all] Error 1
>
> It's worthy to note that I've also compiled the 4.99.7 version 
> *without any problem*
>
> make.log : Paste2.org - Viewing Paste G2a9BEmz 
> <http://paste2.org/G2a9BEmz>
>
> I sure hope i'm not bugging you guys:)
>
> With regards,
> Clinciu Andrei George
>
>
> "Vorba buna, zambetul si fapta binefacatoare sunt raze ale soarelui 
> rasfrante in sufletul omului."
> "A good word, a smile and a good deed are just like rays of the sun 
> reflected in man's soul."   by Nicolae Iorga


-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

[naviserver-devel] Compilation error NaviServer 4.99.8

[Clinciu A. <the...@ya...>]

Hi
I've tried compiling the 4.99.8 and I remembered why I even went to get the bitbucket version, because of failure to compile. 
Doing:./configure --prefix=/opt/ns --enable-symbols --enable-threads
make -j8 or simply 'make' Also trying with autoconf (maybe the configure missed something?)This however was not an issue for the bitbucket version at that time
Error:libnsd.so: undefined reference to `NS_finalshutdown'collect2: error: ld returned 1 exit statusmake[1]: *** [nsd] Error 1make[1]: Leaving directory `/home/lostone/naviserver/naviserver-4.99.8/nsd'make: *** [all] Error 1
It's worthy to note that I've also compiled the 4.99.7 version without any problem
make.log : Paste2.org - Viewing Paste G2a9BEmz
I sure hope i'm not bugging you guys:) 
With regards,
Clinciu Andrei George


"Vorba buna, zambetul si fapta binefacatoare sunt raze ale soarelui rasfrante in sufletul omului."
"A good word, a smile and a good deed are just like rays of the sun reflected in man's soul."   by Nicolae Iorga

Re: [naviserver-devel] Naviserver crash

[Gustaf N. <ne...@wu...>]

Dear Andrei,

In your config-file, you are loading 6 (!) different SQL drivers, and 
you are loading nsdbipg 3 times.
Is this intended?

it is strange that you have closewait larger than keep wait. Is this as 
well intended?

    ns_param    closewait        7            ;# default: 2; timeout in 
seconds for close on socket
    ns_param    keepwait         0     ;# timeout in seconds for keep-alive

openacs.org uses e.g. keepwait 5 and closewait 2

Can you reproduce the problem with the truncated images at your site 
with one of the various
sample configurations, such as e.g. with ns/conf/nsd-config.tcl ?

best regards
-g


Am 24.09.15 um 10:41 schrieb Clinciu Andrei:
> Hi all,
>
> I'd really appreciate the help of someone more experienced with 
> NaviServer since i'm currently using it in production and it seems 
> there are a few things that I can't seem to figure out! Thanks for all 
> your support so far! I've learnt a lot of things about naviserver in 
> the past 2 years.
>
> I've recently compiled (a week ago) the newest version from bitbucket. 
> Server information:
> cat /etc/issue
> *Debian GNU/Linux 7 \n \l*
> uname -a
> *Linux unitedbrainpower 2.6.32-042stab094.7 #1 SMP Wed Oct 22 12:43:21 
> MSK 2014 x86_64 GNU/Linux*
>
> And there are 2 things that I've noted that crash the server every few 
> hours:
>
> 1. First is this assert info, nothing else, no other information.
> *nsd: set.c:89: Ns_SetUpdate: Assertion `value != ((void *)0)' failed.*
> I've been able to capture a place where this error occurs, I've hosted 
> a fossil page via CGI, and If i go to the link i get the same error. 
>  But it seems to generate the error even if CGI is disabled (and I 
> can't pinpoint it!).
>
>  This seems to crash the server aprox *5 times *a day looking into the 
> logs, not talking about me pointing it to the CGI (which i've disabled).
>
> 2. Another error that seems to "crash" the server is:
> *[23/Sep/2015:06:44:12][32506.7f5118eca700][] Fatal: received fatal 
> signal 11*
>
> My server crashes on average *4 times a day *because of this.
>
> Log:
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: exiting: exceeded max connections per thread
> [23/Sep/2015:06:44:12][32506.7f5118dc9700][-driver:nssock-] Notice: 
> NsEnsureRunningConnectionThreads wantCreate 1 waiting 0 idle 3 current 4
>
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_6/0 cols 0: deallocate dbipg_4
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_7/0 cols 0: deallocate dbipg_1
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_8/0 cols 0: deallocate dbipg_3
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_9/0 cols 0: deallocate dbipg_6
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_10/0 cols 0: deallocate dbipg_5
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_11/0 cols 0: deallocate dbipg_2
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_12/0 cols 0: deallocate dbipg_0
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_13/0 cols 0: deallocate dbipg_4
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_14/0 cols 0: deallocate dbipg_7
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_15/0 cols 0: deallocate dbipg_1
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_16/0 cols 0: deallocate dbipg_6
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_17/0 cols 0: deallocate dbipg_0
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_18/0 cols 0: deallocate dbipg_8
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_19/0 cols 0: deallocate dbipg_12
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_20/0 cols 0: deallocate dbipg_11
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_21/0 cols 0: deallocate dbipg_9
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_22/0 cols 0: deallocate dbipg_17
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_23/0 cols 0: deallocate dbipg_3
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_24/0 cols 0: deallocate dbipg_15
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_25/0 cols 0: deallocate dbipg_10
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_26/0 cols 0: deallocate dbipg_20
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_27/0 cols 0: deallocate dbipg_23
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_28/0 cols 0: deallocate dbipg_19
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_29/0 cols 0: deallocate dbipg_5
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_30/0 cols 0: deallocate dbipg_13
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_31/0 cols 0: deallocate dbipg_2
> [23/Sep/2015:06:44:12][32506.7f5118eca700][] Fatal: received fatal 
> signal 11
>
> View config at: http://pastebin.com/qAKiTSSb
>
> *Any ideas?*
>
> 3. Images are truncated or sent corrupted (same happens to JS/CSS!)
>
> I've played tens of times with all the settings, enabled and disabled 
> everything from spoolers to fastpath and even adp rendering.
> At the moment I set * ns_param**writerstreaming**false * and the 
> keepalive is 5 seconds, the page renders in 5 seconds! (in any 
> browser, on anyother pc) And this doesn't solve the images problem.
>
> While it seems that after a few refreshes the page works as expected, 
> the images are not sent correctly.
> I've reviewed the information in the access log and a 25kb image is 
> sent first as a 17kb and then the remaining on the next refresh.
>
> Each document has a preview (random image selected from the images of 
> the document, each page is cached so you see the same image if you 
> refresh) and in Firefox I can render it ultimately. But in Opera, 
> Chrome and Internet Explorer it seems to never show the image, only 
> after maybe the 3d refresh.
>
> You can view what i'm talking about at http://bibliografie.info 
> <http://bibliografie.info./> by pressing F12 while loading..
>
>  My (deleted out database configuration options) configuration file 
> looks like this: http://pastebin.com/qAKiTSSb
>
> How can I reduce the number of crashes? I can live with a crash per 
> day but having 10 (or more!) per day is annoying since it disrupts 
> stuff like sessions (saved in cache), file transfers and other things.
>
> Thanks!
>
> With regards,
> Clinciu Andrei George
>
> "Vorba buna, zambetul si fapta binefacatoare sunt raze ale soarelui 
> rasfrante in sufletul omului."
> "A good word, a smile and a good deed are just like rays of the sun 
> reflected in man's soul."   by Nicolae Iorga
>

Re: [naviserver-devel] Naviserver crash

[Gustaf N. <ne...@wu...>]

Dear Andrei,

from some distance, the problem looks like a version mismatch in your 
installation.
Unless for NaviServer developer, it is recommended to use the released 
versions,
and not "some catch of the day" of various repositories at bitbucket. Using
releases makes it as well much easier to reproduce potential problems.

Please use NaviServer 4.99.8 and the matching modules from sourceforge.
In case the problem persists, please activate core dumps on your machine,
compile with -g and submit the bug-report with a backtrace (in case of
questions, feel free to ask me).

all the best
-g


[1] https://sourceforge.net/projects/naviserver/files/naviserver/4.99.8/


Am 24.09.15 um 10:41 schrieb Clinciu Andrei:
> Hi all,
>
> I'd really appreciate the help of someone more experienced with 
> NaviServer since i'm currently using it in production and it seems 
> there are a few things that I can't seem to figure out! Thanks for all 
> your support so far! I've learnt a lot of things about naviserver in 
> the past 2 years.
>
> I've recently compiled (a week ago) the newest version from bitbucket. 
> Server information:
> cat /etc/issue
> *Debian GNU/Linux 7 \n \l*
> uname -a
> *Linux unitedbrainpower 2.6.32-042stab094.7 #1 SMP Wed Oct 22 12:43:21 
> MSK 2014 x86_64 GNU/Linux*
>
> And there are 2 things that I've noted that crash the server every few 
> hours:
>
> 1. First is this assert info, nothing else, no other information.
> *nsd: set.c:89: Ns_SetUpdate: Assertion `value != ((void *)0)' failed.*
> I've been able to capture a place where this error occurs, I've hosted 
> a fossil page via CGI, and If i go to the link i get the same error. 
>  But it seems to generate the error even if CGI is disabled (and I 
> can't pinpoint it!).
>
>  This seems to crash the server aprox *5 times *a day looking into the 
> logs, not talking about me pointing it to the CGI (which i've disabled).
>
> 2. Another error that seems to "crash" the server is:
> *[23/Sep/2015:06:44:12][32506.7f5118eca700][] Fatal: received fatal 
> signal 11*
>
> My server crashes on average *4 times a day *because of this.
>
> Log:
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: exiting: exceeded max connections per thread
> [23/Sep/2015:06:44:12][32506.7f5118dc9700][-driver:nssock-] Notice: 
> NsEnsureRunningConnectionThreads wantCreate 1 waiting 0 idle 3 current 4
>
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_6/0 cols 0: deallocate dbipg_4
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_7/0 cols 0: deallocate dbipg_1
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_8/0 cols 0: deallocate dbipg_3
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_9/0 cols 0: deallocate dbipg_6
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_10/0 cols 0: deallocate dbipg_5
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_11/0 cols 0: deallocate dbipg_2
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_12/0 cols 0: deallocate dbipg_0
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_13/0 cols 0: deallocate dbipg_4
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_14/0 cols 0: deallocate dbipg_7
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_15/0 cols 0: deallocate dbipg_1
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_16/0 cols 0: deallocate dbipg_6
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_17/0 cols 0: deallocate dbipg_0
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_18/0 cols 0: deallocate dbipg_8
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_19/0 cols 0: deallocate dbipg_12
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_20/0 cols 0: deallocate dbipg_11
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_21/0 cols 0: deallocate dbipg_9
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_22/0 cols 0: deallocate dbipg_17
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_23/0 cols 0: deallocate dbipg_3
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_24/0 cols 0: deallocate dbipg_15
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_25/0 cols 0: deallocate dbipg_10
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_26/0 cols 0: deallocate dbipg_20
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_27/0 cols 0: deallocate dbipg_23
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_28/0 cols 0: deallocate dbipg_19
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_29/0 cols 0: deallocate dbipg_5
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_30/0 cols 0: deallocate dbipg_13
> [23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] 
> Notice: dbipg: prepare dbipg_31/0 cols 0: deallocate dbipg_2
> [23/Sep/2015:06:44:12][32506.7f5118eca700][] Fatal: received fatal 
> signal 11
>
> View config at: http://pastebin.com/qAKiTSSb
>
> *Any ideas?*
>
> 3. Images are truncated or sent corrupted (same happens to JS/CSS!)
>
> I've played tens of times with all the settings, enabled and disabled 
> everything from spoolers to fastpath and even adp rendering.
> At the moment I set * ns_param**writerstreaming**false * and the 
> keepalive is 5 seconds, the page renders in 5 seconds! (in any 
> browser, on anyother pc) And this doesn't solve the images problem.
>
> While it seems that after a few refreshes the page works as expected, 
> the images are not sent correctly.
> I've reviewed the information in the access log and a 25kb image is 
> sent first as a 17kb and then the remaining on the next refresh.
>
> Each document has a preview (random image selected from the images of 
> the document, each page is cached so you see the same image if you 
> refresh) and in Firefox I can render it ultimately. But in Opera, 
> Chrome and Internet Explorer it seems to never show the image, only 
> after maybe the 3d refresh.
>
> You can view what i'm talking about at http://bibliografie.info 
> <http://bibliografie.info./> by pressing F12 while loading..
>
>  My (deleted out database configuration options) configuration file 
> looks like this: http://pastebin.com/qAKiTSSb
>
> How can I reduce the number of crashes? I can live with a crash per 
> day but having 10 (or more!) per day is annoying since it disrupts 
> stuff like sessions (saved in cache), file transfers and other things.
>
> Thanks!
>
> With regards,
> Clinciu Andrei George
>
> "Vorba buna, zambetul si fapta binefacatoare sunt raze ale soarelui 
> rasfrante in sufletul omului."
> "A good word, a smile and a good deed are just like rays of the sun 
> reflected in man's soul."   by Nicolae Iorga
>
>

-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

[naviserver-devel] Naviserver crash

[Clinciu A. <the...@ya...>]

Hi all,
I'd really appreciate the help of someone more experienced with NaviServer since i'm currently using it in production and it seems there are a few things that I can't seem to figure out! Thanks for all your support so far! I've learnt a lot of things about naviserver in the past 2 years. 
I've recently compiled (a week ago) the newest version from bitbucket. Server information:cat /etc/issueDebian GNU/Linux 7 \n \luname -aLinux unitedbrainpower 2.6.32-042stab094.7 #1 SMP Wed Oct 22 12:43:21 MSK 2014 x86_64 GNU/Linux
And there are 2 things that I've noted that crash the server every few hours:
1. First is this assert info, nothing else, no other information.nsd: set.c:89: Ns_SetUpdate: Assertion `value != ((void *)0)' failed.
I've been able to capture a place where this error occurs, I've hosted a fossil page via CGI, and If i go to the link i get the same error.  But it seems to generate the error even if CGI is disabled (and I can't pinpoint it!).
 This seems to crash the server aprox 5 times a day looking into the logs, not talking about me pointing it to the CGI (which i've disabled). 
2. Another error that seems to "crash" the server is:[23/Sep/2015:06:44:12][32506.7f5118eca700][] Fatal: received fatal signal 11
My server crashes on average 4 times a day because of this. 
Log:[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: exiting: exceeded max connections per thread[23/Sep/2015:06:44:12][32506.7f5118dc9700][-driver:nssock-] Notice: NsEnsureRunningConnectionThreads wantCreate 1 waiting 0 idle 3 current 4
[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_6/0 cols 0: deallocate dbipg_4[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_7/0 cols 0: deallocate dbipg_1[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_8/0 cols 0: deallocate dbipg_3[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_9/0 cols 0: deallocate dbipg_6[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_10/0 cols 0: deallocate dbipg_5[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_11/0 cols 0: deallocate dbipg_2[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_12/0 cols 0: deallocate dbipg_0[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_13/0 cols 0: deallocate dbipg_4[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_14/0 cols 0: deallocate dbipg_7[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_15/0 cols 0: deallocate dbipg_1[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_16/0 cols 0: deallocate dbipg_6[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_17/0 cols 0: deallocate dbipg_0[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_18/0 cols 0: deallocate dbipg_8[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_19/0 cols 0: deallocate dbipg_12[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_20/0 cols 0: deallocate dbipg_11[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_21/0 cols 0: deallocate dbipg_9[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_22/0 cols 0: deallocate dbipg_17[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_23/0 cols 0: deallocate dbipg_3[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_24/0 cols 0: deallocate dbipg_15[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_25/0 cols 0: deallocate dbipg_10[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_26/0 cols 0: deallocate dbipg_20[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_27/0 cols 0: deallocate dbipg_23[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_28/0 cols 0: deallocate dbipg_19[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_29/0 cols 0: deallocate dbipg_5[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_30/0 cols 0: deallocate dbipg_13[23/Sep/2015:06:44:12][32506.7f5118eca700][-conn:ubpserver1:2-] Notice: dbipg: prepare dbipg_31/0 cols 0: deallocate dbipg_2[23/Sep/2015:06:44:12][32506.7f5118eca700][] Fatal: received fatal signal 11
View config at: http://pastebin.com/qAKiTSSb
Any ideas?
3. Images are truncated or sent corrupted (same happens to JS/CSS!)
I've played tens of times with all the settings, enabled and disabled everything from spoolers to fastpath and even adp rendering. At the moment I set    ns_param writerstreaming false  and the keepalive is 5 seconds, the page renders in 5 seconds! (in any browser, on anyother pc) And this doesn't solve the images problem.
While it seems that after a few refreshes the page works as expected, the images are not sent correctly.I've reviewed the information in the access log and a 25kb image is sent first as a 17kb and then the remaining on the next refresh.

Each document has a preview (random image selected from the images of the document, each page is cached so you see the same image if you refresh) and in Firefox I can render it ultimately. But in Opera, Chrome and Internet Explorer it seems to never show the image, only after maybe the 3d refresh.

You can view what i'm talking about at http://bibliografie.info by pressing F12 while loading..

 My (deleted out database configuration options) configuration file looks like this: http://pastebin.com/qAKiTSSb
How can I reduce the number of crashes? I can live with a crash per day but having 10 (or more!) per day is annoying since it disrupts stuff like sessions (saved in cache), file transfers and other things.
Thanks!
With regards,
Clinciu Andrei George

"Vorba buna, zambetul si fapta binefacatoare sunt raze ale soarelui rasfrante in sufletul omului."
"A good word, a smile and a good deed are just like rays of the sun reflected in man's soul."   by Nicolae Iorga

Re: [naviserver-devel] Remove SSLv3 from protocols in nsssl?

[Cesáreo G. R. <ce...@ce...>]

Hi Gustaf,

You're welcome ;-).

I don't upgrade my certificate to get an A+ rating yet. I did use sha1 
when I've create my certificates almost 2 years ago and I had to 
re-create it.

I mean, to get an A+ rating, some people (like me) have to re-create 
certificates using SHA2.

Thanks
Cesáreo

El 18/septiembre/15 a las 15:32, Gustaf Neumann escribió:
> Hi Cesáreo,
>
> thanks for noting! We had this already removed (e.g. on
> next-scripting.org, which has a A+ rating),
> but it was not yet adapted in the README file.
>
> -gn
> Am 18.09.15 um 20:55 schrieb Cesáreo García Rodicio:
>> Hi
>>
>> To get an A rating in SSLlabs SSL Server Test I had to remove SSLv3
>> (poodle attack[2]) in nssl configuration example [3]
>>
>> << ns_param      protocols            "!SSLv2"
>>    >> ns_param      protocols            "!SSLv2:!SSLv3"
>>
>> Regards,
>> Cesáreo
>>
>> [1] https://www.ssllabs.com/ssltest/analyze.html
>> [2]
>> https://community.qualys.com/blogs/securitylabs/2014/10/15/ssl-3-is-dead-killed-by-the-poodle-attack
>> [3] https://bitbucket.org/naviserver/nsssl
>>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>

Re: [naviserver-devel] Remove SSLv3 from protocols in nsssl?

[Gustaf N. <ne...@wu...>]

Hi Cesáreo,

thanks for noting! We had this already removed (e.g. on 
next-scripting.org, which has a A+ rating),
but it was not yet adapted in the README file.

-gn
Am 18.09.15 um 20:55 schrieb Cesáreo García Rodicio:
> Hi
>
> To get an A rating in SSLlabs SSL Server Test I had to remove SSLv3
> (poodle attack[2]) in nssl configuration example [3]
>
> << ns_param      protocols            "!SSLv2"
>   >> ns_param      protocols            "!SSLv2:!SSLv3"
>
> Regards,
> Cesáreo
>
> [1] https://www.ssllabs.com/ssltest/analyze.html
> [2]
> https://community.qualys.com/blogs/securitylabs/2014/10/15/ssl-3-is-dead-killed-by-the-poodle-attack
> [3] https://bitbucket.org/naviserver/nsssl
>

[naviserver-devel] Remove SSLv3 from protocols in nsssl?

[Cesáreo G. R. <ce...@ce...>]

Hi

To get an A rating in SSLlabs SSL Server Test I had to remove SSLv3 
(poodle attack[2]) in nssl configuration example [3]

<< ns_param      protocols            "!SSLv2"
 >> ns_param      protocols            "!SSLv2:!SSLv3"

Regards,
Cesáreo

[1] https://www.ssllabs.com/ssltest/analyze.html
[2] 
https://community.qualys.com/blogs/securitylabs/2014/10/15/ssl-3-is-dead-killed-by-the-poodle-attack
[3] https://bitbucket.org/naviserver/nsssl

Re: [naviserver-devel] ns_ssl and binary data

[Gustaf N. <ne...@wu...>]

Hi David,

The implicit type conversion on the variables are a pain, but there are 
already a couple
of places where this happens. Using always the Tcl_GetByteArrayFromObj 
can lead
to encoding problems. One has actually the binary/non-binary problem and the
encoding problem.

I have developed a few test cases for ns_http and ns_ssl which should 
help to
make the code more stable and to refine certain corner cases. Furthermore
i added an api call for "binary" content-types for which 
Tcl_GetByteArrayFromObj()
should always be used. The interface should be probably extended for 
ensuring
as well binary transmission for gzipped content, i will check this the 
next days.

-gn

Am 30.07.15 um 12:33 schrieb David Osborne:
> Thanks Gustaf - I have just tried these changes and it appear to work 
> just fine.
>
> I just have to be careful when handling the binary content to pass to 
> the -body option, if I inadvertently force an internal string 
> representation to be generated then if will be corrupted by ns_ssl in 
> transit since it will treat it like a string.
>
> Previously I had:
>
> set content [ns_conn content -binary]
> if { $content ne "" } {
>    lappend cmd -body $content
> }
>
> which didn't work with this code change, but sticking to bytearray 
> aware commands worked:
>
> set content [ns_conn content -binary]
> if { [string length $content] > 0 } {
>    lappend cmd -body $content
> }
>
>
> On 29 July 2015 at 13:55, Gustaf Neumann <ne...@wu... 
> <mailto:ne...@wu...>> wrote:
>
>
>     hi david,
>
>     i've commit a change that handles binary data different form
>     non-binary
>     data, as used on several
>     places in NaviServer. With this change, the problem should be solved.
>
>     all the best
>     -g
>
>
>
> ------------------------------------------------------------------------------
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] ns_ssl and binary data

[David O. <da...@qc...>]

Thanks Gustaf - I have just tried these changes and it appear to work just
fine.

I just have to be careful when handling the binary content to pass to the
-body option, if I inadvertently force an internal string representation to
be generated then if will be corrupted by ns_ssl in transit since it will
treat it like a string.

Previously I had:

set content [ns_conn content -binary]
if { $content ne "" } {
   lappend cmd -body $content
}

which didn't work with this code change, but sticking to bytearray aware
commands worked:

set content [ns_conn content -binary]
if { [string length $content] > 0 } {
   lappend cmd -body $content
}


On 29 July 2015 at 13:55, Gustaf Neumann <ne...@wu...> wrote:

>
> hi david,
>
> i've commit a change that handles binary data different form non-binary
> data, as used on several
> places in NaviServer. With this change, the problem should be solved.
>
> all the best
> -g
>
>

Re: [naviserver-devel] Disabling directorylisting

[Gustaf N. <ne...@wu...>]

Hi David,
i see nothing wrong and no bad side effects with this change.
please make a pull request
-g

Am 23.07.15 um 10:46 schrieb David Osborne:
> Hi,
>
> I came across this while trying to turn fastpath directorylisting off. 
> From the docs:
>
> directorylisting
>
>     Style of directory listings, Can be *fancy* or *simple*. (string,
>     defaults to simple)
>
> directoryproc
>
>     Name of Tcl proc to use to display directory listings. One can
>     either specify *directoryproc*, or *directoryadp*, but not both.
>     (string, defaults to _ns_dirlist)
>
>
> The docs don't explicitly say how to disable them as far as I can see. 
> But fastpath.tcl seems to default to a value of "none" if no config 
> value is set.
> Should Naviserver support this value explicitly?
>
> I tried:
>
> ns_param directorylisting none
>
> This then gave be a Tcl error:
>
> invalid command name "none"
>     while executing
> "none"
>     while executing callback
> ns:tclrequest none
> (context: request proc)
>
> It seems the value "none" is being used as the directoryproc. When 
> directoryproc is not set, and directorylisting is not "simple" or 
> "fancy", dirproc is set to p which will be "none":
>
>     p = Ns_ConfigString(path, "directorylisting", "simple");
>     if (p != NULL && (STREQ(p, "simple") || STREQ(p, "fancy"))) {
>         p = "_ns_dirlist";
>     }
>     servPtr->fastpath.dirproc = Ns_ConfigString(path, "directoryproc", p);
>
> To make fastpath return a 404 instead of a dir listing I needed both :
>
> ns_param directorylisting none
> ns_param directoryproc _ns_dirlist
>
> Is there another way of doing this?
> Strikes me that directoryproc should default to _ns_dirlist no matter 
> what directorylisting is set to?
> Could naviserver drop this check in fastpath.c, and leave the default 
> value of directorylisting to be set in fastpath.tcl as in the 
> following commit?
>
> https://bitbucket.org/davidqc/naviserver/commits/afd8de3e5bdc6813ab49c6bc5713faeaaf68b854
>
> Regards,
> -- 
> David Osborne
> Qcode Software Limited
>
>
> ------------------------------------------------------------------------------
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] ns_ssl and binary data

[Gustaf N. <ne...@wu...>]

Am 29.07.15 um 13:44 schrieb David Osborne:
> I found a commit for aolserver which makes the change to ns_http which 
> I think I am looking for which treats the data referenced by bodyPtr 
> as a byte array when appending it to the request in httpPtr->ds....
> Would there be any consequences of storing a bytearray inside this 
> dynamic string as happens in aolserver?
hi david,

i've commit a change that handles binary data different form non-binary 
data, as used on several
places in NaviServer. With this change, the problem should be solved.

all the best
-g

Re: [naviserver-devel] ns_ssl and binary data

[David O. <da...@qc...>]

Hi,
I found a commit for aolserver which makes the change to ns_http which I
think I am looking for which treats the data referenced by bodyPtr as a
byte array when appending it to the request in httpPtr->ds.
This stops invalid utf-8 bytes being transformed as when using
Tcl_GetStringFromObj which is what is currently used.

https://github.com/aolserver/aolserver/commit/a6bbca2a509be0e78ffd523860e4904cbb344494#diff-d729ed51b22c10b4e307ad9f4fac22feR522

This allows ns_ssl to POST multipart/form-data requests which contains
binary content, eg. image/png

Would there be any consequences of storing a bytearray inside this dynamic
string as happens in aolserver?

--- a/nsssl.c Wed Jul 01 11:36:49 2015 +0100
+++ b/nsssl.c Wed Jul 29 12:39:37 2015 +0100
@@ -1216,7 +1216,7 @@

     if (bodyPtr != NULL) {
         int len = 0;
- const char *body = Tcl_GetStringFromObj(bodyPtr, &len);
+ const char *body = (char *) Tcl_GetByteArrayFromObj(bodyPtr, &len);
  Ns_DStringPrintf(&httpPtr->ds, "Content-Length: %d\r\n\r\n", len);
         Tcl_DStringAppend(&httpPtr->ds, body, len);
     } else {



On 23 July 2015 at 16:52, David Osborne <da...@qc...> wrote:

>
> Question - do ns_http / ns_ssl support raw binary data being passed in via
> the -body argument?
>
> *ns_http queue* ?*-method M*? ?*-headers S*? ?*-body B*? ?*-timeout T*?
> *url*
>
> *-body* body is the value which will be sent as the HTTP request body.
> I'm coming across this receiving a multipart/form-data POST and attempting
> to proxy these requests to an upstream server using ns_ssl.
>
> Regards,
> --
> David Osborne
> Qcode Software Limited
>
>

[naviserver-devel] ns_ssl and binary data

[David O. <da...@qc...>]

Question - do ns_http / ns_ssl support raw binary data being passed in via
the -body argument?

*ns_http queue* ?*-method M*? ?*-headers S*? ?*-body B*? ?*-timeout T*?
*url*

*-body* body is the value which will be sent as the HTTP request body.
I'm coming across this receiving a multipart/form-data POST and attempting
to proxy these requests to an upstream server using ns_ssl.

Regards,
-- 
David Osborne
Qcode Software Limited

[naviserver-devel] Disabling directorylisting

[David O. <da...@qc...>]

Hi,

I came across this while trying to turn fastpath directorylisting off. From
the docs:

directorylisting

Style of directory listings, Can be *fancy* or *simple*. (string, defaults
to simple)
directoryproc

Name of Tcl proc to use to display directory listings. One can either
specify *directoryproc*, or *directoryadp*, but not both. (string, defaults
to _ns_dirlist)

The docs don't explicitly say how to disable them as far as I can see. But
fastpath.tcl seems to default to a value of "none" if no config value is
set.
Should Naviserver support this value explicitly?

I tried:

ns_param directorylisting none

This then gave be a Tcl error:

invalid command name "none"
    while executing
"none"
    while executing callback
ns:tclrequest none
(context: request proc)

It seems the value "none" is being used as the directoryproc. When
directoryproc is not set, and directorylisting is not "simple" or "fancy",
dirproc is set to p which will be "none":

    p = Ns_ConfigString(path, "directorylisting", "simple");
    if (p != NULL && (STREQ(p, "simple") || STREQ(p, "fancy"))) {
        p = "_ns_dirlist";
    }
    servPtr->fastpath.dirproc = Ns_ConfigString(path, "directoryproc", p);

To make fastpath return a 404 instead of a dir listing I needed both :

ns_param directorylisting none
ns_param directoryproc _ns_dirlist

Is there another way of doing this?
Strikes me that directoryproc should default to _ns_dirlist no matter what
directorylisting is set to?
Could naviserver drop this check in fastpath.c, and leave the default value
of directorylisting to be set in fastpath.tcl as in the following commit?

https://bitbucket.org/davidqc/naviserver/commits/afd8de3e5bdc6813ab49c6bc5713faeaaf68b854

Regards,
-- 
David Osborne
Qcode Software Limited

Re: [naviserver-devel] Escaping characters in access log

[David O. <da...@qc...>]

Thanks very much Gustaf. That looks great .

On 14 July 2015 at 17:59, Gustaf Neumann <ne...@wu...> wrote:

>  Dear all,
>
> This is again a very reasonable request. Since most access-log analyzer
> are developed
> against apache rules, it seems that sticking to apache rules is sensible.
> ... although
> missing a few lines of hacking attempts is usually not an issue.
>
> i've added a small addition to the tip version that performs apache-style
> substitutions
> in the query fraction of the access log. The updated version performs
> apache-style
> escaping for all double-quoted fields depending potentially on external
> input,
> such as the user agent field or the referrer field.
>
> all the best
> -g
>
> Am 14.07.15 um 09:05 schrieb David Osborne:
>
> Hi,
>
>  We're coming up against a problem where we attempt to parse data in a
> naviserver access log to analyse server use.
>
>  We were relying on the combined log format being parsable but are
> running into difficulties when non-percent encoded characters are making
> their way into the logged request.
>
>  For example, the URL for testing for a XSS exploit:
>
>  /tiki-list_file_gallery.php/>"><script>alert(document.domain)</script>
>
>  This will be logged to the access log as:
>
>  9.9.9.9 - - [14/Jul/2015:14:55:34 +0100] "GET
> /tiki-list_file_gallery.php/>"><script>alert(document.domain)</script>
> HTTP/1.0" 404 737 "" "curl/7.26.0" "1436882134.386210 0.038129 0.000129
> 0.000016 0.000152"
>
>  Because of the unescaped quote we can't reliably parse this entry.
>
>  I wasn't sure what the server should do in cases like this. The quote
> should technically be percent encoded but clients like curl allow the raw
> character to be sent.
>
>  Apache escapes quotes by prefixing a backslash before logging:
> http://httpd.apache.org/docs/2.2/mod/mod_log_config.html
> "Exceptions from this rule are " and \, which are escaped by prepending a
> backslash, and all whitespace characters, which are written in their
> C-style notation (\n, \t, etc)"
>
> Nginx replaces quotes in the log with \x22:
>
> http://trac.nginx.org/nginx/changeset?old_path=%2Fnginx&old=66dc85397a9006d5ecdd74c56d9eac1fd479b5d6&new_path=%2Fnginx&new=66dc85397a9006d5ecdd74c56d9eac1fd479b5d6
>
> Do we have any means of doing something similar in Naviserver?
>
>  --
>  David
>
>
>
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>

Re: [naviserver-devel] Escaping characters in access log

[Gustaf N. <ne...@wu...>]

Dear all,

This is again a very reasonable request. Since most access-log analyzer 
are developed
against apache rules, it seems that sticking to apache rules is 
sensible. ... although
missing a few lines of hacking attempts is usually not an issue.

i've added a small addition to the tip version that performs 
apache-style substitutions
in the query fraction of the access log. The updated version performs 
apache-style
escaping for all double-quoted fields depending potentially on external 
input,
such as the user agent field or the referrer field.

all the best
-g

Am 14.07.15 um 09:05 schrieb David Osborne:
> Hi,
>
> We're coming up against a problem where we attempt to parse data in a 
> naviserver access log to analyse server use.
>
> We were relying on the combined log format being parsable but are 
> running into difficulties when non-percent encoded characters are 
> making their way into the logged request.
>
> For example, the URL for testing for a XSS exploit:
>
> /tiki-list_file_gallery.php/>"><script>alert(document.domain)</script>
>
> This will be logged to the access log as:
>
> 9.9.9.9 - - [14/Jul/2015:14:55:34 +0100] "GET 
> /tiki-list_file_gallery.php/>"><script>alert(document.domain)</script> 
> HTTP/1.0" 404 737 "" "curl/7.26.0" "1436882134.386210 0.038129 
> 0.000129 0.000016 0.000152"
>
> Because of the unescaped quote we can't reliably parse this entry.
>
> I wasn't sure what the server should do in cases like this. The quote 
> should technically be percent encoded but clients like curl allow the 
> raw character to be sent.
>
> Apache escapes quotes by prefixing a backslash before logging:
> http://httpd.apache.org/docs/2.2/mod/mod_log_config.html
> "Exceptions from this rule are |"| and |\|, which are escaped by 
> prepending a backslash, and all whitespace characters, which are 
> written in their C-style notation (|\n|, |\t|, etc)"
>
> Nginx replaces quotes in the log with \x22:
> http://trac.nginx.org/nginx/changeset?old_path=%2Fnginx&old=66dc85397a9006d5ecdd74c56d9eac1fd479b5d6&new_path=%2Fnginx&new=66dc85397a9006d5ecdd74c56d9eac1fd479b5d6
>
> Do we have any means of doing something similar in Naviserver?
>
> -- 
> David

[naviserver-devel] Escaping characters in access log

[David O. <da...@qc...>]

Hi,

We're coming up against a problem where we attempt to parse data in a
naviserver access log to analyse server use.

We were relying on the combined log format being parsable but are running
into difficulties when non-percent encoded characters are making their way
into the logged request.

For example, the URL for testing for a XSS exploit:

/tiki-list_file_gallery.php/>"><script>alert(document.domain)</script>

This will be logged to the access log as:

9.9.9.9 - - [14/Jul/2015:14:55:34 +0100] "GET
/tiki-list_file_gallery.php/>"><script>alert(document.domain)</script>
HTTP/1.0" 404 737 "" "curl/7.26.0" "1436882134.386210 0.038129 0.000129
0.000016 0.000152"

Because of the unescaped quote we can't reliably parse this entry.

I wasn't sure what the server should do in cases like this. The quote
should technically be percent encoded but clients like curl allow the raw
character to be sent.

Apache escapes quotes by prefixing a backslash before logging:
http://httpd.apache.org/docs/2.2/mod/mod_log_config.html
"Exceptions from this rule are " and \, which are escaped by prepending a
backslash, and all whitespace characters, which are written in their
C-style notation (\n, \t, etc)"

Nginx replaces quotes in the log with \x22:
http://trac.nginx.org/nginx/changeset?old_path=%2Fnginx&old=66dc85397a9006d5ecdd74c56d9eac1fd479b5d6&new_path=%2Fnginx&new=66dc85397a9006d5ecdd74c56d9eac1fd479b5d6

Do we have any means of doing something similar in Naviserver?

-- 
David

Re: [naviserver-devel] Nsssl SNI

[Jeff R. <dv...@di...>]

I was just considering this exact same thing, and it appears the answer 
is no.  You should be able to serve multiple vhosts on different ip 
addresses (or ports) by running nsssl multiple times with different cert 
configs, but that isn't particularly helpful.

I haven't explored this completely, but to add SNI support to nsssl I 
think the cleanest approach config-wise would be to add a "servers" and 
"certs" section underneath nsssl to map hostnames to certificates as 
well as servers, ala nssock.

For example:
ns_section ns/module/nsssl/servers
ns_param server1 www.example.com
ns_param server2 www.example2.com

ns_section ns/module/nsssl/certs
ns_param www.example.com /usr/local/ssl/certs/server1.pem
ns_param www.exmaple2.com /usr/local/ssl/certs/server2.pem

Dynamic vhosts could perhaps be supported by defining the cert file for 
a given domain to be a standard name under a "certs" subdirectory in the 
vhost tree (i.e., servers/${servername}/host.com/certs/host.com.pem). 
I would address the explicit configuration above first, however.

This SO post points at the implementation strategy: 
http://stackoverflow.com/questions/5113333/how-to-implement-server-name-indication-sni

Implementing this is not on my immediate to-do list (we're using ELB for 
termination) but it may become a concern sometime soon.

-J


David Osborne wrote:
> Hi there,
>
> Is there any way to replicate the behaviour of SNI aware https servers
> using naviserver nsssl?
> Namely, where different certificates can be presented on the same ssl
> port depending on the servername sent by the TLS client
>
> https://www.domain.com -> nsssl.server.com:443 <http://nsssl.server.com:443>
>        <- www.domain.com <http://www.domain.com> cert
>
> https://sub.domain.com -> nsssl.server.com:443 <http://nsssl.server.com:443>
>        <- sub.domain.com <http://sub.domain.com> cert
>
> (I don't think SNI is supported by nsssl - please correct me if I'm wrong)

[naviserver-devel] Nsssl SNI

[David O. <da...@qc...>]

Hi there,

Is there any way to replicate the behaviour of SNI aware https servers
using naviserver nsssl?
Namely, where different certificates can be presented on the same ssl port
depending on the servername sent by the TLS client

https://www.domain.com -> nsssl.server.com:443
      <- www.domain.com cert

https://sub.domain.com -> nsssl.server.com:443
      <- sub.domain.com cert

(I don't think SNI is supported by nsssl - please correct me if I'm wrong)


-- 
David Osborne
Qcode Software Limited
http://www.qcode.co.uk

Re: [naviserver-devel] Naviserver as reverse proxy - header fields

[David O. <da...@qc...>]

Thanks Gustaf.
No problem... I'll get the pull requests done shortly.

On 26 June 2015 at 12:18, Gustaf Neumann <ne...@wu...> wrote:

>  Both are valid requests and should go to the source base. david, could
> you issue a pull request on bitbucket?
> -g
>
> --
> Univ.Prof. Dr. Gustaf Neumann
> WU Vienna
> Institute of Information Systems and New Media
> Welthandelsplatz 1, A-1020 Vienna, Austria
>
>
>
> ------------------------------------------------------------------------------
> Monitor 25 network devices or servers for free with OpManager!
> OpManager is web-based network management software that monitors
> network devices and physical & virtual servers, alerts via email & sms
> for fault. Monitor 25 devices for free with no restriction. Download now
> http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>

Re: [naviserver-devel] Naviserver as reverse proxy - header fields

[Gustaf N. <ne...@wu...>]

Both are valid requests and should go to the source base. david, could 
you issue a pull request on bitbucket?
-g

Am 26.06.15 um 11:51 schrieb David Osborne:
> Hi,
>
> We've been looking at using Naviserver as a reverse proxy so it can 
> serve up fastpath requests, and also forward requests to an upstream 
> app server when required.
>
> There was a couple of issues we came across that I'd like to ask about.
> The proxy server code follows the same basic idea as Gustaf's post 
> here: http://sourceforge.net/p/naviserver/mailman/message/31063859/
>
> 1. Host header fields.
> When we forward a request upstream using ns_ssl I couldn't find a way 
> of getting the Host header field to be forwarded untouched by ns_ssl.
>
> I pass the query headers into ns_ssl which include a Host field 
> containing the host originally request by the client, and ns_ssl 
> always overwrites it with the value of the upstream Host. I don't 
> think this behaviour is wrong by any means, but, in Apache for 
> example, there is an option which turns on the behaviour we were 
> looking for - proxyPreserveHost
> http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypreservehost
>
> Could there be scope for such an argument for ns_ssl (and ns_http as 
> well I guess) or is there a better way to deal with this?
> Something like this:
> https://bitbucket.org/davidqc/nsssl/branches/compare/davidqc/nsssl:tip%0Dnaviserver/nsssl:default#diff
>
>
> 2. Duplicate headers added by ns_respond.
> Once upstream has replied to the proxy, the response is passed back to 
> the client using ns_respond with the headers we received from 
> upstream. What we are seeing is duplicate content-type and 
> content-length headers being received by the client.
> It seems like this may be because Ns_ConnUpdateHeaders is using a case 
> sensitive Ns_SetUpdate to update the content-type header. So if 
> content-type already exists, it will add Content-Type as well.
>
> Should this behaviour be that a case insensitive Ns_SetIUpdate is used 
> like the following or are there other ways around this?:
> https://bitbucket.org/davidqc/naviserver/branches/compare/davidqc/naviserver:tip%0Dnaviserver/naviserver:default#diff
>
>
> Thanks
> -- 
> David Osborne
> Qcode Software Limited
> http://www.qcode.co.uk
>
>
>
> ------------------------------------------------------------------------------
> Monitor 25 network devices or servers for free with OpManager!
> OpManager is web-based network management software that monitors
> network devices and physical & virtual servers, alerts via email & sms
> for fault. Monitor 25 devices for free with no restriction. Download now
> http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

[naviserver-devel] Naviserver as reverse proxy - header fields

[David O. <da...@qc...>]

Hi,

We've been looking at using Naviserver as a reverse proxy so it can serve
up fastpath requests, and also forward requests to an upstream app server
when required.

There was a couple of issues we came across that I'd like to ask about.
The proxy server code follows the same basic idea as Gustaf's post here:
http://sourceforge.net/p/naviserver/mailman/message/31063859/

1. Host header fields.
When we forward a request upstream using ns_ssl I couldn't find a way of
getting the Host header field to be forwarded untouched by ns_ssl.

I pass the query headers into ns_ssl which include a Host field containing
the host originally request by the client, and ns_ssl always overwrites it
with the value of the upstream Host. I don't think this behaviour is wrong
by any means, but, in Apache for example, there is an option which turns on
the behaviour we were looking for - proxyPreserveHost
http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypreservehost

Could there be scope for such an argument for ns_ssl (and ns_http as well I
guess) or is there a better way to deal with this?
Something like this:
https://bitbucket.org/davidqc/nsssl/branches/compare/davidqc/nsssl:tip%0Dnaviserver/nsssl:default#diff


2. Duplicate headers added by ns_respond.
Once upstream has replied to the proxy, the response is passed back to the
client using ns_respond with the headers we received from upstream. What we
are seeing is duplicate content-type and content-length headers being
received by the client.
It seems like this may be because Ns_ConnUpdateHeaders is using a case
sensitive Ns_SetUpdate to update the content-type header. So if
content-type already exists, it will add Content-Type as well.

Should this behaviour be that a case insensitive Ns_SetIUpdate is used like
the following or are there other ways around this?:
https://bitbucket.org/davidqc/naviserver/branches/compare/davidqc/naviserver:tip%0Dnaviserver/naviserver:default#diff


Thanks
-- 
David Osborne
Qcode Software Limited
http://www.qcode.co.uk

[naviserver-devel] logging

[Jeff R. <dv...@di...>]

Hi all,

I ran into a bug trying to change the loglevel for nsdb:
"ns_db verbose" crashes with an assertion failure.  It looks like this 
was recently changed to change the log level for the entire ns_db module 
rather than just for one handle (a change that IMHO makes sense), but 
this part doesn't work right.

Related, it would be exceedingly useful to set messages at a particular 
loglevel to go to a logfile other than the default server log.  However 
it doesn't appear you can do this.  The feature isn't supported by the 
default logger, and logging filters don't provide a mechanism (e.g., 
ns_break) to indicate that the message has been successfully logged and 
further filters should be skipped.

Log filters also process everything, there's no matching as in request 
filters.  This could be worked around easily enough, but it would make 
things a little more structured.  The usage I'm envisioning would be 
something like:
ns_logctl filter Debug(sql) ns:logtofile /tmp/sqldebug.log

Lastly, would it be reasonable to allow ns_log calls to give a loglevel 
that is not already defined (which would default to being disabled)?  Or 
maybe that would make sense only when the loglevel is of the form
"Level(subsystem)"  (e.g., Debug(sql)) and the "Level" is already known. 
  Then I could sprinkle module-specific ns_log debug statements 
everywhere without needing to pre-declare all the levels I might use.

Cheers,
-J

[naviserver-devel] NaviServer 4.99.8 available

[Gustaf N. <ne...@wu...>]

Dear friends,

NaviServer 4.99.8 is available. The new version is tagged with
naviserver-4.99.8 in mercurial and is as well available at source-forge
(naviserver, modules, documentation pages). I'll prepare as well
an annoucement relative to 4.99.6 for c.l.tcl and OpenACS.

Below is the section for 4.99.8 from the NEWS file.

all the best

-gustaf neumann

======================================
NaviServer 4.99.8, released 2015-04-13
======================================

Changes relative to 4.99.7
   55 files changed, 1088 insertions(+), 527 deletions(-)

New Features:
   * ns_md5, ns_sha1: added binary support
     When binary data is passed to this function, use
     Tcl byte-array operations instead of string operations
   * Added ability to debug configuration of nscgi via
     "ns_logctl severity Debug(cgi) on"
   * Added config parameter "compresspreinit": Setting this parameter
     to true (default is false) will cause the compression stream
     buffers to be allocated and initialized at server startup.
     Without it, they will be allocated and initialized as needed. This
     change drops the initial memory footprint of a server with a
     default config significantly.

Bug Fixes:
   * Fixed bug reported by Wolfgang Winkler, when " ns_urldecode --" was
     called (switched to regular argv parser)
   * Fixed bug, when "ns_conn content" was called without content
     potential race conditions on thread exits
   * Fixed potential race conditions on thread exits
   * ns_md5: Code generation was broken
     (probably since a long time, due to a mix up of somewhat tricky casts)
   * Fixed warning in interaction between TCP_CORK and nsssl
   * Fixed bug, where one thread frees a nsv-array, but an internal
     representation of an Tcl_Obj for this array was still active in
     another thread
   * Fixed bug where ns_imgsize returned an error where it should return
     success and width and height returned as 0 (according to documentation)
   * Avoid potential access of string past null character
   * Fixed a bug with in https client commands (ns_ssl) when paths and 
parameters
     are passed.
   * Fixed nsphp compilation (and "make php").
     Many thanks to Branden Graves for feedback and testing.

Documentation improvements:
   * Added documentation for ns_md5
   * Improved documentation for ns_img commands
   * Improved documentation for nscgi

Configuration Changes:
   * Improved sample configuration for OpenACS and nsssl
   * Improved Makefiles (reduce redundancy for CFLAGS)
   * Improved rpath handling in configure.ac for Linux distros,
     where TCL_CC_SEARCH_FLAGS and TCL_LD_SEARCH_FLAGS are set
     empty, like e.g.Debian

Code Changes:
   * Extended Regression Test:
     - Added test set for ns_md5, compared results with other 
implementations
     - Added binary regression test for ns_md5 and ns_sha1
     - Added test set for ns_md5
     - Improved robustness of tests for ns_parseargs
     - Added tests for "ns_urldecode --"
     - Added test set infrastructure (nstest::https, server setup) and
       test cases for nsssl
     - Added test set for ns_img* commands
     - Number of regression tests passed 1000

   * Reduced implicit type conversions and other minor code cleanups
   * Protect against potential buffer overruns