You can subscribe to this list here.
2005 |
Jan
|
Feb
(53) |
Mar
(62) |
Apr
(88) |
May
(55) |
Jun
(204) |
Jul
(52) |
Aug
|
Sep
(1) |
Oct
(94) |
Nov
(15) |
Dec
(68) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2006 |
Jan
(130) |
Feb
(105) |
Mar
(34) |
Apr
(61) |
May
(41) |
Jun
(92) |
Jul
(176) |
Aug
(102) |
Sep
(247) |
Oct
(69) |
Nov
(32) |
Dec
(140) |
2007 |
Jan
(58) |
Feb
(51) |
Mar
(11) |
Apr
(20) |
May
(34) |
Jun
(37) |
Jul
(18) |
Aug
(60) |
Sep
(41) |
Oct
(105) |
Nov
(19) |
Dec
(14) |
2008 |
Jan
(3) |
Feb
|
Mar
(7) |
Apr
(5) |
May
(123) |
Jun
(5) |
Jul
(1) |
Aug
(29) |
Sep
(15) |
Oct
(21) |
Nov
(51) |
Dec
(3) |
2009 |
Jan
|
Feb
(36) |
Mar
(29) |
Apr
|
May
|
Jun
(7) |
Jul
(4) |
Aug
|
Sep
(4) |
Oct
|
Nov
(13) |
Dec
|
2010 |
Jan
|
Feb
|
Mar
(9) |
Apr
(11) |
May
(16) |
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2012 |
Jan
(7) |
Feb
(3) |
Mar
|
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
|
Oct
(92) |
Nov
(28) |
Dec
(16) |
2013 |
Jan
(9) |
Feb
(2) |
Mar
|
Apr
(4) |
May
(4) |
Jun
(6) |
Jul
(14) |
Aug
(12) |
Sep
(4) |
Oct
(13) |
Nov
(1) |
Dec
(6) |
2014 |
Jan
(23) |
Feb
(19) |
Mar
(10) |
Apr
(14) |
May
(11) |
Jun
(6) |
Jul
(11) |
Aug
(15) |
Sep
(41) |
Oct
(95) |
Nov
(23) |
Dec
(11) |
2015 |
Jan
(3) |
Feb
(9) |
Mar
(19) |
Apr
(3) |
May
(1) |
Jun
(3) |
Jul
(11) |
Aug
(1) |
Sep
(15) |
Oct
(5) |
Nov
(2) |
Dec
|
2016 |
Jan
(7) |
Feb
(11) |
Mar
(8) |
Apr
(1) |
May
(3) |
Jun
(17) |
Jul
(12) |
Aug
(3) |
Sep
(5) |
Oct
(19) |
Nov
(12) |
Dec
(6) |
2017 |
Jan
(30) |
Feb
(23) |
Mar
(12) |
Apr
(32) |
May
(27) |
Jun
(7) |
Jul
(13) |
Aug
(16) |
Sep
(6) |
Oct
(11) |
Nov
|
Dec
(12) |
2018 |
Jan
(1) |
Feb
(5) |
Mar
(6) |
Apr
(7) |
May
(23) |
Jun
(3) |
Jul
(2) |
Aug
(1) |
Sep
(6) |
Oct
(6) |
Nov
(10) |
Dec
(3) |
2019 |
Jan
(26) |
Feb
(15) |
Mar
(9) |
Apr
|
May
(8) |
Jun
(14) |
Jul
(10) |
Aug
(10) |
Sep
(4) |
Oct
(2) |
Nov
(20) |
Dec
(10) |
2020 |
Jan
(10) |
Feb
(14) |
Mar
(29) |
Apr
(11) |
May
(25) |
Jun
(21) |
Jul
(23) |
Aug
(12) |
Sep
(19) |
Oct
(6) |
Nov
(8) |
Dec
(12) |
2021 |
Jan
(29) |
Feb
(9) |
Mar
(8) |
Apr
(8) |
May
(2) |
Jun
(2) |
Jul
(9) |
Aug
(9) |
Sep
(3) |
Oct
(4) |
Nov
(12) |
Dec
(13) |
2022 |
Jan
(4) |
Feb
|
Mar
(4) |
Apr
(12) |
May
(15) |
Jun
(7) |
Jul
(10) |
Aug
(2) |
Sep
|
Oct
(1) |
Nov
(8) |
Dec
|
2023 |
Jan
(15) |
Feb
|
Mar
(23) |
Apr
(1) |
May
(2) |
Jun
(10) |
Jul
|
Aug
(22) |
Sep
(19) |
Oct
(2) |
Nov
(20) |
Dec
|
2024 |
Jan
(1) |
Feb
|
Mar
(16) |
Apr
(15) |
May
(6) |
Jun
(4) |
Jul
(1) |
Aug
(1) |
Sep
|
Oct
(13) |
Nov
(18) |
Dec
(6) |
2025 |
Jan
(12) |
Feb
|
Mar
(2) |
Apr
(1) |
May
(11) |
Jun
(5) |
Jul
(4) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Gustaf N. <ne...@wu...> - 2017-03-02 22:06:04
|
Am 02.03.17 um 19:14 schrieb iu...@iu...: > Running NS instaltion script (install-ns.sh <http://install-ns.sh>) I > noticed that heanet.dl.sourceforge.net > <http://heanet.dl.sourceforge.net> is down. I tried to access their > repos but it seems NS sources were removed. The download url changed recently. get a new version of install-ns.sh from https://github.com/gustafn/install-ns all the best -gn |
From: <iu...@iu...> - 2017-03-02 18:33:54
|
<html><body><span style="font-family:Verdana; color:#000000; font-size:10pt;"><div>Hi there,</div><div><br></div><div>Running NS instaltion script (<a href="http://install-ns.sh">install-ns.sh</a>) I noticed that <span style="font-size: 10pt;"><a href="http://heanet.dl.sourceforge.net">heanet.dl.sourceforge.net</a> is down. I tried to access their repos but it seems NS sources were removed. </span></div><div><span style="font-size: 10pt;"><br></span></div><div>examples:</div><div><span style=""><a href="http://heanet.dl.sourceforge.net/sourceforge/tcllib/Tcllib-1.18.tar.bz2">http://heanet.dl.sourceforge.net/sourceforge/tcllib/Tcllib-1.18.tar.bz2</a></span></div><div><span style=""></span><span style=""><a href="http://heanet.dl.sourceforge.net/sourceforge/naviserver/naviserver">http://heanet.dl.sourceforge.net/sourceforge/naviserver/naviserver</a>-</span><span style="">4.99.14</span><span style="">.tar.gz</span></div><div><span style=""></span><span style="font-size: 10pt;"> </span></div><div><span style="font-size: 10pt;"><br></span></div><div><span style="">Has anyone got the same problem?</span></div><div><span style=""><br></span></div><div><span style=""><br></span></div><div><span style="">Best wishes,</span></div><div><span style="">Iuri</span></div></span></body></html> |
From: David O. <da...@qc...> - 2017-03-01 11:20:17
|
On 28 February 2017 at 17:55, Gustaf Neumann <ne...@wu...> wrote: > Am 28.02.17 um 17:02 schrieb David Osborne: > > In that case, we get a ~5 second delay, > do you see the same behavior for static content (fastpath) and dynamic > content? > -g > Yes, identical results returning content via fastpath, and using ns_return. |
From: Gustaf N. <ne...@wu...> - 2017-02-28 17:56:01
|
Am 28.02.17 um 17:02 schrieb David Osborne: > In that case, we get a ~5 second delay, do you see the same behavior for static content (fastpath) and dynamic content? -g |
From: David O. <da...@qc...> - 2017-02-28 16:02:48
|
Revproxy is working quite well for a variety of requests now. One more issue we are encountering while spooling responses back to the client, is a fairly sizeable delay before the connection is being closed. This happens when the proxy is proxying to another port on localhost (which is how we run our development environments). I couldn't reproduce when upstream was a different server. For example, requesting a single file, the revproxy code very quickly receives the reply from upstream, then the spool callback proceeds to return the result to client in 4096 chunks. These happen rapidly until the final chunk. In that case, we get a ~5 second delay, then finally the last spool callback is triggered, the final chunk of data is sent, then the connection is auto closed by the spool proc. In the following log extract the delay can be seen between 18:25:52 & 18:25:57 Revproxy output <https://gist.github.com/davidqc/a665905f4e44ab35c4e5356d7067ce90> This was a quick test case based on the latest code: Naviserver Config <https://gist.github.com/davidqc/980ace1188f0537f3d616b847ccc2f3b> Naviserver Init <https://gist.github.com/davidqc/0f0d5210fa16012f35afec4fde5f86c9> Add a static file to pages/, then https://www.test.co.uk/proxy/file.html The content is displayed but the connection does not close for > 5 seconds. Any ideas why this might be happening? I'll continue to try to narrow it down... Regards, David > On 24 February 2017 at 19:32, Gustaf Neumann <ne...@wu...> wrote: > >> Am 24.02.17 um 18:21 schrieb David Osborne: >> >> Thanks, that's working great for me. >> Have tested it with large POST data, binary data, UTF-8 data and all gets >> handled perfectly. >> >> Only issue I've come across is testing the behaviour when the proxy can't >> connect to the backend. I'm getting a segfault for every request in that >> case. >> >> tested proxy https connection to non-existing host ... fine. >> tested proxy https connection to the non-listening port ... fine >> tested proxy https connection to a non-https port ... bingo >> >> Fixed in https://bitbucket.org/naviserver/naviserver/commits/9264a64b >> f2d81e6c142a2bab2b5f0deb3acd7c14 >> >> all the best >> -gn >> >> ------------------------------------------------------------ >> ------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, SlashDot.org! http://sdm.link/slashdot >> _______________________________________________ >> naviserver-devel mailing list >> nav...@li... >> https://lists.sourceforge.net/lists/listinfo/naviserver-devel >> >> > |
From: David O. <da...@qc...> - 2017-02-27 17:29:57
|
Perfect, thanks! On 24 February 2017 at 19:32, Gustaf Neumann <ne...@wu...> wrote: > Am 24.02.17 um 18:21 schrieb David Osborne: > > Thanks, that's working great for me. > Have tested it with large POST data, binary data, UTF-8 data and all gets > handled perfectly. > > Only issue I've come across is testing the behaviour when the proxy can't > connect to the backend. I'm getting a segfault for every request in that > case. > > tested proxy https connection to non-existing host ... fine. > tested proxy https connection to the non-listening port ... fine > tested proxy https connection to a non-https port ... bingo > > Fixed in https://bitbucket.org/naviserver/naviserver/commits/ > 9264a64bf2d81e6c142a2bab2b5f0deb3acd7c14 > > all the best > -gn > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > -- David Osborne Qcode Software Limited http://www.qcode.co.uk T: +44 (0)1463 896484 |
From: Gustaf N. <ne...@wu...> - 2017-02-24 19:32:39
|
Am 24.02.17 um 18:21 schrieb David Osborne: > Thanks, that's working great for me. > Have tested it with large POST data, binary data, UTF-8 data and all > gets handled perfectly. > > Only issue I've come across is testing the behaviour when the proxy > can't connect to the backend. I'm getting a segfault for every request > in that case. tested proxy https connection to non-existing host ... fine. tested proxy https connection to the non-listening port ... fine tested proxy https connection to a non-https port ... bingo Fixed in https://bitbucket.org/naviserver/naviserver/commits/9264a64bf2d81e6c142a2bab2b5f0deb3acd7c14 all the best -gn |
From: David O. <da...@qc...> - 2017-02-24 17:21:23
|
(In asnwer to the previous question, yes this is debian 8.7 with openssl 1.0.1t) On 24 February 2017 at 06:24, Gustaf Neumann <ne...@wu...> wrote: > There are reports that SSL writes with large content cause problems. > I've uploaded a version of revproxy that avoids memory bloats on huge > files and that chunks POST data. > Thanks, that's working great for me. Have tested it with large POST data, binary data, UTF-8 data and all gets handled perfectly. Only issue I've come across is testing the behaviour when the proxy can't connect to the backend. I'm getting a segfault for every request in that case. Not making much progress debugging since I don't understand socket programming well enough, but it looks like the code doesn't pickup on the ssl connection error (error:00000000:lib(0):func(0):reason(0)) which is reported in the interpreter (gdb) bt #0 0x00007ffff4a0cf10 in Send (sock=0x7fffb800dd80, bufs=0x7fffd77fc950, nbufs=4, UNUSED_timeoutPtr=<optimized out>, UNUSED_flags=<optimized out>) at nsssl.c:584 #1 0x00007ffff7b5239d in DriverSend (interp=0x7fffc8008f70, connChanPtr=0x7fffc80bf740, bufs=0x7fffd77fc950, nbufs=4, flags=0, timeoutPtr=0x7fffd77fc850, flags=0) at connchan.c:628 #2 0x00007ffff7b515c2 in ConnChanOpenObjCmd (clientData=0x7fffc804f750, interp=0x7fffc8008f70, objc=0, objv=0x7fffc80bf7b8) at connchan.c:856 #3 0x00007ffff7b86714 in Ns_SubcmdObjv (subcmdSpec=0x7fffd77fcae0, clientData=0x7fffdc0570b0, interp=0x7fffc8008f70, objc=11, objv=0x7fffc80171b8) at tclobjv.c:1506 #4 0x00007ffff7b522e5 in NsTclConnChanObjCmd (clientData=<optimized out>, interp=<optimized out>, objc=<optimized out>, objv=<optimized out>) at connchan.c:1304 #5 0x00007ffff71bce59 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #6 0x00007ffff720395e in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #7 0x00007ffff7246ce9 in TclObjInterpProcCore () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #8 0x00007ffff71bce59 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #9 0x00007ffff720395e in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #10 0x00007ffff7202897 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #11 0x00007ffff71be6e6 in TclEvalObjEx () from /usr/lib/x86_64-linux-gnu/ libtcl8.5.so #12 0x00007ffff71c5690 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #13 0x00007ffff71bce59 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #14 0x00007ffff71bd312 in Tcl_EvalObjv () from /usr/lib/x86_64-linux-gnu/ libtcl8.5.so #15 0x00007ffff71be9a2 in TclEvalObjEx () from /usr/lib/x86_64-linux-gnu/ libtcl8.5.so #16 0x00007ffff724642f in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #17 0x00007ffff71bce59 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #18 0x00007ffff720395e in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #19 0x00007ffff7246ce9 in TclObjInterpProcCore () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #20 0x00007ffff71bce59 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #21 0x00007ffff720395e in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #22 0x00007ffff7246ce9 in TclObjInterpProcCore () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #23 0x00007ffff71bce59 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #24 0x00007ffff71bdb29 in ?? () from /usr/lib/x86_64-linux-gnu/libtcl8.5.so #25 0x00007ffff71bd473 in Tcl_EvalEx () from /usr/lib/x86_64-linux-gnu/ libtcl8.5.so #26 0x00007ffff7b87918 in NsTclFilterProc (arg=0x871820, conn=0x6a4e30, why=NS_FILTER_PRE_AUTH) at tclrequest.c:537 #27 0x00007ffff7b5d89f in NsRunFilters (conn=conn@entry=0x6a4e30, why=why@entry=NS_FILTER_PRE_AUTH) at filter.c:161 #28 0x00007ffff7b6ace4 in ConnRun (connPtr=0x6a4e30, UNUSED_argPtr=0x6b3340) at queue.c:1998 #29 NsConnThread (arg=0x6b3340) at queue.c:1714 #30 0x00007ffff74ba86d in NsThreadMain (arg=<optimized out>) at thread.c:232 #31 0x00007ffff74bb8a9 in ThreadMain (arg=<optimized out>) at pthread.c:830 #32 0x00007ffff5e520a4 in start_thread (arg=0x7fffd77fe700) at pthread_create.c:309 #33 0x00007ffff635362d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 (gdb) frame 4 #4 0x00007ffff7b522e5 in NsTclConnChanObjCmd (clientData=<optimized out>, interp=<optimized out>, objc=<optimized out>, objv=<optimized out>) at connchan.c:1304 1304 return Ns_SubcmdObjv(subcmds, clientData, interp, objc, objv); (gdb) frame 2 #2 0x00007ffff7b515c2 in ConnChanOpenObjCmd (clientData=0x7fffc804f750, interp=0x7fffc8008f70, objc=0, objv=0x7fffc80bf7b8) at connchan.c:856 856 nSent = DriverSend(interp, connChanPtr, buf, 4, 0u, &connChanPtr->sendTimeout); (gdb) print interp $18 = (Tcl_Interp *) 0x7fffc8008f70 (gdb) print *interp $19 = { result = 0x7fffc8009148 "ssl connect failed: error:00000000:lib(0):func(0):reason(0)", freeProc = 0x0, errorLine = 1 } |
From: Gustaf N. <ne...@wu...> - 2017-02-24 06:24:35
|
There are reports that SSL writes with large content cause problems. I've uploaded a version of revproxy that avoids memory bloats on huge files and that chunks POST data. -g Am 23.02.17 um 21:25 schrieb Gustaf Neumann: > actually, the approach with the "ns_getcontent" should not be necessary > and is not desirable for large content. What i can see from the sinppet > is that the request content was spooled to a file (due to the maxupload > settings) which needs probably more detailed handling (my tests were > mostly with GET and WebSockets). > > I'll try to setup a test environment similar to yours - may be at the > weekend. My assumption is that you are using this under linux and > OpenSSL 1.0.*. Is this correct? |
From: Gustaf N. <ne...@wu...> - 2017-02-23 20:25:33
|
actually, the approach with the "ns_getcontent" should not be necessary and is not desirable for large content. What i can see from the sinppet is that the request content was spooled to a file (due to the maxupload settings) which needs probably more detailed handling (my tests were mostly with GET and WebSockets). I'll try to setup a test environment similar to yours - may be at the weekend. My assumption is that you are using this under linux and OpenSSL 1.0.*. Is this correct? g |
From: David O. <da...@qc...> - 2017-02-23 17:30:40
|
After this, I've been trying to get POST requests to work. I needed to change the revproxy code a little since, for me, the backend would wait forever for the POST body unless I explicitly sent it after opening the backend channel. This will now work fine for POSTs with a body of ~32,768 bytes and under. But anything more than that, again, the backend will again wait forever for some input from the front end with: [23/Feb/2017:17:20:22][19157.7fa9c17fa700][-driver:nsssl:0-] Debug(ns:driver): SockRead wait for more input It seems notable that the backend receives data in 16384 byte chunks and this starts failing when the content is around 2x 16384 Should "ns_connchann write" be happy sending data of this size in a single request or am I abusing it? [23/Feb/2017:17:29:17][19157.7fa9c17fa700][-driver:nsssl:0-] Debug(ns:driver): SockRead: require tmp file for content spooling (length 101332 > readahead 16384) [23/Feb/2017:17:29:17][19157.7fa9c17fa700][-driver:nsssl:0-] Debug(ns:driver): SockRead receive from network 16384 bytes [23/Feb/2017:17:29:17][19157.7fa9c17fa700][-driver:nsssl:0-] Debug(ns:driver): SockRead wait for more input [23/Feb/2017:17:29:17][19157.7fa9c17fa700][-driver:nsssl:0-] Debug(ns:driver): === PollWait returned 1, trigger[0] 0 [23/Feb/2017:17:29:17][19157.7fa9c17fa700][-driver:nsssl:0-] Debug(ns:driver): SockRead receive from network 16384 bytes [23/Feb/2017:17:29:17][19157.7fa9c17fa700][-driver:nsssl:0-] Debug(ns:driver): SockRead wait for more input --- a/revproxy-procs.tcl Tue Nov 01 16:27:48 2016 +0100 +++ b/revproxy-procs.tcl Thu Feb 23 17:07:59 2017 +0000 @@ -62,6 +62,8 @@ {*}$validation_callback -url $url } + set content [ns_getcontent -binary true -as_file false] + if {[catch { # # Open backend channel, get frontend channel and connect these. @@ -73,7 +75,13 @@ $url] set frontendChan [ns_connchan detach] log notice "back $backendChan front $frontendChan method [ns_conn method] version 1.0 $url" - + + # POST requests + if { [string length $content] > 0 } { + # We have a body, send it to the backend + ns_connchan write $backendChan $content + } + ns_connchan callback $backendChan [list ::revproxy::backendReply $backendChan $frontendChan $url 0] rex ns_connchan callback $frontendChan [list ::revproxy::spool $frontendChan $backendChan client 0] rex On 23 February 2017 at 11:54, Gustaf Neumann <ne...@wu...> wrote: > good news! yes, sure, the change is right. merged already. -gn > > > |
From: Gustaf N. <ne...@wu...> - 2017-02-23 11:54:12
|
good news! yes, sure, the change is right. merged already. -gn Am 23.02.17 um 12:08 schrieb David Osborne: > Thanks very much - I'm able to get that working for the second nsd > server now, but had to make a couple of changes. > Pull request if you want to take a look: > https://bitbucket.org/naviserver/naviserver/pull-requests/12/even-if-protocol-matches-carry-on/diff |
From: David O. <da...@qc...> - 2017-02-23 11:09:01
|
Thanks very much - I'm able to get that working for the second nsd server now, but had to make a couple of changes. Pull request if you want to take a look: https://bitbucket.org/naviserver/naviserver/pull-requests/12/even-if-protocol-matches-carry-on/diff On 22 February 2017 at 17:18, Gustaf Neumann <ne...@wu...> wrote: > Am 22.02.17 um 16:29 schrieb Gustaf Neumann: > > Am 22.02.17 um 16:10 schrieb David Osborne: > > > >> - is the connection to the backend via https? >> > > Yes > > Without foing into details: When the connection to the backend is via > https, the different driver can make a difference (since there might be > different ceritificates, etc.) > > I will look into this > > i've commited a prototypical "-driver" option to "ns_connchan open" (on > bitbucket) > please check > > -gn > > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > |
From: Gustaf N. <ne...@wu...> - 2017-02-22 17:18:10
|
Am 22.02.17 um 16:29 schrieb Gustaf Neumann: > Am 22.02.17 um 16:10 schrieb David Osborne: >> >> - is the connection to the backend via https? >> >> >> Yes > Without foing into details: When the connection to the backend is via > https, the different driver can make a difference (since there might > be different ceritificates, etc.) > > I will look into this i've commited a prototypical "-driver" option to "ns_connchan open" (on bitbucket) please check -gn |
From: Gustaf N. <ne...@wu...> - 2017-02-22 15:29:16
|
Am 22.02.17 um 16:10 schrieb David Osborne: > > - is the connection to the backend via https? > > > Yes Without foing into details: When the connection to the backend is via https, the different driver can make a difference (since there might be different ceritificates, etc.) I will look into this -g |
From: David O. <da...@qc...> - 2017-02-22 15:10:22
|
On 22 February 2017 at 13:49, Gustaf Neumann <ne...@wu...> wrote: > Am 22.02.17 um 13:48 schrieb David Osborne: > > Short question: > Is there a way to influence which driver thread "ns_connchan open" will > use? > > ... no, since ns_connchan does not use the driver threads, and when in the > revproxy code issues the connection to the backend it is already in a > connection thread (where the filter is running). After the setup in the > connection thread, the callback execution happens in a "socks" thread. > > However, opening connections uses the driver's setup information > (certificate, ciphers, etc), but only, when doing "ns_connchan open > https://...". The connchan handles "conn0" ... are per-server, not > per-driver, so it looks to me, as if in your case, the "conn0" was already > closed. > > Some questions: > - do you have multiple servers defined in one nsd? > Yes > - is the connection to the backend via https? > Yes > - how do the https driver configurations differ? > The only difference as I recall is they listen on different ports with a different backend destinations, but otherwise identical. For example: ns_section "ns/server/proxy_www/modules" ns_param nsssl1 nsssl.so ns_section "ns/server/proxy_erp/modules" ns_param nsssl2 nsssl.so ns_section "ns/server/proxy_www/module/nsssl1" ns_param port $https_port ns_param address $address ns_param hostname $hostname ... ns_section "ns/server/proxy_erp/module/nsssl2" ns_param port $https_erp_port ns_param address $address ns_param hostname $hostname ... I jumped to the conclusion regarding the driver threads by examining the output of "ns_connchan list" at the point the ns_connchan callbacks are defined. I'll see if I can put together a simple test case using the revproxy module. But in our environment, the case that works, both channels are listed as nsssl1: - Request received for server *proxy_www* - The backend and frontend channels are set up: - back *conn2 *front conn3 method GET version 1.0 https://localhost:8001/index.html <https://localhost:8001/index.html> - "ns_connchan list" shows: - *conn2 *{} 1487772742.608164 *nsssl1 *874 0 {} - conn3 {} 1487772742.604384 *nsssl1 *172.31.100.23 0 0 {} - The backendReply callback is registered against backend *conn2*. - backendReply is called when the socket becomes READABLE - *conn2 *is read... etc etc The case that doesn't work, the new backend channel is listed as nsssl1 where-as this connection thread is servicing a nsssl2 request. - Request received for server *proxy_erp* - The backend and frontend channels are set up: - back *conn4 *front conn5 method GET version 1.0 https://localhost:8002/index.html <https://localhost:8002/index.html> - "ns_connchan list" shows (differing driver modules) - *conn4 *{} 1487772797.297281 *nsssl1 *625 0 {} - conn0 -socks- 1487767354.759715 nsssl1 625 0 {} - conn5 {} 1487772797.293823 *nsssl2 *172.31.100.23 0 0 {} - The backendReply callback is registered against backend *conn4*. - backendReply is called when *conn4* becomes READABLE - Then error saying *conn4* does not exist |
From: Gustaf N. <ne...@wu...> - 2017-02-22 13:49:39
|
Am 22.02.17 um 13:48 schrieb David Osborne: > Short question: > Is there a way to influence which driver thread "ns_connchan open" > will use? ... no, since ns_connchan does not use the driver threads, and when in the revproxy code issues the connection to the backend it is already in a connection thread (where the filter is running). After the setup in the connection thread, the callback execution happens in a "socks" thread. However, opening connections uses the driver's setup information (certificate, ciphers, etc), but only, when doing "ns_connchan open https://...". The connchan handles "conn0" ... are per-server, not per-driver, so it looks to me, as if in your case, the "conn0" was already closed. Some questions: - do you have multiple servers defined in one nsd? - is the connection to the backend via https? - how do the https driver configurations differ? For debugging: - the revproxy code has a variable "verbose". When set to 1, it gives more details. - even more details can be obtained via "ns_logctl severity Debug(ns:driver) on" (e.g. add this to the end of your startup file) all the best -gn > > Longer question: > We have 2 driver threads listening on different ports, nsssl1 & nsssl2. > Using backenReply/spool works ok for GET requests which come in on > driver nsssl1 (we have issues with POSTs working but I've not looked > at that yet). > > But if the request comes in on driver nsssl2, when we "ns_connchan > open" to our backend, the channel is on nsssl1, so then when the > backendReply callback is triggered it tries to read the handle and > can't find it. > > eg: > [22/Feb/2017:12:42:34][6739.7f6113fff700][-socks-] Error: connchan > "conn0" does not exist > connchan "conn0" does not exist > while executing > "ns_connchan read $from" > (procedure "backendReply" line 6) > invoked from within > "backendReply conn0 conn1 https://localhost:8002/login_form.html 0 r" > (context: connchan proc) > > What would be the best way to avoid this issue? > > Thanks, > -- |
From: David O. <da...@qc...> - 2017-02-22 12:48:33
|
Hi, We've been looking at using the new revproxy module in our environment. I'm not sure we can use the module as is, but we're interested in using the spool & backendReply procs in out our code. I think we should be able to replace ns_http queue/ns_http wait with the same ns_connchan callbacks as utilised in the revproxy code. Short question: Is there a way to influence which driver thread "ns_connchan open" will use? Longer question: We have 2 driver threads listening on different ports, nsssl1 & nsssl2. Using backenReply/spool works ok for GET requests which come in on driver nsssl1 (we have issues with POSTs working but I've not looked at that yet). But if the request comes in on driver nsssl2, when we "ns_connchan open" to our backend, the channel is on nsssl1, so then when the backendReply callback is triggered it tries to read the handle and can't find it. eg: [22/Feb/2017:12:42:34][6739.7f6113fff700][-socks-] Error: connchan "conn0" does not exist connchan "conn0" does not exist while executing "ns_connchan read $from" (procedure "backendReply" line 6) invoked from within "backendReply conn0 conn1 https://localhost:8002/login_form.html 0 r" (context: connchan proc) What would be the best way to avoid this issue? Thanks, -- David Qcode Software Limited |
From: Brian F. <Bri...@qu...> - 2017-02-21 11:15:53
|
Thanks Gustaf Worked very well! Brian From: Gustaf Neumann [mailto:ne...@wu...] Sent: 20 February 2017 19:48 To: nav...@li... Subject: Re: [naviserver-devel] Differences in usage of "open" command between Naviserver and AOLserver Am 20.02.17 um 16:01 schrieb Brian Fenton: HI Gustaf I'd guess this is not a Windows thing, as I can reproduce the problem in standard TCL on both Windows and Linux. But I haven't looked at Naviserver on Linux to confirm it there. This worked fine for years in AOLserver on both Windows and Linux. It seems to be that Naviserver's version of "open" is closer to the standard TCL version. naviserver is not "closer" to Tcl, it uses the Tcl unmodified (including open). I see in the AOLserver sources no place, where it overwrites/modifies the "open" command. When should I write the "exit\n" to the pipe? I've tried a few variations on the following with no success: set fp [open "|[file join $::env(ORACLE_HOME) bin sqlplus] $user_pass @$file" "r+"] puts $fp "exit\n" while { [gets $fp line] >= 0 } { append output $line } close $fp The following should work. this is a little script (that works under Unix) that emulates sqlplus - as i understand it. Hope this helps -g ================================================================================ #!/usr/bin/env tclsh puts "GOT args <$argv>" # # Read a line from stdin to emulate, what sqlplus is doing # set line [gets stdin] puts "GOT line <$line>" if {0} { # # Place this file into /tmp/sqlplus.tcl, make it executable, call a # tclsh, and issue in this shell the following cmds: # set user_pass pw set file test.sql set fp [open "| /tmp/sqlplus.tcl $user_pass @$file" "r+"] fconfigure $fp -buffering line puts $fp "exit" while { [gets $fp line] >= 0 } { append output $line \n } close $fp puts output:\n$output } ================================================================================ |
From: Gustaf N. <ne...@wu...> - 2017-02-20 19:47:55
|
Am 20.02.17 um 16:01 schrieb Brian Fenton: > > HI Gustaf > > I’d guess this is not a Windows thing, as I can reproduce the problem > in standard TCL on both Windows and Linux. But I haven’t looked at > Naviserver on Linux to confirm it there. > > This worked fine for years in AOLserver on both Windows and Linux. It > seems to be that Naviserver’s version of “open” is closer to the > standard TCL version. > naviserver is not "closer" to Tcl, it uses the Tcl unmodified (including open). I see in the AOLserver sources no place, where it overwrites/modifies the "open" command. > > When should I write the “exit\n” to the pipe? I’ve tried a few > variations on the following with no success: > > set fp [open "|[file join $::env(ORACLE_HOME) bin sqlplus] $user_pass > @$file" "r+"] > > puts $fp "exit\n" > > while { [gets $fp line] >= 0 } { > > append output $line > > } > > close $fp > The following should work. this is a little script (that works under Unix) that emulates sqlplus - as i understand it. Hope this helps -g ================================================================================ #!/usr/bin/env tclsh puts "GOT args <$argv>" # # Read a line from stdin to emulate, what sqlplus is doing # set line [gets stdin] puts "GOT line <$line>" if {0} { # # Place this file into /tmp/sqlplus.tcl, make it executable, call a # tclsh, and issue in this shell the following cmds: # set user_pass pw set file test.sql set fp [open "| /tmp/sqlplus.tcl $user_pass @$file" "r+"] fconfigure $fp -buffering line puts $fp "exit" while { [gets $fp line] >= 0 } { append output $line \n } close $fp puts output:\n$output } ================================================================================ |
From: Brian F. <Bri...@qu...> - 2017-02-20 15:05:40
|
HI Gustaf I'd guess this is not a Windows thing, as I can reproduce the problem in standard TCL on both Windows and Linux. But I haven't looked at Naviserver on Linux to confirm it there. This worked fine for years in AOLserver on both Windows and Linux. It seems to be that Naviserver's version of "open" is closer to the standard TCL version. When should I write the "exit\n" to the pipe? I've tried a few variations on the following with no success: set fp [open "|[file join $::env(ORACLE_HOME) bin sqlplus] $user_pass @$file" "r+"] puts $fp "exit\n" while { [gets $fp line] >= 0 } { append output $line } close $fp thanks Brian From: Gustaf Neumann [mailto:ne...@wu...] Sent: 20 February 2017 12:48 To: nav...@li... Subject: Re: [naviserver-devel] Differences in usage of "open" command between Naviserver and AOLserver Hi Brian, I am not sure, how this ever worked, ... and maybe there is a big difference in the end-of-file treament of sqlplus under windows, but altering the tcl command "open" does not look right (this is used for all "open" operations). Doesn't the anwer in http://serverfault.com/questions/87035/run-oracle-sql-script-and-exit-from-sqlplus-exe-via-command-prompt help? (writing "exit\n" to the pipe, open pipe with "r+")? -g Am 20.02.17 um 13:01 schrieb Brian Fenton: Hi all We're in the early stages of an attempt to migrate from AOLserver to Naviserver. For now, we're just looking at the Windows version of Naviserver, but I guess this might be an issue on Unix versions too. We've already run into an issue with some OpenACS code that that uses "open" to open a pipe to Oracle's SQL*Plus command, and then runs a SQL script. It looks like AOLserver was doing something extra, which Naviserver doesn't seem to do. As a bit of background, when you run a SQL script in SQL*Plus, the script needs to have an EXIT command on the last line, which tells SQL*Plus to exit when finished. For some reason, in OpenACS, none of the provided SQL scripts have this EXIT, but with AOLserver it was never an issue. It seemed to close the channel after SQL*Plus was finished running. With Naviserver, the pipe hangs while waiting on SQL*Plus to exit. The obvious fix is to add the EXIT to the 1000s of SQL scripts included in OpenACS, but I'm hoping there's a smarter solution (something along the lines of this, if necessary http://serverfault.com/a/87038/9044 ). Here's an example of what I'm talking about: Create a file called hello.sql which contains the following line: select 'brian says hi' from dual; >From your command line, run it as follows: sqlplus user/pass@SID @hello.sql You will find that it doesn't exit the SQL*Plus session, unless you add an EXIT to the end of hello.sql Here is a version of the OpenACS code, that works on AOLserver (without the EXIT command in the SQL script) but is hanging on Naviserver: set file "c:/temp/hello.sql" cd [file dirname $file] set user_pass "user/pass@SID" set fp [open "|[file join $::env(ORACLE_HOME) bin sqlplus] $user_pass @$file" "r"] set output "" while { [gets $fp line] >= 0 } { append output $line } close $fp ns_log Notice $output If hello.sql has an EXIT, you will see output similar to below. Otherwise, it just waits, and you have to kill it. SQL*Plus: Release 12.1.0.2.0 Production on Fri Feb 17 16:45:34 2017Copyright (c) 1982, 2014, Oracle. All rights reserved.Connected to:Oracle Database 11g Relea se 11.2.0.3.0 - 64bit Production'BRIANSAYSHI'----------------------------------- ----brian says hiDisconnected from Oracle Database 11g Release 11.2.0.3.0 - 64bi t Production Any thoughts? Brian |
From: Gustaf N. <ne...@wu...> - 2017-02-20 12:48:32
|
Hi Brian, I am not sure, how this ever worked, ... and maybe there is a big difference in the end-of-file treament of sqlplus under windows, but altering the tcl command "open" does not look right (this is used for all "open" operations). Doesn't the anwer in http://serverfault.com/questions/87035/run-oracle-sql-script-and-exit-from-sqlplus-exe-via-command-prompt help? (writing "exit\n" to the pipe, open pipe with "r+")? -g Am 20.02.17 um 13:01 schrieb Brian Fenton: > > Hi all > > We’re in the early stages of an attempt to migrate from AOLserver to > Naviserver. For now, we’re just looking at the Windows version of > Naviserver, but I guess this might be an issue on Unix versions too. > > We’ve already run into an issue with some OpenACS code that that uses > “open” to open a pipe to Oracle’s SQL*Plus command, and then runs a > SQL script. It looks like AOLserver was doing something extra, which > Naviserver doesn’t seem to do. > > As a bit of background, when you run a SQL script in SQL*Plus, the > script needs to have an EXIT command on the last line, which tells > SQL*Plus to exit when finished. For some reason, in OpenACS, none of > the provided SQL scripts have this EXIT, but with AOLserver it was > never an issue. It seemed to close the channel after SQL*Plus was > finished running. > > > With Naviserver, the pipe hangs while waiting on SQL*Plus to exit. The > obvious fix is to add the EXIT to the 1000s of SQL scripts included in > OpenACS, but I’m hoping there’s a smarter solution (something along > the lines of this, if necessary http://serverfault.com/a/87038/9044 ). > > Here’s an example of what I’m talking about: > > *Create a file called hello.sql which contains the following line:* > > select 'brian says hi' from dual; > > *From your command line, run it as follows:* > sqlplus user/pass@SID @hello.sql > > You will find that it doesn’t exit the SQL*Plus session, unless you > add an EXIT to the end of hello.sql > > *Here is a version of the OpenACS code, that works on AOLserver > (without the EXIT command in the SQL script) but is hanging on > Naviserver:* > > set file "c:/temp/hello.sql" > > cd [file dirname $file] > > set user_pass "user/pass@SID" > > set fp [open "|[file join $::env(ORACLE_HOME) bin sqlplus] $user_pass > @$file" "r"] > > set output "" > > while { [gets $fp line] >= 0 } { > > append output $line > > } > > close $fp > > ns_log Notice $output > > *If hello.sql has an EXIT, you will see output similar to below. > Otherwise, it just waits, and you have to kill it.* > > SQL*Plus: Release 12.1.0.2.0 Production on Fri Feb 17 16:45:34 > 2017Copyright (c) > > 1982, 2014, Oracle. All rights reserved.Connected to:Oracle Database > 11g Relea > > se 11.2.0.3.0 - 64bit > Production'BRIANSAYSHI'----------------------------------- > > ----brian says hiDisconnected from Oracle Database 11g Release > 11.2.0.3.0 - 64bi > > t Production > > Any thoughts? > > Brian > > > > > |
From: Brian F. <Bri...@qu...> - 2017-02-20 12:18:03
|
Hi all We're in the early stages of an attempt to migrate from AOLserver to Naviserver. For now, we're just looking at the Windows version of Naviserver, but I guess this might be an issue on Unix versions too. We've already run into an issue with some OpenACS code that that uses "open" to open a pipe to Oracle's SQL*Plus command, and then runs a SQL script. It looks like AOLserver was doing something extra, which Naviserver doesn't seem to do. As a bit of background, when you run a SQL script in SQL*Plus, the script needs to have an EXIT command on the last line, which tells SQL*Plus to exit when finished. For some reason, in OpenACS, none of the provided SQL scripts have this EXIT, but with AOLserver it was never an issue. It seemed to close the channel after SQL*Plus was finished running. With Naviserver, the pipe hangs while waiting on SQL*Plus to exit. The obvious fix is to add the EXIT to the 1000s of SQL scripts included in OpenACS, but I'm hoping there's a smarter solution (something along the lines of this, if necessary http://serverfault.com/a/87038/9044 ). Here's an example of what I'm talking about: Create a file called hello.sql which contains the following line: select 'brian says hi' from dual; >From your command line, run it as follows: sqlplus user/pass@SID @hello.sql You will find that it doesn't exit the SQL*Plus session, unless you add an EXIT to the end of hello.sql Here is a version of the OpenACS code, that works on AOLserver (without the EXIT command in the SQL script) but is hanging on Naviserver: set file "c:/temp/hello.sql" cd [file dirname $file] set user_pass "user/pass@SID" set fp [open "|[file join $::env(ORACLE_HOME) bin sqlplus] $user_pass @$file" "r"] set output "" while { [gets $fp line] >= 0 } { append output $line } close $fp ns_log Notice $output If hello.sql has an EXIT, you will see output similar to below. Otherwise, it just waits, and you have to kill it. SQL*Plus: Release 12.1.0.2.0 Production on Fri Feb 17 16:45:34 2017Copyright (c) 1982, 2014, Oracle. All rights reserved.Connected to:Oracle Database 11g Relea se 11.2.0.3.0 - 64bit Production'BRIANSAYSHI'----------------------------------- ----brian says hiDisconnected from Oracle Database 11g Release 11.2.0.3.0 - 64bi t Production Any thoughts? Brian This e-mail transmission may contain confidential information that is intended for the individual or entity named on the e-mail address. If you are not the intended recipient, please reply to the sender so that Quest Computing Ltd can arrange for the proper delivery and then please delete the message from your inbox. If you have received this e-mail in error, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, 31-33 Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in Ireland No. 146435. |
From: David O. <da...@qc...> - 2017-02-17 09:50:10
|
Thanks again Gustaf, That looks great. Just as an aside on this particular use case, the reason we're building with --enable-symbols is debhelper has a dh_strip helper which will strip the symbols out and makes a separate debug package. So I'd want the symbols, but not for the asserts to be enabled. I'll try your suggestions - thanks again. On 16 February 2017 at 02:26, Gustaf Neumann <ne...@wu...> wrote: > Am 15.02.17 um 16:22 schrieb David Osborne: > > We spotted that, if the following conditions are met, then a malformed > HTTP request will make a Naviserver assert fail (4.99.15). > > - Naviserver is compiled with --enable-symbols > - At least 1 writerthread is running > - A malformed HTTP request is made to a fastpath file which is bigger > than writersize > > I have 2 questions. > > 1. Is there a way of compiling Naviserver with debug symbols included, > but with NDEBUG flag set as to not abort when an assert fails? > > The management of the NDEBUG flag is "inherited" via the TEA build rules > of Tcl (m4/tcl.m4). When built with "--enable-symbols", the flag is not > added, meaning that assert() calls will be evaluated. The assumption from > Tcl is that versions built with "--enable-symbols" are for developers, who > want as well asserts to be tested - but not for production sites. One can > override these settings by either editing include/Makefile.global, or via > providing the flag on the command line, like e.g.: > > make "CFLAGS_DEFAULT=-Os -DNDEBUG" > > 2. Is the failing assert showing us something interesting? > > The request is actually not a "malformed request", but a request > interpreted as a pre-HTTP/1.0 request (e.g. a request conforming to HTTP > 0.9, where the HTTP-version was not specified in the request line, and > where no reply header was returned; note that HTTP/1.0 was introduced in > 1996, more than 20 years ago). NaviServer still tries to handle such > requests, but there was a sanity check in the writer thread code, saying > that when the writer thread has to send the reply header, it must not be > zero, which does not hold for HTTP 0.9. > > The fix for this case is in bitbucket [1]. > > All the best, and many thanks for the detailed report. > -g > > [1] https://bitbucket.org/naviserver/naviserver/commits/ > 258264d620c1bb0164b5d6521d62b6cda064940a?at=default > > [13/Feb/2017:18:06:40][22169.7fb8a27fc700][-driver:nssock:0-] Notice: > pre-HTTP/1.0 request <GET /index.html> > nsd: driver.c:4872: NsWriterQueue: Assertion `wrSockPtr->headerString == > ((void *)0)' failed. > Aborted > > > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > |
From: Gustaf N. <ne...@wu...> - 2017-02-16 02:26:44
|
Am 15.02.17 um 16:22 schrieb David Osborne: > We spotted that, if the following conditions are met, then a malformed > HTTP request will make a Naviserver assert fail (4.99.15). > > * Naviserver is compiled with --enable-symbols > * At least 1 writerthread is running > * A malformed HTTP request is made to a fastpath file which is > bigger than writersize > > I have 2 questions. > > 1. Is there a way of compiling Naviserver with debug symbols included, > but with NDEBUG flag set as to not abort when an assert fails? The management of the NDEBUG flag is "inherited" via the TEA build rules of Tcl (m4/tcl.m4). When built with "--enable-symbols", the flag is not added, meaning that assert() calls will be evaluated. The assumption from Tcl is that versions built with "--enable-symbols" are for developers, who want as well asserts to be tested - but not for production sites. One can override these settings by either editing include/Makefile.global, or via providing the flag on the command line, like e.g.: make "CFLAGS_DEFAULT=-Os -DNDEBUG" > 2. Is the failing assert showing us something interesting? The request is actually not a "malformed request", but a request interpreted as a pre-HTTP/1.0 request (e.g. a request conforming to HTTP 0.9, where the HTTP-version was not specified in the request line, and where no reply header was returned; note that HTTP/1.0 was introduced in 1996, more than 20 years ago). NaviServer still tries to handle such requests, but there was a sanity check in the writer thread code, saying that when the writer thread has to send the reply header, it must not be zero, which does not hold for HTTP 0.9. The fix for this case is in bitbucket [1]. All the best, and many thanks for the detailed report. -g [1] https://bitbucket.org/naviserver/naviserver/commits/258264d620c1bb0164b5d6521d62b6cda064940a?at=default > [13/Feb/2017:18:06:40][22169.7fb8a27fc700][-driver:nssock:0-] Notice: > pre-HTTP/1.0 request <GET /index.html> > nsd: driver.c:4872: NsWriterQueue: Assertion `wrSockPtr->headerString > == ((void *)0)' failed. > Aborted |