Menu
▾
▴
naviserver-devel
|
From: Wolfgang W. <wol...@di...> - 2013-10-09 07:13:45
Attachments:
sockcallback.patch
|
Hello! We are using websockets on naviserver. After connections are established, we create a callback for the read event with ns_sockcallback and push them in the background with ns_chan. We hit a limit of exactly 100 connections, which went away, when we commented out the ns_sockcallback call. The problem is an invalid memory allocation. I've attached a patch, which allows us to open much more connections, although we still get a memory corruption at around 1000 connections, somtimes more, sometimes less. TCL is compiled with Gustaf Neumanns memory allocator path. This is the backtrace with the linux standard memory allocator *** glibc detected *** /usr/local/naviserver/bin/nsd: malloc(): memory corruption: 0x000000000b371b60 *** ======= Backtrace: ========= /lib/libc.so.6(+0x71e16)[0x7f9de5cf5e16] /lib/libc.so.6(+0x74ead)[0x7f9de5cf8ead] /lib/libc.so.6(__libc_malloc+0x70)[0x7f9de5cfac70] /usr/local/lib/libtcl8.5.so(Tcl_Alloc+0x15)[0x7f9de64b8475] /usr/local/lib/libtcl8.5.so(+0xafdf2)[0x7f9de652ddf2] /usr/local/lib/libtcl8.5.so(+0x330de)[0x7f9de64b10de] /usr/local/lib/libtcl8.5.so(Tcl_EvalEx+0x16)[0x7f9de64b1aa6] /usr/local/naviserver/lib/libnsd.so(Ns_TclEvalCallback+0x12b)[0x7f9de6e4682b] /usr/local/naviserver/lib/libnsd.so(NsTclTraceProc+0x1c)[0x7f9de6e4a60c] /usr/local/naviserver/lib/libnsd.so(+0x5984a)[0x7f9de6e4a84a] /usr/local/naviserver/lib/libnsd.so(+0x59c45)[0x7f9de6e4ac45] /usr/local/naviserver/lib/libnsd.so(Ns_TclAllocateInterp+0x15)[0x7f9de6e4aef5] /usr/local/naviserver/lib/libnsd.so(NsTclSockProc+0x42)[0x7f9de6e56eb2] /usr/local/naviserver/lib/libnsd.so(+0x5250f)[0x7f9de6e4350f] /usr/local/naviserver/lib/libnsthread.so(NsThreadMain+0x7e)[0x7f9de67a067e] /usr/local/naviserver/lib/libnsthread.so(+0x5029)[0x7f9de67a1029] /lib/libpthread.so.0(+0x68ca)[0x7f9de586a8ca] /lib/libc.so.6(clone+0x6d)[0x7f9de5d53b6d] We are running some test with the google allocator, but the results are similar. Wolfgang -- *Wolfgang Winkler* Geschäftsführung wol...@di... mobil +43.699.19971172 dc:*büro* digital concepts Novak Winkler OG Software & Design Landstraße 68, 5. Stock, 4020 Linz www.digital-concepts.com <http://www.digital-concepts.com> tel +43.732.997117.72 tel +43.699.1997117.72 Firmenbuchnummer: 192003h Firmenbuchgericht: Landesgericht Linz PS: BESUCHEN SIE UNSERE NEUE SHOP INFO SEITE: www.shop-info.at |
|
From: Gustaf N. <ne...@wu...> - 2013-10-09 08:55:19
|
Am 09.10.13 08:47, schrieb Wolfgang Winkler: > Hello! > > We are using websockets on naviserver. After connections are > established, we create a callback for the read event with > ns_sockcallback and push them in the background with ns_chan. We hit a > limit of exactly 100 connections, which went away, when we commented > out the ns_sockcallback call. > > The problem is an invalid memory allocation. I've attached a patch, > which allows us to open much more connections, although we still get a > memory corruption at around 1000 connections, somtimes more, sometimes > less. Dear Wolfgang, many thanks for the patch. The allocation unit was clearly wrong (number of entries vs. number of bytes); the same problem exists as well in aolserver. The proposed change has from my understanding 2 flaws (a memory leak, and an underallocation when max is low (e.g. 100) and the number of entries in the hash table is high (e.g. 1000), then only 200 entries are allocated). The second problem might be related to your still existing problem, Please check your code again with the following patch: https://bitbucket.org/naviserver/naviserver/commits/c35cd3d2394e61dc55d4c3118d7a14a7a774cd52 best regards -gustaf neumann > TCL is compiled with Gustaf Neumanns memory allocator path. This is > the backtrace with the linux standard memory allocator > > *** glibc detected *** /usr/local/naviserver/bin/nsd: malloc(): > memory corruption: 0x000000000b371b60 *** > ======= Backtrace: ========= > /lib/libc.so.6(+0x71e16)[0x7f9de5cf5e16] > /lib/libc.so.6(+0x74ead)[0x7f9de5cf8ead] > /lib/libc.so.6(__libc_malloc+0x70)[0x7f9de5cfac70] > /usr/local/lib/libtcl8.5.so(Tcl_Alloc+0x15)[0x7f9de64b8475] > /usr/local/lib/libtcl8.5.so(+0xafdf2)[0x7f9de652ddf2] > /usr/local/lib/libtcl8.5.so(+0x330de)[0x7f9de64b10de] > /usr/local/lib/libtcl8.5.so(Tcl_EvalEx+0x16)[0x7f9de64b1aa6] > /usr/local/naviserver/lib/libnsd.so(Ns_TclEvalCallback+0x12b)[0x7f9de6e4682b] > /usr/local/naviserver/lib/libnsd.so(NsTclTraceProc+0x1c)[0x7f9de6e4a60c] > /usr/local/naviserver/lib/libnsd.so(+0x5984a)[0x7f9de6e4a84a] > /usr/local/naviserver/lib/libnsd.so(+0x59c45)[0x7f9de6e4ac45] > /usr/local/naviserver/lib/libnsd.so(Ns_TclAllocateInterp+0x15)[0x7f9de6e4aef5] > /usr/local/naviserver/lib/libnsd.so(NsTclSockProc+0x42)[0x7f9de6e56eb2] > /usr/local/naviserver/lib/libnsd.so(+0x5250f)[0x7f9de6e4350f] > /usr/local/naviserver/lib/libnsthread.so(NsThreadMain+0x7e)[0x7f9de67a067e] > /usr/local/naviserver/lib/libnsthread.so(+0x5029)[0x7f9de67a1029] > /lib/libpthread.so.0(+0x68ca)[0x7f9de586a8ca] > /lib/libc.so.6(clone+0x6d)[0x7f9de5d53b6d] > > We are running some test with the google allocator, but the results > are similar. > > Wolfgang > > -- > > *Wolfgang Winkler* > Geschäftsführung > wol...@di... > mobil +43.699.19971172 > |
|
From: Wolfgang W. <wol...@di...> - 2013-10-09 10:27:39
|
Dear Gustaf! I tested the patch and it is working perfectly. Once again I'm impressed with NaviServer. I tested with 5200 websocket connections. Every 10 seconds each connection sends a "still alive" message. Posting a 1000 character message from the server to all 5200 sockets takes between 200 and 350 ms. The load on the server never exceeded 0.5 and typically hovers around 0.1. best regards Wolfgang Am 2013-10-09 10:55, schrieb Gustaf Neumann: > Am 09.10.13 08:47, schrieb Wolfgang Winkler: >> Hello! >> >> We are using websockets on naviserver. After connections are >> established, we create a callback for the read event with >> ns_sockcallback and push them in the background with ns_chan. We hit >> a limit of exactly 100 connections, which went away, when we >> commented out the ns_sockcallback call. >> >> The problem is an invalid memory allocation. I've attached a patch, >> which allows us to open much more connections, although we still get >> a memory corruption at around 1000 connections, somtimes more, >> sometimes less. > > Dear Wolfgang, > > many thanks for the patch. The allocation unit was clearly wrong > (number of entries vs. number of bytes); the same problem exists > as well in aolserver. > > The proposed change has from my understanding 2 flaws > (a memory leak, and an underallocation when max is low > (e.g. 100) and the number of entries in the hash table is > high (e.g. 1000), then only 200 entries are allocated). > The second problem might be related to your still > existing problem, > > Please check your code again with the following patch: > https://bitbucket.org/naviserver/naviserver/commits/c35cd3d2394e61dc55d4c3118d7a14a7a774cd52 > > best regards > -gustaf neumann > >> TCL is compiled with Gustaf Neumanns memory allocator path. This is >> the backtrace with the linux standard memory allocator >> >> *** glibc detected *** /usr/local/naviserver/bin/nsd: malloc(): >> memory corruption: 0x000000000b371b60 *** >> ======= Backtrace: ========= >> /lib/libc.so.6(+0x71e16)[0x7f9de5cf5e16] >> /lib/libc.so.6(+0x74ead)[0x7f9de5cf8ead] >> /lib/libc.so.6(__libc_malloc+0x70)[0x7f9de5cfac70] >> /usr/local/lib/libtcl8.5.so(Tcl_Alloc+0x15)[0x7f9de64b8475] >> /usr/local/lib/libtcl8.5.so(+0xafdf2)[0x7f9de652ddf2] >> /usr/local/lib/libtcl8.5.so(+0x330de)[0x7f9de64b10de] >> /usr/local/lib/libtcl8.5.so(Tcl_EvalEx+0x16)[0x7f9de64b1aa6] >> /usr/local/naviserver/lib/libnsd.so(Ns_TclEvalCallback+0x12b)[0x7f9de6e4682b] >> /usr/local/naviserver/lib/libnsd.so(NsTclTraceProc+0x1c)[0x7f9de6e4a60c] >> /usr/local/naviserver/lib/libnsd.so(+0x5984a)[0x7f9de6e4a84a] >> /usr/local/naviserver/lib/libnsd.so(+0x59c45)[0x7f9de6e4ac45] >> /usr/local/naviserver/lib/libnsd.so(Ns_TclAllocateInterp+0x15)[0x7f9de6e4aef5] >> /usr/local/naviserver/lib/libnsd.so(NsTclSockProc+0x42)[0x7f9de6e56eb2] >> /usr/local/naviserver/lib/libnsd.so(+0x5250f)[0x7f9de6e4350f] >> /usr/local/naviserver/lib/libnsthread.so(NsThreadMain+0x7e)[0x7f9de67a067e] >> /usr/local/naviserver/lib/libnsthread.so(+0x5029)[0x7f9de67a1029] >> /lib/libpthread.so.0(+0x68ca)[0x7f9de586a8ca] >> /lib/libc.so.6(clone+0x6d)[0x7f9de5d53b6d] >> >> We are running some test with the google allocator, but the results >> are similar. >> >> Wolfgang >> >> -- >> >> *Wolfgang Winkler* >> Geschäftsführung >> wol...@di... >> mobil +43.699.19971172 >> > > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk > > > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- *Wolfgang Winkler* Geschäftsführung wol...@di... mobil +43.699.19971172 dc:*büro* digital concepts Novak Winkler OG Software & Design Landstraße 68, 5. Stock, 4020 Linz www.digital-concepts.com <http://www.digital-concepts.com> tel +43.732.997117.72 tel +43.699.1997117.72 Firmenbuchnummer: 192003h Firmenbuchgericht: Landesgericht Linz PS: BESUCHEN SIE UNSERE NEUE SHOP INFO SEITE: www.shop-info.at |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X