naviserver-devel — Developer discussion

Re: [naviserver-devel] Naviserver and Tcl 8.6

[Cyan O. <cya...@gm...>]

Thanks, I'll give it a try on 8.6.1 first, since I've got a ton of code
that requires 8.6 features that it would be nice to use and see what bugs
shake loose.

If it doesn't pan out I'll fall back to 8.5 but at least it'll show up
areas that need work.  8.6 is pretty huge in terms of core changes so I'm
expecting quite a lot of breakage.

Cyan
On Jan 3, 2014 6:28 PM, "Gustaf Neumann" <ne...@wu...> wrote:

>  My recommendation would be to use in production naviserver with tcl
> 8.5.*. The main reason for the recommendation is lack of testing in
> generation with tcl 8.6, and in particular with interactions with new
> commands and features (e.g. coroutines, NRE etc.).
>
> We don't have a list of known issues or a check list for a thorough test.
> At least in the past, it was
> a good policy to wait a few releases before jumping on the train.
>
> all the best
> -gustaf
>
> Am 03.01.14 16:37, schrieb Cyan Ogilvie:
>
> Hi
>
> I'm looking at upgrading a fairly high load website from a prehistoric
> version of aolserver (3.4 with Tcl 7.6) to the latest naviserver and Tcl
> 8.6.1.
>
> Has anyone run a production website with naviserver and Tcl 8.6 yet?  I
> put together a quick test a few months back and got a basic hello world
> page to work, but the full site segfaulted, although I suspect that is
> mostly due to the mysql db changes.
>
> If this configuration isn't yet recommended for production systems does
> anyone have a list of known issues with it?  I've got a smallish time
> budget I can spend working on them if someone can point me at a list.
>
> Thanks
>
> Cyan
>
>
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>

Re: [naviserver-devel] Naviserver and Tcl 8.6

[Gustaf N. <ne...@wu...>]

My recommendation would be to use in production naviserver with tcl 
8.5.*. The main reason for the recommendation is lack of testing in 
generation with tcl 8.6, and in particular with interactions with new 
commands and features (e.g. coroutines, NRE etc.).

We don't have a list of known issues or a check list for a thorough 
test. At least in the past, it was
a good policy to wait a few releases before jumping on the train.

all the best
-gustaf

Am 03.01.14 16:37, schrieb Cyan Ogilvie:
>
> Hi
>
> I'm looking at upgrading a fairly high load website from a prehistoric 
> version of aolserver (3.4 with Tcl 7.6) to the latest naviserver and 
> Tcl 8.6.1.
>
> Has anyone run a production website with naviserver and Tcl 8.6 yet?  
> I put together a quick test a few months back and got a basic hello 
> world page to work, but the full site segfaulted, although I suspect 
> that is mostly due to the mysql db changes.
>
> If this configuration isn't yet recommended for production systems 
> does anyone have a list of known issues with it?  I've got a smallish 
> time budget I can spend working on them if someone can point me at a list.
>
> Thanks
>
> Cyan
>
>

[naviserver-devel] Naviserver and Tcl 8.6

[Cyan O. <cya...@gm...>]

Hi

I'm looking at upgrading a fairly high load website from a prehistoric
version of aolserver (3.4 with Tcl 7.6) to the latest naviserver and Tcl
8.6.1.

Has anyone run a production website with naviserver and Tcl 8.6 yet?  I put
together a quick test a few months back and got a basic hello world page to
work, but the full site segfaulted, although I suspect that is mostly due
to the mysql db changes.

If this configuration isn't yet recommended for production systems does
anyone have a list of known issues with it?  I've got a smallish time
budget I can spend working on them if someone can point me at a list.

Thanks

Cyan

Re: [naviserver-devel] threads and stability

[Gustaf N. <ne...@wu...>]

Am 02.01.14 20:53, schrieb John Buckman:
> After rebuilding tcl and naviserver (current) from source, with 
> -ltcmalloc, my naviserver is stable.
great. the tip version of naviserver have already some modifications
to help against concurrency bugs in tcl (e.g. reducing the frequency of 
interp
create operations, serializing these etc.).

The tcl-alloc bug that wolfgang winkler has mentioned is avoided by using a
different malloc(). Unfortunately i was not able to reproduce the problem
reliably by using e.g. just a c-program + tcl to help the tcl-core team
to provide a test-bed for fixing the problem.
> One small thing: I can crash naviserver on ctrl-c exit fairly easily:
> [02/Jan/2014:11:41:01][15491.7fee7ed75700][-conn:magnatune3:8] Fatal: 
> nsthreads: pthread_join failed in Ns_ThreadJoin: Invalid argument
i am aware of this. the same problem can appear on server-shutdowns.
as far i can tell this is tcl-version dependent, there is as well some
work going on to make tcl-shutdown cleaner from the tcl-core
community, my hope is that these changes will cause this annoyance
to "go away".

-gustaf

Re: [naviserver-devel] threads and stability

[Gustaf N. <ne...@wu...>]

Am 02.01.14 19:02, schrieb John Buckman:
> Hi Gustaf,
>
> I found today your great article on alternative mallocs:
> https://next-scripting.org/xowiki/docs/misc/thread-mallocs/index1
>
> could you possibly let me know what the correct way is to build Tcl 
> and Naviserver with tcmalloc?
In order to avoid tcl's zippy malloc, one has to patch the tcl sources, 
since
practically every malloc from tcl and naviserver happens finally through
Tcl's ckalloc(), which does not call the system malloc(), but its own
implementation (in a threaded case, this is zippy malloc).
Therefore a patch is required to make tcl call malloc(), and then one can
use the either the "system malloc" shipped with your OS or
one of the various malloc() implementation out there "on the market".

I'll send you a patch for tcl 8.5 in a separate mail. Apply this to the
tcl sources, recompile and install, and use the resulting tcl library
for linking against naviserver.

then one can use e.g.

    export LD_PRELOAD=/usr/lib64/libtcmalloc.so

to use TCMalloc for naviserver

all the best
-gustaf

> My way to do it, was the modify the Tcl Makefile like so:
> LDFLAGS                 = $(LDFLAGS_DEBUG)  -Wl,--export-dynamic 
> -ltcmalloc
>
> but that way doesn't carry the -ltcmalloc through to tclConfig.sh, and 
> so loadable extensions aren't built with tcmalloc, and so I suspect 
> that my way wasn't the right way to do it.
>
> Or should I just hand-modify the Naviserver makefile, and tclConfig.sh 
> to have that -ltcmalloc line on it?
>
> As to your question about stability of naviserver, I see now that I 
> had previously built Naviserver using the Tcl binaries from ActiveTcl 
> and I suspect that's the problem.  I'm in the process of rebuilding 
> entirely from source.
>
> I *have* successfully been using naviserver as a staging server, and 
> for development for the past 6 months. There's lots to love.
>
> My config file is lightweight, just a slight mod from one of the 
> samples I found from naviserver.  My hunch of the source of the 
> problem is that binary ActiveTcl distribution.  I'm using Tcl 8.5.15
>
> -john
>
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] threads and stability

[John B. <jo...@ma...>]

After rebuilding tcl and naviserver (current) from source, with -ltcmalloc, my naviserver is stable.

One small thing: I can crash naviserver on ctrl-c exit fairly easily:
[02/Jan/2014:11:41:01][15491.7fee7ed75700][-conn:magnatune3:8] Fatal: nsthreads: pthread_join failed in Ns_ThreadJoin: Invalid argument

I conducted two stress tests that went well.

------

A trivial stress test with 200 threads in naviserver, calling a page that just does:
<%= [clock seconds] %>

ab -n 2000 -c 200 http://magnatune.com:9000/timer2.adp

yields
Requests per second:    150.44 [#/sec] (mean)

Performance did not substantially change as I upped concurrency, nor does it look at that different with tcmalloc linked in.  However, this tcl test might be too simple to see tcmalloc performance effects.

-----

A "long page load" test, of:
ab -n 2000 -c 200 http://magnatune.com:9000/timer.adp

against:
<%= [after 5000] %>
<%= [clock seconds] %>

was made to see what would happen if requests came in faster than they could be fulfilled.   A minthread of 5 was set, with a maxthread of 200.

Good news: naviserver did indeed spawn new threads to deal with the backlog.

----

Re: [naviserver-devel] threads and stability

[John B. <jo...@ma...>]

Hi Gustaf,

I found today your great article on alternative mallocs:
https://next-scripting.org/xowiki/docs/misc/thread-mallocs/index1

could you possibly let me know what the correct way is to build Tcl and Naviserver with tcmalloc?

My way to do it, was the modify the Tcl Makefile like so:
LDFLAGS                 = $(LDFLAGS_DEBUG)  -Wl,--export-dynamic -ltcmalloc

but that way doesn't carry the -ltcmalloc through to tclConfig.sh, and so loadable extensions aren't built with tcmalloc, and so I suspect that my way wasn't the right way to do it. 

Or should I just hand-modify the Naviserver makefile, and tclConfig.sh to have that -ltcmalloc line on it?

As to your question about stability of naviserver, I see now that I had previously built Naviserver using the Tcl binaries from ActiveTcl and I suspect that's the problem.  I'm in the process of rebuilding entirely from source.

I *have* successfully been using naviserver as a staging server, and for development for the past 6 months. There's lots to love.  

My config file is lightweight, just a slight mod from one of the samples I found from naviserver.  My hunch of the source of the problem is that binary ActiveTcl distribution.  I'm using Tcl 8.5.15

-john

Re: [naviserver-devel] threads and stability

[Wolfgang W. <wol...@di...>]

Am 2014-01-02 12:20, schrieb John Buckman:
>>
>> We had crashes on startup as well. Here is a description and the 
>> solution of the problem:
>> http://sourceforge.net/p/tcl/bugs/5238/
>
> Hi Wolfgang, thanks for the tip.  I don't see anything in your bug log 
> indicating that you had a _startup_ problem, but rather a problem 
> under load, which appears to be a multithreading bug with the tcl malloc.
>
> I'm going to rebuild Tcl with a 3rd party malloc, and I was thinking 
> of using TCMalloc
> http://goog-perftools.sourceforge.net/doc/tcmalloc.html
>

Hi John!

The problem showed under heavy thread creation. When starting up with a 
high minhtreads value, the server crashed 1 out of 3 times. When hitting 
the server with ab or siege after it had started up and a lot of new 
threads where created, the server crashed as well. The stack trace was 
similar. Both problems where fixed with tcmalloc.

wolfgang


PS: BESUCHEN SIE UNSERE NEUE SHOP INFO SEITE: www.shop-info.at

Re: [naviserver-devel] threads and stability

[Gustaf N. <ne...@wu...>]

Hi John,

we are using naviserver on 20-30 installations with quite different 
configurations without the symptoms you are describing, so i would 
expect a problem in the configuration file.

Is your setup heavy-weight (like OpenACS) or rather light-weight?
If one is running a heavy-weight installation with minthreads 100 (as 
you are indicating) with limited resources, the os might kill the 
process (or e.g. an oom-killer).

Find a sample configuration files
     sample-config.tcl 
https://bitbucket.org/naviserver/naviserver/src/ada2e2dd98fde3bd8bfdb4e7ca3531306116705f/sample-config.tcl.in?at=default
     openacs-config.tcl 
https://bitbucket.org/naviserver/naviserver/src/ada2e2dd98fde3bd8bfdb4e7ca3531306116705f/openacs-config.tcl

Do you experience the "startup problem" with this config files as well?
What is exactly this problem? Is it the case that the server starts and
runs fine, but you do not see multiple connection threads created?
Maybe the server is delivering the files quite fast such it does not need
multiple connection threads?

When you use NaviServer 4.99.5 or newer, the request queue management
is quite different. Under normal operations, incoming requests are not 
queued
but directly dispatched to connection threads. Only when all connection
threads are busy, requests are added to the queue. For more details, see:
https://next-scripting.org/xowiki/docs/misc/naviserver-connthreadqueue/

In order to control the eagerness of connection thread creation,
please read the document above or read the comments in the
mentioned config files.

What is the reason for altering minthreads from 1 to 100 and not to a 
more sane value
like e.g. 5 or 10?

what version of naviserver + tcl are you using?

Are you experiencing the problem with real web clients
or with a synthetic benchmark?
Is it possible to post the used configuration file here?

best regards
-gustaf neumann

Am 01.01.14 18:57, schrieb John Buckman:
> Hello navifans,
>
> I'm in the process of trying to move to Naviserver from Aolserver.
>
> I'm finding that naviserver is not launching new threads, and is just qeueing requests up to be handled by a single thread, despite maxthreads being set high.
>
> I can get around that problem if I change
>     ns_section 	"ns/server/${servername}"
>     ns_param   	minthreads      	1
> to
>     ns_section 	"ns/server/${servername}"
>     ns_param   	minthreads      	100
>
> and then the server *does* use multiple threads.  That shouldn't be needed, though.
>
> Is there a config parameter I need to twiddle to enable one-demand thread creation?
>
> I also have the following two other problems:
> 1) if I try a minthreads of 50 or larger, nsd crashes on startup
> 2) apachebench with a concurrency of 50 causes nsd to crash (at 20 threads)
>
> I'm running debian squeeze, with a number of production aolservers that run stably also on the same box.
>
> Any thoughts?
>
> -john
>
>

-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria

Re: [naviserver-devel] threads and stability

[John B. <jo...@ma...>]

> 
> We had crashes on startup as well. Here is a description and the solution of the problem:
> http://sourceforge.net/p/tcl/bugs/5238/

Hi Wolfgang, thanks for the tip.  I don't see anything in your bug log indicating that you had a _startup_ problem, but rather a problem under load, which appears to be a multithreading bug with the tcl malloc.

I'm going to rebuild Tcl with a 3rd party malloc, and I was thinking of using TCMalloc
http://goog-perftools.sourceforge.net/doc/tcmalloc.html

I *think* I built Naviserver using ActiveState's Tcl build, and so I'll rebuild using a from-source Tcl, with your mallow swap, and see if that helps.

Apachebench reports almost 2x the performance with Naviserver vs Aolserver (250 pgs/sec, vs 150 pgs/sec) on a simple [clock seconds] page, so it's very much worth my time to try to get naviserver stable!

-john

Re: [naviserver-devel] threads and stability

[Wolfgang W. <wol...@di...>]

Hello!

We had crashes on startup as well. Here is a description and the 
solution of the problem:

http://sourceforge.net/p/tcl/bugs/5238/

There is a very good information, why you should do this, even when you 
experience no crashes here:

https://next-scripting.org/xowiki/docs/misc/thread-mallocs/index1

Concerning thread creation: use naviserver 4.99.5 from the repository, 
it has some significant improvements. Then check your server section in 
the configuration file. Our look like this:

ns_section "ns/server/${servername}"
   ns_param   directoryfile   $directoryfile
   ns_param   pageroot        $pageroot
   ns_param   enabletclpages  true ;# Parse *.tcl files in pageroot.

   # Maximum number of connection structures
   ns_param   maxconnections    240;# 100; determines queue size as well 
(good number: 100 + maxthreads)

   # Minimal and maximal number of connection threads
   ns_param   minthreads        20
   ns_param   maxthreads        40

   # Connection thread lifetime management
   ns_param connsperthread  10000        ;# Number of connections 
(requests) handled per thread
   ns_param threadtimeout   [expr 30*60] ;# Timeout for idle threads

   # Connection thread creation eagerness
   #ns_param  lowwatermark  10       ;# 10; create additional threads 
above this queue-full percentage
   #ns_param  highwatermark 100      ;# 80; allow concurrent creates 
above this queue-is percentage

The interesting bit ist the "maxconnections" parameter. This is the 
maximum queue length. In the above example, naviserver would put a 
maximum of 240 requests to the queue, before it returns errors to the 
clients. As there is a maximum of 40 threads, the real queue length ist 
200. When number of requests in the queue reaches the lowwatermark (you 
can see the value on startup), threads are created. When it reaches the 
highwatermark, threads are created in parallel. With "ns_server stats" 
you get some statistics, which tell you how many requests have been 
queued. There is also a page in nsstats (nsstats?@page=process) where 
you get the value in percent. If this values are very hight, you should 
consider a higher value for minthreads.

You can get much more information from here:

https://next-scripting.org/xowiki/docs/misc/naviserver-connthreadqueue/index1

With naviserver 4.99.5, the tcmalloc and the settings above, we have a 
very stable system which handles 200 pages/secs over several hours up to 
1500 pages/secs.

regards,

Wolfgang

Am 2014-01-01 18:57, schrieb John Buckman:
> Hello navifans,
>
> I'm in the process of trying to move to Naviserver from Aolserver.
>
> I'm finding that naviserver is not launching new threads, and is just qeueing requests up to be handled by a single thread, despite maxthreads being set high.
>
> I can get around that problem if I change
>     ns_section 	"ns/server/${servername}"
>     ns_param   	minthreads      	1
> to
>     ns_section 	"ns/server/${servername}"
>     ns_param   	minthreads      	100
>
> and then the server *does* use multiple threads.  That shouldn't be needed, though.
>
> Is there a config parameter I need to twiddle to enable one-demand thread creation?
>
> I also have the following two other problems:
> 1) if I try a minthreads of 50 or larger, nsd crashes on startup
> 2) apachebench with a concurrency of 50 causes nsd to crash (at 20 threads)
>
> I'm running debian squeeze, with a number of production aolservers that run stably also on the same box.
>
> Any thoughts?
>
> -john
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.72

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz



PS: BESUCHEN SIE UNSERE NEUE SHOP INFO SEITE: www.shop-info.at

[naviserver-devel] threads and stability

[John B. <jo...@ma...>]

Hello navifans,

I'm in the process of trying to move to Naviserver from Aolserver.  

I'm finding that naviserver is not launching new threads, and is just qeueing requests up to be handled by a single thread, despite maxthreads being set high.

I can get around that problem if I change
   ns_section 	"ns/server/${servername}"
   ns_param   	minthreads      	1
to
   ns_section 	"ns/server/${servername}"
   ns_param   	minthreads      	100

and then the server *does* use multiple threads.  That shouldn't be needed, though.

Is there a config parameter I need to twiddle to enable one-demand thread creation?

I also have the following two other problems:
1) if I try a minthreads of 50 or larger, nsd crashes on startup
2) apachebench with a concurrency of 50 causes nsd to crash (at 20 threads)

I'm running debian squeeze, with a number of production aolservers that run stably also on the same box.

Any thoughts?

-john

Re: [naviserver-devel] nsdbpg SSL mode

[David O. <da...@qc...>]

Amazon Postgresql RDS? That was exactly why we were asking.

(slightly off-topic - apologies)
We found that Postgresql RDS SSL was optional though.. could happily
connect directly to the RDS instance via psql "sslmode=disable"... and also
via nsdbpg but think this connection wouldn't be encrypted.

If you want to try stunnel we had to use the following client config to
connect successfully directly to a Postgresql RDS instance (we haven't
heavily tested this yet be warned):

[postgresql]
protocol = pgsql
client = yes
accept = localhost:5432
connect = pg_rds_server:5432
options = NO_TICKET


-- 
David

On 5 December 2013 23:44, Ian Harding <har...@gm...> wrote:

> Anyone?  I am experimenting with Amazon RDS and it appears to require
> SSL.  My naviserver nsdbipg module seems to barf on it.  psql connects fine.
>
>

Re: [naviserver-devel] nsdbpg SSL mode

[Stephen D. <sd...@gm...>]

On Thu, Dec 5, 2013 at 11:44 PM, Ian Harding <har...@gm...> wrote:
>
> My naviserver nsdbipg module seems to barf on it.
> psql connects fine.

nsbipg is configured with a datasource param which lets you pass any
key=value pairs directly through to libpq.

According to:

  http://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS

...the default sslmode is prefer, which apparently means "don't bother
trying". You actually need sslmode=require.

nsdbpg unfortunately implements it's own datasource parsing so you're
stuck with user:host:db


Are you also using nsssl? Looks like some modifications are required:

  http://www.postgresql.org/docs/9.3/static/libpq-ssl.html#LIBPQ-SSL-INITIALIZE

  If your application initializes libssl and/or libcrypto libraries
and libpq is built
  with SSL support, you should call PQinitOpenSSL to tell libpq that the libssl
  and/or libcrypto libraries have been initialized by your application, so that
  libpq will not also initialize those libraries.

But you could try connecting to the db via ssl with nsssl unloaded, to
confirm that it works.

Not sure what the best way is to coordinate with nsssl who should init
the openssl library.

Re: [naviserver-devel] nsdbpg SSL mode

[Ian H. <har...@gm...>]

Anyone?  I am experimenting with Amazon RDS and it appears to require SSL.
My naviserver nsdbipg module seems to barf on it.  psql connects fine.


On Mon, Nov 18, 2013 at 7:23 AM, David Osborne <da...@qc...> wrote:

>
> Hi,
>
> Can anyone tell us if it's possible to use the Naviserver nsdbpg drivers
> to connect directly via SSL to a SSL enabled Postgresql database?
>
> I think we'd be able to encrypt communication between an application
> server and DB server using stunnel if not, but wondered if this is
> something nsdbpg supports directly?
>
> Regards,
> --
> David
>
>
> ------------------------------------------------------------------------------
> DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
> OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
> Free app hosting. Or install the open source package on any LAMP server.
> Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
> http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
>

Re: [naviserver-devel] naviserver usage

[Ibrahim T. <it...@ar...>]

Congrats. Me proud of us ;)

Thanks for the ongoing efforts Gustav, Stephen, Vlad & Zoran.

-- Ibrahim


On 04-Dec-13 14:22, Gustaf Neumann wrote:
> Dear friends,
>
> not sure, where this is coming from, but the growth rate on the
> naviserver statistics
> at w3techs [1] looks nice:
>
> -g
> [1]: http://w3techs.com/technologies/details/ws-naviserver/all/all
>
> Historical trends in the usage of NaviServer

Re: [naviserver-devel] naviserver usage

[Zoran V. <zv...@ar...>]

Wow! 

Thanks for the excellent work!

Cheers,
Zoran

On 04.12.2013, at 14:22, Gustaf Neumann <ne...@wu...> wrote:

> Dear friends,
> 
> not sure, where this is coming from, but the growth rate on the naviserver statistics
> at w3techs [1] looks nice:
> 
> -g
> [1]: http://w3techs.com/technologies/details/ws-naviserver/all/all
> 
> <ws-naviserver.png>
> 
> 
> ------------------------------------------------------------------------------
> Sponsored by Intel(R) XDK 
> Develop, test and display web and hybrid apps with a single code base.
> Download it for free now!
> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk_______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

[naviserver-devel] naviserver usage

[Gustaf N. <ne...@wu...>]

Dear friends,

not sure, where this is coming from, but the growth rate on the 
naviserver statistics
at w3techs [1] looks nice:

-g
[1]: http://w3techs.com/technologies/details/ws-naviserver/all/all

Historical trends in the usage of NaviServer

[naviserver-devel] nsdbpg SSL mode

[David O. <da...@qc...>]

Hi,

Can anyone tell us if it's possible to use the Naviserver nsdbpg drivers to
connect directly via SSL to a SSL enabled Postgresql database?

I think we'd be able to encrypt communication between an application server
and DB server using stunnel if not, but wondered if this is something
nsdbpg supports directly?

Regards,
-- 
David

Re: [naviserver-devel] naviserver and Tcl 8.4

[Gustaf N. <ne...@wu...>]

Great!  Have a nice weekend! -gustaf

Am 12.10.13 14:08, schrieb Zoran Vasiljevic:
> On 11.10.2013, at 15:02, Gustaf Neumann <ne...@wu...> wrote:
>
>> Could the 8.4 camp live with an situation where
>> - we (jeff, david, ... me) don't test under tcl 8.4
> We can.
>
>> - we stay try to stay away from the tcl 8.5 C-API [4]
> Please. If not possible let us speak about that
> and we will find a solution.
>
>> - but we can use tcl 8.5 functions in the scripts?
> I see no problem here.
>
> Cheers
> Zoran
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA

Re: [naviserver-devel] naviserver and Tcl 8.4

[Zoran V. <zv...@ar...>]

On 11.10.2013, at 15:02, Gustaf Neumann <ne...@wu...> wrote:

> Could the 8.4 camp live with an situation where
> - we (jeff, david, ... me) don't test under tcl 8.4

We can.

> - we stay try to stay away from the tcl 8.5 C-API [4]

Please. If not possible let us speak about that
and we will find a solution.

> - but we can use tcl 8.5 functions in the scripts?

I see no problem here.

Cheers
Zoran

Re: [naviserver-devel] naviserver and Tcl 8.4

[Gustaf N. <ne...@wu...>]

from my point of view, just the tcl-level benefits are currently 
important [1], but these just relate to the few scripts in the 
naviserver distro. However, even there we have already one obvious tcl 
8.5  incompatibility in nstrace.tcl (using dict [2]), which can 
certainly programmed around for tcl 8.4, but my larger concerns are 
actually, that we do not test under tcl 8.4 and at some time we should 
move to tcl 8.5 anyhow. OpenACS has done this move [3], even MacPorts is 
considering this now :)

Could the 8.4 camp live with an situation where
- we (jeff, david, ... me) don't test under tcl 8.4
- we stay try to stay away from the tcl 8.5 C-API [4]
- but we can use tcl 8.5 functions in the scripts?

The other option is care for tcl8.4 compatibility for the next release
-gustaf

[1] http://wiki.tcl.tk/10630
[2] 
https://bitbucket.org/naviserver/naviserver/commits/10bf51a04b2fd746a871d2ef13b75a87a7101f6f
[3] http://openacs.org/forums/message-view?message_id=3357449
[4] http://wiki.tcl.tk/20591

Am 11.10.13 10:32, schrieb Zoran Vasiljevic:
> Hi!
>
> We are still using Tcl 8.4 and will do so for some
> time to come. I can however understand that could
> be potentially counter-productive for others as it
> may influence future developments.
>
> The server as-is currently links against 8.4 API.
> Do you have an idea what are the benefits of 8.5+
> API that would justify the move?
>   
> Cheers
> Zoran
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 
Univ.Prof. Dr. Gustaf Neumann
Institute of Information Systems and New Media
WU Vienna
Augasse 2-6, A-1090 Vienna, AUSTRIA

Re: [naviserver-devel] naviserver and Tcl 8.4

[Zoran V. <zv...@ar...>]

On 10.10.2013, at 23:14, Gustaf Neumann <ne...@wu...> wrote:

> Since nobody speaks up in favor of keeping Tcl 8.4 as requirement, i think
> we should go ahead and use tcl 8.5 when convenient (and therefore making
> Tcl 8.5 a requirement in the next release).

Hi!

We are still using Tcl 8.4 and will do so for some 
time to come. I can however understand that could 
be potentially counter-productive for others as it 
may influence future developments. 

The server as-is currently links against 8.4 API.
Do you have an idea what are the benefits of 8.5+ 
API that would justify the move?
 
Cheers
Zoran

[naviserver-devel] naviserver and Tcl 8.4

[Gustaf N. <ne...@wu...>]

Since nobody speaks up in favor of keeping Tcl 8.4 as requirement, i think
we should go ahead and use tcl 8.5 when convenient (and therefore making
Tcl 8.5 a requirement in the next release).

-gustaf neumann
>> In aolserver the code is wrapped to prevent it from executing on pre-8.5
>> tcl.  Is that still needed, or is 8.4 dead enough to have 8.5 as a
>> default requirement?
> so far, i have tried to keep non-optional stuff in naviserver tcl 8.4
> compatible, but
> i have stopped testing against 8.4. so, i would not mind if we would
> require tcl 8.5,
> but i know there are naviserver + tcl 8.4 installations out there....
>
> -g
>
>

Re: [naviserver-devel] patch for ns_sockcallback

[Wolfgang W. <wol...@di...>]

Dear Gustaf!

I tested the patch and it is working perfectly. Once again I'm impressed 
with NaviServer.

I tested with 5200 websocket connections. Every 10 seconds each 
connection sends a "still alive" message. Posting a 1000 character 
message from the server to all  5200 sockets takes between 200 and 350 
ms. The load on the server never exceeded 0.5 and typically hovers 
around 0.1.

best regards

Wolfgang

Am 2013-10-09 10:55, schrieb Gustaf Neumann:
> Am 09.10.13 08:47, schrieb Wolfgang Winkler:
>> Hello!
>>
>> We are using websockets on naviserver. After connections are 
>> established, we create a callback for the read event with 
>> ns_sockcallback and push them in the background with ns_chan. We hit 
>> a limit of exactly 100 connections, which went away, when we 
>> commented out the ns_sockcallback call.
>>
>> The problem is an invalid memory allocation. I've attached a patch, 
>> which allows us to open much more connections, although we still get 
>> a memory corruption at around 1000 connections, somtimes more, 
>> sometimes less. 
>
> Dear Wolfgang,
>
> many thanks for the patch. The allocation unit was clearly wrong
> (number of entries vs. number of bytes); the same problem exists
> as well in aolserver.
>
> The proposed change has from my understanding 2 flaws
> (a memory leak, and an underallocation when max is low
> (e.g. 100) and the number of entries in the hash table is
> high (e.g. 1000), then only 200 entries are allocated).
> The second problem might be related to your still
> existing problem,
>
> Please check your code again with the following patch:
> https://bitbucket.org/naviserver/naviserver/commits/c35cd3d2394e61dc55d4c3118d7a14a7a774cd52
>
> best regards
> -gustaf neumann
>
>> TCL is compiled with Gustaf Neumanns memory allocator path. This is 
>> the backtrace with the linux standard memory allocator
>>
>>  *** glibc detected *** /usr/local/naviserver/bin/nsd: malloc(): 
>> memory corruption: 0x000000000b371b60 ***
>> ======= Backtrace: =========
>> /lib/libc.so.6(+0x71e16)[0x7f9de5cf5e16]
>> /lib/libc.so.6(+0x74ead)[0x7f9de5cf8ead]
>> /lib/libc.so.6(__libc_malloc+0x70)[0x7f9de5cfac70]
>> /usr/local/lib/libtcl8.5.so(Tcl_Alloc+0x15)[0x7f9de64b8475]
>> /usr/local/lib/libtcl8.5.so(+0xafdf2)[0x7f9de652ddf2]
>> /usr/local/lib/libtcl8.5.so(+0x330de)[0x7f9de64b10de]
>> /usr/local/lib/libtcl8.5.so(Tcl_EvalEx+0x16)[0x7f9de64b1aa6]
>> /usr/local/naviserver/lib/libnsd.so(Ns_TclEvalCallback+0x12b)[0x7f9de6e4682b]
>> /usr/local/naviserver/lib/libnsd.so(NsTclTraceProc+0x1c)[0x7f9de6e4a60c]
>> /usr/local/naviserver/lib/libnsd.so(+0x5984a)[0x7f9de6e4a84a]
>> /usr/local/naviserver/lib/libnsd.so(+0x59c45)[0x7f9de6e4ac45]
>> /usr/local/naviserver/lib/libnsd.so(Ns_TclAllocateInterp+0x15)[0x7f9de6e4aef5]
>> /usr/local/naviserver/lib/libnsd.so(NsTclSockProc+0x42)[0x7f9de6e56eb2]
>> /usr/local/naviserver/lib/libnsd.so(+0x5250f)[0x7f9de6e4350f]
>> /usr/local/naviserver/lib/libnsthread.so(NsThreadMain+0x7e)[0x7f9de67a067e]
>> /usr/local/naviserver/lib/libnsthread.so(+0x5029)[0x7f9de67a1029]
>> /lib/libpthread.so.0(+0x68ca)[0x7f9de586a8ca]
>> /lib/libc.so.6(clone+0x6d)[0x7f9de5d53b6d]
>>
>> We are running some test with the google allocator, but the results 
>> are similar.
>>
>> Wolfgang
>>
>> -- 
>>
>> *Wolfgang Winkler*
>> Geschäftsführung
>> wol...@di...
>> mobil +43.699.19971172
>>
>
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel


-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.72

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz



PS: BESUCHEN SIE UNSERE NEUE SHOP INFO SEITE: www.shop-info.at