naviserver-devel — Developer discussion

Re: [naviserver-devel] nsproxy API

[Zoran V. <zv...@ar...>]

On 04.10.2006, at 16:52, Andrew Piskorski wrote:

> Designing better, high level, Tcl-ish APIs is non-trivial.  And if
> you're going to design some better high level API for waking up a
> thread, why even pollute your thinking with low-level "condition
> variable" stuff at all?  Zoran, I think you've already got some nice
> message passing style stuff in your Threads extension, maybe it would
> make more sense to add some friendlier "Heh thread X, wake up, you've
> got work to do now!" style API there?
>
> And if so, I'd say leave ns_cond alone, it's just a low-level wrapper
> around pthread_cond_*, and that's fine.  Futzing with it sounds likely
> to be wasted effort at best, and perhaps a source of very tricky
> destabalizing bugs at worst.

Every word true. I just gave it as an example to illustrate
what Stephen ment by "doing the Tcl way". I know that exposing
the low level C API to the Tcl level isn't going to please
everybody. And this is true. Just, to get "some" things right,
you need to step out of the circle and offer someting completely
different.
Out of the experience with the tcl threading extension I can say
that people *mostly* just use it in a most-simple fashion: post
scripts to threads and wait for results. But, ocasionally, they
need "alternate" way of synchronization and that is where mutexes
and condvars come into play. And, I must say, I really do not know
how to "solve" that in a different way. I'm not even going to try.

Speaking of that... the naviserver users (either current or future)
may really benefit of the "alternate" thread communication I did for
the Tcl threading extension (basically message-passing thing), but
this is another story.

Re: [naviserver-devel] ns_cond wait (Was: hosting java-script server-side?)

[Zoran V. <zv...@ar...>]

On 04.10.2006, at 15:52, Stephen Deasey wrote:
>
> Hmm...  I think this does work. Here's how I read it:

First time arround, there will be 3 lookups.
The next round should be simple deref.
Allright this might compute. If you have this in a proc
body the "themutex" will be a literal objects and will/should
last "longer".
I did it under the debugger in the nscp session and every
time I get another object as "themutex". This is most probably
because the code does not compile into bytecodes...

OK. I can accept that. Still, you put this into frequently
called procedure and there you go (literals are freed AFAIK,
when procedure scope is exited): again you have global locking.

>
>> If one however does:
>>
>>   set m themutex
>>   set t thecond
>>
>>    ns_mutex lock $m
>>    while {} {
>>       ns_cond wait $c $m
>>    }
>>    ns_mutex unlock $m
>>
>> it WILL work. But how are you going to convey this information
>> to the Tcl programmer??? He's implicitly serializing his app
>> at the place he does not expect (lookup of the internal hash
>> table).
>
>
> Actually, the above might NOT work... :-)

The first two init lines I havent counted for. It is just the
remaining and in fact most specifically the
    ns_cond wait
that was the problem as it may awake and sleep again.

>
> But I'm having a hard time coming up with real world scenarios where
> that might happen...

This is OK. In the compiled code, things start to look somehow  
different.
But also when you use the "themutex" to put it in the nsv array
you loose the caching effect of the object. Unlike with handles
where you get the "real thing" immediately and need not global
lock.

Keeping the handles "away" from the user, means you need to manage
handles yourself because the underlying C code needs handles/pointers.
Every time you do that, you need to lock. In this case the global lock.
So, nothing is for free. By allowing the tcl programmer some freedom,
you charge him with some performance/concurrency.

The best illustration:

lexxsrv:nscp 7> ns_mutex unlock themutex
Connection closed by foreign host.

Here no check is done: you get a core. But if the themutex was
locked before, all would be fine.
In the Tcl threading extension I check mutex for a locked state
and TCL_ERROR if lock was attempted on an already locked mutex
OR an attempt was made to unlock never-locked mutex.
This requires lots of low-level plumbing BUT it gives the Tcl
coder maximum comfort.

Neither way is "right" or "wrong".  So you can't generally
"avoid" using handles as this will lead you to situation
where you must sacrify some speed/concurrency for getting
the comfortable API.

Re: [naviserver-devel] nsproxy API

[Andrew P. <at...@pi...>]

On Wed, Oct 04, 2006 at 12:05:48PM +0200, Zoran Vasiljevic wrote:
> I've been thinking about that...
> Basically, what you try to avoid is to replicate common
> C-idioms to Tcl because Tcl is not C after all.
>
> Would this include common threading paradigms like

>       cond wait cond mutex

ns_cond has exactly the same semantics and usage style as the C
pthread_cond_*() functions which underly it, and thus is precisely as
confusing and low-level as they are.  This is annoying, especially
when using ns_cond for the first time.

However, it also has the important ADVANTAGE that the ns_cond
implementation is simple, and that you know that ns_cond* and
pthread_cond_* are in fact intended to work EXACTLY the same way.

ns_cond has been extremely useful to me on the rare occasions when I
needed it, but the average Naviserver user probably never uses it even
once.  And AOLserver (and thus Naviserver) has had ns_conf for many,
many years.  I bet the original ns_cond implementor couldn't think of
a much better design, so he did the obvious simple thing: just wrap
the ugly C APIs as is.

Designing better, high level, Tcl-ish APIs is non-trivial.  And if
you're going to design some better high level API for waking up a
thread, why even pollute your thinking with low-level "condition
variable" stuff at all?  Zoran, I think you've already got some nice
message passing style stuff in your Threads extension, maybe it would
make more sense to add some friendlier "Heh thread X, wake up, you've
got work to do now!" style API there?

And if so, I'd say leave ns_cond alone, it's just a low-level wrapper
around pthread_cond_*, and that's fine.  Futzing with it sounds likely
to be wasted effort at best, and perhaps a source of very tricky
destabalizing bugs at worst.

-- 
Andrew Piskorski <at...@pi...>
http://www.piskorski.com/

Re: [naviserver-devel] ns_cond wait (Was: hosting java-script server-side?)

[Stephen D. <sd...@gm...>]

On 10/4/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> On 04.10.2006, at 11:24, Zoran Vasiljevic wrote:
>
> > I think I see where the problem is and will fix that now.
>
> I've fixed that but this opens another discussion.
>
> Basically you wanted to avoid using handles. Then you said,
> OK, I can use string names and turn them to handles on as-needed
> basis by looking them up in an internal table. Everytime this
> lookup happens I need to global lock the table then lookup the
> (whatever)handle using it string name.
>
> To avoid that global lock, you change/update the object holding the
> string rep with the calculated handle and you count on the fact
> that next usage of the same object will avoid having to to the
> (costly) lookup.


Right.

And the reason the handle name doesn't have to be looked up again
because we cheat -- the mutexes associated with the handle names live
forever. You can't delete or reassign them. So if you ever find a
pointer to a mutex in one of these handle names, it's valid.


> Did you think about such uses:
>
>    ns_mutex lock themutex
>    while {} {
>       ns_cond wait thecond themutex
>    }
>    ns_mutex unlock themutex
>
> because here EVERY time you will have to lock the global
> lock to get the handle of the "thing", as the "trick" to
> save it in the object will not work! The "themutex" or
> "thecond" could/will always be different objects if used
> this way, so there is no gain.


Hmm...  I think this does work. Here's how I read it:

There are 3 string objects with the value "themutex" in the above
code.  The first time the code runs, there will be 3 global look ups
in the hash table of mutex names to convert the name "themutex" to a
pointer to an actual mutex.

On the first line, in the call to "ns_mutex lock themutex", the last
argument is just a Tcl string object so we have to look up the name in
the mutex hash table. The name doesn't exist, so a new mutex is
allocated, put in the hash table, and a pointer to it is stored in the
Tcl object. The Tcl object is marked as an Address obj type (it's
string rep is not invalidated, we just change the type).

In the call to ns_cond and the last call to ns_mutex, "themutex" is
also just a Tcl string object. So again the mutex hash table has to be
consulted. This time the mutex exists however, so we simply change the
string object to an Address type and store a pointer to the existing
mutex in it.

There are 3 individual Tcl objects, each with the string
representation "themutex". Each now points to exactly the same mutex
in memory.

The second time the code is run, the 3 "themutex" objects retain their
type (Address), so we don't have to look up the name in the mutex hash
table as a pointer to the mutex is stored in the Tcl object itself. We
get access to the mutex with a simple pointer dereference.

So, the premise is that the code will be executed more than once. The
first time will be 'slow', but after that there is no locking at all
to convert the name to an actual mutex.

And it works specifically in the above case because mutex objects are
seperate from Tcl objects which point to them.


> If one however does:
>
>   set m themutex
>   set t thecond
>
>    ns_mutex lock $m
>    while {} {
>       ns_cond wait $c $m
>    }
>    ns_mutex unlock $m
>
> it WILL work. But how are you going to convey this information
> to the Tcl programmer??? He's implicitly serializing his app
> at the place he does not expect (lookup of the internal hash
> table).


Actually, the above might NOT work... :-)

In the above code, $m is a single Tcl string object with the value
"themutex" and, after the first call to "ns_mutex lock $m", we convert
it to a an Address type and point it at a mutex.

There is a single Tcl object pointing to a single mutex, where as in
the original code there were 3 Tcl objects pointing to a single mutex.
There will be a single look up in the mutex hash table, where as in
the original code there will be 3. But only on the first time the code
is run.

The while loop will be 'efficient' because the underlying mutex object
is just a pointer dereference away.

However, what if the whole code block was run in a loop?

At the bottom of the loop, $m is an Address type pointing to our
mutex. At the top of the loop $m is reset:

    set m themutex

The value of the literal "themutex" Tcl string object is copied and
used as the new value of $m. Because the string representation of $m
has changed (even if the actual text is literaly the same, there is no
checking for this), the internal representation is invalidated. i.e.
it is no longer an Address type.

So at the top of the loop we again have to convert m to an Address
type be looking up the name in the mutex hash table. The mutex will
exist in the hash table, but we need to get a pointer to it.

Any operation which looses the type information will break the caching
of the mutex pointer in the object. So, passing the name through an
nsv array or cache will strip the type info.

But I'm having a hard time coming up with real world scenarios where
that might happen...

Re: [naviserver-devel] ns_cond wait (Was: hosting java-script server-side?)

[Zoran V. <zv...@ar...>]

On 04.10.2006, at 11:24, Zoran Vasiljevic wrote:

> I think I see where the problem is and will fix that now.

I've fixed that but this opens another discussion.

Basically you wanted to avoid using handles. Then you said,
OK, I can use string names and turn them to handles on as-needed
basis by looking them up in an internal table. Everytime this
lookup happens I need to global lock the table then lookup the
(whatever)handle using it string name.

To avoid that global lock, you change/update the object holding the
string rep with the calculated handle and you count on the fact
that next usage of the same object will avoid having to to the
(costly) lookup.

Did you think about such uses:

   ns_mutex lock themutex
   while {} {
      ns_cond wait thecond themutex
   }
   ns_mutex unlock themutex

because here EVERY time you will have to lock the global
lock to get the handle of the "thing", as the "trick" to
save it in the object will not work! The "themutex" or
"thecond" could/will always be different objects if used
this way, so there is no gain.

If one however does:

  set m themutex
  set t thecond

   ns_mutex lock $m
   while {} {
      ns_cond wait $c $m
   }
   ns_mutex unlock $m

it WILL work. But how are you going to convey this information
to the Tcl programmer??? He's implicitly serializing his app
at the place he does not expect (lookup of the internal hash
table).

I believe this is not very optimal and requires some other
"means" of hiding this information (the handle), potentially
using complete different paradigms of programming (more suitable
to Tcl way of doing things - if there is such a "way" at all).

Re: [naviserver-devel] nsproxy API

[Zoran V. <zv...@ar...>]

On 03.10.2006, at 01:01, Stephen Deasey wrote:

>     ns_serialize {
>         # whatever
>     }

I've been thinking about that...
Basically, what you try to avoid is to replicate common
C-idioms to Tcl because Tcl is not C after all.
Would this include common threading paradigms like

   mutex lock
   while (true) {
      cond wait cond mutex
   }
   mutex ulock

?

To what extent are you prepared to go?
More specifically, how would you code this
particular thing in a Tcl-like way?

  

Re: [naviserver-devel] ns_cond wait (Was: hosting java-script server-side?)

[Zoran V. <zv...@ar...>]

On 04.10.2006, at 09:42, Zoran Vasiljevic wrote:

(answering my own questions)

>
> But... I believe there is a "hole" there. Look:
>
> lexxsrv:nscp 7> ns_mutex lock themutex
> lexxsrv:nscp 8> ns_cond wait thecond themutex
> invalid address "themutex"
> lexxsrv:nscp 9>
>
> I could however:
>
> lexxsrv:nscp 9> ns_cond broadcast thecond
> lexxsrv:nscp 10>

There is a slight "glicht" as this one works
fine:

lexxsrv:nscp 1> set m mux
mux
lexxsrv:nscp 2> ns_mutex lock $m
lexxsrv:nscp 3> ns_cond wait thecond $m 100

This is because you expect the mutex argument of the ns_cond
to be of the Address type which is not always true.
I think I see where the problem is and will fix that now.

Re: [naviserver-devel] nsproxy API

[Zoran V. <zv...@ar...>]

On 03.10.2006, at 01:01, Stephen Deasey wrote:

>
> It's not just the cleanup-up after error.  Why does the Tcl programmer
> need to get and return resources?  It's kind of awkward.

Not necessarily... Think of a handle as a "reservation". I take
the handle and I "own" the resource for some time. In case of
nsproxy, I can run two or more "things" one after another, possibly
sharing the common state in the proxy slave.

>
> For example:
>
>     ns_serialize {
>         # whatever
>     }
>
> is better than the mutex example above because you also don't have to
> mess around with the mutex handle and nsv arrays. The Tcl code in the
> code block serves as a key into a hash table of locks, created on
> demand.  We can let the code block shimmer to an object which contains
> both a byte code script and a mutex pointer. It will run as fast as
> possible the second time round.
>
> More complicated syntax for a more complicated operation:
>
>     ns_serialize -lock mylock {
>         # whatever
>     }
>
> Now you could run two different code blocks using the same lock. But
> you still don't have to allocate mutex objects, pass around their
> names in nsv arrays, and so on.
>
> I don't think the above is dumbed-down. It's a much nicer way to work
> with Tcl, and you just wouldn't do this kind of thing from C.


Guess what... exactly this I have done in the Tcl threading extension
for the thread::eval command:

   thread::eval ?-lock mutex? arg ?arg ...?

(this is perhaps a misnomer as script is evaluated in the _current_
  and not in some other thread)

Which means: I completely understand what you say. Just in some cases
it is feasible (like in example for the mutex) to avoid fiddling with
handles and in some cases (nsproxy) it opens new posibilities.


>
>
> Anyway, back to nsproxy.  Thinking about it further, it looks like the
> explicit handles are actually there to allow a particular kind of Tcl
> programming, not necessarily because explicit handle management is
> needed. i.e.:
>
>     set x [create ...
>     $x method ...
>
> Which is perhaps not a bad way to manage the handles if you absolutely
> need them. But if you can instead do:
>
>     ns_proxy eval {
>         # whatever
>     }
>
> which in the case of a default pool and a single evaluation, you can,
> it's got to be better than the explicit handle management, right?

As said, in this special case: yes. But otherwise: no. What I CAN do
with explicit handle managment is:

   set proxy [ns_proxy get thepool]
   $proxy "set a 1"
   $proxy "puts $a"

This is not possible w/o explicit handle. It is also the reservation
aspect that is important when using the handle. This aspect is of
course nil for mutex, condvar or other "things".


> This is basically identical to the db module.  People hate manging db
> handles, which is why the ACS style db_* API is so popular.

It is all the matter of the programming case. In some cases it is
absolutely stupid to use handles, I agree. In some cases it opens
new possibilities.

>
>
> I was also wondering about the ns_proxy send/wait/receive.  Why are
> wait and receive separate commands?

Because you can wait, get the timeout, do something and then go repeat
waiting. It makes sense. You can't achieve this with a longer wait
timeout OR with a timed [ns_proxy eval].
Allright, you can argue: one can [ns_proxy receive proxy ?timeout?]
in which case you have the same behaviour. Correct. But what difference
would it make, really?

>
> Also, does there need to be so many timeouts?  The waittimeout is 100
> msec. That means if all else goes well, but the wait time takes 101
> msec, the evaluation will not be successful. But looking at the other
> defaults, the script evaluator was prepared to wait (gettimeout 5000 +
> evaltimeout infinite + sendtimeout 1000 + recvtimeout 1000), or
> between six seconds and forever...  Wouldn't a single timeout do?

There are lots of "places" something can go "wrong" at the communication
path. Hence so many timeouts. At every place you send something or
receive something, there is a timeout. Timeout to send chunk of
data to proxy isn't the same as the timeout to wait for the proxy to
respond after feeding it some command to execute. Basically, one can
hardwire "sane" values for those communication timeouts (and there are
sane values set there as defaults) but somebody may come into the need
of adjusting them during runtime. You however do not need to do that,
as sane defaults are provided everywhere.


>
> It would also be great if the timeout error code began with the
> NS_TIMEOUT symbol, which if not caught will automatically trigger a
> Server Busy error page.

This is not a problem.

Re: [naviserver-devel] nsproxy API

[Zoran V. <zv...@ar...>]

On 03.10.2006, at 00:05, Stephen Deasey wrote:


>
> I'm not sure how much real code sharing there would be between a
> combined job/proxy module. The back ends are different, processes can
> have more limits, the focus on parallelism versus isolation...

Apart from the "unified" API there would be no other "benefits".
Code sharing would be rather minimal. It would perhaps be easier
to use since you do not need to remember just-another-api.
But I'm perfectly happy with both. We would have to find a consensus
on that and I'll do it.

>
>
>>       The oher thing, I do not like the "default" pool. I think  
>> the pool
>>       (being the place to put common things shared across all slaves)
>>       should be given explicitly.
>
>
> It would allow a simple 'set' command to configure parameters, like
> the limits stuff.  The proxy pool 'default' already exists, and then
> you adjust it if you need to.
>

Hmmm... set "what"?

> With a default pool, we could alias the Tcl 'exec' command to run in a
> proxy pool.

This is the good reason.

>
> Although you can configure the binary to bootstrap the proxy
> processes, that's probably going to be rare. In fact most situations
> would probably be covered by a sane, default setup.

True.

>
> If there's no default pool, then all user code which uses nsproxy will
> create it's own pool. With multiple pools there may be some which run
> out of resources while others have spare -- the pools are a hard
> partitioning of resources.

True. But pools are there because of the different:

   timers
   initialization scripts

I use the nsproxy in 2 places in the code now. Both require different
init scripts. One requires indefinite timer (like simple exec) and the
one "can" kill the process if the computation went more than 30 seconds.

It is true what you say about partitioning of resources, though...
Still, when I have to choose, I'd rather opt for explicit pool
rather than having a "default" pool. I'd really leave this to the
programmer. The programmer can do trivial Tcl wrapper to redirect
Tcl exec to the proxy if needed.

>
>
>>    b. If we go that far then ns_exec_xxx is also not a suitable  
>> name as
>>       we are not exec'ing anything really.
>
>
> A common reason to use the proxy module is to isolate calls to the Tcl
> exec command. The proxies must also execute an external process
> (default nsproxy).  Just trying to get away from 'proxy'. Maybe
> something with 'ext', for 'external' in the name?

To summarize. What you object is:

   a. handle management (should work without handles)
   b. name of the module (should not be proxy)
   c. invent default pool
   d. ?


For a:
I understand the syntactical need, but this is how I see that from
the practical side. By having a handle, I can run "things" in the
proxy (lets call it slave process from now on) one after another with
a possible "state" between the runs. This might sometimes be needed.
If I hide the handle, how could I force my two or three consecutive
operations to be executed in the same slave?

For b.
I do not care how we call it. We can call it ns_cocacola if you like.
The name contest is open...

For c.
I'd rather stick to explicit pool naming. I'd leave this "default"
to the programmer. The programmer might do something like
(not 100% right but serves the illustration purpose):

    ns_proxy config default
    rename ::exec tcl::exec
    proc ::exec args {
        ns_proxy eval default $args
    }

This covers overloading of Tcl exec command. If you can convince me
that there are other benefits of having the default pool I can
think about them. I just do not see any at this point.

[naviserver-devel] ns_cond wait (Was: hosting java-script server-side?)

[Zoran V. <zv...@ar...>]

On 03.10.2006, at 00:13, Stephen Deasey wrote:

>
> (It's different in naviserver. you can pass in a string name for the
> mutex, and if it doesn't exits, it's automatically created. this
> mostly works...)

Interesting... (I did not know that, which stresses the importance of
documentation again...)

But... I believe there is a "hole" there. Look:

lexxsrv:nscp 7> ns_mutex lock themutex
lexxsrv:nscp 8> ns_cond wait thecond themutex
invalid address "themutex"
lexxsrv:nscp 9>

I could however:

lexxsrv:nscp 9> ns_cond broadcast thecond
lexxsrv:nscp 10>

So it is the [ns_cond wait] that does not play exactly by the rules.
Should I look into that or would you?
I'm asking this because I'd like to use this feature and drop our
Tcl wrappers which do basically the same thing...

Re: [naviserver-devel] nsproxy API

[Rick C. <rc...@Kn...>]

Uh, maybe "ns_process"?

-- ReC

-----Original Message-----
From: nav...@li...
[mailto:nav...@li...] On Behalf Of
Stephen Deasey
Sent: Monday, October 02, 2006 3:06 PM
To: nav...@li...
Subject: Re: [naviserver-devel] nsproxy API

On 9/19/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> But lets go two steps back...
>
> You say you wanted to get ns_job and ns_proxy together. You also say
> that this is not feasible because of the "eval" command.
> Why?
> The ns_job could be expanded easily to integrate the "eval" command,
> but it would not make much sense (i.e. practical use).


Well, exactly -- it doesn't make any sense.  I think it would be a
mistake to encourage people to do the wrong thing.

There was a guy on the AOLserver list the other day, obviously
experienced and smart, who 'forgot' that the proxy script ran in an
external process, when that's basically the entire point, right?

If we give people an ns_job eval command, they'll get confused about
the purpose of job pools, too.

I'm not sure how much real code sharing there would be between a
combined job/proxy module. The back ends are different, processes can
have more limits, the focus on parallelism versus isolation...


>       The oher thing, I do not like the "default" pool. I think the
pool
>       (being the place to put common things shared across all slaves)
>       should be given explicitly.


It would allow a simple 'set' command to configure parameters, like
the limits stuff.  The proxy pool 'default' already exists, and then
you adjust it if you need to.

With a default pool, we could alias the Tcl 'exec' command to run in a
proxy pool.

Although you can configure the binary to bootstrap the proxy
processes, that's probably going to be rare. In fact most situations
would probably be covered by a sane, default setup.

If there's no default pool, then all user code which uses nsproxy will
create it's own pool. With multiple pools there may be some which run
out of resources while others have spare -- the pools are a hard
partitioning of resources.


>    b. If we go that far then ns_exec_xxx is also not a suitable name
as
>       we are not exec'ing anything really.


A common reason to use the proxy module is to isolate calls to the Tcl
exec command. The proxies must also execute an external process
(default nsproxy).  Just trying to get away from 'proxy'. Maybe
something with 'ext', for 'external' in the name?

------------------------------------------------------------------------
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share
your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDE
V
_______________________________________________
naviserver-devel mailing list
nav...@li...
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] nsproxy API

[Stephen D. <sd...@gm...>]

On 9/19/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> But lets go two steps back...
>
> You say you wanted to get ns_job and ns_proxy together. You also say
> that this is not feasible because of the "eval" command.
> Why?
> The ns_job could be expanded easily to integrate the "eval" command,
> but it would not make much sense (i.e. practical use).


Well, exactly -- it doesn't make any sense.  I think it would be a
mistake to encourage people to do the wrong thing.

There was a guy on the AOLserver list the other day, obviously
experienced and smart, who 'forgot' that the proxy script ran in an
external process, when that's basically the entire point, right?

If we give people an ns_job eval command, they'll get confused about
the purpose of job pools, too.

I'm not sure how much real code sharing there would be between a
combined job/proxy module. The back ends are different, processes can
have more limits, the focus on parallelism versus isolation...


>       The oher thing, I do not like the "default" pool. I think the pool
>       (being the place to put common things shared across all slaves)
>       should be given explicitly.


It would allow a simple 'set' command to configure parameters, like
the limits stuff.  The proxy pool 'default' already exists, and then
you adjust it if you need to.

With a default pool, we could alias the Tcl 'exec' command to run in a
proxy pool.

Although you can configure the binary to bootstrap the proxy
processes, that's probably going to be rare. In fact most situations
would probably be covered by a sane, default setup.

If there's no default pool, then all user code which uses nsproxy will
create it's own pool. With multiple pools there may be some which run
out of resources while others have spare -- the pools are a hard
partitioning of resources.


>    b. If we go that far then ns_exec_xxx is also not a suitable name as
>       we are not exec'ing anything really.


A common reason to use the proxy module is to isolate calls to the Tcl
exec command. The proxies must also execute an external process
(default nsproxy).  Just trying to get away from 'proxy'. Maybe
something with 'ext', for 'external' in the name?

Re: [naviserver-devel] nsproxy API

[Stephen D. <sd...@gm...>]

On 9/18/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> On 18.09.2006, at 23:03, Stephen Deasey wrote:
>
> >
> > I mean that
> >
> >     Ns_ProxyGet()
> >
> > might be appropriate for C programmers, but that doesn't mean that
> >
> >     ns_proxy get
> >
> > is for Tcl programmers.  It's not just about level of difficulty
> > either. Mapping the low level C APIs directly seems to add a lot of
> > difficulty, because you have to take account of the Tcl environment
> > and all the crazy things that you just don't allow in the C API.
>
> Aha. Well, in that particular case, the [ns_proxy get] is not bad at
> all!
> You get the handle of "something" that you use to do all kinds of
> things.
> Later on, when you're done with it, you [ns_proxy put] itback, allowing
> others to use the same "something"
>
> How would you make this in a "Tcl way"?
>
> After  all, you [open] the channel, then you [puts] to it and then
> [close] it. What is not OK here?
>
> I think I know where are you heading and I can understand the idea
> but this particular example may not be optimal.
>
> I believe you are thinking about something like this:
>
>    ns_mutex lock $lock
>    do bla bla bla
>    ns_mutex unlock $lock
>
> The "optimal" way to do that would really be:
>
>     ns_mutex lock $lock {
>        do bla bla bla
>     }
>
> as it will save you the trouble if "do bla bla" throws error.
> In the latter case you must ugly things like
>
>    ns_mutex lock $lock
>    if {[catch {do bla bla} err]} {
>        ns_mutex unlock $lock
>        error $err
>    }
>    ns_mutex unlock $lock
>
> which is just plain rubbish (because of which we had to invent
> our own do/catch/finally command)...
>


It's not just the cleanup-up after error.  Why does the Tcl programmer
need to get and return resources?  It's kind of awkward.

For example:

    ns_serialize {
        # whatever
    }

is better than the mutex example above because you also don't have to
mess around with the mutex handle and nsv arrays. The Tcl code in the
code block serves as a key into a hash table of locks, created on
demand.  We can let the code block shimmer to an object which contains
both a byte code script and a mutex pointer. It will run as fast as
possible the second time round.

More complicated syntax for a more complicated operation:

    ns_serialize -lock mylock {
        # whatever
    }

Now you could run two different code blocks using the same lock. But
you still don't have to allocate mutex objects, pass around their
names in nsv arrays, and so on.

I don't think the above is dumbed-down. It's a much nicer way to work
with Tcl, and you just wouldn't do this kind of thing from C.


Anyway, back to nsproxy.  Thinking about it further, it looks like the
explicit handles are actually there to allow a particular kind of Tcl
programming, not necessarily because explicit handle management is
needed. i.e.:

    set x [create ...
    $x method ...

Which is perhaps not a bad way to manage the handles if you absolutely
need them. But if you can instead do:

    ns_proxy eval {
        # whatever
    }

which in the case of a default pool and a single evaluation, you can,
it's got to be better than the explicit handle management, right?
This is basically identical to the db module.  People hate manging db
handles, which is why the ACS style db_* API is so popular.


I was also wondering about the ns_proxy send/wait/receive.  Why are
wait and receive separate commands?

Also, does there need to be so many timeouts?  The waittimeout is 100
msec. That means if all else goes well, but the wait time takes 101
msec, the evaluation will not be successful. But looking at the other
defaults, the script evaluator was prepared to wait (gettimeout 5000 +
evaltimeout infinite + sendtimeout 1000 + recvtimeout 1000), or
between six seconds and forever...  Wouldn't a single timeout do?

It would also be great if the timeout error code began with the
NS_TIMEOUT symbol, which if not caught will automatically trigger a
Server Busy error page.

Re: [naviserver-devel] hosting java-script server-side?

[Stephen D. <sd...@gm...>]

On 9/18/06, Vlad Seryakov <vl...@cr...> wrote:
> >
> > People are writing a lot of sucky Tcl for AOLserver.  Here's an
> > example from a couple of days ago:
> >
> >
> > proc startworker {name} {
> >        nsv_set $name mutex [ns_mutex create]
> >         nsv_set $name cond  [ns_cond create]
> >        nsv_set $name queue {}
> >        nsv_set $name tid [ns_thread begindetached [list workermain $name]]
> > }
> >
> > Jumping through hoops with low level mutexes and and variables etc.,
> > you see it all the time.  Why can't we have code that looks like this?
> >
> >     ns_serialize {
> >         # Only one copy of this code runs at a time
> >     }
> >
> >
> > Which brings us back to nsproxy and handle management... :-)
>
>
> You just showed that it is not Tcl sucks but a programmer that writes
> bad code in Tcl without using it properly or not in full. That example
> of your shows that programmer that wrote it has more experience in C and
>   uses Tcl in pure procedure way, which is not that bad, the code works.
>


The programmer above used the API correctly, it's the API that sucks.

To lock a mutex you need to first create it and save the handle. You
need the handle in all the threads which will use the mutex, so you
put it in a global nsv array where all threads can retrieve it.

So at run time, you lock the nsv array, get the name, unlock the nsv
array, lock your mutex, do your stuff, unlock the mutex. There's
double the necessary locking and it's a real pain in the ass.

(It's different in naviserver. you can pass in a string name for the
mutex, and if it doesn't exits, it's automatically created. this
mostly works...)

Re: [naviserver-devel] hosting java-script server-side?

[Zoran V. <zv...@ar...>]

On 27.09.2006, at 20:06, Vlad Seryakov wrote:

> Exactly, because there is no third party projects or repositories of
> software for Naviserver (or Aolserver) except OpenACS we should  
> host and
> keep it. Like examples directory with small useful apps

Interesting idea...
I will see if we can get something from our code base...

Re: [naviserver-devel] hosting java-script server-side?

[Vlad S. <vl...@cr...>]

Exactly, because there is no third party projects or repositories of 
software for Naviserver (or Aolserver) except OpenACS we should host and 
keep it. Like examples directory with small useful apps

Bernd Eidenschink wrote:
>> My suggestion is to include some apps or additional Tcl modules that
>> implements something useful and can be used as an example or even as a
>> foundation of the web site.
> 
> It could be something with very little dependancies, e.g. one simple helper 
> API file sourced by an ADP on demand if not already sourced on server 
> startup, then some simple things like
> - uploading (multiple) images, displaying the results of ns_jpeg|gif|pngsize
> - setting/reading cookies
> - server stats (already there)
> - usage if registered procs/filters
> - ...
> 
> Something like that?
> 
> Bernd.
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] hosting java-script server-side?

[Bernd E. <eid...@we...>]

> My suggestion is to include some apps or additional Tcl modules that
> implements something useful and can be used as an example or even as a
> foundation of the web site.

It could be something with very little dependancies, e.g. one simple helper 
API file sourced by an ADP on demand if not already sourced on server 
startup, then some simple things like
- uploading (multiple) images, displaying the results of ns_jpeg|gif|pngsize
- setting/reading cookies
- server stats (already there)
- usage if registered procs/filters
- ...

Something like that?

Bernd.

Re: [naviserver-devel] hosting java-script server-side?

[Zoran V. <zv...@ar...>]

On 27.09.2006, at 18:04, Vlad Seryakov wrote:

> My suggestion is to include some apps or additional Tcl modules that
> implements something useful and can be used as an example or even as a
> foundation of the web site.

WHAT?

All that we done is of "internal" nature, means we can't make
it public...

So, what would that be?

Re: [naviserver-devel] hosting java-script server-side?

[Vlad S. <vl...@cr...>]

On another note, i've been thinking about Naviserver distribution and 
here are my thoughts:

- for us who is using it already and have our own systems bare-bone 
server is just a platform and we do not need anything more

- for new users who want to try NS and see if it is useful or not 
bare-bone system means they need to learn Tcl and Naviserver API before
they even can see how it works and what it can do.

My suggestion is to include some apps or additional Tcl modules that 
implements something useful and can be used as an example or even as a 
foundation of the web site.

That means, those modules should be easily installed and/or referred 
from docs or main page somehow so it is easy to install them and see.

Is it something worth trying?


Zoran Vasiljevic wrote:
> On 27.09.2006, at 17:00, Vlad Seryakov wrote:
> 
>> Also, when do you plan to release 5.0?
> 
> A Christmas present to all of us?
> Hm? Should be possible...
> 
> 
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] hosting java-script server-side?

[Vlad S. <vl...@cr...>]

I remember i've done it with my OCaml module, it registers handlers for 
.cmo files and handles them in C, not via Tcl. I had to provide API to 
ns_conn and other stuff of course but it is not that complicated as it 
looks.

Zoran Vasiljevic wrote:
> On 27.09.2006, at 17:31, Vlad Seryakov wrote:
> 
>> We are usuing PHP with RoundCube Webmail and Drupal
>> (http://www.drupal.org) system and it never crashed and works very  
>> stable.
> 
> Amazing just how many CMS systems are out there...
> 
>> I thought that Javascript can be used same way, as external module.
>> And Tcl will not be overhead beacuse if my conn thread does not  
>> call any
>> Tcl script Tcl interp is not used, instead i can register Javascript
>> server-side proc that will be called directly through the module.
> 
> It is much more than that: you need to open some of the things
> now only visible to the Tcl level, to Javascript as well in order
> to do something "meaningful". But is not impossible. It is just work...
> 
>   
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] hosting java-script server-side?

[Zoran V. <zv...@ar...>]

On 27.09.2006, at 17:31, Vlad Seryakov wrote:

>
> We are usuing PHP with RoundCube Webmail and Drupal
> (http://www.drupal.org) system and it never crashed and works very  
> stable.

Amazing just how many CMS systems are out there...

> I thought that Javascript can be used same way, as external module.
> And Tcl will not be overhead beacuse if my conn thread does not  
> call any
> Tcl script Tcl interp is not used, instead i can register Javascript
> server-side proc that will be called directly through the module.

It is much more than that: you need to open some of the things
now only visible to the Tcl level, to Javascript as well in order
to do something "meaningful". But is not impossible. It is just work...

  

Re: [naviserver-devel] hosting java-script server-side?

[Vlad S. <vl...@cr...>]

Regarding PHP, now it has useful feature that if module is not 
thread-safe it will not be compiled when threading is enabled.
We are usuing PHP with RoundCube Webmail and Drupal 
(http://www.drupal.org) system and it never crashed and works very stable.

I thought that Javascript can be used same way, as external module.
And Tcl will not be overhead beacuse if my conn thread does not call any 
Tcl script Tcl interp is not used, instead i can register Javascript 
server-side proc that will be called directly through the module.

Bernd Eidenschink wrote:
>> The response(s) so far were almost unisono: NO.
>> Summary (rough):  We like Tcl and we are happy with
>> C/Tcl server as-is now. There is no need for any
>> alternative page-construction language (PHP, JS,
>> whatever) because what we have is good (enough)
>> and its fairly easy to learn Tcl anyway, so why
>> bother with anyhing else?
>>
>> It is good to know where are we when thinking about
>> evolvement of the server code for the next few years.
> 
> PHP is available, thanks to Vlad. We already use the latest version on a 
> production system with NaviServer. We needed to integrate some PHP code from 
> a customer and are happy to do it with NaviServer. You can evaluate code in 
> both directions: PHP from within TCL/ADP pages and TCL from within PHP.
> That's nice. (**)
> 
> I did not answer to your original mail because (a) I don't know if it would be 
> the best strategical solution in your specific situation to let new 
> candidates introduce an additional language you (and your colleagues) have to 
> take care of and build knowledge, rather them learning the language the 
> company is familiar with; and (b) if Server-Side Javascript is or becomes 
> a 'Hype', worth enough to integrate.
> 
> That said, I never understood why every effort to introduce a new language to 
> the webserver over the last years crashes everytime on the 
> TCL-is-so-great-and-Hypes-come-and-go-firewall. I doubt one would need a 
> level of integration and status TCL has with the server to see interesting 
> combinations of the new possibilites and to attract more people to the 
> server. What's so bad (besides working effort) to try, fail or succeed?
> 
> Bernd.
> 
> 
> (**) Of course, the natural brother and sister are a forking Apache and PHP, 
> so you don't run into threading issues some binary PHP extension might have.
> On the production system we have modest requirements for the PHP interface we 
> needed to integrate.
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] hosting java-script server-side?

[Zoran V. <zv...@ar...>]

On 27.09.2006, at 17:00, Vlad Seryakov wrote:

>
> Also, when do you plan to release 5.0?

A Christmas present to all of us?
Hm? Should be possible...

Re: [naviserver-devel] hosting java-script server-side?

[Zoran V. <zv...@ar...>]

On 27.09.2006, at 17:19, Bernd Eidenschink wrote:

> That said, I never understood why every effort to introduce a new  
> language to
> the webserver over the last years crashes everytime on the
> TCL-is-so-great-and-Hypes-come-and-go-firewall.

Weird, isn't it?

> What's so bad (besides working effort) to try, fail or succeed?

Time needed to get this done, which one could (better?) invest in
polishing the things we have already.
I believe this is the main reason (and, this is a good reason indeed).

At the moment this Tcl/C marriage begins to itch us too much I will
have to do something about (an alternative).

Re: [naviserver-devel] hosting java-script server-side?

[Bernd E. <eid...@we...>]

> The response(s) so far were almost unisono: NO.
> Summary (rough):  We like Tcl and we are happy with
> C/Tcl server as-is now. There is no need for any
> alternative page-construction language (PHP, JS,
> whatever) because what we have is good (enough)
> and its fairly easy to learn Tcl anyway, so why
> bother with anyhing else?
>
> It is good to know where are we when thinking about
> evolvement of the server code for the next few years.

PHP is available, thanks to Vlad. We already use the latest version on a 
production system with NaviServer. We needed to integrate some PHP code from 
a customer and are happy to do it with NaviServer. You can evaluate code in 
both directions: PHP from within TCL/ADP pages and TCL from within PHP.
That's nice. (**)

I did not answer to your original mail because (a) I don't know if it would be 
the best strategical solution in your specific situation to let new 
candidates introduce an additional language you (and your colleagues) have to 
take care of and build knowledge, rather them learning the language the 
company is familiar with; and (b) if Server-Side Javascript is or becomes 
a 'Hype', worth enough to integrate.

That said, I never understood why every effort to introduce a new language to 
the webserver over the last years crashes everytime on the 
TCL-is-so-great-and-Hypes-come-and-go-firewall. I doubt one would need a 
level of integration and status TCL has with the server to see interesting 
combinations of the new possibilites and to attract more people to the 
server. What's so bad (besides working effort) to try, fail or succeed?

Bernd.


(**) Of course, the natural brother and sister are a forking Apache and PHP, 
so you don't run into threading issues some binary PHP extension might have.
On the production system we have modest requirements for the PHP interface we 
needed to integrate.