Menu
▾
▴
naviserver-devel
|
From: Vlad S. <vl...@cr...> - 2006-09-27 16:05:07
|
On another note, i've been thinking about Naviserver distribution and here are my thoughts: - for us who is using it already and have our own systems bare-bone server is just a platform and we do not need anything more - for new users who want to try NS and see if it is useful or not bare-bone system means they need to learn Tcl and Naviserver API before they even can see how it works and what it can do. My suggestion is to include some apps or additional Tcl modules that implements something useful and can be used as an example or even as a foundation of the web site. That means, those modules should be easily installed and/or referred from docs or main page somehow so it is easy to install them and see. Is it something worth trying? Zoran Vasiljevic wrote: > On 27.09.2006, at 17:00, Vlad Seryakov wrote: > >> Also, when do you plan to release 5.0? > > A Christmas present to all of us? > Hm? Should be possible... > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2006-09-27 16:17:32
|
On 27.09.2006, at 18:04, Vlad Seryakov wrote: > My suggestion is to include some apps or additional Tcl modules that > implements something useful and can be used as an example or even as a > foundation of the web site. WHAT? All that we done is of "internal" nature, means we can't make it public... So, what would that be? |
|
From: Bernd E. <eid...@we...> - 2006-09-27 16:47:44
|
> My suggestion is to include some apps or additional Tcl modules that > implements something useful and can be used as an example or even as a > foundation of the web site. It could be something with very little dependancies, e.g. one simple helper API file sourced by an ADP on demand if not already sourced on server startup, then some simple things like - uploading (multiple) images, displaying the results of ns_jpeg|gif|pngsize - setting/reading cookies - server stats (already there) - usage if registered procs/filters - ... Something like that? Bernd. |
|
From: Vlad S. <vl...@cr...> - 2006-09-27 18:06:50
|
Exactly, because there is no third party projects or repositories of software for Naviserver (or Aolserver) except OpenACS we should host and keep it. Like examples directory with small useful apps Bernd Eidenschink wrote: >> My suggestion is to include some apps or additional Tcl modules that >> implements something useful and can be used as an example or even as a >> foundation of the web site. > > It could be something with very little dependancies, e.g. one simple helper > API file sourced by an ADP on demand if not already sourced on server > startup, then some simple things like > - uploading (multiple) images, displaying the results of ns_jpeg|gif|pngsize > - setting/reading cookies > - server stats (already there) > - usage if registered procs/filters > - ... > > Something like that? > > Bernd. > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2006-09-27 18:14:51
|
On 27.09.2006, at 20:06, Vlad Seryakov wrote: > Exactly, because there is no third party projects or repositories of > software for Naviserver (or Aolserver) except OpenACS we should > host and > keep it. Like examples directory with small useful apps Interesting idea... I will see if we can get something from our code base... |
|
From: Stephen D. <sd...@gm...> - 2006-10-04 03:29:36
|
On 9/18/06, Vlad Seryakov <vl...@cr...> wrote:
> >
> > People are writing a lot of sucky Tcl for AOLserver. Here's an
> > example from a couple of days ago:
> >
> >
> > proc startworker {name} {
> > nsv_set $name mutex [ns_mutex create]
> > nsv_set $name cond [ns_cond create]
> > nsv_set $name queue {}
> > nsv_set $name tid [ns_thread begindetached [list workermain $name]]
> > }
> >
> > Jumping through hoops with low level mutexes and and variables etc.,
> > you see it all the time. Why can't we have code that looks like this?
> >
> > ns_serialize {
> > # Only one copy of this code runs at a time
> > }
> >
> >
> > Which brings us back to nsproxy and handle management... :-)
>
>
> You just showed that it is not Tcl sucks but a programmer that writes
> bad code in Tcl without using it properly or not in full. That example
> of your shows that programmer that wrote it has more experience in C and
> uses Tcl in pure procedure way, which is not that bad, the code works.
>
The programmer above used the API correctly, it's the API that sucks.
To lock a mutex you need to first create it and save the handle. You
need the handle in all the threads which will use the mutex, so you
put it in a global nsv array where all threads can retrieve it.
So at run time, you lock the nsv array, get the name, unlock the nsv
array, lock your mutex, do your stuff, unlock the mutex. There's
double the necessary locking and it's a real pain in the ass.
(It's different in naviserver. you can pass in a string name for the
mutex, and if it doesn't exits, it's automatically created. this
mostly works...)
|
|
From: Zoran V. <zv...@ar...> - 2006-10-04 07:42:51
|
On 03.10.2006, at 00:13, Stephen Deasey wrote: > > (It's different in naviserver. you can pass in a string name for the > mutex, and if it doesn't exits, it's automatically created. this > mostly works...) Interesting... (I did not know that, which stresses the importance of documentation again...) But... I believe there is a "hole" there. Look: lexxsrv:nscp 7> ns_mutex lock themutex lexxsrv:nscp 8> ns_cond wait thecond themutex invalid address "themutex" lexxsrv:nscp 9> I could however: lexxsrv:nscp 9> ns_cond broadcast thecond lexxsrv:nscp 10> So it is the [ns_cond wait] that does not play exactly by the rules. Should I look into that or would you? I'm asking this because I'd like to use this feature and drop our Tcl wrappers which do basically the same thing... |
|
From: Zoran V. <zv...@ar...> - 2006-10-04 09:24:09
|
On 04.10.2006, at 09:42, Zoran Vasiljevic wrote: (answering my own questions) > > But... I believe there is a "hole" there. Look: > > lexxsrv:nscp 7> ns_mutex lock themutex > lexxsrv:nscp 8> ns_cond wait thecond themutex > invalid address "themutex" > lexxsrv:nscp 9> > > I could however: > > lexxsrv:nscp 9> ns_cond broadcast thecond > lexxsrv:nscp 10> There is a slight "glicht" as this one works fine: lexxsrv:nscp 1> set m mux mux lexxsrv:nscp 2> ns_mutex lock $m lexxsrv:nscp 3> ns_cond wait thecond $m 100 This is because you expect the mutex argument of the ns_cond to be of the Address type which is not always true. I think I see where the problem is and will fix that now. |
|
From: Zoran V. <zv...@ar...> - 2006-10-04 10:27:28
|
On 04.10.2006, at 11:24, Zoran Vasiljevic wrote:
> I think I see where the problem is and will fix that now.
I've fixed that but this opens another discussion.
Basically you wanted to avoid using handles. Then you said,
OK, I can use string names and turn them to handles on as-needed
basis by looking them up in an internal table. Everytime this
lookup happens I need to global lock the table then lookup the
(whatever)handle using it string name.
To avoid that global lock, you change/update the object holding the
string rep with the calculated handle and you count on the fact
that next usage of the same object will avoid having to to the
(costly) lookup.
Did you think about such uses:
ns_mutex lock themutex
while {} {
ns_cond wait thecond themutex
}
ns_mutex unlock themutex
because here EVERY time you will have to lock the global
lock to get the handle of the "thing", as the "trick" to
save it in the object will not work! The "themutex" or
"thecond" could/will always be different objects if used
this way, so there is no gain.
If one however does:
set m themutex
set t thecond
ns_mutex lock $m
while {} {
ns_cond wait $c $m
}
ns_mutex unlock $m
it WILL work. But how are you going to convey this information
to the Tcl programmer??? He's implicitly serializing his app
at the place he does not expect (lookup of the internal hash
table).
I believe this is not very optimal and requires some other
"means" of hiding this information (the handle), potentially
using complete different paradigms of programming (more suitable
to Tcl way of doing things - if there is such a "way" at all).
|
|
From: Stephen D. <sd...@gm...> - 2006-10-04 14:00:17
|
On 10/4/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> On 04.10.2006, at 11:24, Zoran Vasiljevic wrote:
>
> > I think I see where the problem is and will fix that now.
>
> I've fixed that but this opens another discussion.
>
> Basically you wanted to avoid using handles. Then you said,
> OK, I can use string names and turn them to handles on as-needed
> basis by looking them up in an internal table. Everytime this
> lookup happens I need to global lock the table then lookup the
> (whatever)handle using it string name.
>
> To avoid that global lock, you change/update the object holding the
> string rep with the calculated handle and you count on the fact
> that next usage of the same object will avoid having to to the
> (costly) lookup.
Right.
And the reason the handle name doesn't have to be looked up again
because we cheat -- the mutexes associated with the handle names live
forever. You can't delete or reassign them. So if you ever find a
pointer to a mutex in one of these handle names, it's valid.
> Did you think about such uses:
>
> ns_mutex lock themutex
> while {} {
> ns_cond wait thecond themutex
> }
> ns_mutex unlock themutex
>
> because here EVERY time you will have to lock the global
> lock to get the handle of the "thing", as the "trick" to
> save it in the object will not work! The "themutex" or
> "thecond" could/will always be different objects if used
> this way, so there is no gain.
Hmm... I think this does work. Here's how I read it:
There are 3 string objects with the value "themutex" in the above
code. The first time the code runs, there will be 3 global look ups
in the hash table of mutex names to convert the name "themutex" to a
pointer to an actual mutex.
On the first line, in the call to "ns_mutex lock themutex", the last
argument is just a Tcl string object so we have to look up the name in
the mutex hash table. The name doesn't exist, so a new mutex is
allocated, put in the hash table, and a pointer to it is stored in the
Tcl object. The Tcl object is marked as an Address obj type (it's
string rep is not invalidated, we just change the type).
In the call to ns_cond and the last call to ns_mutex, "themutex" is
also just a Tcl string object. So again the mutex hash table has to be
consulted. This time the mutex exists however, so we simply change the
string object to an Address type and store a pointer to the existing
mutex in it.
There are 3 individual Tcl objects, each with the string
representation "themutex". Each now points to exactly the same mutex
in memory.
The second time the code is run, the 3 "themutex" objects retain their
type (Address), so we don't have to look up the name in the mutex hash
table as a pointer to the mutex is stored in the Tcl object itself. We
get access to the mutex with a simple pointer dereference.
So, the premise is that the code will be executed more than once. The
first time will be 'slow', but after that there is no locking at all
to convert the name to an actual mutex.
And it works specifically in the above case because mutex objects are
seperate from Tcl objects which point to them.
> If one however does:
>
> set m themutex
> set t thecond
>
> ns_mutex lock $m
> while {} {
> ns_cond wait $c $m
> }
> ns_mutex unlock $m
>
> it WILL work. But how are you going to convey this information
> to the Tcl programmer??? He's implicitly serializing his app
> at the place he does not expect (lookup of the internal hash
> table).
Actually, the above might NOT work... :-)
In the above code, $m is a single Tcl string object with the value
"themutex" and, after the first call to "ns_mutex lock $m", we convert
it to a an Address type and point it at a mutex.
There is a single Tcl object pointing to a single mutex, where as in
the original code there were 3 Tcl objects pointing to a single mutex.
There will be a single look up in the mutex hash table, where as in
the original code there will be 3. But only on the first time the code
is run.
The while loop will be 'efficient' because the underlying mutex object
is just a pointer dereference away.
However, what if the whole code block was run in a loop?
At the bottom of the loop, $m is an Address type pointing to our
mutex. At the top of the loop $m is reset:
set m themutex
The value of the literal "themutex" Tcl string object is copied and
used as the new value of $m. Because the string representation of $m
has changed (even if the actual text is literaly the same, there is no
checking for this), the internal representation is invalidated. i.e.
it is no longer an Address type.
So at the top of the loop we again have to convert m to an Address
type be looking up the name in the mutex hash table. The mutex will
exist in the hash table, but we need to get a pointer to it.
Any operation which looses the type information will break the caching
of the mutex pointer in the object. So, passing the name through an
nsv array or cache will strip the type info.
But I'm having a hard time coming up with real world scenarios where
that might happen...
|
|
From: Zoran V. <zv...@ar...> - 2006-10-04 15:03:19
|
On 04.10.2006, at 15:52, Stephen Deasey wrote:
>
> Hmm... I think this does work. Here's how I read it:
First time arround, there will be 3 lookups.
The next round should be simple deref.
Allright this might compute. If you have this in a proc
body the "themutex" will be a literal objects and will/should
last "longer".
I did it under the debugger in the nscp session and every
time I get another object as "themutex". This is most probably
because the code does not compile into bytecodes...
OK. I can accept that. Still, you put this into frequently
called procedure and there you go (literals are freed AFAIK,
when procedure scope is exited): again you have global locking.
>
>> If one however does:
>>
>> set m themutex
>> set t thecond
>>
>> ns_mutex lock $m
>> while {} {
>> ns_cond wait $c $m
>> }
>> ns_mutex unlock $m
>>
>> it WILL work. But how are you going to convey this information
>> to the Tcl programmer??? He's implicitly serializing his app
>> at the place he does not expect (lookup of the internal hash
>> table).
>
>
> Actually, the above might NOT work... :-)
The first two init lines I havent counted for. It is just the
remaining and in fact most specifically the
ns_cond wait
that was the problem as it may awake and sleep again.
>
> But I'm having a hard time coming up with real world scenarios where
> that might happen...
This is OK. In the compiled code, things start to look somehow
different.
But also when you use the "themutex" to put it in the nsv array
you loose the caching effect of the object. Unlike with handles
where you get the "real thing" immediately and need not global
lock.
Keeping the handles "away" from the user, means you need to manage
handles yourself because the underlying C code needs handles/pointers.
Every time you do that, you need to lock. In this case the global lock.
So, nothing is for free. By allowing the tcl programmer some freedom,
you charge him with some performance/concurrency.
The best illustration:
lexxsrv:nscp 7> ns_mutex unlock themutex
Connection closed by foreign host.
Here no check is done: you get a core. But if the themutex was
locked before, all would be fine.
In the Tcl threading extension I check mutex for a locked state
and TCL_ERROR if lock was attempted on an already locked mutex
OR an attempt was made to unlock never-locked mutex.
This requires lots of low-level plumbing BUT it gives the Tcl
coder maximum comfort.
Neither way is "right" or "wrong". So you can't generally
"avoid" using handles as this will lead you to situation
where you must sacrify some speed/concurrency for getting
the comfortable API.
|
|
From: Stephen D. <sd...@gm...> - 2006-10-04 16:44:35
|
On 10/4/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> On 04.10.2006, at 15:52, Stephen Deasey wrote:
> >
> > Hmm... I think this does work. Here's how I read it:
>
> First time arround, there will be 3 lookups.
> The next round should be simple deref.
> Allright this might compute. If you have this in a proc
> body the "themutex" will be a literal objects and will/should
> last "longer".
Not just proc bodies, but also sourced files and looping commands. You
have to go out of your way to evaluate a piece of Tcl code and not
have it compiled to byte code. Like use the nscp control port... :-)
> I did it under the debugger in the nscp session and every
> time I get another object as "themutex". This is most probably
> because the code does not compile into bytecodes...
>
> OK. I can accept that. Still, you put this into frequently
> called procedure and there you go (literals are freed AFAIK,
> when procedure scope is exited): again you have global locking.
I'm not sure what you're saying here. What is freed, and when?
> >
> >> If one however does:
> >>
> >> set m themutex
> >> set t thecond
> >>
> >> ns_mutex lock $m
> >> while {} {
> >> ns_cond wait $c $m
> >> }
> >> ns_mutex unlock $m
> >>
> >> it WILL work. But how are you going to convey this information
> >> to the Tcl programmer??? He's implicitly serializing his app
> >> at the place he does not expect (lookup of the internal hash
> >> table).
> >
> >
> > Actually, the above might NOT work... :-)
>
> The first two init lines I havent counted for. It is just the
> remaining and in fact most specifically the
> ns_cond wait
> that was the problem as it may awake and sleep again.
>
> >
> > But I'm having a hard time coming up with real world scenarios where
> > that might happen...
>
> This is OK. In the compiled code, things start to look somehow
> different.
> But also when you use the "themutex" to put it in the nsv array
> you loose the caching effect of the object.
Right. But why would you ever do this? This was a corner case example
I gave to show that it *could* happen, if you tried real hard and
looked at it funny, but you wouldn't actually do this, right?
> Unlike with handles
> where you get the "real thing" immediately and need not global
> lock.
In the case of the thread objects this is the case, but they are
special (or weird). The internal rep of a thread object (mutex, cond
var etc.) is a serialised C pointer. Given the string you can
re-create the pointer (and if you try hard enough you can create an
invalid pointer and crash the server, hence weird).
But for handles in general, such as nsproxy, this is not the case.
Remember, if you can refer to thread objects by name then you don't
*need* to put the name in an nsv array, for example. And if you do,
ns_mutex create does in fact still return a handle.
> Keeping the handles "away" from the user, means you need to manage
> handles yourself because the underlying C code needs handles/pointers.
> Every time you do that, you need to lock. In this case the global lock.
But you don't. That's the point. If this isn't the case, then I've
done something wrong and the code needs to be reverted.
> So, nothing is for free. By allowing the tcl programmer some freedom,
> you charge him with some performance/concurrency.
>
> The best illustration:
>
> lexxsrv:nscp 7> ns_mutex unlock themutex
> Connection closed by foreign host.
>
> Here no check is done: you get a core. But if the themutex was
> locked before, all would be fine.
I guess this is a good example of trade offs -- performance over
safety. But it's not a good example of the differences between names
and handles, because you can do exactly the same thing:
% set m [ns_mutex create]
t0xfd3dd9 a0xa654800 ns:mutex
% ns_mutex unlock $m
The trade off here is balanced, you can have one thing (safety) or you
can have the other (performance). Your choice. For both handles and
names.
On the other hand, the choice between names and handles is not
balanced. Assuming names actually work (you've sort of said they don't
above), they are equally as fast. So the choice is fast and easy or
fast and hard.
Fast and easy, please!
There's also the question of whether names allow the same kind of
expressive power as handles, in the nsproxy case for example. I think
they are, but that's another thread.
> In the Tcl threading extension I check mutex for a locked state
> and TCL_ERROR if lock was attempted on an already locked mutex
> OR an attempt was made to unlock never-locked mutex.
> This requires lots of low-level plumbing BUT it gives the Tcl
> coder maximum comfort.
>
> Neither way is "right" or "wrong". So you can't generally
> "avoid" using handles as this will lead you to situation
> where you must sacrify some speed/concurrency for getting
> the comfortable API.
|
|
From: Zoran V. <zv...@ar...> - 2006-10-04 17:13:35
|
On 04.10.2006, at 18:44, Stephen Deasey wrote:
>>
>> OK. I can accept that. Still, you put this into frequently
>> called procedure and there you go (literals are freed AFAIK,
>> when procedure scope is exited): again you have global locking.
>
>
> I'm not sure what you're saying here. What is freed, and when?
>
The literal object "themutex" is gone after procedure exits
AFAIK. This MIGHT need to be double-checked though as I'm
not 100% sure if the literals get garbage collected at that
point OR at interp teardown.
>
>
> Right. But why would you ever do this? This was a corner case example
> I gave to show that it *could* happen, if you tried real hard and
> looked at it funny, but you wouldn't actually do this, right?
Correct.
>
>
>> Unlike with handles
>> where you get the "real thing" immediately and need not global
>> lock.
>
>
> In the case of the thread objects this is the case, but they are
> special (or weird). The internal rep of a thread object (mutex, cond
> var etc.) is a serialised C pointer. Given the string you can
> re-create the pointer (and if you try hard enough you can create an
> invalid pointer and crash the server, hence weird).
>
> But for handles in general, such as nsproxy, this is not the case.
>
> Remember, if you can refer to thread objects by name then you don't
> *need* to put the name in an nsv array, for example. And if you do,
> ns_mutex create does in fact still return a handle.
BUT: to be able to refer them by name I need a lookup table.
And if this is to be thread-wide I need to lock that table globally.
>
>
>> Keeping the handles "away" from the user, means you need to manage
>> handles yourself because the underlying C code needs handles/
>> pointers.
>> Every time you do that, you need to lock. In this case the global
>> lock.
>
>
> But you don't. That's the point. If this isn't the case, then I've
> done something wrong and the code needs to be reverted.
You don't?? You do!
ns_mutex create
returns handle. But I can say:
ns_mutex lock junklock
and it will *hide* a newly created mutex and tag it with
a "junkmutex" in a thread-wide table. There you go.
The user never "see's" that real mutex handle. He knows
only "junklock". With some object tricks you "remember"
the real handle so to avoid lookup in a locked table.
But when this shimmers away, you're out of bussiness.
OTOH, when I say
set mutexhandle [ns_mutex create]
I can do whatever I want with mutexhandle, it will always point to
that damn mutex. I need not do some other lookup in addition.
I can put it into nsv, transfer it over network back to myself...
Thats what I mean by removing the handle from users. If you
do that you need to do more, introduce more locks etc.
Nothing is for free...
>
>
>> So, nothing is for free. By allowing the tcl programmer some freedom,
>> you charge him with some performance/concurrency.
>>
>> The best illustration:
>>
>> lexxsrv:nscp 7> ns_mutex unlock themutex
>> Connection closed by foreign host.
>>
>> Here no check is done: you get a core. But if the themutex was
>> locked before, all would be fine.
>
>
> I guess this is a good example of trade offs -- performance over
> safety. But it's not a good example of the differences between names
> and handles, because you can do exactly the same thing:
>
> % set m [ns_mutex create]
> t0xfd3dd9 a0xa654800 ns:mutex
> % ns_mutex unlock $m
>
> The trade off here is balanced, you can have one thing (safety) or you
> can have the other (performance). Your choice. For both handles and
> names.
>
> On the other hand, the choice between names and handles is not
> balanced. Assuming names actually work (you've sort of said they don't
> above), they are equally as fast. So the choice is fast and easy or
> fast and hard.
>
> Fast and easy, please!
>
> There's also the question of whether names allow the same kind of
> expressive power as handles, in the nsproxy case for example. I think
> they are, but that's another thread.
Ah... the handle is an address of the "thing". The name is a tag.
You can tag the same address with several names but only the address
matters, as this is now the lower bits are working. So, when you
have a name of a "thing" you need a table to associate it with
an address in order to find/use it. Right?
If yes, then you need to protect accesses to that table when the
name you use refers to some thread-wide thing. Right?
When you use handles DIRECTLY, the implementation is normally
responsible to make a as-fast-as-possble transformation
to get the thing "address":
t0xfd3dd9 a0xa654800 ns:mutex
that 0x nubmers is actually the hidden address of the mutex.
What I say is: everytime you make something "nicer" or "easier"
to use, you need to do more work, potentially reducing performance
or concurrency or scalability. This case of tagging thread sync
primitives with names and using the names instead of their handles
is an example. I do not say that what you've done is bad. Actually
it is the best one can do. BUT: it costs concurrency and people
must know that it might be unpredictable and dependent on some
hidden implicit object behaviour.
Bottom line: I do not object. I find it good in any way. It is just
important to know that one might come into concurrency issues at that
point. And it is not always 100% predictable unless you read thru the
lines.
|
|
From: Stephen D. <sd...@gm...> - 2006-10-04 18:01:00
|
On 10/4/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> On 04.10.2006, at 18:44, Stephen Deasey wrote:
>
> >>
> >> OK. I can accept that. Still, you put this into frequently
> >> called procedure and there you go (literals are freed AFAIK,
> >> when procedure scope is exited): again you have global locking.
> >
> >
> > I'm not sure what you're saying here. What is freed, and when?
> >
>
> The literal object "themutex" is gone after procedure exits
> AFAIK. This MIGHT need to be double-checked though as I'm
> not 100% sure if the literals get garbage collected at that
> point OR at interp teardown.
If that were true then my scheme would be useless. But I don't think
it is. I added a log line to GetArgs in nsd/tclthread.c:
if (Ns_TclGetOpaqueFromObj(objv[2], type, &addr) != TCL_OK
&& Ns_TclGetAddrFromObj(interp, objv[2], type, &addr) != TCL_OK) {
Ns_Log(Warning, "tclthread.c:GetArgs: objv[2] not an
address object. Looking up in hash table...");
Tcl_ResetResult(interp);
Ns_MasterLock();
...
i.e. if the 'name' of the thread primitive given to the command does
not already contain a pointer to the underlying object, and it can not
be converted to one by deserialising a pointer address, then take the
global lock and look the name up in the hash table. Log this case.
% proc lockunlock args {ns_mutex lock m; ns_mutex unlock m}
% lockunlock
[04/Oct/2006:18:39:25][8985.3086293920][-thread-1208673376-] Warning:
tclthread.c:GetArgs: objv[2] not an address object. Looking up
in hash table...
% lockunlock
% lockunlock
The first time the proc is run it is compiled and the name is indeed
looked up in the hash of mutex names. And that is of course 'slow'
because there's a lock around the hash table.
But that is the *only* time this happens. On the second and third
attempt, no locking or look up!
What's interesting from the above is that there is only a single log
line, but there are two literal "m" objects. Apparently Tcl is doing
some optimising behind the scenes...
> > Right. But why would you ever do this? This was a corner case example
> > I gave to show that it *could* happen, if you tried real hard and
> > looked at it funny, but you wouldn't actually do this, right?
>
> Correct.
>
> >
> >
> >> Unlike with handles
> >> where you get the "real thing" immediately and need not global
> >> lock.
> >
> >
> > In the case of the thread objects this is the case, but they are
> > special (or weird). The internal rep of a thread object (mutex, cond
> > var etc.) is a serialised C pointer. Given the string you can
> > re-create the pointer (and if you try hard enough you can create an
> > invalid pointer and crash the server, hence weird).
> >
> > But for handles in general, such as nsproxy, this is not the case.
> >
> > Remember, if you can refer to thread objects by name then you don't
> > *need* to put the name in an nsv array, for example. And if you do,
> > ns_mutex create does in fact still return a handle.
>
> BUT: to be able to refer them by name I need a lookup table.
> And if this is to be thread-wide I need to lock that table globally.
But only ONCE! The look up is cached.
> >> Keeping the handles "away" from the user, means you need to manage
> >> handles yourself because the underlying C code needs handles/
> >> pointers.
> >> Every time you do that, you need to lock. In this case the global
> >> lock.
> >
> >
> > But you don't. That's the point. If this isn't the case, then I've
> > done something wrong and the code needs to be reverted.
>
> You don't?? You do!
>
> ns_mutex create
>
> returns handle. But I can say:
>
> ns_mutex lock junklock
>
> and it will *hide* a newly created mutex and tag it with
> a "junkmutex" in a thread-wide table. There you go.
> The user never "see's" that real mutex handle. He knows
> only "junklock". With some object tricks you "remember"
> the real handle so to avoid lookup in a locked table.
> But when this shimmers away, you're out of bussiness.
Right. But it won't shimmer away, because it is a literal name and you
have no need to manipulate it in any way that would cause it to
shimmer, such as putting it in an nsv array.
Now, I'm not sure what you're getting at with the "junklock" business.
If you mean the user could have a typo in their code and an extra lock
will be created behind their back, well, that's the nature of Tcl.
Same goes for variables, right?
Although you could define ns_mutex create to take a name and force
people to use this in their initialisation code, and then in calls to
lock and unlock you wouldn't create on demand, you'd just do the look
up (first time only!), and throw an error if the name doesn't exist.
But maybe I'm missing your point here...
> OTOH, when I say
>
> set mutexhandle [ns_mutex create]
>
> I can do whatever I want with mutexhandle, it will always point to
> that damn mutex. I need not do some other lookup in addition.
> I can put it into nsv, transfer it over network back to myself...
It's not that you *can* put it into an nsv array, it's that you *have*
to, because how else are you going to communicate the serialised
pointer which is a mutex handle?
And you *have* to do it because what good is a mutex without more than
one thread?
And if you have more than one thread you have more than one interp,
each with it's own global namesapce. So how do you refer to a single
mutex from within each interp?
> Thats what I mean by removing the handle from users. If you
> do that you need to do more, introduce more locks etc.
> Nothing is for free...
It is in fact more or less free. Compiling the Tcl source to byte code
incurs a lock around the mutex hash table. It is a compile time
expense. At run time there is no locking.
nsv arrays *also* incur locking. Plus you have the effort of having to
cache the mutex in a thread local variable, or else you incur the nsv
locking cost each time.
So I think this is in fact a case of a free lunch!
As far as I can see, the only thing that will make this not true, is
some real-world, non-contrived case where the mutex name has to
shimmer.
|
|
From: Stephen D. <sd...@gm...> - 2006-10-04 18:26:46
|
On 10/4/06, Stephen Deasey <sd...@gm...> wrote:
> On 10/4/06, Zoran Vasiljevic <zv...@ar...> wrote:
> >
> > On 04.10.2006, at 18:44, Stephen Deasey wrote:
> >
> > >>
> > >> OK. I can accept that. Still, you put this into frequently
> > >> called procedure and there you go (literals are freed AFAIK,
> > >> when procedure scope is exited): again you have global locking.
> > >
> > >
> > > I'm not sure what you're saying here. What is freed, and when?
> > >
> >
> > The literal object "themutex" is gone after procedure exits
> > AFAIK. This MIGHT need to be double-checked though as I'm
> > not 100% sure if the literals get garbage collected at that
> > point OR at interp teardown.
>
>
> If that were true then my scheme would be useless. But I don't think
> it is. I added a log line to GetArgs in nsd/tclthread.c:
>
> if (Ns_TclGetOpaqueFromObj(objv[2], type, &addr) != TCL_OK
> && Ns_TclGetAddrFromObj(interp, objv[2], type, &addr) != TCL_OK) {
>
> Ns_Log(Warning, "tclthread.c:GetArgs: objv[2] not an
> address object. Looking up in hash table...");
>
> Tcl_ResetResult(interp);
> Ns_MasterLock();
> ...
>
>
> i.e. if the 'name' of the thread primitive given to the command does
> not already contain a pointer to the underlying object, and it can not
> be converted to one by deserialising a pointer address, then take the
> global lock and look the name up in the hash table. Log this case.
>
>
> % proc lockunlock args {ns_mutex lock m; ns_mutex unlock m}
> % lockunlock
> [04/Oct/2006:18:39:25][8985.3086293920][-thread-1208673376-] Warning:
> tclthread.c:GetArgs: objv[2] not an address object. Looking up
> in hash table...
> % lockunlock
> % lockunlock
>
> The first time the proc is run it is compiled and the name is indeed
> looked up in the hash of mutex names. And that is of course 'slow'
> because there's a lock around the hash table.
>
> But that is the *only* time this happens. On the second and third
> attempt, no locking or look up!
>
> What's interesting from the above is that there is only a single log
> line, but there are two literal "m" objects. Apparently Tcl is doing
> some optimising behind the scenes...
>
Hmm.. what's most interesting about the above is, if Tcl notices two
literal "m" objects in the above code and coalesces them into one,
where does it put them? If it puts them in a global cache, not a
proc-local one, then we have problems:
% proc lockunlock args {ns_mutex lock x; ns_mutex unlock x}
% proc broadcast args {ns_cond broadcast x}
% broadcast
[04/Oct/2006:19:15:41][9066.3086289824][-thread-1208677472-] Warning:
tclthread.c:GetArgs: objv[2] not an address object. Looking up
in hash table...
% broadcast
% lockunlock
[04/Oct/2006:19:15:52][9066.3086289824][-thread-1208677472-] Warning:
tclthread.c:GetArgs: objv[2] not an address object. Looking up
in hash table...
% lockunlock
% broadcast
[04/Oct/2006:19:15:59][9066.3086289824][-thread-1208677472-] Warning:
tclthread.c:GetArgs: objv[2] not an address object. Looking up
in hash table...
Er, there's the object shimmering...
So to work efficiently the name you give to the mutex must be unique.
As is the case with naming procs etc. It will still work correctly
with a non-unique name, but it will incur a locking overhead should
your name clash with some other literal, and should that other literal
be used in a way which would cause shimering. Damn.
So maybe we can force unique names by requiring mutexes to be
explicitly created, as mentioned before. i.e.:
init.tcl:
ns_mutex create foo_lock
foo.tcl
proc foo_do args {
ns_mutex lock foo_lock
...
}
Or not. I don't know...
|
|
From: Zoran V. <zv...@ar...> - 2006-10-04 19:03:11
|
On 04.10.2006, at 20:26, Stephen Deasey wrote: > > Or not. I don't know... Not. It does not have anything to do with unique names. So far I recall, the procedure context stores list of literals used in the proc. When the proc scope exits the literal table is cleaned and all literal objects are decr-ref-counted (ulimately put back on the per-thread list of free objects). I'm not SURE that this is so, I recall seeing this at the time I was chasing a very weird bug resulting in wrong usage of Tcl objects. A peek into Tcl sources will surely reval that, but even w/o that, my little example illustrates exactly such behaviour. We might need to understand how come you get different behaviour but this way or another, the objects get lost. BUT... this way or another, what you did is perfectly OK as there is no other way one can do that. It is just that people need to be aware of such possiblity and operate either on the handle of the mutex or on its name. In our code I have done the same on the Tcl level. I invented the "getlock" call which returned the mutex handle, cacheing it in between in the nsv array. You could improve the code by inventing similar bucket based locking as done in nsv in order to avoid using that one single global lock. |
|
From: Zoran V. <zv...@ar...> - 2006-10-04 18:34:59
|
On 04.10.2006, at 20:00, Stephen Deasey wrote:
> On 10/4/06, Zoran Vasiljevic <zv...@ar...> wrote:
>>
>> On 04.10.2006, at 18:44, Stephen Deasey wrote:
>>
>>>>
>>>> OK. I can accept that. Still, you put this into frequently
>>>> called procedure and there you go (literals are freed AFAIK,
>>>> when procedure scope is exited): again you have global locking.
>>>
>>>
>>> I'm not sure what you're saying here. What is freed, and when?
>>>
>>
>> The literal object "themutex" is gone after procedure exits
>> AFAIK. This MIGHT need to be double-checked though as I'm
>> not 100% sure if the literals get garbage collected at that
>> point OR at interp teardown.
>
>
> If that were true then my scheme would be useless. But I don't think
> it is. I added a log line to GetArgs in nsd/tclthread.c:
>
> if (Ns_TclGetOpaqueFromObj(objv[2], type, &addr) != TCL_OK
> && Ns_TclGetAddrFromObj(interp, objv[2], type, &addr) !
> = TCL_OK) {
>
> Ns_Log(Warning, "tclthread.c:GetArgs: objv[2] not an
> address object. Looking up in hash table...");
>
> Tcl_ResetResult(interp);
> Ns_MasterLock();
> ...
>
>
> i.e. if the 'name' of the thread primitive given to the command does
> not already contain a pointer to the underlying object, and it can not
> be converted to one by deserialising a pointer address, then take the
> global lock and look the name up in the hash table. Log this case.
>
>
> % proc lockunlock args {ns_mutex lock m; ns_mutex unlock m}
> % lockunlock
> [04/Oct/2006:18:39:25][8985.3086293920][-
> thread-1208673376-] Warning:
> tclthread.c:GetArgs: objv[2] not an address object. Looking up
> in hash table...
> % lockunlock
> % lockunlock
>
Hmhmhmhmhm.... look what I get after doing this change in tclthread.c
(exactly the same spot):
if (Ns_TclGetOpaqueFromObj(objv[2], type, &addr) != TCL_OK
&& Ns_TclGetAddrFromObj(interp, objv[2], type, &addr) !=
TCL_OK) {
Ns_Log(2, "MISS THE OBJ CACHE : %p", objv[2]);
Tcl_ResetResult(interp);
Ns_MasterLock();
I do following from the nscp line:
server1:nscp 9> proc lu count {ns_log notice LOCK.$count; ns_mutex
lock m; ns_mutex unlock m; ns_log notice UNLOCK.$count}
server1:nscp 11> lu 1
server1:nscp 12> lu 2
server1:nscp 13> lu 3
and this is what comes in the log:
[04/Oct/2006:20:25:22][3026.41967104][-nscp:3-] Notice: LOCK.1
[04/Oct/2006:20:25:22][3026.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f4ff8
[04/Oct/2006:20:25:22][3026.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f4ff8
[04/Oct/2006:20:25:22][3026.41967104][-nscp:3-] Notice: UNLOCK.1
[04/Oct/2006:20:25:24][3026.41967104][-nscp:3-] Notice: LOCK.2
[04/Oct/2006:20:25:24][3026.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f4ff8
[04/Oct/2006:20:25:24][3026.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f4ff8
[04/Oct/2006:20:25:24][3026.41967104][-nscp:3-] Notice: UNLOCK.2
[04/Oct/2006:20:25:27][3026.41967104][-nscp:3-] Notice: LOCK.3
[04/Oct/2006:20:25:27][3026.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f4ff8
[04/Oct/2006:20:25:27][3026.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f4ff8
[04/Oct/2006:20:25:27][3026.41967104][-nscp:3-] Notice: UNLOCK.3
So, what now?
> The first time the proc is run it is compiled and the name is indeed
> looked up in the hash of mutex names. And that is of course 'slow'
> because there's a lock around the hash table.
>
> But that is the *only* time this happens. On the second and third
> attempt, no locking or look up!
>
> What's interesting from the above is that there is only a single log
> line, but there are two literal "m" objects. Apparently Tcl is doing
> some optimising behind the scenes...
I believe quite opposit is happening. The literal "m" gets lost
and so its saved address.
>
> Right. But it won't shimmer away, because it is a literal name and you
> have no need to manipulate it in any way that would cause it to
> shimmer, such as putting it in an nsv array.
>
> Now, I'm not sure what you're getting at with the "junklock" business.
> If you mean the user could have a typo in their code and an extra lock
> will be created behind their back, well, that's the nature of Tcl.
> Same goes for variables, right?
>
> Although you could define ns_mutex create to take a name and force
> people to use this in their initialisation code, and then in calls to
> lock and unlock you wouldn't create on demand, you'd just do the look
> up (first time only!), and throw an error if the name doesn't exist.
>
> But maybe I'm missing your point here...
Yes. You miss the point. The "junkmutex" is just yet another
name. No real "junk" was ment here.
>
>
>> OTOH, when I say
>>
>> set mutexhandle [ns_mutex create]
>>
>> I can do whatever I want with mutexhandle, it will always point to
>> that damn mutex. I need not do some other lookup in addition.
>> I can put it into nsv, transfer it over network back to myself...
>
>
> It's not that you *can* put it into an nsv array, it's that you *have*
> to, because how else are you going to communicate the serialised
> pointer which is a mutex handle?
No way. I have to put it there. You need not necessarily
put the mutex name there as it is "known". This is true.
>
> And you *have* to do it because what good is a mutex without more than
> one thread?
Correct.
>
> And if you have more than one thread you have more than one interp,
> each with it's own global namesapce. So how do you refer to a single
> mutex from within each interp?
Using the handle I put in the nsv, how else?
>
>
>> Thats what I mean by removing the handle from users. If you
>> do that you need to do more, introduce more locks etc.
>> Nothing is for free...
>
>
> It is in fact more or less free. Compiling the Tcl source to byte code
> incurs a lock around the mutex hash table. It is a compile time
> expense. At run time there is no locking.
Hmhmhm... not really and not always. First we have to understand
why do I get those misses of the cache...
>
> nsv arrays *also* incur locking. Plus you have the effort of having to
> cache the mutex in a thread local variable, or else you incur the nsv
> locking cost each time.
Right. But finer grade locking, not the global lock.
Instead, the nsv bucket is locked.
>
> So I think this is in fact a case of a free lunch!
There is no such thing as free lunch. This is not a
theorem, this is an axiom.
>
> As far as I can see, the only thing that will make this not true, is
> some real-world, non-contrived case where the mutex name has to
> shimmer.
See above...
|
|
From: Stephen D. <sd...@gm...> - 2006-10-04 19:33:45
|
On 10/4/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> On 04.10.2006, at 20:00, Stephen Deasey wrote:
>
> > On 10/4/06, Zoran Vasiljevic <zv...@ar...> wrote:
> >>
> >> On 04.10.2006, at 18:44, Stephen Deasey wrote:
> >>
> >>>>
> >>>> OK. I can accept that. Still, you put this into frequently
> >>>> called procedure and there you go (literals are freed AFAIK,
> >>>> when procedure scope is exited): again you have global locking.
> >>>
> >>>
> >>> I'm not sure what you're saying here. What is freed, and when?
> >>>
> >>
> >> The literal object "themutex" is gone after procedure exits
> >> AFAIK. This MIGHT need to be double-checked though as I'm
> >> not 100% sure if the literals get garbage collected at that
> >> point OR at interp teardown.
> >
> >
> > If that were true then my scheme would be useless. But I don't think
> > it is. I added a log line to GetArgs in nsd/tclthread.c:
> >
> > if (Ns_TclGetOpaqueFromObj(objv[2], type, &addr) != TCL_OK
> > && Ns_TclGetAddrFromObj(interp, objv[2], type, &addr) !
> > = TCL_OK) {
> >
> > Ns_Log(Warning, "tclthread.c:GetArgs: objv[2] not an
> > address object. Looking up in hash table...");
> >
> > Tcl_ResetResult(interp);
> > Ns_MasterLock();
> > ...
> >
> >
> > i.e. if the 'name' of the thread primitive given to the command does
> > not already contain a pointer to the underlying object, and it can not
> > be converted to one by deserialising a pointer address, then take the
> > global lock and look the name up in the hash table. Log this case.
> >
> >
> > % proc lockunlock args {ns_mutex lock m; ns_mutex unlock m}
> > % lockunlock
> > [04/Oct/2006:18:39:25][8985.3086293920][-
> > thread-1208673376-] Warning:
> > tclthread.c:GetArgs: objv[2] not an address object. Looking up
> > in hash table...
> > % lockunlock
> > % lockunlock
> >
>
>
> Hmhmhmhmhm.... look what I get after doing this change in tclthread.c
> (exactly the same spot):
>
>
> if (Ns_TclGetOpaqueFromObj(objv[2], type, &addr) != TCL_OK
> && Ns_TclGetAddrFromObj(interp, objv[2], type, &addr) !=
> TCL_OK) {
> Ns_Log(2, "MISS THE OBJ CACHE : %p", objv[2]);
> Tcl_ResetResult(interp);
> Ns_MasterLock();
>
>
> I do following from the nscp line:
>
> server1:nscp 9> proc lu count {ns_log notice LOCK.$count; ns_mutex
> lock m; ns_mutex unlock m; ns_log notice UNLOCK.$count}
> server1:nscp 11> lu 1
> server1:nscp 12> lu 2
> server1:nscp 13> lu 3
>
> and this is what comes in the log:
>
> [04/Oct/2006:20:25:22][3026.41967104][-nscp:3-] Notice: LOCK.1
> [04/Oct/2006:20:25:22][3026.41967104][-nscp:3-] Error: MISS THE OBJ
> CACHE : 0x2f4ff8
> [04/Oct/2006:20:25:22][3026.41967104][-nscp:3-] Error: MISS THE OBJ
> CACHE : 0x2f4ff8
> [04/Oct/2006:20:25:22][3026.41967104][-nscp:3-] Notice: UNLOCK.1
> [04/Oct/2006:20:25:24][3026.41967104][-nscp:3-] Notice: LOCK.2
> [04/Oct/2006:20:25:24][3026.41967104][-nscp:3-] Error: MISS THE OBJ
> CACHE : 0x2f4ff8
> [04/Oct/2006:20:25:24][3026.41967104][-nscp:3-] Error: MISS THE OBJ
> CACHE : 0x2f4ff8
> [04/Oct/2006:20:25:24][3026.41967104][-nscp:3-] Notice: UNLOCK.2
> [04/Oct/2006:20:25:27][3026.41967104][-nscp:3-] Notice: LOCK.3
> [04/Oct/2006:20:25:27][3026.41967104][-nscp:3-] Error: MISS THE OBJ
> CACHE : 0x2f4ff8
> [04/Oct/2006:20:25:27][3026.41967104][-nscp:3-] Error: MISS THE OBJ
> CACHE : 0x2f4ff8
> [04/Oct/2006:20:25:27][3026.41967104][-nscp:3-] Notice: UNLOCK.3
>
> So, what now?
The control port is not typical. It's probably evaluating each line
with the "no-compile" flag. Neither is the runtest shell, which is
what I was using.
If I run the test as a registered proc in a conn thread with a proc
named "foo" and a mutex named "foo", things work fine -- no clash
between the name of the proc "foo" and the mutex "foo".
The experiment with a lockunlock proc and a broadcast proc run in the
same registered proc causes two log lines. The single atom "m" is
being shimmered between mutex and cond type.
Looks like these names live in the same namespace due to Tcl caching
identical strings in an object cache, to save memory.
It's not the end of the world. You just need to make sure the names
used are unique, which is not a completely unreasonable thing to do.
I mean, nsv array names are unique too, which is the alternative if
you have to store handles.
But I don't know if that makes this technique not useful...
Opinions? Other options?
> >> OTOH, when I say
> >>
> >> set mutexhandle [ns_mutex create]
> >>
> >> I can do whatever I want with mutexhandle, it will always point to
> >> that damn mutex. I need not do some other lookup in addition.
> >> I can put it into nsv, transfer it over network back to myself...
> >
> >
> > It's not that you *can* put it into an nsv array, it's that you *have*
> > to, because how else are you going to communicate the serialised
> > pointer which is a mutex handle?
>
> No way. I have to put it there. You need not necessarily
> put the mutex name there as it is "known". This is true.
>
> >
> > And you *have* to do it because what good is a mutex without more than
> > one thread?
>
> Correct.
>
> >
> > And if you have more than one thread you have more than one interp,
> > each with it's own global namesapce. So how do you refer to a single
> > mutex from within each interp?
>
> Using the handle I put in the nsv, how else?
Right. So if handle management was implicit you wouldn't have to
bother shuffling them between interps via nsv arrays. Which would be a
good thing.
> > nsv arrays *also* incur locking. Plus you have the effort of having to
> > cache the mutex in a thread local variable, or else you incur the nsv
> > locking cost each time.
>
> Right. But finer grade locking, not the global lock.
> Instead, the nsv bucket is locked.
The global lock is just my laziness. Although if the handle is cached
it doesn't matter because the lock is only taken once, the first time.
And it has to be cached or else there's no point doing this...
|
|
From: Zoran V. <zv...@ar...> - 2006-10-04 19:51:28
|
On 04.10.2006, at 21:33, Stephen Deasey wrote:
> The control port is not typical. It's probably evaluating each line
> with the "no-compile" flag. Neither is the runtest shell, which is
> what I was using.
NO! It is FAR more complex than you think...
I added this:
proc lu count {
ns_log notice LOCK.$count
ns_mutex lock m
ns_mutex unlock m
ns_log notice UNLOCK.$count
}
into tcl/util.c and restarted the server.
Then I went to ctrl port and did:
server1:nscp 1> lu 1
server1:nscp 2> lu 2
server1:nscp 3> lu 3
and got
[04/Oct/2006:21:37:35][5931.41968128][-nscp:2-] Notice: LOCK.1
[04/Oct/2006:21:37:35][5931.41968128][-nscp:2-] Error: MISS THE OBJ
CACHE : 0x2f3858
[04/Oct/2006:21:37:35][5931.41968128][-nscp:2-] Notice: UNLOCK.1
[04/Oct/2006:21:37:38][5931.41968128][-nscp:2-] Notice: LOCK.2
[04/Oct/2006:21:37:38][5931.41968128][-nscp:2-] Notice: UNLOCK.2
[04/Oct/2006:21:37:38][5931.41968128][-nscp:2-] Notice: LOCK.3
[04/Oct/2006:21:37:38][5931.41968128][-nscp:2-] Notice: UNLOCK.3
Now, you would naively (as myself) think all is green.
Its not.
Because when I do following (on the ncp line)
server1:nscp 11> proc lu count {ns_log notice LOCK.$count; ns_mutex
lock m; ns_mutex unlock m; ns_log notice UNLOCK.$count}
i.e. redefine the proc "lu" then I get the cache misses again.
As you might expect because the ncp is "probably" not compiling
the proc. Allright, but when I logout the ncp and log in again
the completely new thread is created and the "lu" procedure
is again created from the blueprint. AND... if I now repeat
my test i get:
[04/Oct/2006:21:39:08][5931.41967104][-nscp:3-] Notice: LOCK.1
[04/Oct/2006:21:39:08][5931.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f34b0
[04/Oct/2006:21:39:08][5931.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f34b0
[04/Oct/2006:21:39:08][5931.41967104][-nscp:3-] Notice: UNLOCK.1
[04/Oct/2006:21:39:10][5931.41967104][-nscp:3-] Notice: LOCK.2
[04/Oct/2006:21:39:10][5931.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f34b0
[04/Oct/2006:21:39:10][5931.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f34b0
[04/Oct/2006:21:39:10][5931.41967104][-nscp:3-] Notice: UNLOCK.2
[04/Oct/2006:21:39:12][5931.41967104][-nscp:3-] Notice: LOCK.3
[04/Oct/2006:21:39:12][5931.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f34b0
[04/Oct/2006:21:39:12][5931.41967104][-nscp:3-] Error: MISS THE OBJ
CACHE : 0x2f34b0
[04/Oct/2006:21:39:12][5931.41967104][-nscp:3-] Notice: UNLOCK.3
As you see... it is much more complicated and I can't really
explain w/o deeply looking into it. There is something
happening there which we still do not understand.
> But I don't know if that makes this technique not useful...
>
It is useful. But it has side-effects we will yet
have to understand.
>
> Right. So if handle management was implicit you wouldn't have to
> bother shuffling them between interps via nsv arrays. Which would be a
> good thing.
Of course. I never sad something different.
>
>
> The global lock is just my laziness. Although if the handle is cached
> it doesn't matter because the lock is only taken once, the first time.
> And it has to be cached or else there's no point doing this...
It is cached *sometimes* and we do not know why not, when not.
At least I don't as I can reproduce the above behaviour easly.
|
|
From: Mike <nee...@gm...> - 2006-10-04 22:27:21
|
<lots of interesting text cut> Zoran, Stephen, To me, what this discussion says most of all is that the current approaches to handling synchronization primitives in naviserver are way too complicated for human consumption. Perhaps the "correct" approach is to actually examine real-world use of these primitives and find a "better" approach to the problem that can leverage the power of Tcl (something closer to the stuff Stephen was pointing out earlier) but not necessarily losing the power of free handles that Zoran believes are useful. I don't have the vaguest idea what that mechanism may be - but all you guys are doing right now is throwing around hypothetical use cases. I have yet to see a solid example in this thread that would lead me to believe one would ever want to use a mutex or condition variable anywhere in naviserver at all. (I don't mean to be say this harshly - but I feel like the discussion you guys are having right now is missing a bigger picture.) |
|
From: Zoran V. <zv...@ar...> - 2006-10-05 08:19:47
|
On 05.10.2006, at 00:27, Mike wrote: > <lots of interesting text cut> > > Zoran, Stephen, > To me, what this discussion says most of all is that the current > approaches to handling synchronization primitives in naviserver are > way too complicated for human consumption. Well, we have most (all?) basic blocks exported to Tcl level. This allows you (a Tcl programmer) to build just about anything that C-programmer can do in that area. Unfortunately, using the same idioms that C-programmer is used to, which may seem odd to you. > Perhaps the "correct" approach is to actually examine real-world use > of these primitives and find a "better" approach to the problem that > can leverage the power of Tcl (something closer to the stuff Stephen > was pointing out earlier) but not necessarily losing the power of free > handles that Zoran believes are useful. It is true that we'd yet to find something like that. It isn't easy though. At the moment we are discussing whether handles or tags are better, obviously neglecting the fact that the whole concept is actually "foreign" to the Tcl programmer. > I don't have the vaguest idea > what that mechanism may be - but all you guys are doing right now is > throwing around hypothetical use cases. I have yet to see a solid > example in this thread that would lead me to believe one would ever > want to use a mutex or condition variable anywhere in naviserver at > all. Ah... in some cases you have no choice! Every time you have some code that may execute in 2 connection (or other) threads at the same time and that accesses the same resource (a file or piece of memory) you need to lock and for that you need all those sync primitives. In some cases there are "abstractions" or "helpers" like ns_job or nsv where most of that ugly locking issues is hidden from you. But you can't abstract everything. From what I see, we are already very good: there is nsv which allows you to store/access scalar values across threads w/o explicit locking. Then we have caches which are thread-wide, also we have ns_job so you can start parallel jobs and wait for them, all w/o any special locking or synchronization of your own. The only piece that I miss is the ability to park a thread as a "script evaluator" in the way that Tcl threading extension does. So with all those tools, you can mostly avoid any own synchronization. But you can't really avoid it for some special cases. |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X