Menu
▾
▴
naviserver-devel
|
From: Vlad S. <vl...@cr...> - 2006-07-14 14:02:53
|
In the nstk module i have generic caching of compiled pages and file.tcl
can be modified to perform per-thread .tcl caching of bytecode as well.
proc ns_sourcefile { path } {
set proc0 [info procs filecache_$path]
set cookie0 $stat(mtime):$stat(ctime):$stat(ino):$stat(dev)
# Verify file modification time
if { $proc0 == "" || [$proc0 1] != $cookie0 } {
set code [ns_fileread $path]
proc filecache_$path {{getcookie 0}} "
if { \$getcookie } { return $cookie0 }
uplevel #$level { $code }
"
}
# Run the proc
filecache_$path
}
Zoran Vasiljevic wrote:
> Am 13.07.2006 um 23:40 schrieb Stephen Deasey:
>
>> I was suggesting that maybe there should be a away around this as in
>> some cases, like ours, we specifically make sure that all interps are
>> identical: we create one master interp at start up and then clone it
>> for each new thread. No one seemed excited by this though...
>
> You can't do that, as when you compile:
>
> set a 1
>
> in intrep A and interp B the "1" will result in different literal
> object so bytecode rep must be recompiled anyways. Even if you
> have one thread and several interps, each time the compiled
> bytecode object gets evaluated in the different interp, it gets
> recompiled.
>
> Therefore I would suggest people who need last microsecond of
> performance to put most of the frequently used code in procedures.
>
> Cheers
> Zoran
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
--
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/
|
|
From: Zoran V. <zv...@ar...> - 2006-07-14 14:11:55
|
Am 14.07.2006 um 16:04 schrieb Vlad Seryakov: > In the nstk module i have generic caching of compiled pages and > file.tcl > can be modified to perform per-thread .tcl caching of bytecode as > well. Yes, this is a trivial, poor-man's cache which is simple and works. Cheers, Zoran |
|
From: Mike <nee...@gm...> - 2006-07-12 20:10:35
|
On 7/12/06, Vlad Seryakov <vl...@cr...> wrote:
> I could be wrong, never used sendfile before.
>
> The code you can check is driver.c, WriterThread, and bufsize parameter
> can be used to define buffer.
Excerpt from sendfile(2) on FreeBSD:
When using a socket marked for non-blocking I/O, sendfile() may send
fewer bytes than requested. In this case, the number of bytes success-
fully written is returned in *sbytes (if specified), and the error EAGAIN
is returned.
Overall it looks like it would be very easy to change WriterThread()
to use sendfile() instead of read+memmove+(something). However the
(something) appears to be NsSockRead() which I am guessing is a bit
more complicated than just an indirect call to write()... since
probably that code-path supports sending through zlib or over an SSL
socket...
> I was considering using mmap, but somewhere i heard that too many
> mmapped files could be a problem for the server.
mmap is good because it kind of allows zero-copy from usercode..
however it's not useable in my situation directly because of the
32-bit address space restriction - can't mmap 65GB (650MB*100) of
address space... The solution I've seen others use is to mmap in 16MB
chunks and move the mmaped chunk as the transfer progresses along the
file.
|
|
From: Zoran V. <zv...@ar...> - 2006-07-12 21:02:24
|
Am 12.07.2006 um 22:04 schrieb Mike: > > mmap is good because it kind of allows zero-copy from usercode.. > however it's not useable in my situation directly because of the > 32-bit address space restriction - can't mmap 65GB (650MB*100) of > address space... The solution I've seen others use is to mmap in 16MB > chunks and move the mmaped chunk as the transfer progresses along the > file. I wouldn't be surprised to see some clever sendfile implementations mmap'ing the file and there you go with the 32-bit problem! I do not know... I believe you may be good advised to look into the actual platform implementation of the sendfile. Cheers Zoran |
|
From: Vlad S. <vl...@cr...> - 2006-07-13 20:00:41
|
This is from file.tcl,
# And here's the magic part. We're using "for" here to translate the
# text source file into bytecode, which will be associated with the
# Tcl_Obj we just cached (as its internal representation). "eval"
# doesn't do this as the eval provided in Tcl uses the TCL_EVAL_DIRECT
# flag, and hence interprets the text directly.
#
uplevel [for [lindex $pair 1] {0} {} {}]
Stephen Deasey wrote:
> On 7/13/06, Vlad Seryakov <vl...@cr...> wrote:
>> I meant .tcl file cacheing, which is in tcl/file.tcl when
>> enabletclpages=on then all requests to .tcl are cached in global cache.
>
> Right, the Tcl is read from disk and cached in memory. But only the
> Tcl *source*. It needs to be byte code compiled before use. This is
> what Andrew is talking about.
>
> Our cache stringifys everything that goes into it -- you can't share
> Tcl objects between threads. Therefore, every time you retrieve the
> Tcl page from cache (not disk), you still have to byte code compile
> it.
>
> aolserver3.x-ad13 cached the byte code per-interp.
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
>
--
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/
|
|
From: Stephen D. <sd...@gm...> - 2006-07-13 20:07:51
|
On 7/13/06, Vlad Seryakov <vl...@cr...> wrote:
> This is from file.tcl,
>
> # And here's the magic part. We're using "for" here to translate the
> # text source file into bytecode, which will be associated with the
> # Tcl_Obj we just cached (as its internal representation). "eval"
> # doesn't do this as the eval provided in Tcl uses the TCL_EVAL_DIRECT
> # flag, and hence interprets the text directly.
> #
>
> uplevel [for [lindex $pair 1] {0} {} {}]
>
It's all lies without per-interp caches...
|
|
From: Stephen D. <sd...@gm...> - 2006-07-13 21:02:37
|
On 7/13/06, Andrew Piskorski <at...@pi...> wrote: > On Thu, Jul 13, 2006 at 09:28:00PM +0100, Stephen Deasey wrote: > > > Except... Rob found a neat way to cheat by passing the script as an > > arg to a do-nothing 'for' command. 'for' compiles the script for you. > > Very smart and hellish to read... > > Ok, some Tcl commands cause their script arguments to be compiled to > bytecode, others simply interpret them directly. And Tcl interps > automatically cache their own bytecode as necessary. > > So... Are you saying that Rob changed AOLserver 3.x such that it used > 'for' rather than 'eval' (or whatever else) to run the code in *.tcl > files? And THAT made all the difference, because as long as the *.tcl > file had not changed, the Tcl interp would automatically find and use > the bytecode it had already created the first time that particular > *.tcl file was run? No, it uses eval at runtime. But the object to be evaled already has it's byte code representation. The byte code is generated as the object is put into the cache. > But if that's the case, then what does any of this have to do with > per-thread caches? As you said, Naviserver is already cacheing the > string source of *.tcl pages, so what's preventing you from using the > same for/eval trick Rob used, for each interp? > (I must be missing something here...) > Our caches are server-wide, not per-interp. Server-wide caches have their values stringified. AOLserver 4.5 has the same restriction with it's new ns_cache implementation, btw. We talked about adding per-interp caches. It's not a high priority for me as I think converting to use the ADP code is the best solution, for the reasons mentioned before. |
|
From: Andrew P. <at...@pi...> - 2006-07-13 21:15:32
|
On Thu, Jul 13, 2006 at 10:02:35PM +0100, Stephen Deasey wrote: > Our caches are server-wide, not per-interp. Server-wide caches have > their values stringified. Must they always though? That is also how the nsv_* commands worked, but I think Zoran's tsv_* commands do special work to retain the internal rep of many objects across tsv_set and tsv_get. -- Andrew Piskorski <at...@pi...> http://www.piskorski.com/ |
|
From: Zoran V. <zv...@ar...> - 2006-07-13 21:23:18
|
Am 13.07.2006 um 23:15 schrieb Andrew Piskorski: > Must they always though? That is also how the nsv_* commands worked, > but I think Zoran's tsv_* commands do special work to retain the > internal rep of many objects across tsv_set and tsv_get. Yes, but only for certain "known" types of Tcl objects. You can't cache bytecodes thread-wide because they contain literals and they are entered per-reference in bytecode rep. Even my tsv "tricks" will not work. Moreover, bytecodes get re-compiled when jumping interp so this all bytecode caching stuff is really not something I would do from the cost/gain perspective. The only real practical caching of bytecodes is moving more code into Tcl procedures and putting less code in the page itself. Cheers Zoran |
|
From: Stephen D. <sd...@gm...> - 2006-07-13 21:30:14
|
On 7/13/06, Andrew Piskorski <at...@pi...> wrote: > On Thu, Jul 13, 2006 at 10:02:35PM +0100, Stephen Deasey wrote: > > Our caches are server-wide, not per-interp. Server-wide caches have > > their values stringified. > > Must they always though? That is also how the nsv_* commands worked, > but I think Zoran's tsv_* commands do special work to retain the > internal rep of many objects across tsv_set and tsv_get. > Tcl objects may reference other objects. Think of a list. So it's not enough to copy an object's internal rep, you need to follow the references and do a deep-copy. That's real ugly because then you need to know the details of each object's internal rep. What happens when you come accross an object you don't know about? Punt, I guess: stringify it. It may or may not be worth while adding per-interp caches to the cache code, but I'm pretty sure ADP is the way to go for Tcl page caching. |
|
From: Zoran V. <zv...@ar...> - 2006-07-13 21:38:11
|
Am 13.07.2006 um 23:30 schrieb Stephen Deasey:
> On 7/13/06, Andrew Piskorski <at...@pi...> wrote:
>> On Thu, Jul 13, 2006 at 10:02:35PM +0100, Stephen Deasey wrote:
>>> Our caches are server-wide, not per-interp. Server-wide caches have
>>> their values stringified.
>>
>> Must they always though? That is also how the nsv_* commands worked,
>> but I think Zoran's tsv_* commands do special work to retain the
>> internal rep of many objects across tsv_set and tsv_get.
>>
>
> Tcl objects may reference other objects. Think of a list. So it's
> not enough to copy an object's internal rep, you need to follow the
> references and do a deep-copy. That's real ugly because then you need
> to know the details of each object's internal rep.
>
> What happens when you come accross an object you don't know about?
>
> Punt, I guess: stringify it.
EXACTLY this is what tsv is doing. For "known" objects it does
a deep copy. It also allows for registering user-callbacks to
process their own objects. But if it encounters object it does
not understand, it stringifies it.
Bytecodes cannot be "magically" processed as they contain
references to literal objects which are stored per-interp
so:
set a 1
The "1" is a literal which gets stored in the per-interp
literal table. Therefore bytecodes must be recompiled when
jumping interpreters. Which is what would hnappen all the
time in our scenario.
Hence I do not hold much about caching bytecodes.
>
> It may or may not be worth while adding per-interp caches to the cache
> code, but I'm pretty sure ADP is the way to go for Tcl page caching.
I must second that. With the extension that it may not be worth
adding per-interp/thread caches IMO.
Cheers
Zoran
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X