Menu
▾
▴
naviserver-devel
|
From: Vlad S. <vl...@cr...> - 2005-12-11 17:37:25
|
Helo Guys, Can we close nscache incorporating into the core? I will be soon producing some product involving naviserver which uses nscache and having it in the core would simplify life a little bit. But if it will not be backward compatible, this will introduce more problems because ther are a lot of software using ns_cache API. Can we just put existing module into the core? It's been working good in the past. -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2005-12-11 22:06:43
|
On 11.12.2005, at 18:39, Vlad Seryakov wrote: > > Can we just put existing module into the core? It's been working > good in the past. I have nothing against. We use this module for years and it just works fine. I also like Stephens less-locking approach but only one thing bothers me there: inability to create new caches during runtime. I'd also accept "just creating and not tearing down" caches as this would suffice us. Cheers Zoran |
|
From: Vlad S. <vl...@cr...> - 2005-12-11 22:52:51
|
Yes, "just-create" policy is fine, clearing the cache but keeping it empty is not bad. Reduceing locks with nscache is actually a big deal but at this point compatibility with old API is a higher priority, too much production code. Let's see what Stephen will say :-)) Zoran Vasiljevic wrote: > > On 11.12.2005, at 18:39, Vlad Seryakov wrote: > >> >> Can we just put existing module into the core? It's been working good >> in the past. > > > I have nothing against. We use this module for years and it just works > fine. > I also like Stephens less-locking approach but only one thing bothers me > there: inability to create new caches during runtime. > I'd also accept "just creating and not tearing down" caches as this would > suffice us. > > Cheers > Zoran > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Stephen D. <sd...@gm...> - 2005-12-23 19:03:46
|
On 12/11/05, Vlad Seryakov <vl...@cr...> wrote: > Yes, "just-create" policy is fine, clearing the cache but keeping it > empty is not bad. Reduceing locks with nscache is actually a big deal > but at this point compatibility with old API is a higher priority, too > much production code. > > Let's see what Stephen will say :-)) > > Zoran Vasiljevic wrote: > > > > On 11.12.2005, at 18:39, Vlad Seryakov wrote: > > > >> > >> Can we just put existing module into the core? It's been working good > >> in the past. > > > > > > I have nothing against. We use this module for years and it just works > > fine. > > I also like Stephens less-locking approach but only one thing bothers m= e > > there: inability to create new caches during runtime. > > I'd also accept "just creating and not tearing down" caches as this wo= uld > > suffice us. Yes, good points. I've uploaded a new patch which merges everything into the core with minimal changes to existing C code. Let me know how this looks. Regarding API compatibility for Tcl code, I thought a wrapper ns_cache command could be added to the tcl/cache.tcl file. What do you think? |
|
From: Vlad S. <vl...@cr...> - 2005-12-24 00:15:17
|
Looks like a good patch, can you just commit it. Happy Holidays! Stephen Deasey wrote: > On 12/11/05, Vlad Seryakov <vl...@cr...> wrote: > >>Yes, "just-create" policy is fine, clearing the cache but keeping it >>empty is not bad. Reduceing locks with nscache is actually a big deal >>but at this point compatibility with old API is a higher priority, too >>much production code. >> >>Let's see what Stephen will say :-)) >> >>Zoran Vasiljevic wrote: >> >>>On 11.12.2005, at 18:39, Vlad Seryakov wrote: >>> >>> >>>>Can we just put existing module into the core? It's been working good >>>>in the past. >>> >>> >>>I have nothing against. We use this module for years and it just works >>>fine. >>>I also like Stephens less-locking approach but only one thing bothers me >>>there: inability to create new caches during runtime. >>>I'd also accept "just creating and not tearing down" caches as this would >>>suffice us. > > > Yes, good points. I've uploaded a new patch which merges everything > into the core with minimal changes to existing C code. Let me know > how this looks. > > Regarding API compatibility for Tcl code, I thought a wrapper ns_cache > command could be added to the tcl/cache.tcl file. What do you think? > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_idv37&alloc_id865&op=click > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2005-12-24 10:10:30
|
Am 23.12.2005 um 20:03 schrieb Stephen Deasey: > Yes, good points. I've uploaded a new patch which merges everything > into the core with minimal changes to existing C code. Let me know > how this looks. Great! > > Regarding API compatibility for Tcl code, I thought a wrapper ns_cache > command could be added to the tcl/cache.tcl file. What do you think? You mean "ns_cache eval" -> "ns_cache_eval" stuff? Yes, just place them there. There are not too many of them anyways. When I look thru our code, we just use ns_cache (create, eval, flush) so I will go and rename them to ns_cache_create etc. And, commit that code. And, happy christmas to you all! Thanks, Zoran |
|
From: Stephen D. <sd...@gm...> - 2005-12-30 02:11:41
|
On 12/24/05, Zoran Vasiljevic <zv...@ar...> wrote: > > Am 23.12.2005 um 20:03 schrieb Stephen Deasey: > > > Yes, good points. I've uploaded a new patch which merges everything > > into the core with minimal changes to existing C code. Let me know > > how this looks. > > Great! > > > > > Regarding API compatibility for Tcl code, I thought a wrapper ns_cache > > command could be added to the tcl/cache.tcl file. What do you think? > > You mean "ns_cache eval" -> "ns_cache_eval" stuff? Yes, just place > them there. There are not too many of them anyways. > When I look thru our code, we just use ns_cache (create, eval, flush) > so I will go and rename them to ns_cache_create etc. > > And, commit that code. OK. It shouldn't cause any problems with existing code, and you can still load the nscache module, so I'll go ahead and we can see how it works out. It's only ~600 lines of code for the Tcl commands, and ~500 chopped out of cache.c, so I think it's going in the right direction. Still to think about are per-thread Tcl caches. The main reason for these are that you can put Tcl objects in the cache, not just their string rep, which is particularly useful for already-compiled byte code -- i.e. for *.tcl pages. The best solution for caching byte code might be to share the mechanism with ADP pages. They currently have a pre-thread cache of script blocks, and a per-server cache of text blocks, for each page.=20 AOLserver 4.5 adds a per-server output cache. We could use this for *.tcl pages as well, perhaps negating the need for per-thread Tcl caches. That still leaves Tcl objects like lists or dictionaries that you may not want to stringify on each access. Input welcome... |
|
From: Andrew P. <at...@pi...> - 2005-12-30 05:32:46
|
On Thu, Dec 29, 2005 at 07:11:35PM -0700, Stephen Deasey wrote: > Still to think about are per-thread Tcl caches. The main reason for > these are that you can put Tcl objects in the cache, not just their > string rep, which is particularly useful for already-compiled byte > code -- i.e. for *.tcl pages. Tcl code in *.tcl pages (not in procs) can be, has been, and I hope currently still is cached in OpenACS using the mechanism Rob Mayoff added to AOLserver 3.3 long ago. Essentially, all Tcl code in each *.tcl page is automatically "compiled" to Tcl procs on first execution, and those procs' bytecode is then cached via ns_cache. Each connection thread does its own entirely independent cacheing, but it worked fine and apparently was a substantial speed up for some ACS sites when switching from AOLserver 3.3+ad11 or 3.3+ad12 (I forget which) to 3.3+ad13. Actually, I'm not sure whether that code is currently active or not. It sounds as if the feature may have broke during the AOLserver 3.x to 4.x transition and never been repaired. See here for more info: http://sourceforge.net/tracker/?func=detail&aid=689515&group_id=3152&atid=353152 Does Rob's cacheing cover the functionality you want, or are you thinking of cacheing of some other sort of Tcl objects? > The best solution for caching byte code might be to share the > mechanism with ADP pages. They currently have a pre-thread cache of > script blocks, and a per-server cache of text blocks, for each page. > AOLserver 4.5 adds a per-server output cache. We could use this for > *.tcl pages as well, perhaps negating the need for per-thread Tcl > caches. That still leaves Tcl objects like lists or dictionaries that > you may not want to stringify on each access. Ah. I think Rob's code above solely cached compiled Tcl bytecode (in what form I don't know), not Tcl_Obj representations of anything. I've also no idea whether it plays nicely with Zoran's Ttrace extension or not. But, we already have centralized server-wide cacheing of Tcl_Obj's via Zoran's Tcl Threads Extension, right? So, is there any real use case for a per-thread Tcl_Obj cache? (Offhand I can't think of one, but then I don't think I've ever really used ns_cache directly at all, only the simplified util_memoize facility of OpenACS, so I'm not the right person to ask.) -- Andrew Piskorski <at...@pi...> http://www.piskorski.com/ |
|
From: Stephen D. <sd...@gm...> - 2005-12-30 07:46:13
|
On 12/29/05, Andrew Piskorski <at...@pi...> wrote: > On Thu, Dec 29, 2005 at 07:11:35PM -0700, Stephen Deasey wrote: > > > Still to think about are per-thread Tcl caches. The main reason for > > these are that you can put Tcl objects in the cache, not just their > > string rep, which is particularly useful for already-compiled byte > > code -- i.e. for *.tcl pages. > > Tcl code in *.tcl pages (not in procs) can be, has been, and I hope > currently still is cached in OpenACS using the mechanism Rob Mayoff > added to AOLserver 3.3 long ago. Essentially, all Tcl code in each > *.tcl page is automatically "compiled" to Tcl procs on first > execution, and those procs' bytecode is then cached via ns_cache. > Each connection thread does its own entirely independent cacheing, but > it worked fine and apparently was a substantial speed up for some ACS > sites when switching from AOLserver 3.3+ad11 or 3.3+ad12 (I forget > which) to 3.3+ad13. > > Actually, I'm not sure whether that code is currently active or not. > It sounds as if the feature may have broke during the AOLserver 3.x to > 4.x transition and never been repaired. See here for more info: > > http://sourceforge.net/tracker/?func=3Ddetail&aid=3D689515&group_id=3D3= 152&atid=3D353152 ACS hasn't used that for a long time. It handles *.adp and *.tcl pages itself. ACS creates procs named after the file to evaluate at runtime. It does this for each thread the first time a *.tcl page is called. There is no eviction policy, therefore it's more of a memory leak than a cache. Rob's old method was to slurp the contents of the *.tcl file into a variable and eval it. The variable would be byte code compiled the first time it was evaled. Because of limitations in Tcl, byte code objects cannot be shared across threads, so to save the byte code it was necessary to have a per-thread Tcl cache. > Does Rob's cacheing cover the functionality you want, or are you > thinking of cacheing of some other sort of Tcl objects? Rob's per-thread Tcl cache can cache any Tcl object. It's just a hash table with an eviction policy (max size or time limit). > > The best solution for caching byte code might be to share the > > mechanism with ADP pages. They currently have a pre-thread cache of > > script blocks, and a per-server cache of text blocks, for each page. > > AOLserver 4.5 adds a per-server output cache. We could use this for > > *.tcl pages as well, perhaps negating the need for per-thread Tcl > > caches. That still leaves Tcl objects like lists or dictionaries that > > you may not want to stringify on each access. > > Ah. I think Rob's code above solely cached compiled Tcl bytecode (in > what form I don't know), not Tcl_Obj representations of anything. > I've also no idea whether it plays nicely with Zoran's Ttrace > extension or not. > > But, we already have centralized server-wide cacheing of Tcl_Obj's via > Zoran's Tcl Threads Extension, right? So, is there any real use case > for a per-thread Tcl_Obj cache? (Offhand I can't think of one, but > then I don't think I've ever really used ns_cache directly at all, > only the simplified util_memoize facility of OpenACS, so I'm not the > right person to ask.) Kind of. The Tcl thread package has nsv arrays, with the added twist that certain Tcl objects do not have to be serialised to their string form. This isn't a cache though -- there's no eviction policy. Also, compared to a per-thread Tcl cache there will be locking overhead. It's a trade off though. If you have a small amount of data and/or if the data is expensive to serialise or is to be evaled, it may make sense to put it in a per-thread cache. Otherwise, save the memory and share it between threads. If you look at the old nscache module you'll see basically two completely independent code paths for per-thread and shared caches. I didn't want to go that far. I also thought that perhaps the main use case was for caching byte code, and that might be better done with the ADP caching mechanism. But I don't know, there may be situations where a per-thread cache is worht while and we should add that...? |
|
From: Zoran V. <zv...@ar...> - 2005-12-30 13:20:50
|
Am 30.12.2005 um 08:46 schrieb Stephen Deasey: > > But I don't know, there may be situations where a per-thread cache is > worht while and we should add that...? Hm... for a typical web application (whatever that is) where you have short lived connections, it is not worth the work, IMO. But if you have a long running operation in a thread, a per-thread cache seems like a good idea. There you can stuff (and reuse) Tcl objects w/o shimmering to/from string. To be honest, at the moment I do not have any need for such thing but on the general level, an option to create a cache as "global" vs. thread-private is a powerful functionality. Cheers Zoran |
|
From: Vlad S. <vl...@cr...> - 2005-12-30 14:45:02
|
I have web applications and server applications running on our server and in my experience so far i always prefer global caches over per-thread cache. If you need something local in long running thread, you can use Tcl global variables to keep state during the processing, even namespace variables. It is just one Tcl interpreter anyway. Just my 2 cents. Zoran Vasiljevic wrote: > > Am 30.12.2005 um 08:46 schrieb Stephen Deasey: > >> >> But I don't know, there may be situations where a per-thread cache is >> worht while and we should add that...? > > > Hm... for a typical web application (whatever that is) where you have > short lived connections, it is not worth the work, IMO. > But if you have a long running operation in a thread, a per-thread > cache seems like a good idea. There you can stuff (and reuse) Tcl > objects w/o shimmering to/from string. > To be honest, at the moment I do not have any need for such thing > but on the general level, an option to create a cache as "global" > vs. thread-private is a powerful functionality. > > Cheers > Zoran > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2005-12-30 14:52:19
|
Am 30.12.2005 um 15:46 schrieb Vlad Seryakov: > I have web applications and server applications running on our =20 > server and in my experience so far i always prefer global caches =20 > over per-thread cache. If you need something local in long running =20 > thread, you can use Tcl global variables to keep state during the =20 > processing, even namespace variables. It is just one Tcl =20 > interpreter anyway. =10True. But the state can be accumulating over time, though, leading to extensive memory usage. Hence the cache, as it can selecively expire things. But... I haven't any code needing per-thread caches, yet. As a tool in the toolbox, OK, but I will not press on that. I'm perfectly happy with global caches. Zoran= |
|
From: Vlad S. <vl...@cr...> - 2005-12-30 15:01:45
|
On that note i have another idea i'd like to discuss before i even start coding a prototype. There is thread on aolserver mailing list about upload prgoress, so i thought would it be a good idea to have global url-specific cache of all uploads, let's say, all requestes with content-length > 0. It will have only 2 alues, current size and total length and will last only for the time of upload. It will belong to Sock structure of the Request, so on close it will be freed as well. Making it url-specific will give Web developer ability to generate uniaue urls for upload and then request statistics, requests with the same url will not override each other, server will update statistics for the first created cache only, subsequent uploads with the same url will not show anything or show old values which is fine for security reasons. Overhead is minimal and it will add one new commmand like ns_upload_stats url. SockRead will handle it all, so no other places are affected. Is it worth trying? Zoran Vasiljevic wrote: > > Am 30.12.2005 um 15:46 schrieb Vlad Seryakov: > >> I have web applications and server applications running on our server >> and in my experience so far i always prefer global caches over >> per-thread cache. If you need something local in long running thread, >> you can use Tcl global variables to keep state during the processing, >> even namespace variables. It is just one Tcl interpreter anyway. > > > True. But the state can be accumulating over time, though, > leading to extensive memory usage. Hence the cache, as it > can selecively expire things. > > But... > > I haven't any code needing per-thread caches, yet. > As a tool in the toolbox, OK, but I will not press > on that. I'm perfectly happy with global caches. > > Zoran > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_idv37&alloc_id865&op=click > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2005-12-30 15:07:30
|
Am 30.12.2005 um 16:02 schrieb Vlad Seryakov: > Is it worth trying? Yesterday! Zoran |
|
From: Vlad S. <vl...@cr...> - 2005-12-30 17:18:26
Attachments:
upload.patch
|
Patch is attached, works great Zoran Vasiljevic wrote: > > Am 30.12.2005 um 16:02 schrieb Vlad Seryakov: > >> Is it worth trying? > > > Yesterday! > Zoran > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2005-12-30 17:23:43
|
Am 30.12.2005 um 18:19 schrieb Vlad Seryakov: > Patch is attached, works great Why don't you put that in RFE section in SourceForge? This way get get a nice record for future refs. Cheers Zoran |
|
From: Zoran V. <zv...@ar...> - 2005-12-30 17:28:42
|
Am 30.12.2005 um 18:19 schrieb Vlad Seryakov: > Patch is attached, works great > Dummy question: what happens I have several uploads at a time all handled by the same url? Zoran |
|
From: Vlad S. <vl...@cr...> - 2005-12-30 17:32:10
|
That is the key, if developer is not prepared unique urls, that statistics is useless because every upload overwrites the same stats. Zoran Vasiljevic wrote: > > Am 30.12.2005 um 18:19 schrieb Vlad Seryakov: > >> Patch is attached, works great >> > > Dummy question: what happens I have several uploads at a time > all handled by the same url? > > Zoran > > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2005-12-30 17:36:57
|
Am 30.12.2005 um 18:33 schrieb Vlad Seryakov: > That is the key, if developer is not prepared unique urls, that > statistics is useless because every upload overwrites the same stats. > OK. I will talk to one of our Web-GUI guys. He's been bugging me about that upload progress since we switched to 4.0 server. I will check with him about the unique url approach and see if this is feasible. I do not see any problem with that but I'm not that much in that corner of the world... Zoran |
|
From: Vlad S. <vl...@cr...> - 2005-12-30 17:41:39
|
Unique url or unique parameters, you can upload using /some/url/for/post?arg=1&arg2=2 urls as well, parameters will be ignored but can be used in upload stats using request->line instead of request->url. This way it is much easier to develop and support. Zoran Vasiljevic wrote: > > Am 30.12.2005 um 18:33 schrieb Vlad Seryakov: > >> That is the key, if developer is not prepared unique urls, that >> statistics is useless because every upload overwrites the same stats. >> > > OK. I will talk to one of our Web-GUI guys. He's been bugging me about > that upload progress since we switched to 4.0 server. I will check with > him about the unique url approach and see if this is feasible. I do not > see any problem with that but I'm not that much in that corner of the > world... > > Zoran > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2005-12-30 17:49:31
|
Am 30.12.2005 um 18:42 schrieb Vlad Seryakov: > Unique url or unique parameters, you can upload using /some/url/for/ > post?arg=1&arg2=2 urls as well, parameters will be ignored but can > be used in upload stats using request->line instead of request- > >url. This way it is much easier to develop and support. yes. this is true. the request line seems more appropriate to me. Zoran |
|
From: Zoran V. <zv...@ar...> - 2006-01-11 17:33:37
|
Am 30.12.2005 um 03:11 schrieb Stephen Deasey: > > Still to think about are per-thread Tcl caches. The main reason for > these are that you can put Tcl objects in the cache, not just their > string rep, which is particularly useful for already-compiled byte > code -- i.e. for *.tcl pages. > > The best solution for caching byte code might be to share the > mechanism with ADP pages. They currently have a pre-thread cache of > script blocks, and a per-server cache of text blocks, for each page. > AOLserver 4.5 adds a per-server output cache. We could use this for > *.tcl pages as well, perhaps negating the need for per-thread Tcl > caches. That still leaves Tcl objects like lists or dictionaries that > you may not want to stringify on each access. > > Input welcome... Hmhmhmhm... those adp caches are pretty-much app-dependent (adp), right? OTOH, the cache module is absolutely generic: put X in, get X out whatever X is, or? We do not have any use for that. If I need such thing, I create a namespaced array and stuff things there. Admitently, this is not a real cache where things are expired, but during the (short) connection lifetime I do not have that many things I'd like to keep arround... I don't know... If anybody has the need to make this beast, feel free to do that. I'm perfectly happy with the global cache as-is now. Cheers Zoran |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X