perl-cache-users — The Perl Cache users listserver

[Perl-cache-users] Cache::Cache and locking

[Ethan R. <et...@en...>]

Hi.

It looks to me from the README file that the various Cache::Cache 
modules make a point of avoiding locking of caches; for instance, the 
README indicating that Cache::FileCache avoids file locking entirely by 
using temporary files.  Given that the basic assumption of $cache->set() 
is that the data/object to be cached is already built, it makes perfect 
sense to take this approach.

However, in systems where you cache on demand, it can be desirable to 
lock things so that multiple processes don't concurrently build the 
target structure.  One approach we've used is to use a locked file 
during the build process, and only the process with the lock on that 
file enters the actual build routine, during which the expensive, 
application-specific logic takes place.  Any other processes that need 
the cache will wait while the lock file remains in place, and once the 
lock file is free, they'll check for the actual cache file again and 
presumably find it present and up-to-date.  This causes multiple 
processes to potentially wait on the outcome of one, but at least the 
extra processes aren't wasting system resources doing redundant, 
arbitrarily-complex builds.

I'm working on a project in which this kind of locking during the build 
process is essential.  While I would like to use the Cache::Cache family 
(Cache::FileCache for now), this approach seems not only at odds with 
the storage mechanisms available but the very interface of Cache::Cache 
itself -- for the sort of build process I've described, get() and set() 
essentially need to be a single atomic operation (from the perspective 
of the caller).  My current intention is to make a new Cache::Cache 
subclass that does file-based caching but with locks, with the special 
addition that you can pass a subref as the item to be cached, such that 
set() would execute the subref and cache the result and thus bring the 
build process into the portion of code in which the file is locked.

But, before pursuing this, I thought I'd check in and see if anybody 
else has tackled this kind of issue within Cache::Cache and come away 
with satisfactory solutions.  Or if anybody has any guidance or 
suggestions to offer.  Would I be better off starting from scratch for 
my Cache::Cache subclass, or could I perhaps find it helpful to start 
from Cache::FileCache?

Any suggestions will be much-appreciated.

Thanks.
- Ethan

-- 
Ethan Rowe
End Point Corporation
et...@en...

Re: [Perl-cache-users] Help with caching DBD::mysql statement handles ($sth)

[danny m. <dan...@gm...>]

Just a follow up to my yesterday's mail, with the help of DeWitt I was
able to find the right method I needed to use in DBD::mysql and cache
all my results at once so I can loop through them whenever I want.

My code looks like this now:

use Cache::FileCache;
my $cache = Cache::FileCache->new({
	namespace => "test",
	cache_root =>"cache/"
	});
my $all_arrays = $cache->get("testcache");
if (not defined $all_arrays){
	use DBI qw/:sql_types/;
	my $dbtype = "mysql";
	my $dbname = "test_db";
	my $dbuser = "test_user";
	my $dbpass = "12345678";
	my $tbl = "mybooks";
	my $default_cache_time = "30 minutes" ;
	my $dsn = "DBI:$dbtype:dbname=$dbname;host=localhost";
	my $dbh = DBI->connect($dsn, $dbuser, $dbpass);
	my $sql = "select id, name from $tbl order by id desc";
	my $sth = $dbh->prepare($sql);
	$sth->execute();
	$all_arrays = $sth->fetchall_arrayref ;
	$sth->finish;
	$dbh->disconnect ;
	$cache->set("testcache",$all_arrays,$default_cache_time);
}
foreach $r (@{$all_arrays}) {
	print "@$r\n";
}

Obviously, I can play with @$r by getting @$r[0] and @$r[1] and so on...

Thank you again DeWitt for showing me mysql's fetchall_arrayref and
other methods!

--Danny

Re: [Perl-cache-users] Help with caching DBD::mysql statement handles ($sth)

[DeWitt C. <de...@un...>]

Hi,

I didn't even know this list still existed!

The Cache::Cache library uses the Storable module for object
persistence.  While this library has been remarkably solid throughout
the years, the major limitation is that it can only persist data, not
live objects.  As such you would need to use a different means for
keeping a DB statement handle cached.

Are you seeing a major overhead in preparing statements?  I would
expect that even if you did, you would be able to simply pre-prepare
each statement when launching the app, or alternately, on first use.
Caching them to disk, or caching them for a set length of time
wouldn't seem to offer any advantages.

Hope this helps!

-DeWitt


On 9/12/06, danny miles <dan...@gm...> wrote:
> Hi list,
>
> I am using DBD::mysql to connect to my db. I'm trying to cache my
> already-executed statement handle ($sth) so i can reuse it over and
> over again.
>
> My code looks like this:
>
> my $cache = Cache::FileCache->new();
>
> my $sth = $cache->get_object(md5_hex($sql));
>
> if (not defined $sth){
>   $sth = $dbh->prepare($sql);
>   $sth->execute();
>   $cache->set_object(md5_hex($sql),$sth);
> }
> while(my @r = $sth->fetchrow_array()) {....my loop.....}
> .......the rest of my code......
>
> And the error I get is:
> Error: Can't store CODE items at P:/site/lib/Cache/CacheUtils.pm line 70.
>
> Can someone give me an advise as to whether this is a right direction
> I am going or how else can I accomplish such a mysql object caching.
>
> Any comment would be appreciated.
>
> Thanks,
>
> Danny
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Perl-cache-users mailing list
> Per...@li...
> https://lists.sourceforge.net/lists/listinfo/perl-cache-users
>

[Perl-cache-users] Help with caching DBD::mysql statement handles ($sth)

[danny m. <dan...@gm...>]

Hi list,

I am using DBD::mysql to connect to my db. I'm trying to cache my
already-executed statement handle ($sth) so i can reuse it over and
over again.

My code looks like this:

my $cache = Cache::FileCache->new();

my $sth = $cache->get_object(md5_hex($sql));

if (not defined $sth){
  $sth = $dbh->prepare($sql);
  $sth->execute();
  $cache->set_object(md5_hex($sql),$sth);
}
while(my @r = $sth->fetchrow_array()) {....my loop.....}
.......the rest of my code......

And the error I get is:
Error: Can't store CODE items at P:/site/lib/Cache/CacheUtils.pm line 70.

Can someone give me an advise as to whether this is a right direction
I am going or how else can I accomplish such a mysql object caching.

Any comment would be appreciated.

Thanks,

Danny

[Perl-cache-users] FASD project: Online survey launched

[<ben...@id...>]

Dear Open Source developer

I am doing a research project on "Fun and Software Development" in which I kindly invite you to participate.
You will find the online survey under http://fasd.ethz.ch/qsf/. The questionnaire consists of 53 questions and you will need about 15 minutes to complete it.

With the FASD project (Fun and Software Development) we want to define the motivational significance of fun when software developers decide to engage in Open Source projects. What is special about our research project is that a similar survey is planned with software developers in commercial firms. This procedure allows the immediate comparison between the involved individuals and the conditions of production of these two development models. Thus we hope to obtain substantial new insights to the phenomenon of Open Source Development.


With many thanks for your participation,
Benno Luthiger


PS:
The results of the survey will be published under http://www.isu.unizh.ch/fuehrung/blprojects/FASD/.
We have set up the mailing list fa...@we... for this study. Please see http://fasd.ethz.ch/qsf/mailinglist_en.html for registration to this mailing list.

_______________________________________________________________________

Benno Luthiger
Swiss Federal Institute of Technology Zurich
8092 Zurich

Mail: benno.luthiger(at)id.ethz.ch
_______________________________________________________________________

Re: [Perl-cache-users] Shared Memory on Windows 2000

[DeWitt C. <de...@un...>]

Hi:

I highly recommend using FileCache instead.  FileCache is faster,
scales better, is more portable, etc., than SharedMemoryCache.  There
is no reason not to chose FileCache over SharedMemoryCache.  If could
make the SharedMemoryCache go away then I would.

Hopefully this will help.  Cheers,

-DeWitt


On Tue, Apr 22, 2003 at 12:50:16PM -0700, Maxim Maltchevski wrote:
> Hi everyone:
> I'm trying to use a shared memory module with the
> ActiveState Perl 5.6.1 build 633. I want to read a
> huge XML-doc file into the memory and access it from
> different script instances instead of re-reading it
> into each instance. I know that would be on problem on
> UNIX using IPC modules. The Cache::Cache module is
> installed just fine from the ActiveState ppd-file but
> I'm unable to install IPC::ShareLite which is required
> by the Cache::SharedMemoryCache. As I understand
> IPC::* modules are not yet ported to Windows. Does
> anyone know, if there is a workaround for that or
> maybe some other module I could use? I searched the
> Net extensively but didn't find any solution.
> Any help is appreciated.
> Thank you,
> 
> 
> =====
> Maxim Maltchevski
> 
> 
> 
> __________________________________________________
> Do you Yahoo!?
> The New Yahoo! Search - Faster. Easier. Bingo
> http://search.yahoo.com
> 
> 
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Perl-cache-users mailing list
> Per...@li...
> https://lists.sourceforge.net/lists/listinfo/perl-cache-users

[Perl-cache-users] Shared Memory on Windows 2000

[Maxim M. <mal...@ya...>]

Hi everyone:
I'm trying to use a shared memory module with the
ActiveState Perl 5.6.1 build 633. I want to read a
huge XML-doc file into the memory and access it from
different script instances instead of re-reading it
into each instance. I know that would be on problem on
UNIX using IPC modules. The Cache::Cache module is
installed just fine from the ActiveState ppd-file but
I'm unable to install IPC::ShareLite which is required
by the Cache::SharedMemoryCache. As I understand
IPC::* modules are not yet ported to Windows. Does
anyone know, if there is a workaround for that or
maybe some other module I could use? I searched the
Net extensively but didn't find any solution.
Any help is appreciated.
Thank you,


=====
Maxim Maltchevski



__________________________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo
http://search.yahoo.com