It's worth noting that caching is against the EULA/TOS (depending on what they call it) for places like Syndetics and Google Books (and presumably Amazon, but I haven't checked them).
Our stance has been that a small, time specific cache is acceptable because it is reasonable for delivery of a service. Long term or mass storage would be a pretty clear violation though. You are basically harvesting all the paid for content from their systems and keeping it.
Electronic Services Officer (Systems Team)
Division of Academic Information Services
University of Southern Queensland
Phone: +61 7 4631 1501
Fax: +61 7 4631 1841
From: Alan Rykhus [mailto:alan.rykhus@...]
Sent: Tuesday, 3 November 2009 2:00 AM
Subject: Re: [VuFind-General] Book covers alternatives
I modified our bookcover code to store the images once we obtained them.
I then check to see if the bookcover is available locally before I go to
find it elsewhere. As far as the disk usage, you should only have as
many bookcovers as you have unique ISBN codes in your database, so it
should not grow forever, only until you get all of the images. Nowadays,
what's 2.2 gigs? If it was using up 200 gigs, I still might not worry,
I'd just add more disk. The speed difference is worth it.
just my 2 cents -- al
On Mon, 2009-11-02 at 09:02 -0600, Demian Katz wrote:
> The book cover directories are just a file cache to speed things up
> and reduce bandwidth use. You could change the code to bypass the
> cache, but a better solution may be to institute some kind of cleanup
> routine. I'm assuming these 2.2 gigs have accumulated over a long
> period of time, and if you do some analysis, you'll probably find that
> many of them are old and have not been accessed in a long time. If
> not, you are probably being hit by spiders and should look into
> robots.txt settings as another line of defense. There are tools for
> every operating system that you can schedule to run regularly and
> delete all files older than a particular date -- once you find the
> best expiration age for your needs, you should be able to solve your
> disk space issues without losing the performance benefits of the
> Please let me know if you need more specifics -- I'm going to be
> tuning this for my own installation in the near future, and I'm happy
> to compare notes.
> - Demian
> From: Philip Shafer [mailto:shafer@...]
> Sent: Monday, November 02, 2009 9:49 AM
> To: vufind-general@...
> Subject: [VuFind-General] Book covers alternatives
> Currently our book covers are taking up 2.2Gigs of diskspace. At this
> rate we will be quickly running out of disk space, has any libraries
> dealt with this issue, and if so what were your solutions? Are there
> limits on bandwidths for book cover usage from Syndetics or Google?
> Would it be easier to just directly link to the image instead of
> downloading a local copy of the file?
> Philip Shafer
> Library System Services
> Rowan University Library
> 201 Mullica Hill Rd
> Glassboro, NJ 08028
> 856-256-4924 Fax
PALS, A Program of the Minnesota State Colleges and Universities
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
VuFind-General mailing list
This email (including any attached files) is confidential and is for the
intended recipient(s) only. If you received this email by mistake,
please, as a courtesy, tell the sender, then delete this email.
The views and opinions are the originator's and do not necessarily
reflect those of the University of Southern Queensland. Although all
reasonable precautions were taken to ensure that this email contained no
viruses at the time it was sent we accept no liability for any losses
arising from its receipt.
The University of Southern Queensland is a registered provider of
education with the Australian Government (CRICOS Institution Code No's.
QLD 00244B / NSW 02225M)