I'm working on a project expecting heavy load during launch (tens of
thousands of requests per hour)
Did anyone do any work preparing vufind for this kind of load?
From: Till Kinstler <kinstler@gm...> - 2012-01-25 11:58:39
What do you mean by "requests"? HTTP requests or "user requests/clicks"?
We are handling about 200,000 HTTP requests per day on the web frontend
of http://finden.nationallizenzen.de/ (including CSS, images requests etc.).
Our Solr search backend gets about 1 Million requests per day (a search
in our frontend generates about 3 to 15 requests in Solr, because we
load data from additional, linked records).
Our current setup evolved rather pragmatically with needs and
opportunities, so was not designed by a master plan...:
The VuFind PHP web frontend and MySQL currently run on one single (by
today's standards rather small) server machine (8 GB RAM, 2 Intel Xeon
X5450 CPUs @ 3 Ghz with 4 CPU cores each). The server hosts some
additional web services on top (like a HTTP load balancer for the Solr
search servers, see below).
The bottleneck for us is/was not the PHP frontend, but the Solr search
backend. Our index holds about 25 million records and is about 80 GB on
So we needed to scale Solr (also because we are running now an
additional Solr index holding about 100 million records, but we don't
provide a VuFind interface to that (yet?)).
Our "Solr farm" currently consists of three pretty average machines with
some memory (96 GB RAM each, 2 Intel Xeon X5660 CPUs @ 2,8 Ghz with 6
cores each). The amount of RAM is needed because of our index size. So
if your index is smaller, you might get along with less...
Each of these three Solr machines hosts the complete index cores (the
one for finden.nationallizenzen.de, and the other, larger one). We don't
use Solr sharding currently as scaling strategy (we found, that it is
not necessary for search performance (yet) and would only complicate
index maintenance). There is a load balancer in front of these three
Solr server that distributes the search requests...
So, I'd say you can handle tens of thousands requests per hour without
too much hardware effort.
The two things to investigate are the PHP frontend and the Solr search
backend. It might be a good idea to run them on seperate machines
(especially if your Solr index is large, so put it on a machine with
suitable RAM). Test your setup using HTTP load testing software (like
Apache Jmeter). If you are not happy with results, try to find the
bottleneck(s) (first question: is it the web frontend or Solr?). Then
apply common scaling mechanisms for HTTP applications like load
balancing (very easy with Solr, because it is stateless, with the VuFind
frontend you need a load balancer that manages persistent sessions)...
Am 25.01.2012 10:07, schrieb Tulie Amichal:
> Hi All
> I'm working on a project expecting heavy load during launch (tens of
> thousands of requests per hour)
> Did anyone do any work preparing vufind for this kind of load?
> טולי עמיכל
> tulie.amichal@... <mailto:tulie.amichal@...>
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> Vufind-tech mailing list