One solution that might be worth considering, if you have the resources, would be to set up two Solr servers in a master-slave configuration. You could post new records to the master and have it replicate periodically to the slave. This way, if adding records makes the system busy, end users will not be affected.

That being said, I don't know why adding a record should make the whole system busy for a minute. Are you running an optimize after every import? That might be the problem.... Perhaps you could do a commit instead to reduce the load.

- Demian

From: Steinberg, Jan [Jan.Steinberg@gesis.org]
Sent: Friday, January 03, 2014 4:38 AM
To: vufind-general@lists.sourceforge.net
Subject: [VuFind-General] real time user content integration

Hello everyone,

 

first of all I’d like to wish you all a Happy New Year!

 

But now to business:

We think about giving logged in users the opportunity to real time integrate metadata to the Solr index of our Vufind installation (1.x). In our tests we used a simple php form to fetch the data, we validated the content, generated the import file and sent it to the index.

Problem is that the index has to be rebuilt and the system is busy for over 60 seconds afterwards.

Is there a feasible way to do it without this time gap?

I know that normally the system’s not made for such actions but maybe it’s possible anyway?

 

Thanks in advance for your feedback!

 

Jan

 

 

--

GESIS - Leibniz Institute for the Social Sciences

Computational Social Science (CSS)

Team Knowledge Discovery

Jan Steinberg, M.A.

 

Unter Sachsenhausen 6-8,  D-50667 Köln
Tel: + 49 (0) 221 / 47 694 – 215
Fax: + 49 (0) 221 / 47 694 – 560