Hi Steve,

already apologies in advance for the vagueness of this answer but there have been several performance related optimizations to the stats between 1.6.2 and 3.0.

The latest one, SOLR sharding by year, was added in 3.0. This is especially useful for those institutions who have accumulated multiple years of SOLR stats.

https://wiki.duraspace.org/display/DSDOC3x/Managing+Usage+Statistics#ManagingUsageStatistics-Solr ShardingByYear

Enabling auto commit was also something not included in the DSpace 1.6 version of the stats by default:
https://atmire.com/website/?q=content/increasing-dspace-performance

Hope this helps! 200,000 usage events is indeed not a huge number so there should be a way to optimize.

Can I also ask you to share your findings about the spiders here?
https://jira.duraspace.org/browse/DS-790

best regards,

Bram

--
logo 
Bram Luyten @mire
2888 Loker Avenue East, Suite 315, Carlsbad, CA. 92010
Esperantolaan 4, Heverlee 3001, Belgium
 www.atmire.com


On Thu, Dec 20, 2012 at 3:36 AM, Ian Boston <ib236@cam.ac.uk> wrote:
Hi,

I was having a problem recently with stats in ds3, caused by excessive SQL queries building parent collections. There was a patch shared on list about a week ago by Andrea. It might help ?

Ian


On Thursday, December 20, 2012, Steve Swinsburg wrote:
Does anyone ever update their solr stats? Does anyone know about the performance issue I am seeing here?

thanks,
Steve


On 18/12/2012, at 5:15 PM, Steve Swinsburg <steve.swinsburg@anu.edu.au> wrote:

Hi all,

We have identified a number of new spider IP addresses from Google and other indexers being responsible for vastly inflating our stats. I've created a local spider filter list with the IP addresses and I am running the stats updater:
dspace stats-util -m

to reprocess the stats and mark them appropriately, then will remove them via:
dspace stats-util -f

However the mark is taking hours. Likewise if I go ahead and just delete them based on the new rules, via:
dspace stats-util -i

Is that normal? We only have about 200,000 views to process.

Version 1.6.2 but about to rollout an upgrade. If the performance has improved in 1.8.2 we can wait a week or so.


regards,
Steve

------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette


------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette