Re: [Scribeserver-users] Configuration options and memory usage

[Anthony G. <an...@fa...>]

Alex,

See the Scribe Wiki for more information about configuration options:
http://scribeserver.wiki.sourceforge.net/

If your Scribe instance is using up a ton of memory, this is most likely due to there being too many simultaneous connections trying to log large amounts of data.  Scribe is built using a Thrift TNonBlocking server, and Thrift will attempt to allocate a buffer for every open connection.  (Thrift developers are currently working on a feature to help manage the number of Thrift connections, but this has not yet been released.)

The only way to reduce this memory usage is to either limit the number of simultaneous connections or reduce the amount of data sent in each call to Scribe.  I'm not sure how you have configured Scribe, but turning on 'use_conn_pool=yes' on may help.  (You would need to use connection pooling on all the Scribe servers that are sending messages to the Scribe server that is using too much memory).  Feel free to send me a copy of the Scribe configuration(s) you are using and I will take a look.

-Anthony