[Jcyclone-users] JCyclone purpose (Re: a few questions)
Brought to you by:
jm7
|
From: Quartz <qua...@ya...> - 2006-12-21 23:59:59
|
> Unfortunately, I'm not familiar with inner magic of JCyclone enough to > know how well it does self-tuning automatically. The thread manager mesures how full is a stage's queue and attempt to maintain the (usually bounded) processing capacity of the stage to a minimum that avoid event loss / queue backpressure. The processing capacity is either altered by the number of threads allocated to the stage (provided) or through other means. Personally, I think adding/removing threads is far too dangerous (not to mention a nightmare to profile!). If a stage A feeds a stage B that is about 100 times slower, the natural balancing would give 100 threads to stage B and 1 to A in order to have the B queue catching up and not have event loss. This is not practical for large stage layout or larger processing capacities discrepancies. Most coders will endup putting a max thread to each stage, and will have to deal with event loss. I prefer playing with thread priorities (a custom thread manager I hacked in), but that doesn't entirely solve the loss problem. Anyhow, dealing with event loss is inevitable in SEDA and that lead to blocking enqueuing, which defeats the 'event-driven' driving concept, but brings a 2nd purpose to JCyclone: lossless data processing. It is pretty much pointless to do do self-tuning when there are few or no asynchronously processing stages. But sometimes, having a staged processing framework is handy. So, JCyclone as a few primary goals: 1-It's a stage container. It will delegate the processing of queue content (events) to a custom "handler". It has surrounding elements like the runtime environment, the config, the stage managers, profiler, the stage wrappers, the queue and some life cycling and control api. 2-It's a stage processing capacity balancer. 3-It can delegate the admission control (AC) of the queue. A few AC delegate supporting classes provided can assert a size limit, a avg rate limit, etc... You should already know how you want to solve your event processing problems in a regular multithreaded environment before you try to deploy a solution with a SEDA implementation. At this point, you will be already aware of so many runtime and threading details that sandstorm/jcyclone purpose will become much clearer... Once your solution is clearly screaming for at least 1 of the 3 main features (stage balancing, stage container, queueing admision control), then JCyclone can help you avoid some headaches. It serves handlers like a servlet container serves servlets: you still have all the processing work to do, but not the http/socket crap to deal with. JCyclone suggest to take care of the queueing/threading crap while you take care of the event processing. > so if you are doing fan in/fan out, you may have to do the > load-balancing yourself inside your fan-out stage. not sure if there's > built-in functionality in JCyclone to handle this. The stage handlers are the ones calling a sink.enqueue(), therefore the handler class coder is responsible for such load balancing. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |