[Jcyclone-users] JCyclone purpose (Re: a few questions)

[Quartz <qua...@ya...>]

> Unfortunately, I'm not familiar with inner magic of JCyclone enough to
> know how well it does self-tuning automatically.

The thread manager mesures how full is a stage's queue and attempt to maintain the (usually
bounded) processing capacity of the stage to a minimum that avoid event loss / queue backpressure.
The processing capacity is either altered by the number of threads allocated to the stage
(provided) or through other means.

Personally, I think adding/removing threads is far too dangerous (not to mention a nightmare to
profile!). If a stage A feeds a stage B that is about 100 times slower, the natural balancing
would give 100 threads to stage B and 1 to A in order to have the B queue catching up and not have
event loss. This is not practical for large stage layout or larger processing capacities
discrepancies. Most coders will endup putting a max thread to each stage, and will have to deal
with event loss.
I prefer playing with thread priorities (a custom thread manager I hacked in), but that doesn't
entirely solve the loss problem.

Anyhow, dealing with event loss is inevitable in SEDA and that lead to blocking enqueuing, which
defeats the 'event-driven' driving concept, but brings a 2nd purpose to JCyclone: lossless data
processing. It is pretty much pointless to do do self-tuning when there are few or no
asynchronously processing stages. But sometimes, having a staged processing framework is handy.


So, JCyclone as a few primary goals:

1-It's a stage container. It will delegate the processing of queue content (events) to a custom
"handler". It has surrounding elements like the runtime environment, the config, the stage
managers, profiler, the stage wrappers, the queue and some life cycling and control api.

2-It's a stage processing capacity balancer.

3-It can delegate the admission control (AC) of the queue. A few AC delegate supporting classes
provided can assert a size limit, a avg rate limit, etc...


You should already know how you want to solve your event processing problems in a regular
multithreaded environment before you try to deploy a solution with a SEDA implementation. At this
point, you will be already aware of so many runtime and threading details that sandstorm/jcyclone
purpose will become much clearer...

Once your solution is clearly screaming for at least 1 of the 3 main features (stage balancing,
stage container, queueing admision control), then JCyclone can help you avoid some headaches. It
serves handlers like a servlet container serves servlets: you still have all the processing work
to do, but not the http/socket crap to deal with. JCyclone suggest to take care of the
queueing/threading crap while you take care of the event processing.




> so if you are doing fan in/fan out, you may have to do the
> load-balancing yourself inside your fan-out stage. not sure if there's
> built-in functionality in JCyclone to handle this.

The stage handlers are the ones calling a sink.enqueue(),
therefore the handler class coder is responsible for such load balancing.



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com