asterisk-java-users

[Asterisk-java-users] Advanced Event Filtering

[Aaron E. <aar...@gm...>]

I would like to propose a feature for more advanced event filtering,
beyond what AMI provides.

I have a situation where I am interested in extension state changes
only.  So I can cut down the number of events by using the "cal" event
mask.

However, I am connecting to about a dozen asterisk servers all with
relatively high load.

I am concerned that with the number of events flowing into my
application I am worried about performance.

Event though my event listener may only process these extension status
change events, discarding all others, the asterisk java API will still
be building a very large number of these event objects at a very quick
rate.

Although the objects will be immediately available for garbage
collection, at the rate they are produced, the heap space could fill
quickly causing very frequent garbage collection.  Frequent garbage
collection can be very detrimental to JVM performance, depending on
the implementation.

I might be being overly cautious here. If you think I am, please say
something as I don't really want to make more work for myself
unnecessarily.

In order to supply the functionality I am describing, I think I may
add some kind of HashSet of eventTypes to act as a filter. If the set
is null or empty, then that means all events. Otherwise, it would be
limited to those in the set.

In order to avoid creating these ManagerEvent objects, I'm thinking
I'll inject the filter check in the buildEvent method of the
ManagerReaderImpl class.

This brings me to my next point/request.  While I could provide my own
ManagerReaderImpl by overriding the factory method in
ManagerConnectionImpl, the buildEvent method is private. :(

This means I can't just extend and override the ManagerReaderImpl.  In
fact, if I make a copy of this implementation to my application's
source tree, there are a great deal of classes and interfaces that are
not visible in the org.asteriskjava.manager.internal package (the
Dispatcher interface for example).  I guess that is kind of the point
of the "internal" package since these aren't really supposed to be
part of the API for developers.

However, in this case, I have no choice but to modify the source
rather than supplying my own implementation (short of copying a great
many classes to my applications source tree).

At the very least, I would propose making some of the methods of the
ManagerReaderImpl and ManagerWriterImpl classes protected rather than
private.

Comments?

-aaron

Re: [Asterisk-java-users] Advanced Event Filtering

[Martin S. <ma...@be...>]

> -----Original Message-----
> From: ast...@li...=20
> [mailto:ast...@li...] On=20
> Behalf Of Aaron Evans
> Sent: Thursday, August 02, 2007 10:16 AM
> To: ast...@li...
> Subject: [Asterisk-java-users] Advanced Event Filtering

> Event though my event listener may only process these extension status
> change events, discarding all others, the asterisk java API will still
> be building a very large number of these event objects at a very quick
> rate.

This is true, and I think filtering isn't a bad idea. In fact, there has
been talk of filtering the data even further up the pipe, in Asterisk
itself.
=20
> This brings me to my next point/request.  While I could provide my own
> ManagerReaderImpl by overriding the factory method in
> ManagerConnectionImpl, the buildEvent method is private. :(
> This means I can't just extend and override the ManagerReaderImpl.  In
> fact, if I make a copy of this implementation to my application's
> source tree, there are a great deal of classes and interfaces that are
> not visible in the org.asteriskjava.manager.internal package (the
> Dispatcher interface for example).  I guess that is kind of the point
> of the "internal" package since these aren't really supposed to be
> part of the API for developers.
>
> At the very least, I would propose making some of the methods of the
> ManagerReaderImpl and ManagerWriterImpl classes protected rather than
> private.

I think the important reason that these implementations are private is
because there are assumptions that are made inside of each class that
you won't necessarily obey if you extend one, in turn breaking your
application. In closed source projects, you'd almost *have* to do this
so that your users don't break the default implementation and then blame
you (or worse, do something malicious with your product). In open source
world, I can see how this seems less fruitful, as someone can and will
copy your private implementation, as you're suggesting you will be
forced to do.

> In order to avoid creating these ManagerEvent objects, I'm thinking
> I'll inject the filter check in the buildEvent method of the
> ManagerReaderImpl class.

An alternative approach would be to make filtering more flexible for all
users, by explicitly defining filter objects in the API. These filter
objects would be required to make decisions based on something like the
Map<String, String> that ManagerReaderImpl uses.

> However, in this case, I have no choice but to modify the source
> rather than supplying my own implementation (short of copying a great
> many classes to my applications source tree).

At the risk of sounding too harsh -- copying classes means you'll have
to actually deal with the implementation details of the class directly,
whereas someone who is just extending an implementation is much more
likely to step on the toes of the original implementation for the worse.
I actually like that kind of design -- you can force the programmer to
be fully involved in the internals and make their own mistakes or stay
out of your working implementation.

I'd much prefer, however, in this case, an open public API for the
filtering, and then allow a user/developer to simply provide an
implementation for that.

Martin Smith, Systems Developer
ma...@be...
Bureau of Economic and Business Research
University of Florida
(352) 392-0171 Ext. 221=20

Re: [Asterisk-java-users] Advanced Event Filtering

[Aaron E. <aar...@gm...>]

On 8/2/07, Martin Smith <ma...@be...> wrote:
> > -----Original Message-----
> > From: ast...@li...
> > [mailto:ast...@li...] On
> > Behalf Of Aaron Evans
> > Sent: Thursday, August 02, 2007 10:16 AM
> > To: ast...@li...
> > Subject: [Asterisk-java-users] Advanced Event Filtering
>
> > Event though my event listener may only process these extension status
> > change events, discarding all others, the asterisk java API will still
> > be building a very large number of these event objects at a very quick
> > rate.
>
> This is true, and I think filtering isn't a bad idea. In fact, there has
> been talk of filtering the data even further up the pipe, in Asterisk
> itself.
>

Yeah, I wish there were more event filtering options within asterisk.

> > This brings me to my next point/request.  While I could provide my own
> > ManagerReaderImpl by overriding the factory method in
> > ManagerConnectionImpl, the buildEvent method is private. :(
> > This means I can't just extend and override the ManagerReaderImpl.  In
> > fact, if I make a copy of this implementation to my application's
> > source tree, there are a great deal of classes and interfaces that are
> > not visible in the org.asteriskjava.manager.internal package (the
> > Dispatcher interface for example).  I guess that is kind of the point
> > of the "internal" package since these aren't really supposed to be
> > part of the API for developers.
> >
> > At the very least, I would propose making some of the methods of the
> > ManagerReaderImpl and ManagerWriterImpl classes protected rather than
> > private.
>
> I think the important reason that these implementations are private is
> because there are assumptions that are made inside of each class that
> you won't necessarily obey if you extend one, in turn breaking your
> application. In closed source projects, you'd almost *have* to do this
> so that your users don't break the default implementation and then blame
> you (or worse, do something malicious with your product). In open source
> world, I can see how this seems less fruitful, as someone can and will
> copy your private implementation, as you're suggesting you will be
> forced to do.
>
> > In order to avoid creating these ManagerEvent objects, I'm thinking
> > I'll inject the filter check in the buildEvent method of the
> > ManagerReaderImpl class.
>
> An alternative approach would be to make filtering more flexible for all
> users, by explicitly defining filter objects in the API. These filter
> objects would be required to make decisions based on something like the
> Map<String, String> that ManagerReaderImpl uses.
>

Yes, that would be good.

> > However, in this case, I have no choice but to modify the source
> > rather than supplying my own implementation (short of copying a great
> > many classes to my applications source tree).
>
> At the risk of sounding too harsh -- copying classes means you'll have
> to actually deal with the implementation details of the class directly,
> whereas someone who is just extending an implementation is much more
> likely to step on the toes of the original implementation for the worse.
> I actually like that kind of design -- you can force the programmer to
> be fully involved in the internals and make their own mistakes or stay
> out of your working implementation.
>

Not harsh at all, I agree with you 100%.  Copying classes is the last
thing I want to do.

> I'd much prefer, however, in this case, an open public API for the
> filtering, and then allow a user/developer to simply provide an
> implementation for that.
>

Agreed, that is the better approach but I don't think I'll be able to
wait for these features and I'll have to hack something out myself for
the time being.  If what I come up with seems pretty good, I have no
problem contributing it back.  I'll be modifying the source directly
and then building the customized jar for my application.

> Martin Smith, Systems Developer
> ma...@be...
> Bureau of Economic and Business Research
> University of Florida
> (352) 392-0171 Ext. 221
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> Asterisk-java-users mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/asterisk-java-users
>

Re: [Asterisk-java-users] Advanced Event Filtering

[Stefan R. <ste...@re...>]

Hi,

my first thought when reading this thread was: If it's not broken, don't
try to fix it. It seems you did not experience any problems with garbage
collection (though you might run some load tests to verify that in your
environment). Java is really smart handling small short-lived objects.
The garbage collector has a young generation (eden space) designed for
just that purpose.

The next thing that comes to my mind is what you might actually achieve
by filtering events at the java side. You will still have create some
strings and build the map representing the events (note that the
management api makes no guarantees regarding the order of the attributes
so you have to parse at least until you encounter the "Event: XXX"
attribute). What you could save is creating the actual event object, but
well...

I would recommend to really get some numbers using a decent profiler. If
you can provide something that is convincing we might very well have a
look at supporting it in future Asterisk-Java releases.

=3DStefan

--=20
reuter network consulting
Neusser Str. 110
50760 Koeln
Germany
Telefon: +49 221 1305699-0
Telefax: +49 221 1305699-90
E-Mail:  ste...@re...
Jabber:  ste...@re...

Steuernummern 215/5140/1791 USt-IdNr. DE220701760

Re: [Asterisk-java-users] Advanced Event Filtering

[Brett S. <bs...@no...>]

I would be very surprised if you have any problems with the garbage 
collector.

We have done a lot of work in this area for a large client running a 
very heavily loaded java based web server (tomcat).
The garbage collector is super fast.

The only problem we ever had was if we let the java app swap memory out 
to the page file.
This is super dangerous with java.
[This took us about 6 weeks of heavy diagnosis to discover the root 
cause of, hence the experience with the java GC. This is the only bug 
that I've every resolved by 'touch'.  I happened to put my hand on the 
system unit during garbage collection and felt the drive rumbling :)]

The Tenured area of the garbage collector can grow quite big and remain 
untouch for lengthy periods of time.
The OS will then page this area out to disk. When a tenured garbage 
collection occurs the tenured area has to be pulled from disk. This can 
turn a GC which normally takes 300 milliseconds into a 2 minute process.

Regards,
Brett.



Stefan Reuter wrote:
> Hi,
>
> my first thought when reading this thread was: If it's not broken, don't
> try to fix it. It seems you did not experience any problems with garbage
> collection (though you might run some load tests to verify that in your
> environment). Java is really smart handling small short-lived objects.
> The garbage collector has a young generation (eden space) designed for
> just that purpose.
>
> The next thing that comes to my mind is what you might actually achieve
> by filtering events at the java side. You will still have create some
> strings and build the map representing the events (note that the
> management api makes no guarantees regarding the order of the attributes
> so you have to parse at least until you encounter the "Event: XXX"
> attribute). What you could save is creating the actual event object, but
> well...
>
> I would recommend to really get some numbers using a decent profiler. If
> you can provide something that is convincing we might very well have a
> look at supporting it in future Asterisk-Java releases.
>
> =Stefan
>
>   
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> ------------------------------------------------------------------------
>
> _______________________________________________
> Asterisk-java-users mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/asterisk-java-users
>   

Re: [Asterisk-java-users] Advanced Event Filtering

[Aaron E. <aar...@gm...>]

As a general rule, I try to keep my swap at 0 always. Swap is very
bad, not just for java. :D

On 8/2/07, Brett Sutton <bs...@no...> wrote:
> I would be very surprised if you have any problems with the garbage
> collector.
>
> We have done a lot of work in this area for a large client running a
> very heavily loaded java based web server (tomcat).
> The garbage collector is super fast.
>
> The only problem we ever had was if we let the java app swap memory out
> to the page file.
> This is super dangerous with java.
> [This took us about 6 weeks of heavy diagnosis to discover the root
> cause of, hence the experience with the java GC. This is the only bug
> that I've every resolved by 'touch'.  I happened to put my hand on the
> system unit during garbage collection and felt the drive rumbling :)]
>
> The Tenured area of the garbage collector can grow quite big and remain
> untouch for lengthy periods of time.
> The OS will then page this area out to disk. When a tenured garbage
> collection occurs the tenured area has to be pulled from disk. This can
> turn a GC which normally takes 300 milliseconds into a 2 minute process.
>
> Regards,
> Brett.
>
>
>
> Stefan Reuter wrote:
> > Hi,
> >
> > my first thought when reading this thread was: If it's not broken, don't
> > try to fix it. It seems you did not experience any problems with garbage
> > collection (though you might run some load tests to verify that in your
> > environment). Java is really smart handling small short-lived objects.
> > The garbage collector has a young generation (eden space) designed for
> > just that purpose.
> >
> > The next thing that comes to my mind is what you might actually achieve
> > by filtering events at the java side. You will still have create some
> > strings and build the map representing the events (note that the
> > management api makes no guarantees regarding the order of the attributes
> > so you have to parse at least until you encounter the "Event: XXX"
> > attribute). What you could save is creating the actual event object, but
> > well...
> >
> > I would recommend to really get some numbers using a decent profiler. If
> > you can provide something that is convincing we might very well have a
> > look at supporting it in future Asterisk-Java releases.
> >
> > =Stefan
> >
> >
> > ------------------------------------------------------------------------
> >
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems?  Stop.
> > Now Search log events and configuration files using AJAX and a browser.
> > Download your FREE copy of Splunk now >>  http://get.splunk.com/
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Asterisk-java-users mailing list
> > Ast...@li...
> > https://lists.sourceforge.net/lists/listinfo/asterisk-java-users
> >
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> Asterisk-java-users mailing list
> Ast...@li...
> https://lists.sourceforge.net/lists/listinfo/asterisk-java-users
>