nioframework-discussion — for general discussion around the NIO framework

Re: [Nioframework-discussion] client-server specify end of message bytebuffer

[Klearchos K. <kle...@eu...>]

Hello Ronny,

thank you for answering. I have seen your example but my original problem is
that I wanted to send binary data
like images.

You suggest converting the images to Base64 string?
I will try this as well.

Regards,
Klearchos



----- Original Message ----- 
From: "Ronny Standtke" <Ron...@gm...>
To: <nio...@li...>
Cc: "Klearchos Klearchou" <kle...@eu...>
Sent: Saturday, January 16, 2010 6:38 PM
Subject: Re: [Nioframework-discussion] client-server specify end of message
bytebuffer


> Hi Klearchos,
>
> > I cannot find out how to specify the beginning and the end of one
> >  serialised object that I have send with the client.
>
> You can use the framing transformers for that. FramingOutputForwarder adds
a
> length header to every incoming frame, FramingInputForwarder parses and
> removes this header again. There are even two examples in the library:
>
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/examples/FramingClient.java?revision=203&view=markup
>
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/examples/FramingServer.java?revision=203&view=markup
>
> >  Additionally when the client exits the server raises the following
error.
>
> I tried reproducing the problem with the EchoClient and EchoServer example
by
> killing the EchoClient while being connected to the EchoServer but could
not
> reproduce the Exception. What OS are you using?
>
> >  How should I close the client's connection ?
>
> The exception is not really a problem, the exception handling of the NIO
> library does all the right things for you. Does the Exception still occur
when
> you close the channel before exiting the client? See:
>
http://java.sun.com/javase/6/docs/api/java/nio/channels/spi/AbstractInterruptibleChannel.html#close%28%29
>
> Regards
>
> Ronny
>

Re: [Nioframework-discussion] client-server specify end of message bytebuffer

[Ronny S. <Ron...@gm...>]

Hi Klearchos,

> I cannot find out how to specify the beginning and the end of one
>  serialised object that I have send with the client.

You can use the framing transformers for that. FramingOutputForwarder adds a 
length header to every incoming frame, FramingInputForwarder parses and 
removes this header again. There are even two examples in the library:
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/examples/FramingClient.java?revision=203&view=markup
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/examples/FramingServer.java?revision=203&view=markup

>  Additionally when the client exits the server raises the following error. 

I tried reproducing the problem with the EchoClient and EchoServer example by 
killing the EchoClient while being connected to the EchoServer but could not 
reproduce the Exception. What OS are you using?

>  How should I close the client's connection ?

The exception is not really a problem, the exception handling of the NIO 
library does all the right things for you. Does the Exception still occur when 
you close the channel before exiting the client? See:
http://java.sun.com/javase/6/docs/api/java/nio/channels/spi/AbstractInterruptibleChannel.html#close%28%29

Regards

Ronny

Re: [Nioframework-discussion] Setting connection timeout

[Ronny S. <Ron...@gm...>]

Hi John

Sorry for the terrible delay and a happy 2010!

> One question. If I try and make a connection to a non existent host, then I
> first get a  java.net.NoRouteToHostException: No route to host exception
>  and then shortly after a java.net.ConnectException: Connection timeout.

I tried to reproduce this case in a unit test here:
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/test/ch/unifr/nio/framework/ResolveFailedTest.java?view=markup
but everything worked like expected.

Could you please provide a code snippet or (even better) a unit test for your 
case?

Ronny

[Nioframework-discussion] client-server specify end of message bytebuffer

[Klearchos K. <kle...@eu...>]

Hello to NIO Users,

I was trying to find a quick way to stream data between several machines until finally to write them to 
the destination.

I read your examples.
I mainly used the NonBlockingEchoClient and the EchoServer.

I cannot find out how to specify the beginning and the end of one serialised object that I have send with the client.
Additionally when the client exits the server raises the following error.
How should I close the client's connection ?

Dec 22, 2009 2:17:20 PM ch.unifr.nio.framework.FrameworkTools handleStackTrace
WARNING: java.io.IOException: An existing connection was forcibly closed by the
remote host
        at sun.nio.ch.SocketDispatcher.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:25)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233)
        at sun.nio.ch.IOUtil.read(IOUtil.java:200)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
        at ch.unifr.nio.framework.transform.ChannelReader.read(ChannelReader.jav
a:114)
        at ch.unifr.nio.framework.HandlerAdapter.run(HandlerAdapter.java:149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:885)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:907)
        at java.lang.Thread.run(Thread.java:619)

Exception on channel: java.io.IOException: An existing connection was forcibly c
losed by the remote host

Thank you in advance

Re: [Nioframework-discussion] Setting connection timeout

[John P. <byh...@gm...>]

Thanks. I had arrived at the same looking into the codebase. Thanks man.

One question. If I try and make a connection to a non existent host, then I
first get a  java.net.NoRouteToHostException: No route to host exception and
then shortly after a java.net.ConnectException: Connection timeout.

 In my code which is a blocking client to a MPD server, I already abort the
operation to send some data to the server. My question is: is there
something that I should do in connectFailed to clear any pending operations,
so that when I next try a connection there is not some lingering state.

John


On Sat, Oct 10, 2009 at 12:13 PM, Ronny Standtke <Ron...@gm...>wrote:

> Hi John
>
> > I've recently started back using the framework and is everthing is going
> > ok, but I need to set the connection timeout  and can't seem to get it
> > right.
>
> Please take a look at the non-blocking example here:
> http://nioframework.sourceforge.net/?q=node/12
> You can enforce a connection timeout by using the version of
> registerClientSocketChannelHandler() with the timeout parameter (line 38 in
> the example is without the timeout parameter).
>
> When you use the timeout parameter, the Dispatcher runs an internal
> TimeoutHandler that calls ClientSocketChannelHandler.connectFailed() when
> the
> time runs out.
>
> > channel.socket().setSoTimeout(5000);
>
> This should not be necessary as all operations after connecting to the
> socket
> are non-blocking.
>
> Please let me know if this information was helpful for you.
>
> > P.S. The library looks like its coming on well. keep up the good work.
>
> Thanks! :-)
>
> Ronny
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Nioframework-discussion mailing list
> Nio...@li...
> https://lists.sourceforge.net/lists/listinfo/nioframework-discussion
>

Re: [Nioframework-discussion] Setting connection timeout

[Ronny S. <Ron...@gm...>]

Hi John

> I've recently started back using the framework and is everthing is going
> ok, but I need to set the connection timeout  and can't seem to get it
> right.

Please take a look at the non-blocking example here:
http://nioframework.sourceforge.net/?q=node/12
You can enforce a connection timeout by using the version of 
registerClientSocketChannelHandler() with the timeout parameter (line 38 in 
the example is without the timeout parameter).

When you use the timeout parameter, the Dispatcher runs an internal 
TimeoutHandler that calls ClientSocketChannelHandler.connectFailed() when the 
time runs out.

> channel.socket().setSoTimeout(5000);

This should not be necessary as all operations after connecting to the socket 
are non-blocking.

Please let me know if this information was helpful for you.

> P.S. The library looks like its coming on well. keep up the good work.

Thanks! :-)

Ronny

[Nioframework-discussion] Setting connection timeout

[John P. <byh...@gm...>]

I've recently started back using the framework and is everthing is going ok,
but I need to set the connection timeout  and can't seem to get it right.

SocketChannel channel = SocketChannel.open(socketAddress);
channel.socket().setSoTimeout(5000);
...

The code above does not change the connection timeout from about a minute or
more to 5 seconds.

What am I missing?

John
P.S. The library looks like its coming on well. keep up the good work.

Re: [Nioframework-discussion] Minor bug in ChannelReader.java - can't be reused after close.

[Ronny S. <Ron...@gm...>]

Hi Richard

Sorry for the terrible delay, the last four weeks where horribly packed with 
other things...

> The boolean closed takes the default value of false, and is set true
> when fill reads -1. There is however no point in the code where it is
> set false again, when in fact the queue can legitimately and
> successfully be re-used. I believe setChannel() should mark this.closed
> = false.

Fixed in svn. Thanks again for the hint.

Best regards

Ronny

Re: [Nioframework-discussion] Minor bug in ChannelReader.java - can't be reused after close.

[Ronny S. <Ron...@gm...>]

Hi Richard

Sorry for the late response and thank you for the hint. I will take a look at 
it ASAP.

Regards

Ronny

[Nioframework-discussion] Minor bug in ChannelReader.java - can't be reused after close.

[Ridgway, R. \(London\) <Ric...@ml...>]

Hi all,

I have to admit I am using an old version of the code (Taking the 'if it
isn't broke, don't fix it' approach!), but I found a bug in
ByteBufferInputQueue.java. I took a quick look at the latest code from
source forge, and believe this bug is still present, but renamed and
repackaged into ChannelReader.java.

The boolean closed takes the default value of false, and is set true
when fill reads -1. There is however no point in the code where it is
set false again, when in fact the queue can legitimately and
successfully be re-used. I believe setChannel() should mark this.closed
= false.


Thanks again for the code Ronny

Regards,

Richard

--------------------------------------------------------------------------
This message w/attachments (message) may be privileged, confidential or proprietary, and if you are not an intended recipient, please notify the sender, do not use or share it and delete it. Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Merrill Lynch. Subject to applicable law, Merrill Lynch may monitor, review and retain e-communications (EC) traveling through its networks/systems. The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or error-free. References to "Merrill Lynch" are references to any company in the Merrill Lynch & Co., Inc. group of companies, which are wholly-owned by Bank of America Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a Condition to Any Banking Service or Activity * Are Not Insured by Any Federal Government Agency. Attachments that are part of this E-communication may have additional important disclosures and disclaimers, which you should read. This message is subject to terms available at the following link: http://www.ml.com/e-communications_terms/. By messaging with Merrill Lynch you consent to the foregoing.
--------------------------------------------------------------------------
 

[Nioframework-discussion] v1.1beta

[Ronny S. <Ron...@gm...>]

Hi all,

I just released v1.1beta. So now you can try out the new examples on our 
website with a released version of the NIO Framework.

Happy testing!

Ronny

Re: [Nioframework-discussion] IOException

[Ronny S. <Ron...@gm...>]

Hi Brian,

> Unfortunately, I can't seem to catch the exception in my ChannelHandler. I
> have  public void channelException(Exception exception)  overridden but it
> doesn't get called.

Very strange. If you take a look at the sources of the v1.0 release:
http://nioframework.svn.sourceforge.net/viewvc/nioframework/tags/NIO_Framework_v1.0/src/ch/unifr/nio/framework/HandlerAdapter.java?view=markup
you can see that in line 173 we call "channelHandler.channelException(ex)" so 
the call should be made!?

> Any idea of how I can catch this exception so I can 
> cleanup my internal data structures (e.g. I have data structures that will
> tell me who is currently connected, without this cleanup, I have a bunch of
> ghosts)

At first we should understand why channelException() is not called in your 
application. Can you add a logger to your application? Something like this:

Logger nioLogger = Logger.getLogger("ch.unifr.nio.framework");
nioLogger.setLevel(Level.FINEST);
// append log to a rotating set of 2 files of max 50 MByte each
FileHandler fileHandler = new FileHandler(
	"%h/niotest.log", 52428800, 2, true);
fileHandler.setFormatter(NIOFormatter.getInstance());
nioLogger.addHandler(fileHandler);

This will produce very detailled log files of the NIO Framework operations in 
your home directory. Could you try this and take a look at the log file? 
Maybe you can also add a logger call to your implementation of 
channelException() so that it can be spotted in the log file.

Thank you and a Happy New Year

Ronny

[Nioframework-discussion] IOException

[Brian G. <br...@br...>]

Hi Guys,

I get the following error occurring quite frequently on a highly trafficked server:

Exception on channel: java.io.IOException: Connection reset by peer
Dec 28, 2008 8:18:50 PM ch.unifr.nio.framework.FrameworkTools handleStackTrace
WARNING: java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcher.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(Unknown Source)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
        at sun.nio.ch.IOUtil.read(Unknown Source)
        at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
        at ch.unifr.nio.framework.transform.ChannelReader.read(ChannelReader.java:110)
        at ch.unifr.nio.framework.HandlerAdapter.run(HandlerAdapter.java:148)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

I believe this is just the client disconnecting for some unknown reason. Which is fine, except I need to do some cleanup in my ChannelHandler. Unfortunately, I can't seem to catch the exception in my ChannelHandler. I have  public void channelException(Exception exception)  overridden but it doesn't get called. Any idea of how I can catch this exception so I can cleanup my internal data structures (e.g. I have data structures that will tell me who is currently connected, without this cleanup, I have a bunch of ghosts)


Thanks,

Brian Gruber
www.populatetheweb.com<http://www.populatetheweb.com>

Re: [Nioframework-discussion] How to reconnect base on NonBlockingEchoClient

[Ronny S. <Ron...@gm...>]

Hi Eva,

> It works,but still have problems.
>  It works when server terminates the connection and inputClosed() is
> called. But it does't work when client close the channel on it's own. It
> seems the ErraticNonBlockingEchoClient class doesn't catch the event of the
> connection finishing. So the client just close the channel in
> TroubleMaker class but never reconnect.

This is because IP is packet oriented. You only notice that a connection is 
down when you actually try to send a package. Try running the 
ErraticNonBlockingEchoClient and input some strings right after the internal 
TroubleMaker closed the channel. Then you can observe how channelException() 
reconnects.

Greetings

Ronny

Re: [Nioframework-discussion] How to reconnect base on NonBlockingEchoClient

[wing e. <wi...@gm...>]

Hi Ronny,


> Hi Eva,
>
>> In my project, I need to reconnect the server when
>> inputClosed/channelException happens. I tried closing the channel and
>> socket then connecting again, but it didn't work. I tried
>> registerClientSocketChannelHandler too, it didn't work either.
>>
>> What I do now is new another NonBlockingEchoClient but with the
>> previous dispather and other stuffs remaining, which means those
>> stuffs wont disapear until the application exits . So if it cant't
>> reconnect, NonBlockingEchoClient seems useless for me.
>
> I just added another example that shows how to do all that. It has a private
> class "TroubleMaker" that constantly closes the channel. Please update to the
> latest svn trunk and take a look at
> src/ch/unifr/nio/framework/examples/ErraticNonBlockingEchoClient.java
>
> Alternatively, you can browse the source here:
> http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/examples/ErraticNonBlockingEchoClient.java?view=markup
>
> The basic changes are:
> - make the Dispatcher and the OutputHandler member fields
> - interrupt running OutputHandler when reconnecting
> - listen to interrupts in the endless while loop of OutputHandler
> - store the last user input so that it can be resent in case of interrrupt
>
>
> Please let us know whether this example was helpful or not.
>
> Regards
>
> Ronny
>


Thx for your works, .

It works,but still have problems.
 It works when server terminates the connection and inputClosed() is called.
But it does't work when client close the channel on it's own. It seems
the ErraticNonBlockingEchoClient class doesn't catch the event of the
connection finishing. So the client just close the channel in
TroubleMaker class but never reconnect.
Anyway, I found the reason why I failed to reconnect: I didnt cancel
the key! - -

Regards

Eva

Re: [Nioframework-discussion] How to reconnect base on NonBlockingEchoClient

[Ronny S. <Ron...@gm...>]

Hi Eva,

> In my project, I need to reconnect the server when
> inputClosed/channelException happens. I tried closing the channel and
> socket then connecting again, but it didn't work. I tried
> registerClientSocketChannelHandler too, it didn't work either.
>
> What I do now is new another NonBlockingEchoClient but with the
> previous dispather and other stuffs remaining, which means those
> stuffs wont disapear until the application exits . So if it cant't
> reconnect, NonBlockingEchoClient seems useless for me.

I just added another example that shows how to do all that. It has a private 
class "TroubleMaker" that constantly closes the channel. Please update to the 
latest svn trunk and take a look at 
src/ch/unifr/nio/framework/examples/ErraticNonBlockingEchoClient.java

Alternatively, you can browse the source here:
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/examples/ErraticNonBlockingEchoClient.java?view=markup

The basic changes are:
- make the Dispatcher and the OutputHandler member fields
- interrupt running OutputHandler when reconnecting
- listen to interrupts in the endless while loop of OutputHandler
- store the last user input so that it can be resent in case of interrrupt


Please let us know whether this example was helpful or not.

Regards

Ronny

Re: [Nioframework-discussion] Methods for handling message correctly

[Ronny S. <Ron...@gm...>]

> > and getStringByDelimiter(String delimiter) methods
>
> Shouldn't this functionality be implemented in a separate transformer? This
> functionality may be needed in a transformation chain, even when not
> transforming ByteBuffers to Strings. Something like:
>
>   StringSplitTransformer extends AbstractTransformer<String, String>
>
> Should be very simple. Do you feel like implementing it? :-)

Yesterday evening I added a new Transformer to split strings. The source is 
here:
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/transform/SplitStringTransformer.java?view=markup

Please take a look at it, try it out and tell me if it fits your needs.

Regards

Ronny

Re: [Nioframework-discussion] Methods for handling message correctly

[Ronny S. <Ron...@gm...>]

Hi Eva

Thank you very much for your feedback.

> I think it's necessary to add setEncoding(String encoding)

I just added support for charsets in both the ByteBufferToStringTransformer 
and StringToByteBufferTransformer. Please, take a look at it here:
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/transform/ByteBufferToStringTransformer.java?view=markup
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/transform/StringToByteBufferTransformer.java?view=markup

You can also just download svn trunk to test it out.

> and getStringByDelimiter(String delimiter) methods 

Shouldn't this functionality be implemented in a separate transformer? This 
functionality may be needed in a transformation chain, even when not 
transforming ByteBuffers to Strings. Something like:

  StringSplitTransformer extends AbstractTransformer<String, String>

Should be very simple. Do you feel like implementing it? :-)

Sorry, I didn't have enough time to look into your reconnect problem. I will 
try in the next couple of days.

Regards

Ronny

[Nioframework-discussion] How to reconnect base on NonBlockingEchoClient

[wing e. <wi...@gm...>]

Hi,all.
In my project, I need to reconnect the server when
inputClosed/channelException happens. I tried closing the channel and
socket then connecting again, but it didn't work. I tried
registerClientSocketChannelHandler too, it didn't work either.

What I do now is new another NonBlockingEchoClient but with the
previous dispather and other stuffs remaining, which means those
stuffs wont disapear until the application exits . So if it cant't
reconnect, NonBlockingEchoClient seems useless for me.

[Nioframework-discussion] Methods for handling message correctly

[wing e. <wi...@gm...>]

Hi,Ronny.
I think it's necessary to add setEncoding(String encoding)  and
getStringByDelimiter(String delimiter) methods to string or bytebuffer
transformer classes, to handle different encoding problem between
client and server and completeness of application protocol messages.

[Nioframework-discussion] Methods for handling message correctly

[wing e. <wi...@gm...>]

Hi,Ronny.
I think it's necessary to add setEncoding(String encoding)  and
getStringByDelimiter(String delimiter) methods to string or bytebuffer
transformer classes, to handle different encoding problem between
client and server and completeness of application protocol messages.

[Nioframework-discussion] Methods for handling message correctly

[wing e. <wi...@gm...>]

Hi,Ronny.
I think it's necessary to add setEncoding(String encoding)  and
getStringByDelimiter(String delimiter) methods to string or bytebuffer
transformer classes, to handle different encoding problem between
client and server and completeness of application protocol messages.

[Nioframework-discussion] methods for handling message correctly

[wing e. <wi...@gm...>]

It's necessary to add setEncoding(String encoding)  and
getStringByDelimiter(String delimiter) methods to string or bytebuffer
transformer classes.

[Nioframework-discussion] methods for handling message correctly

[wing e. <wi...@gm...>]

It's necessary to add setEncoding(String encoding)  and
getStringByDelimiter(String delimiter) methods to string or bytebuffer
transformer classes.

Re: [Nioframework-discussion] Thread Safety (Static Data Structures)

[Ronny S. <Ron...@gm...>]

Hi Brian,

> 1. If I want to have one client send a message to another connected client,
> I realize that I need to have some sort of global (static) data structure
> that holds a list of connected clients. Better yet, just a reference to the
> connected client's ChannelWriter objects.

I would provide a sendMessage() method at the client's ChannelHandler and 
store all ChannelHandler references in a global data structure. This is more 
encapsulated because this way you can add and change transformations "in 
front" of the ChannelWriters and ChannelReaders without touching the rest of 
your code. 

> 3. Then if a client sends some data, I will handle it under the 
> handleInput() of my ChannelHandler Hopefully, so far so good.

In the upcoming Version 1.1 of the NIO Framework handleInput() will be 
replaced by customized transformation classes. See here for a simple example:
http://nioframework.svn.sourceforge.net/viewvc/nioframework/trunk/src/ch/unifr/nio/framework/examples/EchoServer.java?view=markup
This is more consistent and removes some problems with the previous approach, 
e.g. if you have SSL transformations between your socket and your application 
handleInput() was called even when there was just some handshake data 
exchanged and no input available for the application...

> 4. Under the Classic Multi-threaded approach, I would usually synchronize
> access to the static HashMap , but, do I need to do that here?

The NIO Framework is internally also multi-threaded. Yes, there is only a 
single selector thread in the Dispatcher but all the real work is then 
offloaded to an Executor which uses by default a cached thread pool. This way 
we automatically scale on multi-core systems. So, you should also syncronize 
access to your global data structure or just use a concurrent collection 
class, e.g. ConcurrentHashMap.

Greetings

Ronny