xsocket-develop — xSocket Support Forum

Re: [xSocket-develop] Simple Server Sometimes Seeing Data Corruption on Client Side

[Gregor R. <gre...@go...>]

Hi Rod,

would you please submit this as a bug (
http://sourceforge.net/tracker/?atid=850942&group_id=169583). Please also
add a junit test, which contains the essential xSocket-related part of your
client and server-side code.

Thanks
Gregor

2008/5/3 Rod Magnuson <rod...@gm...>:

> I've got a simple server that sends/receives null terminated Strings
> to/from a flash based client.
>
> Flush mode is async.
>
> I'm using the 2.0 final jar.
>
> I'm seeing times when the client is receiving mangled data. The length of
> the data is correct, just some of the contents are wrong. This does not
> happen often and is hard to reproduce. When it does happen other clients
> receive the correct data.
>
> I write out the messages with:
>
>     String END_OF_MESSAGE_DELIM = new String(new char[] { 0 });
>     connection.write(message + END_OF_MESSAGE_DELIM);
>     connection.flush();
>
> It could be either client or server. Ideas on how to disqualify the
> server?
>
> Thanks,
> Rod
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
>
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>
>

[xSocket-develop] Simple Server Sometimes Seeing Data Corruption on Client Side

[Rod M. <rod...@gm...>]

I've got a simple server that sends/receives null terminated Strings to/from
a flash based client.

Flush mode is async.

I'm using the 2.0 final jar.

I'm seeing times when the client is receiving mangled data. The length of
the data is correct, just some of the contents are wrong. This does not
happen often and is hard to reproduce. When it does happen other clients
receive the correct data.

I write out the messages with:

    String END_OF_MESSAGE_DELIM = new String(new char[] { 0 });
    connection.write(message + END_OF_MESSAGE_DELIM);
    connection.flush();

It could be either client or server. Ideas on how to disqualify the server?

Thanks,
Rod

Re: [xSocket-develop] Too many open files

[Gregor R. <gre...@go...>]

Hi Christiaan,

did you check the behaviour of your application against the 'Too Many Open
Files support pattern' (
http://support.bea.com/application_content/product_portlets/support_patterns/wls/TooManyOpenFilesPattern.html).
If you don't use persistent connections, this could be the reason for the
problem.

Gregor



2008/4/28 Christiaan Willemsen <cwi...@te...>:

> Hi There,
>
> I have an application that can do about 20 simultanious socket
> connections to a http server.
>
> I handle the http stuff myself. This works fine for a while, but then I
> get these exceptions:
>
> 27286 java.net.SocketException: Too many open files
> 27287         at sun.nio.ch.Net.socket0(Native Method)
> 27288         at sun.nio.ch.Net.socket(Net.java:94)
> 27289         at
> sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:84)
> 27290         at
>
> sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:37)
> 27291         at
> java.nio.channels.SocketChannel.open(SocketChannel.java:105)
> 27292         at
>
> org.xsocket.connection.spi.DefaultIoProvider.openSocket(DefaultIoProvider.java:522)
> 27293         at
>
> org.xsocket.connection.spi.DefaultIoProvider.createClientIoHandler(DefaultIoProvider.java:358)
> 27294         at
>
> org.xsocket.connection.NonBlockingConnection.createClientIoHandler(NonBlockingConnection.java:1181)
> 27295         at
>
> org.xsocket.connection.NonBlockingConnection.<init>(NonBlockingConnection.java:522)
> 27296         at
>
> org.xsocket.connection.NonBlockingConnection.<init>(NonBlockingConnection.java:510)
> 27297         at
>
> org.xsocket.connection.NonBlockingConnection.<init>(NonBlockingConnection.java:326)
> 27298         at
>
> com.technocon.controlserver.util.location.Map24LocationProviderHTTP.getLocation(Map24LocationProviderHTTP.java:86)
> 27299         at
>
> com.technocon.controlserver.backend.udpserver.events.Logpoint.readData(Logpoint.java:83)
> 27300         at
>
> com.technocon.controlserver.backend.udpserver.events.Logpoint.<init>(Logpoint.java:50)
> 27301         at
>
> com.technocon.controlserver.backend.udpserver.UDPEventFactory.composeEvent(UDPEventFactory.java:61)
> 27302         at
>
> com.technocon.controlserver.backend.udpserver.DatagramPackets.extractPackets(DatagramPackets.java:58)
> 27303         at
>
> com.technocon.controlserver.backend.udpserver.DatagramPackets.<init>(DatagramPackets.java:39)
> 27304         at
>
> com.technocon.controlserver.backend.udpserver.UDPBackend.onDatagram(UDPBackend.java:212)
> 27305         at
>
> org.xsocket.datagram.AbstractEndpoint$HandlerProcessor.run(AbstractEndpoint.java:329)
> 27306         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
> 27307         at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
> 27308         at java.lang.Thread.run(Thread.java:619)
>
> I have no clue why this happens, since there are by for not enough
> sockets in use to be able to account for this error
>
> I'm using JDK 6.0 and xsocket 2.0
>
> --
> Christiaan Willemsen
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
>
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>

[xSocket-develop] Too many open files

[Christiaan W. <cwi...@te...>]

Hi There,

I have an application that can do about 20 simultanious socket 
connections to a http server.

I handle the http stuff myself. This works fine for a while, but then I 
get these exceptions:

27286 java.net.SocketException: Too many open files
27287         at sun.nio.ch.Net.socket0(Native Method)
27288         at sun.nio.ch.Net.socket(Net.java:94)
27289         at 
sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:84)
27290         at 
sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:37)
27291         at 
java.nio.channels.SocketChannel.open(SocketChannel.java:105)
27292         at 
org.xsocket.connection.spi.DefaultIoProvider.openSocket(DefaultIoProvider.java:522)
27293         at 
org.xsocket.connection.spi.DefaultIoProvider.createClientIoHandler(DefaultIoProvider.java:358)
27294         at 
org.xsocket.connection.NonBlockingConnection.createClientIoHandler(NonBlockingConnection.java:1181)
27295         at 
org.xsocket.connection.NonBlockingConnection.<init>(NonBlockingConnection.java:522)
27296         at 
org.xsocket.connection.NonBlockingConnection.<init>(NonBlockingConnection.java:510)
27297         at 
org.xsocket.connection.NonBlockingConnection.<init>(NonBlockingConnection.java:326)
27298         at 
com.technocon.controlserver.util.location.Map24LocationProviderHTTP.getLocation(Map24LocationProviderHTTP.java:86)
27299         at 
com.technocon.controlserver.backend.udpserver.events.Logpoint.readData(Logpoint.java:83)
27300         at 
com.technocon.controlserver.backend.udpserver.events.Logpoint.<init>(Logpoint.java:50)
27301         at 
com.technocon.controlserver.backend.udpserver.UDPEventFactory.composeEvent(UDPEventFactory.java:61)
27302         at 
com.technocon.controlserver.backend.udpserver.DatagramPackets.extractPackets(DatagramPackets.java:58)
27303         at 
com.technocon.controlserver.backend.udpserver.DatagramPackets.<init>(DatagramPackets.java:39)
27304         at 
com.technocon.controlserver.backend.udpserver.UDPBackend.onDatagram(UDPBackend.java:212)
27305         at 
org.xsocket.datagram.AbstractEndpoint$HandlerProcessor.run(AbstractEndpoint.java:329)
27306         at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
27307         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
27308         at java.lang.Thread.run(Thread.java:619)

I have no clue why this happens, since there are by for not enough 
sockets in use to be able to account for this error

I'm using JDK 6.0 and xsocket 2.0

-- 
Christiaan Willemsen

[xSocket-develop] xSocket V2.0 final and xSocket-http, xSocket-multiplexed released

[Gregor R. <gre...@go...>]

Hi,

xSocket V2.0 has been released. Base on the new core release, the HTTP
extension module and the multiplexed extension module has been updated (and
fixed).

xSocket V2.0. Issues:
 * Deprecated artefacts have been removed and the API has been cleaned
 * Significant performance improvements (low level NIO classes have been
reimplemented)
* The Low-level provider SPI has been removed
 * New configuration features
 * Fine granularly thread management (on call back method level)


Gregor

Re: [xSocket-develop] Proxy example - piping data to file

[<gre...@we...>]

Hi Martin,

the HttpServer.setWriteTransferRate() is a unsupported method. 

Currently throttling is not supported. A built-in throttling support should be available in future versions. Please file a feature request, with your requirements (http://sourceforge.net/tracker/?atid=850945&group_id=169583).

Thanks
Gregor

> -----Ursprüngliche Nachricht-----
> Von: "Martin Thelian" <Mar...@gm...>
> Gesendet: 25.04.08 08:00:50
> An: gre...@we...
> Betreff: Re: [xSocket-develop] Proxy example - piping data to file

Hi,
> 
> does configuring the transfer rate via HttpServer.setWriteTransferRate(
> ) work with this approach? I've tried to limit the transfer rate to 
> 5KB/s but currently my test-download uses 200KB/s.
> 
> Martin
> 
> gre...@we... schrieb:Hi Martin, the xSocket Junit test 
> BodyForwardingTest (http://xsocket.svn.sourceforge.net/viewvc/xsocket/
> xsocket/http/tags/V2_0_alpha4/src/test/java/org/xsocket/connection/
> http/BodyForwardingTest.java) contains a ForwardHandler which shows 
> you how to control the message/body streaming. One fast way to fetch 
> the complete message header is to call the toString method ... 
> HttpResponseHeader resHdr = response.getResponseHeader(); String 
> hdrCopy = resHdr.toString(); FileChannel fileChannel = new .... 
> fileChannel.write(...) ... The body data can be fetched by duplicating 
> the body ByteBuffer : ... ByteBuffer[] data = bodyDataSource.
> readByteBufferByLength(available); for (ByteBuffer buf : data) { 
> fileChannel.write(buf.duplicate()); } dataSink.write(data); ... Take 
> care with the NonThreaded annotations of the example code. If you are 
> writing to a file, the handler should run in a multithreaded mode. 
> The Nonthreaded annotation should be removed. Gregor -----Ursprüngliche
>  Nachricht----- Von: "Martin Thelian" <Mar...@gm...>Gesendet:
> 10.04.08 09:56:16 An: xso...@li...: [
> xSocket-develop] Proxy example - piping data to file Hi, I would like 
> to use xsocket as described in [1] chapter "3.2 Reusing message 
> objects - a proxy example" and additionally I would like to pipe the 
> response-message into a file while transfering it back to the client. 
> What is the best way to do this? Thanks, Martin [1] http://xsocket.
> sourceforge.net/http/tutorial/V2/TutorialHttp.htm-- Psssst! Schon vom 
> neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.
> net/de/go/multimessenger----------------------------------------------
> --------------------------- This SF.net email is sponsored by the 
> 2008 JavaOne(SM) Conference Don't miss this year's exciting event. 
> There's still time to save $100. Use priority code J8TL2D2. http://ad.
> doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone__
> _____________________________________________ xSocket-develop mailing 
> lis...@li...://lists.sourceforge.
> net/lists/listinfo/xsocket-develop
> 
> 


-- 
Gregor Roth

Re: [xSocket-develop] http listen on port 80 problem?

[Gregor R. <gre...@go...>]

Hi

this is a known, ugly bug, which haven been fixed in the current development
trunk. If the HTTP header Host doesn't contains a port, xSocket-http
2.0-alpha-4 will not enter the onRequest method on the server side:

// doesn't works on the server side for xSocket-http 2.0-alpha-4
GET /Handler.class HTTP/1.1
Host: 192.168.1.90
User-Agent: xSocket-http/2.0-alpha-4

// works on the server side for xSocket-http 2.0-alpha-4
GET /Handler.class HTTP/1.1
Host: 192.168.1.90:80 <http://192.168.1.90/>
User-Agent: xSocket-http/2.0-alpha-4

The new, fixed version xSocket-http 2.0-alpha-5 will be released in 1-2
weeks.

Gregor

2008/4/23 luyongfugx <luy...@16...>:

>
>
>   i use the xsocket http extension,now i have a problem like this:
> i can listen and handle request on any other port but 80,when i listen on
> port 80,
> it can run,but when the request come,the server just don't enter the
> onRequest function,and the client just waiting.
> i have try this on another machine,but the problem remain.can you give me
> a suggestion?
>
>
>
> ------------------------------
> 自信源自成熟 全新换代上市 东风雪铁龙新爱丽舍 <http://popme.163.com/link/004151_0416_9352.html>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
>
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>
>

[xSocket-develop] http listen on port 80 problem?

[luyongfugx <luy...@16...>]

 
 
  
i use the xsocket http extension,now i have a problem like this:
i can listen and handle request on any other port but 80,when i listen on port 80,
it can run,but when the request come,the server just don't enter the onRequest function,and the client just waiting.
i have try this on another machine,but the problem remain.can you give me a suggestion? 
 

Re: [xSocket-develop] suspendRead() and resumeRead() cause program hang?

[Gregor R. <gre...@go...>]

 Hello fchen,

thanks for your investigation. I'm going to cross check this. I saw that
your bug fix is based on xSocket V1.2.x. For maintenance reasons I'm going
to fix the bug.

Currently I try to reproduce our reported bug on xSocket 2.0 (The dispatcher
has been reimplemented by V2.0). The final version of xSocket 2.0 will be
released in 1-2 weeks.

Gregor


2008/4/22 fulichen <ful...@du...>:

>  Hello Gregor,
>
> Thanks for your response, but we find suspendRead() and resumeRead() will
> cause program hang, is it a bug (maybe there is race condition)? or how can
> we fix our test?
>
> The test program is following, and the test environment Java version
> 1.6.0_03 on Windows Vista.
> TestClient result hanging  a while after start (but not each TestClient
> test, try some times will get hang)
>
> =================================
> public class TestClient implements IDataHandler
> {
>  //-------------------------------
>  static public void main(String[] args)throws IOException,
> InterruptedException
>  {
>   TestClient c = new TestClient();
>   NonBlockingConnection nbc = new NonBlockingConnection("127.0.0.1", 1000,
> c);
>   Thread.sleep(100000000);
>  }
>  public boolean onData(INonBlockingConnection connection)throws
> IOException
>  {
>   System.out.println("onData enter");
>   try
>   {
>    connection.suspendRead(); // here call to suspendRead
>    int available = connection.available();
>    System.out.println("onData available = " + available);
>    byte[] data = connection.readBytesByLength(available);
>   }
>   finally
>   {
>    connection.resumeRead(); // call resumeRead here
>    System.out.println("onData exit");
>   }
>   return true;
>  }
> }
> =================================
> public class TestServer
> {
>  //-------------------------------
>  static public void main(String[] args)throws IOException
>  {
>   TestServer s = new TestServer();
>   s.doServer();
>  }
>  void doServer() throws IOException
>  {
>   ServerSocket ssck = new ServerSocket(1000);
>   Socket sck = ssck.accept();
>   OutputStream out = new BufferedOutputStream(sck.getOutputStream());
>   for (int i = 0; i < 100000000; i++)
>   {
>    String s = "Message " + i + "\r\n";
>    //1.write out a string
>    out.write(s.getBytes());
>    //2.write out bytes length
>    ByteBuffer buffer = ByteBuffer.allocate(4);
>    buffer.putInt(i);
>    buffer.rewind();
>    out.write(buffer.getInt());
>    //3.write out bytes
>    byte[] data = new byte[i];
>    out.write(data);
>    out.flush();
>   }
>  }
> }
> ========================================
> TestClient result
> onData enter
> onData available = 10027
> onData exit
> onData enter
> onData available = 16
> onData exit
> onData enter
> onData available = 3752
> onData exit
> onData enter
> onData available = 10044
> onData exit
> onData enter
> onData available = 3752
> onData exit
> ----- Original Message ----- From: <gre...@we...>
> To: "fulichen" <ful...@du...>
> Sent: Monday, April 21, 2008 7:21 PM
> Subject: Re: [xSocket-develop] OutOfMemory exception
>
> Hi fchen,
>
> by calling the <connection>.suspendRead() you can stop reading the socket.
> Reading will be resumed by calling the <connection>.reaumeRead() method. For
> an example see
> http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/LimitedReadTest.java?view=markup
>
> Gregor
>
>
> > -----Ursprgliche Nachricht-----
> > Von: "fulichen" <ful...@du...>
> > Gesendet: 21.04.08 12:35:09
> > An: <xso...@li...>
> > Betreff: [xSocket-develop] OutOfMemory exception
>
> Our application need to handle many many connection,
> > and the data need time to process,
> > because xSocket Dispatcher read data when it is avialable,
> > our application often run out of memory.
> >
> > How we limit Dispatcher data reading, or what can we do to prevent
> > OutOfMemoryException?
> >
> > fchen-----------------------------------------------------------------
> > -------- This SF.net email is sponsored by the 2008 JavaOne(SM)
> > Conference Don't miss this year's exciting event. There's still time
> > to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/
> > clk;198757673;13503038;p?http://java.sun.com/javaone__________________
> > _____________________________ xSocket-develop mailing list xSocket-
> > de...@li... https://lists.sourceforge.net/lists/
> >
> > listinfo/xsocket-develop
>
>
>
> --
> Gregor Roth
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
>
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>
>

Re: [xSocket-develop] suspendRead() and resumeRead() cause program hang?

[fulichen <ful...@du...>]

Hi Gregor,

Thanks for your quickly response, and we find the bug occur from the race condition among IoSocketHandler's 3 methods(onWriteableEvent(), suspendRead(), resumeRead()), could we synchronized these 3 methods to resolve this probem like the following code? (but this approach maybe loose some performace)

fchen.

====================================================================
 Object suspendReadLocker = new Object(); // new data member for suspendRead
 
 int onWriteableEvent() throws IOException {
  assert (IoSocketDispatcher.isDispatcherThread());

  int sent = 0;

  synchronized(suspendReadLocker) { // synchronized with suspendReadLocker
   if (suspendRead) {
    if (LOG.isLoggable(Level.FINEST)) {
     LOG.finest("[" + getId() + "] writeable event occured. update interested to none (because suspendRead is set) and write data to socket");
    }
    updateInterestedSetNonen();
 
   } else {
//   if (LOG.isLoggable(Level.FINEST)) {
//    LOG.finest("[" + getId() + "] writeable event occured. update interested to read and write data to socket");
//   }
    updateInterestedSetRead();
   }
  }
  ...
  ...
 }

 public void suspendRead() throws IOException {
  synchronized(suspendReadLocker) { // synchronized with suspendReadLocker
   suspendRead = true;
   // update to write (why?). Reason:
   //  * avoid race conditions in which current write need will be swallowed
   //  * write falls back to `none interested set`
   updateInterestedSetWrite();
  }
 }
 
 public void resumeRead() throws IOException {
  synchronized(suspendReadLocker) { // synchronized with suspendReadLocker
   if (suspendRead) {
    suspendRead = false;
 
    // update to write (why not read?). Reason:
    //  * avoid race conditions in which current write need will be swallowed
    //  * write falls back to `read interested set` if there is no data to write
    updateInterestedSetWrite();
   }
  }
 }


----- Original Message ----- 
From: <gre...@we...>
To: <xso...@li...>
Cc: "fulichen" <ful...@du...>
Sent: Tuesday, April 22, 2008 3:10 PM
Subject: Re: [xSocket-develop] suspendRead() and resumeRead() cause program hang?


Hi fchen,

the problem here is, that the read buffer size will be managed by the application specific app handler (worker thread) by using the suspend an resume read methods. The socket will be read by a dedicated IO-Thread (Dispacher-Thread). For this reason race conditions exists, which can cause  memory problems in your use case.

My suggested approach doesn’t work good to limit the read buffer size. Sorry. The suspend and resume read method hasn’t been introduced for such a use case. 

I introduced a new parameter MaxReadBufferThreshold. This parameter will be managed by calling the <connection>.setMaxReadBufferThreshold(...) method. If the xSocket connection’s internal app-level read buffer exceeds this size the connection will stop reading. This will be controlled within the IO-Thread. Please note this is a threshold value. By exceeding this threshold the read will be stopped, which means the read buffer size will exceed this threshold (max size = maxReadBufferThreshold + socket read buffer size  * 2).

For an example see http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/LimitedReadTest.java?view=markup. I also attached a bin jar of the current snapshot.

Gregor 





> -----Ursprüngliche Nachricht-----
> Von: "fulichen" <ful...@du...>
> Gesendet: 22.04.08 03:57:22
> An: <xso...@li...>
> Betreff: [xSocket-develop] suspendRead() and resumeRead() cause program hang?


> Hello Gregor,
> 
> Thanks for your response, but we find suspendRead() and resumeRead() 
> will cause program hang, is it a bug (maybe there is race condition)? 
> or how can we fix our test?
> 
> The test program is following, and the test environment Java version 
> 1.6.0_03 on Windows Vista.
> TestClient result hanging a while after start (but not each 
> TestClient test, try some times will get hang)
> 
> =================================
> public class TestClient implements IDataHandler
> {
>  //-------------------------------
>  static public void main(String[] args)throws IOException, InterruptedE
> xception
>  {
>  TestClient c = new TestClient();
>  NonBlockingConnection nbc = new NonBlockingConnection("127.0.0.1", 
> 1000, c);
>  Thread.sleep(100000000);
>  }
>  public boolean onData(INonBlockingConnection connection)throws 
> IOException
>  {
>  System.out.println("onData enter");
>  try
>  {
>  connection.suspendRead(); // here call to suspendRead
>  int available = connection.available();
>  System.out.println("onData available = " + available);
>  byte[] data = connection.readBytesByLength(available);
>  }
>  finally
>  {
>  connection.resumeRead(); // call resumeRead here
>  System.out.println("onData exit");
>  }
>  return true;
>  }
> }
> =================================
> public class TestServer
> {
>  //-------------------------------
>  static public void main(String[] args)throws IOException
>  {
>  TestServer s = new TestServer();
>  s.doServer();
>  }
>  void doServer() throws IOException
>  {
>  ServerSocket ssck = new ServerSocket(1000);
>  Socket sck = ssck.accept();
>  OutputStream out = new BufferedOutputStream(sck.getOutputStream());
>  for (int i = 0; i < 100000000; i++)
>  {
>  String s = "Message " + i + "\r\n";
>  //1.write out a string
>  out.write(s.getBytes());
>  //2.write out bytes length
>  ByteBuffer buffer = ByteBuffer.allocate(4);
>  buffer.putInt(i);
>  buffer.rewind();
>  out.write(buffer.getInt());
>  //3.write out bytes
>  byte[] data = new byte[i];
>  out.write(data);
>  out.flush();
>  }
>  }
> }
> ========================================
> TestClient result
> onData enter
> onData available = 10027
> onData exit
> onData enter
> onData available = 16
> onData exit
> onData enter
> onData available = 3752
> onData exit
> onData enter
> onData available = 10044
> onData exit
> onData enter
> onData available = 3752
> onData exit
> 
> ----- Original Message ----- 
> From: <gre...@we...>
> To: "fulichen" <ful...@du...>
> Sent: Monday, April 21, 2008 7:21 PM
> Subject: Re: [xSocket-develop] OutOfMemory exception
> 
> Hi fchen,
> 
> by calling the <connection>.suspendRead() you can stop reading the 
> socket. Reading will be resumed by calling the <connection>.
> reaumeRead() method. For an example see http://xsocket.svn.sourceforge.
> net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/
> connection/LimitedReadTest.java?view=markup
> 
> Gregor 
> 
> > -----Ursprî…sgliche Nachricht-----
> > Von: "fulichen" <ful...@du...>
> > Gesendet: 21.04.08 12:35:09
> > An: <xso...@li...>
> > Betreff: [xSocket-develop] OutOfMemory exception
> 
> Our application need to handle many many connection, 
> > and the data need time to process,
> > because xSocket Dispatcher read data when it is avialable, 
> > our application often run out of memory.
> > 
> > How we limit Dispatcher data reading, or what can we do to prevent 
> > OutOfMemoryException?
> > 
> > fchen---------------------------------------------------------------
> --
> > -------- This SF.net email is sponsored by the 2008 JavaOne(SM) 
> > Conference Don't miss this year's exciting event. There's still 
> time 
> > to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/
> > clk;198757673;13503038;p?http://java.sun.com/javaone________________
> __
> > _____________________________ xSocket-develop mailing list xSocket-
> > de...@li...://lists.sourceforge.net/lists/
> > 
> > listinfo/xsocket-develop
> 
> -- 
> Gregor Roth
> ----------------------------------------------------------------------
> --- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $
> 100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;
> 198757673;13503038;p?http://java.sun.com/javaone______________________
> _________________________ xSocket-develop mailing list xSocket-
> de...@li... https://lists.sourceforge.net/lists/
> 
> listinfo/xsocket-develop



-- 
Gregor Roth

Re: [xSocket-develop] xsocket-http Base64 Encoding/Decoding

[<gre...@we...>]

Hi Martin

it’s strange why JSE API doesn’t include a base64 en/decoder (using the sun.misc.BASE64Decoder of Sun's JVM impl. is not recommended). 

xSocket will not implement such a en/decoder. There a plenty of free Base64 decoders. E.g. Apache commons codec (http://commons.apache.org/codec/), which xSocket uses for the unit tests.

Gregor



> -----Ursprüngliche Nachricht-----
> Von: "Martin Thelian" <Mar...@gm...>
> Gesendet: 22.04.08 08:47:31
> An: xso...@li...
> Betreff: [xSocket-develop] xsocket-http Base64 Encoding/Decoding


> Hi,
> 
> I'd like to authenticate a user using basic www-authentication. Does 
> xsocket provide functions for base64 encoding/decoding or do I need to 
> implement it on my own?
> 
> Thanks,
> Martin
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
> 



-- 
Gregor Roth

Re: [xSocket-develop] suspendRead() and resumeRead() cause program hang?

[<gre...@we...>]

Hi fchen,

the problem here is, that the read buffer size will be managed by the application specific app handler (worker thread) by using the suspend an resume read methods. The socket will be read by a dedicated IO-Thread (Dispacher-Thread). For this reason race conditions exists, which can cause  memory problems in your use case.

My suggested approach doesn’t work good to limit the read buffer size. Sorry. The suspend and resume read method hasn’t been introduced for such a use case. 

I introduced a new parameter MaxReadBufferThreshold. This parameter will be managed by calling the <connection>.setMaxReadBufferThreshold(...) method. If the xSocket connection’s internal app-level read buffer exceeds this size the connection will stop reading. This will be controlled within the IO-Thread. Please note this is a threshold value. By exceeding this threshold the read will be stopped, which means the read buffer size will exceed this threshold (max size = maxReadBufferThreshold + socket read buffer size  * 2).

For an example see http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/LimitedReadTest.java?view=markup. I also attached a bin jar of the current snapshot.

Gregor 





> -----Ursprüngliche Nachricht-----
> Von: "fulichen" <ful...@du...>
> Gesendet: 22.04.08 03:57:22
> An: <xso...@li...>
> Betreff: [xSocket-develop] suspendRead() and resumeRead() cause program hang?


> Hello Gregor,
> 
> Thanks for your response, but we find suspendRead() and resumeRead() 
> will cause program hang, is it a bug (maybe there is race condition)? 
> or how can we fix our test?
> 
> The test program is following, and the test environment Java version 
> 1.6.0_03 on Windows Vista.
> TestClient result hanging a while after start (but not each 
> TestClient test, try some times will get hang)
> 
> =================================
> public class TestClient implements IDataHandler
> {
>  //-------------------------------
>  static public void main(String[] args)throws IOException, InterruptedE
> xception
>  {
>  TestClient c = new TestClient();
>  NonBlockingConnection nbc = new NonBlockingConnection("127.0.0.1", 
> 1000, c);
>  Thread.sleep(100000000);
>  }
>  public boolean onData(INonBlockingConnection connection)throws 
> IOException
>  {
>  System.out.println("onData enter");
>  try
>  {
>  connection.suspendRead(); // here call to suspendRead
>  int available = connection.available();
>  System.out.println("onData available = " + available);
>  byte[] data = connection.readBytesByLength(available);
>  }
>  finally
>  {
>  connection.resumeRead(); // call resumeRead here
>  System.out.println("onData exit");
>  }
>  return true;
>  }
> }
> =================================
> public class TestServer
> {
>  //-------------------------------
>  static public void main(String[] args)throws IOException
>  {
>  TestServer s = new TestServer();
>  s.doServer();
>  }
>  void doServer() throws IOException
>  {
>  ServerSocket ssck = new ServerSocket(1000);
>  Socket sck = ssck.accept();
>  OutputStream out = new BufferedOutputStream(sck.getOutputStream());
>  for (int i = 0; i < 100000000; i++)
>  {
>  String s = "Message " + i + "\r\n";
>  //1.write out a string
>  out.write(s.getBytes());
>  //2.write out bytes length
>  ByteBuffer buffer = ByteBuffer.allocate(4);
>  buffer.putInt(i);
>  buffer.rewind();
>  out.write(buffer.getInt());
>  //3.write out bytes
>  byte[] data = new byte[i];
>  out.write(data);
>  out.flush();
>  }
>  }
> }
> ========================================
> TestClient result
> onData enter
> onData available = 10027
> onData exit
> onData enter
> onData available = 16
> onData exit
> onData enter
> onData available = 3752
> onData exit
> onData enter
> onData available = 10044
> onData exit
> onData enter
> onData available = 3752
> onData exit
> 
> ----- Original Message ----- 
> From: <gre...@we...>
> To: "fulichen" <ful...@du...>
> Sent: Monday, April 21, 2008 7:21 PM
> Subject: Re: [xSocket-develop] OutOfMemory exception
> 
> Hi fchen,
> 
> by calling the <connection>.suspendRead() you can stop reading the 
> socket. Reading will be resumed by calling the <connection>.
> reaumeRead() method. For an example see http://xsocket.svn.sourceforge.
> net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/
> connection/LimitedReadTest.java?view=markup
> 
> Gregor 
> 
> > -----Ursprgliche Nachricht-----
> > Von: "fulichen" <ful...@du...>
> > Gesendet: 21.04.08 12:35:09
> > An: <xso...@li...>
> > Betreff: [xSocket-develop] OutOfMemory exception
> 
> Our application need to handle many many connection, 
> > and the data need time to process,
> > because xSocket Dispatcher read data when it is avialable, 
> > our application often run out of memory.
> > 
> > How we limit Dispatcher data reading, or what can we do to prevent 
> > OutOfMemoryException?
> > 
> > fchen---------------------------------------------------------------
> --
> > -------- This SF.net email is sponsored by the 2008 JavaOne(SM) 
> > Conference Don't miss this year's exciting event. There's still 
> time 
> > to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/
> > clk;198757673;13503038;p?http://java.sun.com/javaone________________
> __
> > _____________________________ xSocket-develop mailing list xSocket-
> > de...@li...://lists.sourceforge.net/lists/
> > 
> > listinfo/xsocket-develop
> 
> -- 
> Gregor Roth
> ----------------------------------------------------------------------
> --- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $
> 100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;
> 198757673;13503038;p?http://java.sun.com/javaone______________________
> _________________________ xSocket-develop mailing list xSocket-
> de...@li... https://lists.sourceforge.net/lists/
> 
> listinfo/xsocket-develop



-- 
Gregor Roth

[xSocket-develop] xsocket-http Base64 Encoding/Decoding

[Martin T. <Mar...@gm...>]

Hi,

I'd like to authenticate a user using basic www-authentication. Does 
xsocket provide functions for base64 encoding/decoding or do I need to 
implement it on my own?

Thanks,
Martin

[xSocket-develop] suspendRead() and resumeRead() cause program hang?

[fulichen <ful...@du...>]

Hello Gregor,

Thanks for your response, but we find suspendRead() and resumeRead() will cause program hang, is it a bug (maybe there is race condition)? or how can we fix our test?

The test program is following, and the test environment Java version 1.6.0_03 on Windows Vista.
TestClient result hanging  a while after start (but not each TestClient test, try some times will get hang)

=================================
public class TestClient implements IDataHandler
{
 //-------------------------------
 static public void main(String[] args)throws IOException, InterruptedException
 {
  TestClient c = new TestClient();
  NonBlockingConnection nbc = new NonBlockingConnection("127.0.0.1", 1000, c);
  Thread.sleep(100000000);
 }
 public boolean onData(INonBlockingConnection connection)throws IOException
 {
  System.out.println("onData enter");
  try
  {
   connection.suspendRead(); // here call to suspendRead
   int available = connection.available();
   System.out.println("onData available = " + available);
   byte[] data = connection.readBytesByLength(available);
  }
  finally
  {
   connection.resumeRead(); // call resumeRead here
   System.out.println("onData exit");
  }
  return true;
 }
}
=================================
public class TestServer
{
 //-------------------------------
 static public void main(String[] args)throws IOException
 {
  TestServer s = new TestServer();
  s.doServer();
 }
 void doServer() throws IOException
 {
  ServerSocket ssck = new ServerSocket(1000);
  Socket sck = ssck.accept();
  OutputStream out = new BufferedOutputStream(sck.getOutputStream());
  for (int i = 0; i < 100000000; i++)
  {
   String s = "Message " + i + "\r\n";
   //1.write out a string
   out.write(s.getBytes());
   //2.write out bytes length
   ByteBuffer buffer = ByteBuffer.allocate(4);
   buffer.putInt(i);
   buffer.rewind();
   out.write(buffer.getInt());
   //3.write out bytes
   byte[] data = new byte[i];
   out.write(data);
   out.flush();
  }
 }
}
========================================
TestClient result
onData enter
onData available = 10027
onData exit
onData enter
onData available = 16
onData exit
onData enter
onData available = 3752
onData exit
onData enter
onData available = 10044
onData exit
onData enter
onData available = 3752
onData exit

----- Original Message ----- 
From: <gre...@we...>
To: "fulichen" <ful...@du...>
Sent: Monday, April 21, 2008 7:21 PM
Subject: Re: [xSocket-develop] OutOfMemory exception


Hi fchen,

by calling the <connection>.suspendRead() you can stop reading the socket. Reading will be resumed by calling the <connection>.reaumeRead() method. For an example see http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/LimitedReadTest.java?view=markup

Gregor 


> -----Ursprgliche Nachricht-----
> Von: "fulichen" <ful...@du...>
> Gesendet: 21.04.08 12:35:09
> An: <xso...@li...>
> Betreff: [xSocket-develop] OutOfMemory exception

Our application need to handle many many connection, 
> and the data need time to process,
> because xSocket Dispatcher read data when it is avialable, 
> our application often run out of memory.
> 
> How we limit Dispatcher data reading, or what can we do to prevent 
> OutOfMemoryException?
> 
> fchen-----------------------------------------------------------------
> -------- This SF.net email is sponsored by the 2008 JavaOne(SM) 
> Conference Don't miss this year's exciting event. There's still time 
> to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/
> clk;198757673;13503038;p?http://java.sun.com/javaone__________________
> _____________________________ xSocket-develop mailing list xSocket-
> de...@li... https://lists.sourceforge.net/lists/
> 
> listinfo/xsocket-develop



-- 
Gregor Roth

Re: [xSocket-develop] OutOfMemory exception

[<gre...@we...>]

Hi fchen,

by calling the <connection>.suspendRead() you can stop reading the socket. Reading will be resumed by calling the <connection>.reaumeRead() method. For an example see http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/LimitedReadTest.java?view=markup

Gregor 

> -----Ursprüngliche Nachricht-----
> Von: "fulichen" <ful...@du...>
> Gesendet: 21.04.08 12:35:09
> An: <xso...@li...>
> Betreff: [xSocket-develop] OutOfMemory exception

Our application need to handle many many connection, 
> and the data need time to process,
> because xSocket Dispatcher read data when it is avialable, 
> our application often run out of memory.
> 
> How we limit Dispatcher data reading, or what can we do to prevent 
> OutOfMemoryException?
> 
> fchen-----------------------------------------------------------------
> -------- This SF.net email is sponsored by the 2008 JavaOne(SM) 
> Conference Don't miss this year's exciting event. There's still time 
> to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/
> clk;198757673;13503038;p?http://java.sun.com/javaone__________________
> _____________________________ xSocket-develop mailing list xSocket-
> de...@li... https://lists.sourceforge.net/lists/
> 
> listinfo/xsocket-develop



-- 
Gregor Roth

[xSocket-develop] OutOfMemory exception

[fulichen <ful...@du...>]

Our application need to handle many many connection, 
and the data need time to process,
because xSocket Dispatcher read data when it is avialable, 
our application often run out of memory.

How we limit Dispatcher data reading, or what can we do to prevent OutOfMemoryException?

fchen

Re: [xSocket-develop] Running out of connections with HttpClient?

[<gre...@we...>]

Hi Ian,

the constructor you used -> new PostRequest(node.getAddress()+"/", new String[0]) is a convenience constructor to create a x-www-form-urlencoded POST request. By setting the body (post.setBodyDataSource(bytes, null)) you declare the encoding with null. Based on this xSocket-http creates a POST request header like

POST / HTTP/1.1
Content-Length: 3
Content-Type: application/x-www-form-urlencoded; charset=null
HOST: localhost:30002

As you see the charset is set with null. This is a bug, which I have already fixed in the trunk. I suppose this causes the strange behaviour you reported.

I'm not sure if you really want to use the content-type x-www-form-urlencoded. The PostRequest of the current trunk has been enhanced by a constructor, which accepts the parameters url, contentType and a byte array. E.g.
new PostRequest("http://localhost:30002/", "application/octet-stream", new byte[] { 65, 87, 78});

Regarding to the HttpClient you don't have to close a request or response. The HttpClient will automatically close or reuse the underlying http connection. What you should do, is to close the HttpClient. By closing the HttpClient the internal cached (open) connections will be closed.

Gregor




> -----Ursprüngliche Nachricht-----
> Von: "Ian Swett" <is...@ya...>
> Gesendet: 14.04.08 13:04:21
> An: xso...@li...
> Betreff: [xSocket-develop] Running out of connections with HttpClient?


> 
> I am trying to use only the http client portion of
> xsocket at first, and it appears I'm running out of
> connections around 250, presumably because the
> existing connection isn't being closed properly. 
> Eventually the first batch of requests timeout, and it
> will do another 250 posts.  
> 
> I can't find a close() or release() method on
> PostRequest or HttpResponse, so I'm not sure what I
> should be doing to allow HttpClient to reclaim the
> request.
> 
> Here is my code for posting a byte[] as the body, it
> is very simple.  In my simple performance test, I'm
> trying to call this 10000 times.  
> 
> 
> PostRequest post = new
> PostRequest(node.getAddress()+"/", new String[0]);
> post.setBodyDataSource(bytes, null);
> HttpResponse response = client.call(post);
> 
> 
> I'm using the default HttpClient:
> 
> client = new HttpClient();
> 
> 
> Thanks, Ian
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
> 



-- 
Gregor Roth

Re: [xSocket-develop] Running out of connections with HttpClient?

[Gregor R. <gre...@go...>]

Hi Ian,

the constructor you used -> new PostRequest(node.getAddress()+"/", new
String[0]) is a convenience constructor to create a x-www-form-urlencoded
POST request. By setting the body (post.setBodyDataSource(bytes, null)) you
declare the encoding with null. Based on this xSocket-http creates a POST
request header like

POST / HTTP/1.1
Content-Length: 3
Content-Type: application/x-www-form-urlencoded; charset=null
HOST: localhost:30002

As you see the charset is set with null. This is a bug, which I have already
fixed in the trunk. I suppose this causes the strange behaviour you
reported.

I'm not sure if you really want to use the content-type
x-www-form-urlencoded. The PostRequest of the current trunk has been
enhanced by a constructor, which accepts the parameters url, contentType and
a byte array. E.g.
new PostRequest("http://localhost:30002/", "application/octet-stream", new
byte[] { 65, 87, 78});

Regarding to the HttpClient you don't have to close a request or response.
The HttpClient will automatically close or reuse the underlying http
connection. What you should do, is to close the HttpClient. By closing the
HttpClient the internal cached (open) connections will be closed.

Gregor



2008/4/13, Ian Swett <is...@ya...>:
>
> I am trying to use only the http client portion of
> xsocket at first, and it appears I'm running out of
> connections around 250, presumably because the
> existing connection isn't being closed properly.
> Eventually the first batch of requests timeout, and it
> will do another 250 posts.
>
> I can't find a close() or release() method on
> PostRequest or HttpResponse, so I'm not sure what I
> should be doing to allow HttpClient to reclaim the
> request.
>
> Here is my code for posting a byte[] as the body, it
> is very simple.  In my simple performance test, I'm
> trying to call this 10000 times.
>
>
> PostRequest post = new
> PostRequest(node.getAddress()+"/", new String[0]);
> post.setBodyDataSource(bytes, null);
> HttpResponse response = client.call(post);
>
>
> I'm using the default HttpClient:
>
> client = new HttpClient();
>
>
> Thanks, Ian
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
> Don't miss this year's exciting event. There's still time to save $100.
> Use priority code J8TL2D2.
>
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>

[xSocket-develop] Running out of connections with HttpClient?

[Ian S. <is...@ya...>]

I am trying to use only the http client portion of
xsocket at first, and it appears I'm running out of
connections around 250, presumably because the
existing connection isn't being closed properly. 
Eventually the first batch of requests timeout, and it
will do another 250 posts.  

I can't find a close() or release() method on
PostRequest or HttpResponse, so I'm not sure what I
should be doing to allow HttpClient to reclaim the
request.

Here is my code for posting a byte[] as the body, it
is very simple.  In my simple performance test, I'm
trying to call this 10000 times.  


PostRequest post = new
PostRequest(node.getAddress()+"/", new String[0]);
post.setBodyDataSource(bytes, null);
HttpResponse response = client.call(post);


I'm using the default HttpClient:

client = new HttpClient();


Thanks, Ian

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [xSocket-develop] Proxy example - piping data to file

[Martin T. <Mar...@gm...>]

Hi,

thank you for your hints. Now it works :-)

One pitfall was that if I'm using
|> responseCtx.send(respHdr);

instead of
|> responseCtx.send(respHdr,respHdr.getContentLength());

if the content-length header is available, then the body is transfered 
using transfer-encoding but without setting the proper transfer-encoding 
header.

Another bug I've found is
> java.lang.ArrayIndexOutOfBoundsException: 1
>         at 
> org.xsocket.connection.http.HttpRequestHeader.getParamMap(HttpRequestHeader.java:580)
>         at 
> org.xsocket.connection.http.HttpRequestHeader.getParameterMap(HttpRequestHeader.java:535)
>         at 
> org.paxle.crawler.proxy.impl.ProxyResponseHandler.shouldCrawl(ProxyResponseHandler.java:128)
>         at 
> org.paxle.crawler.proxy.impl.ProxyResponseHandler.onResponse(ProxyResponseHandler.java:81)
>         at 
> org.xsocket.connection.http.client.HttpClientConnection$ResponseHandlerAdapter.performOnResponse(HttpClientConnection.java:967)
>         at 
> org.xsocket.connection.http.client.HttpClientConnection$ResponseHandlerAdapter.run(HttpClientConnection.java:961)
>         at 
> org.xsocket.connection.http.AbstractHttpConnection.performPendingTasks(AbstractHttpConnection.java:796)
>         at 
> org.xsocket.connection.http.AbstractHttpConnection.access$500(AbstractHttpConnection.java:67)
>         at 
> org.xsocket.connection.http.AbstractHttpConnection$MultithreadedTaskProcessor.run(AbstractHttpConnection.java:826)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>         at java.lang.Thread.run(Thread.java:595)
The query-parameter parser seems to assume that an http-query-parameter 
always has a value.

Regards,
Martin


gre...@we... schrieb:
> Hi Martin,
>
> the xSocket Junit test BodyForwardingTest (http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/http/tags/V2_0_alpha4/src/test/java/org/xsocket/connection/http/BodyForwardingTest.java)  contains a ForwardHandler which shows you how to control the message/body streaming.
>
> One fast way to fetch the complete message header is to call the toString method
>
>    ...
>    HttpResponseHeader resHdr = response.getResponseHeader();
>  
>    String hdrCopy = resHdr.toString();
>    FileChannel fileChannel = new ....
>    fileChannel.write(...)
>    ...
>
>
> The body data can be fetched by duplicating the body ByteBuffer :
>
>     ...
>     ByteBuffer[] data = bodyDataSource.readByteBufferByLength(available);
>  
>     for (ByteBuffer buf : data) {
>        fileChannel.write(buf.duplicate());
>     }
>
>     dataSink.write(data);
>     ...
>
> Take care with the NonThreaded annotations of the example code. If you are writing to a file, the handler should run in a multithreaded mode. The Nonthreaded annotation should be removed.
>
>
> Gregor
>   

Re: [xSocket-develop] Proxy example - piping data to file

[<gre...@we...>]

Hi Martin,

the xSocket Junit test BodyForwardingTest (http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/http/tags/V2_0_alpha4/src/test/java/org/xsocket/connection/http/BodyForwardingTest.java)  contains a ForwardHandler which shows you how to control the message/body streaming.

One fast way to fetch the complete message header is to call the toString method

   ...
   HttpResponseHeader resHdr = response.getResponseHeader();
 
   String hdrCopy = resHdr.toString();
   FileChannel fileChannel = new ....
   fileChannel.write(...)
   ...


The body data can be fetched by duplicating the body ByteBuffer :

    ...
    ByteBuffer[] data = bodyDataSource.readByteBufferByLength(available);
 
    for (ByteBuffer buf : data) {
       fileChannel.write(buf.duplicate());
    }

    dataSink.write(data);
    ...

Take care with the NonThreaded annotations of the example code. If you are writing to a file, the handler should run in a multithreaded mode. The Nonthreaded annotation should be removed.


Gregor



> -----Ursprüngliche Nachricht-----
> Von: "Martin Thelian" <Mar...@gm...>
> Gesendet: 10.04.08 09:56:16
> An: xso...@li...
> Betreff: [xSocket-develop] Proxy example - piping data to file


> 
> Hi,
> 
> I would like to use xsocket as described in [1] chapter "3.2 Reusing message objects - a proxy example" and additionally I would like to pipe the response-message into a file while transfering it back to the client. 
> 
> What is the best way to do this?
> 
> Thanks,
> Martin
> 
> [1] http://xsocket.sourceforge.net/http/tutorial/V2/TutorialHttp.htm
> -- 
> Psssst! Schon vom neuen GMX MultiMessenger gehört?
> Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop

[xSocket-develop] Proxy example - piping data to file

[Martin T. <Mar...@gm...>]

Hi,

I would like to use xsocket as described in [1] chapter "3.2 Reusing message objects - a proxy example" and additionally I would like to pipe the response-message into a file while transfering it back to the client. 

What is the best way to do this?

Thanks,
Martin

[1] http://xsocket.sourceforge.net/http/tutorial/V2/TutorialHttp.htm
-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger

Re: [xSocket-develop] scalability and latency

[Gregor R. <gre...@go...>]

Hi James,

this is a typical scenario which should be supported by xSocket-http.

Unfortunately, performance (scalability, latency, …) diagnostics &
optimizations haven't been done for xSocket-http yet (for resource/time
reasons). I would suspect, that the new module xSocket-http causes this
undesired effects.

I have to take a deeper look into this. Could you send me minimal code, that
I'm able to run some test and make some diagnostics?

Which platform do you use (windows, linux,…)?

Gregor

2008/4/4, James Olsen <Jam...@co...>:
>
>  I'm experimenting with xSocket 2.0-beta-2 and xSocket-http 2.0-alpha-4 to
> implement a messaging server that pushes messages to http clients. The
> messages are small (approx 15 bytes) but frequent (enqueued at a rate of 25
> per second per client).  These messages are in 10 logical topics and only
> the latest available message from each topic is sent, i.e. when a new one is
> queued to be sent it overwrites any message already queued for the same
> topic.  The server infrastructure will also concatenate several smaller
> messages to make one larger message if messages are available on multiple
> topics at the time of writing to the BodyDataSink.  Thus the number of
> messages dequeued and sent to the clients is automatically throttled by the
> capability of the underlying comms.
>
>
>
> I have the same setup with Tomcat 6 using the NIO connector and find that
> I can add large numbers of users and still get good performance.  With the
> hardware I have it can deliver 70,000 messages per second.  More users mean
> fewer messages per user but the overall throughput remains the same and
> latency is low and fairly consistent.
>
>
>
> However I'm finding that this does not scale as well with xSocket.  The
> max message throughput is lower and reduces as more clients are added.  The
> average latency is higher and can be highly variable (>10 seconds!)
> depending on the number of background message writing Threads and the number
> of Dispatchers.  Making the number of message writing Threads equal to the
> number of Dispatchers and keeping them both quite low (<= 3) reduces the
> latency variability but the average is still significantly higher than with
> Tomcat.  SYNC flushing is also better than ASYNC with this setup because it
> causes more message concatenation (fewer larger messages sent).
>
>
>
> The xSocket API has been a joy to work with and allows me to do more than
> with Tomcat/Comet (e.g. if desired the client can continue to write to the
> connection) and the memory use is incredibly low, however the performance
> for my scenario is worse than I was expecting.
>
>
>
> Is there something I might be missing in my config or is this not a
> scenario that xSocket is currently optimal for?
>
>
> ***********************************************************************************************
> This email, its contents and any files attached are a confidential
> communication and are
> intended only for the named addressees indicated in the message.
>
> If you are not the named addressee or if you have received this email in
> error, you may not,
> without the consent of Cognotec, copy, use or rely on any information or
> attachments in any way.
> Please notify the sender by return email and delete it from your email
> system.
>
> Unless separately agreed, Cognotec does not accept any responsibility for
> the accuracy or
> completeness of the contents of this email or its attachments. Please note
> that any views,
> opinion or advice contained in this communication are those of the sending
> individual
> and not those of Cognotec and Cognotec shall have no liability whatsoever
> in relation to
> this communication (or its content) unless separately agreed.
>
> ***********************************************************************************************
>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
>
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>
>

[xSocket-develop] scalability and latency

[James O. <Jam...@co...>]

I'm experimenting with xSocket 2.0-beta-2 and xSocket-http 2.0-alpha-4
to implement a messaging server that pushes messages to http clients.
The messages are small (approx 15 bytes) but frequent (enqueued at a
rate of 25 per second per client).  These messages are in 10 logical
topics and only the latest available message from each topic is sent,
i.e. when a new one is queued to be sent it overwrites any message
already queued for the same topic.  The server infrastructure will also
concatenate several smaller messages to make one larger message if
messages are available on multiple topics at the time of writing to the
BodyDataSink.  Thus the number of messages dequeued and sent to the
clients is automatically throttled by the capability of the underlying
comms.

 

I have the same setup with Tomcat 6 using the NIO connector and find
that I can add large numbers of users and still get good performance.
With the hardware I have it can deliver 70,000 messages per second.
More users mean fewer messages per user but the overall throughput
remains the same and latency is low and fairly consistent.

 

However I'm finding that this does not scale as well with xSocket.  The
max message throughput is lower and reduces as more clients are added.
The average latency is higher and can be highly variable (>10 seconds!)
depending on the number of background message writing Threads and the
number of Dispatchers.  Making the number of message writing Threads
equal to the number of Dispatchers and keeping them both quite low (<=
3) reduces the latency variability but the average is still
significantly higher than with Tomcat.  SYNC flushing is also better
than ASYNC with this setup because it causes more message concatenation
(fewer larger messages sent).

 

The xSocket API has been a joy to work with and allows me to do more
than with Tomcat/Comet (e.g. if desired the client can continue to write
to the connection) and the memory use is incredibly low, however the
performance for my scenario is worse than I was expecting.

 

Is there something I might be missing in my config or is this not a
scenario that xSocket is currently optimal for?




***********************************************************************************************
This email, its contents and any files attached are a confidential communication and are
intended only for the named addressees indicated in the message. 

If you are not the named addressee or if you have received this email in error, you may not, 
without the consent of Cognotec, copy, use or rely on any information or attachments in any way. 
Please notify the sender by return email and delete it from your email system.  

Unless separately agreed, Cognotec does not accept any responsibility for the accuracy or
completeness of the contents of this email or its attachments. Please note that any views,
opinion or advice contained in this communication are those of the sending individual
and not those of Cognotec and Cognotec shall have no liability whatsoever in relation to
this communication (or its content) unless separately agreed.
***********************************************************************************************

[xSocket-develop] Fwd: how alpha is alpha?

[Gregor R. <gre...@go...>]

 <xSo...@li...>
.. sorry my former response does have a typo. I is april/2008 not march/2008

Gregor

---------- Forwarded message ----------
From: Gregor Roth <gre...@go...>
Date: 29.03.2008 10:01
Subject: Re: [xSocket-develop] how alpha is alpha?
To: xSo...@li...
Cc: Raoul Duke <ra...@gm...>

Hi Raoul,

I added xSocket's understanding of alpha/beta/final to the home page. See
http://xsocket.sourceforge.net/UnderstandingOfAlphaBetaFinal.htm


The state of the current xSocket's component is as follows:


   - the final version xSocket core 2.0 will be release in march/2008.
   Only minor issues are open (typo checks, JavaDoc, tutorial update).


   - The first beta of xSocket multiplexed will be release in march/2008.
   Planning for the final version is round about 1-2 month later.


   - The next version of xSocket http will also be released in
   march/2008. The component will stay in alpha mode (beta will be available in
   round about 1-2 months later). The alpha is full functional. Bugs are not
   known for the current implementation. But more test/example/early
   applications are required to see that the API is high flexible and high
   usable. Major API changes are not expected for xSocket http, but it can not
   be excluded. For this reasons xSocket http stays in alpha mode.


Feedback and support of the community is very welcome. As more community
feedback and support as shorter the pre final phases

I hope this helps
Gregor



2008/3/29, Raoul Duke <ra...@gm...>:
>
> hi,
>
> i'm interested in xSocket for implementing client async HTTP requests.
> what does alpha/beta mean? :-) when do you think things will get out
> of alpha/beta?
>
> many thanks!
>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
>
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> xSocket-develop mailing list
> xSo...@li...
> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>